Reproducibility of prompts in computer-aided detection (CAD) of breast cancer

CAD e mamografia: por que usar esta ferramenta?

A mamografia representa o melhor método de detecção precoce do câncer de mama, porém cerca de 10% a 30% das lesões mamárias são perdidas no rastreamento, devido a limitações próprias dos observadores humanos. A detecção auxiliada por computador (computer-aided detection - CAD) é uma tecnologia relativamente nova que tem sido implementada em alguns serviços de mamografia, com o intuito de prover uma dupla leitura. Estudos clínicos têm demonstrado que o CAD aumenta a sensibilidade de detecção do câncer da mama, por radiologistas, em até 21%. Um sistema CAD é útil em situações em que exista alta variabilidade interobservador, falta de observadores treinados, ou na impossibilidade de se realizar a dupla leitura com dois ou mais radiologistas. O objetivo desta revisão está baseado na necessidade de atualizar a comunidade médica acerca desta ferramenta, como um método auxiliar, quantitativo, não operador-dependente, e que visa a melhorar a qualidade do diagnóstico do câncer de mama.

Impact of a CAD system in a screen-film mammography screening program: a prospective study

OBJECTIVE

The purpose of our study was to perform a prospective assessment of the impact of a CAD system in a screen-film mammography screening program during a period of 3 years.

MATERIALS AND METHODS

Our study was carried out on a population of 21,855 asymptomatic women (45-65 years). Mammograms were processed in a CAD system and independently interpreted by one of six radiologists. We analyzed the following parameters: sensitivity of radiologist's interpretation (without and with CAD), detection increase, recall rate and positive predictive value of biopsy, CAD's marks, radiologist's false negatives and comparative analysis of carcinomas detected and non-detected by CAD.

RESULTS

Detection rate was 4.3‰. CAD supposed an increase of 0.1‰ in detection rate and 1% in the total number of cases (p<0.005). The impact on recall rate was not significant (0.4%) and PPV of percutaneous biopsy was unchanged by CAD (20.23%). CAD's marks were 2.7 per case and 0.7 per view. Radiologist's false negatives were 13 lesions which were initially considered as CAD's false positives.

CONCLUSIONS

CAD supposed a significant increase in detection, without modifications in recall rates and PPV of biopsy. However, better results could have been achieved if radiologists had considered actionable those cases marked by CAD but initially misinterpreted.

Computer-aided detection in full-field digital mammography: sensitivity and reproducibility in serial examinations

PURPOSE

To retrospectively evaluate the sensitivity and reproducibility of a computer-aided detection (CAD) system applied to serial digital mammograms obtained in women with breast cancer, with histologic analysis as the reference standard.

MATERIALS AND METHODS

This study was institutional review board approved, and patient informed consent was waived. A commercially available CAD system was applied to initial and follow-up digital mammograms obtained in 93 women with breast cancer (mean age, 52 years; age range, 32-81 years). The mean interval between mammographic examinations was 23 days (range, 7-58 days). There were 119 visible lesion components (70 masses, 49 microcalcifications). Sensitivity, false-positive mark rate, and reproducibility of the CAD system were evaluated for both sets of mammograms with the t test.

RESULTS

Sensitivities of the CAD system at initial and follow-up digital mammography were 91% and 89%, respectively, for detection of masses. Sensitivity of the CAD system for detection of microcalcifications was 100% at both initial and follow-up digital mammography. Overall false-positive mark rates were 0.29 per image and 0.27 per image at initial and follow-up digital mammography, respectively. When craniocaudal and mediolateral oblique views were considered separately, sensitivities were 76% and 75%, respectively, for masses and 96% and 92%, respectively, for microcalcifications. The reproducibility of CAD marks was 80% for true-positive masses, 92% for true-positive microcalcifications, 9% for false-positive masses, and 8% for false-positive microcalcifications (P < .001).

CONCLUSION

The sensitivity of the CAD system was consistently high for detection of breast cancer on initial and short-term follow-up digital mammograms. Reproducibility was significantly higher for true-positive CAD marks than for false-positive CAD marks.

SUPPLEMENTAL MATERIAL

http://radiology.rsnajnls.org/cgi/content/full/246/1/71/DC1.

Reproducibility of computer-aided detection marks in digital mammography

Objective

To evaluate the performance and reproducibility of a computer-aided detection (CAD) system in mediolateral oblique (MLO) digital mammograms taken serially, without release of breast compression.

Materials and Methods

A CAD system was applied preoperatively to the full-field digital mammograms of two MLO views taken without release of breast compression in 82 patients (age range: 33 - 83 years; mean age: 49 years) with previously diagnosed breast cancers. The total number of visible lesion components in 82 patients was 101: 66 masses and 35 microcalcifications. We analyzed the sensitivity and reproducibility of the CAD marks.

Results

The sensitivity of the CAD system for first MLO views was 71% (47/66) for masses and 80% (28/35) for microcalcifications. The sensitivity of the CAD system for second MLO views was 68% (45/66) for masses and 17% (6/35) for microcalcifications. In 84 ipsilateral serial MLO image sets (two patients had bilateral cancers), identical images, regardless of the existence of CAD marks, were obtained for 35% (29/84) and identical images with CAD marks were obtained for 29% (23/78). Identical images, regardless of the existence of CAD marks, for contralateral MLO images were 65% (52/80) and identical images with CAD marks were obtained for 28% (11/39). The reproducibility of CAD marks for the true positive masses in serial MLO views was 84% (42/50) and that for the true positive microcalcifications was 0% (0/34).

Conclusion

The CAD system in digital mammograms showed a high sensitivity for detecting masses and microcalcifications. However, reproducibility of microcalcification marks was very low in MLO views taken serially without release of breast compression. Minute positional change and patient movement can alter the images and result in a significant effect on the algorithm utilized by the CAD for detecting microcalcifications.

Ductal Carcinoma in Situ: Computer-aided Detection in Screening Mammography

PURPOSE

To retrospectively evaluate the sensitivity of computer-aided detection (CAD) in depicting ductal carcinoma in situ (DCIS) on screening mammograms by using biopsy proved lesion location as the reference standard.

MATERIALS AND METHODS

Institutional review board approval was obtained, with a waiver of patient informed consent for this HIPAA-compliant study. Findings of all image-guided biopsies with a pathologic diagnosis of DCIS during a 1-year period were reviewed. Fifty-eight lesions in 55 women (average age, 61.41 years +/- 12.89 [standard deviation]) were available for review. The screening mammogram of the affected breast and, if available, the prior screening mammogram were digitized by the CAD system. An assessment was then made as to whether the CAD system marked the area of DCIS on the current and prior mammograms. Patient age, location and mammographic size of the lesion, type of lesion, and breast density were recorded and were analyzed by using chi2, Fisher exact, or Cochran-Mantel-Haenzel tests, where applicable.

RESULTS

CAD identified DCIS in 53 (91%) of 58 lesions on craniocaudal (CC) and mediolateral oblique (MLO) views of screening mammograms obtained in the year of the diagnosis. On screening mammograms obtained prior to the year of the diagnosis (34 patients), no radiologically or CAD-detected lesion was present on 11 (32%) of 34 mammograms. CAD identified DCIS in 16 (70%) of 23 lesions on one of the two views. Seven (30%) of 23 lesions had mammographic findings at retrospective review that were not identified with CAD.

CONCLUSION

CAD had a high sensitivity in the depiction of DCIS.

“Hippocrates-mst”: a prototype for computer-aided microcalcification analysis and risk assessment for breast cancer

One of the most common cancer types among women is breast cancer. Regular mammographic examinations increase the possibility for early diagnosis and treatment and significantly improve the chance of survival for patients with breast cancer. Clustered microcalcifications have been considered as important indicators of the presence of breast cancer. We present "Hippocrates-mst", a prototype system for computer-aided risk assessment of breast cancer. Our research has been focused in developing software to locate microcalcifications on X-ray mammography images, quantify their critical features and classify them according to their probability of being cancerous. A total of 260 cases (187 benign and 73 malignant) have been examined and the performance of the prototype is presented through receiver operating characteristic (ROC) analysis. The system is showing high levels of sensitivity identifying correctly 98.63% of malignant cases.

A method to improve visual similarity of breast masses for an interactive computer-aided diagnosis environment

The purpose of this study was to develop and test a method for selecting "visually similar" regions of interest depicting breast masses from a reference library to be used in an interactive computer-aided diagnosis (CAD) environment. A reference library including 1000 malignant mass regions and 2000 benign and CAD-generated false-positive regions was established. When a suspicious mass region is identified, the scheme segments the region and searches for similar regions from the reference library using a multifeature based k-nearest neighbor (KNN) algorithm. To improve selection of reference images, we added an interactive step. All actual masses in the reference library were subjectively rated on a scale from 1 to 9 as to their "visual margins speculations". When an observer identifies a suspected mass region during a case interpretation he/she first rates the margins and the computerized search is then limited only to regions rated as having similar levels of spiculation (within +/-1 scale difference). In an observer preference study including 85 test regions, two sets of the six "similar" reference regions selected by the KNN with and without the interactive step were displayed side by side with each test region. Four radiologists and five nonclinician observers selected the more appropriate ("similar") reference set in a two alternative forced choice preference experiment. All four radiologists and five nonclinician observers preferred the sets of regions selected by the interactive method with an average frequency of 76.8% and 74.6%, respectively. The overall preference for the interactive method was highly significant (p < 0.001). The study demonstrated that a simple interactive approach that includes subjectively perceived ratings of one feature alone namely, a rating of margin "spiculation," could substantially improve the selection of "visually similar" reference images.

Performance and reproducibility of a computerized mass detection scheme for digitized mammography using rotated and resampled images: an assessment

OBJECTIVE

Our objective was to compare the performance and reproducibility of a computer-aided detection (CAD) scheme that uses multiple rotated and resampled images with an in-house-developed CAD scheme (single-image-based) and a commercial CAD product in detecting masses depicted on digitized mammograms.

MATERIALS AND METHODS

Ninety-two film mammograms (acquired from 23 patients) were selected. Forty-four mass regions associated with malignancy were visually identified. A commercial CAD system was used to scan and process each image four times, for a total of 368 digitized images depicting 176 mass regions. Images were processed using two CAD schemes developed in our laboratory. One uses the detection results generated from a single image, and the other averages five detection scores generated after processing the originally digitized image and four slightly rotated and resampled images. A region-based analysis was used to compare reproducibility and performance levels among the two in-house schemes and the commercial system.

RESULTS

The commercial system detected a total of 98 mass regions (55.7% sensitivity) and 136 false-positive regions (an average of 0.37 per image). Among the detected mass regions, 76 represented 19 regions that were detected on all four scans and 22 represented 10 regions that were not fully reproducible. Eighty-eight false-positive detections represented 22 reproducible detections on all four scans. Our single-image-based scheme identified 87 mass regions and 160 false-positive regions. Seventeen mass regions and 28 false-positive regions were detected on all four scans. The multiple-image-based scheme identified 98 mass regions and 132 false-positive regions. Twenty-three mass regions were detected on all four scans. One hundred twelve of the 132 false-positive regions represented 28 reproducible detections.

CONCLUSION

Averaging detection scores from multiple rotated and resampled images generated from a single digitization of a film can reduce variations in detection scores. Our multiple-image-based scheme improved both performance and reproducibility over the single-image-based scheme. The multiple-image-based scheme yielded an overall performance comparable to that of the commercial system but with improved reproducibility.

A method to test the reproducibility and to improve performance of computer-aided detection schemes for digitized mammograms

The purpose of this study is to develop a new method for assessment of the reproducibility of computer-aided detection (CAD) schemes for digitized mammograms and to evaluate the possibility of using the implemented approach for improving CAD performance. Two thousand digitized mammograms (representing 500 cases) with 300 depicted verified masses were selected in the study. Series of images were generated for each digitized image by resampling after a series of slight image rotations. A CAD scheme developed in our laboratory was applied to all images to detect suspicious mass regions. We evaluated the reproducibility of the scheme using the detection sensitivity and false-positive rates for the original and resampled images. We also explored the possibility of improving CAD performance using three methods of combining results from the original and resampled images, including simple grouping, averaging output scores, and averaging output scores after grouping. The CAD scheme generated a detection score (from 0 to 1) for each identified suspicious region. A region with a detection score >0.5 was considered as positive. The CAD scheme detected 238 masses (79.3% case-based sensitivity) and identified 1093 false-positive regions (average 0.55 per image) in the original image dataset. In eleven repeated tests using original and ten sets of rotated and resampled images, the scheme detected a maximum of 271 masses and identified as many as 2359 false-positive regions. Two hundred and eighteen masses (80.4%) and 618 false-positive regions (26.2%) were detected in all 11 sets of images. Combining detection results improved reproducibility and the overall CAD performance. In the range of an average false-positive detection rate between 0.5 and 1 per image, the sensitivity of the scheme could be increased approximately 5% after averaging the scores of the regions detected in at least four images. At low false-positive rate (e.g., < or =average 0.3 per image), the grouping method alone could increase CAD sensitivity by 7%. The study demonstrated that reproducibility of a CAD scheme can be tested using a set of slightly rotated and resampled images. Because the reproducibility of true-positive detections is generally higher than that of false-positive detections, combining detection results generated from subsets of rotated and resampled images could improve both reproducibility and overall performance of CAD schemes.

Computer-Aided Detection of Amorphous Calcifications

OBJECTIVE

Computer-aided detection (CAD) systems have been used successfully to detect malignant calcifications on mammography, with sensitivities ranging from 86% to 99%. Amorphous calcifications are a subset of small indistinct calcifications of intermediate concern that have a 20% likelihood of being malignant and that are frequently overlooked on mammography. The purpose of our study was to determine the sensitivity of one commercially available CAD system for detecting amorphous calcifications.

MATERIALS AND METHODS

A commercially available CAD system evaluated mammograms of 82 patients with 85 mammographically detected and histologically sampled groups of amorphous calcifications (21 malignant, 14 high risk, and 50 benign). The sensitivity of the system for detecting the calcifications on at least one image of the two-view mammographic examination (case sensitivity) and on each individual mammographic image (image sensitivity) was determined. Findings were correlated with results from large core needle biopsy or surgical excision in each case.

RESULTS

The CAD system detected amorphous calcifications in 43 of 85 cases (case sensitivity, 51%) and in 59 of 146 mammographic images (image sensitivity, 40%). The case sensitivities by histologic outcome were 57% for malignant calcifications, 29% for high-risk calcifications, and 54% for benign calcifications. An average of 2.0 false-positive marks were displayed per case.

CONCLUSION

The CAD sensitivity for malignant amorphous calcifications is markedly lower than previously reported for all malignant calcifications. Breast imaging radiologists who use CAD systems should continue to search diligently for these difficult-to-detect lesions.

Comprendre et utiliser le CAD (Aide informatisée au diagnostic) en mammographie

Generalization of breast screening programs requires efficient double reading of mammograms, which allows reduction of false negative interpretations, but it may be difficult to achieve. CAD (Computed Aided Detection) systems are dramatically improving and can now assist in the detection of suspicious mammographic lesions, either suspicious microcalcifications, masses or architectural distortion. Characterization of the lesions is improving as well. CAD mammography might complete or substitute to "human" double reading. The aim of this review is to present the main CAD systems commercially available, review the principles of CAD and discuss the results of CAD mammography. Specifically, the role of CAD within breast screening program, according to the results of recent prospective studies will be discussed.

Computer-aided detection performance in mammographic examination of masses: assessment

PURPOSE

To compare performance of two computer-aided detection (CAD) systems and an in-house scheme applied to five groups of sequentially acquired screening mammograms.

MATERIALS AND METHODS

Two hundred nineteen film-based mammographic examinations, classified into five groups, were included in this study. Group 1 included 58 examinations in which verified malignant masses were detected during screening; group 2, 39 in which all available latest examinations were performed prior to diagnosis of these malignant masses (subset of 39 women from group 1); group 3, 22 in which findings were interpreted as negative but were verified as cancer within 1 year from the negative interpretation (missed cancers); group 4, 50 in which findings were negative and patients were not recalled for additional procedures; and group 5, 50 in which patients were recalled for additional procedures and findings were negative for cancer. In all examinations, images were processed with two Food and Drug Administration-approved commercially available CAD systems and an in-house scheme. Performance levels in terms of true-positive detection rates and number of false-positive identifications per image and per examination were compared.

RESULTS

Mass detection rates in positive examinations (group 1) were 67%-72%. Detection rates among three systems were not significantly different (P > .05). In 50 negative screening examinations (group 4), false-positive rates ranged from 1.08 to 1.68 per four-view examination. Performance level differences among systems were significant for false-positive rates (P = .008). Performance of all systems was at levels lower than publicly suggested in some retrospective studies. False-positive CAD cueing rates were significantly higher for negative examinations in which patients were recalled (group 5) than they were for those in which patients were not recalled (group 4) (P < or = .002).

CONCLUSION

Performance of CAD systems for mass detection at mammography varies significantly, depending on examination and system used. Actual performance of all systems in clinical environment can be improved.