Does the accuracy of single reading with CAD (computer-aided detection) compare with that of double reading?: A review of the literature

Framing the fallibility of Computer-Aided Detection aids cancer detection

Computer-Aided Detection (CAD) has been proposed to help operators search for cancers in mammograms. Previous studies have found that although accurate CAD leads to an improvement in cancer detection, inaccurate CAD leads to an increase in both missed cancers and false alarms. This is known as the over-reliance effect. We investigated whether providing framing statements of CAD fallibility could keep the benefits of CAD while reducing over-reliance. In Experiment 1, participants were told about the benefits or costs of CAD, prior to the experiment. Experiment 2 was similar, except that participants were given a stronger warning and instruction set in relation to the costs of CAD. The results showed that although there was no effect of framing in Experiment 1, a stronger message in Experiment 2 led to a reduction in the over-reliance effect. A similar result was found in Experiment 3 where the target had a lower prevalence. The results show that although the presence of CAD can result in over-reliance on the technology, these effects can be mitigated by framing and instruction sets in relation to CAD fallibility.

Is single reading with computer-aided detection (CAD) as good as double reading in mammography screening? A systematic review

BACKGROUND

In accordance with European guidelines, mammography screening comprises independent readings by two breast radiologists (double reading). CAD (computer-aided detection) has been suggested to complement or replace one of the two readers (single reading + CAD).The aim of this systematic review is to address the following question: Is the reading of mammographic x-ray images by a single breast radiologist together with CAD at least as accurate as double reading?

METHODS

The electronic literature search included the databases Pub Med, EMBASE and The Cochrane Library. Two independent reviewers assessed abstracts and full-text articles.

RESULTS

1049 abstracts were identified, of which 996 were excluded with reference to inclusion and exclusion criteria; 53 full-text articles were assessed for eligibility. Finally, four articles were included in the qualitative analysis, and one in a GRADE synthesis.

CONCLUSIONS

The scientific evidence is insufficient to determine whether the accuracy of single reading + CAD is at least equivalent to that obtained in standard practice, i.e. double reading where two breast radiologists independently read the mammographic images.

The evolving role of new imaging methods in breast screening

The potential to avert breast cancer deaths through screening means that efforts continue to identify methods which may enhance early detection. While the role of most new imaging technologies remains in adjunct screening or in the work-up of mammography-detected abnormalities, some of the new breast imaging tests (such as MRI) have roles in screening groups of women defined by increased cancer risk. This paper highlights the evidence and the current role of new breast imaging technologies in screening, focusing on those that have broader application in population screening, including digital mammography, breast ultrasound in women with dense breasts, and computer-aided detection. It highlights that evidence on new imaging in screening comes mostly from non-randomised studies that have quantified test detection capability as adjunct to mammography, or have compared measures of screening performance for new technologies with that of conventional mammography. Two RCTs have provided high-quality evidence on the equivalence of digital and conventional mammography and on outcomes of screen-reading complemented by CAD. Many of these imaging technologies enhance cancer detection but also increase recall and false positives in screening.

Mammographic features of breast cancers at single reading with computer-aided detection and at double reading in a large multicenter prospective trial of computer-aided detection: CADET II

PURPOSE

To evaluate the mammographic features of breast cancer that favor lesion detection with single reading and computer-aided detection (CAD) or with double reading.

MATERIALS AND METHODS

The Computer Aided Detection Evaluation Trial II study was approved by the ethics committee, and all participants provided written informed consent. A total of 31,057 women were recruited from three screening centers between September 2006 and August 2007. They were randomly allocated to the double reading group, the single reading with CAD group, or the double reading and single reading with CAD group at a ratio of 1:1:28, respectively. In this study, cancers in the women whose mammograms were read with both single reading with CAD and double reading were retrospectively reviewed. The original mammograms were obtained for each case and reviewed by two of three experienced breast radiologists in consensus. The method of detection was noted. The size and predominant mammographic feature of the cancer were recorded, as was the breast density. CAD marking data were reviewed to determine if the cancer had been correctly marked.

RESULTS

A total of 227 cancers were detected in 28,204 women. A total of 170 cases were recalled with both reading regimens. Lesion types were masses (66%), microcalcifications (25%), parenchymal deformities (6%), and asymmetric densities (3%). The ability of the reading regimens to correctly prompt the reader to recall cases varied significantly by lesion type (P < .001). More parenchymal deformities were recalled with double reading, whereas more asymmetric densities were recalled with single reading with CAD. There was no difference in the ability of either reading regimen to prompt the reader to correctly recall masses or microcalcifications. CAD correctly prompted 100% of microcalcifications, 87% of mass lesions, 80% of asymmetric densities, and 50% of parenchymal deformities. CAD correctly marked 93% of spiculated masses compared with 80% of ill-defined masses (P = .054). There was a significant trend for cancers detected with double reading to occur only in women with a denser mammographic background pattern (P = .02). Size had no effect on lesion detection.

CONCLUSION

Readers using either single reading with CAD or double reading need to be aware of the strengths and weaknesses of reading regimens to avoid missing the more challenging cancer cases.

Modelling the impact of changes in sensitivity on the outcomes of the UK breast screening programme

OBJECTIVES

To assess the impact on breast cancer mortality of improving the sensitivity of breast screening programmes.

METHODS

A Markov model was populated with data obtained from published statistics describing the UK National Health Service Breast Screening Programme and the incidence and mortality of breast cancer in the UK. The model was used to study the impact of changes to the sensitivity of screening. The effects on cancer detection rates and breast cancers and total mortality was studied for a cohort of women followed from age 45 to age 89.

RESULTS

Running the model from age 45 to 89, with sensitivity set at the baseline value of 75%, predicts the detection at screening of 44 cancers per thousand of population and the detection outside screening of 82 cancers per thousand of population. Running the model with values of sensitivity from 75% to 95% shows the proportion of cancers detected at screening increasing as screening improves, and deaths from breast cancer falling. The drop in breast cancer deaths is however modest. Increasing sensitivity from 75% to 85% reduces the number of breast cancer deaths from 28 to 27 per thousand.

CONCLUSIONS

Likely achievable improvements in the sensitivity of screening do not have a marked effect on breast cancer mortality.

Strategies for digital mammography interpretation in a clinical patient population

Mammography is the basic imaging modality for early detection of breast cancer. The aim of this prospective study was to evaluate the impact of different mammogram reading strategies on the diagnostic yield in a consecutive patient population referred for digital mammography to a hospital. First, the effect of using computer-aided detection (CAD) software on the performance of mammogram readers was studied. Furthermore, the impact of employing technologists as either prereaders or double readers was assessed, as compared to the conventional strategy of single reading by a radiologist. Digital mammograms of 1,048 consecutive patients were evaluated by a radiologist and 3 technologists with and without the use of CAD software. ROC analysis was used to study the effects of the different strategies. In the conventional strategy, an overall area under the curve (AUC) of 0.92 was found, corresponding to a sensitivity of 84% and specificity of 94%. When applying CAD software, the AUCs were similar before and after CAD for all readers (mean of 0.95). Employing technologists in prereading and double reading of mammograms resulted in a mean AUC of 0.91 and 0.96, respectively. In the prereading strategy, the corresponding sensitivity and specificity were 81 and 96%; in the double reading strategy they were 96 and 79%, respectively. Concluding, in this clinical population, systematic application of CAD software by either radiologist or technologists failed to improve the diagnostic yield. Furthermore, employing technologists as double readers of mammograms was the most effective strategy in improving breast cancer detection in daily clinical practice.

[Mammography screening in Germany]

Available data suggest that early detection of breast cancer by mammography screening can reduce mortality by about 25%. Intensified monitoring of women with a family history of breast cancer and regular general screening have recently been introduced in Germany. The screening program is expected to be fully established by 2008. Following its successful introduction (participation rates between 65 and 80%), the German screening program will be conducted and evaluated in accordance with the European guidelines. At least in the screening trials that were conducted prior to the now established screening program the quality criteria were more than fulfilled (e.g. cancer detection rate in Bremen 8.7, Wiesbaden 9.4, Weser-Ems region 8.3/1000). Additional parameters that can be taken into account for quality assurance are the overdiagnosis bias, lead time bias, length bias and selection bias. Moreover, there are some factors that are specific to the German program compared with the breast cancer screening programs already established in other countries. One of these is the intensified screening program for high-risk women (ca. 5% of all carcinomas) and as a result fewer women with an increased genetic risk of breast cancer will be represented in the general screening program. The German screening program involves only a few university centers and hospital-based physicians, which may have adverse effects on research and development as well as mammography training in the future. Therefore, the screening program should also provide for the investigation of new techniques or emerging techniques (e.g. CAD systems in screening mammography) in the future.

Computer-assisted diagnosis (CAD) in mammography: comparison of diagnostic accuracy of a new algorithm (Cyclopus, Medicad) with two commercial systems

PURPOSE

The study compares the diagnostic accuracy (correct identification of cancer) of a new computer-assisted diagnosis (CAD) system (Cyclopus) with two other commercial systems (R2 and CADx).

MATERIALS AND METHODS

Cyclopus was tested on a set of 120 mammograms on which the two compared commercial systems had been previously tested. The set consisted of mammograms reported as negative, preceding 31 interval cancers reviewed as screening error or minimal sign, and of 89 verified negative controls randomly selected from the same screening database.

RESULTS

Cyclopus sensitivity was 74.1% (R2=54.8%; CADx=41.9%) and was higher for interval cancers reviewed as screening error (90.9%; R2=54.5%; CADx=81.8%) compared with those reviewed as minimal sign (65.0%; R2=55.0%; CADx=20.0%). Specificity was 15.7% (R2=29.2%; CADx=17.9%). Overall accuracy was 30.8% (R2=35.8%; CADx=24.1%). The positive predictive value of a case with CAD marks [regions of interest (ROI)] was 23.4% (23/98; R2=16.0%; CADx=15.1%). Average ROI number per view among negative controls was 1.13 (R2=0.93; CADx=0.99). Cyclopus was more sensitive for masses compared with isolated microcalcifications (208 vs 62 ROI; R2=90 vs 213; CADx=192 vs 130).

CONCLUSIONS

Compared with two other commercial systems, Cyclopus was more sensitive (R2 p=0.14; CADx p=0.02) and less specific (R2 p=0.02; CADx p=0.64).

Single reading with computer-aided detection for screening mammography

BACKGROUND

The sensitivity of screening mammography for the detection of small breast cancers is higher when the mammogram is read by two readers rather than by a single reader. We conducted a trial to determine whether the performance of a single reader using a computer-aided detection system would match the performance achieved by two readers.

METHODS

The trial was designed as an equivalence trial, with matched-pair comparisons between the cancer-detection rates achieved by single reading with computer-aided detection and those achieved by double reading. We randomly assigned 31,057 women undergoing routine screening by film mammography at three centers in England to double reading, single reading with computer-aided detection, or both double reading and single reading with computer-aided detection, at a ratio of 1:1:28. The primary outcome measures were the proportion of cancers detected according to regimen and the recall rates within the group receiving both reading regimens.

RESULTS

The proportion of cancers detected was 199 of 227 (87.7%) for double reading and 198 of 227 (87.2%) for single reading with computer-aided detection (P=0.89). The overall recall rates were 3.4% for double reading and 3.9% for single reading with computer-aided detection; the difference between the rates was small but significant (P<0.001). The estimated sensitivity, specificity, and positive predictive value for single reading with computer-aided detection were 87.2%, 96.9%, and 18.0%, respectively. The corresponding values for double reading were 87.7%, 97.4%, and 21.1%. There were no significant differences between the pathological attributes of tumors detected by single reading with computer-aided detection alone and those of tumors detected by double reading alone.

CONCLUSIONS

Single reading with computer-aided detection could be an alternative to double reading and could improve the rate of detection of cancer from screening mammograms read by a single reader. (ClinicalTrials.gov number, NCT00450359.)

Does computer-aided detection have a role in the arbitration of discordant double-reading opinions in a breast-screening programme?

AIMS

To investigate whether a computer-aided detection (CAD) system could act as an arbitrator of discordant double-reading opinions, replacing the need for an independent third film reader.

METHODS

The mammograms of the 240 women that underwent arbitration by an independent third reader were identified from the 16,629 women attending our screening centre between July 2003 and April 2004. Mammograms of the arbitration cases were digitized and analysed by a CAD system. To assess the ability of CAD to act as the arbitrator, the site of the CAD prompts was retrospectively compared to the site of any abnormality noted by the original film readers. If a CAD prompt was placed on a region marked by one of the film readers then the decision of CAD as the arbitrator was that the women should be recalled for further assessment. If no mark was placed then the region was considered low risk and the decision was not to recall. The decision of CAD as the arbitrator was retrospectively compared with the original recall decision of the independent third reader.

RESULTS

There were 21 cancer cases in the group of women undergoing arbitration, diagnosed both at the original screening episode and subsequently. The independent third reader recalled 15/18 (83%) of the cancers that corresponded with the arbitrated lesion. CAD as the arbitrator would have recalled 16/18 (89%) of the cancers that corresponded to the arbitrated lesion. CAD acting as the arbitrator would have resulted in a significant increase in normal women being recalled to assessment in the arbitration group (P<0.001). The extra 50 recalls would have potentially increased the overall recall rate to assessment from 3.1 to 3.4%; a relative increase of 10%.

CONCLUSIONS

The main effect of CAD acting as an arbitrator of discordant double-reading opinions is to increase the recall rate, significantly above what is found when arbitration is performed by an independent third reader. Using CAD as an arbitrator may be an option to deal with discordant double-reading opinions when no other method of consensus or arbitration is available.