Is radiologists' volume of mammography reading related to accuracy? A critical review of the literature

Optimum screening mammography reading volumes: evidence from the NHS Breast Screening Programme

OBJECTIVES

Minimum caseload standards for professionals examining breast screening mammograms vary from 480 (US) to 5000 (Europe). We measured the relationship between the number of women's mammograms examined per year and reader performance.

METHODS

We extracted routine records from the English NHS Breast Screening Programme for readers examining between 1000 and 45,000 mammograms between April 2014 and March 2017. We measured the relationship between the volume of cases read and screening performance (cancer detection rate, recall rate, positive predictive value of recall (PPV) and discrepant cancers) using linear logistic regression. We also examined the effect of reader occupational group on performance.

RESULTS

In total, 759 eligible mammography readers (445 consultant radiologists, 235 radiography advanced practitioners, 79 consultant radiographers) examined 6.1 million women's mammograms during the study period. PPV increased from 12.9 to 14.4 to 17.0% for readers examining 2000, 5000 and 10000 cases per year respectively. This was driven by decreases in recall rates from 5.8 to 5.3 to 4.5 with increasing volume read, and no change in cancer detection rate (from 7.6 to 7.6 to 7.7). There was no difference in cancer detection rate with reader occupational group. Consultant radiographers had higher recall rate and lower PPV compared to radiologists (OR 1.105, p = 0.012; OR 0.874, p = 0.002, unadjusted).

CONCLUSION

Positive predictive value of screening increases with the total volume of cases examined per reader, through decreases in numbers of cases recalled with no concurrent change in numbers of cancers detected.

KEY POINTS

• In the English Breast Screening Programme, readers who examined a larger number of cases per year had a higher positive predictive value, because they recalled fewer women for further tests but detected the same number of cancers. • Reader type did not affect cancer detection rate, but consultant radiographers had a higher recall rate and lower positive predictive value than consultant radiologists, although this was not adjusted for length of experience.

Clinical reporting of radiographs by radiographers: Policy and practice guidance for regional imaging networks

OBJECTIVES

Radiographer reporting is an essential component of imaging across the United Kingdom. Since the previous policy and practice guidance in 2004 the role and contribution of reporting radiographers has changed significantly. The move to imaging networks further reinforces the need for consistency in scope of practice and clinical governance for radiographer reporting.

KEY FINDINGS

This guidance provides a consistent, evidence-based template for planning a reporting service, resourcing, clinical governance, preceptorship, volume and frequency of reporting, a peer learning framework and expected standards.

CONCLUSION

Developed for North Central and East London, this framework and standards will help reduce unwarranted variation.

IMPLICATIONS FOR PRACTICE

Consistency in practice could help maximise the contribution of radiographer reporting.

Performance of 4 years of population-based mammography screening for breast cancer combined with ultrasound in Tyrol / Austria

BACKGROUND

Systems for the delivery of screening mammography vary among countries and these differences can influence screening effectiveness. We evaluated the performance of organized mammography screening for breast cancer combined with ultrasound in Tyrol / Austria, an approach that differs from many other population-based screening programs.

METHODS

Data on women aged 40-69 years screened in the period from June 2008 to May 2012 were collected within the framework of an organized screening program. A total of 272,555 invitations were sent to the target population living in Tyrol and 176,957 screening examinations were performed. We analyzed the main performance indicators as defined by European Union (EU) guidelines and some important estimates of harms.

RESULTS

The estimated 2‑year participation rate was 56.9%. As ultrasound is implemented as second-line screening procedure, 76.2% of all women screened underwent supplementary ultrasound. In total 2322 women were recalled for further assessment (13.1 per 1000 screens) and 1351 biopsies were performed (7.6 per 1000 screens). The positive predictive value was 28.2% for assessment and 48.5% for biopsies. The cancer detection rate was 3.7 per 1000 screens and the proportion of all stage II+ screen-detected cancers was 35.5%. The interval cancer rate was 0.33 and 0.47 per 1000 screens in the first and second years, respectively. The estimated cumulative risk for a false positive screening result and an unnecessary biopsy for women following the invitation approach was 21.1% and 9.4%, respectively.

CONCLUSION

The performance of our population-based screening approach combining mammography and ultrasound is very favorable and potential harm is kept very low compared to other European mammography screening programs for breast cancer.

Factors influencing accuracy of referral and the likelihood of false positive referral by optometrists in Bradford, United Kingdom

AIMS

Levels of false positive referral to ophthalmology departments can be high. This study aimed to evaluate commonality between false positive referrals in order to find the factors which may influence referral accuracy.

METHODS

In 2007/08, a sample of 431 new Ophthalmology referrals from the catchment area of Bradford Royal Infirmary were retrospectively analysed.

RESULTS

The proportion of false positive referrals generated by optometrists decreases with experience at a rate of 6.2% per year since registration (p<0.0001). Community services which involved further investigation done by the optometrist before directly referring to the hospital were 2.7 times less likely to refer false positively than other referral formats (p=0.007). Male optometrists were about half as likely to generate a false positive referral than females (OR=0.51, p=0.008) and as multiple/corporate practices in the Bradford area employ less experienced and more female staff, independent practices generate about half the number of false positive referrals (OR=0.52, p=0.005).

CONCLUSIONS

Clinician experience has the greatest effect on referral accuracy although there is also a significant effect of gender with women tending to refer more false positives. This may be due to a different approach to patient care and possibly a greater sensitivity to litigation. The improved accuracy of community services (which often refer directly after further investigation) supports further growth of these schemes.

The National Mammography Database: Preliminary Data

OBJECTIVE

The purposes of our study were to analyze screening mammography data submitted to the National Mammography Database (NMD) since its inception to confirm data collection feasibility, to draw parallels to data from the Breast Cancer Surveillance Consortium (BCSC), and to examine trends over time. We also retrospectively evaluated practice-level variation in terms of practice type, practice setting, census region, and annual volume.

MATERIALS AND METHODS

Data from 90 mammography facilities in the NMD registry were analyzed. The registry receives mammography data collected as part of standard clinical practice, including self-reported demographic information, clinical findings, screening mammography interpretation, and biopsy results. Outcome metrics calculated were cancer detection rate, recall rate, and positive predictive values for biopsy recommended (PPV2) and biopsy performed (PPV3).

RESULTS

The NMD successfully collected and analyzed data for 3,181,437 screening mammograms performed between January 2008 and December 2012. Mean values for outcomes were cancer detection rate of 3.43 per 1000 (95% CI, 3.2-3.7), recall rate of 10% (95% CI, 9.3-10.7%), PPV2 of 18.5% (95% CI, 16.7-20.2%), and PPV3 of 29.2% (95% CI, 26.2-32.3%). No statistically significant difference was seen in performance measurements on the basis of practice type, practice setting, census region, or annual volume. NMD performance measurements parallel those reported by the BCSC.

CONCLUSION

The NMD has become the fastest growing mammography registry in the United States, providing nationwide performance metrics and permitting comparison with published benchmarks. Our study shows the feasibility of using the NMD to audit mammography facilities and to provide current, ongoing benchmark data.

Reader practice in mammography screen reporting in Australia

Reader variability is a problem in mammography image reporting and compromises the efficacy of screening programmes. The purpose of this exploratory study was to survey reader practice in reporting screening mammograms in Australia to identify aspects of practice that warrant further investigation. Mammography reporting practice and influences on concentration and attention were investigated by using an original questionnaire distributed to screen readers in Australia. A response rate of 71% (83 out of 117) was achieved. Demographic data indicated that the majority of readers were over 46 years of age (73%), have been reporting on screening mammograms for over 10 years (61%), take less than 1 min to report upon a screening mammogram examination (66%), report up to 200 examinations in a single session (83%) and take up to 2 h to report one session (61%). A majority report on more than 5000 examinations annually (66%); 93% of participants regard their search strategy as systematic, 87% agreed that their concentration can vary throughout a session, 64% agreed that the relatively low number of positives can lead to lapses in concentration and attention and almost all (94%) participants agreed that methods to maximise concentration should be explored. Participants identified a range of influences on concentration within their working environment including volume of images reported in one session, image types and aspects of the physical environment. This study has provided important evidence of the need to investigate adverse influences on concentration during mammography screen reporting.

Screening-detected breast cancers: discordant independent double reading in a population-based screening program

PURPOSE

To analyze discordant and concordant screening-detected breast cancers in a nationwide population-based screening program by using independent double reading with consensus.

MATERIALS AND METHODS

The study is a part of the evaluation of the Norwegian Breast Cancer Screening Program and is covered by the Cancer Registry regulation. Analyses were based on prospective initial interpretation scores of 1 033 870 screenings that included 5611 breast cancers. A five-point scale for probability of cancer was used in the initial interpretation. Screening mammograms with a score of 2 or higher by either radiologist were discussed at consensus meetings where the decision whether to recall was made. A score of 1 by one reader and 2 or higher by the other was defined as a discordant interpretation and discordant cancer, whereas a score of 2 or higher by both readers was defined as a concordant recall and cancer.

RESULTS

Discordant interpretation was present in 5.3% (54 447 of 1 033 870) of the screenings, whereas 2.1% (21 928 of 1 033 870) were concordant positive interpretations. Of the screening-detected cancers, 23.6% (1326 of 5611) were diagnosed in women who were recalled because of screenings with discordant interpretation. One hundred seventeen interval breast cancers were diagnosed among the 40 312 screenings that were dismissed at consensus; these were 6.5% of all interval cancers. A significantly higher proportion of microcalcifications alone was present in discordant cancers (24.9% [304 of 1219]) compared with concordant cancers (17.7% [704 of 3972]) (P < .001).

CONCLUSION

Independent double reading with consensus at mammography screening has the potential to increase the cancer detection rate compared with single reading. Mammograms with microcalcifications alone are significantly more common among discordant cancers.

Computer-aided detection in full-field digital mammography in a clinical population: performance of radiologist and technologists

The purpose of the study was to evaluate the impact of a computer-aided detection (CAD) system on the performance of mammogram readers in interpreting digital mammograms in a clinical population. Furthermore, the ability of a CAD system to detect breast cancer in digital mammography was studied in comparison to the performance of radiologists and technologists as mammogram readers. Digital mammograms of 1,048 consecutive patients were evaluated by a radiologist and three technologists. Abnormalities were recorded and an imaging conclusion was given as a BI-RADS score before and after CAD analysis. Pathology results during 12 months follow up were used as a reference standard for breast cancer. Fifty-one malignancies were found in 50 patients. Sensitivity and specificity were computed before and after CAD analysis and provided with 95% CIs. In order to assess the detection rate of malignancies by CAD and the observers, the pathological locations of these 51 breast cancers were matched with the locations of the CAD marks and the mammographic locations that were considered to be suspicious by the observers. For all observers, the sensitivity rates did not change after application of CAD. A mean sensitivity of 92% was found for all technologists and 84% for the radiologist. For two technologists, the specificity decreased (from 84 to 83% and from 77 to 75%). For the radiologist and one technologist, the application of CAD did not have any impact on the specificity rates (95 and 83%, respectively). CAD detected 78% of all malignancies. Five malignancies were indicated by CAD without being noticed as suspicious by the observers. In conclusion, the results show that systematic application of CAD in a clinical patient population failed to improve the overall sensitivity of mammogram interpretation by the readers and was associated with an increase in false-positive results. However, CAD marked five malignancies that were missed by the different readers.

Accuracy of short-interval follow-up mammograms by patient and radiologist characteristics

OBJECTIVE

The purpose of our study was to examine the accuracy of short-interval follow-up mammograms and evaluate patient and radiologist characteristics associated with accuracy.

MATERIALS AND METHODS

We evaluated 45,007 initial short-interval follow-up mammograms from the Breast Cancer Surveillance Consortium interpreted 3-9 months after a probably benign assessment on a screening or diagnostic examination between 1994 and 2004. We linked these mammograms with patient characteristics and breast cancer diagnoses within 12 months. A subset of short-interval follow-up examinations (n = 13,907) was merged with radiologist characteristics collected from survey data from 130 interpreting radiologists. Using logistic regression, we fit generalized estimating equations to model sensitivity and specificity of short-interval follow-up mammograms by patient and radiologist characteristics.

RESULTS

For every 1,000 women, 8.0 women (0.8%) were diagnosed with breast cancer within 6 months and 11.3 (1.1%) within 12 months. Sensitivity was 83.3% (95% CI, 79.4-87.3%) for cancers diagnosed within 6 months and 60.5% (56.2-64.7%) for those diagnosed within 12 months. Specificity was 97.2% (96.9-97.6%) at 6 months and 97.3% (96.9-97.6%) at 12 months. Sensitivity at 12 months increased among women with unilateral short-interval follow-up mammograms (odds ratio, 1.56 [95% CI, 1.06-2.29]) and when the interpreting radiologist spent more than 10 hours a week in breast imaging (odds ratio, 3.25 [1.00-10.52]).

CONCLUSION

Initial short-interval follow-up mammography examinations had a lower sensitivity for detecting breast cancer within 12 months than other diagnostic mammograms (61% for short-interval follow-up vs 80% for diagnostic mammograms reported in the literature). However, sensitivity within the 6-month interval that is usually recommended for subsequent follow-up was 83%. Accuracy of short-interval follow-up mammograms was influenced by few patient and radiologist characteristics.

Provider’s volume and quality of breast cancer detection and treatment

For many health conditions, the process or result of medical procedures improves with increasing caseload. The evidence about breast cancer has not been thoroughly assessed. This review synthesizes the literature about provider's volume and performance in either breast cancer screening with mammography or treatment. Articles published in English between 1990 and 2006 were identified by a computerized search and by review of reference lists. In screening with mammography, the reading volume of the radiologist and the screening volume of the facility influence different components of performance. The most conclusive evidence for breast cancer treatment concerns the association between the surgeon's caseload and the process or end-results of therapeutic interventions. Although the mechanisms of these associations still need to be clarified, large provider's volume in screening mammography or breast cancer treatment is often related to the quality of medical interventions.