Influence of annual interpretive volume on screening mammography performance in the United States

Feasibility of predicting a screening digital breast tomosynthesis recall using features extracted from the electronic medical record

PURPOSE

Tools to predict a screening mammogram recall at the time of scheduling could improve patient care. We extracted patient demographic and breast care history information within the electronic medical record (EMR) for women undergoing digital breast tomosynthesis (DBT) to identify which factors were associated with a screening recall recommendation.

METHOD

In 2018, 21,543 women aged 40 years or greater who underwent screening DBT at our institution were identified. Demographic information and breast care factors were extracted automatically from the EMR. The primary outcome was a screening recall recommendation of BI-RADS 0. A multivariable logistic regression model was built and included age, race, ethnicity groups, family breast cancer history, personal breast cancer history, surgical breast cancer history, recall history, and days since last available screening mammogram.

RESULTS

Multiple factors were associated with a recall on the multivariable model: history of breast cancer surgery (OR: 2.298, 95% CI: 1.854, 2.836); prior recall within the last five years (vs no prior, OR: 0.768, 95% CI: 0.687, 0.858); prior screening mammogram within 0-18 months (vs no prior, OR: 0.601, 95% CI: 0.520, 0.691), prior screening mammogram within 18-30 months (vs no prior, OR: 0.676, 95% CI: 0.520, 0.691); and age (normalized OR: 0.723, 95% CI: 0.690, 0.758).

CONCLUSIONS

It is feasible to predict a DBT screening recall recommendation using patient demographics and breast care factors that can be extracted automatically from the EMR.

Reducing ' probably benign ' assessments in normal mammograms: The role of radiologist experience

Rationale and objectives

to investigate the relationship between radiologists' experience in reporting mammograms, their caseloads, and the classification of category '3' or 'Probably Benign' on normal mammograms.

Materials and Methods

A total of 92 board-certified radiologists participated. Self-reported parameters related to experience, including age, years since qualifying as a radiologist, years of experience reading mammograms, number of mammograms read per year, and hours spent reading mammograms per week, were documented. To assess the radiologists' accuracy, "Probably Benign fractions" was calculated by dividing the number of "Probably Benign findings" given by each radiologist in the normal cases by the total number of normal cases Probably Benign fractions were correlated with various factors, such as the radiologists' experience.

Results

The results of the statistical analysis revealed a significant negative correlation between radiologist experience and 'Probably Benign' fractions for normal images. Specifically, for normal cases, the number of mammograms read per year (r = -0.29, P = 0.006) and the number of mammograms read over the radiologist's lifetime (r = -0.21, P = 0.049) were both negatively correlated with 'Probably Benign' fractions.

Conclusion

The results indicate that a relationship exists between increased reading volumes and reduced assessments of 'Probably Benign' in normal mammograms. The implications of these findings extend to the effectiveness of screening programs and the recall rates.

National Performance Benchmarks for Screening Digital Breast Tomosynthesis: Update from the Breast Cancer Surveillance Consortium

Background It is important to establish screening mammography performance benchmarks for quality improvement efforts. Purpose To establish performance benchmarks for digital breast tomosynthesis (DBT) screening and evaluate performance trends over time in U.S. community practice. Materials and Methods In this retrospective study, DBT screening examinations were collected from five Breast Cancer Surveillance Consortium (BCSC) registries between 2011 and 2018. Performance measures included abnormal interpretation rate (AIR), cancer detection rate (CDR), sensitivity, specificity, and false-negative rate (FNR) and were calculated based on the American College of Radiology Breast Imaging Reporting and Data System, fifth edition, and compared with concurrent BCSC DM screening examinations, previously published BCSC and National Mammography Database benchmarks, and expert opinion acceptable performance ranges. Benchmarks were derived from the distribution of performance measures across radiologists (n = 84 or n = 73 depending on metric) and were presented as percentiles. Results A total of 896 101 women undergoing 2 301 766 screening examinations (458 175 DBT examinations [median age, 58 years; age range, 18-111 years] and 1 843 591 DM examinations [median age, 58 years; age range, 18-109 years]) were included in this study. DBT screening performance measures were as follows: AIR, 8.3% (95% CI: 7.5, 9.3); CDR per 1000 screens, 5.8 (95% CI: 5.4, 6.1); sensitivity, 87.4% (95% CI: 85.2, 89.4); specificity, 92.2% (95% CI: 91.3, 93.0); and FNR per 1000 screens, 0.8 (95% CI: 0.7, 1.0). When compared with BCSC DM screening examinations from the same time period and previously published BCSC and National Mammography Database performance benchmarks, all performance measures were higher for DBT except sensitivity and FNR, which were similar to concurrent and prior DM performance measures. The following proportions of radiologists achieved acceptable performance ranges with DBT: 97.6% for CDR, 91.8% for sensitivity, 75.0% for AIR, and 74.0% for specificity. Conclusion In U.S. community practice, large proportions of radiologists met acceptable performance ranges for screening performance metrics with DBT. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Lee and Moy in this issue.

Do Reader Characteristics Affect Diagnostic Efficacy in Screening Mammography? A Systematic Review

To examine reader characteristics associated with diagnostic efficacy in the interpretation of screening mammograms. A systematic search of the literature was conducted using databases such as Cochrane, Scopus, Medline, Embase, Web of Science, and PubMed. Search terms were combined with "AND" or "OR" and included: "Radiologist's characteristics AND performance"; "radiologist experience AND screening mammography"; "annual volume read AND diagnostic efficacy"; "screening mammography performance OR diagnostic efficacy". Studies were included if they assessed reader performance in screening mammography interpretation, breast readers, used a reference standard to assess the performance, and were published in the English language. Twenty-eight studies were reviewed. Increasing reader's age was associated with lower false positive rates. No association was found between gender and performance. Half of the studies showed no association between years of reading mammograms and performance. Most studies showed that high reading volume was more likely to be associated with increased sensitivity, cancer detection rates (CDR), lower recall rate, and lower false positive rates. Inconsistent associations were found between fellowship training in breast imaging and reader performance. Specialization in breast imaging was associated with better CDR, sensitivity, and specificity. Limited studies were available to establish the association between performance and factors such as time spent in breast imaging (n = 2), screening focus (n = 1), formal rotation in mammography (n = 1), owner of practice (n = 1), and practice type (n = 1). No individual characteristics is associated with versatility in diagnostic efficacy, albeit reading volume and specialization in breast imaging appear to be associated with with increased sensitivity and CDR without significantly affecting other performance metrics.

Live Mammographic Screening Interpretation Versus Offline Same-Day Screening Interpretation at a Tertiary Cancer Center

OBJECTIVES

The aim of this study was to compare screening mammography performance metrics for immediate (live) interpretation versus offline interpretation at a cancer center.

METHODS

An institutional review board-approved, retrospective comparison of screening mammography metrics at a cancer center for January 1, 2018, to December 31, 2019 (live period), and September 1, 2020, to March 31, 2022 (offline period), was performed. Before July 2020, screening examinations were interpreted while patients waited (live period), and diagnostic workup was performed concurrently. After the coronavirus disease 2019 shutdown from March to mid-June 2020, offline same-day interpretation was instituted. Patients with abnormal screening results returned for separate diagnostic evaluation. Screening metrics of positive predictive value 1 (PPV1), cancer detection rate (CDR), and abnormal interpretation rate (AIR) were compared for 17 radiologists who interpreted during both periods. Statistical significance was assessed using χ² analysis.

RESULTS

In the live period, there were 7,105 screenings, 635 recalls, and 51 screen-detected cancers. In the offline period, there were 7,512 screenings, 586 recalls, and 47 screen-detected cancers. Comparison of live screening metrics versus offline metrics produced the following results: AIR, 8.9% (635 of 7,105) versus 7.8% (586 of 7,512) (P = .01); PPV1, 8.0% (51 of 635) versus 8.0% (47 of 586); and CDR, 7.2/1,000 versus 6.3/1,000 (P = .50). When grouped by >10% AIR or <10% AIR for the live period, the >10% AIR group showed a significant decrease in AIR for offline interpretation (from 12.7% to 9.7%, P < .001), whereas the <10% AIR group showed no significant change (from 7.4% to 6.7%, P = .17).

CONCLUSIONS

Conversion to offline screening interpretation from immediate interpretation at a cancer center was associated with lower AIR and similar CDR and PPV1. This effect was seen largely in radiologists with AIR > 10% in the live setting.

Patterns of Screening Recall Behavior Among Subspecialty Breast Radiologists

RATIONALE AND OBJECTIVES

Determine whether there are patterns of lesion recall among breast imaging subspecialists interpreting screening mammography, and if so, whether recall patterns correlate to morphologies of screen-detected cancers.

MATERIALS AND METHODS

This Institutional Review Board-approved, retrospective review included all screening examinations January 3, 2012-October 1, 2018 interpreted by fifteen breast imaging subspecialists at a large academic medical center and two outpatient imaging centers. Natural language processing identified radiologist recalls by lesion type (mass, calcifications, asymmetry, architectural distortion); proportions of callbacks by lesion types were calculated per radiologist. Hierarchical cluster analysis grouped radiologists based on recall patterns. Groups were compared to overall practice and each other by proportions of lesion types recalled, and overall and lesion-specific positive predictive value-1 (PPV1).

RESULTS

Among 161,859 screening mammograms with 13,086 (8.1%) recalls, Hierarchical cluster analysis grouped 15 radiologists into five groups. There was substantial variation in proportions of lesions recalled: calcifications 13%-18% (Chi-square 45.69, p < 0.00001); mass 16%-44% (Chi-square 498.42, p < 0.00001); asymmetry 13%-47% (Chi-square 660.93, p < 0.00001) architectural distortion 6%-20% (Chi-square 283.81, p < 0.00001). Radiologist groups differed significantly in overall PPV1 (range 5.6%-8.8%; Chi-square 17.065, p = 0.0019). PPV1 by lesion type varied among groups: calcifications 9.2%-15.4% (Chi-square 2.56, p = 0.6339); mass 5.6%-8.5% (Chi-square 1.31, p = 0.8597); asymmetry 3.4%-5.9% (Chi-square 2.225, p = 0.6945); architectural distortion 5.6%-10.8% (Chi-square 5.810, p = 0.2138). Proportions of recalled lesions did not consistently correlate to proportions of screen-detected cancer.

CONCLUSION

Breast imaging subspecialists have patterns for screening mammography recalls, suggesting differential weighting of imaging findings for perceived malignant potential. Radiologist recall patterns are not always predictive of screen-detected cancers nor lesion-specific PPV1s.

A machine learning model based on readers' characteristics to predict their performances in reading screening mammograms

OBJECTIVES

Proposing a machine learning model to predict readers' performances, as measured by the area under the receiver operating characteristics curve (AUC) and lesion sensitivity, using the readers' characteristics.

METHODS

Data were collected from 905 radiologists and breast physicians who completed at least one case-set of 60 mammographic images containing 40 normal and 20 biopsy-proven cancer cases. Nine different case-sets were available. Using a questionnaire, we collected radiologists' demographic details, such as reading volume and years of experience. These characteristics along with a case set difficulty measure were fed into two ensemble of regression trees to predict the readers' AUCs and lesion sensitivities. We calculated the Pearson correlation coefficient between the predicted values by the model and the actual AUC and lesion sensitivity. The usefulness of the model to categorize readers as low and high performers based on different criteria was also evaluated. The performances of the models were evaluated using leave-one-out cross-validation.

RESULTS

The Pearson correlation coefficient between the predicted AUC and actual one was 0.60 (p < 0.001). The model's performance for differentiating the reader in the first and fourth quartile based on the AUC values was 0.86 (95% CI 0.83-0.89). The model reached an AUC of 0.91 (95% CI 0.88-0.93) for distinguishing the readers in the first quartile from the fourth one based on the lesion sensitivity.

CONCLUSION

A machine learning model can be used to categorize readers as high- or low-performing. Such model could be useful for screening programs for designing a targeted quality assurance and optimizing the double reading practice.

Cumulative risk of a false-positive screening result: A retrospective cohort study using empirical data from 10 biennial screening rounds in BreastScreen Norway

BACKGROUND

False-positive screening results are an inevitable and commonly recognized disadvantage of mammographic screening. This study estimated the cumulative probability of experiencing a first false-positive screening result in women attending 10 biennial screening rounds in BreastScreen Norway, which targets women aged 50 to 69 years.

METHODS

This retrospective cohort study analyzed screening outcomes from 421,545 women who underwent 1,894,523 screening examinations during 1995-2019. Empirical data were used to calculate the cumulative risk of experiencing a first false-positive screening result and a first false-positive screening result that involved an invasive procedure over 10 screening rounds. Logistic regression was used to evaluate the effect of adjusting for irregular attendance, age at screening, and number of screens attended.

RESULTS

The cumulative risk of experiencing a first false-positive screening result was 18.04% (95% confidence interval [CI], 18.00%-18.07%). It was 5.01% (95% CI, 5.01%-5.02%) for experiencing a false-positive screening result that involved an invasive procedure. Adjusting for irregular attendance or age at screening did not appreciably affect these estimates. After adjustments for the number of screens attended, the cumulative risk of a first false-positive screening result was 18.28% (95% CI, 18.24%-18.32%), and the risk of a false-positive screening result including an invasive procedure was 5.11% (95% CI, 5.11%-5.22%). This suggested that there was minimal bias from dependent censoring.

CONCLUSIONS

Nearly 1 in 5 women will experience a false-positive screening result if they attend 10 biennial screening rounds in BreastScreen Norway. One in 20 will experience a false-positive screening result with an invasive procedure.

LAY SUMMARY

A false-positive screening result occurs when a woman attending mammographic screening is called back for further assessment because of suspicious findings, but the assessment does not detect breast cancer. Further assessment includes additional imaging. Usually, it involves ultrasound, and sometimes, it involves a biopsy. This study has evaluated the chance of experiencing a false-positive screening result among women attending 10 screening examinations over 20 years in BreastScreen Norway. Nearly 1 in 5 women will experience a false-positive screening result over 10 screening rounds. One in 20 women will experience a false-positive screening result involving a biopsy.

Classification of Breast Cancer in Mammograms with Deep Learning Adding a Fifth Class

Breast cancer is one of the diseases of most profound concern, with the most prevalence worldwide, where early detections and diagnoses play the leading role against this disease achieved through imaging techniques such as mammography. Radiologists tend to have a high false positive rate for mammography diagnoses and an accuracy of around 82%. Currently, deep learning (DL) techniques have shown promising results in the early detection of breast cancer by generating computer-aided diagnosis (CAD) systems implementing convolutional neural networks (CNNs). This work focuses on applying, evaluating, and comparing the architectures: AlexNet, GoogLeNet, Resnet50, and Vgg19 to classify breast lesions after using transfer learning with fine-tuning and training the CNN with regions extracted from the MIAS and INbreast databases. We analyzed 14 classifiers, involving 4 classes as several researches have done it before, corresponding to benign and malignant microcalcifications and masses, and as our main contribution, we also added a 5th class for the normal tissue of the mammary parenchyma increasing the correct detection; in order to evaluate the architectures with a statistical analysis based on the received operational characteristics (ROC), the area under the curve (AUC), F1 Score, accuracy, precision, sensitivity, and specificity. We generate the best results with the CNN GoogLeNet trained with five classes on a balanced database with an AUC of 99.29%, F1 Score of 91.92%, the accuracy of 91.92%, precision of 92.15%, sensitivity of 91.70%, and specificity of 97.66%, concluding that GoogLeNet is optimal as a classifier in a CAD system to deal with breast cancer.

The Impact of Radiologist Screening Mammogram Reading Volume on Performance in the Ontario Breast Screening Program

PURPOSE

Although some studies have shown increasing radiologists' mammography volumes improves performance, there is a lack of evidence specific to digital mammography and breast screening program performance targets. This study evaluates the relationship between digital screening volume and meeting performance targets.

METHODS

This retrospective cohort study included 493 radiologists in the Ontario Breast Screening Program who interpreted 1,762,173 screening mammograms in participants ages 50-90 between 2014 and 2016. Associations between annual screening volume and meeting performance targets for abnormal call rate, positive predictive value (PPV), invasive cancer detection rate (CDR), sensitivity, and specificity were modeled using mixed-effects multivariate logistic regression.

RESULTS

Most radiologists read 500-999 (36.7%) or 1,000-1,999 (31.0%) screens annually, and 18.5% read ≥2,000. Radiologists who read ≥2,000 annually were more likely to meet abnormal call rate (OR = 3.85; 95% CI: 1.17-12.61), PPV (OR = 5.36; 95% CI: 2.53-11.34), invasive CDR (OR = 4.14; 95% CI: 1.50-11.46), and specificity (OR = 4.07; 95% CI: 1.89-8.79) targets versus those who read 100-499 screens. Radiologists reading 1,000-1,999 screens annually were more likely to meet PPV (OR = 2.32; 95% CI: 1.22-4.40), invasive CDR (OR = 3.36; 95% CI: 1.49-7.59) and specificity (OR = 2.00; 95% CI: 1.04-3.84) targets versus those who read 100-499 screens. No significant differences were observed for sensitivity.

CONCLUSIONS

Annual reading volume requirements of 1,000 in Canada are supported as screening volume above 1,000 was strongly associated with achieving performance targets for nearly all measures. Increasing the minimum volume to 2,000 may further reduce the potential limitations of screening due to false positives, leading to improvements in overall breast screening program quality.

Radiologist Characteristics Associated with Interpretive Performance of Screening Mammography: A National Mammography Database (NMD) Study

Background Factors affecting radiologists' performance in screening mammography interpretation remain poorly understood. Purpose To identify radiologists characteristics that affect screening mammography interpretation performance. Materials and Methods This retrospective study included 1223 radiologists in the National Mammography Database (NMD) from 2008 to 2019 who could be linked to Centers for Medicare & Medicaid Services (CMS) datasets. NMD screening performance metrics were extracted. Acceptable ranges were defined as follows: recall rate (RR) between 5% and 12%; cancer detection rate (CDR) of at least 2.5 per 1000 screening examinations; positive predictive value of recall (PPV1) between 3% and 8%; positive predictive value of biopsies recommended (PPV2) between 20% and 40%; positive predictive value of biopsies performed (PPV3) between the 25th and 75th percentile of study sample; invasive CDR of at least the 25th percentile of the study sample; and percentage of ductal carcinoma in situ (DCIS) of at least the 25th percentile of the study sample. Radiologist characteristics extracted from CMS datasets included demographics, subspecialization, and clinical practice patterns. Multivariable stepwise logistic regression models were performed to identify characteristics independently associated with acceptable performance for the seven metrics. The most influential characteristics were defined as those independently associated with the majority of the metrics (at least four). Results Relative to radiologists practicing in the Northeast, those in the Midwest were more likely to achieve acceptable RR, PPV1, PPV2, and CDR (odds ratio [OR], 1.4-2.5); those practicing in the West were more likely to achieve acceptable RR, PPV2, and PPV3 (OR, 1.7-2.1) but less likely to achieve acceptable invasive CDR (OR, 0.6). Relative to general radiologists, breast imagers were more likely to achieve acceptable PPV1, invasive CDR, percentage DCIS, and CDR (OR, 1.4-4.4). Those performing diagnostic mammography were more likely to achieve acceptable PPV1, PPV2, PPV3, invasive CDR, and CDR (OR, 1.9-2.9). Those performing breast US were less likely to achieve acceptable PPV1, PPV2, percentage DCIS, and CDR (OR, 0.5-0.7). Conclusion The geographic location of the radiology practice, subspecialization in breast imaging, and performance of diagnostic mammography are associated with better screening mammography performance; performance of breast US is associated with lower performance. ^©RSNA, 2021 Online supplemental material is available for this article.

Optimum screening mammography reading volumes: evidence from the NHS Breast Screening Programme

OBJECTIVES

Minimum caseload standards for professionals examining breast screening mammograms vary from 480 (US) to 5000 (Europe). We measured the relationship between the number of women's mammograms examined per year and reader performance.

METHODS

We extracted routine records from the English NHS Breast Screening Programme for readers examining between 1000 and 45,000 mammograms between April 2014 and March 2017. We measured the relationship between the volume of cases read and screening performance (cancer detection rate, recall rate, positive predictive value of recall (PPV) and discrepant cancers) using linear logistic regression. We also examined the effect of reader occupational group on performance.

RESULTS

In total, 759 eligible mammography readers (445 consultant radiologists, 235 radiography advanced practitioners, 79 consultant radiographers) examined 6.1 million women's mammograms during the study period. PPV increased from 12.9 to 14.4 to 17.0% for readers examining 2000, 5000 and 10000 cases per year respectively. This was driven by decreases in recall rates from 5.8 to 5.3 to 4.5 with increasing volume read, and no change in cancer detection rate (from 7.6 to 7.6 to 7.7). There was no difference in cancer detection rate with reader occupational group. Consultant radiographers had higher recall rate and lower PPV compared to radiologists (OR 1.105, p = 0.012; OR 0.874, p = 0.002, unadjusted).

CONCLUSION

Positive predictive value of screening increases with the total volume of cases examined per reader, through decreases in numbers of cases recalled with no concurrent change in numbers of cancers detected.

KEY POINTS

• In the English Breast Screening Programme, readers who examined a larger number of cases per year had a higher positive predictive value, because they recalled fewer women for further tests but detected the same number of cancers. • Reader type did not affect cancer detection rate, but consultant radiographers had a higher recall rate and lower positive predictive value than consultant radiologists, although this was not adjusted for length of experience.

Understanding the Mammography Audit

Screening mammography aims to identify small, node-negative breast cancers when they are still curable while maintaining an acceptable range of false-positive recalls and biopsies. The mammography audit is a powerful tool to help radiologists understand their performance with respect to that goal. This article defines audit terms and describes how to use collected and derived data to perform a mammography audit. Accepted benchmarks are discussed as well as their applicability to radiologists and breast imaging practices in the United States. Special considerations regarding volumes and radiologist characteristics are explored, because these factors may affect audit results.

Development and Assessment of a New Global Mammographic Image Feature Analysis Scheme to Predict Likelihood of Malignant Cases

This study aims to develop and evaluate a new computer-aided diagnosis (CADx) scheme based on analysis of global mammographic image features to predict likelihood of cases being malignant. An image dataset involving 1,959 cases was retrospectively assembled. Suspicious lesions were detected and biopsied in each case. Among them, 737 cases are malignant and 1,222 are benign. Each case includes four mammograms of craniocaudal and mediolateral oblique view of left and right breasts. CADx scheme is applied to pre-process mammograms, generate two image maps in frequency domain using discrete cosine transform and fast Fourier transform, compute bilateral image feature differences from left and right breasts, and apply a support vector machine (SVM) to predict likelihood of the case being malignant. Three sub-groups of image features were computed from the original mammograms and two transformation maps. Four SVMs using three sub-groups of image features and fusion of all features were trained and tested using a 10-fold cross-validation method. The computed areas under receiver operating characteristic curves (AUCs) range from 0.85 to 0.91 using image features computed from one of three sub-groups, respectively. By fusion of all image features computed in three sub-groups, the fourth SVM yields a significantly higher performance with AUC = 0.96±0.01 (p<0.01). This study demonstrates feasibility of developing a new global image feature analysis based CADx scheme of mammograms with high performance. By avoiding difficulty and possible errors in breast lesion segmentation, this new CADx approach is more efficient in development and potentially more robust in future application.

Telemammography: Technical Advances Improve Patient Access in Breast Care

Screening mammography's efficacy in reducing breast cancer deaths depends on patient compliance with screening recommendations and the radiologist's interpretative skills. Reasons for suboptimal screening compliance may be multifactorial, including possible limitations in access. Additionally, while studies show experienced breast radiologists are more accurate in their mammographic interpretation, only a minority of the nation's mammograms are interpreted by breast imaging specialists. To simultaneously optimize the benefit of early breast cancer detection while minimizing the harms associated with a false positive interpretation, delivery models that help improve access to breast expertise should be considered. Telemammography is one such delivery model that may be underutilized in current practice. While radiologists and other stakeholders of healthcare have accepted teleradiology interpretation of non-mammography studies as routine, telemammography use and acceptance is less well known. In this article, we review the operational components of a telemammography practice in today's information- and technology-dependent society. Current use of telemammography and remaining potential challenges are discussed. Telemammography can improve healthcare delivery and access by bringing together patients and breast expertise. If accepted, use of telemammography can help meet Centers for Disease Control's Healthy People 2020 goals related to breast cancer.

Low-Volume and High-Volume Readers of Neurological and Musculoskeletal MRI: Achieving Subspecialization in Radiology

OBJECTIVE

Differentiate high- versus low-volume radiologists who interpret neurological (Neuro) MRI or musculoskeletal (MSK) MRI and measure the proportion of Neuro and MSK MRIs read by low-volume radiologists.

METHODS

We queried the 2015 Medicare Physician and Other Supplier Public Use File for radiologists who submitted claims for Neuro or MSK MRIs. Radiologists were classified as high-volume versus low-volume based on their work relative value units (wRVUs) focus or volume of studies interpreted using three different methodologies: Method 1, percentage of wRVUs in Neuro or MSK MRI; Method 2, absolute number of Neuro or MSK MRIs interpreted; and Method 3, both percentage and absolute number. Multiple thresholds with each methodology were tested, and the percentage of Neuro or MSK MRIs interpreted by low-volume radiologists was calculated for each threshold.

RESULTS

With Method 1, 33% of Neuro MRI and 50% of MSK MRI studies were interpreted by a radiologist whose wRVUs in Neuro or MSK MRI were less than 20% (Method 1). With Method 2, 22% of Neuro MRIs and 37% of MSK MRIs were interpreted by radiologists who read fewer than the mean number of Neuro or MSK MRIs interpreted by an "average full-time radiologist" whose wRVUs in Neuro or MSK MRI were approximately 20%. With Method 3, 38% of Neuro MRIs and 57% of MSK MRIs were interpreted by "low-volume" radiologists. If instead a 50% wRVU threshold is used for Methods One, Two, and Three, then 70%, 58%, and 77% of Neuro MRIs and 86%, 80%, and 90% of MSK MRIs are read by low-volume radiologists.

DISCUSSION

A large number of radiologists read a low volume of Neuro or MSK MRIs; these low-volume Neuro or MSK MRI radiologists read a substantial portion of Neuro or MSK MRIs. It is unknown which of the methods for distinguishing low-volume radiologists, combined with which threshold, may best correlate with high-performing or low-performing radiologists.

Association of volume of self-directed versus assigned interpretive work with diagnostic performance of radiologists: an observational study

OBJECTIVES

To understand the sources of variability in diagnostic performance among experienced radiologists.

DESIGN

All prostate MRI examinations performed between 2016 and 2018 were retrospectively reviewed.

SETTING

University hospital in Japan.

PARTICIPANTS

Data derived from 334 pathology-proven cases (male, mean age: 70 years; range: 35-90 years) that were interpreted by 10 experienced radiologists were subjected to the analysis.

PRIMARY AND SECONDARY OUTCOME MEASURES

Diagnostic performance measures of the radiologists were compared with candidate factors, including interpretive volume of prostate MRIs, volume of self-directed and assigned total annual interpretive work, and years of experience. The potential influence of fatigue was also evaluated by examining the effect of the report's issue time.

RESULTS

There were 186 prostate cancer cases. Performance was based on accuracy, sensitivity and specificity (86%, 85% and 84%, respectively). While performance was not correlated with the volume of prostate MRIs, per se (ρ=-0.15, p=0.69; ρ=-0.01, p=0.99; ρ=-0.33, p=0.36) or the total MRIs assigned for each radiologist (p>0.6) or years of experience (p>0.4), all measures were strongly correlated with voluntary work represented by the interpretive volume of abdominal CTs (r=0.79, p<0.01; r=0.80, p<0.01; r=0.64, p=0.048). The performance did not differ based on the issue time of the report (morning, afternoon and evening) (χ²(2)=3.65, p=0.16).

CONCLUSIONS

Greater autonomy, represented as enhanced self-directed interpretive work, was most significantly correlated with the performance of prostate MRI interpretation. The lack of a correlation between the performance and assigned volume confirms the complexity of human learning. Together, these findings support the hypothesis that successful promotion of internal drivers could have a pervasive positive impact on improving diagnostic performance.

Prognostic Case Volume Thresholds in Patients With Head and Neck Squamous Cell Carcinoma

IMPORTANCE

Though described as an important prognostic indicator, facility case volume thresholds for patients with head and neck squamous cell carcinoma (HNSCC) have not been previously developed to date.

OBJECTIVE

To identify prognostic case volume thresholds of facilities that manage HNSCC.

DESIGN, SETTING, AND PARTICIPANTS

Retrospective analysis of 351 052 HNSCC cases reported from January 1, 2004, through December 31, 2014, by Commission of Cancer-accredited cancer centers from the US National Cancer Database. Data were analyzed from August 1, 2018, to April 5, 2019.

EXPOSURES

Treatment of HNSCC at facilities with varying case volumes.

MAIN OUTCOMES AND MEASURES

Using all-cause mortality outcomes among adult patients with HNSCC, 10 groups with increasing facility case volume were created and thresholds were identified where group survival differed compared with each of the 2 preceding groups (univariate log-rank analysis). Groups were collapsed at these thresholds and the prognostic value was confirmed using multivariable Cox regression. Prognostic meaning of these thresholds was assessed in subgroups by category (localized [I/II] and advanced [III/IV]), without metastasis (M0), with metastasis (M1), and anatomic subsites (nonoropharyngeal HNSCC and oropharyngeal HNSCC with known human papillomavirus status).

RESULTS

Of 250 229 eligible patients treated at 1229 facilities in the United States, there were 185 316 (74.1%) men and 64 913 (25.9%) women and the mean (SD) age was 62.8 (12.1) years. Three case volume thresholds were identified (low: ≤54 cases per year; moderate: >54 to ≤165 cases per year; and high: >165 cases per year). Compared with the moderate-volume group, multivariate analysis found that treatment at low-volume facilities (LVFs) was associated with a higher risk of mortality (hazard ratio [HR], 1.09; 99% CI, 1.07-1.11), whereas treatment at high-volume facilities (HVFs) was associated with a lower risk of mortality (HR, 0.92; 99% CI, 0.89-0.94). Subgroup analysis with Bonferroni correction revealed that only the moderate- vs low- threshold had meaningful differences in outcomes in localized stage (I/II) cancers, (LVFs vs moderate-volume facilities [MVFs]: HR, 1.09 [99% CI, 1.05-1.13]; HVF vs MVF: HR, 0.95 [99% CI, 0.90-1.00]), whereas both thresholds were meaningful in advanced stage (III/IV) cancers (LVF vs MVF: HR, 1.09 [99% CI, 1.06-1.12]; HVF vs MVF: HR, 0.91 [99% CI, 0.88-0.94]). Survival differed by prognostic thresholds for both M0 (LVF vs MVF: HR, 1.09 [99% CI, 1.07-1.12]; HVF vs MVF: HR, 0.91 [99% CI, 0.89-0.94]) and nonoropharyngeal HNSCC (LVF vs MVF: HR, 1.10 [99% CI, 1.07-1.13]; HVF vs MVF: HR, 0.93 [99% CI, 0.90-0.97]) site cases, but not for M1 (LVF vs MVF: HR, 1.00 [99% CI, 0.92-1.09]; HVF vs MVF: HR, 0.94 [99% CI, 0.83-1.07]) or oropharyngeal HNSCC cases (when controlling for human papillomavirus status) (LVF vs MVF: HR, 1.10 [99% CI, 0.99-1.23]; HVF vs MVF: HR, 1.07 [99% CI, 0.94-1.22]).

CONCLUSIONS AND RELEVANCE

Higher volume facility threshold results appear to be associated with increases in survival rates for patients treated for HNSCC at MVFs or HVFs compared with LVFs, which suggests that these thresholds may be used as quality markers.

Digital Breast Tomosynthesis: Radiologist Learning Curve

Background There is growing evidence that digital breast tomosynthesis (DBT) results in lower recall rates and higher cancer detection rates when compared with digital mammography. However, whether DBT interpretative performance changes with experience (learning curve effect) is unknown. Purpose To evaluate screening DBT performance by cumulative DBT volume within 2 years after adoption relative to digital mammography (DM) performance 1 year before DBT adoption. Materials and Methods This prospective study included 106 126 DBT and 221 248 DM examinations in 271 362 women (mean age, 57.5 years) from 2010 to 2017 that were interpreted by 104 radiologists from 53 facilities in the Breast Cancer Surveillance Consortium. Conditional logistic regression was used to estimate within-radiologist effects of increasing cumulative DBT volume on recall and cancer detection rates relative to DM and was adjusted for examination-level characteristics. Changes were also evaluated by subspecialty and breast density. Results Before DBT adoption, DM recall rate was 10.4% (95% confidence interval [CI]: 9.5%, 11.4%) and cancer detection rate was 4.0 per 1000 screenings (95% CI: 3.6 per 1000 screenings, 4.5 per 1000 screenings); after DBT adoption, DBT recall rate was lower (9.4%; 95% CI: 8.2%, 10.6%; P = .02) and cancer detection rate was similar (4.6 per 1000 screenings; 95% CI: 4.0 per 1000 screenings, 5.2 per 1000 screenings; P = .12). Relative to DM, DBT recall rate decreased for a cumulative DBT volume of fewer than 400 studies (odds ratio [OR] = 0.83; 95% CI: 0.78, 0.89) and remained lower as volume increased (400-799 studies, OR = 0.8 [95% CI: 0.75, 0.85]; 800-1199 studies, OR = 0.81 [95% CI: 0.76, 0.87]; 1200-1599 studies, OR = 0.78 [95% CI: 0.73, 0.84]; 1600-2000 studies, OR = 0.81 [95% CI: 0.75, 0.88]; P < .001). Improvements were sustained for breast imaging subspecialists (OR range, 0.67-0.85; P < .02) and readers who were not breast imaging specialists (OR range, 0.80-0.85; P < .001). Recall rates decreased more in women with nondense breasts (OR range, 0.68-0.76; P < .001) than in those with dense breasts (OR range, 0.86-0.90; P ≤ .05; P interaction < .001). Cancer detection rates for DM and DBT were similar, regardless of DBT volume (P ≥ .10). Conclusion Early performance improvements after digital breast tomosynthesis (DBT) adoption were sustained regardless of DBT volume, radiologist subspecialty, or breast density. © RSNA, 2019 See also the editorial by Hooley in this issue.

Patient, Radiologist, and Examination Characteristics Affecting Screening Mammography Recall Rates in a Large Academic Practice

OBJECTIVE

The aims of this study were to evaluate patient, radiologist, and examination characteristics affecting screening mammography recall rates in an academic breast imaging practice and to identify modifiable factors that could reduce recall variation.

METHODS

This institutional review board-approved retrospective study included screening mammographic examinations in female patients interpreted by 13 breast imaging specialists at an academic center and two outpatient centers from October 1, 2012, to May 31, 2015. Patient demographics were extracted via electronic medical record. Natural language processing captured breast density, BI-RADS assignment, and current and prior screening examination findings. Radiologists' annual screening volumes, clinical experience, and concentration in breast imaging were calculated. Risk aversion, stress from uncertainty, and malpractice concerns were derived via survey. Univariate and multivariate analyses assessed patient, radiologist, and examination characteristics associated with likelihood of mammography recall. The Pearson product-moment correlation coefficient was used to assess the relationship between cancer detection rate and recall rate.

RESULTS

Overall, 5,678 of 61,198 screening examinations (9.3%) were recalled. In multivariate analysis, patient and radiologist characteristics associated with higher odds of recall included patient's age < 50 years (P < .0001), prior mammographic findings (calcification [P < .0001], mass [P < .0001], higher density category [P < .0001]), baseline examination (P < .0001), annual reading volume < 1,250 examinations (P = .0282), and <10 years of experience (P = .0036). Radiologist's risk aversion, stress from uncertainty, malpractice concerns, and cancer detection rates were not associated with higher recall rates (r = -0.36, P = .23).

CONCLUSIONS

In addition to patient and examination factors, screening recall variations were associated with radiologists' annual reading volume and experience. Interventions targeting radiologist factors (screening volumes, second review of potential recalls) may help reduce unwarranted variation in screening recall.

Errors in Mammography Cannot be Solved Through Technology Alone

Mammography has been the frontline screening tool for breast cancer for decades. However, high error rates in the form of false negatives (FNs) and false positives (FPs) have persisted despite technological improvements. Radiologists still miss between 10% and 30% of cancers while 80% of woman recalled for additional views have normal outcomes, with 40% of biopsied lesions being benign. Research show that the majority of cancers missed is actually visible and looked at, but either go unnoticed or are deemed to be benign. Causal agents for these errors include human related characteristics resulting in contributory search, perception and decision-making behaviours. Technical, patient and lesion factors are also important relating to positioning, compression, patient size, breast density and presence of breast implants as well as the nature and subtype of the cancer itself, where features such as architectural distortion and triple-negative cancers remain challenging to detect on screening. A better understanding of these causal agents as well as the adoption of technological and educational interventions, which audits reader performance and provide immediate perceptual feedback, should help. This paper reviews the current status of our knowledge around error rates in mammography and explores the factors impacting it. It also presents potential solutions for maximizing diagnostic efficacy thus benefiting the millions of women who undergo this procedure each year.

Assessment of global and local region-based bilateral mammographic feature asymmetry to predict short-term breast cancer risk

This study aims to develop and test a new imaging marker-based short-term breast cancer risk prediction model. An age-matched dataset of 566 screening mammography cases was used. All 'prior' images acquired in the two screening series were negative, while in the 'current' screening images, 283 cases were positive for cancer and 283 cases remained negative. For each case, two bilateral cranio-caudal view mammograms acquired from the 'prior' negative screenings were selected and processed by a computer-aided image processing scheme, which segmented the entire breast area into nine strip-based local regions, extracted the element regions using difference of Gaussian filters, and computed both global- and local-based bilateral asymmetrical image features. An initial feature pool included 190 features related to the spatial distribution and structural similarity of grayscale values, as well as of the magnitude and phase responses of multidirectional Gabor filters. Next, a short-term breast cancer risk prediction model based on a generalized linear model was built using an embedded stepwise regression analysis method to select features and a leave-one-case-out cross-validation method to predict the likelihood of each woman having image-detectable cancer in the next sequential mammography screening. The area under the receiver operating characteristic curve (AUC) values significantly increased from 0.5863  ±  0.0237 to 0.6870  ±  0.0220 when the model trained by the image features extracted from the global regions and by the features extracted from both the global and the matched local regions (p  =  0.0001). The odds ratio values monotonically increased from 1.00-8.11 with a significantly increasing trend in slope (p  =  0.0028) as the model-generated risk score increased. In addition, the AUC values were 0.6555  ±  0.0437, 0.6958  ±  0.0290, and 0.7054  ±  0.0529 for the three age groups of 37-49, 50-65, and 66-87 years old, respectively. AUC values of 0.6529  ±  0.1100, 0.6820  ±  0.0353, 0.6836  ±  0.0302 and 0.8043  ±  0.1067 were yielded for the four mammography density sub-groups (BIRADS from 1-4), respectively. This study demonstrated that bilateral asymmetry features extracted from local regions combined with the global region in bilateral negative mammograms could be used as a new imaging marker to assist in the prediction of short-term breast cancer risk.

Performance of 4 years of population-based mammography screening for breast cancer combined with ultrasound in Tyrol / Austria

BACKGROUND

Systems for the delivery of screening mammography vary among countries and these differences can influence screening effectiveness. We evaluated the performance of organized mammography screening for breast cancer combined with ultrasound in Tyrol / Austria, an approach that differs from many other population-based screening programs.

METHODS

Data on women aged 40-69 years screened in the period from June 2008 to May 2012 were collected within the framework of an organized screening program. A total of 272,555 invitations were sent to the target population living in Tyrol and 176,957 screening examinations were performed. We analyzed the main performance indicators as defined by European Union (EU) guidelines and some important estimates of harms.

RESULTS

The estimated 2‑year participation rate was 56.9%. As ultrasound is implemented as second-line screening procedure, 76.2% of all women screened underwent supplementary ultrasound. In total 2322 women were recalled for further assessment (13.1 per 1000 screens) and 1351 biopsies were performed (7.6 per 1000 screens). The positive predictive value was 28.2% for assessment and 48.5% for biopsies. The cancer detection rate was 3.7 per 1000 screens and the proportion of all stage II+ screen-detected cancers was 35.5%. The interval cancer rate was 0.33 and 0.47 per 1000 screens in the first and second years, respectively. The estimated cumulative risk for a false positive screening result and an unnecessary biopsy for women following the invitation approach was 21.1% and 9.4%, respectively.

CONCLUSION

The performance of our population-based screening approach combining mammography and ultrasound is very favorable and potential harm is kept very low compared to other European mammography screening programs for breast cancer.

Effectiveness and cost-effectiveness of double reading in digital mammography screening: A systematic review and meta-analysis

PURPOSE

Double reading is the strategy of choice for mammogram interpretation in screening programmes. It remains, however, unknown whether double reading is still the strategy of choice in the context of digital mammography. Our aim was to determine the effectiveness and cost-effectiveness of double reading versus single reading of digital mammograms in screening programmes.

METHODS

We performed a systematic review by searching the PubMed, Embase, and Cochrane Library databases up to April 2017. We used the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies) tool and CHEERS (Consolidated Health Economic Evaluation Reporting Standards) checklist to assess the methodological quality of the diagnostic studies and economic evaluations, respectively. A proportion's meta-analysis approach, 95% Confidence Intervals (95% CI) and test of heterogeneity (P values) were used for pooled results. Costs are expressed US$ PPP (United States Dollar purchasing power parities). The PROSPERO ID of this Systematic Review's protocol is CRD42014013804.

RESULTS

Of 1473 potentially relevant hits, four high-quality studies were included. The pooled cancer detection rate of double reading was 6.01 per 1000 screens (CI: 4.47‰-7.77‰), and it was 5.65 per 1000 screens (CI: 3.95‰-7.65‰) for single reading (P=0.76). The pooled proportion of false-positives of double reading was 47.03 per 1000 screens (CI: 39.13‰-55.62‰) and it was 40.60 per 1000 screens (CI: 38.58‰-42.67‰) for single reading (P=0.12). One study reported, for double reading, an ICER (Incremental Cost-Effectiveness Ratio) of 16,684 Euros (24,717 US$ PPP; 2015 value) per detected cancer. Single reading+CAD (computer-aided-detection) was cost-effective in Japan.

CONCLUSION

The evidence of benefit for double reading compared to single reading for digital mammography interpretation is scarce. Double reading seems to increase operational costs, have a not significantly higher false-positive rate, and a similar cancer detection rate.

Applying a new bilateral mammographic density segmentation method to improve accuracy of breast cancer risk prediction

PURPOSE

How to optimally detect bilateral mammographic asymmetry and improve risk prediction accuracy remains a difficult and unsolved issue. Our aim was to find an effective mammographic density segmentation method to improve accuracy of breast cancer risk prediction.

METHODS

A dataset including 168 negative mammography screening cases was used. We applied a mutual threshold to bilateral mammograms of left and right breasts to segment the dense breast regions. The mutual threshold was determined by the median grayscale value of all pixels in both left and right breast regions. For each case, we then computed three types of image features representing asymmetry, mean and the maximum of the image features, respectively. A two-stage classification scheme was developed to fuse the three types of features. The risk prediction performance was tested using a leave-one-case-out cross-validation method.

RESULTS

By using the new density segmentation method, the computed area under the receiver operating characteristic curve was 0.830 ± 0.033 and overall prediction accuracy was 81.0%, significantly higher than those of 0.633 ± 0.043 and 57.1% achieved by using the previous density segmentation method ([Formula: see text], t-test).

CONCLUSIONS

A new mammographic density segmentation method based on a bilateral mutual threshold can be used to more effectively detect bilateral mammographic density asymmetry and help significantly improve accuracy of near-term breast cancer risk prediction.

Early Detection and Screening for Breast Cancer

OBJECTIVE

To review the history, current status, and future trends related to breast cancer screening.

DATA SOURCES

Peer-reviewed articles, web sites, and textbooks.

CONCLUSION

Breast cancer remains a complex, heterogeneous disease. Serial screening with mammography is the most effective method to detect early stage disease and decrease mortality. Although politics and economics may inhibit organized mammography screening programs in many countries, the judicious use of proficient clinical and self-breast examination can also identify small tumors leading to reduced morbidity.

IMPLICATIONS FOR NURSING PRACTICE

Oncology nurses have exciting opportunities to lead, facilitate, and advocate for delivery of high-quality screening services targeting individuals and communities. A practical approach is needed to translate the complexities and controversies surrounding breast cancer screening into improved care outcomes.

Classification of normal screening mammograms is strongly influenced by perceived mammographic breast density

INTRODUCTION

To investigate how breast screen readers classify normal screening cases using descriptors of normal mammographic features and to assess test cases for suitability for a single reading strategy.

METHODS

Fifteen breast screen readers interpreted a test set of 29 normal screening cases and classified them by firstly rating their perceived difficulty to reach a 'normal' decision, secondly identifying the cases' salient normal mammographic features and thirdly assessing the cases' suitability for a single reading strategy.

RESULTS

The relationship between the perceived difficulty in making 'normal' decisions and the normal mammographic features was investigated. Regular ductal pattern (T_b  = -0.439, P = 0.001), uniform density (T_b  = -0.527, P < 0.001), non-dense breasts (T_b  = -0.736, P < 0.001), symmetrical mammographic features (T_b  = -0.474, P = 0.001) and overlapped density (T_b  = 0.630, P < 0.001) had a moderate to strong correlation with the difficulty to make 'normal' decisions. Cases with regular ductal pattern (T_b  = 0.447, P = 0.002), uniform density (T_b  = 0.550, P < 0.001), non-dense breasts (T_b  = 0.748, P < 0.001) and symmetrical mammographic features (T_b  = 0.460, P = 0.001) were considered to be more suitable for single reading, whereas cases with overlapped density were not (T_b  = -0.679, P < 0.001).

CONCLUSION

The findings suggest that perceived mammographic breast density has a major influence on the difficulty for readers to classify cases as normal and hence their suitability for single reading.

Shuffling your way out of change blindness

Change detection is typically discussed in the literature as a 2-state phenomenon. Small differences between otherwise identical images are easy to detect when the images are superimposed in space and alternated in time ("shuffled"). However, change blindness results from any disruption that prevents the critical change from generating the sole salient transient. Here we show that different presentation strategies produce different degrees of change blindness for the same change. Specifically, shuffling the images supports faster change detection than viewing the same 2 images side by side, even when the images contain a number of distracting differences. In Experiment 1, pairs of photographs having a 50 % chance of containing a difference were viewed either in alternation, in a "Shuffle" condition, or simultaneously, in a "Side-by-Side" condition. Change detection was about 6 seconds faster when the images were viewed in the "Shuffle" mode. In Experiment 2, we presented two images that were slightly laterally shifted relative to each other (0-48 pixels). The RT benefit for the Shuffle viewing mode was very strong when the relative shift was small, to insignificant when there was a large difference between the two images. However, at large shifts, Shuffle maintained an accuracy advantage. It seems that changes are easier to detect when comparing images in a Shuffle condition rather than Side-by-Side. This has important implications for real world tasks like radiology where detection of change is critical.

Radiologists' interpretive skills in screening vs. diagnostic mammography: are they related?

PURPOSE

This study aims to determine whether radiologists who perform well in screening also perform well in interpreting diagnostic mammography.

MATERIALS AND METHODS

We evaluated the accuracy of 468 radiologists interpreting 2,234,947 screening and 196,164 diagnostic mammograms. Adjusting for site, radiologist, and patient characteristics, we identified radiologists with performance in the highest tertile and compared to those with lower performance.

RESULTS

A moderate correlation was noted for radiologists' accuracy when interpreting screening versus their accuracy on diagnostic examinations: sensitivity (r_spearman=0.51, 95% CI: 0.22, 0.80; P=.0006) and specificity (r_spearman=0.40, 95% CI: 0.30, 0.49; P<.0001).

CONCLUSION

Different educational approaches to screening and diagnostic imaging should be considered.

Validation of a Medicare Claims-based Algorithm for Identifying Breast Cancers Detected at Screening Mammography

BACKGROUND

The breast cancer detection rate is a benchmark measure of screening mammography quality, but its computation requires linkage of mammography interpretive performance information with cancer incidence data. A Medicare claims-based measure of detected breast cancers could simplify measurement of this benchmark and facilitate mammography quality assessment and research.

OBJECTIVES

To validate a claims-based algorithm that can identify with high positive predictive value (PPV) incident breast cancers that were detected at screening mammography.

RESEARCH DESIGN

Development of a claims-derived algorithm using classification and regression tree analyses within a random half-sample of Medicare screening mammography claims followed by validation of the algorithm in the remaining half-sample using clinical data on mammography results and cancer incidence from the Breast Cancer Surveillance Consortium (BCSC).

SUBJECTS

Female fee-for-service Medicare enrollees aged 68 years and older who underwent screening mammography from 2001 to 2005 within BCSC registries in 4 states (CA, NC, NH, and VT), enabling linkage of claims and BCSC mammography data (N=233,044 mammograms obtained by 104,997 women).

MEASURES

Sensitivity, specificity, and PPV of algorithmic identification of incident breast cancers that were detected by radiologists relative to a reference standard based on BCSC mammography and cancer incidence data.

RESULTS

An algorithm based on subsequent codes for breast cancer diagnoses and treatments and follow-up mammography identified incident screen-detected breast cancers with 92.9% sensitivity [95% confidence interval (CI), 91.0%-94.8%], 99.9% specificity (95% CI, 99.9%-99.9%), and a PPV of 88.0% (95% CI, 85.7%-90.4%).

CONCLUSIONS

A simple claims-based algorithm can accurately identify incident breast cancers detected at screening mammography among Medicare enrollees. The algorithm may enable mammography quality assessment using Medicare claims alone.

Using multiscale texture and density features for near-term breast cancer risk analysis

PURPOSE

To help improve efficacy of screening mammography by eventually establishing a new optimal personalized screening paradigm, the authors investigated the potential of using the quantitative multiscale texture and density feature analysis of digital mammograms to predict near-term breast cancer risk.

METHODS

The authors' dataset includes digital mammograms acquired from 340 women. Among them, 141 were positive and 199 were negative/benign cases. The negative digital mammograms acquired from the "prior" screening examinations were used in the study. Based on the intensity value distributions, five subregions at different scales were extracted from each mammogram. Five groups of features, including density and texture features, were developed and calculated on every one of the subregions. Sequential forward floating selection was used to search for the effective combinations. Using the selected features, a support vector machine (SVM) was optimized using a tenfold validation method to predict the risk of each woman having image-detectable cancer in the next sequential mammography screening. The area under the receiver operating characteristic curve (AUC) was used as the performance assessment index.

RESULTS

From a total number of 765 features computed from multiscale subregions, an optimal feature set of 12 features was selected. Applying this feature set, a SVM classifier yielded performance of AUC = 0.729 ± 0.021. The positive predictive value was 0.657 (92 of 140) and the negative predictive value was 0.755 (151 of 200).

CONCLUSIONS

The study results demonstrated a moderately high positive association between risk prediction scores generated by the quantitative multiscale mammographic image feature analysis and the actual risk of a woman having an image-detectable breast cancer in the next subsequent examinations.

Factors associated with breast screening radiologists' annual mammogram reading volume in Italy

PURPOSE

Screening mammogram reading volume (SMRV) and total (screening and clinical) mammogram reading volume (TMRV) per year are strongly associated with the radiologist's diagnostic performance in breast cancer screening. The current article reports the prevalence and correlates of a SMRV and a TMRV ≥5000 among Italian breast screening radiologists.

MATERIALS AND METHODS

A questionnaire survey was carried out in 2013-2014 by the Italian Group for Mammography Screening (GISMa). The questionnaire included items of information for radiologist's experience-related characteristics and for facility-level factors supposedly associated with SMRV and TMRV. Multivariate analysis was performed using backward stepwise multiple logistic regression models.

RESULTS

Data for 235 radiologists from 51 local screening programmes were received. Of the 222 radiologists who were eligible, 133 (59.9 %) reported a SMRV ≥5000 and 163 (73.4 %) a TMRV ≥5000. Multivariate factors positively associated with both characteristics included: the number of years of experience reading mammograms; the percentage of total working time dedicated to breast imaging and breast care; the participation in diagnostic assessment; and the availability of digital tomosynthesis at facility. Full-time dedication to breast imaging and breast care was associated with the highest odds ratio for a SMRV and a TMRV ≥5000, i.e. 11.80 and 46.74, respectively, versus a percentage of time ≤50 %. An early (<2000) year of implementation of the screening programme and the availability of vacuum-assisted biopsy at facility were associated with a SMRV and, respectively, a TMRV ≥5000.

CONCLUSIONS

Increasing the proportion of radiologists with full-time dedication to breast imaging and breast care qualified as the most effective approach to improve SMRV and TMRV.

Imaging-based screening: maximizing benefits and minimizing harms

Advanced imaging technologies play a central role in screening asymptomatic patients. However, the balance between imaging-based screening's potential benefits versus risks is sometimes unclear. Radiologists will have to address ongoing concerns, including high false-positive rates, incidental findings outside the organ of interest, overdiagnosis, and potential risks from radiation exposure. In this article, we provide a brief overview of these recurring controversies and suggest the following as areas that radiologists should focus on in order to tip the balance toward more benefits and less harms for patients undergoing imaging-based screening: interpretive variability, abnormal finding thresholds, and personalized, risk-based screening.

Influences of Radiology Trainees on Screening Mammography Interpretation

PURPOSE

Participation of radiology trainees in screening mammographic interpretation is a critical component of radiology residency and fellowship training. The aim of this study was to investigate and quantify the effects of trainee involvement on screening mammographic interpretation and diagnostic outcomes.

METHODS

Screening mammograms interpreted at an academic medical center by six dedicated breast imagers over a three-year period were identified, with cases interpreted by an attending radiologist alone or in conjunction with a trainee. Trainees included radiology residents, breast imaging fellows, and fellows from other radiology subspecialties during breast imaging rotations. Trainee participation, patient variables, results of diagnostic evaluations, and pathology were recorded.

RESULTS

A total of 47,914 mammograms from 34,867 patients were included, with an overall recall rate for attending radiologists reading alone of 14.7% compared with 18.0% when involving a trainee (P < .0001). Overall cancer detection rate for attending radiologists reading alone was 5.7 per 1,000 compared with 5.2 per 1,000 when reading with a trainee (P = .517). When reading with a trainee, dense breasts represented a greater portion of recalls (P = .0001), and more frequently, greater than one abnormality was described in the breast (P = .013). Detection of ductal carcinoma in situ versus invasive carcinoma or invasive cancer type was not significantly different. The mean size of cancers in patients recalled by attending radiologists alone was smaller, and nodal involvement was less frequent, though not statistically significantly.

CONCLUSIONS

These results demonstrate a significant overall increase in recall rate when interpreting screening mammograms with radiology trainees, with no change in cancer detection rate. Radiology faculty members should be aware of this potentiality and mitigate tendencies toward greater false positives.

The National Mammography Database: Preliminary Data

OBJECTIVE

The purposes of our study were to analyze screening mammography data submitted to the National Mammography Database (NMD) since its inception to confirm data collection feasibility, to draw parallels to data from the Breast Cancer Surveillance Consortium (BCSC), and to examine trends over time. We also retrospectively evaluated practice-level variation in terms of practice type, practice setting, census region, and annual volume.

MATERIALS AND METHODS

Data from 90 mammography facilities in the NMD registry were analyzed. The registry receives mammography data collected as part of standard clinical practice, including self-reported demographic information, clinical findings, screening mammography interpretation, and biopsy results. Outcome metrics calculated were cancer detection rate, recall rate, and positive predictive values for biopsy recommended (PPV2) and biopsy performed (PPV3).

RESULTS

The NMD successfully collected and analyzed data for 3,181,437 screening mammograms performed between January 2008 and December 2012. Mean values for outcomes were cancer detection rate of 3.43 per 1000 (95% CI, 3.2-3.7), recall rate of 10% (95% CI, 9.3-10.7%), PPV2 of 18.5% (95% CI, 16.7-20.2%), and PPV3 of 29.2% (95% CI, 26.2-32.3%). No statistically significant difference was seen in performance measurements on the basis of practice type, practice setting, census region, or annual volume. NMD performance measurements parallel those reported by the BCSC.

CONCLUSION

The NMD has become the fastest growing mammography registry in the United States, providing nationwide performance metrics and permitting comparison with published benchmarks. Our study shows the feasibility of using the NMD to audit mammography facilities and to provide current, ongoing benchmark data.

Breast cancer screening in England and the United States: a comparison of provision and utilisation

OBJECTIVES

Comparing breast cancer screening across countries within the context of some of the benefits and harms offers the opportunity to improve effectiveness through mutual learning.

METHODS

This paper describes the provision of breast cancer screening in England and the United States. The various recommendations for accessing breast cancer screening in the two countries are set out and the organisation of services including quality assurance, incentives and performance mechanisms considered.

RESULTS

In the United States, younger women are routinely screened; they are less likely to benefit and more likely to be harmed. The utilisation of breast cancer screening amongst eligible women is broadly comparable in the two countries. However, there are differences in technical performance; the reasons for these including radiological reading procedures and cultural factors are explored.

CONCLUSIONS

Despite a well-functioning screening programme, breast cancer mortality and survival in England are poor relative to other countries. Emphasis for American improvement should be on reducing false-positive recall rates, while the English NHS could supplement existing efforts to understand and improve comparatively poor survival and mortality.

Patient and Radiologist Characteristics Associated With Accuracy of Two Types of Diagnostic Mammograms

OBJECTIVE

Earlier studies of diagnostic mammography found wide unexplained variability in accuracy among radiologists. We assessed patient and radiologist characteristics associated with the interpretive performance of two types of diagnostic mammography.

MATERIALS AND METHODS

Radiologists interpreting mammograms in seven regions of the United States were invited to participate in a survey that collected information on their demographics, practice setting, breast imaging experience, and self-reported interpretive volume. Survey data from 244 radiologists were linked to data on 274,401 diagnostic mammograms performed for additional evaluation of a recent abnormal screening mammogram or to evaluate a breast problem, between 1998 and 2008. These data were also linked to patients' risk factors and follow-up data on breast cancer. We measured interpretive performance by false-positive rate, sensitivity, and AUC. Using logistic regression, we evaluated patient and radiologist characteristics associated with false-positive rate and sensitivity for each diagnostic mammogram type.

RESULTS

Mammograms performed for additional evaluation of a recent mammogram had an overall false-positive rate of 11.9%, sensitivity of 90.2%, and AUC of 0.894; examinations done to evaluate a breast problem had an overall false-positive rate of 7.6%, sensitivity of 83.9%, and AUC of 0.871. Multiple patient characteristics were associated with measures of interpretive performance, and radiologist academic affiliation was associated with higher sensitivity for both indications for diagnostic mammograms.

CONCLUSION

These results indicate the potential for improved radiologist training, using evaluation of their own performance relative to best practices, and for improved clinical outcomes with health care system changes to maximize access to diagnostic mammography interpretation in academic settings.

Facility Mammography Volume in Relation to Breast Cancer Screening Outcomes

OBJECTIVES

To clarify the relationship between facility-level mammography interpretive volume and breast cancer screening outcomes.

METHODS

We calculated annual mammography interpretive volumes from 2000-2009 for 116 facilities participating in the U.S. Breast Cancer Surveillance Consortium (BCSC). Radiology, pathology, cancer registry, and women's self-report information were used to determine the indication for each exam, cancer characteristics, and patient characteristics. We examined the effect of annual total volume and percentage of mammograms that were screening on cancer detection rates using multinomial logistic regression adjusting for age, race/ethnicity, time since last mammogram, and BCSC registries. "Good prognosis" tumours were defined as screen-detected invasive cancers that were <15 mm, early stage, and lymph node negative at diagnosis.

RESULTS

From 3,098,481 screening mammograms, 9,899 cancers were screen-detected within one year of the exam. Approximately 80% of facilities had annual total interpretive volumes of >2,000 mammograms, and 42% had >5,000. Higher total volume facilities were significantly more likely to diagnose invasive tumours with good prognoses (odds ratio [OR] 1.32; 95% confidence interval [CI] 1.10-1.60, for total volume of 5,000-10,000/year v. 1,000-2,000/year; p-for-trend <0.001). A concomitant decrease in tumours with poor prognosis was seen (OR 0.78; 95%CI 0.63-0.98 for total volume of 5,000-10,000/year v. 1,000-2,000/year).

CONCLUSIONS

Mammography facilities with higher total interpretive volumes detected more good prognosis invasive tumours and fewer poor prognosis invasive tumours, suggesting that women attending these facilities may be more likely to benefit from screening.

A new quantitative image analysis method for improving breast cancer diagnosis using DCE-MRI examinations

PURPOSE

To investigate the feasibility of applying a new quantitative image analysis method to improve breast cancer diagnosis performance using dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) by integrating background parenchymal enhancement (BPE) features into the decision making process.

METHODS

A dataset involving 115 DCE-MRI examinations was used in this study. Each examination depicts one identified suspicious breast tumor. Among them, 75 cases were verified as malignant and 40 were benign by the biopsy results. A computer-aided detection scheme was applied to segment breast regions and the suspicious tumor depicted on the sequentially scanned MR images of each case. We then computed 18 kinetic features in which 6 were computed from the segmented breast tumor and 12 were BPE features from the background parenchymal regions (excluding the tumor). Support vector machine (SVM) based statistical learning classifiers were trained and optimized using different combinations of features that were computed either from tumor only or from both tumor and BPE. Each SVM was tested using a leave-one-case-out validation method and assessed using an area under the receiver operating characteristic curve (AUC).

RESULTS

When using kinetic features computed from tumors only, the maximum AUC is 0.865 ± 0.035. After fusing with the BPE features, AUC increased to 0.919 ± 0.029. At 90% specificity, the tumor classification sensitivity increased by 13.2%.

CONCLUSIONS

The proposed quantitative BPE features provide valuable supplementary information to the kinetic features of breast tumors in DCE-MRI. Their addition to computer-aided diagnosis methodologies could improve breast cancer diagnosis based on DCE-MRI examinations.

Criteria for identifying radiologists with acceptable screening mammography interpretive performance on basis of multiple performance measures

OBJECTIVE

Using a combination of performance measures, we updated previously proposed criteria for identifying physicians whose performance interpreting screening mammography may indicate suboptimal interpretation skills.

MATERIALS AND METHODS

In this study, six expert breast imagers used a method based on the Angoff approach to update criteria for acceptable mammography performance on the basis of two sets of combined performance measures: set 1, sensitivity and specificity for facilities with complete capture of false-negative cancers; and set 2, cancer detection rate (CDR), recall rate, and positive predictive value of a recall (PPV1) for facilities that cannot capture false-negative cancers but have reliable cancer follow-up information for positive mammography results. Decisions were informed by normative data from the Breast Cancer Surveillance Consortium (BCSC).

RESULTS

Updated combined ranges for acceptable sensitivity and specificity of screening mammography are sensitivity≥80% and specificity≥85% or sensitivity 75-79% and specificity 88-97%. Updated ranges for CDR, recall rate, and PPV1 are: CDR≥6 per 1000, recall rate 3-20%, and any PPV1; CDR 4-6 per 1000, recall rate 3-15%, and PPV1≥3%; or CDR 2.5-4.0 per 1000, recall rate 5-12%, and PPV1 3-8%. Using the original criteria, 51% of BCSC radiologists had acceptable sensitivity and specificity; 40% had acceptable CDR, recall rate, and PPV1. Using the combined criteria, 69% had acceptable sensitivity and specificity and 62% had acceptable CDR, recall rate, and PPV1.

CONCLUSION

The combined criteria improve previous criteria by considering the interrelationships of multiple performance measures and broaden the acceptable performance ranges compared with previous criteria based on individual measures.

Predictors of preoperative MRI for breast cancer: differences by data source

AIM

Investigate how the results of predictive models of preoperative MRI for breast cancer change based on available data.

MATERIALS & METHODS

A total of 1919 insured women aged ≥18 with stage 0-III breast cancer diagnosed 2002-2009. Four models were compared using nested multivariable logistic, backwards stepwise regression; model fit was assessed via area under the curve (AUC), R2.

RESULTS

MRI recipients (n = 245) were more recently diagnosed, younger, less comorbid, with higher stage disease. Significant variables included: Model 1/Claims (AUC = 0.76, R2 = 0.10): year, age, location, income; Model 2/Cancer Registry (AUC = 0.78, R2 = 0.12): stage, breast density, imaging indication; Model 3/Medical Record (AUC = 0.80, R2 = 0.13): radiologic recommendations; Model 4/Risk Factor Survey (AUC = 0.81, R2 = 0.14): procedure count.

CONCLUSION

Clinical variables accounted for little of the observed variability compared with claims data.

The influence of mammographic technologists on radiologists' ability to interpret screening mammograms in community practice

RATIONALE AND OBJECTIVES

To determine whether the mammographic technologist has an effect on the radiologists' interpretative performance of screening mammography in community practice.

MATERIALS AND METHODS

In this institutional review board-approved retrospective cohort study, we included Carolina Mammography Registry data from 372 radiologists and 356 mammographic technologists from 1994 to 2009 who performed 1,003,276 screening mammograms. Measures of interpretative performance (recall rate, sensitivity, specificity, positive predictive value [PPV1], and cancer detection rate [CDR]) were ascertained prospectively with cancer outcomes collected from the state cancer registry and pathology reports. To determine if the mammographic technologist influenced the radiologists' performance, we used mixed effects logistic regression models, including a radiologist-specific random effect and taking into account the clustering of examinations across women, separately for screen-film mammography (SFM) and full-field digital mammography (FFDM).

RESULTS

Of the 356 mammographic technologists included, 343 performed 889,347 SFM examinations, 51 performed 113,929 FFDM examinations, and 38 performed both SFM and FFDM examinations. A total of 4328 cancers were reported for SFM and 564 cancers for FFDM. The technologists had a statistically significant effect on the radiologists' recall rate, sensitivity, specificity, and CDR for both SFM and FFDM (P values <.01). For PPV1, variability by technologist was observed for SFM (P value <.0001) but not for FFDM (P value = .088).

CONCLUSIONS

The interpretative performance of radiologists in screening mammography varies substantially by the technologist performing the examination. Additional studies should aim to identify technologist characteristics that may explain this variation.

Reduction of false-positive recalls using a computerized mammographic image feature analysis scheme

The high false-positive recall rate is one of the major dilemmas that significantly reduce the efficacy of screening mammography, which harms a large fraction of women and increases healthcare cost. This study aims to investigate the feasibility of helping reduce false-positive recalls by developing a new computer-aided diagnosis (CAD) scheme based on the analysis of global mammographic texture and density features computed from four-view images. Our database includes full-field digital mammography (FFDM) images acquired from 1052 recalled women (669 positive for cancer and 383 benign). Each case has four images: two craniocaudal (CC) and two mediolateral oblique (MLO) views. Our CAD scheme first computed global texture features related to the mammographic density distribution on the segmented breast regions of four images. Second, the computed features were given to two artificial neural network (ANN) classifiers that were separately trained and tested in a ten-fold cross-validation scheme on CC and MLO view images, respectively. Finally, two ANN classification scores were combined using a new adaptive scoring fusion method that automatically determined the optimal weights to assign to both views. CAD performance was tested using the area under a receiver operating characteristic curve (AUC). The AUC = 0.793  ±  0.026 was obtained for this four-view CAD scheme, which was significantly higher at the 5% significance level than the AUCs achieved when using only CC (p = 0.025) or MLO (p = 0.0004) view images, respectively. This study demonstrates that a quantitative assessment of global mammographic image texture and density features could provide useful and/or supplementary information to classify between malignant and benign cases among the recalled cases, which may eventually help reduce the false-positive recall rate in screening mammography.

Educational interventions to improve screening mammography interpretation: a randomized controlled trial

OBJECTIVE

The objective of our study was to conduct a randomized controlled trial of educational interventions that were created to improve performance of screening mammography interpretation.

MATERIALS AND METHODS

We randomly assigned physicians who interpret mammography to one of three groups: self-paced DVD, live expert-led educational seminar, or control. The DVD and seminar interventions used mammography cases of varying difficulty and provided associated teaching points. Interpretive performance was compared using a pretest-posttest design. Sensitivity, specificity, and positive predictive value (PPV) were calculated relative to two outcomes: cancer status and consensus of three experts about recall. The performance measures for each group were compared using logistic regression adjusting for pretest performance.

RESULTS

One hundred two radiologists completed all aspects of the trial. After adjustment for preintervention performance, the odds of improved sensitivity for correctly identifying a lesion relative to expert recall were 1.34 times higher for DVD participants than for control subjects (95% CI, 1.00-1.81; p = 0.050). The odds of an improved PPV for correctly identifying a lesion relative to both expert recall (odds ratio [OR] = 1.94; 95% CI, 1.24-3.05; p = 0.004) and cancer status (OR = 1.81; 95% CI, 1.01-3.23; p = 0.045) were significantly improved for DVD participants compared with control subjects, with no significant change in specificity. For the seminar group, specificity was significantly lower than the control group (OR relative to expert recall = 0.80; 95% CI, 0.64-1.00; p = 0.048; OR relative to cancer status = 0.79; 95% CI, 0.65-0.95; p = 0.015).

CONCLUSION

In this randomized controlled trial, the DVD educational intervention resulted in a significant improvement in screening mammography interpretive performance on a test set, which could translate into improved interpretative performance in clinical practice.