Analysis of Participant Factors That Affect the Diagnostic Performance of Screening Mammography: A Report of the Alliance for Breast Cancer Screening in Korea

Screening mammography performance according to breast density: a comparison between radiologists versus standalone intelligence detection

BACKGROUND

Artificial intelligence (AI) algorithms for the independent assessment of screening mammograms have not been well established in a large screening cohort of Asian women. We compared the performance of screening digital mammography considering breast density, between radiologists and AI standalone detection among Korean women.

METHODS

We retrospectively included 89,855 Korean women who underwent their initial screening digital mammography from 2009 to 2020. Breast cancer within 12 months of the screening mammography was the reference standard, according to the National Cancer Registry. Lunit software was used to determine the probability of malignancy scores, with a cutoff of 10% for breast cancer detection. The AI's performance was compared with that of the final Breast Imaging Reporting and Data System category, as recorded by breast radiologists. Breast density was classified into four categories (A-D) based on the radiologist and AI-based assessments. The performance metrics (cancer detection rate [CDR], sensitivity, specificity, positive predictive value [PPV], recall rate, and area under the receiver operating characteristic curve [AUC]) were compared across breast density categories.

RESULTS

Mean participant age was 43.5 ± 8.7 years; 143 breast cancer cases were identified within 12 months. The CDRs (1.1/1000 examination) and sensitivity values showed no significant differences between radiologist and AI-based results (69.9% [95% confidence interval [CI], 61.7-77.3] vs. 67.1% [95% CI, 58.8-74.8]). However, the AI algorithm showed better specificity (93.0% [95% CI, 92.9-93.2] vs. 77.6% [95% CI, 61.7-77.9]), PPV (1.5% [95% CI, 1.2-1.9] vs. 0.5% [95% CI, 0.4-0.6]), recall rate (7.1% [95% CI, 6.9-7.2] vs. 22.5% [95% CI, 22.2-22.7]), and AUC values (0.8 [95% CI, 0.76-0.84] vs. 0.74 [95% CI, 0.7-0.78]) (all P < 0.05). Radiologist and AI-based results showed the best performance in the non-dense category; the CDR and sensitivity were higher for radiologists in the heterogeneously dense category (P = 0.059). However, the specificity, PPV, and recall rate consistently favored AI-based results across all categories, including the extremely dense category.

CONCLUSIONS

AI-based software showed slightly lower sensitivity, although the difference was not statistically significant. However, it outperformed radiologists in recall rate, specificity, PPV, and AUC, with disparities most prominent in extremely dense breast tissue.

Diagnostic performance of screening mammography according to menstrual cycle among Asian women

PURPOSE

To investigate the performance metrics of screening mammography according to menstrual cycle week in premenopausal Asian women.

METHODS

This retrospective study included 69,556 premenopausal Asian women who underwent their first screening mammography between 2011 and 2019. The presence or absence of a breast cancer diagnosis within 12 months after the index screening mammography served as the reference standard, determined by linking the study data to the national cancer registry data. Menstrual cycles were calculated, and participants were assigned to groups according to weeks 1-4. The performance metrics included cancer detection rate (CDR), sensitivity, specificity, and positive predictive value (PPV), with comparisons across menstrual cycles.

RESULTS

Among menstrual cycles, the lowest CDR at 4.7 per 1000 women (95% confidence interval [CI], 3.8-5.8 per 1000 women) was observed in week 4 (all P < 0.05). The highest sensitivity of 72.7% (95% CI, 61.4-82.3) was observed in week 1, although the results failed to reach statistical significance. The highest specificity of 80.4% (95% CI, 79.5-81.3%) was observed in week 1 (P = 0.01). The lowest PPV of 2.2% (95% CI, 1.8-2.7) was observed in week 4 (all P < 0.05).

CONCLUSION

Screening mammography tended to show a higher performance during week 1 and a lower performance during week 4 of the menstrual cycle among Asian women. These results emphasize the importance of timing recommendations that consider menstrual cycles to optimize the effectiveness of screening mammography for breast cancer detection.

Effect of artificial intelligence-based computer-aided diagnosis on the screening outcomes of digital mammography: a matched cohort study

OBJECTIVE

To investigate whether artificial intelligence-based computer-aided diagnosis (AI-CAD) can improve radiologists' performance when used to support radiologists' interpretation of digital mammography (DM) in breast cancer screening.

METHODS

A retrospective database search identified 3158 asymptomatic Korean women who consecutively underwent screening DM between January and December 2019 without AI-CAD support, and screening DM between February and July 2020 with image interpretation aided by AI-CAD in a tertiary referral hospital using single reading. Propensity score matching was used to match the DM with AI-CAD group in a 1:1 ratio with the DM without AI-CAD group according to age, breast density, experience level of the interpreting radiologist, and screening round. Performance measures were compared with the McNemar test and generalized estimating equations.

RESULTS

A total of 1579 women who underwent DM with AI-CAD were matched with 1579 women who underwent DM without AI-CAD. Radiologists showed higher specificity (96% [1500 of 1563] vs 91.6% [1430 of 1561]; p < 0.001) and lower abnormal interpretation rates (AIR) (4.9% [77 of 1579] vs 9.2% [145 of 1579]; p < 0.001) with AI-CAD than without. There was no significant difference in the cancer detection rate (CDR) (AI-CAD vs no AI-CAD, 8.9 vs 8.9 per 1000 examinations; p = 0.999), sensitivity (87.5% vs 77.8%; p = 0.999), and positive predictive value for biopsy (PPV3) (35.0% vs 35.0%; p = 0.999) according to AI-CAD support.

CONCLUSIONS

AI-CAD increases the specificity for radiologists without decreasing sensitivity as a supportive tool in the single reading of DM for breast cancer screening.

CLINICAL RELEVANCE STATEMENT

This study shows that AI-CAD could improve the specificity of radiologists' DM interpretation in the single reading system without decreasing sensitivity, suggesting that it can benefit patients by reducing false positive and recall rates.

KEY POINTS

• In this retrospective-matched cohort study (DM without AI-CAD vs DM with AI-CAD), radiologists showed higher specificity and lower AIR when AI-CAD was used to support decision-making in DM screening. • CDR, sensitivity, and PPV for biopsy did not differ with and without AI-CAD support.

The usefulness of a three-protein signature blood assay (Mastocheck®) for follow-up after breast cancer surgery

PURPOSE

Mastocheck®, a proteomic-based blood assay, has been developed for early diagnosis of breast cancer. The purpose of this study is whether Mastocheck® is useful as a postoperative follow-up.

METHODS

A total of 255 patients were analyzed. The patients were classified into longitudinal monitoring and recurrence/nonrecurrence cohorts. The longitudinal monitoring cohort consisted of 111 patients. In this cohort, blood analyses were performed three times (before surgery, 8 weeks after surgery, and between 6 months and one year after surgery), and a comparative analysis of the values of Mastocheck® and individual proteins at each time point was performed. The recurrence/nonrecurrence cohort consisted of 144 patients who had been followed up for more than 1 year, and the blood marker values at the time of local recurrence were compared to those of nonrecurrence patients.

RESULTS

In the longitudinal monitoring cohort analysis, in 81 of 111 patients were diagnosed with breast cancer with Mastocheck® and the sensitivity was 73.0%. Of 111 patients in the longitudinal monitoring cohort, 108 had two blood analyses (before and 8 weeks after surgery), and three serial blood analyses were performed on 53 patients. The Mastocheck® value that were in the cancer range of 73.0% (in 81 of 111 patients) of patients before surgery, was within the normal range of 68.5% (in 74 of 108 patients) at 8 weeks after surgery and 88.7% (in 47 of 53 patients) from 6 months to 1 year after surgery. The value of Mastocheck® was significantly decreased after surgery compared to before surgery (p < 0.001). In the recurrence/nonrecurrence cohort analysis, the Mastocheck® values were in the cancer range in 38 out of 63 recurrence patients and within the normal range in 66 of 81 nonrecurrence patients (sensitivity of 60.3% and specificity of 80.2%).

CONCLUSIONS

Mastocheck® is expected to be used as a blood marker tool to aid in the early detection of recurrence during follow-up after breast cancer surgery.

Enhancement Technique Based on the Breast Density Level for Mammogram for Computer-Aided Diagnosis

Mass detection in mammograms has a limited approach to the presence of a mass in overlapping denser fibroglandular breast regions. In addition, various breast density levels could decrease the learning system's ability to extract sufficient feature descriptors and may result in lower accuracy performance. Therefore, this study is proposing a textural-based image enhancement technique named Spatial-based Breast Density Enhancement for Mass Detection (SbBDEM) to boost textural features of the overlapped mass region based on the breast density level. This approach determines the optimal exposure threshold of the images' lower contrast limit and optimizes the parameters by selecting the best intensity factor guided by the best Blind/Reference-less Image Spatial Quality Evaluator (BRISQUE) scores separately for both dense and non-dense breast classes prior to training. Meanwhile, a modified You Only Look Once v3 (YOLOv3) architecture is employed for mass detection by specifically assigning an extra number of higher-valued anchor boxes to the shallower detection head using the enhanced image. The experimental results show that the use of SbBDEM prior to training mass detection promotes superior performance with an increase in mean Average Precision (mAP) of 17.24% improvement over the non-enhanced trained image for mass detection, mass segmentation of 94.41% accuracy, and 96% accuracy for benign and malignant mass classification. Enhancing the mammogram images based on breast density is proven to increase the overall system's performance and can aid in an improved clinical diagnosis process.

Evaluation of participation and performance indicators in a breast cancer screening program in Saudi Arabia

OBJECTIVES

To evaluate early performance indicators for breast cancer screening at the King Abdulaziz University Hospital in Saudi Arabia.

METHODS

This study retrospectively evaluated data from women who underwent their first breast cancer screening program in Jeddah, Saudi Arabia between 2012 and 2019. Data on screening results were used to estimate performance indicators and generate descriptive statistics.

RESULTS

Of the 16000 women invited from 2012 to 2019, a total of 1911 (11.9%) participated. The majority of women (68.8%) were between 40 and 55 years old. Based on the screening process results, 26.6%, 40.1%, 9.7%, 1.3%, 0.7%, and 5.2% of women had BI-RADS scores of R1, R2, R3, R4, R5, and R0 respectively. The remaining 16.3% did not have mammogram records. The recall rate, or the percentage of women who underwent further evaluation, was 19.9%; 18.9% underwent a biopsy procedure. In addition, 1.6% of women had cancer screen-detected, although only 0.7% were diagnosed with breast cancer.

CONCLUSION

In light of the low participation and high recall rates, it is essential that the screening program utilizes performance indicators to optimize resource utilization and ensure the quality of the service provided. Additionally, a national framework and standardized performance indicators could mitigate this problem for other cancer screening programs.

Effectiveness of the Korean National Cancer Screening Program in reducing breast cancer mortality

High incidences of breast cancer (BC) are reported in Asian women in their forties, and it is not clear whether mammographic screening reduces mortality among them. This study evaluated the effect of BC screening on mortality in Korea. We conducted a nationwide prospective cohort study of women invited to the Korean National Cancer Screening Program (KNCSP) between 2002 and 2003 (N = 8,300,682), with data linkage to the Korea Central Cancer Registry and death certificates through 2014 and 2015, respectively. Exposure to mammographic screening was defined using a modified never/ever approach. The primary study outcome was adjusted mortality rate ratio (MRR) for BC among screened and non-screened women estimated by Poisson regression. An adjusted MRR for all cause-death other than BC was examined to account for selection bias in the cohort. BC incidence rates for screened and non-screened women were 84.41 and 82.88 per 100,000 women-years, respectively. BC mortality rates for screened and non-screened women were 5.81 and 13.43 per 100,000 women-years, respectively, with an adjusted MRR for BC of 0.43 (95% CI, 0.41-0.44). The adjusted MRR for all-cause death excluding BC was 0.52 (95% CI, 0.52-0.52). The greatest reduction in BC mortality was noted for women aged 45-54 years, and there was no observable reduction in mortality after the age of 70 years. In conclusion, the KNCSP has been effective in reducing BC mortality among Korean women aged 40-69 years.

Automated breast US as the primary screening test for breast cancer among East Asian women aged 40-49 years: a multicenter prospective study

OBJECTIVES

To prospectively evaluate the diagnostic performance of screening ABUS as the primary screening test for breast cancer among Korean women aged 40-49 years.

METHODS

This prospective, multicenter study included asymptomatic Korean women aged 40-49 years from three academic centers between February 2017 and October 2019. Each participant underwent ABUS without mammography, and the ABUS images were interpreted at each hospital with double-reading by two breast radiologists. Biopsy and at least 1 year of follow-up was considered the reference standard. Diagnostic performance of ABUS screening and subgroup analyses according to patient and tumor characteristics were evaluated.

RESULTS

Reference standard data were available for 959 women. The recall rate was 9.8% (95% confidence interval [CI]: 7.9%, 11.7%; 94 of 959 women) and the cancer detection yield was 5.2 per 1000 women (95% CI: -0.6, 11.1; 5 of 959 women). There was only one interval cancer. The sensitivity was 83.3% (95% CI: 53.5%, 100%; 5 of 6 cancers) and the specificity was 90.7% (95% CI: 88.8%, 92.5%; 864 of 95. women). The positive predictive values of biopsies performed (PPV₃) was 20.0% (95% CI: 4.3%, 35.7%; 5 of 25 women). Women with heterogeneous background echotexture had a higher recall rate (p = .009) and lower specificity (p = .036). Women with body mass index values < 25 kg/m² had a higher mean recall rate (p = .046).

CONCLUSION

In East Asia, screening automated breast US may be an alternative to screening mammography for detecting breast cancers in women aged 40-49 years.

KEY POINTS

• Automated breast US screening for breast cancer in asymptomatic women aged 40-49 is effective with 5.2 per 1000 cancer detection yield. • Women with heterogeneous background echotexture had a higher recall rate and lower specificity. • Women with body mass index < 25 kg/m² had a higher recall rate.

Automated Breast Ultrasound Screening for Dense Breasts

Mammography is the primary screening method for breast cancers. However, the sensitivity of mammographic screening is lower for dense breasts, which are an independent risk factor for breast cancers. Automated breast ultrasound (ABUS) is used as an adjunct to mammography for screening breast cancers in asymptomatic women with dense breasts. It is an effective screening modality with diagnostic accuracy comparable to that of handheld ultrasound (HHUS). Radiologists should be familiar with the unique display mode, imaging features, and artifacts in ABUS, which differ from those in HHUS. The purpose of this study was to provide a comprehensive review of the clinical significance of dense breasts and ABUS screening, describe the unique features of ABUS, and introduce the method of use and interpretation of ABUS.

Effect of Different Types of Mammography Equipment on Screening Outcomes: A Report by the Alliance for Breast Cancer Screening in Korea

Objective

To investigate the effects of different types of mammography equipment on screening outcomes by comparing the performance of film-screen mammography (FSM), computed radiography mammography (CRM), and digital mammography (DM).

Materials and Methods

We retrospectively enrolled 128756 sets of mammograms from 10 hospitals participating in the Alliance for Breast Cancer Screening in Korea between 2005 and 2010. We compared the diagnostic accuracy of the types of mammography equipment by analyzing the area under the receiver operating characteristic curve (AUC) with a 95% confidence interval (CI); performance indicators, including recall rate, cancer detection rate (CDR), positive predictive value₁ (PPV₁), sensitivity, specificity, and interval cancer rate (ICR); and the types of breast cancer pathology.

Results

The AUCs were 0.898 (95% CI, 0.878–0.919) in DM, 0.860 (0.815–0.905) in FSM, and 0.866 (0.828–0.903) in CRM (p = 0.150). DM showed better performance than FSM and CRM in terms of the recall rate (14.8 vs. 24.8 and 19.8%), CDR (3.4 vs. 2.2 and 2.1 per 1000 examinations), PPV₁ (2.3 vs. 0.9 and 1.1%), and specificity (85.5 vs. 75.3 and 80.3%) (p < 0.001) but not in terms of sensitivity (86.3 vs. 87.4 and 86.3%) and ICR (0.6 vs. 0.4 and 0.4). The proportions of carcinoma in situ (CIS) were 27.5%, 13.6%, and 11.8% for DM, CRM, and FSM, respectively (p = 0.003).

Conclusion

In comparison to FSM and CRM, DM showed better performance in terms of the recall rate, CDR, PPV₁, and specificity, although the AUCs were similar, and more CISs were detected using DM. The application of DM may help to improve the quality of mammography screenings. However, the overdiagnosis issue of CIS using DM should be evaluated.

Interpretive Performance and Inter-Observer Agreement on Digital Mammography Test Sets

Objective

To evaluate the interpretive performance and inter-observer agreement on digital mammographs among radiologists and to investigate whether radiologist characteristics affect performance and agreement.

Materials and Methods

The test sets consisted of full-field digital mammograms and contained 12 cancer cases among 1000 total cases. Twelve radiologists independently interpreted all mammograms. Performance indicators included the recall rate, cancer detection rate (CDR), positive predictive value (PPV), sensitivity, specificity, false positive rate (FPR), and area under the receiver operating characteristic curve (AUC). Inter-radiologist agreement was measured. The reporting radiologist characteristics included number of years of experience interpreting mammography, fellowship training in breast imaging, and annual volume of mammography interpretation.

Results

The mean and range of interpretive performance were as follows: recall rate, 7.5% (3.3–10.2%); CDR, 10.6 (8.0–12.0 per 1000 examinations); PPV, 15.9% (8.8–33.3%); sensitivity, 88.2% (66.7–100%); specificity, 93.5% (90.6–97.8%); FPR, 6.5% (2.2–9.4%); and AUC, 0.93 (0.82–0.99). Radiologists who annually interpreted more than 3000 screening mammograms tended to exhibit higher CDRs and sensitivities than those who interpreted fewer than 3000 mammograms (p = 0.064). The inter-radiologist agreement showed a percent agreement of 77.2–88.8% and a kappa value of 0.27–0.34. Radiologist characteristics did not affect agreement.

Conclusion

The interpretative performance of the radiologists fulfilled the mammography screening goal of the American College of Radiology, although there was inter-observer variability. Radiologists who interpreted more than 3000 screening mammograms annually tended to perform better than radiologists who did not.

Five-Year Overall Survival of Interval Breast Cancers is Better than Non- Interval Cancers from Korean Breast Cancer Registry

Objective: Interval breast cancer (IC) is a limitation of breast cancer screening. We investigated data from a large scaled breast cancer dataset of patients with breast cancer who underwent breast cancer screening in order to recapitulate the overall survival (OS) of patients with ICs compared to those with non-ICs. Methods: A total of 27,141 patients in the Korean breast cancer registry with breast cancer who had ever participated in biannual national breast cancer screening programs between 2009 and 2013 were enrolled. We compared the social, pregnancy-associated, and pathologic characteristics between the IC and non-IC groups and identified the significant prognostic factors for OS. Results: The proportion of ICs was 1.3% (370/27,141) in this study population. ICs were correlated with age 45-55 years at diagnosis, higher levels of education, early menopause (<50 years), hormone replacement therapy, specific provinces (Kangwon, Kyungnam, Jeju, and Dae-jeon), and family history of breast cancer. Low-to-intermediate nuclear grade, early stage (stage 0-I), and low Ki-67 level were also correlated with IC proportion. Non-ICs were associated with an increased risk of five-year mortality (hazard ratio [HR] 7.4; 95% confidence interval [CI]:1.85-29.66; p = 0.005) compared to ICs. Lymph node metastasis, residence (Kyung-nam province), low education status, high histologic grade, and asymptomatic cancers increased the HR of five-year OS. Conclusion: ICs occurred unequally in specific province and relatively high-educated women in Korea. They were also diagnosed with early-stage breast cancer with a favorable recurrence risk, and their outcome was better than those of patients with other breast cancers in breast cancer screening.

Prevalence of Women with Dense Breasts in Korea: Results from a Nationwide Cross-sectional Study

Purpose

Women with dense breast are known to be at high risk for breast cancer, but their prevalence and number of Korean women are unknown. The current study was to investigate the distribution of mammographic breast density by age of women undergoing screening mammography, and to estimate the prevalence of Korean women with dense breasts, quantitatively.

Materials and Methods

For obtaining a nationwide representative sample, 6,481 mammograms were collected from 86 screening units participated in the National Cancer Screening Program for breast cancer. Based on the American College of Radiology Breast Imaging Reporting and Data System classification, breast density was evaluated by six breast radiologists, qualitatively. We applied these breast density distributions to age-specific counts of the Korean women population derived to mid-year 2017 to estimate the number of Korean women with dense breasts.

Results

Overall, 54.4% (95% confidence interval [CI], 52.9% to 55.8%) of women 40 to 69 years of age had heterogeneously or extremely dense breasts, and this proportion was inversely associated with age. Based on the age distribution of Korean women, we estimated that 6,083,000 women (95% CI, 5,919,600 to 6,245,600) age 40-69 years in Korean have dense breasts. Women aged 40-49 years (n=3,450,000) accounted for 56.7% of this group.

Conclusion

More than half of Korean women aged 40 and over have dense breasts. To prevent breast cancer effectively and efficiently, it is necessary to develop a new personalized prevention strategy considering her status of breast density.

Age of Data in Contemporary Research Articles Published in Representative General Radiology Journals

Objective

To analyze and compare the age of data in contemporary research articles published in representative general radiology journals.

Materials and Methods

We searched for articles reporting original research studies analyzing patient data that were published in the print issues of the Korean Journal of Radiology (KJR), European Radiology (ER), and Radiology in 2017. Eligible articles were reviewed to extract data collection period (time from first patient recruitment to last patient follow-up) and age of data (time between data collection end and publication). The journals were compared in terms of the proportion of articles reporting the data collection period to the level of calendar month and regarding the age of data.

Results

There were 50, 492, and 254 eligible articles in KJR, ER, and Radiology, respectively. Of these, 44 (88%; 95% confidence interval [CI]: 75.8-94.8%), 359 (73%; 95% CI: 68.9-76.7%), and 211 (83.1%; 95% CI: 78-87.2%) articles, respectively, provided enough details of data collection period, revealing a significant difference between ER and Radiology (p = 0.002). The age of data was significantly greater in KJR (median age: 826 days; range: 299-2843 days) than in ER (median age: 570 days; range: 56-4742 days; p < 0.001) and Radiology (median age: 618; range: 75-4271 days; p < 0.001).

Conclusion

Korean Journal of Radiology did not fall behind ER or Radiology in reporting of data collection period, but showed a significantly greater age of data than ER and Radiology, suggesting that KJR should take measures to improve the timeliness of its data.