Performance of an Artificial Intelligence System for Breast Cancer Detection on Screening Mammograms from BreastScreen Norway

Purpose To explore the stand-alone breast cancer detection performance, at different risk score thresholds, of a commercially available artificial intelligence (AI) system. Materials and Methods This retrospective study included information from 661 695 digital mammographic examinations performed among 242 629 female individuals screened as a part of BreastScreen Norway, 2004-2018. The study sample included 3807 screen-detected cancers and 1110 interval breast cancers. A continuous examination-level risk score by the AI system was used to measure performance as the area under the receiver operating characteristic curve (AUC) with 95% CIs and cancer detection at different AI risk score thresholds. Results The AUC of the AI system was 0.93 (95% CI: 0.92, 0.93) for screen-detected cancers and interval breast cancers combined and 0.97 (95% CI: 0.97, 0.97) for screen-detected cancers. In a setting where 10% of the examinations with the highest AI risk scores were defined as positive and 90% with the lowest scores as negative, 92.0% (3502 of 3807) of the screen-detected cancers and 44.6% (495 of 1110) of the interval breast cancers were identified with AI. In this scenario, 68.5% (10 987 of 16 040) of false-positive screening results (negative recall assessment) were considered negative by AI. When 50% was used as the cutoff, 99.3% (3781 of 3807) of the screen-detected cancers and 85.2% (946 of 1110) of the interval breast cancers were identified as positive by AI, whereas 17.0% (2725 of 16 040) of the false-positive results were considered negative. Conclusion The AI system showed high performance in detecting breast cancers within 2 years of screening mammography and a potential for use to triage low-risk mammograms to reduce radiologist workload. Keywords: Mammography, Breast, Screening, Convolutional Neural Network (CNN), Deep Learning Algorithms Supplemental material is available for this article. © RSNA, 2024 See also commentary by Bahl and Do in this issue.

AI Risk Score on Screening Mammograms Preceding Breast Cancer Diagnosis

Background Few studies have evaluated the role of artificial intelligence (AI) in prior screening mammography. Purpose To examine AI risk scores assigned to screening mammography in women who were later diagnosed with breast cancer. Materials and Methods Image data and screening information of examinations performed from January 2004 to December 2019 as part of BreastScreen Norway were used in this retrospective study. Prior screening examinations from women who were later diagnosed with cancer were assigned an AI risk score by a commercially available AI system (scores of 1-7, low risk of malignancy; 8-9, intermediate risk; and 10, high risk of malignancy). Mammographic features of the cancers based on the AI score were also assessed. The association between AI score and mammographic features was tested with a bivariate test. Results A total of 2787 prior screening examinations from 1602 women (mean age, 59 years ± 5.1 [SD]) with screen-detected (n = 1016) or interval (n = 586) cancers showed an AI risk score of 10 for 389 (38.3%) and 231 (39.4%) cancers, respectively, on the mammograms in the screening round prior to diagnosis. Among the screen-detected cancers with AI scores available two screening rounds (4 years) before diagnosis, 23.0% (122 of 531) had a score of 10. Mammographic features were associated with AI score for invasive screen-detected cancers (P < .001). Density with calcifications was registered for 13.6% (43 of 317) of screen-detected cases with a score of 10 and 4.6% (15 of 322) for those with a score of 1-7. Conclusion More than one in three cases of screen-detected and interval cancers had the highest AI risk score at prior screening, suggesting that the use of AI in mammography screening may lead to earlier detection of breast cancers. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Mehta in this issue.

Possible strategies for use of artificial intelligence in screen-reading of mammograms, based on retrospective data from 122,969 screening examinations

OBJECTIVES

Artificial intelligence (AI) has shown promising results when used on retrospective data from mammographic screening. However, few studies have explored the possible consequences of different strategies for combining AI and radiologists in screen-reading.

METHODS

A total of 122,969 digital screening examinations performed between 2009 and 2018 in BreastScreen Norway were retrospectively processed by an AI system, which scored the examinations from 1 to 10; 1 indicated low suspicion of malignancy and 10 high suspicion. Results were merged with information about screening outcome and used to explore consensus, recall, and cancer detection for 11 different scenarios of combining AI and radiologists.

RESULTS

Recall was 3.2%, screen-detected cancer 0.61% and interval cancer 0.17% after independent double reading and served as reference values. In a scenario where examinations with AI scores 1-5 were considered negative and 6-10 resulted in standard independent double reading, the estimated recall was 2.6% and screen-detected cancer 0.60%. When scores 1-9 were considered negative and score 10 double read, recall was 1.2% and screen-detected cancer 0.53%. In these two scenarios, potential rates of screen-detected cancer could be up to 0.63% and 0.56%, if the interval cancers selected for consensus were detected at screening. In the former scenario, screen-reading volume would be reduced by 50%, while the latter would reduce the volume by 90%.

CONCLUSION

Several theoretical scenarios with AI and radiologists have the potential to reduce the volume in screen-reading without affecting cancer detection substantially. Possible influence on recall and interval cancers must be evaluated in prospective studies.

KEY POINTS

• Different scenarios using artificial intelligence in combination with radiologists could reduce the screen-reading volume by 50% and result in a rate of screen-detected cancer ranging from 0.59% to 0.60%, compared to 0.61% after standard independent double reading • The use of artificial intelligence in combination with radiologists has the potential to identify negative screening examinations with high precision in mammographic screening and to reduce the rate of interval cancer.

Artificial Intelligence Evaluation of 122 969 Mammography Examinations from a Population-based Screening Program

Background Artificial intelligence (AI) has shown promising results for cancer detection with mammographic screening. However, evidence related to the use of AI in real screening settings remain sparse. Purpose To compare the performance of a commercially available AI system with routine, independent double reading with consensus as performed in a population-based screening program. Furthermore, the histopathologic characteristics of tumors with different AI scores were explored. Materials and Methods In this retrospective study, 122 969 screening examinations from 47 877 women performed at four screening units in BreastScreen Norway from October 2009 to December 2018 were included. The data set included 752 screen-detected cancers (6.1 per 1000 examinations) and 205 interval cancers (1.7 per 1000 examinations). Each examination had an AI score between 1 and 10, where 1 indicated low risk of breast cancer and 10 indicated high risk. Threshold 1, threshold 2, and threshold 3 were used to assess the performance of the AI system as a binary decision tool (selected vs not selected). Threshold 1 was set at an AI score of 10, threshold 2 was set to yield a selection rate similar to the consensus rate (8.8%), and threshold 3 was set to yield a selection rate similar to an average individual radiologist (5.8%). Descriptive statistics were used to summarize screening outcomes. Results A total of 653 of 752 screen-detected cancers (86.8%) and 92 of 205 interval cancers (44.9%) were given a score of 10 by the AI system (threshold 1). Using threshold 3, 80.1% of the screen-detected cancers (602 of 752) and 30.7% of the interval cancers (63 of 205) were selected. Screen-detected cancer with AI scores not selected using the thresholds had favorable histopathologic characteristics compared to those selected; opposite results were observed for interval cancer. Conclusion The proportion of screen-detected cancers not selected by the artificial intelligence (AI) system at the three evaluated thresholds was less than 20%. The overall performance of the AI system was promising according to cancer detection. © RSNA, 2022.

Artificial Intelligence Evaluation of 122 969 Mammography Examinations from a Population-based Screening Program

Background Artificial intelligence (AI) has shown promising results for cancer detection with mammographic screening. However, evidence related to the use of AI in real screening settings remain sparse. Purpose To compare the performance of a commercially available AI system with routine, independent double reading with consensus as performed in a population-based screening program. Furthermore, the histopathologic characteristics of tumors with different AI scores were explored. Materials and Methods In this retrospective study, 122 969 screening examinations from 47 877 women performed at four screening units in BreastScreen Norway from October 2009 to December 2018 were included. The data set included 752 screen-detected cancers (6.1 per 1000 examinations) and 205 interval cancers (1.7 per 1000 examinations). Each examination had an AI score between 1 and 10, where 1 indicated low risk of breast cancer and 10 indicated high risk. Threshold 1, threshold 2, and threshold 3 were used to assess the performance of the AI system as a binary decision tool (selected vs not selected). Threshold 1 was set at an AI score of 10, threshold 2 was set to yield a selection rate similar to the consensus rate (8.8%), and threshold 3 was set to yield a selection rate similar to an average individual radiologist (5.8%). Descriptive statistics were used to summarize screening outcomes. Results A total of 653 of 752 screen-detected cancers (86.8%) and 92 of 205 interval cancers (44.9%) were given a score of 10 by the AI system (threshold 1). Using threshold 3, 80.1% of the screen-detected cancers (602 of 752) and 30.7% of the interval cancers (63 of 205) were selected. Screen-detected cancer with AI scores not selected using the thresholds had favorable histopathologic characteristics compared to those selected; opposite results were observed for interval cancer. Conclusion The proportion of screen-detected cancers not selected by the artificial intelligence (AI) system at the three evaluated thresholds was less than 20%. The overall performance of the AI system was promising according to cancer detection. © RSNA, 2022.

Mammographic features and screening outcome in a randomized controlled trial comparing digital breast tomosynthesis and digital mammography

PURPOSE

To compare the distribution of mammographic features among women recalled for further assessment after screening with digital breast tomosynthesis (DBT) versus digital mammography (DM), and to assess associations between features and final outcome of the screening, including immunohistochemical subtypes of the tumour.

METHODS

This randomized controlled trial was performed in Bergen, Norway, and included 28,749 women, of which 1015 were recalled due to mammographic findings. Mammographic features were classified according to a modified BI-RADS-scale. The distribution were compared using 95 % confidence intervals (CI).

RESULTS

Asymmetry was the most common feature of all recalls, 24.3 % (108/444) for DBT and 38.9 % (222/571) for DM. Spiculated mass was most common for breast cancer after screening with DBT (36.8 %, 35/95, 95 %CI: 27.2-47.4) while calcifications (23.0 %, 20/87, 95 %CI: 14.6-33.2) was the most frequent after DM. Among women screened with DBT, 0.13 % (95 %CI: 0.08-0.21) had benign outcome after recall due to indistinct mass while the percentage was 0.28 % (95 %CI: 0.20-0.38) for DM. The distributions were 0.70 % (95 %CI: 0.57-0.85) versus 1.46 % (95 %CI: 1.27-1.67) for asymmetry and 0.24 % (95 %CI: 0.16-0.33) versus 0.54 % (95 %CI: 0.43-0.68) for obscured mass, among women screened with DBT versus DM, respectively. Spiculated mass was the most common feature among women diagnosed with non-luminal A-like cancer after DBT and after DM.

CONCLUSIONS

Spiculated mass was the dominant feature for breast cancer among women screened with DBT while calcifications was the most frequent feature for DM. Further studies exploring the clinical relevance of mammographic features visible particularly on DBT are warranted.

Terminal digit preference: a source of measurement error in breast cancer diameter reporting

Objectives: Women diagnosed with breast cancer are offered treatment and therapy based on tumor characteristics, including tumor diameter. There is scarce knowledge whether tumor diameter is accurately reported, or whether it is unconsciously rounded to the nearest half-centimeter (terminal digit preference). This study aimed to assess the precision (number of digits) of breast cancer tumor diameters and whether they are affected by terminal digit preference. Furthermore, we aimed to assess the agreement between mammographic and histopathologic tumor diameter measurements.Material and Methods: This national registry study included reported mammographic and registered histopathologic tumor diameter information from the Cancer Registry of Norway for invasive breast cancers diagnosed during 2012-2016. Terminal digit preference was assessed using histograms. Agreement between mammographic and histopathologic measurements was assessed using the intraclass correlation coefficient (ICC) and Bland-Altman plots.Results: Mammographic, histopathologic, or both tumor measurements were available for 7792, 13,541 and 6865 cases, respectively. All mammographic and 97.2% of histopathologic tumor diameters were recorded using whole mm. Terminal digits of zero or five were observed among 38.7% and 34.8% of mammographic and histopathologic measurements, respectively. There was moderate agreement between the two measurement methods (ICC = 0.52, 95% CI: 0.50-0.53). On average, mammographic measurements were 1.26 mm larger (95% limits of agreement: -22.29-24.73) than histopathologic measurements. This difference increased with increasing tumor size.Conclusion: Terminal digit preference was evident among breast cancer tumor diameters in this nationwide study. Further studies are needed to investigate the potential extent of under-staging and under-treatment resulting from this measurement error.

Abstract 1201: Breast cancer risk factors and volumetric breast density in a national breast cancer screening program

Background: Studying associations between percent or absolute volumetric breast density (VBD) with age and menopausal status, and whether the associations are modified by demographic, lifestyle, reproductive, or hormonal exposures, can uncover underlying biological mechanisms and improve breast cancer risk prediction.

Material and methods: The cohort consisted of women (aged 49-71 years) who participated in the Norwegian Breast Cancer Screening Program (NBCPS) between 2007 and 2014, had information on VBD and completed questionnaires on standard breast cancer risk factors as part of the program (n=46 428). We estimated least squared means of percent and absolute VBD associated with age at mammography, menopausal status, age at menopause, reproductive and hormonal factors (ages at menarche and first birth, number of pregnancies lasting ≥6 months, duration of breastfeeding, oral contraceptives, and menopausal hormone therapy), self-reported height and body mass index (BMI), education, and lifestyle factors (physical activity, alcohol intake, and smoking).

Results: For a 5-year increase in age, the reduction in percent VBD was -0.18% in pre- and perimenopausal and -0.08% in postmenopausal women, and the reduction in absolute VBD was -0.11 cm³ in pre- and perimenopausal and -0.03 cm³ in postmenopausal women (p for interaction by menopausal status &lt;0.001). In multivariate analyses, the associations between demographic, lifestyle, reproductive and hormonal risk factors and percent and absolute VBD were highly significant, however the magnitude of the effects were modest (1-2%), and the range of percent VBD across levels of risk factors rather narrow. The strongest association was with BMI, which was inversely associated with percent VBD, with a threefold higher percent VBD in women with BMI&lt;20 kg/m² than in women with BMI&gt;33 kg/m² (12.9% versus 3.9%). Interestingly, BMI was positively associated with absolute VBD, with 1.5

times higher VBD in women with BMI≥33 kg/cm² (37.9 cm³ versus 58.4 cm³). Models were adjusted for BMI, education, and parity.

Conclusion: This large cohort analysis found percent and absolute VBD to decrease with increasing age both among pre/perimenopausal and postmenopausal women. The rate of decline was larger among pre/perimenopausal women. Percent and absolute VBD are associated with several established breast cancer risk factors, especially BMI, where the direction of the association differed for percent and absolute VBD.

Citation Format: Kirsti V. Hjerkind, Merete Ellingjord-Dale, Anna L. Johansson, Hildegunn S. Aase, Solveig R. Hoff, Solveig Hofvind, Siri Fagerheim, Linda Vos, Isabel dos Santos Silva, Giske Ursin. Breast cancer risk factors and volumetric breast density in a national breast cancer screening program [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2018; 2018 Apr 14-18; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2018;78(13 Suppl):Abstract nr 1201.

Asymptomatic breast cancer in non-participants of the national screening programme in Norway: a confounding factor in evaluation?

OBJECTIVE

To evaluate the extent and histopathological characteristics of asymptomatic breast cancer detected outside the Norwegian Breast Cancer Screening Program (NBCSP) in women targeted by the programme.

METHODS

Our study included 568 primary breast cancers (523 invasive and 45 ductal carcinoma in situ) diagnosed in 553 women aged 50-70, residing in Møre og Romsdal County, 2002-2008. The cancers were divided into screening-detected cancers in the NBCSP, interval cancers (ICs) and cancers detected in women not participating in the NBCSP (never participated and lapsed attendees), and further into asymptomatic and symptomatic cancers. Nottingham Prognostic Index (NPI) was used for comparisons across the groups and the distributions were compared using chi-square tests for statistical significance.

RESULTS

Twenty percent (19/97) of the ICs and 32% (69/213) of the breast cancers in non-participants were asymptomatic, with opportunistic screening as the most frequent detection method (42%, 8/19 for ICs and 54%, 37/69 for non-participants). There were no differences in distribution of NPI prognostic categories across subgroups of asymptomatic invasive cancers (screening-detected cancers in the NBCSP, asymptomatic ICs and asymptomatic cancers in non-participants) or between subgroups of symptomatic invasive cancers (symptomatic ICs and symptomatic cancers in non-participants). Asymptomatic cancers had a significantly more favourable distribution of NPI prognostic categories compared with symptomatic cancers (P < 0.001). The proportion of invasive cancers with excellent/good NPI was 53% (164/310) for all asymptomatic and 25% (52/211) for all symptomatic invasive cancers.

CONCLUSIONS

A considerable percentage of breast cancers detected outside the organized screening programme were asymptomatic, with a prognostic profile comparable with screening-detected breast cancers in the NBCSP. Individual data regarding the detection method for all breast cancers are needed for a complete evaluation of the organized screening programme in Norway.

Asymptomatic Breast Cancer in Non-Participants of the National Screening Programme in Norway: A Confounding Factor in Evaluation?

Objective

To evaluate the extent and histopathological characteristics of asymptomatic breast cancer detected outside the Norwegian Breast Cancer Screening Program (NBCSP) in women targeted by the programme.

Methods

Our study included 568 primary breast cancers (523 invasive and 45 ductal carcinoma in situ) diagnosed in 553 women aged 50–70, residing in Møre og Romsdal County, 2002–2008. The cancers were divided into screening-detected cancers in the NBCSP, interval cancers (ICs) and cancers detected in women not participating in the NBCSP (never participated and lapsed attendees), and further into asymptomatic and symptomatic cancers. Nottingham Prognostic Index (NPI) was used for comparisons across the groups and the distributions were compared using chi-square tests for statistical significance.

Results

Twenty percent (19/97) of the ICs and 32% (69/213) of the breast cancers in non-participants were asymptomatic, with opportunistic screening as the most frequent detection method (42%, 8/19 for ICs and 54%, 37/69 for non-participants). There were no differences in distribution of NPI prognostic categories across subgroups of asymptomatic invasive cancers (screening-detected cancers in the NBCSP, asymptomatic ICs and asymptomatic cancers in non-participants) or between subgroups of symptomatic invasive cancers (symptomatic ICs and symptomatic cancers in non-participants). Asymptomatic cancers had a significantly more favourable distribution of NPI prognostic categories compared with symptomatic cancers ( P < 0.001). The proportion of invasive cancers with excellent/good NPI was 53% (164/310) for all asymptomatic and 25% (52/211) for all symptomatic invasive cancers.

Conclusions

A considerable percentage of breast cancers detected outside the organized screening programme were asymptomatic, with a prognostic profile comparable with screening-detected breast cancers in the NBCSP. Individual data regarding the detection method for all breast cancers are needed for a complete evaluation of the organized screening programme in Norway.

Breast cancer: missed interval and screening-detected cancer at full-field digital mammography and screen-film mammography-- results from a retrospective review

PURPOSE

To compare the percentages and mammographic features of cancers missed at full-field digital mammography (FFDM) and screen-film mammography (SFM) in women who participated in the Norwegian Breast Cancer Screening Program in 2002-2008.

MATERIALS AND METHODS

Social Science Data Services approval was obtained; the requirement for informed consent was waived. Cases were all the interval and screening-detected cancers from 35&emsp14;127 FFDM and 52&emsp14;444 SFM examinations in two Norwegian counties. Prior and diagnostic FFDM examinations of 49 interval and 86 screening-detected breast cancers were reviewed by four breast radiologists and compared with a review of SFM examinations of 81 interval and 123 screening-detected cancers. Cancers were classified as missed or true, mammographic features were described, percentages were compared by using the χ(2) or Fisher exact test, and 95% confidence intervals (CIs) were calculated.

RESULTS

The percentages of interval and screening-detected cancers missed at FFDM and SFM did not differ significantly. (interval cancers missed: 33% [16 of 49] at FFDM vs 30% [24 of 81] at SFM [P = .868]; screening-detected cancers missed: 20% [17 of 86] at FFDM vs 21% [26 of 123] at SFM [P = .946]). Asymmetry was present in 27% (95% CI: 13.3%, 45.5%) of prior mammograms of cancers missed at FFDM and 10% (95% CI: 3.3%, 21.8%) of those missed at SFM (P = .070). Calcifications were observed in 18% (95% CI: 7.0%, 35.5%) of the cancers missed at FFDM and 34% (95% CI: 21.2%, 48.8%) of those missed at SFM (P = .185). Average mammographic tumor size of missed cancers manifesting as masses was 10.4 mm at FFDM and 13.6 mm at SFM (P = .036).

CONCLUSION

The use of FFDM has not reduced the challenge of missed cancers. Cancers missed at FFDM tend to have different mammographic features than those missed at SFM.

The occurrence of asymptomatic and symptomatic simple hepatic cysts. A prospective, hospital-based study

AIM

To examine the prevalence of asymptomatic and symptomatic liver cysts in a university hospital patient population using modern US equipment.

METHODS

Abdominal US scans of 1541 cases referred during the period 21 January to 11 November 2000 were examined for hepatic cysts.

RESULTS

Of 1541 cases, 174 (11.3%) were found to have hepatic cysts, i.e. 109 female (12.5%) and 65 (9.7%) male patients (9.7%). In 413 individuals younger than 40 years, no cysts were found. Above the age of 40 years, prevalence increased with age.

CONCLUSION

By using modern US equipment, we found a higher prevalence of hepatic cysts than that reported in previous studies. Patient selection and the prevalence of liver cysts in the population from which the patients were referred may have influenced our results.