Diagnostic Performance of AI for Cancers Registered in A Mammography Screening Program: A Retrospective Analysis

Comparative analysis of diagnostic performance in mammography: A reader study on the impact of AI assistance

PURPOSE

This study evaluates the impact of artificial intelligence (AI) assistance on the diagnostic performance of radiologists with varying levels of experience in interpreting mammograms in a Malaysian tertiary referral center, particularly in women with dense breasts.

METHODS

A retrospective study including 434 digital mammograms interpreted by two general radiologists (12 and 6 years of experience) and two trainees (2 years of experience). Diagnostic performance was assessed with and without AI assistance (Lunit INSIGHT MMG), using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC). Inter-reader agreement was measured using kappa statistics.

RESULTS

AI assistance significantly improved the diagnostic performance of all reader groups across all metrics (p < 0.05). The senior radiologist consistently achieved the highest sensitivity (86.5% without AI, 88.0% with AI) and specificity (60.5% without AI, 59.2% with AI). The junior radiologist demonstrated the highest PPV (56.9% without AI, 74.6% with AI) and NPV (90.3% without AI, 92.2% with AI). The trainees showed the lowest performance, but AI significantly enhanced their accuracy. AI assistance was particularly beneficial in interpreting mammograms of women with dense breasts.

CONCLUSION

AI assistance significantly enhances the diagnostic accuracy and consistency of radiologists in mammogram interpretation, with notable benefits for less experienced readers. These findings support the integration of AI into clinical practice, particularly in resource-limited settings where access to specialized breast radiologists is constrained.

Artificial intelligence in mammography: a systematic review of the external validation

Objective

To conduct a systematic review of external validation studies on the use of different Artificial Intelligence algorithms in breast cancer screening with mammography.

Data source

Our systematic review was conducted and reported following the PRISMA statement, using the PubMed, EMBASE, and Cochrane databases with the search terms "Artificial Intelligence," "Mammography," and their respective MeSH terms. We filtered publications from the past ten years (2014 - 2024) and in English.

Study selection

A total of 1,878 articles were found in the databases used in the research. After removing duplicates (373) and excluding those that did not address our PICO question (1,475), 30 studies were included in this work.

Data collection

The data from the studies were collected independently by five authors, and it was subsequently synthesized based on sample data, location, year, and their main results in terms of AUC, sensitivity, and specificity.

Data synthesis

It was demonstrated that the Area Under the ROC Curve (AUC) and sensitivity were similar to those of radiologists when using independent Artificial Intelligence. When used in conjunction with radiologists, statistically higher accuracy in mammogram evaluation was reported compared to the assessment by radiologists alone.

Conclusion

AI algorithms have emerged as a means to complement and enhance the performance and accuracy of radiologists. They also assist less experienced professionals in detecting possible lesions. Furthermore, this tool can be used to complement and improve the analyses conducted by medical professionals.

Diagnostic capabilities of artificial intelligence as an additional reader in a breast cancer screening program

OBJECTIVES

We aimed to evaluate the early-detection capabilities of AI in a screening program over its duration, with a specific focus on the detection of interval cancers, the early detection of cancers with the assistance of AI from prior visits, and its impact on workload for various reading scenarios.

MATERIALS AND METHODS

The study included 22,621 mammograms of 8825 women within a 10-year biennial two-reader screening program. The statistical analysis focused on 5136 mammograms from 4282 women due to data retrieval issues, among whom 105 were diagnosed with breast cancer. The AI software assigned scores from 1 to 100. Histopathology results determined the ground truth, and Youden's index was used to establish a threshold. Tumor characteristics were analyzed with ANOVA and chi-squared test, and different workflow scenarios were evaluated using bootstrapping.

RESULTS

The AI software achieved an AUC of 89.6% (86.1-93.2%, 95% CI). The optimal threshold was 30.44, yielding 72.38% sensitivity and 92.86% specificity. Initially, AI identified 57 screening-detected cancers (83.82%), 15 interval cancers (51.72%), and 4 missed cancers (50%). AI as a second reader could have led to earlier diagnosis in 24 patients (average 29.92 ± 19.67 months earlier). No significant differences were found in cancer-characteristics groups. A hybrid triage workflow scenario showed a potential 69.5% reduction in workload and a 30.5% increase in accuracy.

CONCLUSION

This AI system exhibits high sensitivity and specificity in screening mammograms, effectively identifying interval and missed cancers and identifying 23% of cancers earlier in prior mammograms. Adopting AI as a triage mechanism has the potential to reduce workload by nearly 70%.

CLINICAL RELEVANCE STATEMENT

The study proposes a more efficient method for screening programs, both in terms of workload and accuracy.

KEY POINTS

• Incorporating AI as a triage tool in screening workflow improves sensitivity (72.38%) and specificity (92.86%), enhancing detection rates for interval and missed cancers. • AI-assisted triaging is effective in differentiating low and high-risk cases, reduces radiologist workload, and potentially enables broader screening coverage. • AI has the potential to facilitate early diagnosis compared to human reading.

Differences and similarities in false interpretations by AI CAD and radiologists in screening mammography

OBJECTIVE

We aimed to evaluate the false interpretations between artificial intelligence (AI) and radiologists in screening mammography to get a better understanding of how the distribution of diagnostic mistakes might change when moving from entirely radiologist-driven to AI-integrated breast cancer screening.

METHODS AND MATERIALS

This retrospective case-control study was based on a mammography screening cohort from 2008 to 2015. The final study population included screening examinations for 714 women diagnosed with breast cancer and 8029 randomly selected healthy controls. Oversampling of controls was applied to attain a similar cancer proportion as in the source screening cohort. We examined how false-positive (FP) and false-negative (FN) assessments by AI, the first reader (RAD 1) and the second reader (RAD 2), were associated with age, density, tumor histology and cancer invasiveness in a single- and double-reader setting.

RESULTS

For each reader, the FN assessments were distributed between low- and high-density females with 53 (42%) and 72 (58%) for AI; 59 (36%) and 104 (64%) for RAD 1 and 47 (36%) and 84 (64%) for RAD 2. The corresponding numbers for FP assessments were 1820 (47%) and 2016 (53%) for AI; 1568 (46%) and 1834 (54%) for RAD 1 and 1190 (43%) and 1610 (58%) for RAD 2. For ductal cancer, the FN assessments were 79 (77%) for AI CAD; with 120 (83%) for RAD 1 and with 96 (16%) for RAD 2. For the double-reading simulation, the FP assessments were distributed between younger and older females with 2828 (2.5%) and 1554 (1.4%) for RAD 1 + RAD 2; 3850 (3.4%) and 2940 (2.6%) for AI+RAD 1 and 3430 (3%) and 2772 (2.5%) for AI+RAD 2.

CONCLUSION

The most pronounced decrease in FN assessments was noted for females over the age of 55 and for high density-women. In conclusion, AI could have an important complementary role when combined with radiologists to increase sensitivity for high-density and older females.

ADVANCES IN KNOWLEDGE

Our results highlight the potential impact of integrating AI in breast cancer screening, particularly to improve interpretation accuracy. The use of AI could enhance screening outcomes for high-density and older females.

A Narrative Review of the Use of Artificial Intelligence in Breast, Lung, and Prostate Cancer

Artificial intelligence (AI) has been an important topic within radiology. Currently, AI is used clinically to assist with the detection of lesions through detection systems. However, a number of recent studies have demonstrated the increased value of neural networks in radiology. With an increasing number of screening requirements for cancers, this review aims to study the accuracy of the numerous AI models used in the detection and diagnosis of breast, lung, and prostate cancers. This study summarizes pertinent findings from reviewed articles and provides analysis on the relevancy to clinical radiology. This study found that whereas AI is showing continual improvement in radiology, AI alone does not surpass the effectiveness of a radiologist. Additionally, it was found that there are multiple variations on how AI should be integrated with a radiologist's workflow.

Future of Breast Radiology

The landscape of breast imaging has transformed significantly since mammography's introduction in the 1960s, accelerated by ultrasound and imageguided biopsies in the 1990s. The emergence of magnetic resonance imaging (MRI) in the 2000s added a valuable dimension to advanced imaging. Multimodality and multiparametric imaging have firmly established breast radiology's pivotal role in managing breast disorders. A shift from conventional to digital radiology emerged in the late 20th and early 21st centuries, enabling advanced techniques like digital breast tomosynthesis, contrast-enhanced mammography, and artificial intelligence (AI) integration. AI's impending integration into breast radiology may enhance diagnostics and workflows. It involves computer-aided diagnosis (CAD) algorithms, workflow support algorithms, and data processing algorithms. CAD systems, developed since the 1980s, optimize cancer detection rates by addressing false positives and negatives. Radiologists' roles will evolve into specialized clinicians collaborating with AI for efficient patient care and utilizing advanced techniques with multiparametric imaging and radiomics. Wearable technologies, non-contrast MRI, and innovative modalities like photoacoustic imaging show potential to enhance diagnostics. Imaging-guided therapy, notably cryotherapy, and theranostics, gains traction. Theranostics, integrating therapy and diagnostics, holds potential for precise treatment. Advanced imaging, AI, and novel therapies will revolutionize breast radiology, offering refined diagnostics and personalized treatments. Personalized screening, AI's role, and imaging-guided therapies will shape the future of breast radiology.

Comparison of Characteristics of Breast Cancer Detected through Different Imaging Modalities in a Large Cohort of Hong Kong Chinese Women: Implication of Imaging Choice on Upcoming Local Screening Program

Background

We compared the clinico-radio-pathological characteristics of breast cancer detected through mammogram (MMG) and ultrasound (USG) and discuss the implication of the choice of imaging as the future direction of our recently launched local screening program.

Methods

Retrospective study of 14613 Hong Kong Chinese female patients with histologically confirmed breast cancer registered in the Hong Kong Breast Cancer Registry between January 2006 and February 2020. Patients were classified into four groups based on the mode of breast cancer detection (detectable by both mammogram and ultrasound (MMG+/USG+), mammogram only (MMG+/USG-), ultrasound only (MMG-/USG+), or not detectable by either (MMG-/USG-). Characteristics of breast cancer detected were compared, including patient demographics, breast density on MMG, mode of presentation, tumour size, histological type, and staging. Types of mammographic abnormalities were also evaluated for MMG+ subgroups.

Results

85% of the cancers were detectable by MMG, while USG detected an additional 9%. MMG+/USG+ cancers were larger, more advanced in stage, often of symptomatic presentation, and commonly manifested as mammographic mass. MMG+/USG- cancers were more likely of asymptomatic presentation, manifested as microcalcifications, and of earlier stage and to be ductal carcinoma in situ. MMG-/USG+ cancers were more likely seen in young patients and those with denser breasts and more likely of symptomatic presentation. MMG-/USG- cancers were often smaller and found in denser breasts.

Conclusion

Mammogram has a good detection rate of cancers in our local population. It has superiority in detecting early cancers by detecting microcalcifications. Our current study agrees that ultrasound is one of the key adjunct tools of breast cancer detection.

Impact of artificial intelligence in breast cancer screening with mammography

Objectives

To demonstrate that radiologists, with the help of artificial intelligence (AI), are able to better classify screening mammograms into the correct breast imaging reporting and data system (BI-RADS) category, and as a secondary objective, to explore the impact of AI on cancer detection and mammogram interpretation time.

Methods

A multi-reader, multi-case study with cross-over design, was performed, including 314 mammograms. Twelve radiologists interpreted the examinations in two sessions delayed by a 4 weeks wash-out period with and without AI support. For each breast of each mammogram, they had to mark the most suspicious lesion (if any) and assign it with a forced BI-RADS category and a level of suspicion or “continuous BI-RADS 100”.

Cohen’s kappa correlation coefficient evaluating the inter-observer agreement for BI-RADS category per breast, and the area under the receiver operating characteristic curve (AUC), were used as metrics and analyzed.

Results

On average, the quadratic kappa coefficient increased significantly when using AI for all readers [κ = 0.549, 95% CI (0.528–0.571) without AI and κ = 0.626, 95% CI (0.607–0.6455) with AI]. AUC was significantly improved when using AI (0.74 vs 0.77, p = 0.004). Reading time was not significantly affected for all readers (106 s without AI and vs 102 s with AI; p = 0.754).

Conclusions

When using AI, radiologists were able to better assign mammograms with the correct BI-RADS category without slowing down the interpretation time.