Machine Learning for Workflow Applications in Screening Mammography: Systematic Review and Meta-Analysis

The efficacy of artificial intelligence (AI) in detecting interval cancers in the national screening program of a middle-income country

AIM

We aimed to investigate the efficiency and accuracy of an artificial intelligence (AI) algorithm for detecting interval cancers in a middle-income country's national screening program.

MATERIAL AND METHODS

A total of 2,129,486 mammograms reported as BIRADS 1 and 2 were matched with the national cancer registry for interval cancers (IC). The IC group consisted of 442 cases, of which 36 were excluded due to having mammograms incompatible with the AI system. A control group of 446 women with two negative consequent mammograms was defined as time-proven normal and constituted the normal group. The cancer risk scores of both groups were determined from 1 to 10 with the AI system. The sensitivity and specificity values of the AI system were defined in terms of IC detection. The IC group was divided into subgroups with six-month intervals according to their time from screening to diagnosis: 0-6 months, 6-12 months, 12-18 months, and 18-24 months. The diagnostic performance of the AI system for all patients was evaluated using receiver operating characteristics (ROC) curve analysis. The diagnostic performance of the AI system for major and minor findings that expert readers determined was re-evaluated.

RESULTS

AI labeled 53% of ICs with the highest score of 10. The sensitivity of AI in detecting ICs was 53.7% and 38.5% at specificities of 90% and 95%, respectively. Area under the curve (AUC) of AI in detecting major signs was 0.93 (95% CI: 0.90-0.95) with a sensitivity of 81.6% and 72.4% at specificities of 90% and 95%, respectively (95% CI: 0.73-0.88 and 95% CI: 0.60-0.82 respectively) and minor signs was 0.87 (95% CI: 0.87-0.92) with a sensitivity of 70% and 53% at a specificity of 90% and 95%, respectively (95% CI: 0.65-0.82 and 95% CI: 0.52-0.71 respectively). In subgroup analysis for time to diagnosis, the AUC value of the AI system was higher in the 0-6 month period than in later periods.

CONCLUSION

This study showed the potential of AI in detecting ICs in initial mammograms and reducing human errors and undetected cancers.

Performance of artificial intelligence in 7533 consecutive prevalent screening mammograms from the BreastScreen Australia program

OBJECTIVES

To assess the performance of an artificial intelligence (AI) algorithm in the Australian mammography screening program which routinely uses two independent readers with arbitration of discordant results.

METHODS

A total of 7533 prevalent round mammograms from 2017 were available for analysis. The AI program classified mammograms into deciles on the basis of breast cancer (BC) risk. BC diagnoses, including invasive BC (IBC) and ductal carcinoma in situ (DCIS), included those from the prevalent round, interval cancers, and cancers identified in the subsequent screening round two years later. Performance was assessed by sensitivity, specificity, positive and negative predictive values, and the proportion of women recalled by the radiologists and identified as higher risk by AI.

RESULTS

Radiologists identified 54 women with IBC and 13 with DCIS with a recall rate of 9.7%. In contrast, 51 of 54 of the IBCs and 12/13 cases of DCIS were within the higher AI score group (score 10), a recall equivalent of 10.6% (a difference of 0.9% (CI -0.03 to 1.89%, p = 0.06). When IBCs were identified in the 2017 round, interval cancers classified as false negatives or with minimal signs in 2017, and cancers from the 2019 round were combined, the radiologists identified 54/67 and 59/67 were in the highest risk AI category (sensitivity 80.6% and 88.06 % respectively, a difference that was not different statistically).

CONCLUSIONS

As the performance of AI was comparable to that of expert radiologists, future AI roles in screening could include replacing one reader and supporting arbitration, reducing workload and false positive results.

CLINICAL RELEVANCE STATEMENT

AI analysis of consecutive prevalent screening mammograms from the Australian BreastScreen program demonstrated the algorithm's ability to match the cancer detection of experienced radiologists, additionally identifying five interval cancers (false negatives), and the majority of the false positive recalls.

KEY POINTS

• The AI program was almost as sensitive as the radiologists in terms of identifying prevalent lesions (51/54 for invasive breast cancer, 63/67 when including ductal carcinoma in situ). • If selected interval cancers and cancers identified in the subsequent screening round were included, the AI program identified more cancers than the radiologists (59/67 compared with 54/67, sensitivity 88.06 % and 80.6% respectively p = 0.24). • The high negative predictive value of a score of 1-9 would indicate a role for AI as a triage tool to reduce the recall rate (specifically false positives).

Population-wide evaluation of artificial intelligence and radiologist assessment of screening mammograms

OBJECTIVES

To validate an AI system for standalone breast cancer detection on an entire screening population in comparison to first-reading breast radiologists.

MATERIALS AND METHODS

All mammography screenings performed between August 4, 2014, and August 15, 2018, in the Region of Southern Denmark with follow-up within 24 months were eligible. Screenings were assessed as normal or abnormal by breast radiologists through double reading with arbitration. For an AI decision of normal or abnormal, two AI-score cut-off points were applied by matching at mean sensitivity (AIsens) and specificity (AIspec) of first readers. Accuracy measures were sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and recall rate (RR).

RESULTS

The sample included 249,402 screenings (149,495 women) and 2033 breast cancers (72.6% screen-detected cancers, 27.4% interval cancers). AIsens had lower specificity (97.5% vs 97.7%; p < 0.0001) and PPV (17.5% vs 18.7%; p = 0.01) and a higher RR (3.0% vs 2.8%; p < 0.0001) than first readers. AIspec was comparable to first readers in terms of all accuracy measures. Both AIsens and AIspec detected significantly fewer screen-detected cancers (1166 (AIsens), 1156 (AIspec) vs 1252; p < 0.0001) but found more interval cancers compared to first readers (126 (AIsens), 117 (AIspec) vs 39; p < 0.0001) with varying types of cancers detected across multiple subgroups.

CONCLUSION

Standalone AI can detect breast cancer at an accuracy level equivalent to the standard of first readers when the AI threshold point was matched at first reader specificity. However, AI and first readers detected a different composition of cancers.

CLINICAL RELEVANCE STATEMENT

Replacing first readers with AI with an appropriate cut-off score could be feasible. AI-detected cancers not detected by radiologists suggest a potential increase in the number of cancers detected if AI is implemented to support double reading within screening, although the clinicopathological characteristics of detected cancers would not change significantly.

KEY POINTS

• Standalone AI cancer detection was compared to first readers in a double-read mammography screening population. • Standalone AI matched at first reader specificity showed no statistically significant difference in overall accuracy but detected different cancers. • With an appropriate threshold, AI-integrated screening can increase the number of detected cancers with similar clinicopathological characteristics.

Beyond regulatory compliance: evaluating radiology artificial intelligence applications in deployment

The implementation of artificial intelligence (AI) applications in routine practice, following regulatory approval, is currently limited by practical concerns around reliability, accountability, trust, safety, and governance, in addition to factors such as cost-effectiveness and institutional information technology support. When a technology is new and relatively untested in a field, professional confidence is lacking and there is a sense of the need to go above the baseline level of validation and compliance. In this article, we propose an approach that goes beyond standard regulatory compliance for AI apps that are approved for marketing, including independent benchmarking in the lab as well as clinical audit in practice, with the aims of increasing trust and preventing harm.

The role of machine learning in advancing diabetic foot: a review

Background

Diabetic foot complications impose a significant strain on healthcare systems worldwide, acting as a principal cause of morbidity and mortality in individuals with diabetes mellitus. While traditional methods in diagnosing and treating these conditions have faced limitations, the emergence of Machine Learning (ML) technologies heralds a new era, offering the promise of revolutionizing diabetic foot care through enhanced precision and tailored treatment strategies.

Objective

This review aims to explore the transformative impact of ML on managing diabetic foot complications, highlighting its potential to advance diagnostic accuracy and therapeutic approaches by leveraging developments in medical imaging, biomarker detection, and clinical biomechanics.

Methods

A meticulous literature search was executed across PubMed, Scopus, and Google Scholar databases to identify pertinent articles published up to March 2024. The search strategy was carefully crafted, employing a combination of keywords such as "Machine Learning," "Diabetic Foot," "Diabetic Foot Ulcers," "Diabetic Foot Care," "Artificial Intelligence," and "Predictive Modeling." This review offers an in-depth analysis of the foundational principles and algorithms that constitute ML, placing a special emphasis on their relevance to the medical sciences, particularly within the specialized domain of diabetic foot pathology. Through the incorporation of illustrative case studies and schematic diagrams, the review endeavors to elucidate the intricate computational methodologies involved.

Results

ML has proven to be invaluable in deriving critical insights from complex datasets, enhancing both the diagnostic precision and therapeutic planning for diabetic foot management. This review highlights the efficacy of ML in clinical decision-making, underscored by comparative analyses of ML algorithms in prognostic assessments and diagnostic applications within diabetic foot care.

Conclusion

The review culminates in a prospective assessment of the trajectory of ML applications in the realm of diabetic foot care. We believe that despite challenges such as computational limitations and ethical considerations, ML remains at the forefront of revolutionizing treatment paradigms for the management of diabetic foot complications that are globally applicable and precision-oriented. This technological evolution heralds unprecedented possibilities for treatment and opportunities for enhancing patient care.

Positive Predictive Values of Abnormality Scores From a Commercial Artificial Intelligence-Based Computer-Aided Diagnosis for Mammography

OBJECTIVE

Artificial intelligence-based computer-aided diagnosis (AI-CAD) is increasingly used in mammography. While the continuous scores of AI-CAD have been related to malignancy risk, the understanding of how to interpret and apply these scores remains limited. We investigated the positive predictive values (PPVs) of the abnormality scores generated by a deep learning-based commercial AI-CAD system and analyzed them in relation to clinical and radiological findings.

MATERIALS AND METHODS

From March 2020 to May 2022, 656 breasts from 599 women (mean age 52.6 ± 11.5 years, including 0.6% [4/599] high-risk women) who underwent mammography and received positive AI-CAD results (Lunit Insight MMG, abnormality score ≥ 10) were retrospectively included in this study. Univariable and multivariable analyses were performed to evaluate the associations between the AI-CAD abnormality scores and clinical and radiological factors. The breasts were subdivided according to the abnormality scores into groups 1 (10-49), 2 (50-69), 3 (70-89), and 4 (90-100) using the optimal binning method. The PPVs were calculated for all breasts and subgroups.

RESULTS

Diagnostic indications and positive imaging findings by radiologists were associated with higher abnormality scores in the multivariable regression analysis. The overall PPV of AI-CAD was 32.5% (213/656) for all breasts, including 213 breast cancers, 129 breasts with benign biopsy results, and 314 breasts with benign outcomes in the follow-up or diagnostic studies. In the screening mammography subgroup, the PPVs were 18.6% (58/312) overall and 5.1% (12/235), 29.0% (9/31), 57.9% (11/19), and 96.3% (26/27) for score groups 1, 2, 3, and 4, respectively. The PPVs were significantly higher in women with diagnostic indications (45.1% [155/344]), palpability (51.9% [149/287]), fatty breasts (61.2% [60/98]), and certain imaging findings (masses with or without calcifications and distortion).

CONCLUSION

PPV increased with increasing AI-CAD abnormality scores. The PPVs of AI-CAD satisfied the acceptable PPV range according to Breast Imaging-Reporting and Data System for screening mammography and were higher for diagnostic mammography.

Transfer learning for the generalization of artificial intelligence in breast cancer detection: a case-control study

BACKGROUND

Some researchers have questioned whether artificial intelligence (AI) systems maintain their performance when used for women from populations not considered during the development of the system.

PURPOSE

To evaluate the impact of transfer learning as a way of improving the generalization of AI systems in the detection of breast cancer.

MATERIAL AND METHODS

This retrospective case-control Finnish study involved 191 women diagnosed with breast cancer and 191 matched healthy controls. We selected a state-of-the-art AI system for breast cancer detection trained using a large US dataset. The selected baseline system was evaluated in two experimental settings. First, we examined our private Finnish sample as an independent test set that had not been considered in the development of the system (unseen population). Second, the baseline system was retrained to attempt to improve its performance in the unseen population by means of transfer learning. To analyze performance, we used areas under the receiver operating characteristic curve (AUCs) with DeLong's test.

RESULTS

Two versions of the baseline system were considered: ImageOnly and Heatmaps. The ImageOnly and Heatmaps versions yielded mean AUC values of 0.82±0.008 and 0.88±0.003 in the US dataset and 0.56 (95% CI=0.50-0.62) and 0.72 (95% CI=0.67-0.77) when evaluated in the unseen population, respectively. The retrained systems achieved AUC values of 0.61 (95% CI=0.55-0.66) and 0.69 (95% CI=0.64-0.75), respectively. There was no statistical difference between the baseline system and the retrained system.

CONCLUSION

Transfer learning with a small study sample did not yield a significant improvement in the generalization of the system.

Diagnostic capabilities of artificial intelligence as an additional reader in a breast cancer screening program

OBJECTIVES

We aimed to evaluate the early-detection capabilities of AI in a screening program over its duration, with a specific focus on the detection of interval cancers, the early detection of cancers with the assistance of AI from prior visits, and its impact on workload for various reading scenarios.

MATERIALS AND METHODS

The study included 22,621 mammograms of 8825 women within a 10-year biennial two-reader screening program. The statistical analysis focused on 5136 mammograms from 4282 women due to data retrieval issues, among whom 105 were diagnosed with breast cancer. The AI software assigned scores from 1 to 100. Histopathology results determined the ground truth, and Youden's index was used to establish a threshold. Tumor characteristics were analyzed with ANOVA and chi-squared test, and different workflow scenarios were evaluated using bootstrapping.

RESULTS

The AI software achieved an AUC of 89.6% (86.1-93.2%, 95% CI). The optimal threshold was 30.44, yielding 72.38% sensitivity and 92.86% specificity. Initially, AI identified 57 screening-detected cancers (83.82%), 15 interval cancers (51.72%), and 4 missed cancers (50%). AI as a second reader could have led to earlier diagnosis in 24 patients (average 29.92 ± 19.67 months earlier). No significant differences were found in cancer-characteristics groups. A hybrid triage workflow scenario showed a potential 69.5% reduction in workload and a 30.5% increase in accuracy.

CONCLUSION

This AI system exhibits high sensitivity and specificity in screening mammograms, effectively identifying interval and missed cancers and identifying 23% of cancers earlier in prior mammograms. Adopting AI as a triage mechanism has the potential to reduce workload by nearly 70%.

CLINICAL RELEVANCE STATEMENT

The study proposes a more efficient method for screening programs, both in terms of workload and accuracy.

KEY POINTS

• Incorporating AI as a triage tool in screening workflow improves sensitivity (72.38%) and specificity (92.86%), enhancing detection rates for interval and missed cancers. • AI-assisted triaging is effective in differentiating low and high-risk cases, reduces radiologist workload, and potentially enables broader screening coverage. • AI has the potential to facilitate early diagnosis compared to human reading.

Covalent Organic Frameworks: Opportunities for Rational Materials Design in Cancer Therapy

Nanomedicines are extensively used in cancer therapy. Covalent organic frameworks (COFs) are crystalline organic porous materials with several benefits for cancer therapy, including porosity, design flexibility, functionalizability, and biocompatibility. This review examines the use of COFs in cancer therapy from the perspective of reticular chemistry and function-oriented materials design. First, the modification sites and functionalization methods of COFs are discussed, followed by their potential as multifunctional nanoplatforms for tumor targeting, imaging, and therapy by integrating functional components. Finally, some challenges in the clinical translation of COFs are presented with the hope of promoting the development of COF-based anticancer nanomedicines and bringing COFs closer to clinical trials.

Why Medical Students Pursue Radiology: A Current Longitudinal Survey on Motivations and Controversial Issues in Radiology

RATIONALE AND OBJECTIVES

Radiology is an increasingly competitive specialty. Various current factors influence medical students' decision to pursue a radiology career, including artificial intelligence (AI), remote reading, and COVID-19. This study seeks to determine the decision-making factors of all alumni from our medical school who matched into a radiology residency, and to gather opinions on emerging radiology topics.

MATERIALS AND METHODS

A survey querying decision-making factors and opinions on current radiology topics was distributed to all alumni from our medical school (first graduating class in 2011) who previously matched into a diagnostic or interventional radiology residency program (n = 57). Wilcoxon Rank-Sum and Fisher's Exact tests were used to determine statistical significance.

RESULTS

Forty-three of fifty-seven responses were received (75% response rate). The most influential factor that sparked respondents' interest in radiology was a radiology elective (25/43, 58%). Students who will finish radiology training in 2023 or later were more likely to be influenced by a mentor (15/23, 65%) than those who finished radiology training before 2023 (5/20, 25%) (p = 0.04). Respondents reported a 1.6/5 concern about AI negatively impacting their future career in radiology. There was 1.7/5 concern about performing radiology procedures on patients during the COVID-19 pandemic. Respondents predicted that remote reading would have a 3.2/5 positive impact on helping them achieve their preferred lifestyle. Job satisfaction among attending radiologists is rated at 4.3/5.

CONCLUSION

Radiology electives had the greatest influence in piquing students' interest in radiology, while mentorship is assuming increasing influence. AI is perceived as a relatively minimal threat to negatively impact radiologists' jobs. Respondents had little concern about performing radiology procedures during the COVID-19 pandemic. Remote reading is viewed as having a moderately positive impact on lifestyle. Responding radiologists enjoy notably high job satisfaction.

Role of Ethics in Developing AI-Based Applications in Medicine: Insights From Expert Interviews and Discussion of Implications

BACKGROUND

The integration of artificial intelligence (AI)-based applications in the medical field has increased significantly, offering potential improvements in patient care and diagnostics. However, alongside these advancements, there is growing concern about ethical considerations, such as bias, informed consent, and trust in the development of these technologies.

OBJECTIVE

This study aims to assess the role of ethics in the development of AI-based applications in medicine. Furthermore, this study focuses on the potential consequences of neglecting ethical considerations in AI development, particularly their impact on patients and physicians.

METHODS

Qualitative content analysis was used to analyze the responses from expert interviews. Experts were selected based on their involvement in the research or practical development of AI-based applications in medicine for at least 5 years, leading to the inclusion of 7 experts in the study.

RESULTS

The analysis revealed 3 main categories and 7 subcategories reflecting a wide range of views on the role of ethics in AI development. This variance underscores the subjectivity and complexity of integrating ethics into the development of AI in medicine. Although some experts view ethics as fundamental, others prioritize performance and efficiency, with some perceiving ethics as potential obstacles to technological progress. This dichotomy of perspectives clearly emphasizes the subjectivity and complexity surrounding the role of ethics in AI development, reflecting the inherent multifaceted nature of this issue.

CONCLUSIONS

Despite the methodological limitations impacting the generalizability of the results, this study underscores the critical importance of consistent and integrated ethical considerations in AI development for medical applications. It advocates further research into effective strategies for ethical AI development, emphasizing the need for transparent and responsible practices, consideration of diverse data sources, physician training, and the establishment of comprehensive ethical and legal frameworks.

Value of magnetic resonance imaging radiomics features in predicting histologic grade of invasive ductal carcinoma of the breast

BACKGROUND

Breast cancer has the second highest mortality rate of all cancers and occurs mainly in women.

OBJECTIVE

To investigate the relationship between magnetic resonance imaging (MRI) radiomics features and histological grade of invasive ductal carcinoma (IDC) of the breast and to evaluate its diagnostic efficacy.

METHODS

The two conventional MRI quantitative indicators, i.e. the apparent diffusion coefficient (ADC) and the initial enhancement rate, were collected from 112 patients with breast cancer. The breast cancer lesions were manually segmented in dynamic contrast-enhanced MRI (DCE-MRI) and ADC images, the differences in radiomics features between Grades I, II and III IDCs were compared and the diagnostic efficacy was evaluated.

RESULTS

The ADC values (0.77 ± 0.22 vs 0.91 ± 0.22 vs 0.92 ± 0.20, F= 4.204, p< 0.01), as well as the B_sum_variance (188.51 ± 67.803 vs 265.37 ± 77.86 vs 263.74 ± 82.58, F= 6.040, p< 0.01), L_energy (0.03 ± 0.02 vs 0.13 ± 0.11 vs 0.12 ± 0.14, F= 7.118, p< 0.01) and L_sum_average (0.78 ± 0.32 vs 16.34 ± 4.23 vs 015.45 ± 3.74, F= 21.860, p< 0.001) values of patients with Grade III IDC were significantly lower than those of patients with Grades I and II IDC. The B_uniform (0.15 ± 0.12 vs 0.11 ± 0.04 vs 0.12 ± 0.03, F= 3.797, p< 0.01) and L_SRE (0.85 ± 0.07 vs 0.78 ± 0.03 vs 0.79 ± 0.32, F= 3.024, p< 0.01) values of patients with Grade III IDC were significantly higher than those of patients with Grades I and II IDC. All differences were statistically significant (p< 0.05). The ADC radiomics signature model had a higher area-under-the-curve value in identifying different grades of IDC than the ADC value model and the DCE radiomics signature model (0.869 vs 0.711 vs 0.682). The accuracy (0.812 vs 0.647 vs 0.710), specificity (0.731 vs 0.435 vs 0.342), positive predictive value (0.815 vs 0.663 vs 0.669) and negative predictive value (0.753 vs 0.570 vs 0.718) of the ADC radiomics signature model were all significantly better than the ADC value model and the DCE radiomics signature model.

CONCLUSION

ADC values and breast MRI radiomics signatures are significant in identifying the histological grades of IDC, with the ADC radiomics signatures having greater value.

Systematic reviews of machine learning in healthcare: a literature review

INTRODUCTION

The increasing availability of data and computing power has made machine learning (ML) a viable approach to faster, more efficient healthcare delivery.

METHODS

A systematic literature review (SLR) of published SLRs evaluating ML applications in healthcare settings published between1 January 2010 and 27 March 2023 was conducted.

RESULTS

In total 220 SLRs covering 10,462 ML algorithms were reviewed. The main application of AI in medicine related to the clinical prediction and disease prognosis in oncology and neurology with the use of imaging data. Accuracy, specificity, and sensitivity were provided in 56%, 28%, and 25% SLRs respectively. Internal and external validation was reported in 53% and less than 1% of the cases respectively. The most common modeling approach was neural networks (2,454 ML algorithms), followed by support vector machine and random forest/decision trees (1,578 and 1,522 ML algorithms, respectively).

EXPERT OPINION

The review indicated considerable reporting gaps in terms of the ML's performance, both internal and external validation. Greater accessibility to healthcare data for developers can ensure the faster adoption of ML algorithms into clinical practice.

Breast cancer detection accuracy of AI in an entire screening population: a retrospective, multicentre study

BACKGROUND

Artificial intelligence (AI) systems are proposed as a replacement of the first reader in double reading within mammography screening. We aimed to assess cancer detection accuracy of an AI system in a Danish screening population.

METHODS

We retrieved a consecutive screening cohort from the Region of Southern Denmark including all participating women between Aug 4, 2014, and August 15, 2018. Screening mammograms were processed by a commercial AI system and detection accuracy was evaluated in two scenarios, Standalone AI and AI-integrated screening replacing first reader, with first reader and double reading with arbitration (combined reading) as comparators, respectively. Two AI-score cut-off points were applied by matching at mean first reader sensitivity (AIsens) and specificity (AIspec). Reference standard was histopathology-proven breast cancer or cancer-free follow-up within 24 months. Coprimary endpoints were sensitivity and specificity, and secondary endpoints were positive predictive value (PPV), negative predictive value (NPV), recall rate, and arbitration rate. Accuracy estimates were calculated using McNemar's test or exact binomial test.

RESULTS

Out of 272,008 screening mammograms from 158,732 women, 257,671 (94.7%) with adequate image data were included in the final analyses. Sensitivity and specificity were 63.7% (95% CI 61.6%-65.8%) and 97.8% (97.7-97.8%) for first reader, and 73.9% (72.0-75.8%) and 97.9% (97.9-98.0%) for combined reading, respectively. Standalone AIsens showed a lower specificity (-1.3%) and PPV (-6.1%), and a higher recall rate (+ 1.3%) compared to first reader (p < 0.0001 for all), while Standalone AIspec had a lower sensitivity (-5.1%; p < 0.0001), PPV (-1.3%; p = 0.01) and NPV (-0.04%; p = 0.0002). Compared to combined reading, Integrated AIsens achieved higher sensitivity (+ 2.3%; p = 0.0004), but lower specificity (-0.6%) and PPV (-3.9%) as well as higher recall rate (+ 0.6%) and arbitration rate (+ 2.2%; p < 0.0001 for all). Integrated AIspec showed no significant difference in any outcome measures apart from a slightly higher arbitration rate (p < 0.0001). Subgroup analyses showed higher detection of interval cancers by Standalone AI and Integrated AI at both thresholds (p < 0.0001 for all) with a varying composition of detected cancers across multiple subgroups of tumour characteristics.

CONCLUSIONS

Replacing first reader in double reading with an AI could be feasible but choosing an appropriate AI threshold is crucial to maintaining cancer detection accuracy and workload.

The value of machine learning in preoperative identification of lymph node metastasis status in endometrial cancer: a systematic review and meta-analysis

Background

The early identification of lymph node metastasis status in endometrial cancer (EC) is a serious challenge in clinical practice. Some investigators have introduced machine learning into the early identification of lymph node metastasis in EC patients. However, the predictive value of machine learning is controversial due to the diversity of models and modeling variables. To this end, we carried out this systematic review and meta-analysis to systematically discuss the value of machine learning for the early identification of lymph node metastasis in EC patients.

Methods

A systematic search was conducted in Pubmed, Cochrane, Embase, and Web of Science until March 12, 2023. PROBAST was used to assess the risk of bias in the included studies. In the process of meta-analysis, subgroup analysis was performed according to modeling variables (clinical features, radiomic features, and radiomic features combined with clinical features) and different types of models in various variables.

Results

This systematic review included 50 primary studies with a total of 103,752 EC patients, 12,579 of whom had positive lymph node metastasis. Meta-analysis showed that among the machine learning models constructed by the three categories of modeling variables, the best model was constructed by combining radiomic features with clinical features, with a pooled c-index of 0.907 (95%CI: 0.886-0.928) in the training set and 0.823 (95%CI: 0.757-0.890) in the validation set, and good sensitivity and specificity. The c-index of the machine learning model constructed based on clinical features alone was not inferior to that based on radiomic features only. In addition, logistic regression was found to be the main modeling method and has ideal predictive performance with different categories of modeling variables.

Conclusion

Although the model based on radiomic features combined with clinical features has the best predictive efficiency, there is no recognized specification for the application of radiomics at present. In addition, the logistic regression constructed by clinical features shows good sensitivity and specificity. In this context, large-sample studies covering different races are warranted to develop predictive nomograms based on clinical features, which can be widely applied in clinical practice.

Systematic review registration

https://www.crd.york.ac.uk/PROSPERO, identifier CRD42023420774.

Artificial intelligence for triaging of breast cancer screening mammograms and workload reduction: A meta-analysis of a deep learning software

OBJECTIVE

Deep learning (DL) has shown promising results for improving mammographic breast cancer diagnosis. However, the impact of artificial intelligence (AI) on the breast cancer screening process has not yet been fully elucidated in terms of potential workload reduction. We aim to assess if AI-based triaging of breast cancer screening mammograms could reduce the radiologist's workload with non-inferior sensitivity.

METHODS

PubMed, EMBASE, Cochrane Central, and Web of Science databases were systematically searched for studies that evaluated AI algorithms on computer-aided triage of breast cancer screening mammograms. We extracted data from homogenous studies and performed a proportion meta-analysis with a random-effects model to examine the radiologist's workload reduction (proportion of low-risk mammograms that could be theoretically ruled out from human's assessment) and the software's sensitivity to breast cancer detection.

RESULTS

Thirteen studies were selected for full review, and three studies that used the same commercially available DL algorithm were included in the meta-analysis. In the 156,852 examinations included, the threshold of 7 was identified as optimal. With these parameters, radiologist workload decreased by 68.3% (95%CI 0.655-0.711, I² = 98.76%, p < 0.001), while achieving a sensitivity of 93.1% (95%CI 0.882-0.979, I² = 83.86%, p = 0.002) and a specificity of 68.7% (95% CI 0.684-0.723, I² = 97.5%, p < 0.01).

CONCLUSIONS

The deployment of DL computer-aided triage of breast cancer screening mammograms reduces the radiology workload while maintaining high sensitivity. Although the implementation of AI remains complex and heterogeneous, it is a promising tool to optimize healthcare resources.

Mammography Breast Cancer Screening Triage Using Deep Learning: A UK Retrospective Study

Background Breast screening enables early detection of cancers; however, most women have normal mammograms, resulting in repetitive and resource-intensive reading tasks. Purpose To investigate if deep learning (DL) algorithms can be used to triage mammograms by identifying normal results to reduce workload or flag cancers that may be overlooked. Materials and Methods In this retrospective study, three commercial DL algorithms were investigated using consecutive mammograms from two UK Breast Screening Program sites from January 2015 to December 2017 and January 2017 to December 2018 on devices from two mammography vendors. Normal mammograms with a 3-year follow-up and histopathologically proven cancer detected at screening, the subsequent round, or in the 3-year interval were included. Two algorithm thresholds were set: in scenario A, 99.0% sensitivity for rule-out triage to a lone reader, and in scenario B, approximately 1.0% additional recall providing a rule-in triage for further assessment. Both thresholds were then applied to the screening workflow in scenario C. The sensitivity and specificity were used to assess the overall predictive performance of each DL algorithm. Results The data set comprised 78 849 patients (median age, 59 years [IQR, 53-63 years]) and 887 screening-detected, 439 interval, and 688 subsequent screening round-detected cancers. In scenario A (rule-out triage), models DL-1, DL-2, and DL-3 triaged 35.0% (27 565 of 78 849), 53.2% (41 937 of 78 849), and 55.6% (43 869 of 78 849) of mammograms, respectively, with 0.0% (0 of 887) to 0.1% (one of 887) of screening-detected cancers undetected. In scenario B, DL algorithms triaged in 4.6% (20 of 439) to 8.2% (36 of 439) of interval and 5.2% (36 of 688) to 6.1% (42 of 688) of subsequent-round cancers when applied after the routine double-reading workflow. Combining both approaches in scenario C resulted in an overall noninferior specificity (difference, -0.9%; P < .001) and superior sensitivity (difference, 2.7%; P < .001) for the adaptive workflow compared with routine double reading for all three algorithms. Conclusion Rule-out and rule-in DL-adapted triage workflows can improve the efficiency and efficacy of mammography breast cancer screening. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Nishikawa and Lu in this issue.

An automated surgical decision-making framework for partial or radical nephrectomy based on 3D-CT multi-level anatomical features in renal cell carcinoma

OBJECTIVES

To determine whether 3D-CT multi-level anatomical features can provide a more accurate prediction of surgical decision-making for partial or radical nephrectomy in renal cell carcinoma.

METHODS

This is a retrospective study based on multi-center cohorts. A total of 473 participants with pathologically proved renal cell carcinoma were split into the internal training and the external testing set. The training set contains 412 cases from five open-source cohorts and two local hospitals. The external testing set includes 61 participants from another local hospital. The proposed automatic analytic framework contains the following modules: a 3D kidney and tumor segmentation model constructed by 3D-UNet, a multi-level feature extractor based on the region of interest, and a partial or radical nephrectomy prediction classifier by XGBoost. The fivefold cross-validation strategy was used to get a robust model. A quantitative model interpretation method called the Shapley Additive Explanations was conducted to explore the contribution of each feature.

RESULTS

In the prediction of partial versus radical nephrectomy, the combination of multi-level features achieved better performance than any single-level feature. For the internal validation, the AUROC was 0.93 ± 0.1, 0.94 ± 0.1, 0.93 ± 0.1, 0.93 ± 0.1, and 0.93 ± 0.1, respectively, as determined by the fivefold cross-validation. The AUROC from the optimal model was 0.82 ± 0.1 in the external testing set. The tumor shape Maximum 3D Diameter plays the most vital role in the model decision.

CONCLUSIONS

The automated surgical decision framework for partial or radical nephrectomy based on 3D-CT multi-level anatomical features exhibits robust performance in renal cell carcinoma. The framework points the way towards guiding surgery through medical images and machine learning.

CLINICAL RELEVANCE STATEMENT

We proposed an automated analytic framework that can assist surgeons in partial or radical nephrectomy decision-making. The framework points the way towards guiding surgery through medical images and machine learning.

KEY POINTS

• The 3D-CT multi-level anatomical features provide a more accurate prediction of surgical decision-making for partial or radical nephrectomy in renal cell carcinoma. • The data from multicenter study and a strict fivefold cross-validation strategy, both internal validation set and external testing set, can be easily transferred to different tasks of new datasets. • The quantitative decomposition of the prediction model was conducted to explore the contribution of each extracted feature.

Mammography diagnosis of breast cancer screening through machine learning: a systematic review and meta-analysis

Breast cancer was the fourth leading cause of cancer-related death worldwide, and early mammography screening could decrease the breast cancer mortality. Artificial intelligence (AI)-assisted diagnose system based on machine learning (ML) methods can help improve the screening accuracy and efficacy. This study aimed to systematically review and make a meta-analysis on the diagnostic accuracy of mammography diagnosis of breast cancer through various ML methods. Springer Link, Science Direct (Elsevier), IEEE Xplore, PubMed and Web of Science were searched for relevant studies published from January 2000 to September 2021. The study was registered with the PROSPERO International Prospective Register of Systematic Reviews (protocol no. CRD42021284227). A Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) was used to assess the included studies, and reporting was evaluated using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA). The pooled summary estimates for sensitivity, specificity, the area under the receiver operating characteristic curve (AUC) for three ML methods (convolutional neural network [CNN], artificial neural network [ANN], support vector machine [SVM]) were calculated. A total of 32 studies with 23,804 images were included in the meta-analysis. The overall pooled estimate for sensitivity, specificity and AUC was 0.914 [95% CI 0.868-0.945], 0.916 [95% CI 0.873-0.945] and 0.945 for mammography diagnosis of breast cancer through three ML methods. The pooled sensitivity, specificity and AUC of CNN were 0.961 [95% CI 0.886-0.988], 0.950 [95% CI 0.924-0.967] and 0.974. The pooled sensitivity, specificity and AUC of ANN were 0.837 [95% CI 0.772-0.886], 0.894 [95% CI 0.764-0.957] and 0.881. The pooled sensitivity, specificity and AUC of SVM were 0.889 [95% CI 0.807-0.939], 0.843 [95% CI 0.724-0.916] and 0.913. Machine learning methods (especially CNN) show excellent performance in mammography diagnosis of breast cancer screening based on retrospective studies. More rigorous prospective studies are needed to evaluate the longitudinal performance of AI.

Breast cancer pre-clinical screening using infrared thermography and artificial intelligence: a prospective, multicentre, diagnostic accuracy cohort study

BACKGROUND

Given the limited access to breast cancer (BC) screening, the authors developed and validated a mobile phone-artificial intelligence-based infrared thermography (AI-IRT) system for BC screening.

MATERIALS AND METHODS

This large prospective clinical trial assessed the diagnostic performance of the AI-IRT system. The authors constructed two datasets and two models, performed internal and external validation, and compared the diagnostic accuracy of the AI models and clinicians. Dataset A included 2100 patients recruited from 19 medical centres in nine regions of China. Dataset B was used for independent external validation and included 102 patients recruited from Langfang People's Hospital.

RESULTS

The area under the receiver operating characteristic curve of the binary model for identifying low-risk and intermediate/high-risk patients was 0.9487 (95% CI: 0.9231-0.9744) internally and 0.9120 (95% CI: 0.8460-0.9790) externally. The accuracy of the binary model was higher than that of human readers (0.8627 vs. 0.8088, respectively). In addition, the binary model was better than the multinomial model and used different diagnostic thresholds based on BC risk to achieve specific goals.

CONCLUSIONS

The accuracy of AI-IRT was high across populations with different demographic characteristics and less reliant on manual interpretations, demonstrating that this model can improve pre-clinical screening and increase screening rates.

Overview of trials on artificial intelligence algorithms in breast cancer screening - A roadmap for international evaluation and implementation

Accumulating evidence from retrospective studies demonstrate at least non-inferior performance when using AI algorithms with different strategies versus double-reading in mammography screening. In addition, AI algorithms for mammography screening can reduce work load by moving to single human reading. Prospective trials are essential to avoid unintended adverse consequences before incorporation of AI algorithms into UK's National Health Service (NHS) Breast Screening Programme (BSP). A stakeholders' meeting was organized in Newnham College, Cambridge, UK to undertake a review of the current evidence to enable consensus discussion on next steps required before implementation into a screening programme. It was concluded that a multicentre multivendor testing platform study with opt-out consent is preferred. AI thresholds from different vendors should be determined while maintaining non-inferior screening performance results, particularly ensuring recall rates are not increased. Automatic recall of cases using an agreed high sensitivity AI score versus automatic rule out with a low AI score set at a high sensitivity could be used. A human reader should still be involved in decision making with AI-only recalls requiring human arbitration. Standalone AI algorithms used without prompting maintain unbiased screening reading performance, but reading with prompts should be tested prospectively and ideally provided for arbitration.

AI Risk Score on Screening Mammograms Preceding Breast Cancer Diagnosis

Background Few studies have evaluated the role of artificial intelligence (AI) in prior screening mammography. Purpose To examine AI risk scores assigned to screening mammography in women who were later diagnosed with breast cancer. Materials and Methods Image data and screening information of examinations performed from January 2004 to December 2019 as part of BreastScreen Norway were used in this retrospective study. Prior screening examinations from women who were later diagnosed with cancer were assigned an AI risk score by a commercially available AI system (scores of 1-7, low risk of malignancy; 8-9, intermediate risk; and 10, high risk of malignancy). Mammographic features of the cancers based on the AI score were also assessed. The association between AI score and mammographic features was tested with a bivariate test. Results A total of 2787 prior screening examinations from 1602 women (mean age, 59 years ± 5.1 [SD]) with screen-detected (n = 1016) or interval (n = 586) cancers showed an AI risk score of 10 for 389 (38.3%) and 231 (39.4%) cancers, respectively, on the mammograms in the screening round prior to diagnosis. Among the screen-detected cancers with AI scores available two screening rounds (4 years) before diagnosis, 23.0% (122 of 531) had a score of 10. Mammographic features were associated with AI score for invasive screen-detected cancers (P < .001). Density with calcifications was registered for 13.6% (43 of 317) of screen-detected cases with a score of 10 and 4.6% (15 of 322) for those with a score of 1-7. Conclusion More than one in three cases of screen-detected and interval cancers had the highest AI risk score at prior screening, suggesting that the use of AI in mammography screening may lead to earlier detection of breast cancers. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Mehta in this issue.

Artificial Intelligence Improves Detection of Supplemental Screening Ultrasound-detected Breast Cancers in Mammography

Despite recent advances in artificial intelligence (AI) software with improved performance in mammography screening for breast cancer, insufficient data are available on its performance in detecting cancers that were initially missed on mammography. In this study, we aimed to determine whether AI software-aided mammography could provide additional value in identifying cancers detected through supplemental screening ultrasound. We searched our database from 2017 to 2018 and included 238 asymptomatic patients (median age, 50 years; interquartile range, 45-57 years) diagnosed with breast cancer using supplemental ultrasound. Two unblinded radiologists retrospectively reviewed the mammograms using commercially available AI software and identified the reasons for missed detection. Clinicopathological characteristics of AI-detected and AI-undetected cancers were compared using univariate and multivariate logistic regression analyses. A total of 253 cancers were detected in 238 patients using ultrasound. In an unblinded review, the AI software failed to detect 187 of the 253 (73.9%) mammography cases with negative findings in retrospective observations. The AI software detected 66 cancers (26.1%), of which 42 (63.6%) exhibited indiscernible findings obscured by overlapping dense breast tissues, even with the knowledge of magnetic resonance imaging and post-wire localization mammography. The remaining 24 cases (36.4%) were considered interpretive errors by the radiologists. Invasive tumor size was associated with AI detection after multivariable analysis (odds ratio, 2.2; 95% confidence intervals, 1.5-3.3; p < 0.001). In the control group of 160 women without cancer, the AI software identified 19 false positives (11.9%, 19/160). Although most ultrasound-detected cancers were not detected on mammography with the use of AI, the software proved valuable in identifying breast cancers with indiscernible abnormalities or those that clinicians may have overlooked.

Ovarian cancer beyond imaging: integration of AI and multiomics biomarkers

High-grade serous ovarian cancer is the most lethal gynaecological malignancy. Detailed molecular studies have revealed marked intra-patient heterogeneity at the tumour microenvironment level, likely contributing to poor prognosis. Despite large quantities of clinical, molecular and imaging data on ovarian cancer being accumulated worldwide and the rise of high-throughput computing, data frequently remain siloed and are thus inaccessible for integrated analyses. Only a minority of studies on ovarian cancer have set out to harness artificial intelligence (AI) for the integration of multiomics data and for developing powerful algorithms that capture the characteristics of ovarian cancer at multiple scales and levels. Clinical data, serum markers, and imaging data were most frequently used, followed by genomics and transcriptomics. The current literature proves that integrative multiomics approaches outperform models based on single data types and indicates that imaging can be used for the longitudinal tracking of tumour heterogeneity in space and potentially over time. This review presents an overview of studies that integrated two or more data types to develop AI-based classifiers or prediction models.Relevance statement Integrative multiomics models for ovarian cancer outperform models using single data types for classification, prognostication, and predictive tasks.Key points• This review presents studies using multiomics and artificial intelligence in ovarian cancer.• Current literature proves that integrative multiomics outperform models using single data types.• Around 60% of studies used a combination of imaging with clinical data.• The combination of genomics and transcriptomics with imaging data was infrequently used.

Performance of a Breast Cancer Detection AI Algorithm Using the Personal Performance in Mammographic Screening Scheme

Background The Personal Performance in Mammographic Screening (PERFORMS) scheme is used to assess reader performance. Whether this scheme can assess the performance of artificial intelligence (AI) algorithms is unknown. Purpose To compare the performance of human readers and a commercially available AI algorithm interpreting PERFORMS test sets. Materials and Methods In this retrospective study, two PERFORMS test sets, each consisting of 60 challenging cases, were evaluated by human readers between May 2018 and March 2021 and were evaluated by an AI algorithm in 2022. AI considered each breast separately, assigning a suspicion of malignancy score to features detected. Performance was assessed using the highest score per breast. Performance metrics, including sensitivity, specificity, and area under the receiver operating characteristic curve (AUC), were calculated for AI and humans. The study was powered to detect a medium-sized effect (odds ratio, 3.5 or 0.29) for sensitivity. Results A total of 552 human readers interpreted both PERFORMS test sets, consisting of 161 normal breasts, 70 malignant breasts, and nine benign breasts. No difference was observed at the breast level between the AUC for AI and the AUC for human readers (0.93% and 0.88%, respectively; P = .15). When using the developer's suggested recall score threshold, no difference was observed for AI versus human reader sensitivity (84% and 90%, respectively; P = .34), but the specificity of AI was higher (89%) than that of the human readers (76%, P = .003). However, it was not possible to demonstrate equivalence due to the size of the test sets. When using recall thresholds to match mean human reader performance (90% sensitivity, 76% specificity), AI showed no differences inperformance, with a sensitivity of 91% (P =. 73) and a specificity of 77% (P = .85). Conclusion Diagnostic performance of AI was comparable with that of the average human reader when evaluating cases from two enriched test sets from the PERFORMS scheme. © RSNA, 2023 See also the editorial by Philpotts in this issue.

Prediction of prognosis in COVID-19 patients using machine learning: A systematic review and meta-analysis

BACKGROUND

Accurate prediction of prognostic outcomes in patients with COVID-19 could facilitate clinical decision-making and medical resource allocation. However, little is known about the ability of machine learning (ML) to predict prognosis in COVID-19 patients.

OBJECTIVE

This study aimed to systematically examine the prognostic value of ML in patients with COVID-19.

METHODS

A systematic search was conducted in PubMed, Web of Science, Embase, Cochrane Library, and IEEE Xplore up to December 15, 2021. Studies predicting the prognostic outcomes of COVID-19 patients using ML were eligible for inclusion. Risk of bias was evaluated by a tailored checklist based on Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Pooled sensitivity, specificity, and area under the receiver operating curve (AUC) were calculated to evaluate model performance.

RESULTS

A total of 33 studies that described 35 models were eligible for inclusion, with 27 models presenting mortality, four intensive care unit (ICU) admission, and four use of ventilation. For predicting mortality, ML gave a pooled sensitivity of 0.86 (95% CI, 0.79-0.90), a specificity of 0.87 (95% CI, 0.80-0.92), and an AUC of 0.93 (95% CI, 0.90-0.95). For the prediction of ICU admission, ML had a sensitivity of 0.86 (95% CI, 0.78-0.92), a specificity of 0.81 (95% CI, 0.66-0.91), and an AUC of 0.91 (95% CI, 0.88-0.93). For the prediction of ventilation, ML had a sensitivity of 0.81 (95% CI, 0.68-0.90), a specificity of 0.78 (95% CI, 0.66-0.87), and an AUC of 0.87 (95% CI, 0.83-0.89). Meta-regression analyses indicated that algorithm, population, study design, and source of dataset influenced the pooled estimate.

CONCLUSION

This meta-analysis demonstrated the satisfactory performance of ML in predicting prognostic outcomes in patients with COVID-19, suggesting the potential value of ML to support clinical decision-making. However, improvements to methodology and validation are still necessary before its application in routine clinical practice.

Standalone AI for Breast Cancer Detection at Screening Digital Mammography and Digital Breast Tomosynthesis: A Systematic Review and Meta-Analysis

Background There is considerable interest in the potential use of artificial intelligence (AI) systems in mammographic screening. However, it is essential to critically evaluate the performance of AI before it can become a modality used for independent mammographic interpretation. Purpose To evaluate the reported standalone performances of AI for interpretation of digital mammography and digital breast tomosynthesis (DBT). Materials and Methods A systematic search was conducted in PubMed, Google Scholar, Embase (Ovid), and Web of Science databases for studies published from January 2017 to June 2022. Sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) values were reviewed. Study quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 and Comparative (QUADAS-2 and QUADAS-C, respectively). A random effects meta-analysis and meta-regression analysis were performed for overall studies and for different study types (reader studies vs historic cohort studies) and imaging techniques (digital mammography vs DBT). Results In total, 16 studies that include 1 108 328 examinations in 497 091 women were analyzed (six reader studies, seven historic cohort studies on digital mammography, and four studies on DBT). Pooled AUCs were significantly higher for standalone AI than radiologists in the six reader studies on digital mammography (0.87 vs 0.81, P = .002), but not for historic cohort studies (0.89 vs 0.96, P = .152). Four studies on DBT showed significantly higher AUCs in AI compared with radiologists (0.90 vs 0.79, P < .001). Higher sensitivity and lower specificity were seen for standalone AI compared with radiologists. Conclusion Standalone AI for screening digital mammography performed as well as or better than radiologists. Compared with digital mammography, there is an insufficient number of studies to assess the performance of AI systems in the interpretation of DBT screening examinations. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Scaranelo in this issue.

Prioritizing Screening Mammograms for Immediate Interpretation and Diagnostic Evaluation on the Basis of Risk for Recall

PURPOSE

The aim of this study was to develop a prioritization strategy for scheduling immediate screening mammographic interpretation and possible diagnostic evaluation.

METHODS

A population-based cohort with screening mammograms performed from 2012 to 2020 at 126 radiology facilities from 7 Breast Cancer Surveillance Consortium registries was identified. Classification trees identified combinations of clinical history (age, BI-RADS® density, time since prior mammogram, history of false-positive recall or biopsy result), screening modality (digital mammography, digital breast tomosynthesis), and facility characteristics (profit status, location, screening volume, practice type, academic affiliation) that grouped screening mammograms by recall rate, with ≥12/100 considered high and ≥16/100 very high. An efficiency ratio was estimated as the percentage of recalls divided by the percentage of mammograms.

RESULTS

The study cohort included 2,674,051 screening mammograms in 925,777 women, with 235,569 recalls. The most important predictor of recall was time since prior mammogram, followed by age, history of false-positive recall, breast density, history of benign biopsy, and screening modality. Recall rates were very high for baseline mammograms (21.3/100; 95% confidence interval, 19.7-23.0) and high for women with ≥5 years since prior mammogram (15.1/100; 95% confidence interval, 14.3-16.1). The 9.2% of mammograms in subgroups with very high and high recall rates accounted for 19.2% of recalls, an efficiency ratio of 2.1 compared with a random approach. Adding women <50 years of age with dense breasts accounted for 20.3% of mammograms and 33.9% of recalls (efficiency ratio = 1.7). Results including facility-level characteristics were similar.

CONCLUSIONS

Prioritizing women with baseline mammograms or ≥5 years since prior mammogram for immediate interpretation and possible diagnostic evaluation could considerably reduce the number of women needing to return for diagnostic imaging at another visit.

Artificial Intelligence and Cancer Control: Toward Prioritizing Justice, Equity, Diversity, and Inclusion (JEDI) in Emerging Decision Support Technologies

PURPOSE FOR REVIEW

This perspective piece has two goals: first, to describe issues related to artificial intelligence-based applications for cancer control as they may impact health inequities or disparities; and second, to report on a review of systematic reviews and meta-analyses of artificial intelligence-based tools for cancer control to ascertain the extent to which discussions of justice, equity, diversity, inclusion, or health disparities manifest in syntheses of the field's best evidence.

RECENT FINDINGS

We found that, while a significant proportion of existing syntheses of research on AI-based tools in cancer control use formal bias assessment tools, the fairness or equitability of models is not yet systematically analyzable across studies. Issues related to real-world use of AI-based tools for cancer control, such as workflow considerations, measures of usability and acceptance, or tool architecture, are more visible in the literature, but still addressed only in a minority of reviews. Artificial intelligence is poised to bring significant benefits to a wide range of applications in cancer control, but more thorough and standardized evaluations and reporting of model fairness are required to build the evidence base for AI-based tool design for cancer and to ensure that these emerging technologies promote equitable healthcare.

Artificial Intelligence for Indication of Invasive Assessment of Calcifications in Mammography Screening

PURPOSE

Lesion-related evaluation of the diagnostic performance of an individual artificial intelligence (AI) system to assess mamographically detected and histologically proven calcifications.

MATERIALS AND METHODS

This retrospective study included 634 women of one screening unit (July 2012 - June 2018) who completed the invasive assessment of calcifications. For each leasion, the AI-system calculated a score between 0 and 98. Lesions scored > 0 were classified as AI-positive. The performance of the system was evaluated based on its positive predictive value of invasive assessment (PPV3), the false-negative rate and the true-negative rate.

RESULTS

The PPV3 increased across the categories (readers: 4a: 21.2 %, 4b: 57.7 %, 5: 100 %, overall 30.3 %; AI: 4a: 20.8 %, 4b: 57.8 %, 5: 100 %, overall: 30.7 %). The AI system yielded a false-negative rate of 7.2 % (95 %-CI: 4.3 %: 11.4 %) and a true-negative rate of 9.1 % (95 %-CI: 6.6 %; 11.9 %). These rates were highest in category 4a, 12.5 % and 10.4 % retrospectively. The lowest median AI score was observed for benign lesions (61, interquartile range (IQR): 45-74). Invasive cancers yielded the highest median AI score (81, IQR: 64-86). Median AI scores for ductal carcinoma in situ were: 74 (IQR: 63-84) for low grade, 70 (IQR: 52-79) for intermediate grade and 74 (IQR: 66-83) for high grade.

CONCLUSION

At the lowest threshold, the AI system yielded calcification-related PPV3 values that increased across categories, similar as seen in human evaluation. The strongest loss in AI-based breast cancer detection was observed for invasively assessed calcifications with the lowest suspicion of malignancy, yet with a comparable decrease in the false-positive rate. An AI-score based stratification of malignant lesions could not be determined.

KEY POINTS

· The AI-based PPV3 for calcifications is comparable to human assessment.. · AI showed a lower detection performance of screen-positive and screen-negative lesions in category 4a.. · Histological subgroups could not be discriminated by AI scores..

CITATION FORMAT

· Weigel S, Brehl AK, Heindel W et al. Artificial Intelligence for Indication of Invasive Assessment of Calcifications in Mammography Screening. Fortschr Röntgenstr 2023; 195: 38 - 46.

Performance of Radiologists and Radiographers in Double Reading Mammograms: The UK National Health Service Breast Screening Program

Background Double reading can be used in screening mammography, but it is labor intensive. There is limited evidence on whether trained radiographers (ie, technologists) may be used to provide double reading. Purpose To compare the performance of radiologists and radiographers double reading screening mammograms, considering reader experience level. Materials and Methods In this retrospective study, performance and experience data were obtained for radiologists and radiographer readers of all screening mammograms in England from April 2015 to March 2016. Cancer detection rate (CDR), recall rate (RR), and positive predictive value (PPV) of recall based on biopsy-proven findings were calculated for first readers. Performance metrics were analyzed according to reader professional group and years of reading experience using the analysis of variance test. P values less than .05 were considered to indicate statistically significant difference. Results During the study period, 401 readers (224 radiologists and 177 radiographers) double read 1 404 395 screening digital mammograms. There was no difference in CDR between radiologist and radiographer readers (mean, 7.84 vs 7.53 per 1000 examinations, respectively; P = .08) and no difference for readers with more than 10 years of experience compared with 5 years or fewer years of experience, regardless of professional group (mean, 7.75 vs 7.71 per 1000 examinations respectively, P = .87). No difference in the mean RR was observed between radiologists and radiographer readers (5.0% vs 5.2%, respectively, P = .63). A lower RR was seen for readers with more than 10 years of experience compared with 5 years or fewer, regardless of professional group (mean, 4.8% vs 5.8%, respectively; P = .001). No variation in PPV was observed between them (P = .42), with PPV values of 17.1% for radiologists versus 16.1% for radiographers. A higher PPV was seen for readers with more than 10 years of experience compared with 5 years or less, regardless of professional group (mean, 17.5% and 14.9%, respectively; P = .02). Conclusion No difference in performance was observed between radiographers and radiologists reading screening mammograms in a program that used double reading. Published under a CC BY 4.0 license Online supplemental material is available for this article. See also the editorial by Hooley and Durand in this issue.

Artificial intelligence for digital breast tomosynthesis: Impact on diagnostic performance, reading times, and workload in the era of personalized screening

The ultimate goals of the application of artificial intelligence (AI) to digital breast tomosynthesis (DBT) are the reduction of reading times, the increase of diagnostic performance, and the reduction of interval cancer rates. In this review, after outlining the journey from computer-aided detection/diagnosis systems to AI applied to digital mammography (DM), we summarize the results of studies where AI was applied to DBT, noting that long-term advantages of DBT screening and its crucial ability to decrease the interval cancer rate are still under scrutiny. AI has shown the capability to overcome some shortcomings of DBT in the screening setting by improving diagnostic performance and by reducing recall rates (from -2 % to -27 %) and reading times (up to -53 %, with an average 20 % reduction), but the ability of AI to reduce interval cancer rates has not yet been clearly investigated. Prospective validation is needed to assess the cost-effectiveness and real-world impact of AI models assisting DBT interpretation, especially in large-scale studies with low breast cancer prevalence. Finally, we focus on the incoming era of personalized and risk-stratified screening that will first see the application of contrast-enhanced breast imaging to screen women with extremely dense breasts. As the diagnostic advantage of DBT over DM was concentrated in this category, we try to understand if the application of AI to DM in the remaining cohorts of women with heterogeneously dense or non-dense breast could close the gap in diagnostic performance between DM and DBT, thus neutralizing the usefulness of AI application to DBT.

The Systematic Review of Artificial Intelligence Applications in Breast Cancer Diagnosis

Several studies have demonstrated the value of artificial intelligence (AI) applications in breast cancer diagnosis. The systematic review of AI applications in breast cancer diagnosis includes several studies that compare breast cancer diagnosis and AI. However, they lack systematization, and each study appears to be conducted uniquely. The purpose and contributions of this study are to offer elaborative knowledge on the applications of AI in the diagnosis of breast cancer through citation analysis in order to categorize the main area of specialization that attracts the attention of the academic community, as well as thematic issue analysis to identify the species being researched in each category. In this study, a total number of 17,900 studies addressing breast cancer and AI published between 2012 and 2022 were obtained from these databases: IEEE, Embase: Excerpta Medica Database Guide-Ovid, PubMed, Springer, Web of Science, and Google Scholar. We applied inclusion and exclusion criteria to the search; 36 studies were identified. The vast majority of AI applications used classification models for the prediction of breast cancer. Howbeit, accuracy (99%) has the highest number of performance metrics, followed by specificity (98%) and area under the curve (0.95). Additionally, the Convolutional Neural Network (CNN) was the best model of choice in several studies. This study shows that the quantity and caliber of studies that use AI applications in breast cancer diagnosis will continue to rise annually. As a result, AI-based applications are viewed as a supplement to doctors' clinical reasoning, with the ultimate goal of providing quality healthcare that is both affordable and accessible to everyone worldwide.

Beyond fine-tuning: Classifying high resolution mammograms using function-preserving transformations

The task of classifying mammograms is very challenging because the lesion is usually small in the high resolution image. The current state-of-the-art approaches for medical image classification rely on using the de-facto method for convolutional neural networks-fine-tuning. However, there are fundamental differences between natural images and medical images, which based on existing evidence from the literature, limits the overall performance gain when designed with algorithmic approaches. In this paper, we propose to go beyond fine-tuning by introducing a novel framework called MorphHR, in which we highlight a new transfer learning scheme. The idea behind the proposed framework is to integrate function-preserving transformations, for any continuous non-linear activation neurons, to internally regularise the network for improving mammograms classification. The proposed solution offers two major advantages over the existing techniques. Firstly and unlike fine-tuning, the proposed approach allows for modifying not only the last few layers but also several of the first ones on a deep ConvNet. By doing this, we can design the network front to be suitable for learning domain specific features. Secondly, the proposed scheme is scalable to hardware. Therefore, one can fit high resolution images on standard GPU memory. We show that by using high resolution images, one prevents losing relevant information. We demonstrate, through numerical and visual experiments, that the proposed approach yields to a significant improvement in the classification performance over state-of-the-art techniques, and is indeed on a par with radiology experts. Moreover and for generalisation purposes, we show the effectiveness of the proposed learning scheme on another large dataset, the ChestX-ray14, surpassing current state-of-the-art techniques.

Automated Identification of Multiple Findings on Brain MRI for Improving Scan Acquisition and Interpretation Workflows: A Systematic Review

We conducted a systematic review of the current status of machine learning (ML) algorithms’ ability to identify multiple brain diseases, and we evaluated their applicability for improving existing scan acquisition and interpretation workflows. PubMed Medline, Ovid Embase, Scopus, Web of Science, and IEEE Xplore literature databases were searched for relevant studies published between January 2017 and February 2022. The quality of the included studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 tool. The applicability of ML algorithms for successful workflow improvement was qualitatively assessed based on the satisfaction of three clinical requirements. A total of 19 studies were included for qualitative synthesis. The included studies performed classification tasks (n = 12) and segmentation tasks (n = 7). For classification algorithms, the area under the receiver operating characteristic curve (AUC) ranged from 0.765 to 0.997, while accuracy, sensitivity, and specificity ranged from 80% to 100%, 72% to 100%, and 65% to 100%, respectively. For segmentation algorithms, the Dice coefficient ranged from 0.300 to 0.912. No studies satisfied all clinical requirements for successful workflow improvements due to key limitations pertaining to the study’s design, study data, reference standards, and performance reporting. Standardized reporting guidelines tailored for ML in radiology, prospective study designs, and multi-site testing could help alleviate this.

Research on primary Sjögren's syndrome in 2004-2021: a Web of Science-based cross-sectional bibliometric analysis

The extent, range, and nature of available research in the field of primary Sjögren's syndrome (pSS) have not been understood fully. This study aimed to map the literature available on pSS, and identify global hotspots and trends in the research. Papers on pSS published between 2004 and 2021 were searched from Web of Science Core Collection. The quantity and citations of publications, and the research hotspots and trends in the field of pSS were analyzed and presented visually by Microsoft Excel and Citespace software. A total of 3606 papers mainly from 526 institutions in 83 countries/regions were included for analysis. The number of publications presented an overall upward trend in the field of pSS from 2004 to 2021. The USA ranked first in the number of publications (n = 661), followed by China (n = 491), Italy (n = 405), France (n = 351), and Japan (n = 292). Moreover, seven of the top ten countries by the number of publications on pSS were from Europe. The University of Groningen (n = 661), Xavier Mariette (n = 95), and Clinical and Experimental Rheumatology (n = 184) were the most prolific affiliation, author, and journal, respectively. Vitali C (n = 2009) and Arthritis and Rheumatism (n = 3918) held the record for the most cited papers by an author and journal, respectively. At present, the hot keywords in the field of pSS include disease activity, ultrasonography, management, consensus, and data-driven. Lymphoid organization, clinical phenotypes outcome, salivary gland ultrasonography, and Toll-like receptor are the emerging research trends in pSS. Research on pSS is flourishing. Current research of pSS mainly focuses on disease activity, ultrasonography, and management. While, the emerging research trends in pSS are lymphoid organization, clinical phenotypes outcome, salivary gland ultrasonography, and Toll-like receptor.

Artificial Intelligence (AI) for Screening Mammography, From the AI Special Series on AI Applications

Artificial intelligence (AI) applications for screening mammography are being marketed for clinical use in the interpretative domains of lesion detection and diagnosis, triage, and breast density assessment, and in the noninterpretive domains of breast cancer risk assessment, image quality control, image acquisition, and dose reduction. Evidence in support of these nascent applications, particularly for lesion detection and diagnosis, is largely based on multireader studies with cancer-enriched datasets rather than rigorous clinical evaluation aligned with the application's specific intended clinical use. This article reviews commercial AI algorithms for screening mammography that are currently available for clinical practice, their use, and evidence supporting their performance. Clinical implementation considerations, such as workflow integration, governance, and ethical issues, are also described. In addition, the future of AI for screening mammography is discussed, including the development of interpretive and noninterpretive AI applications and strategic priorities for research and development.

Screen-detected and interval breast cancer after concordant and discordant interpretations in a population based screening program using independent double reading

OBJECTIVES

To analyze rates, odds ratios (OR), and characteristics of screen-detected and interval cancers after concordant and discordant initial interpretations and consensus in a population-based screening program.

METHODS

Data were extracted from the Cancer Registry of Norway for 487,118 women who participated in BreastScreen Norway, 2006-2017, with 2 years of follow-up. All mammograms were independently interpreted by two radiologists, using a score from 1 (negative) to 5 (high suspicion of cancer). A score of 2+ by one of the two radiologists was defined as discordant and 2+ by both radiologists as concordant positive. Consensus was performed on all discordant and concordant positive, with decisions of recall for further assessment or dismiss. OR was estimated with logistic regression with 95% confidence interval (CI), and histopathological tumor characteristics were analyzed for screen-detected and interval cancer.

RESULTS

Among screen-detected cancers, 23.0% (697/3024) had discordant scores, while 12.8% (117/911) of the interval cancers were dismissed at index screening. Adjusted OR was 2.4 (95% CI: 1.9-2.9) for interval cancer and 2.8 (95% CI: 2.5-3.2) for subsequent screen-detected cancer for women dismissed at consensus compared to women with concordant negative scores. We found 3.4% (4/117) of the interval cancers diagnosed after being dismissed to be DCIS, compared to 20.3% (12/59) of those with false-positive result after index screening.

CONCLUSION

Twenty-three percent of the screen-detected cancers was scored negative by one of the two radiologists. A higher odds of interval and subsequent screen-detected cancer was observed among women dismissed at consensus compared to concordant negative scores. Our findings indicate a benefit of personalized follow-up.

KEY POINTS

• In this study of 487,118 women participating in a screening program using independent double reading with consensus, 23% screen-detected cancers were detected by only one of the two radiologists. • The adjusted odds ratio for interval cancer was 2.4 (95% confidence interval: 1.9, 2.9) for cases dismissed at consensus using concordant negative interpretations as the reference. • Interval cancers diagnosed after being dismissed at consensus or after concordant negative scores had clinically less favorable prognostic tumor characteristics compared to those diagnosed after false-positive results.