Radiologist-Level Performance by Using Deep Learning for Segmentation of Breast Cancers on MRI Scans

Cross-site Validation of AI Segmentation and Harmonization in Breast MRI

This work aims to perform a cross-site validation of automated segmentation for breast cancers in MRI and to compare the performance to radiologists. A three-dimensional (3D) U-Net was trained to segment cancers in dynamic contrast-enhanced axial MRIs using a large dataset from Site 1 (n = 15,266; 449 malignant and 14,817 benign). Performance was validated on site-specific test data from this and two additional sites, and common publicly available testing data. Four radiologists from each of the three clinical sites provided two-dimensional (2D) segmentations as ground truth. Segmentation performance did not differ between the network and radiologists on the test data from Sites 1 and 2 or the common public data (median Dice score Site 1, network 0.86 vs. radiologist 0.85, n = 114; Site 2, 0.91 vs. 0.91, n = 50; common: 0.93 vs. 0.90). For Site 3, an affine input layer was fine-tuned using segmentation labels, resulting in comparable performance between the network and radiologist (0.88 vs. 0.89, n = 42). Radiologist performance differed on the common test data, and the network numerically outperformed 11 of the 12 radiologists (median Dice: 0.85-0.94, n = 20). In conclusion, a deep network with a novel supervised harmonization technique matches radiologists' performance in MRI tumor segmentation across clinical sites. We make code and weights publicly available to promote reproducible AI in radiology.

AI and Data-Driven Advancements in Industry 4.0

Industrial artificial intelligence is rapidly evolving, driven by an unprecedented explosion of diverse data modalities [...].

Early Detection of Breast Cancer in MRI Using AI

RATIONALE AND OBJECTIVES

To develop and evaluate an AI algorithm that detects breast cancer in MRI scans up to one year before radiologists typically identify it, potentially enhancing early detection in high-risk women.

MATERIALS AND METHODS

A convolutional neural network (CNN) AI model, pre-trained on breast MRI data, was fine-tuned using a retrospective dataset of 3029 MRI scans from 910 patients. These contained 115 cancers that were diagnosed within one year of a negative MRI. The model aimed to identify these cancers, with the goal of predicting cancer development up to one year in advance. The network was fine-tuned and tested with 10-fold cross-validation. Mean age of patients was 52 years (range, 18-88 years), with average follow-up of 4.3 years (range 1-12 years).

RESULTS

The AI detected cancers one year earlier with an area under the ROC curve of 0.72 (0.67-0.76). Retrospective analysis by a radiologist of the top 10% highest risk MRIs as ranked by the AI could have increased early detection by up to 30%. (35/115, CI:22.2-39.7%, 30% sensitivity). A radiologist identified a visual correlate to biopsy-proven cancers in 83 of prior-year MRIs (83/115, CI: 62.1-79.4%). The AI algorithm identified the anatomic region where cancer would be detected in 66 cases (66/115, CI:47.8-66.5%); with both agreeing in 54 cases (54/115, CI:%37.5-56.4%).

CONCLUSION

This novel AI-aided re-evaluation of "benign" breasts shows promise for improving early breast cancer detection with MRI. As datasets grow and image quality improves, this approach is expected to become even more impactful.

Performance of an AI-powered visualization software platform for precision surgery in breast cancer patients

Surgery remains the primary treatment modality in the management of early-stage invasive breast cancer. Artificial intelligence (AI)-powered visualization platforms offer the compelling potential to aid surgeons in evaluating the tumor's location and morphology within the breast and accordingly optimize their surgical approach. We sought to validate an AI platform that employs dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) to render three-dimensional (3D) representations of the tumor and 5 additional chest tissues, offering clear visualizations as well as functionalities for quantifying tumor morphology, tumor-to-landmark structure distances, excision volumes, and approximate surgical margins. This retrospective study assessed the visualization platform's performance on 100 cases with ground-truth labels vetted by 2 breast-specialized radiologists. We assessed features including automatic AI-generated clinical metrics (e.g., tumor dimensions) as well as visualization tools including convex hulls at desired margins around the tumor to help visualize lumpectomy volume. The statistical performance of the platform's automated features was robust and within the range of inter-radiologist variability. These detailed 3D tumor and surrounding multi-tissue depictions offer both qualitative and quantitative comprehension of cancer topology and may aid in formulating an optimal surgical approach for breast cancer treatment. We further establish the framework for broader data integration into the platform to enhance precision cancer care.

Transformative insights: Image-based breast cancer detection and severity assessment through advanced AI techniques

In the realm of image-based breast cancer detection and severity assessment, this study delves into the revolutionary potential of sophisticated artificial intelligence (AI) techniques. By investigating image processing, machine learning (ML), and deep learning (DL), the research illuminates their combined impact on transforming breast cancer diagnosis. This integration offers insights into early identification and precise characterization of cancers. With a foundation in 125 research articles, this article presents a comprehensive overview of the current state of image-based breast cancer detection. Synthesizing the transformative role of AI, including image processing, ML, and DL, the review explores how these technologies collectively reshape the landscape of breast cancer diagnosis and severity assessment. An essential aspect highlighted is the synergy between advanced image processing methods and ML algorithms. This combination facilitates the automated examination of medical images, which is crucial for detecting minute anomalies indicative of breast cancer. The utilization of complex neural networks for feature extraction and pattern recognition in DL models further enhances diagnostic precision. Beyond diagnostic improvements, the abstract underscores the substantial influence of AI-driven methods on breast cancer treatment. The integration of AI not only increases diagnostic precision but also opens avenues for individualized treatment planning, marking a paradigm shift toward personalized medicine in breast cancer care. However, challenges persist, with issues related to data quality and interpretability requiring continued research efforts. Looking forward, the abstract envisions future directions for breast cancer identification and diagnosis, emphasizing the adoption of explainable AI techniques and global collaboration for data sharing. These initiatives promise to propel the field into a new era characterized by enhanced efficiency and precision in breast cancer care.

Progressive Dual Priori Network for Generalized Breast Tumor Segmentation

To promote the generalization ability of breast tumor segmentation models, as well as to improve the segmentation performance for breast tumors with smaller size, low-contrast and irregular shape, we propose a progressive dual priori network (PDPNet) to segment breast tumors from dynamic enhanced magnetic resonance images (DCE-MRI) acquired at different centers. The PDPNet first cropped tumor regions with a coarse-segmentation based localization module, then the breast tumor mask was progressively refined by using the weak semantic priori and cross-scale correlation prior knowledge. To validate the effectiveness of PDPNet, we compared it with several state-of-the-art methods on multi-center datasets. The results showed that, comparing against the suboptimal method, the DSC and HD95 of PDPNet were improved at least by 5.13% and 7.58% respectively on multi-center test sets. In addition, through ablations, we demonstrated that the proposed localization module can decrease the influence of normal tissues and therefore improve the generalization ability of the model. The weak semantic priors allow focusing on tumor regions to avoid missing small tumors and low-contrast tumors. The cross-scale correlation priors are beneficial for promoting the shape-aware ability for irregular tumors. Thus integrating them in a unified framework improved the multi-center breast tumor segmentation performance.

3D Breast Cancer Segmentation in DCE-MRI Using Deep Learning With Weak Annotation

BACKGROUND

Deep learning models require large-scale training to perform confidently, but obtaining annotated datasets in medical imaging is challenging. Weak annotation has emerged as a way to save time and effort.

PURPOSE

To develop a deep learning model for 3D breast cancer segmentation in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) using weak annotation with reliable performance.

STUDY TYPE

Retrospective.

POPULATION

Seven hundred and thirty-six women with breast cancer from a single institution, divided into the development (N = 544) and test dataset (N = 192).

FIELD STRENGTH/SEQUENCE

3.0-T, 3D fat-saturated gradient-echo axial T1-weighted flash 3D volumetric interpolated brain examination (VIBE) sequences.

ASSESSMENT

Two radiologists performed a weak annotation of the ground truth using bounding boxes. Based on this, the ground truth annotation was completed through autonomic and manual correction. The deep learning model using 3D U-Net transformer (UNETR) was trained with this annotated dataset. The segmentation results of the test set were analyzed by quantitative and qualitative methods, and the regions were divided into whole breast and region of interest (ROI) within the bounding box.

STATISTICAL TESTS

As a quantitative method, we used the Dice similarity coefficient to evaluate the segmentation result. The volume correlation with the ground truth was evaluated with the Spearman correlation coefficient. Qualitatively, three readers independently evaluated the visual score in four scales. A P-value <0.05 was considered statistically significant.

RESULTS

The deep learning model we developed achieved a median Dice similarity score of 0.75 and 0.89 for the whole breast and ROI, respectively. The volume correlation coefficient with respect to the ground truth volume was 0.82 and 0.86 for the whole breast and ROI, respectively. The mean visual score, as evaluated by three readers, was 3.4.

DATA CONCLUSION

The proposed deep learning model with weak annotation may show good performance for 3D segmentations of breast cancer using DCE-MRI.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY: Stage 2.

Non-inferiority of deep learning ischemic stroke segmentation on non-contrast CT within 16-hours compared to expert neuroradiologists

We determined if a convolutional neural network (CNN) deep learning model can accurately segment acute ischemic changes on non-contrast CT compared to neuroradiologists. Non-contrast CT (NCCT) examinations from 232 acute ischemic stroke patients who were enrolled in the DEFUSE 3 trial were included in this study. Three experienced neuroradiologists independently segmented hypodensity that reflected the ischemic core on each scan. The neuroradiologist with the most experience (expert A) served as the ground truth for deep learning model training. Two additional neuroradiologists' (experts B and C) segmentations were used for data testing. The 232 studies were randomly split into training and test sets. The training set was further randomly divided into 5 folds with training and validation sets. A 3-dimensional CNN architecture was trained and optimized to predict the segmentations of expert A from NCCT. The performance of the model was assessed using a set of volume, overlap, and distance metrics using non-inferiority thresholds of 20%, 3 ml, and 3 mm, respectively. The optimized model trained on expert A was compared to test experts B and C. We used a one-sided Wilcoxon signed-rank test to test for the non-inferiority of the model-expert compared to the inter-expert agreement. The final model performance for the ischemic core segmentation task reached a performance of 0.46 ± 0.09 Surface Dice at Tolerance 5mm and 0.47 ± 0.13 Dice when trained on expert A. Compared to the two test neuroradiologists the model-expert agreement was non-inferior to the inter-expert agreement, [Formula: see text]. The before, CNN accurately delineates the hypodense ischemic core on NCCT in acute ischemic stroke patients with an accuracy comparable to neuroradiologists.

Fibroglandular tissue segmentation in breast MRI using vision transformers: a multi-institutional evaluation

Accurate and automatic segmentation of fibroglandular tissue in breast MRI screening is essential for the quantification of breast density and background parenchymal enhancement. In this retrospective study, we developed and evaluated a transformer-based neural network for breast segmentation (TraBS) in multi-institutional MRI data, and compared its performance to the well established convolutional neural network nnUNet. TraBS and nnUNet were trained and tested on 200 internal and 40 external breast MRI examinations using manual segmentations generated by experienced human readers. Segmentation performance was assessed in terms of the Dice score and the average symmetric surface distance. The Dice score for nnUNet was lower than for TraBS on the internal testset (0.909 ± 0.069 versus 0.916 ± 0.067, P < 0.001) and on the external testset (0.824 ± 0.144 versus 0.864 ± 0.081, P = 0.004). Moreover, the average symmetric surface distance was higher (= worse) for nnUNet than for TraBS on the internal (0.657 ± 2.856 versus 0.548 ± 2.195, P = 0.001) and on the external testset (0.727 ± 0.620 versus 0.584 ± 0.413, P = 0.03). Our study demonstrates that transformer-based networks improve the quality of fibroglandular tissue segmentation in breast MRI compared to convolutional-based models like nnUNet. These findings might help to enhance the accuracy of breast density and parenchymal enhancement quantification in breast MRI screening.

Machine learning-ready remote sensing data for Maya archaeology

In our study, we set out to collect a multimodal annotated dataset for remote sensing of Maya archaeology, that is suitable for deep learning. The dataset covers the area around Chactún, one of the largest ancient Maya urban centres in the central Yucatán Peninsula. The dataset includes five types of data records: raster visualisations and canopy height model from airborne laser scanning (ALS) data, Sentinel-1 and Sentinel-2 satellite data, and manual data annotations. The manual annotations (used as binary masks) represent three different types of ancient Maya structures (class labels: buildings, platforms, and aguadas - artificial reservoirs) within the study area, their exact locations, and boundaries. The dataset is ready for use with machine learning, including convolutional neural networks (CNNs) for object recognition, object localization (detection), and semantic segmentation. We would like to provide this dataset to help more research teams develop their own computer vision models for investigations of Maya archaeology or improve existing ones.

Embryo selection through artificial intelligence versus embryologists: a systematic review

STUDY QUESTION

What is the present performance of artificial intelligence (AI) decision support during embryo selection compared to the standard embryo selection by embryologists?

SUMMARY ANSWER

AI consistently outperformed the clinical teams in all the studies focused on embryo morphology and clinical outcome prediction during embryo selection assessment.

WHAT IS KNOWN ALREADY

The ART success rate is ∼30%, with a worrying trend of increasing female age correlating with considerably worse results. As such, there have been ongoing efforts to address this low success rate through the development of new technologies. With the advent of AI, there is potential for machine learning to be applied in such a manner that areas limited by human subjectivity, such as embryo selection, can be enhanced through increased objectivity. Given the potential of AI to improve IVF success rates, it remains crucial to review the performance between AI and embryologists during embryo selection.

STUDY DESIGN SIZE DURATION

The search was done across PubMed, EMBASE, Ovid Medline, and IEEE Xplore from 1 June 2005 up to and including 7 January 2022. Included articles were also restricted to those written in English. Search terms utilized across all databases for the study were: ('Artificial intelligence' OR 'Machine Learning' OR 'Deep learning' OR 'Neural network') AND ('IVF' OR 'in vitro fertili*' OR 'assisted reproductive techn*' OR 'embryo'), where the character '*' refers the search engine to include any auto completion of the search term.

PARTICIPANTS/MATERIALS SETTING METHODS

A literature search was conducted for literature relating to AI applications to IVF. Primary outcomes of interest were accuracy, sensitivity, and specificity of the embryo morphology grade assessments and the likelihood of clinical outcomes, such as clinical pregnancy after IVF treatments. Risk of bias was assessed using the Modified Down and Black Checklist.

MAIN RESULTS AND THE ROLE OF CHANCE

Twenty articles were included in this review. There was no specific embryo assessment day across the studies-Day 1 until Day 5/6 of embryo development was investigated. The types of input for training AI algorithms were images and time-lapse (10/20), clinical information (6/20), and both images and clinical information (4/20). Each AI model demonstrated promise when compared to an embryologist's visual assessment. On average, the models predicted the likelihood of successful clinical pregnancy with greater accuracy than clinical embryologists, signifying greater reliability when compared to human prediction. The AI models performed at a median accuracy of 75.5% (range 59-94%) on predicting embryo morphology grade. The correct prediction (Ground Truth) was defined through the use of embryo images according to post embryologists' assessment following local respective guidelines. Using blind test datasets, the embryologists' accuracy prediction was 65.4% (range 47-75%) with the same ground truth provided by the original local respective assessment. Similarly, AI models had a median accuracy of 77.8% (range 68-90%) in predicting clinical pregnancy through the use of patient clinical treatment information compared to 64% (range 58-76%) when performed by embryologists. When both images/time-lapse and clinical information inputs were combined, the median accuracy by the AI models was higher at 81.5% (range 67-98%), while clinical embryologists had a median accuracy of 51% (range 43-59%).

LIMITATIONS REASONS FOR CAUTION

The findings of this review are based on studies that have not been prospectively evaluated in a clinical setting. Additionally, a fair comparison of all the studies were deemed unfeasible owing to the heterogeneity of the studies, development of the AI models, database employed and the study design and quality.

WIDER IMPLICATIONS OF THE FINDINGS

AI provides considerable promise to the IVF field and embryo selection. However, there needs to be a shift in developers' perception of the clinical outcome from successful implantation towards ongoing pregnancy or live birth. Additionally, existing models focus on locally generated databases and many lack external validation.

STUDY FUNDING/COMPETING INTERESTS

This study was funded by Monash Data Future Institute. All authors have no conflicts of interest to declare.

REGISTRATION NUMBER

CRD42021256333.

Clinical applications of deep learning in breast MRI

Deep learning (DL) is one of the most powerful data-driven machine-learning techniques in artificial intelligence (AI). It can automatically learn from raw data without manual feature selection. DL models have led to remarkable advances in data extraction and analysis for medical imaging. Magnetic resonance imaging (MRI) has proven useful in delineating the characteristics and extent of breast lesions and tumors. This review summarizes the current state-of-the-art applications of DL models in breast MRI. Many recent DL models were examined in this field, along with several advanced learning approaches and methods for data normalization and breast and lesion segmentation. For clinical applications, DL-based breast MRI models were proven useful in five aspects: diagnosis of breast cancer, classification of molecular types, classification of histopathological types, prediction of neoadjuvant chemotherapy response, and prediction of lymph node metastasis. For subsequent studies, further improvement in data acquisition and preprocessing is necessary, additional DL techniques in breast MRI should be investigated, and wider clinical applications need to be explored.

Visual ensemble selection of deep convolutional neural networks for 3D segmentation of breast tumors on dynamic contrast enhanced MRI

OBJECTIVES

To develop a visual ensemble selection of deep convolutional neural networks (CNN) for 3D segmentation of breast tumors using T1-weighted dynamic contrast-enhanced (T1-DCE) MRI.

METHODS

Multi-center 3D T1-DCE MRI (n = 141) were acquired for a cohort of patients diagnosed with locally advanced or aggressive breast cancer. Tumor lesions of 111 scans were equally divided between two radiologists and segmented for training. The additional 30 scans were segmented independently by both radiologists for testing. Three 3D U-Net models were trained using either post-contrast images or a combination of post-contrast and subtraction images fused at either the image or the feature level. Segmentation accuracy was evaluated quantitatively using the Dice similarity coefficient (DSC) and the Hausdorff distance (HD95) and scored qualitatively by a radiologist as excellent, useful, helpful, or unacceptable. Based on this score, a visual ensemble approach selecting the best segmentation among these three models was proposed.

RESULTS

The mean and standard deviation of DSC and HD95 between the two radiologists were equal to 77.8 ± 10.0% and 5.2 ± 5.9 mm. Using the visual ensemble selection, a DSC and HD95 equal to 78.1 ± 16.2% and 14.1 ± 40.8 mm was reached. The qualitative assessment was excellent (resp. excellent or useful) in 50% (resp. 77%).

CONCLUSION

Using subtraction images in addition to post-contrast images provided complementary information for 3D segmentation of breast lesions by CNN. A visual ensemble selection allowing the radiologist to select the most optimal segmentation obtained by the three 3D U-Net models achieved comparable results to inter-radiologist agreement, yielding 77% segmented volumes considered excellent or useful.

KEY POINTS

• Deep convolutional neural networks were developed using T1-weighted post-contrast and subtraction MRI to perform automated 3D segmentation of breast tumors. • A visual ensemble selection allowing the radiologist to choose the best segmentation among the three 3D U-Net models outperformed each of the three models. • The visual ensemble selection provided clinically useful segmentations in 77% of cases, potentially allowing for a valuable reduction of the manual 3D segmentation workload for the radiologist and greatly facilitating quantitative studies on non-invasive biomarker in breast MRI.

Artificial intelligence and neural networks in radiology – Basics that all radiology residents should know

The area of Artificial Intelligence is developing at a high rate. In the medical field, an extreme amount of data is created every day. As the images and the reports are quantifiable, the field of radiology aspires to deliver better, more efficient clinical care. Artificial intelligence (AI) means the simulation of human intelligence by a system or machine. It has been developed to enable machines to “think”, which means to be able to learn, reason, predict, categorize, and solve problems concerning high amounts of data and make decisions in a more effective manner than before. Different AI methods can help radiologists with pre-screening images and identifying features. In this review, we summarize the basic concepts which are needed to understand AI. As the AI methods are expected to exceed the threshold for clinical usefulness soon, in the near future it will be inevitable to use AI in medicine.

Standardised lesion segmentation for imaging biomarker quantitation: a consensus recommendation from ESR and EORTC

BACKGROUND

Lesion/tissue segmentation on digital medical images enables biomarker extraction, image-guided therapy delivery, treatment response measurement, and training/validation for developing artificial intelligence algorithms and workflows. To ensure data reproducibility, criteria for standardised segmentation are critical but currently unavailable.

METHODS

A modified Delphi process initiated by the European Imaging Biomarker Alliance (EIBALL) of the European Society of Radiology (ESR) and the European Organisation for Research and Treatment of Cancer (EORTC) Imaging Group was undertaken. Three multidisciplinary task forces addressed modality and image acquisition, segmentation methodology itself, and standards and logistics. Devised survey questions were fed via a facilitator to expert participants. The 58 respondents to Round 1 were invited to participate in Rounds 2-4. Subsequent rounds were informed by responses of previous rounds.

RESULTS/CONCLUSIONS

Items with ≥ 75% consensus are considered a recommendation. These include system performance certification, thresholds for image signal-to-noise, contrast-to-noise and tumour-to-background ratios, spatial resolution, and artefact levels. Direct, iterative, and machine or deep learning reconstruction methods, use of a mixture of CE marked and verified research tools were agreed and use of specified reference standards and validation processes considered essential. Operator training and refreshment were considered mandatory for clinical trials and clinical research. Items with a 60-74% agreement require reporting (site-specific accreditation for clinical research, minimal pixel number within lesion segmented, use of post-reconstruction algorithms, operator training refreshment for clinical practice). Items with ≤ 60% agreement are outside current recommendations for segmentation (frequency of system performance tests, use of only CE-marked tools, board certification of operators, frequency of operator refresher training). Recommendations by anatomical area are also specified.