Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancers

Landscape of lncRNAs expressed in Mexican patients with triple‑negative breast cancer

Long non‑coding RNAs (lncRNAs) are key regulators of gene expression, that can regulate a range of carcinogenic processes. Moreover, they exhibit stability in biological fluids, with some displaying tissue specificity. As their expression depends on specific conditions or is linked to the regulation of particular signaling pathways, lncRNAs are promising candidates for providing insights into the likely progression of the disease. This allows for the stratification of patients based on their risk of progression, making them potential prognostic biomarkers in various types of cancer. In addition, the tissue‑specific expression profile of lncRNAs renders them ideal candidates for detection, prognosis and monitoring of cancer progression. The present study aims to provide an overview of differentially expressed lncRNAs in Mexican patients with triple‑negative breast cancer (TNBC), a subtype of breast cancer. The aim was to identify potential prognostic biomarkers that can be applied to improve the clinical management of Mexican patients with TNBC. Human Transcriptome Array 2.0 microarrays were used to analyze the transcriptome of TNBC and luminal tumors, which are reported to have a good prognosis amongst aggressive tumor types. Subsequently, results from these microarrays were validated in a cohort from The Cancer Genome Atlas, an independent cohort of Mexican patients and in breast cancer cell lines (MCF7, ZR75, T47D, MDA‑MB‑231, MDA‑MB‑468 and BT20). A total of 746 differentially expressed transcripts were identified, including 102 lncRNAs in TNBC compared with luminal tumors. Among the lncRNAs with the most significant changes in expression levels, SOX9‑AS was highly expressed in TNBC, whereas the expression of Lnc‑peroxidasin‑3:1 (Lnc‑PXDN‑3:1), Lnc‑RNA Synapse Defective Rho GTPase Homolog (Lnc‑SYDE) and long intergenic non‑coding RNA (LINC)01087 were decreased. In addition, the low expression of lncRNA LINC01087, LINC02568, ACO22196, and lncRNA eosinophil granule ontogeny transcript (Lnc‑EGOT) was associated with poor overall survival (OS). Further analysis revealed that the high expression levels of Lnc‑PXDN‑3:1, Lnc RNA fibrous sheath interacting protein 1‑6:3 and (LINC)00182 were associated with reduced survival in patients with the luminal subtype of breast cancer. Similarly, low expression levels of lncRNAs such as GATA binding protein 3‑1 (Lnc‑GATA‑3‑1), LINC01087, and BX679671.1 in luminal subtypes of breast cancer, as well as LINC00504 and LncRNA rho guanine nucleotide exchange factor 38 intronic transcript 1 (Lnc‑ARHGEF38‑IT1) in basal subtypes have been linked to poorer survival. The interactions and functions of LINC01087 were then investigated, revealing the interaction of LINC01087 with RNAs and transcription factors, highlighting their potential involvement in the estrogen receptor pathway. The present study provided a detailed analysis of the expression of lncRNAs in TNBC, which highlights the role of lncRNAs as a biomarker in the survival outcomes of patients with breast cancer to improve the understanding of transcriptional regulation in TNBC.

Development and Validation of Multiparametric MRI-based Interpretable Deep Learning Radiomics Fusion Model for Predicting Lymph Node Metastasis and Prognosis in Rectal Cancer: A Two-center Study

RATIONALE AND OBJECTIVES

To develop interpretable machine learning models that utilize deep learning (DL) and radiomics based on multiparametric Magnetic resonance imaging (MRI) to predict preoperative lymph node (LN) metastasis in rectal cancer.

MATERIALS AND METHODS

This retrospective study involved 286 cancer patients confirmed by histopathology from center 1 (Training set) and 66 patients from center 2 (External test set). Radiomics features were extracted from T2-weighted imaging (T2WI) and diffusion-weighted imaging (DWI) sequences, whereas DL features were obtained using four models: MobileNet-V3-large, Inception-V3, ResNet50, and VGG16. These DL radiomics (DLR) features were then combined to construct a machine learning model. The Shapley additive interpretation (SHAP) tool was utilized to investigate the interpretability of the model. We evaluated and compared the diagnostic performance of senior and junior radiologists, with and without the aid of the optimal DLR model. Kaplan-Meier survival curve was used to analyze the prognosis of patients.

RESULTS

The DLR model outperforms individual DL models and the radiomics model. The MobileNet-V3-large combination radiomics signature demonstrated the best performance, achieving an AUC of 0.878 on the Training set and 0.752 on the External test set. Compared to the traditional radiomics model, the AUC for the Training set increased by 0.094 and by 0.051 for the External test set. This model facilitated improved diagnostic performance among both junior and senior radiologists. Specifically, the AUC values for junior and senior radiologists increased by 0.162 and 0.232, respectively, on the Training set; and by 0.096 and 0.113, respectively, on the External test set. The DLR model demonstrated strong performance in risk stratification for disease-free survival.

CONCLUSION

The DLR model developed from multiparametric MRI can effectively distinguish cancer LN metastasis and enhance radiologists' diagnostic performance.

Deep learning radiopathomics predicts targeted therapy sensitivity in EGFR-mutant lung adenocarcinoma

BACKGROUND

Ttyrosine kinase inhibitors (TKIs) represent the standard first-line treatment for patients with epidermal growth factor receptor (EGFR)-mutant lung adenocarcinoma. However, not all patients with EGFR mutations respond to TKIs. This study aims to develop a deep learning radiological-pathological-clinical (DLRPC) model that integrates computed tomography (CT) images, hematoxylin and eosin (H&E)-stained aspiration biopsy samples, and clinical data to predict the response in EGFR-mutant lung adenocarcinoma patients undergoing TKIs treatment.

METHODS

We retrospectively analyzed data from 214 lung adenocarcinoma patients who received TKIs treatment from two medical centers between September 2013 and June 2023. The DLRPC model leverages paired CT, pathological images and clinical data, incorporating a clinical-based attention mask to further explore the cross-modality associations. To evaluate its diagnostic performance, we compared the DLRPC model against single-modality models and a decision level fusion model based on Dempster-Shafer theory. Model performances metrics, including area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), were used for evaluation. The Delong test assessed statistically significantly differences in AUC among models.

RESULTS

The DLRPC model demonstrated strong performance, achieving an AUC value of 0.8424. It outperformed the single-modality models (AUC = 0.6894, 0.7753, 0.8052 for CT model, pathology model and clinical model, respectively. P < 0.05). Additionally, the DLRPC model surpassed the decision level fusion model (AUC = 0.8132, P < 0.05).

CONCLUSION

The DLRPC model effectively predicts the response of EGFR-mutant lung adenocarcinoma patients to TKIs, providing a promising tool for personalized treatment decisions in lung cancer management.

Intratumoral and peritumoral radiomics signature based on DCE-MRI can distinguish between luminal and non-luminal breast cancer molecular subtypes

Distinguishing the luminal subtypes of breast cancer (BC) remaining challenging. Thus, the aim of this study was to investigate the feasibility of radiomic signature using intratumoral and peritumoral features obtained from dynamic contrast-enhanced MRI (DCE-MRI) in preoperatively discriminating the luminal from non-luminal type in patients with BC. A total of 305 patients with pathologically confirmed BC from three hospitals were retrospectively enrolled. The LASSO method was then used for selecting features, and the radiomic score (radscore) for each patient was calculated. Based on the radscore, Radiomic signature of intratumoral, peritumoral, and combined intratumoral and peritumoral were established, respectively. The performances of the radiomic signatures were validated with receiver operator characteristic (ROC) curve and decision curve analysis. For predicting molecular subtypes, the AUC for intratumoral radiomic signature was 0.817, 0.838, and 0.883 in the training set, internal validation set, and external validation set, respectively. AUC for the peritumoral radiomic signature was 0.863, 0.895, and 0.889 in the training set, internal validation set, and external validation set, respectively. The AUC for combined intratumoral and peritumoral radiomic signature was 0.956, 0.945, and 0.896 in the training set, internal validation set, and external validation set, respectively. Additional contributing value of combined intratumoral and peritumoral radiomic signature to the intratumoral radiomic signature was statistically significant [NRI, 0.300 (95% CI: 0.117-0.482), P = 0.001 in internal validation set; NRI, 0.224 (95% CI: 0.038-0.410), P = 0.018 in external validation set]. These results indicated that the radiomic signature combining intratumoral and peritumoral features showed good performance in predicting the luminal type of breast cancer.

Deep Learning Algorithm‑Based MRI Radiomics and Pathomics for Predicting Microsatellite Instability Status in Rectal Cancer: A Multicenter Study

RATIONALE AND OBJECTIVES

To develop and validate multimodal deep-learning models based on clinical variables, multiparametric MRI (mp-MRI) and hematoxylin and eosin (HE) stained pathology slides for predicting microsatellite instability (MSI) status in rectal cancer patients.

MATERIALS AND METHODS

A total of 467 surgically confirmed rectal cancer patients from three centers were included in this study. Patients from center 1 were randomly divided into a training set (242 patients) and an internal validation (invad) set (105 patients) in a 7:3 ratio. Patients from centers 2 and 3 (120 patients) were included in an external validation (exvad) set. HE and immunohistochemistry (IHC) staining were analyzed, and MSI status was confirmed by IHC staining. Independent predictive factors were identified through univariate and multivariate analyses based on clinical evaluations and were used to construct a clinical model. Deep learning with ResNet-101 was applied to preoperative MRI (T2WI, DWI, and contrast-enhanced T1WI sequences) and postoperative HE-stained images to calculate deep-learning radiomics score (DLRS) and deep-learning pathomics score (DLPS), respectively, and to DLRS and DLPS models. Receiver operating characteristic (ROC) curves were plotted, and the area under the curve (AUC) was used to evaluate and compare the predictive performance of each model.

RESULTS

Among all rectal cancer patients, 82 (17.6%) had MSI. Long diameter (LD) and pathological T stage (pT) were identified as independent predictors and were used to construct the clinical model. After undergoing deep learning and feature selection, a final set of 30 radiomics features and 30 pathomics features were selected to construct the DLRS and DLPS models. A nomogram combining the clinical model, DLRS, and DLPS was created through weighted linear combination. The AUC values of the clinical model for predicting MSI were 0.714, 0.639, and 0.697 in the training, invad, and exvad sets, respectively. The AUCs of DLPS and DLRS ranged from 0.896 to 0.961 across the training, invad, and exvad sets. The nomogram achieved AUC values of 0.987, 0.987, and 0.974, with sensitivities of 1.0, 0.963, and 1.0 and specificities of 0.919, 0.949, and 0.867 in the training, invad, and exvad sets, respectively. The nomogram outperformed the other three models in all sets, with DeLong test results indicating superior predictive performance in the training set.

CONCLUSION

The nomogram, incorporating clinical data, mp-MRI, and HE staining, effectively reflects tumor heterogeneity by integrating multimodal data. This model demonstrates high predictive accuracy and generalizability in predicting MSI status in rectal cancer patients.

Radiomics of Multimodal Ultrasound for Early Prediction of Pathologic Complete Response to Neoadjuvant Chemotherapy in Breast Cancer

RATIONALE AND OBJECTIVES

To construct and validate a clinical-radiomics model based on radiomics features extracted from two-stage multimodal ultrasound and clinicopathologic information for early predicting pathologic complete response (pCR) to neoadjuvant chemotherapy (NAC) in breast cancer patients treated with NAC.

MATERIALS AND METHODS

Consecutive women with biopsy-proven breast cancer undergoing multimodal US pretreatment and after two cycles of NAC and followed by surgery between January 2014 and November 2023 were retrospectively collected for clinical-radiomics model construction (n = 274) and retrospective test (n = 134). The predictive performance of it was further tested in a subsequent prospective internal test set recruited between January 2024 to July 2024 (n = 76). Finally, a total of 484 patients were enrolled. The clinical-radiomics model predictive performance was compared with radiomics model, clinical model and radiologists' visual assessment by area under the receiver operating characteristic curve (AUC) analysis and DeLong test.

RESULTS

The proposed clinical-radiomics model obtained the AUC values of 0.92 (95%CI: 0.88, 0.94) and 0.85 (95%CI: 0.79, 0.89) in retrospective and prospective test sets, respectively, which were significantly higher than that those of the radiomics model (AUCs: 0.75-0.85), clinical model (AUCs: 0.68-0.72) and radiologists' visual assessments (AUCs:0.59-0.68) (all p < 0.05). In addition, the predictive efficacy of the radiologists was improved under the assistance of the clinical-radiomics model significantly.

CONCLUSION

The clinical-radiomics model developed in this study, which integrated clinicopathologic information and two-stage multimodal ultrasound features, was able to early predict pCR to NAC in breast cancer patients with favorable predictive effectiveness.

Roles of artificial intelligence and high frame-rate contrast-enhanced ultrasound in the differential diagnosis of Breast Imaging Reporting and Data System 4 breast nodules

Background

Breast cancer prevalence and mortality are rising, emphasizing the need for early, accurate diagnosis. Contrast-enhanced ultrasound (CEUS) and artificial intelligence (AI) show promise in distinguishing benign from malignant breast nodules. We compared the diagnostic values of AI, high frame-rate CEUS (HiFR-CEUS), and their combination in Breast Imaging Reporting and Data System (BI-RADS) 4 nodules, using pathology as the gold standard.

Methods

Patients with BI-RADS 4 breast nodules who were hospitalized at the Department of Thyroid and Breast Surgery, Taizhou People's Hospital from December 2021 to June 2022 were enrolled in the study.80 female patients (80 lesions) underwent preoperative AI and/or HiFR-CEUS. We assessed diagnostic outcomes of AI, HiFR-CEUS, and their combination, calculating sensitivity (SE), specificity (SP), accuracy (ACC), positive/negative predictive values (PPV/NPV). Reliability was compared using Kappa statistics, and AI-HiFR-CEUS correlation was analyzed with Pearson's test. Receiver operating characteristic curves were plotted to compare diagnostic accuracy of AI, HiFR-CEUS, and their combined approach in differentiating BI-RADS 4 lesions.

Results

Of the 80 lesions, 18 were pathologically confirmed to be benign, while the remaining 62 were malignant. The SE, SP, ACC, PPV, and NPV were 75.81%, 94.44%, 80.00%, 97.92%, and 53.13% in the AI group, 74.20%, 94.44%, 78.75%, 97.91%, and 51.51% in the HiFR-CEUS group, and 98.39%, 88.89%, 96.25%, 96.83%, and 94.12% in the combination group, respectively. Thus, the SE, ACC, and NPV of the combination group were significantly higher than those of the AI and HiFR-CEUS groups, and the SP of the combination group was lower (all P<0.05); however, no significant difference was found between the groups in terms of the PPV (P>0.05). No statistically significant difference was observed in the diagnostic performance of the AI and HiFR-CEUS groups (all P>0.05). The AI and HiFR-CEUS groups had moderate agreement with the "gold standard" (Kappa =0.551, Kappa =0.530, respectively), while the combination group had high agreement (Kappa =0.890). AI was positively correlated with HiFR-CEUS (r=0.249, P<0.05). The area under the curves (AUCs) of AI, HiFR-CEUS, and both in combination were 0.851±0.039, 0.815±0.047, and 0.936±0.039, respectively. Thus, the AUC of the combination group was significantly higher than those of the AI and HiFR-CEUS groups (Z1=2.207, Z2=2.477, respectively, both P<0.05). The AI group had a higher AUC than the HiFR-CEUS group, but the difference was not statistically significant (Z3=0.554, P>0.05).

Conclusions

Compared with AI alone or HiFR-CEUS alone, the combined use of these two methods had higher diagnostic performance in distinguishing between benign and malignant BI-RADS 4 breast nodules. Thus, our combination method could further improve the diagnostic accuracy and guide clinical decision making.

A novel deep learning radiopathomics model for predicting carcinogenesis promotor cyclooxygenase-2 expression in common bile duct in children with pancreaticobiliary maljunction: a multicenter study

OBJECTIVES

To develop and validate a deep learning radiopathomics model (DLRPM) integrating radiological and pathological imaging data to predict biliary cyclooxygenase-2 (COX-2) expression in children with pancreaticobiliary maljunction (PBM), and to compare its performance with single-modality radiomics, deep learning radiomics (DLR), and pathomics models.

METHODS

This retrospective study included 219 PBM patients, divided into a training set (n = 104; median age, 2.8 years, 75.0% females) and internal test set (n = 71; median age, 2.2 years, 83.1% females) from center I, and an external test set (n = 44; median age, 3.4 years, 65.9% females) from center II. Biliary COX-2 expression was detected using immunohistochemistry. Radiomics, DLR, and pathomics features were extracted from portal venous-phase CT images and H&E-stained histopathological slides, respectively, to build individual single-modality models. These were then integrated to develop the DLRPM, combining three predictive signatures. Model performance was evaluated using AUC, net reclassification index (NRI, for assessing improvement in correct classification) and integrated discrimination improvement (IDI).

RESULTS

The DLRPM demonstrated the highest performance, with AUCs of 0.851 (95% CI, 0.759-0.942) in internal test set and 0.841 (95% CI, 0.721-0.960) in external test set. In comparison, AUCs for the radiomics, DLR, and pathomics models were 0.532-0.602, 0.658-0.660, and 0.787-0.805, respectively. The DLRPM significantly outperformed three single-modality models, as demonstrated by the NRI and IDI tests (all p < 0.05).

CONCLUSION

The multimodal DLRPM could accurately and robustly predict COX-2 expression, facilitating risk stratification and personalized postoperative management in PBM. However, prospective multicenter studies with larger cohorts are needed to further validate its generalizability.

CRITICAL RELEVANCE STATEMENT

Our proposed deep learning radiopathomics model, integrating CT and histopathological images, provides a novel and cost-effective approach to accurately predict biliary cyclooxygenase-2 expression, potentially advancing individualized risk stratification and improving long-term outcomes for pediatric patients with pancreaticobiliary maljunction.

KEY POINTS

Predicting biliary COX-2 expression in pancreaticobiliary maljunction (PBM) is critical but challenging. A deep learning radiopathomics model achieved high predictive accuracy for COX-2. The model supports patient stratification and personalized postoperative management in PBM.

Synergic value of 3D CT-derived body composition and triglyceride glucose body mass for survival prognostic modeling in unresectable pancreatic cancer

Background

Personalized and accurate survival risk prognostication remains a significant challenge in advanced pancreatic ductal adenocarcinoma (PDAC), despite extensive research on prognostic and predictive markers. Patients with PDAC are prone to muscle loss, fat consumption, and malnutrition, which is associated with inferior outcomes. This study investigated the use of three-dimensional (3D) anthropometric parameters derived from computed tomography (CT) scans and triglyceride glucose-body mass index (TyG-BMI) in relation to overall survival (OS) outcomes in advanced PDAC patients. Additionally, a predictive model for 1 year OS was developed based on body components and hematological indicators.

Methods

A retrospective analysis was conducted on 303 patients with locally advanced PDAC or synchronous metastases undergoing first-line chemotherapy, all of whom had undergone pretreatment abdomen-pelvis CT scans. Automatic 3D measurements of subcutaneous and visceral fat volume, skeletal muscle volume, and skeletal muscle density (SMD) were assessed at the L3 vertebral level by an artificial intelligence assisted diagnosis system (HY Medical). Various indicators including TyG-BMI, nutritional indicators [geriatric nutritional risk index (GNRI) and prealbumin], and inflammation indicators [(C-reactive protein (CRP) and neutrophil to lymphocyte ratio (NLR)] were also recorded. All patients underwent follow-up for at least 1 year and a dynamic nomogram for personalized survival prediction was constructed.

Results

We included 211 advanced PDAC patients [mean (standard deviation) age, 63.4 ± 11.2 years; 89 women (42.2) %)]. Factors such as low skeletal muscle index (SMI) (P = 0.011), high visceral to subcutaneous adipose tissue area ratio (VSR) (P < 0.001), high visceral fat index (VFI) (P < 0.001), low TyG-BMI (P = 0.004), and low prealbumin (P = 0.001) were identified as independent risk factors associated with 1 year OS. The area under the curve of the established dynamic nomogram was 0.846 and the calibration curve showed good consistency. High-risk patients (> 211.9 points calculated using the nomogram) had significantly reduced survival rates.

Conclusion

In this study, the proposed nomogram model (with web-based tool) enabled individualized prognostication of OS and could help to guide risk-adapted nutritional treatment for patients with unresectable PDAC or synchronous metastases.

Early and noninvasive prediction of response to neoadjuvant therapy for breast cancer via longitudinal ultrasound and MR deep learning: A multicentre study

RATIONALE AND OBJECTIVES

The early prediction of response to neoadjuvant chemotherapy (NAC) will aid in the development of personalized treatments for patients with breast cancer. This study investigated the value of longitudinal multimodal deep learning (DL) based on breast MR and ultrasound (US) in predicting pathological complete response (pCR) after NAC.

MATERIALS AND METHODS

We retrospectively reviewed the pre-NAC and post-2nd-NAC MR and/or US images of 448 patients enrolled from three centers and extracted DL features from the largest section of the breast tumour using ResNet50. T test, Pearson correlation analysis and least absolute shrinkage and selection operator regression were used to select the most significant DL features for the pre-NAC and post-2nd-NAC MR and US DL models. The stacking model integrates different single-modality DL models and meaningful clinical data. The diagnostic performance of the models was evaluated.

RESULTS

In all the patients, the pCR rate was 36.65%. There was no significant difference in diagnostic performance between the different single-modality DL models (DeLong test, p > 0.05). The stacking model integrating the above four DL models with HER2 status yielded areas under the curves of 0.951-0.979, accuracies of 91.55%-92.65%, sensitivities of 90.63%-93.94%, and specificities of 89.47%-94.44% in the cohorts.

CONCLUSION

Longitudinal multimodal DL can be useful in predicting pCR. The stacking model can be used as a new tool for the early noninvasive prediction of the response to NAC, as evidenced by its excellent performance, and therefore aid the development of personalized treatment strategies for patients with breast cancer.

Ultrasound-based artificial intelligence model for prediction of Ki-67 proliferation index in soft tissue tumors

RATIONALE AND OBJECTIVES

To investigate the value of deep learning (DL) combined with radiomics and clinical and imaging features in predicting the Ki-67 proliferation index of soft tissue tumors (STTs).

MATERIALS AND METHODS

In this retrospective study, a total of 394 patients with STTs admitted from January 2021 to December 2023 in two separate hospitals were collected. Hospital-1 was the training cohort (323 cases, of which 89 and 234 were high and low Ki-67, respectively) and Hospital-2 was the external validation cohort (71 cases, of which 23 and 48 were high and low Ki-67, respectively). Clinical and ultrasound characteristics including age, sex, tumor size, morphology, margins, internal echoes and blood flow were assessed. Risk factors with significant correlations were screened by univariate and multivariate logistic regression analyses. After extracting the radiomics and DL features, the feature fusion model is constructed by Support Vector Machine. The prediction results obtained from separate clinical features, radiomics features and DL features were combined to construct decision fusion models. Finally, the DeLong test was used to compare whether the AUCs between the models were significantly different.

RESULTS

The three feature fusion models and three decision fusion models constructed demonstrated excellent diagnostic performance in predicting Ki-67 expression levels in STTs. Among them, the feature fusion model based on clinical, radiomics, and DL performed the best with an AUC of 0.911 (95% CI: 0.886-0.935) in the training cohort and 0.923 (95% CI: 0.873-0.972) in the validation cohort, and proved to be well-calibrated and clinically useful. The DeLong test showed that the decision fusion models based on clinical, radiomics and DL performed significantly worse than the three feature fusion models on the validation set. There was no statistical difference in diagnostic performance between the other models.

CONCLUSION

The ultrasound-based fusion model of clinical, radiomics, and DL features showed good performance in predicting Ki-67 expression levels in STTs.

Identifying invasiveness to aid lung adenocarcinoma diagnosis using deep learning and pathomics

Most classification efforts for primary subtypes of lung adenocarcinoma (LUAD) have not yet been integrated into clinical practice. This study explores the feasibility of combining deep learning and pathomics to identify tumor invasiveness in LUAD patients, highlighting its potential clinical value in assisting junior and intermediate pathologists. We retrospectively analyzed whole slide image (WSI) data from 289 patients with surgically resected ground-glass nodules (GGNs). First, three ResNet deep learning models were used to identify tumor regions. Second, features from the best-performing model were extracted to build pathomics using machine learning classifiers. Third, the accuracy of pathomics in predicting tumor invasiveness was compared with junior and intermediate pathologists' diagnoses. Performance was evaluated using the area under receiver operator characteristic curve (AUC). On the test cohort, ResNet18 achieved the highest AUC (0.956) and sensitivity (0.832) in identifying tumor areas, with an accuracy of 0.904; Random Forest provided high accuracy and AUC values of 0.814 and 0.807 in assessing tumor invasiveness. Pathology assistance improved diagnostic accuracy for junior and intermediate pathologists, with AUC values increasing from 0.547 to 0.759 and 0.656 to 0.769. This study suggests that deep learning and pathomics can enhance diagnostic accuracy, offering valuable support to pathologists.

Development of metastasis and survival prediction model of luminal and non-luminal breast cancer with weakly supervised learning based on pathomics

Objective

Breast cancer stands as the most prevalent form of cancer among women globally. This heterogeneous disease exhibits varying clinical behaviors. The stratification of breast cancer patients into risk groups, determined by their metastasis and survival outcomes, is pivotal for tailoring personalized treatments and therapeutic interventions. The pathological sections of radical specimens encompass a diverse range of histological information pertinent to the metastasis and survival of patients. In this study, our objective is to develop a deep learning model utilizing pathological images to predict the metastasis and survival outcomes for breast cancer patients.

Methods

This study utilized pathological sections from 204 radical mastectomy specimens obtained between January 2013 and December 2014 at the Second Affiliated Hospital of the Medical College of Zhejiang University. The 204 pathological slices were scanned and transformed into whole slide imaging (WSI), with manual labeling of all tumor areas. The WSI was then partitioned into smaller tiles measuring 512 × 512 pixels. Three networks, namely Densely Connected Convolutional Network 121 (DenseNet121), Residual Network (ResNet50), and Inception_v3, were assessed. Subsequently, we combined patch-level predictions, probability histograms, and Term Frequency-Inverse Document Frequency (TF-IDF) features to create comprehensive participants representations. These features served as the foundational input for developing a machine learning algorithm for metastasis analysis and a Cox regression model for survival analysis.

Result

Our results show that the Inception_v3 model shows a particularly robust patch recognition ability for estrogen receptor (ER) recognition. Our pathological model shows high accuracy in predicting tumor regions. The train area under the curve (AUC) of the Inception_v3 model based on supervised learning is 0.975, which is higher than the model established by weakly supervised learning. But the AUC of the metastasis prediction in training and testing sets is higher than value based on supervised learning. Furthermore, the C-index of the survival prediction model is 0.710 in the testing sets, which is also better than the value by supervised learning.

Conclusion

Our study demonstrates the significant potential of deep learning models in predicting breast cancer metastasis and prognosis, with the pathomic model showing high accuracy in identifying tumor areas and ER status. The integration of clinical features and pathomics signature into a nomogram further provides a valuable tool for clinicians to make individualized treatment decisions.

IDH-mutant glioma risk stratification via whole slide images: Identifying pathological feature associations

This article aims to develop and validate a pathological prognostic model for predicting prognosis in patients with isocitrate dehydrogenase (IDH)-mutant gliomas and reveal the biological underpinning of the prognostic pathological features. The pathomic model was constructed based on whole slide images (WSIs) from a training set (N = 486) and evaluated on internal validation set (N = 209), HPPH validation set (N = 54), and TCGA validation set (N = 352). Biological implications of PathScore and individual pathomic features were identified by pathogenomics set (N = 100). The WSI-based pathological signature was an independent prognostic factor. Incorporating the pathological features into a clinical model resulted in a pathological-clinical model that predicted survival better than either the pathological model or clinical model alone. Ten categories of pathways (metabolism, proliferation, immunity, DNA damage response, disease, migrate, protein modification, synapse, transcription and translation, and complex cellular functions) were significantly correlated with the WSI-based pathological features.

Clinical Application of Artificial Intelligence in Ultrasound Imaging for Oncology

Ultrasound (US) imaging is a widely used tool in oncology because of its noninvasiveness and real-time performance. However, its diagnostic accuracy can be limited by the skills of the examiner when performing manual scanning and by the presence of acoustic shadows that degrade image quality. Artificial intelligence (AI) technologies can support examiners in cancer screening and diagnosis by addressing these limitations. Here, we examine recent advances in AI research and development for US imaging in oncology. Breast cancer has been the most extensively studied cancer, with research predominantly focusing on tumor detection, differentiation between benign and malignant lesions, and prediction of lymph node metastasis. The American College of Radiology developed a medical imaging reporting and data system for various cancers that is often used to evaluate the accuracy of AI models. We will also explore the application of AI in clinical settings for US imaging in oncology. Despite progress, the number of approved AI-equipped software as medical devices for US imaging remains limited in Japan, the United States, and Europe. Practical issues that need to be addressed for clinical application include domain shifts, black boxes, and acoustic shadows. To address these issues, advances in image quality control, AI explainability, and preprocessing of acoustic shadows are essential.

Harnessing Artificial Intelligence to Enhance Global Breast Cancer Care: A Scoping Review of Applications, Outcomes, and Challenges

BACKGROUND

In recent years, Artificial Intelligence (AI) has shown transformative potential in advancing breast cancer care globally. This scoping review seeks to provide a comprehensive overview of AI applications in breast cancer care, examining how they could reshape diagnosis, treatment, and management on a worldwide scale and discussing both the benefits and challenges associated with their adoption.

METHODS

In accordance with PRISMA-ScR and ensuing guidelines on scoping reviews, PubMed, Web of Science, Cochrane Library, and Embase were systematically searched from inception to end of May 2024. Keywords included "Artificial Intelligence" and "Breast Cancer". Original studies were included based on their focus on AI applications in breast cancer care and narrative synthesis was employed for data extraction and interpretation, with the findings organized into coherent themes.

RESULTS

Finally, 84 articles were included. The majority were conducted in developed countries (n = 54). The majority of publications were in the last 10 years (n = 83). The six main themes for AI applications were AI for breast cancer screening (n = 32), AI for image detection of nodal status (n = 7), AI-assisted histopathology (n = 8), AI in assessing post-neoadjuvant chemotherapy (NACT) response (n = 23), AI in breast cancer margin assessment (n = 5), and AI as a clinical decision support tool (n = 9). AI has been used as clinical decision support tools to augment treatment decisions for breast cancer and in multidisciplinary tumor board settings. Overall, AI applications demonstrated improved accuracy and efficiency; however, most articles did not report patient-centric clinical outcomes.

CONCLUSIONS

AI applications in breast cancer care show promise in enhancing diagnostic accuracy and treatment planning. However, persistent challenges in AI adoption, such as data quality, algorithm transparency, and resource disparities, must be addressed to advance the field.

Deep Learning for Distinguishing Mucinous Breast Carcinoma From Fibroadenoma on Ultrasound

PURPOSE

Mucinous breast carcinoma (MBC) tends to be misdiagnosed as fibroadenomas (FA) due to its benign imaging characteristics. We aimed to develop a deep learning (DL) model to differentiate MBC and FA based on ultrasound (US) images. The model could contribute to the diagnosis of MBC for radiologists.

METHODS

In this retrospective study, 884 eligible patients (700 FA patients and 184 MBC patients) with 2257 US images were enrolled. The images were randomly divided into a training set (n = 1805 images) and a test set (n = 452 images) in a ratio of 8:2. First, we used the training set to establish DL model, DL+ age-cutoff model and DL+ age-tree model. Then, we compared the diagnostic performance of three models to get the optimal model. Finally, we evaluated the diagnostic performance of radiologists (4 junior and 4 senior radiologists) with and without the assistance of the optimal model in the test set.

RESULTS

The DL+ age-tree model yielded higher areas under the receiver operating characteristic curve (AUC) than DL model and DL+ age-cutoff model (0.945 vs. 0.835, P < .001; 0.945 vs. 0.931, P < .001, respectively). With the assistance of DL+ age-tree model, both junior and senior radiologists' AUC had significant improvement (0.746-0.818, P = .010, 0.827-0.860, P = .005, respectively).

CONCLUSIONS

The DL+ age-tree model based on US images and age showed excellent performance in the differentiation of MBC and FA. Moreover, it can effectively improve the performance of radiologists with different degrees of experience that may contribute to reducing the misdiagnosis of MBC.

Multimodal integration to identify the invasion status of lung adenocarcinoma intraoperatively

Evaluating the invasiveness of lung adenocarcinoma is crucial for determining the appropriate surgical strategy, impacting postoperative outcomes. This study developed a multimodality model combining radiomics, intraoperative frozen section (FS) pathology, and clinical indicators to predict invasion status. The study enrolled 1,424 patients from two hospitals, divided into multimodal training, radiology testing, and pathology testing cohorts. A prospective validation cohort of 114 patients was selected between March and May 2023. The radiomics + pathology + clinical indicators multimodality model (multi-RPC model) achieved an area under the curve (AUC) of 0.921 (95% confidence interval [CI] 0.899-0.939) in the multimodal training cohort and 0.939 (95% CI 0.878-0.975) in the validation cohort, outperforming single- and dual-modality models. The multi-RPC model's predictive accuracy of 0.860 (95% CI 0.782-0.918) suggests that it could significantly reduce inappropriate surgical procedures, enhancing precision oncology by integrating multimodal information to guide surgical decisions.

Machine learning radiomics based on intra and peri tumor PA/US images distinguish between luminal and non-luminal tumors in breast cancers

PURPOSE

This study aimed to evaluate a radiomics model using Photoacoustic/ultrasound (PA/US) imaging at intra and peri-tumoral area to differentiate Luminal and non-Luminal breast cancer (BC) and to determine the optimal peritumoral area for accurate classification.

MATERIALS AND METHODS

From February 2022 to April 2024, this study continuously collected 322 patients at Shenzhen People's Hospital, using standardized conditions for PA/US imaging of BC. Regions of interest were delineated using ITK-SNAP, with peritumoral regions of 2 mm, 4 mm, and 6 mm automatically expanded using code from the Pyradiomic package. Feature extraction was subsequently performed using Pyradiomics. The study employed Z-score normalization, Spearman correlation for feature correlation, and LASSO regression for feature selection, validated through 10-fold cross-validation. The radiomics model integrated intra and peri-tumoral area, evaluated by receiver operating characteristic curve(ROC), Calibration and Decision Curve Analysis(DCA).

RESULTS

We extracted and selected features from intratumoral and peritumoral PA/US images regions at 2 mm, 4 mm, and 6 mm. The comprehensive radiomics model, integrating these regions, demonstrated enhanced diagnostic performance, especially the 4 mm model which showed the highest area under the curve(AUC):0.898(0.78-1.00) and comparably high accuracy (0.900) and sensitivity (0.937). This model outperformed the standalone clinical model and combined clinical-radiomics model in distinguishing between Luminal and non-Luminal BC, as evidenced in the test set results.

CONCLUSION

This study developed a radiomics model integrating intratumoral and peritumoral at 4 mm region PA/US model, enhancing the differentiation of Luminal from non-Luminal BC. It demonstrated the diagnostic utility of peritumoral characteristics, reducing the need for invasive biopsies and aiding chemotherapy planning, while emphasizing the importance of optimizing tumor surrounding size for improved model accuracy.

Development of ultrasound-based clinical, radiomics and deep learning fusion models for the diagnosis of benign and malignant soft tissue tumors

Objectives

The aim of this study is to develop an ultrasound-based fusion model of clinical, radiomics and deep learning (CRDL) for accurate diagnosis of benign and malignant soft tissue tumors (STTs).

Methods

In this retrospective study, ultrasound images and clinical data of patients with STTs from two hospitals were collected between January 2021 and December 2023. Radiomics features and deep learning features were extracted from the ultrasound images, and the optimal features were selected to construct fusion models using support vector machines. The predictive performance of the model was evaluated based on three aspects: discrimination, calibration and clinical usefulness. The DeLong test was used to compare whether there was a significant difference in AUC between the models. Finally, two radiologists who were unaware of the clinical information performed an independent diagnosis and a model-assisted diagnosis of the tumor to compare the performance of the two diagnoses.

Results

A training cohort of 516 patients from Hospital-1 and an external validation cohort of 78 patients from Hospital-2 were included in the study. The Pre-FM CRDL showed the best performance in predicting STTs, with area under the curve (AUC) of 0.911 (95%CI: 0.894-0.928) and 0.948 (95%CI: 0.906-0.990) for training cohort and external validation cohort, respectively. The DeLong test showed that the Pre-FM CRDL significantly outperformed the clinical models (P< 0.05). In addition, the Pre-FM CRDL can improve the diagnostic accuracy of radiologists.

Conclusion

This study demonstrates the high clinical applicability of the fusion model in the differential diagnosis of STTs.

Tiron enhances the anti-cancer activity of doxorubicin in DMBA-induced breast cancer: Role of Notch signaling/apoptosis/autophagy/oxidative stress

Existing work intended to investigate the outcomes of the localized mitochondrial antioxidant tiron (TR) alone or in combination with doxorubicin (DOX) in 7,12-dimethylbenz[a]anthracene (DMBA)-induced mammary carcinogenesis in rats and the mechanistic pathways behind these effects. Also, to examine the preventive role of TR against DOX-related cardiotoxicity. 64 female Sprague-Dawley rats were randomly assigned into 8 groups: CTRL, DOX, TR, DMBA, DMBA + DOX, DMBA + TR100, DMBA + TR200, and DMBA + DOX + TR200. Rats received TR (100 and 200 mg/kg), DOX (2mg/kg), and DMBA (7.5 mg/kg) for four consecutive weeks. TR alone or combined with DOX not only inhibited oxidative status-related parameters and Notch pathway proteins but also attenuated proliferation markers, and enhanced apoptosis, and autophagy-related genes. Consistently, the histopathological analysis showed better scores in mammary tissues isolated from groups treated with TR only or combined with DOX. Additionally, TR dramatically decreased relative heart weight, myocardial injury biomarkers, and heart oxidative stress parameters while maintaining the myocardial histological integrity. Here we provided evidence that TR acts via modulating Notch signaling/apoptosis/autophagy/oxidative stress to elicit anti-tumor activity and combination with DOX revealed a higher efficacy as a novel anticancer strategy. Moreover, TR could be a potential cardio-protective candidate during DOX-chemotherapy, possibly via its antioxidant activity.

Deep learning-assisted distinguishing breast phyllodes tumours from fibroadenomas based on ultrasound images: a diagnostic study

OBJECTIVES

To evaluate the performance of ultrasound-based deep learning (DL) models in distinguishing breast phyllodes tumours (PTs) from fibroadenomas (FAs) and their clinical utility in assisting radiologists with varying diagnostic experiences.

METHODS

We retrospectively collected 1180 ultrasound images from 539 patients (247 PTs and 292 FAs). Five DL network models with different structures were trained and validated using nodule regions annotated by radiologists on breast ultrasound images. DL models were trained using the methods of transfer learning and 3-fold cross-validation. The model demonstrated the best evaluation index in the 3-fold cross-validation was selected for comparison with radiologists' diagnostic decisions. Two-round reader studies were conducted to investigate the value of DL model in assisting 6 radiologists with different levels of experience.

RESULTS

Upon testing, Xception model demonstrated the best diagnostic performance (area under the receiver-operating characteristic curve: 0.87; 95% CI, 0.81-0.92), outperforming all radiologists (all P < .05). Additionally, the DL model enhanced the diagnostic performance of radiologists. Accuracy demonstrated improvements of 4%, 4%, and 3% for senior, intermediate, and junior radiologists, respectively.

CONCLUSIONS

The DL models showed superior predictive abilities compared to experienced radiologists in distinguishing breast PTs from FAs. Utilizing the model led to improved efficiency and diagnostic performance for radiologists with different levels of experience (6-25 years of work).

ADVANCES IN KNOWLEDGE

We developed and validated a DL model based on the largest available dataset to assist in diagnosing PTs. This model has the potential to allow radiologists to discriminate 2 types of breast tumours which are challenging to identify with precision and accuracy, and subsequently to make more informed decisions about surgical plans.

Deep learning-based automatic pipeline system for predicting lateral cervical lymph node metastasis in patients with papillary thyroid carcinoma using computed tomography: A multi-center study

Objective

The assessment of lateral lymph node metastasis (LLNM) in patients with papillary thyroid carcinoma (PTC) holds great significance. This study aims to develop and evaluate a deep learning-based automatic pipeline system (DLAPS) for diagnosing LLNM in PTC using computed tomography (CT).

Methods

A total of 1,266 lateral lymph nodes (LLNs) from 519 PTC patients who underwent CT examinations from January 2019 to November 2022 were included and divided into training and validation set, internal test set, pooled external test set, and prospective test set. The DLAPS consists of an auto-segmentation network based on RefineNet model and a classification network based on ensemble model (ResNet, Xception, and DenseNet). The performance of the DLAPS was compared with that of manually segmented DL models, the clinical model, and Node Reporting and Data System (Node-RADS). The improvement of radiologists' diagnostic performance under the DLAPS-assisted strategy was explored. In addition, bulk RNA-sequencing was conducted based on 12 LLNs to reveal the underlying biological basis of the DLAPS.

Results

The DLAPS yielded good performance with area under the receiver operating characteristic curve (AUC) of 0.872, 0.910, and 0.822 in the internal, pooled external, and prospective test sets, respectively. The DLAPS significantly outperformed clinical models (AUC 0.731, P<0.001) and Node-RADS (AUC 0.602, P<0.001) in the internal test set. Moreover, the performance of the DLAPS was comparable to that of the manually segmented deep learning (DL) model with AUCs ranging 0.814-0.901 in three test sets. Furthermore, the DLAPS-assisted strategy improved the performance of radiologists and enhanced inter-observer consistency. In clinical situations, the rate of unnecessary LLN dissection decreased from 33.33% to 7.32%. Furthermore, the DLAPS was associated with the cell-cell conjunction in the microenvironment.

Conclusions

Using CT images from PTC patients, the DLAPS could effectively segment and classify LLNs non-invasively, and this system had a good generalization ability and clinical applicability.

Radiomics in breast cancer: Current advances and future directions

Breast cancer is a common disease that causes great health concerns to women worldwide. During the diagnosis and treatment of breast cancer, medical imaging plays an essential role, but its interpretation relies on radiologists or clinical doctors. Radiomics can extract high-throughput quantitative imaging features from images of various modalities via traditional machine learning or deep learning methods following a series of standard processes. Hopefully, radiomic models may aid various processes in clinical practice. In this review, we summarize the current utilization of radiomics for predicting clinicopathological indices and clinical outcomes. We also focus on radio-multi-omics studies that bridge the gap between phenotypic and microscopic scale information. Acknowledging the deficiencies that currently hinder the clinical adoption of radiomic models, we discuss the underlying causes of this situation and propose future directions for advancing radiomics in breast cancer research.

SC-Unext: A Lightweight Image Segmentation Model with Cellular Mechanism for Breast Ultrasound Tumor Diagnosis

Automatic breast ultrasound image segmentation plays an important role in medical image processing. However, current methods for breast ultrasound segmentation suffer from high computational complexity and large model parameters, particularly when dealing with complex images. In this paper, we take the Unext network as a basis and utilize its encoder-decoder features. And taking inspiration from the mechanisms of cellular apoptosis and division, we design apoptosis and division algorithms to improve model performance. We propose a novel segmentation model which integrates the division and apoptosis algorithms and introduces spatial and channel convolution blocks into the model. Our proposed model not only improves the segmentation performance of breast ultrasound tumors, but also reduces the model parameters and computational resource consumption time. The model was evaluated on the breast ultrasound image dataset and our collected dataset. The experiments show that the SC-Unext model achieved Dice scores of 75.29% and accuracy of 97.09% on the BUSI dataset, and on the collected dataset, it reached Dice scores of 90.62% and accuracy of 98.37%. Meanwhile, we conducted a comparison of the model's inference speed on CPUs to verify its efficiency in resource-constrained environments. The results indicated that the SC-Unext model achieved an inference speed of 92.72 ms per instance on devices equipped only with CPUs. The model's number of parameters and computational resource consumption are 1.46M and 2.13 GFlops, respectively, which are lower compared to other network models. Due to its lightweight nature, the model holds significant value for various practical applications in the medical field.

The prediction of pCR and chemosensitivity for breast cancer patients using DLG3, RADL and Pathomics signatures based on machine learning and deep learning

BACKGROUND

Limited studies have investigated the predictive value of multiomics signatures (radiomics, deep learning features, pathological features and DLG3) in breast cancer patients who underwent neoadjuvant chemotherapy (NAC). However, no study has explored the relationships among radiomic, pathomic signatures and chemosensitivity. This study aimed to predict pathological complete response (pCR) using multiomics signatures, and to evaluate the predictive utility of radiomic and pathomic signatures for guiding chemotherapy selection.

METHODS

The oncogenic function of DLG3 was explored in breast cancer cells via DLG3 knockdown. Immunohistochemistry (IHC) was used to evaluate the relationship between DLG3 expression and docetaxel/epirubin sensitivity. Machine learning (ML) and deep learning (DL) algorithms were used to develop multiomics signatures. Survival analysis was conducted by K-M curves and log-rank. Multivariate logistic regression analysis was used to develop nomograms.

RESULTS

A total of 311 patients with malignant breast tumours who underwent NAC were retrospectively included in this multicentre study. Multiomics (DLG3, RADL and PATHO) signatures could accurately predict pCR (AUC: training: 0.900; testing: 0.814; external validation: 0.792). Its performance is also superior to that of clinical TNM staging and the single RADL signature in different cohorts. Patients in the low DLG3 group more easily achieved pCR, and those in the high RADL Signature_pCR and PATHO_Signature_pCR (OR = 7.93, 95 % CI: 3.49-18, P < 0.001) groups more easily achieved pCR. In the TEC regimen NAC group, patients who achieved pCR had a lower DLG3 score (4.00 ± 2.33 vs. 6.43 ± 3.01, P < 0.05). Patients in the low RADL_Signature_DLG3 and PATHO_Signature_DLG3 groups had lower DLG3 IHC scores (P < 0.05). Patients in the high RADL signature, PATHO signature and DLG3 signature groups had worse DFS and OS.

CONCLUSIONS

Multiomics signatures (RADL, PATHO and DLG3) demonstrated great potential in predicting the pCR of breast cancer patients who underwent NAC. The RADL and PATHO signatures are associated with DLG3 status and could help doctors or patients choose proper neoadjuvant chemotherapy regimens (TEC regimens). This simple, structured, convenient and inexpensive multiomics model could help clinicians and patients make treatment decisions.

Deep learning model based on endoscopic images predicting treatment response in locally advanced rectal cancer undergo neoadjuvant chemoradiotherapy: a multicenter study

PURPOSE

Neoadjuvant chemoradiotherapy has been the standard practice for patients with locally advanced rectal cancer. However, the treatment response varies greatly among individuals, how to select the optimal candidates for neoadjuvant chemoradiotherapy is crucial. This study aimed to develop an endoscopic image-based deep learning model for predicting the response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer.

METHODS

In this multicenter observational study, pre-treatment endoscopic images of patients from two Chinese medical centers were retrospectively obtained and a deep learning-based tumor regression model was constructed. Treatment response was evaluated based on the tumor regression grade and was defined as good response and non-good response. The prediction performance of the deep learning model was evaluated in the internal and external test sets. The main outcome was the accuracy of the treatment prediction model, measured by the AUC and accuracy.

RESULTS

This deep learning model achieved favorable prediction performance. In the internal test set, the AUC and accuracy were 0.867 (95% CI: 0.847-0.941) and 0.836 (95% CI: 0.818-0.896), respectively. The prediction performance was fully validated in the external test set, and the model had an AUC of 0.758 (95% CI: 0.724-0.834) and an accuracy of 0.807 (95% CI: 0.774-0.843).

CONCLUSION

The deep learning model based on endoscopic images demonstrated exceptional predictive power for neoadjuvant treatment response, highlighting its potential for guiding personalized therapy.

Deep learning radiomics based on multimodal imaging for distinguishing benign and malignant breast tumours

Objectives

This study aimed to develop a deep learning radiomic model using multimodal imaging to differentiate benign and malignant breast tumours.

Methods

Multimodality imaging data, including ultrasonography (US), mammography (MG), and magnetic resonance imaging (MRI), from 322 patients (112 with benign breast tumours and 210 with malignant breast tumours) with histopathologically confirmed breast tumours were retrospectively collected between December 2018 and May 2023. Based on multimodal imaging, the experiment was divided into three parts: traditional radiomics, deep learning radiomics, and feature fusion. We tested the performance of seven classifiers, namely, SVM, KNN, random forest, extra trees, XGBoost, LightGBM, and LR, on different feature models. Through feature fusion using ensemble and stacking strategies, we obtained the optimal classification model for benign and malignant breast tumours.

Results

In terms of traditional radiomics, the ensemble fusion strategy achieved the highest accuracy, AUC, and specificity, with values of 0.892, 0.942 [0.886-0.996], and 0.956 [0.873-1.000], respectively. The early fusion strategy with US, MG, and MRI achieved the highest sensitivity of 0.952 [0.887-1.000]. In terms of deep learning radiomics, the stacking fusion strategy achieved the highest accuracy, AUC, and sensitivity, with values of 0.937, 0.947 [0.887-1.000], and 1.000 [0.999-1.000], respectively. The early fusion strategies of US+MRI and US+MG achieved the highest specificity of 0.954 [0.867-1.000]. In terms of feature fusion, the ensemble and stacking approaches of the late fusion strategy achieved the highest accuracy of 0.968. In addition, stacking achieved the highest AUC and specificity, which were 0.997 [0.990-1.000] and 1.000 [0.999-1.000], respectively. The traditional radiomic and depth features of US+MG + MR achieved the highest sensitivity of 1.000 [0.999-1.000] under the early fusion strategy.

Conclusion

This study demonstrated the potential of integrating deep learning and radiomic features with multimodal images. As a single modality, MRI based on radiomic features achieved greater accuracy than US or MG. The US and MG models achieved higher accuracy with transfer learning than the single-mode or radiomic models. The traditional radiomic and depth features of US+MG + MR achieved the highest sensitivity under the early fusion strategy, showed higher diagnostic performance, and provided more valuable information for differentiation between benign and malignant breast tumours.