Predicting HER2 Status in Breast Cancer on Ultrasound Images Using Deep Learning Method

Deep Learning and Radiomics in Triple-Negative Breast Cancer: Predicting Long-Term Prognosis and Clinical Outcomes

Triple-negative breast cancer (TNBC) is a unique breast cancer subtype characterized by the lack of estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) expression in tumor cells. TNBC represents about 15% to 20% of all breast cancers and is aggressive and highly malignant. Currently, TNBC diagnosis primarily depends on pathological examination, while treatment efficacy is assessed through imaging, biomarker detection, pathological evaluation, and clinical symptom improvement. Among these, biomarker detection and pathological assessments are invasive, time-intensive procedures that may be difficult for patients with severe comorbidities and high complication risks. Thus, there is an urgent need for new, supportive tools in TNBC diagnosis and treatment. Deep learning and radiomics techniques represent advanced machine learning methodologies and are also emerging outcomes in the medical-engineering field in recent years. They are extensions of conventional imaging diagnostic methods and have demonstrated tremendous potential in image segmentation, reconstruction, recognition, and classification. These techniques hold certain application prospects for the diagnosis of TNBC, assessment of treatment response, and long-term prognosis prediction. This article reviews recent progress in the application of deep learning, ultrasound, MRI, and radiomics for TNBC diagnosis and treatment, based on research from both domestic and international scholars.

Integration of Longitudinal and Transverse Radiomics from Ultrasound Images with Clinical Factors for HER-2 Status Prediction in Invasive Breast Cancer Patients

OBJECTIVE

This study developed a nomogram combining longitudinal and transverse ultrasound radiomics with clinical factors to identify human epidermal growth factor receptor 2 (HER2) status in invasive breast cancer (BC).

MATERIALS AND METHODS

We analyzed 537 invasive BC patients from two hospitals: 436 in the training cohort (Hospital A) and 101 in the test cohort (Hospital B). From longitudinal and transverse ultrasound planes, 788 radiomics features were extracted, with dimensionality reduced using least absolute shrinkage and selection operator regression. A radiomics nomogram integrating clinical predictors and radiomics scores (Rad-scores) was constructed.

RESULTS

Fifteen and sixteen features from longitudinal and transverse ultrasound planes, respectively, were selected to generate Rad-scores, which differed significantly between HER2-positive and HER2-negative groups in both cohorts (p < 0.05). The combined radiomics model outperformed individual models with AUCs of 0.783 and 0.762 in the training and external test cohorts, respectively. Tumor size was an independent clinical predictor. The nomogram, incorporating Rad-scores and tumor size, achieved AUCs of 0.790 (training cohort) and 0.774 (test cohort). Decision curve analysis demonstrated its potential clinical utility.

CONCLUSION

A biplanar ultrasound radiomics nomogram effectively predicts HER2 status in invasive BC, potentially reducing the need for biopsies and supporting personalized treatment strategies.

Deep learning-based computed tomography urography image analysis for prediction of HER2 status in bladder cancer

Purpose: Bladder cancer (BCa) is one of the most common malignant tumors in the urinary system. BCa with HER2 overexpression can benefit from more precise treatments, but HER2 testing is costly and subjective. This study aimed to detect HER2 expression using computed tomography urography (CTU) images. Method: We gathered CTU images from 97 patients with BCa during the excretion phase in Renmin Hospital of Wuhan University, manually delineated the BCa regions, extracted radiomic features using the Pyradiomics package, conducted data dimensionality reduction via principal component analysis (PCA), and trained three models (Logistic Regression [LR], Random Forest [RF] and Multilayer Perceptron [MLP]) to discern the HER2 expression status. Results: Pyradiomics package was used to extract 975 radiological features and the cumulative interpretation area under the variance curve was 90.964 by PCA. Using an MLP-based deep learning model for identifying HER2 overexpression, the area under the curve (AUC) reached 0.79 (95% confidence interval [CI] 0.74-0.86) in the training set and 0.73 (95% CI 0.66-0.77) in the validation set. LR and RF had AUC of 0.69 (95% CI 0.63-0.75) and 0.66 (95% CI 0.61-0.70) in the training set and 0.61 (95% CI 0.55-0.67) and 0.59 (95% CI 0.55-0.63) in the test set, respectively. Conclusion: The study firstly presents a non-invasive method for identifying and detecting HER2 expression in BCa CTU images. It might not only reduce the cost and subjectivity of traditional HER2 testing but also provide a new technical method for the precise treatment of BCa.

Diagnostic performance of ultrasound-based artificial intelligence for predicting key molecular markers in breast cancer: A systematic review and meta-analysis

BACKGROUND

Breast cancer (BC) diagnosis and treatment rely heavily on molecular markers such as HER2, Ki67, PR, and ER. Currently, these markers are identified by invasive methods.

OBJECTIVE

This meta-analysis investigates the diagnostic accuracy of ultrasound-based radiomics as a novel approach to predicting these markers.

METHODS

A comprehensive search of PubMed, EMBASE, and Web of Science databases was conducted to identify studies evaluating ultrasound-based radiomics in BC. Inclusion criteria encompassed research on HER2, Ki67, PR, and ER as key molecular markers. Quality assessment using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Radiomics Quality Score (RQS) was performed. The data extraction step was performed systematically.

RESULTS

Our meta-analysis quantifies the diagnostic accuracy of ultrasound-based radiomics with a sensitivity and specificity of 0.76 and 0.78 for predicting HER2, 0.80, and 0.76 for Ki67 biomarkers. Studies did not provide sufficient data for quantitative PR and ER prediction analysis. The overall quality of studies based on the RQS tool was moderate. The QUADAS-2 evaluation showed that the studies had an unclear risk of bias regarding the flow and timing domain.

CONCLUSION

Our analysis indicated that AI models have a promising accuracy for predicting key molecular biomarkers' status in BC patients. We performed the quantitative analysis for HER2 and Ki67 biomarkers which yielded a moderate to high accuracy. However, studies did not provide adequate data for meta-analysis of ER and PR prediction accuracy of developed models. The overall quality of the studies was acceptable. In future research, studies need to report the results thoroughly. Also, we suggest more prospective studies from different centers.

Radiomics analysis of intratumoral and different peritumoral regions from multiparametric MRI for evaluating HER2 status of breast cancer: A comparative study

Purpose

To investigate the potential of radiomics signatures (RSs) from intratumoral and peritumoral regions on multiparametric magnetic resonance imaging (MRI) to noninvasively evaluate HER2 status in breast cancer.

Method

In this retrospective study, 992 patients with pathologically confirmed breast cancers who underwent preoperative MRI were enrolled. The breast cancer lesions were segmented manually, and the intratumor region of interest (ROIIntra) was dilated by 2, 4, 6 and 8 mm (ROIPeri2mm, ROIPeri4mm, ROIPeri6mm, and ROIPeri8mm, respectively). Quantitative radiomics features were extracted from dynamic contrast-enhanced T1-weighted imaging (DCE-T1), fat-saturated T2-weighted imaging (T2) and diffusion-weighted imaging (DWI). A three-step procedure was performed for feature selection, and RSs were constructed using a support vector machine (SVM) to predict HER2 status.

Result

The best single-area RSs for predicting HER2 status were DCE_Peri4mm-RS, T2_Peri4mm-RS, and DWI_Peri4mm-RS, yielding areas under the curve (AUCs) of 0.716 (95% confidence interval (CI), 0.648-0.778), 0.706 (95% CI, 0.637-0.768), and 0.719 (95% CI, 0.651-0.780), respectively, in the test set. The optimal RSs combining intratumoral and peritumoral regions for evaluating HER2 status were DCE-T1_Intra + DCE_Peri4mm-RS, T2_Intra + T2_Peri6mm-RS and DWI_Intra + DWI_Peri4mm-RS, with AUCs of 0.752 (95% CI, 0.686-0.810), 0.754 (95% CI, 0.688-0.812) and 0.725 (95% CI, 0.657-0.786), respectively, in the test set. Combining three sequences in the ROIIntra, ROIPeri2mm, ROIPeri4mm, ROIPeri6mm and ROIPeri8mm areas, the optimal RS was DCE-T1_Peri4mm + T2_Peri4mm + DWI_Peri4mm-RS, achieving an AUC of 0.795 (95% CI, 0.733-0.849) in the test set.

Conclusion

This study systematically explored the influence of the intratumoral region, different peritumoral sizes and their combination in radiomics analysis for predicting HER2 status in breast cancer based on multiparametric MRI and found the optimal RS.

Diagnostic efficiency of multi-modal MRI based deep learning with Sobel operator in differentiating benign and malignant breast mass lesions-a retrospective study

Purpose

To compare the diagnostic efficiencies of deep learning single-modal and multi-modal for the classification of benign and malignant breast mass lesions.

Methods

We retrospectively collected data from 203 patients (207 lesions, 101 benign and 106 malignant) with breast tumors who underwent breast magnetic resonance imaging (MRI) before surgery or biopsy between January 2014 and October 2020. Mass segmentation was performed based on the three dimensions-region of interest (3D-ROI) minimum bounding cube at the edge of the lesion. We established single-modal models based on a convolutional neural network (CNN) including T2WI and non-fs T1WI, the dynamic contrast-enhanced (DCE-MRI) first phase was pre-contrast T1WI (d1), and Phases 2, 4, and 6 were post-contrast T1WI (d2, d4, d6); and Multi-modal fusion models with a Sobel operator (four_mods:T2WI, non-fs-T1WI, d1, d2). Training set (n = 145), validation set (n = 22), and test set (n = 40). Five-fold cross validation was performed. Accuracy, sensitivity, specificity, negative predictive value, positive predictive value, and area under the ROC curve (AUC) were used as evaluation indicators. Delong's test compared the diagnostic performance of the multi-modal and single-modal models.

Results

All models showed good performance, and the AUC values were all greater than 0.750. Among the single-modal models, T2WI, non-fs-T1WI, d1, and d2 had specificities of 77.1%, 77.2%, 80.2%, and 78.2%, respectively. d2 had the highest accuracy of 78.5% and showed the best diagnostic performance with an AUC of 0.827. The multi-modal model with the Sobel operator performed better than single-modal models, with an AUC of 0.887, sensitivity of 79.8%, specificity of 86.1%, and positive prediction value of 85.6%. Delong's test showed that the diagnostic performance of the multi-modal fusion models was higher than that of the six single-modal models (T2WI, non-fs-T1WI, d1, d2, d4, d6); the difference was statistically significant (p = 0.043, 0.017, 0.006, 0.017, 0.020, 0.004, all were greater than 0.05).

Conclusions

Multi-modal fusion deep learning models with a Sobel operator had excellent diagnostic value in the classification of breast masses, and further increase the efficiency of diagnosis.

Artificial Intelligence for Breast US

US is a widely available, commonly used, and indispensable imaging modality for breast evaluation. It is often the primary imaging modality for the detection and diagnosis of breast cancer in low-resource settings. In addition, it is frequently employed as a supplemental screening tool via either whole breast handheld US or automated breast US among women with dense breasts. In recent years, a variety of artificial intelligence systems have been developed to assist radiologists with the detection and diagnosis of breast lesions on US. This article reviews the background and evidence supporting the use of artificial intelligence tools for breast US, describes implementation strategies and impact on clinical workflow, and discusses potential emerging roles and future directions.

Deep learning radiomics model based on breast ultrasound video to predict HER2 expression status

Purpose

The detection of human epidermal growth factor receptor 2 (HER2) expression status is essential to determining the chemotherapy regimen for breast cancer patients and to improving their prognosis. We developed a deep learning radiomics (DLR) model combining time-frequency domain features of ultrasound (US) video of breast lesions with clinical parameters for predicting HER2 expression status.

Patients and Methods

Data for this research was obtained from 807 breast cancer patients who visited from February 2019 to July 2020. Ultimately, 445 patients were included in the study. Pre-operative breast ultrasound examination videos were collected and split into a training set and a test set. Building a training set of DLR models combining time-frequency domain features and clinical features of ultrasound video of breast lesions based on the training set data to predict HER2 expression status. Test the performance of the model using test set data. The final models integrated with different classifiers are compared, and the best performing model is finally selected.

Results

The best diagnostic performance in predicting HER2 expression status is provided by an Extreme Gradient Boosting (XGBoost)-based time-frequency domain feature classifier combined with a logistic regression (LR)-based clinical parameter classifier of clinical parameters combined DLR, particularly with a high specificity of 0.917. The area under the receiver operating characteristic curve (AUC) for the test cohort was 0.810.

Conclusion

Our study provides a non-invasive imaging biomarker to predict HER2 expression status in breast cancer patients.

Artificial Intelligence in Breast Ultrasound: From Diagnosis to Prognosis-A Rapid Review

BACKGROUND

Ultrasound (US) is a fundamental diagnostic tool in breast imaging. However, US remains an operator-dependent examination. Research into and the application of artificial intelligence (AI) in breast US are increasing. The aim of this rapid review was to assess the current development of US-based artificial intelligence in the field of breast cancer.

METHODS

Two investigators with experience in medical research performed literature searching and data extraction on PubMed. The studies included in this rapid review evaluated the role of artificial intelligence concerning BC diagnosis, prognosis, molecular subtypes of breast cancer, axillary lymph node status, and the response to neoadjuvant chemotherapy. The mean values of sensitivity, specificity, and AUC were calculated for the main study categories with a meta-analytical approach.

RESULTS

A total of 58 main studies, all published after 2017, were included. Only 9/58 studies were prospective (15.5%); 13/58 studies (22.4%) used an ML approach. The vast majority (77.6%) used DL systems. Most studies were conducted for the diagnosis or classification of BC (55.1%). At present, all the included studies showed that AI has excellent performance in breast cancer diagnosis, prognosis, and treatment strategy.

CONCLUSIONS

US-based AI has great potential and research value in the field of breast cancer diagnosis, treatment, and prognosis. More prospective and multicenter studies are needed to assess the potential impact of AI in breast ultrasound.

Development and Validation of an Ultrasound-Based Radiomics Nomogram for Identifying HER2 Status in Patients with Breast Carcinoma

(1) Objective: To evaluate the performance of ultrasound-based radiomics in the preoperative prediction of human epidermal growth factor receptor 2-positive (HER2+) and HER2- breast carcinoma. (2) Methods: Ultrasound images from 309 patients (86 HER2+ cases and 223 HER2- cases) were retrospectively analyzed, of which 216 patients belonged to the training set and 93 patients assigned to the time-independent validation set. The region of interest of the tumors was delineated, and the radiomics features were extracted. Radiomics features underwent dimensionality reduction analyses using the intra-class correlation coefficient (ICC), Mann-Whitney U test, and the least absolute shrinkage and selection operator (LASSO) algorithm. The radiomics score (Rad-score) for each patient was calculated through a linear combination of the nonzero coefficient features. The support vector machine (SVM), K nearest neighbors (KNN), logistic regression (LR), decision tree (DT), random forest (RF), naive Bayes (NB) and XGBoost (XGB) machine learning classifiers were trained to establish prediction models based on the Rad-score. A clinical model based on significant clinical features was also established. In addition, the logistic regression method was used to integrate Rad-score and clinical features to generate the nomogram model. The leave-one-out cross validation (LOOCV) method was used to validate the reliability and stability of the model. (3) Results: Among the seven classifier models, the LR achieved the best performance in the validation set, with an area under the receiver operating characteristic curve (AUC) of 0.786, and was obtained as the Rad-score model, while the RF performed the worst. Tumor size showed a statistical difference between the HER2+ and HER2- groups (p = 0.028). The nomogram model had a slightly higher AUC than the Rad-score model (AUC, 0.788 vs. 0.786), but no statistical difference (Delong test, p = 0.919). The LOOCV method yielded a high median AUC of 0.790 in the validation set. (4) Conclusion: The Rad-score model performs best among the seven classifiers. The nomogram model based on Rad-score and tumor size has slightly better predictive performance than the Rad-score model, and it has the potential to be utilized as a routine modality for preoperatively determining HER2 status in BC patients non-invasively.

Strategies for Enhancing the Multi-Stage Classification Performances of HER2 Breast Cancer from Hematoxylin and Eosin Images

Breast cancer is a significant health concern among women. Prompt diagnosis can diminish the mortality rate and direct patients to take steps for cancer treatment. Recently, deep learning has been employed to diagnose breast cancer in the context of digital pathology. To help in this area, a transfer learning-based model called 'HE-HER2Net' has been proposed to diagnose multiple stages of HER2 breast cancer (HER2-0, HER2-1+, HER2-2+, HER2-3+) on H&E (hematoxylin & eosin) images from the BCI dataset. HE-HER2Net is the modified version of the Xception model, which is additionally comprised of global average pooling, several batch normalization layers, dropout layers, and dense layers with a swish activation function. This proposed model exceeds all existing models in terms of accuracy (0.87), precision (0.88), recall (0.86), and AUC score (0.98) immensely. In addition, our proposed model has been explained through a class-discriminative localization technique using Grad-CAM to build trust and to make the model more transparent. Finally, nuclei segmentation has been performed through the StarDist method.