Deep learning radiomics model based on breast ultrasound video to predict HER2 expression status

Ultrasound-based radiogenomics: status, applications, and future direction

Radiogenomics, an extension of radiomics, explores the relationship between imaging features and underlying gene expression patterns. This field is instrumental in providing reliable imaging surrogates, thus potentially representing an alternative to genetic testing. The rapidly growing area of radiogenomics that utilizes ultrasound (US) imaging seeks to elucidate the connections between US image characteristics and genomic data. In this review, the authors outline the radiogenomics workflow and summarize the applications of US-based radiogenomics. These include the prediction of gene variations, molecular subtypes, and other biological characteristics, as well as the exploration of the relationships between US phenotypes and cancer gene profiles. Although the field faces various challenges, US-based radiogenomics offers promising prospects and avenues for future research.

Identifying radiogenomic associations of breast cancer based on DCE-MRI by using Siamese Neural Network with manufacturer bias normalization

BACKGROUND AND PURPOSE

The immunohistochemical test (IHC) for Human Epidermal Growth Factor Receptor 2 (HER2) and hormone receptors (HR) provides prognostic information and guides treatment for patients with invasive breast cancer. The objective of this paper is to establish a non-invasive system for identifying HER2 and HR in breast cancer using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI).

METHODS

In light of the absence of high-performance algorithms and external validation in previously published methods, this study utilizes 3D deep features and radiomics features to represent the information of the Region of Interest (ROI). A Siamese Neural Network was employed as the classifier, with 3D deep features and radiomics features serving as the network input. To neutralize manufacturer bias, a batch effect normalization method, ComBat, was introduced. To enhance the reliability of the study, two datasets, Predict Your Therapeutic Response with Imaging and moLecular Analysis (I-SPY 1) and I-SPY 2, were incorporated. I-SPY 2 was utilized for model training and validation, while I-SPY 1 was exclusively employed for external validation. Additionally, a breast tumor segmentation network was trained to improve radiomic feature extraction.

RESULTS

The results indicate that our approach achieved an average Area Under the Curve (AUC) of 0.632, with a Standard Error of the Mean (SEM) of 0.042 for HER2 prediction in the I-SPY 2 dataset. For HR prediction, our method attained an AUC of 0.635 (SEM 0.041), surpassing other published methods in the AUC metric. Moreover, the proposed method yielded competitive results in other metrics. In external validation using the I-SPY 1 dataset, our approach achieved an AUC of 0.567 (SEM 0.032) for HR prediction and 0.563 (SEM 0.033) for HER2 prediction.

CONCLUSION

This study proposes a non-invasive system for identifying HER2 and HR in breast cancer. Although the results do not conclusively demonstrate superiority in both tasks, they indicate that the proposed method achieved good performance and is a competitive classifier compared to other reference methods. Ablation studies demonstrate that both radiomics features and deep features for the Siamese Neural Network are beneficial for the model. The introduced manufacturer bias normalization method has been shown to enhance the method's performance. Furthermore, the external validation of the method enhances the reliability of this research. Source code, pre-trained segmentation network, Radiomics and deep features, data for statistical analysis, and Supporting Information of this article are online at: https://github.com/FORRESTHUACHEN/Siamese_Neural_Network_based_Brest_cancer_Radiogenomic.

Deep learning applications in breast cancer histopathological imaging: diagnosis, treatment, and prognosis

Breast cancer is the most common malignant tumor among women worldwide and remains one of the leading causes of death among women. Its incidence and mortality rates are continuously rising. In recent years, with the rapid advancement of deep learning (DL) technology, DL has demonstrated significant potential in breast cancer diagnosis, prognosis evaluation, and treatment response prediction. This paper reviews relevant research progress and applies DL models to image enhancement, segmentation, and classification based on large-scale datasets from TCGA and multiple centers. We employed foundational models such as ResNet50, Transformer, and Hover-net to investigate the performance of DL models in breast cancer diagnosis, treatment, and prognosis prediction. The results indicate that DL techniques have significantly improved diagnostic accuracy and efficiency, particularly in predicting breast cancer metastasis and clinical prognosis. Furthermore, the study emphasizes the crucial role of robust databases in developing highly generalizable models. Future research will focus on addressing challenges related to data management, model interpretability, and regulatory compliance, ultimately aiming to provide more precise clinical treatment and prognostic evaluation programs for breast cancer patients.

Radiogenomics: bridging the gap between imaging and genomics for precision oncology

Genomics allows the tracing of origin and evolution of cancer at molecular scale and underpin modern cancer diagnosis and treatment systems. Yet, molecular biomarker-guided clinical decision-making encounters major challenges in the realm of individualized medicine, consisting of the invasiveness of procedures and the sampling errors due to high tumor heterogeneity. By contrast, medical imaging enables noninvasive and global characterization of tumors at a low cost. In recent years, radiomics has overcomes the limitations of human visual evaluation by high-throughput quantitative analysis, enabling the comprehensive utilization of the vast amount of information underlying radiological images. The cross-scale integration of radiomics and genomics (hereafter radiogenomics) has the enormous potential to enhance cancer decoding and act as a catalyst for digital precision medicine. Herein, we provide a comprehensive overview of the current framework and potential clinical applications of radiogenomics in patient care. We also highlight recent research advances to illustrate how radiogenomics can address common clinical problems in solid tumors such as breast cancer, lung cancer, and glioma. Finally, we analyze existing literature to outline challenges and propose solutions, while also identifying future research pathways. We believe that the perspectives shared in this survey will provide a valuable guide for researchers in the realm of radiogenomics aiming to advance precision oncology.

Prediction of HER2 Status Based on Deep Learning in H&E-Stained Histopathology Images of Bladder Cancer

Epidermal growth factor receptor 2 (HER2) has been widely recognized as one of the targets for bladder cancer immunotherapy. The key to implementing personalized treatment for bladder cancer patients lies in achieving rapid and accurate diagnosis. To tackle this challenge, we have pioneered the application of deep learning techniques to predict HER2 expression status from H&E-stained pathological images of bladder cancer, bypassing the need for intricate IHC staining or high-throughput sequencing methods. Our model, when subjected to rigorous testing within the cohort from the People's Hospital of Wuhan University, which encompasses 106 cases, has exhibited commendable performance on both the validation and test datasets. Specifically, the validation set yielded an AUC of 0.92, an accuracy of 0.86, a sensitivity of 0.87, a specificity of 0.83, and an F1 score of 86.7%. The corresponding metrics for the test set were 0.88 for AUC, 0.67 for accuracy, 0.56 for sensitivity, 0.75 for specificity, and 77.8% for F1 score. Additionally, in a direct comparison with pathologists, our model demonstrated statistically superior performance, with a p-value less than 0.05, highlighting its potential as a powerful diagnostic tool.

Prediction of Receptor Status in Radiomics: Recent Advances in Breast Cancer Research

Breast cancer is a multifactorial heterogeneous disease and the leading cause of cancer-related deaths in women; its diagnosis and treatment require clinical sensitivity and a comprehensive disciplinary research approach. The expression of different receptors on tumor cells not only provides the basis for molecular typing of breast cancer but also has a decisive role in the diagnosis, treatment, and prognosis of breast cancer. To date, immunohistochemistry (IHC), which uses invasive histological sampling, has been extensively used in clinical practice to analyze the status of receptors and to make an accurate diagnosis of breast cancer. As an invasive assay, IHC can provide important biological information on tumors at a single point in time, but cannot predict future changes (due to treatment or tumor mutations) without additional invasive procedures. These issues highlight the need to develop a non-invasive method for predicting receptor status. The emerging field of radiomics may offer a non-invasive approach to identification of receptor status without requiring biopsy. In this paper, we present a review of the latest research results in radiomics for predicting the status of breast cancer receptors, with potential important clinical applications.

The Emergence of the Potential Therapeutic Targets: Ultrasound-Based Radiomics in the Prediction of Human Epidermal Growth Factor Receptor 2-Low Breast Cancer

RATIONALE AND OBJECTIVES

To evaluate whether ultrasound-based radiomics features can effectively predict HER2-low expression in patients with breast cancer (BC).

MATERIAL AND METHODS

Between January 2021 and June 2023, patients who received US scans with pathologically confirmed BC in this multicenter study were included. In total, 383 patients from institution 1 were comprised of training set, 233 patients from institution 2 were comprised of validation set and 149 patients from institution 3 were comprised of external validation set. Radiomics features were derived from conventional ultrasound (US) images. The minimum redundancy and maximum relevancy and the least absolute shrinkage and selector operation algorithm were used to generate an US-based radiomics score (RS). Multivariable logistic regression analysis was used to select variables associated with HER2 expressions. The diagnostic performance of the RS was evaluated through the area under the receiver operating characteristic curve (AUC).

RESULTS

In the training set, the RS yield an AUC of 0.81 (95%CI: 0.76-0.84) for differentiation HER2-zero from HER2-low and -positive cases, and performed well in validation set (AUC 0.84, 95%CI: 0.78-0.88) and external validation set (AUC 0.82, 95%CI: 0.73-0.90). In the subgroups analysis, the RS showed good performance in distinguishing HER2-zero from HER2 1 + , HER2 2 + and HER2-low tumors (AUC range, 0.79-0.87).

CONCLUSION

The RS based on conventional US is proven effective for predicting HER2-low expression in BC.

Diagnostic performance of ultrasound-based artificial intelligence for predicting key molecular markers in breast cancer: A systematic review and meta-analysis

BACKGROUND

Breast cancer (BC) diagnosis and treatment rely heavily on molecular markers such as HER2, Ki67, PR, and ER. Currently, these markers are identified by invasive methods.

OBJECTIVE

This meta-analysis investigates the diagnostic accuracy of ultrasound-based radiomics as a novel approach to predicting these markers.

METHODS

A comprehensive search of PubMed, EMBASE, and Web of Science databases was conducted to identify studies evaluating ultrasound-based radiomics in BC. Inclusion criteria encompassed research on HER2, Ki67, PR, and ER as key molecular markers. Quality assessment using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Radiomics Quality Score (RQS) was performed. The data extraction step was performed systematically.

RESULTS

Our meta-analysis quantifies the diagnostic accuracy of ultrasound-based radiomics with a sensitivity and specificity of 0.76 and 0.78 for predicting HER2, 0.80, and 0.76 for Ki67 biomarkers. Studies did not provide sufficient data for quantitative PR and ER prediction analysis. The overall quality of studies based on the RQS tool was moderate. The QUADAS-2 evaluation showed that the studies had an unclear risk of bias regarding the flow and timing domain.

CONCLUSION

Our analysis indicated that AI models have a promising accuracy for predicting key molecular biomarkers' status in BC patients. We performed the quantitative analysis for HER2 and Ki67 biomarkers which yielded a moderate to high accuracy. However, studies did not provide adequate data for meta-analysis of ER and PR prediction accuracy of developed models. The overall quality of the studies was acceptable. In future research, studies need to report the results thoroughly. Also, we suggest more prospective studies from different centers.

Machine learning and radiomics for segmentation and classification of adnexal masses on ultrasound

Ultrasound-based models exist to support the classification of adnexal masses but are subjective and rely upon ultrasound expertise. We aimed to develop an end-to-end machine learning (ML) model capable of automating the classification of adnexal masses. In this retrospective study, transvaginal ultrasound scan images with linked diagnoses (ultrasound subjective assessment or histology) were extracted and segmented from Imperial College Healthcare, UK (ICH development dataset; n = 577 masses; 1444 images) and Morgagni-Pierantoni Hospital, Italy (MPH external dataset; n = 184 masses; 476 images). A segmentation and classification model was developed using convolutional neural networks and traditional radiomics features. Dice surface coefficient (DICE) was used to measure segmentation performance and area under the ROC curve (AUC), F1-score and recall for classification performance. The ICH and MPH datasets had a median age of 45 (IQR 35-60) and 48 (IQR 38-57) years old and consisted of 23.1% and 31.5% malignant cases, respectively. The best segmentation model achieved a DICE score of 0.85 ± 0.01, 0.88 ± 0.01 and 0.85 ± 0.01 in the ICH training, ICH validation and MPH test sets. The best classification model achieved a recall of 1.00 and F1-score of 0.88 (AUC:0.93), 0.94 (AUC:0.89) and 0.83 (AUC:0.90) in the ICH training, ICH validation and MPH test sets, respectively. We have developed an end-to-end radiomics-based model capable of adnexal mass segmentation and classification, with a comparable predictive performance (AUC 0.90) to the published performance of expert subjective assessment (gold standard), and current risk models. Further prospective evaluation of the classification performance of this ML model against existing methods is required.

Metastatic Lymph Node Detection on Ultrasound Images Using YOLOv7 in Patients with Head and Neck Squamous Cell Carcinoma

Ultrasonography is the preferred modality for detailed evaluation of enlarged lymph nodes (LNs) identified on computed tomography and/or magnetic resonance imaging, owing to its high spatial resolution. However, the diagnostic performance of ultrasonography depends on the examiner's expertise. To support the ultrasonographic diagnosis, we developed YOLOv7-based deep learning models for metastatic LN detection on ultrasonography and compared their detection performance with that of highly experienced radiologists and less experienced residents. We enrolled 462 B- and D-mode ultrasound images of 261 metastatic and 279 non-metastatic histopathologically confirmed LNs from 126 patients with head and neck squamous cell carcinoma. The YOLOv7-based B- and D-mode models were optimized using B- and D-mode training and validation images and their detection performance for metastatic LNs was evaluated using B- and D-mode testing images, respectively. The D-mode model's performance was comparable to that of radiologists and superior to that of residents' reading of D-mode images, whereas the B-mode model's performance was higher than that of residents but lower than that of radiologists on B-mode images. Thus, YOLOv7-based B- and D-mode models can assist less experienced residents in ultrasonographic diagnoses. The D-mode model could raise the diagnostic performance of residents to the same level as experienced radiologists.

Ultrasound-based radiomics model for predicting molecular biomarkers in breast cancer

Background

Breast cancer (BC) is the most common cancer in women and is highly heterogeneous. BC can be classified into four molecular subtypes based on the status of estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2) and proliferation marker protein Ki-67. However, they can only be obtained by biopsy or surgery, which is invasive. Radiomics can noninvasively predict molecular expression via extracting the image features. Nevertheless, there is a scarcity of data available regarding the prediction of molecular biomarker expression using ultrasound (US) images in BC.

Objectives

To investigate the prediction performance of US radiomics for the assessment of molecular profiling in BC.

Methods

A total of 342 patients with BC who underwent preoperative US examination between January 2013 and December 2021 were retrospectively included. They were confirmed by pathology and molecular subtype analysis of ER, PR, HER2 and Ki-67. The radiomics features were extracted and four molecular models were constructed through support vector machine (SVM). Pearson correlation coefficient heatmaps are employed to analyze the relationship between selected features and their predictive power on molecular expression. The receiver operating characteristic curve was used for the prediction performance of US radiomics in the assessment of molecular profiling.

Results

359 lesions with 129 ER- and 230 ER+, 163 PR- and 196 PR+, 265 HER2- and 94 HER2+, 114 Ki-67- and 245 Ki-67+ expression were included. 1314 features were extracted from each ultrasound image. And there was a significant difference of some specific radiomics features between the molecule positive and negative groups. Multiple features demonstrated significant association with molecular biomarkers. The area under curves (AUCs) were 0.917, 0.835, 0.771, and 0.896 in the training set, while 0.868, 0.811, 0.722, and 0.706 in the validation set to predict ER, PR, HER2, and Ki-67 expression respectively.

Conclusion

Ultrasound-based radiomics provides a promising method for predicting molecular biomarker expression of ER, PR, HER2, and Ki-67 in BC.