A deep learning image-based intrinsic molecular subtype classifier of breast tumors reveals tumor heterogeneity that may affect survival

A deep learning model to predict Ki-67 positivity in oral squamous cell carcinoma

Anatomical pathology is undergoing its third revolution, transitioning from analogical to digital pathology and incorporating new artificial intelligence technologies into clinical practice. Aside from classification, detection, and segmentation models, predictive models are gaining traction since they can impact diagnostic processes and laboratory activity, lowering consumable usage and turnaround time. Our research aimed to create a deep-learning model to generate synthetic Ki-67 immunohistochemistry from Haematoxylin and Eosin (H&E) stained images. We used 175 oral squamous cell carcinoma (OSCC) from the University Federico II's Pathology Unit's archives to train our model to generate 4 Tissue Micro Arrays (TMAs). We sectioned one slide from each TMA, first stained with H&E and then re-stained with anti-Ki-67 immunohistochemistry (IHC). In digitised slides, cores were disarrayed, and the matching cores of the 2 stained were aligned to construct a dataset to train a Pix2Pix algorithm to convert H&E images to IHC. Pathologists could recognise the synthetic images in only half of the cases in a specially designed likelihood test. Hence, our model produced realistic synthetic images. We next used QuPath to quantify IHC positivity, achieving remarkable levels of agreement between genuine and synthetic IHC. Furthermore, a categorical analysis employing 3 Ki-67 positivity cut-offs (5%, 10%, and 15%) revealed high positive-predictive values. Our model is a promising tool for collecting Ki-67 positivity information directly on H&E slides, reducing laboratory demand and improving patient management. It is also a valuable option for smaller laboratories to easily and quickly screen bioptic samples and prioritise them in a digital pathology workflow.

Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review

Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Current status and prospects of artificial intelligence in breast cancer pathology: convolutional neural networks to prospective Vision Transformers

Breast cancer is the most prevalent cancer among women, and its diagnosis requires the accurate identification and classification of histological features for effective patient management. Artificial intelligence, particularly through deep learning, represents the next frontier in cancer diagnosis and management. Notably, the use of convolutional neural networks and emerging Vision Transformers (ViT) has been reported to automate pathologists' tasks, including tumor detection and classification, in addition to improving the efficiency of pathology services. Deep learning applications have also been extended to the prediction of protein expression, molecular subtype, mutation status, therapeutic efficacy, and outcome prediction directly from hematoxylin and eosin-stained slides, bypassing the need for immunohistochemistry or genetic testing. This review explores the current status and prospects of deep learning in breast cancer diagnosis with a focus on whole-slide image analysis. Artificial intelligence applications are increasingly applied to many tasks in breast pathology ranging from disease diagnosis to outcome prediction, thus serving as valuable tools for assisting pathologists and supporting breast cancer management.

Clinical evaluation of deep learning-based risk profiling in breast cancer histopathology and comparison to an established multigene assay

PURPOSE

To evaluate the Stratipath Breast tool for image-based risk profiling and compare it with an established prognostic multigene assay for risk profiling in a real-world case series of estrogen receptor (ER)-positive and human epidermal growth factor receptor 2 (HER2)-negative early breast cancer patients categorized as intermediate risk based on classic clinicopathological variables and eligible for chemotherapy.

METHODS

In a case series comprising 234 invasive ER-positive/HER2-negative tumors, clinicopathological data including Prosigna results and corresponding HE-stained tissue slides were retrieved. The digitized HE slides were analysed by Stratipath Breast.

RESULTS

Our findings showed that the Stratipath Breast analysis identified 49.6% of the clinically intermediate tumors as low risk and 50.4% as high risk. The Prosigna assay classified 32.5%, 47.0% and 20.5% tumors as low, intermediate and high risk, respectively. Among Prosigna intermediate-risk tumors, 47.3% were stratified as Stratipath low risk and 52.7% as high risk. In addition, 89.7% of Stratipath low-risk cases were classified as Prosigna low/intermediate risk. The overall agreement between the two tests for low-risk and high-risk groups (N = 124) was 71.0%, with a Cohen's kappa of 0.42. For both risk profiling tests, grade and Ki67 differed significantly between risk groups.

CONCLUSION

The results from this clinical evaluation of image-based risk stratification shows a considerable agreement to an established gene expression assay in routine breast pathology.

Conditional generative adversarial network driven radiomic prediction of mutation status based on magnetic resonance imaging of breast cancer

BACKGROUND

Breast Cancer (BC) is a highly heterogeneous and complex disease. Personalized treatment options require the integration of multi-omic data and consideration of phenotypic variability. Radiogenomics aims to merge medical images with genomic measurements but encounter challenges due to unpaired data consisting of imaging, genomic, or clinical outcome data. In this study, we propose the utilization of a well-trained conditional generative adversarial network (cGAN) to address the unpaired data issue in radiogenomic analysis of BC. The generated images will then be used to predict the mutations status of key driver genes and BC subtypes.

METHODS

We integrated the paired MRI and multi-omic (mRNA gene expression, DNA methylation, and copy number variation) profiles of 61 BC patients from The Cancer Imaging Archive (TCIA) and The Cancer Genome Atlas (TCGA). To facilitate this integration, we employed a Bayesian Tensor Factorization approach to factorize the multi-omic data into 17 latent features. Subsequently, a cGAN model was trained based on the matched side-view patient MRIs and their corresponding latent features to predict MRIs for BC patients who lack MRIs. Model performance was evaluated by calculating the distance between real and generated images using the Fréchet Inception Distance (FID) metric. BC subtype and mutation status of driver genes were obtained from the cBioPortal platform, where 3 genes were selected based on the number of mutated patients. A convolutional neural network (CNN) was constructed and trained using the generated MRIs for mutation status prediction. Receiver operating characteristic area under curve (ROC-AUC) and precision-recall area under curve (PR-AUC) were used to evaluate the performance of the CNN models for mutation status prediction. Precision, recall and F1 score were used to evaluate the performance of the CNN model in subtype classification.

RESULTS

The FID of the images from the well-trained cGAN model based on the test set is 1.31. The CNN for TP53, PIK3CA, and CDH1 mutation prediction yielded ROC-AUC values 0.9508, 0.7515, and 0.8136 and PR-AUC are 0.9009, 0.7184, and 0.5007, respectively for the three genes. Multi-class subtype prediction achieved precision, recall and F1 scores of 0.8444, 0.8435 and 0.8336 respectively. The source code and related data implemented the algorithms can be found in the project GitHub at https://github.com/mattthuang/BC_RadiogenomicGAN .

CONCLUSION

Our study establishes cGAN as a viable tool for generating synthetic BC MRIs for mutation status prediction and subtype classification to better characterize the heterogeneity of BC in patients. The synthetic images also have the potential to significantly augment existing MRI data and circumvent issues surrounding data sharing and patient privacy for future BC machine learning studies.

Vision Transformers for Breast Cancer Human Epidermal Growth Factor Receptor 2 Expression Staging without Immunohistochemical Staining

Accurate staging of human epidermal growth factor receptor 2 (HER2) expression is vital for evaluating breast cancer treatment efficacy. However, it typically involves costly and complex immunohistochemical staining, along with hematoxylin and eosin staining. This work presents customized vision transformers for staging HER2 expression in breast cancer using only hematoxylin and eosin-stained images. The proposed algorithm comprised three modules: a localization module for weakly localizing critical image features using spatial transformers, an attention module for global learning via vision transformers, and a loss module to determine proximity to a HER2 expression level based on input images by calculating ordinal loss. Results, reported with 95% CIs, reveal the proposed approach's success in HER2 expression staging: area under the receiver operating characteristic curve, 0.9202 ± 0.01; precision, 0.922 ± 0.01; sensitivity, 0.876 ± 0.01; and specificity, 0.959 ± 0.02 over fivefold cross-validation. Comparatively, this approach significantly outperformed conventional vision transformer models and state-of-the-art convolutional neural network models (P < 0.001). Furthermore, it surpassed existing methods when evaluated on an independent test data set. This work holds great importance, aiding HER2 expression staging in breast cancer treatment while circumventing the costly and time-consuming immunohistochemical staining procedure, thereby addressing diagnostic disparities in low-resource settings and low-income countries.

A Novel High-Dimensional Kernel Joint Non-Negative Matrix Factorization With Multimodal Information for Lung Cancer Study

Judging and identifying biological activities and biomarkers inside tissues from imaging features of diseases is challenging, so correlating pathological image data with genes inside organisms is of great significance for clinical diagnosis. This paper proposes a high-dimensional kernel non-negative matrix factorization (NMF) method based on muti-modal information fusion. This algorithm can project RNA gene expression data and pathological images (WSI) into a common feature space, where the heterogeneous variables with the largest coefficient in the same projection direction form a co-module. In addition, the miRNA-mRNA and miRNA-lncRNA interaction networks in the ceRNA network are added to the algorithm as a priori information to explore the relationship between the images and the internal activities of the gene. Furthermore, the radial basis kernel function is used to calculate the feature proportion between different kinds of genes mapped in the high-dimensional feature space and projected into the common feature space to explore the gene interaction in the high-dimensional situation. The original feature matrix is regularized to improve biological correlation, and the feature factors are sparse by orthogonal constraints to reduce redundancy. Experimental results show that the proposed NMF method is better than the traditional NMF method in stability, decomposition accuracy, and robustness. Through data analysis applied to lung cancer, genes related to tissue morphology are found, such as COL7A1, CENPF and BIRC5. In addition, gene pairs with a correlation degree exceeding 0.8 are found, and potential biomarkers of significant correlation with survival are obtained such as CAPN8. It has potential application value for the clinical diagnosis of lung cancer.

iCluF: an unsupervised iterative cluster-fusion method for patient stratification using multiomics data

Motivation

Patient stratification is crucial for the effective treatment or management of heterogeneous diseases, including cancers. Multiomic technologies facilitate molecular characterization of human diseases; however, the complexity of data warrants the need for the development of robust data integration tools for patient stratification using machine-learning approaches.

Results

iCluF iteratively integrates three types of multiomic data (mRNA, miRNA, and DNA methylation) using pairwise patient similarity matrices built from each omic data. The intermediate omic-specific neighborhood matrices implement iterative matrix fusion and message passing among the similarity matrices to derive a final integrated matrix representing all the omics profiles of a patient, which is used to further cluster patients into subtypes. iCluF outperforms other methods with significant differences in the survival profiles of 8581 patients belonging to 30 different cancers in TCGA. iCluF also predicted the four intrinsic subtypes of Breast Invasive Carcinomas with adjusted rand index and Fowlkes-Mallows scores of 0.72 and 0.83, respectively. The Gini importance score showed that methylation features were the primary decisive players, followed by mRNA and miRNA to identify disease subtypes. iCluF can be applied to stratify patients with any disease containing multiomic datasets.

Availability and implementation

Source code and datasets are available at https://github.com/GudaLab/iCluF_core.

Recent Advancements in Deep Learning Using Whole Slide Imaging for Cancer Prognosis

This review furnishes an exhaustive analysis of the latest advancements in deep learning techniques applied to whole slide images (WSIs) in the context of cancer prognosis, focusing specifically on publications from 2019 through 2023. The swiftly maturing field of deep learning, in combination with the burgeoning availability of WSIs, manifests significant potential in revolutionizing the predictive modeling of cancer prognosis. In light of the swift evolution and profound complexity of the field, it is essential to systematically review contemporary methodologies and critically appraise their ramifications. This review elucidates the prevailing landscape of this intersection, cataloging major developments, evaluating their strengths and weaknesses, and providing discerning insights into prospective directions. In this paper, a comprehensive overview of the field aims to be presented, which can serve as a critical resource for researchers and clinicians, ultimately enhancing the quality of cancer care outcomes. This review's findings accentuate the need for ongoing scrutiny of recent studies in this rapidly progressing field to discern patterns, understand breakthroughs, and navigate future research trajectories.

Spatial cellular architecture predicts prognosis in glioblastoma

Intra-tumoral heterogeneity and cell-state plasticity are key drivers for the therapeutic resistance of glioblastoma. Here, we investigate the association between spatial cellular organization and glioblastoma prognosis. Leveraging single-cell RNA-seq and spatial transcriptomics data, we develop a deep learning model to predict transcriptional subtypes of glioblastoma cells from histology images. Employing this model, we phenotypically analyze 40 million tissue spots from 410 patients and identify consistent associations between tumor architecture and prognosis across two independent cohorts. Patients with poor prognosis exhibit higher proportions of tumor cells expressing a hypoxia-induced transcriptional program. Furthermore, a clustering pattern of astrocyte-like tumor cells is associated with worse prognosis, while dispersion and connection of the astrocytes with other transcriptional subtypes correlate with decreased risk. To validate these results, we develop a separate deep learning model that utilizes histology images to predict prognosis. Applying this model to spatial transcriptomics data reveal survival-associated regional gene expression programs. Overall, our study presents a scalable approach to unravel the transcriptional heterogeneity of glioblastoma and establishes a critical connection between spatial cellular architecture and clinical outcomes.

Deep learning to automatically evaluate HER2 gene amplification status from fluorescence in situ hybridization images

Human epidermal growth factor receptor 2 (HER2) gene amplification helps identify breast cancer patients who may respond to targeted anti-HER2 therapy. This study aims to develop an automated method for quantifying HER2 fluorescence in situ hybridization (FISH) signals and improve the working efficiency of pathologists. An Aitrox artificial intelligence (AI) model based on deep learning was constructed, and a comparison between the AI model and traditional manual counting was performed. In total, 918 FISH images from 320 consecutive invasive breast cancers were analysed and automatically classified into 5 groups according to the 2018 ASCO/CAP guidelines. The overall classification accuracy was 85.33% (157/184) with a mean average precision of 0.735. In Group 5, the most common group, the consistency was as high as 95.90% (117/122), while the consistency was low in the other groups due to the limited number of cases. The causes of this inconsistency, including clustered HER2 signals, coarse CEP17 signals and some section quality problems, were analysed. The developed AI model is a reliable tool for evaluating HER2 amplification statuses, especially for breast cancer in Group 5; additional cases from multiple centres could further improve the accuracy achieved for other groups.

Deep learning classification of deep ultraviolet fluorescence images toward intra-operative margin assessment in breast cancer

Background

Breast-conserving surgery is aimed at removing all cancerous cells while minimizing the loss of healthy tissue. To ensure a balance between complete resection of cancer and preservation of healthy tissue, it is necessary to assess themargins of the removed specimen during the operation. Deep ultraviolet (DUV) fluorescence scanning microscopy provides rapid whole-surface imaging (WSI) of resected tissues with significant contrast between malignant and normal/benign tissue. Intra-operative margin assessment with DUV images would benefit from an automated breast cancer classification method.

Methods

Deep learning has shown promising results in breast cancer classification, but the limited DUV image dataset presents the challenge of overfitting to train a robust network. To overcome this challenge, the DUV-WSI images are split into small patches, and features are extracted using a pre-trained convolutional neural network-afterward, a gradient-boosting tree trains on these features for patch-level classification. An ensemble learning approach merges patch-level classification results and regional importance to determine the margin status. An explainable artificial intelligence method calculates the regional importance values.

Results

The proposed method's ability to determine the DUV WSI was high with 95% accuracy. The 100% sensitivity shows that the method can detect malignant cases efficiently. The method could also accurately localize areas that contain malignant or normal/benign tissue.

Conclusion

The proposed method outperforms the standard deep learning classification methods on the DUV breast surgical samples. The results suggest that it can be used to improve classification performance and identify cancerous regions more effectively.

Biological insights and novel biomarker discovery through deep learning approaches in breast cancer histopathology

Breast cancer remains a highly prevalent disease with considerable inter- and intra-tumoral heterogeneity complicating prognostication and treatment decisions. The utilization and depth of genomic, transcriptomic and proteomic data for cancer has exploded over recent times and the addition of spatial context to this information, by understanding the correlating morphologic and spatial patterns of cells in tissue samples, has created an exciting frontier of research, histo-genomics. At the same time, deep learning (DL), a class of machine learning algorithms employing artificial neural networks, has rapidly progressed in the last decade with a confluence of technical developments - including the advent of modern graphic processing units (GPU), allowing efficient implementation of increasingly complex architectures at scale; advances in the theoretical and practical design of network architectures; and access to larger datasets for training - all leading to sweeping advances in image classification and object detection. In this review, we examine recent developments in the application of DL in breast cancer histology with particular emphasis of those producing biologic insights or novel biomarkers, spanning the extraction of genomic information to the use of stroma to predict cancer recurrence, with the aim of suggesting avenues for further advancing this exciting field.

Predicting breast cancer types on and beyond molecular level in a multi-modal fashion

Accurately determining the molecular subtypes of breast cancer is important for the prognosis of breast cancer patients and can guide treatment selection. In this study, we develop a deep learning-based model for predicting the molecular subtypes of breast cancer directly from the diagnostic mammography and ultrasound images. Multi-modal deep learning with intra- and inter-modality attention modules (MDL-IIA) is proposed to extract important relations between mammography and ultrasound for this task. MDL-IIA leads to the best diagnostic performance compared to other cohort models in predicting 4-category molecular subtypes with Matthews correlation coefficient (MCC) of 0.837 (95% confidence interval [CI]: 0.803, 0.870). The MDL-IIA model can also discriminate between Luminal and Non-Luminal disease with an area under the receiver operating characteristic curve of 0.929 (95% CI: 0.903, 0.951). These results significantly outperform clinicians' predictions based on radiographic imaging. Beyond molecular-level test, based on gene-level ground truth, our method can bypass the inherent uncertainty from immunohistochemistry test. This work thus provides a noninvasive method to predict the molecular subtypes of breast cancer, potentially guiding treatment selection for breast cancer patients and providing decision support for clinicians.

Predictive Biomarkers for Response to Immunotherapy in Triple Negative Breast Cancer: Promises and Challenges

Triple negative breast cancer (TNBC) is a highly heterogeneous disease with a poor prognosis and a paucity of therapeutic options. In recent years, immunotherapy has emerged as a new treatment option for patients with TNBC. However, this therapeutic evolution is paralleled by a growing need for biomarkers which allow for a better selection of patients who are most likely to benefit from this immune checkpoint inhibitor (ICI)-based regimen. These biomarkers will not only facilitate a better optimization of treatment strategies, but they will also avoid unnecessary side effects in non-responders, and limit the increasing financial toxicity linked to the use of these agents. Huge efforts have been deployed to identify predictive biomarkers for the ICI, but until now, the fruits of this labor remained largely unsatisfactory. Among clinically validated biomarkers, only programmed death-ligand 1 protein (PD-L1) expression has been prospectively assessed in TNBC trials. In addition to this, microsatellite instability and a high tumor mutational burden are approved as tumor agnostic biomarkers, but only a small percentage of TNBC fits this category. Furthermore, TNBC should no longer be approached as a single biological entity, but rather as a complex disease with different molecular, clinicopathological, and tumor microenvironment subgroups. This review provides an overview of the validated and evolving predictive biomarkers for a response to ICI in TNBC.

Deep Learning Based Methods for Breast Cancer Diagnosis: A Systematic Review and Future Direction

Breast cancer is one of the precarious conditions that affect women, and a substantive cure has not yet been discovered for it. With the advent of Artificial intelligence (AI), recently, deep learning techniques have been used effectively in breast cancer detection, facilitating early diagnosis and therefore increasing the chances of patients' survival. Compared to classical machine learning techniques, deep learning requires less human intervention for similar feature extraction. This study presents a systematic literature review on the deep learning-based methods for breast cancer detection that can guide practitioners and researchers in understanding the challenges and new trends in the field. Particularly, different deep learning-based methods for breast cancer detection are investigated, focusing on the genomics and histopathological imaging data. The study specifically adopts the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), which offer a detailed analysis and synthesis of the published articles. Several studies were searched and gathered, and after the eligibility screening and quality evaluation, 98 articles were identified. The results of the review indicated that the Convolutional Neural Network (CNN) is the most accurate and extensively used model for breast cancer detection, and the accuracy metrics are the most popular method used for performance evaluation. Moreover, datasets utilized for breast cancer detection and the evaluation metrics are also studied. Finally, the challenges and future research direction in breast cancer detection based on deep learning models are also investigated to help researchers and practitioners acquire in-depth knowledge of and insight into the area.

Cross-attention multi-branch CNN using DCE-MRI to classify breast cancer molecular subtypes

Purpose

The aim of this study is to improve the accuracy of classifying luminal or non-luminal subtypes of breast cancer by using computer algorithms based on DCE-MRI, and to validate the diagnostic efficacy of the model by considering the patient's age of menarche and nodule size.

Methods

DCE-MRI images of patients with non-specific invasive breast cancer admitted to the Second Affiliated Hospital of Dalian Medical University were collected. There were 160 cases in total, with 84 cases of luminal type (luminal A and luminal B and 76 cases of non-luminal type (HER 2 overexpressing and triple negative). Patients were grouped according to thresholds of nodule sizes of 20 mm and age at menarche of 14 years. A cross-attention multi-branch net CAMBNET) was proposed based on the dataset to predict the molecular subtypes of breast cancer. Diagnostic performance was assessed by accuracy, sensitivity, specificity, F1 and area under the ROC curve (AUC). And the model is visualized with Grad-CAM.

Results

Several classical deep learning models were included for diagnostic performance comparison. Using 5-fold cross-validation on the test dataset, all the results of CAMBNET are significantly higher than the compared deep learning models. The average prediction recall, accuracy, precision, and AUC for luminal and non-luminal types of the dataset were 89.11%, 88.44%, 88.52%, and 96.10%, respectively. For patients with tumor size <20 mm, the CAMBNET had AUC of 83.45% and ACC of 90.29% for detecting triple-negative breast cancer. When classifying luminal from non-luminal subtypes for patients with age at menarche years, our CAMBNET model achieved an ACC of 92.37%, precision of 92.42%, recall of 93.33%, F1of 92.33%, and AUC of 99.95%.

Conclusions

The CAMBNET can be applied in molecular subtype classification of breasts. For patients with menarche at 14 years old, our model can yield more accurate results when classifying luminal and non-luminal subtypes. For patients with tumor sizes ≤20 mm, our model can yield more accurate result in detecting triple-negative breast cancer to improve patient prognosis and survival.

Correlation of NNMT and DKK1 Protein Expression With Clinicopathological Characteristics and Prognosis of Breast Cancer

Background

Nicotinamide N-methyltransferase (NNMT) and Dickkopf-1 (DKK1) play an important role in the development of breast cancer, and the purpose of this study was designed to examine the clinical and prognostic significance of NNMT and DKK1 in breast cancer.

Methods

The GEPIA2 database was used to evaluate the expression and survival of NNMT mRNA and DKK1 mRNA of breast cancer. Then an immunohistochemical study was carried out on 374 cases of breast tissue to identify the protein expression and significance of NNMT and DKK1. Next, the prognostic significance of DKK1 in breast cancer was explored by COX and Kaplan-Meier models.

Results

Protein NNMT expression was correlated with lymph node metastasis and histological grade (P < .05) while protein DKK1 expression was related to tumor size, pT stage, histological grade, and Ki-67 (P < .05). Protein DKK1 was related to disease-specific survival (DSS), and low DKK1 expression indicated a poor prognosis of breast cancer patients (P < .05). Combined expression of protein NNMT and protein DKK1 predicted different prognosis of DSS (P < .05).

Conclusions

Nicotinamide N-methyltransferase and DKK1 were linked to breast cancer malignancy and invasion. Breast cancer patients with low DKK1 expression had a worse prognosis. Oncotypes of NNMT and DKK1 expression predicted patient outcomes.

Artificial intelligence in breast cancer histopathology

This is a review on the use of artificial intelligence for digital breast pathology. A systematic search on PubMed was conducted, identifying 17,324 research papers related to breast cancer pathology. Following a semimanual screening, 664 papers were retrieved and pursued. The papers are grouped into six major tasks performed by pathologists-namely, molecular and hormonal analysis, grading, mitotic figure counting, ki-67 indexing, tumour-infiltrating lymphocyte assessment, and lymph node metastases identification. Under each task, open-source datasets for research to build artificial intelligence (AI) tools are also listed. Many AI tools showed promise and demonstrated feasibility in the automation of routine pathology investigations. We expect continued growth of AI in this field as new algorithms mature.

Deep Learning-Based Prediction of Molecular Tumor Biomarkers from H&E: A Practical Review

Molecular and genomic properties are critical in selecting cancer treatments to target individual tumors, particularly for immunotherapy. However, the methods to assess such properties are expensive, time-consuming, and often not routinely performed. Applying machine learning to H&E images can provide a more cost-effective screening method. Dozens of studies over the last few years have demonstrated that a variety of molecular biomarkers can be predicted from H&E alone using the advancements of deep learning: molecular alterations, genomic subtypes, protein biomarkers, and even the presence of viruses. This article reviews the diverse applications across cancer types and the methodology to train and validate these models on whole slide images. From bottom-up to pathologist-driven to hybrid approaches, the leading trends include a variety of weakly supervised deep learning-based approaches, as well as mechanisms for training strongly supervised models in select situations. While results of these algorithms look promising, some challenges still persist, including small training sets, rigorous validation, and model explainability. Biomarker prediction models may yield a screening method to determine when to run molecular tests or an alternative when molecular tests are not possible. They also create new opportunities in quantifying intratumoral heterogeneity and predicting patient outcomes.

Using Deep Learning to Predict Final HER2 Status in Invasive Breast Cancers That are Equivocal (2+) by Immunohistochemistry

Invasive breast carcinomas are routinely tested for HER2 using immunohistochemistry (IHC), with reflex in situ hybridization (ISH) for those scored as equivocal (2+). ISH testing is expensive, time-consuming, and not universally available. In this study, we trained a deep learning algorithm to directly predict HER2 gene amplification status from HER2 2+ IHC slides. Data included 115 consecutive cases of invasive breast carcinoma scored as 2+ by IHC that had follow-up HER2 ISH testing. An external validation data set was created from 36 HER2 IHC slides prepared at an outside institution. All internal IHC slides were digitized and divided into training (80%), and test (20%) sets with 5-fold cross-validation. Small patches (256×256 pixels) were randomly extracted and used to train convolutional neural networks with EfficientNet B0 architecture using a transfer learning approach. Predictions for slides in the test set were made on individual patches, and these predictions were aggregated to generate an overall prediction for each slide. This resulted in a receiver operating characteristic area under the curve of 0.83 with an overall accuracy of 79% (sensitivity=0.70, specificity=0.82). Analysis of external validation slides resulted in a receiver operating characteristic area under the curve of 0.79 with an overall accuracy of 81% (sensitivity=0.50, specificity=0.82). Although the sensitivity and specificity are not high enough to negate the need for reflexive ISH testing entirely, this approach may be useful for triaging cases more likely to be HER2 positive and initiating treatment planning in centers where HER2 ISH testing is not readily available.

Prediction of Epstein-Barr Virus Status in Gastric Cancer Biopsy Specimens Using a Deep Learning Algorithm

IMPORTANCE

Epstein-Barr virus (EBV)-associated gastric cancer (EBV-GC) is 1 of 4 molecular subtypes of GC and is confirmed by an expensive molecular test, EBV-encoded small RNA in situ hybridization. EBV-GC has 2 histologic characteristics, lymphoid stroma and lace-like tumor pattern, but projecting EBV-GC at biopsy is difficult even for experienced pathologists.

OBJECTIVE

To develop and validate a deep learning algorithm to predict EBV status from pathology images of GC biopsy.

DESIGN, SETTING, AND PARTICIPANTS

This diagnostic study developed a deep learning classifier to predict EBV-GC using image patches of tissue microarray (TMA) and whole slide images (WSIs) of GC and applied it to GC biopsy specimens from GCs diagnosed at Kangbuk Samsung Hospital between 2011 and 2020. For a quantitative evaluation and EBV-GC prediction on biopsy specimens, the area of each class and the fraction in total tissue or tumor area were calculated. Data were analyzed from March 5, 2021, to February 10, 2022.

MAIN OUTCOMES AND MEASURES

Evaluation metrics of predictive model performance were assessed on accuracy, recall, precision, F1 score, area under the receiver operating characteristic curve (AUC), and κ coefficient.

RESULTS

This study included 137 184 image patches from 16 TMAs (708 tissue cores), 24 WSIs, and 286 biopsy images of GC. The classifier was able to classify EBV-GC image patches from TMAs and WSIs with 94.70% accuracy, 0.936 recall, 0.938 precision, 0.937 F1 score, and 0.909 κ coefficient. The classifier was used for predicting and measuring the area and fraction of EBV-GC on biopsy tissue specimens. A 10% cutoff value for the predicted fraction of EBV-GC to tissue (EBV-GC/tissue area) produced the best prediction results in EBV-GC biopsy specimens and showed the highest AUC value (0.8723; 95% CI, 0.7560-0.9501). That cutoff also obtained high sensitivity (0.895) and moderate specificity (0.745) compared with experienced pathologist sensitivity (0.842) and specificity (0.854) when using the presence of lymphoid stroma and a lace-like pattern as diagnostic criteria. On prediction maps, EBV-GCs with lace-like pattern and lymphoid stroma showed the same prediction results as EBV-GC, but cases lacking these histologic features revealed heterogeneous prediction results of EBV-GC and non-EBV-GC areas.

CONCLUSIONS AND RELEVANCE

This study showed the feasibility of EBV-GC prediction using a deep learning algorithm, even in biopsy samples. Use of such an image-based classifier before a confirmatory molecular test will reduce costs and tissue waste.

Deep learning-based prediction of molecular cancer biomarkers from tissue slides: A new tool for precision oncology

Molecular tests are necessary to stratify cancer patients for targeted therapy. However, high cost and technical barriers limit the application of these tests, hindering optimal treatment. Recently, deep learning (DL) has been applied to predict molecular test results from digitized images of tissue slides. Furthermore, treatment response and prognosis can be predicted from tissue slides using DL. In this review, we summarized DL-based studies regarding the prediction of genetic mutation, microsatellite instability, tumor mutational burden, molecular subtypes, gene expression, treatment response, and prognosis directly from hematoxylin- and eosin-stained tissue slides. Although performance needs to be improved, these studies clearly demonstrated the feasibility of DL-based prediction of key molecular features in cancer tissues. With the accumulation of data and technical advances, the performance of the DL system could be improved in the near future. Therefore, we expect that DL could provide cost- and time-effective alternative tools for patient stratification in the era of precision oncology.

DEMoS: a deep learning-based ensemble approach for predicting the molecular subtypes of gastric adenocarcinomas from histopathological images

MOTIVATION

The molecular subtyping of gastric cancer (adenocarcinoma) into four main subtypes based on integrated multiomics profiles, as proposed by The Cancer Genome Atlas (TCGA) initiative, represents an effective strategy for patient stratification. However, this approach requires the use of multiple technological platforms, and is quite expensive and time-consuming to perform. A computational approach that uses histopathological image data to infer molecular subtypes could be a practical, cost- and time-efficient complementary tool for prognostic and clinical management purposes.

RESULTS

Here, we propose a deep learning ensemble approach (called DEMoS) capable of predicting the four recognized molecular subtypes of gastric cancer directly from histopathological images. DEMoS achieved tile-level area under the receiver-operating characteristic curve (AUROC) values of 0.785, 0.668, 0.762 and 0.811 for the prediction of these four subtypes of gastric cancer [i.e. (i) Epstein-Barr (EBV)-infected, (ii) microsatellite instability (MSI), (iii) genomically stable (GS) and (iv) chromosomally unstable tumors (CIN)] using an independent test dataset, respectively. At the patient-level, it achieved AUROC values of 0.897, 0.764, 0.890 and 0.898, respectively. Thus, these four subtypes are well-predicted by DEMoS. Benchmarking experiments further suggest that DEMoS is able to achieve an improved classification performance for image-based subtyping and prevent model overfitting. This study highlights the feasibility of using a deep learning ensemble-based method to rapidly and reliably subtype gastric cancer (adenocarcinoma) solely using features from histopathological images.

AVAILABILITY AND IMPLEMENTATION

All whole slide images used in this study was collected from the TCGA database. This study builds upon our previously published HEAL framework, with related documentation and tutorials available at http://heal.erc.monash.edu.au. The source code and related models are freely accessible at https://github.com/Docurdt/DEMoS.git.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Untangling the web of intratumour heterogeneity

Intratumour heterogeneity (ITH) is a hallmark of cancer that drives tumour evolution and disease progression. Technological and computational advances have enabled us to assess ITH at unprecedented depths, yet this accumulating knowledge has not had a substantial clinical impact. This is in part due to a limited understanding of the functional relevance of ITH and the inadequacy of preclinical experimental models to reproduce it. Here, we discuss progress made in these areas and illuminate future directions.

Deep learning for survival analysis in breast cancer with whole slide image data

MOTIVATION

Whole slide tissue images contain detailed data on the sub-cellular structure of cancer. Quantitative analyses of this data can lead to novel biomarkers for better cancer diagnosis and prognosis and can improve our understanding of cancer mechanisms. Such analyses are challenging to execute because of the sizes and complexity of whole slide image data and relatively limited volume of training data for machine learning methods.

RESULTS

We propose and experimentally evaluate a multi-resolution deep learning method for breast cancer survival analysis. The proposed method integrates image data at multiple resolutions and tumor, lymphocyte and nuclear segmentation results from deep learning models. Our results show that this approach can significantly improve the deep learning model performance compared to using only the original image data. The proposed approach achieves a c-index value of 0.706 compared to a c-index value of 0.551 from an approach that uses only color image data at the highest image resolution. Furthermore, when clinical features (sex, age and cancer stage) are combined with image data, the proposed approach achieves a c-index of 0.773.

AVAILABILITY AND IMPLEMENTATION

https://github.com/SBU-BMI/deep_survival_analysis.

Deep Learning-Based Pathology Image Analysis Enhances Magee Feature Correlation With Oncotype DX Breast Recurrence Score

Background

Oncotype DX Recurrence Score (RS) has been widely used to predict chemotherapy benefits in patients with estrogen receptor-positive breast cancer. Studies showed that the features used in Magee equations correlate with RS. We aimed to examine whether deep learning (DL)-based histology image analyses can enhance such correlations.

Methods

We retrieved 382 cases with RS diagnosed between 2011 and 2015 from the Emory University and the Ohio State University. All patients received surgery. DL models were developed to detect nuclei of tumor cells and tumor-infiltrating lymphocytes (TILs) and segment tumor cell nuclei in hematoxylin and eosin (H&E) stained histopathology whole slide images (WSIs). Based on the DL-based analysis, we derived image features from WSIs, such as tumor cell number, TIL number variance, and nuclear grades. The entire patient cohorts were divided into one training set (125 cases) and two validation sets (82 and 175 cases) based on the data sources and WSI resolutions. The training set was used to train the linear regression models to predict RS. For prediction performance comparison, we used independent variables from Magee features alone or the combination of WSI-derived image and Magee features.

Results

The Pearson's correlation coefficients between the actual RS and predicted RS by DL-based analysis were 0.7058 (p-value = 1.32 × 10-13) and 0.5041 (p-value = 1.15 × 10-12) for the validation sets 1 and 2, respectively. The adjusted R 2 values using Magee features alone are 0.3442 and 0.2167 in the two validation sets, respectively. In contrast, the adjusted R 2 values were enhanced to 0.4431 and 0.2182 when WSI-derived imaging features were jointly used with Magee features.

Conclusion

Our results suggest that DL-based digital pathological features can enhance Magee feature correlation with RS.

Global Increase in Breast Cancer Incidence: Risk Factors and Preventive Measures

Breast cancer is a global cause for concern owing to its high incidence around the world. The alarming increase in breast cancer cases emphasizes the management of disease at multiple levels. The management should start from the beginning that includes stringent cancer screening or cancer registry to effective diagnostic and treatment strategies. Breast cancer is highly heterogeneous at morphology as well as molecular levels and needs different therapeutic regimens based on the molecular subtype. Breast cancer patients with respective subtype have different clinical outcome prognoses. Breast cancer heterogeneity emphasizes the advanced molecular testing that will help on-time diagnosis and improved survival. Emerging fields such as liquid biopsy and artificial intelligence would help to under the complexity of breast cancer disease and decide the therapeutic regimen that helps in breast cancer management. In this review, we have discussed various risk factors and advanced technology available for breast cancer diagnosis to combat the worst breast cancer status and areas that need to be focused for the better management of breast cancer.

Breast Cancer Molecular Subtype Prediction on Pathological Images with Discriminative Patch Selection and Multi-Instance Learning

Molecular subtypes of breast cancer are important references to personalized clinical treatment. For cost and labor savings, only one of the patient’s paraffin blocks is usually selected for subsequent immunohistochemistry (IHC) to obtain molecular subtypes. Inevitable block sampling error is risky due to the tumor heterogeneity and could result in a delay in treatment. Molecular subtype prediction from conventional H&amp;E pathological whole slide images (WSI) using the AI method is useful and critical to assist pathologists to pre-screen proper paraffin block for IHC. It is a challenging task since only WSI-level labels of molecular subtypes from IHC can be obtained without detailed local region information. Gigapixel WSIs are divided into a huge amount of patches to be computationally feasible for deep learning, while with coarse slide-level labels, patch-based methods may suffer from abundant noise patches, such as folds, overstained regions, or non-tumor tissues. A weakly supervised learning framework based on discriminative patch selection and multi-instance learning was proposed for breast cancer molecular subtype prediction from H&amp;E WSIs. Firstly, co-teaching strategy using two networks was adopted to learn molecular subtype representations and filter out some noise patches. Then, a balanced sampling strategy was used to handle the imbalance in subtypes in the dataset. In addition, a noise patch filtering algorithm that used local outlier factor based on cluster centers was proposed to further select discriminative patches. Finally, a loss function integrating local patch with global slide constraint information was used to fine-tune MIL framework on obtained discriminative patches and further improve the prediction performance of molecular subtyping. The experimental results confirmed the effectiveness of the proposed AI method and our models outperformed even senior pathologists, which has the potential to assist pathologists to pre-screen paraffin blocks for IHC in clinic.

Association of machine learning ultrasound radiomics and disease outcome in triple negative breast cancer

Triple negative breast cancer (TNBC) is a breast cancer subtype with unfavorable prognosis. We aimed to establish a machine learning-based ultrasound radiomics model to predict disease-free survival (DFS) in TNBC. Invasive TNBC>T1b between January 2009 and June 2018 with preoperative ultrasound were enrolled and assigned to training and independent test cohort. Radiomics and clinicopathological features related with DFS were selected by univariate and multivariate regression analysis. Training cohort of combined features was resampled with SMOTEENN to balance distribution and put into classifiers. Areas Under Curves (AUCs) of models were compared by DeLong's test. 562 women were included with 68 DFS events observed. Twenty prognostic radiomics features were extracted. Machine learning model by Naïve Bayes combining radiomics, clinicopathological features, and SMOTEENN had an AUC of 0.86 (95% CI 0.84-0.88), with sensitivity of 74.7% and specificity of 80.1% in training cohort. In independent test cohort, this three-combination model delivered an AUC of 0.90 (95% CI 0.83-0.95), higher than models based on radiomics (AUC=0.69, P=0.016) or radiomics + SMOTEENN (AUC=0.73, P=0.019). Integrating machine learning radiomics model based on ultrasound and clinicopathological features can predict DFS events for TNBC patients.

L’intelligence artificielle pourrait-elle intervenir dans l’aide au diagnostic des cancers du sein ? – L’exemple de HER2

HER2 is an important prognostic and predictive biomarker in breast cancer. Its detection makes it possible to define which patients will benefit from a targeted treatment. While assessment of HER2 status by immunohistochemistry in positive vs negative categories is well implemented and reproducible, the introduction of a new "HER2-low" category could raise some concerns about its scoring and reproducibility. We herein described the current HER2 testing methods and the application of innovative machine learning techniques to improve these determinations, as well as the main challenges and opportunities related to the implementation of digital pathology in the up-and-coming AI era.

Predicting Breast Cancer Gene Expression Signature by Applying Deep Convolutional Neural Networks From Unannotated Pathological Images

We proposed a highly versatile two-step transfer learning pipeline for predicting the gene signature defining the intrinsic breast cancer subtypes using unannotated pathological images. Deciphering breast cancer molecular subtypes by deep learning approaches could provide a convenient and efficient method for the diagnosis of breast cancer patients. It could reduce costs associated with transcriptional profiling and subtyping discrepancy between IHC assays and mRNA expression. Four pretrained models such as VGG16, ResNet50, ResNet101, and Xception were trained with our in-house pathological images from breast cancer patient with recurrent status in the first transfer learning step and TCGA-BRCA dataset for the second transfer learning step. Furthermore, we also trained ResNet101 model with weight from ImageNet for comparison to the aforementioned models. The two-step deep learning models showed promising classification results of the four breast cancer intrinsic subtypes with accuracy ranging from 0.68 (ResNet50) to 0.78 (ResNet101) in both validation and testing sets. Additionally, the overall accuracy of slide-wise prediction showed even higher average accuracy of 0.913 with ResNet101 model. The micro- and macro-average area under the curve (AUC) for these models ranged from 0.88 (ResNet50) to 0.94 (ResNet101), whereas ResNet101_imgnet weighted with ImageNet archived an AUC of 0.92. We also show the deep learning model prediction performance is significantly improved relatively to the common Genefu tool for breast cancer classification. Our study demonstrated the capability of deep learning models to classify breast cancer intrinsic subtypes without the region of interest annotation, which will facilitate the clinical applicability of the proposed models.

Integrative multiomics-histopathology analysis for breast cancer classification

Histopathologic evaluation of biopsy slides is a critical step in diagnosing and subtyping breast cancers. However, the connections between histology and multi-omics status have never been systematically explored or interpreted. We developed weakly supervised deep learning models over hematoxylin-and-eosin-stained slides to examine the relations between visual morphological signal, clinical subtyping, gene expression, and mutation status in breast cancer. We first designed fully automated models for tumor detection and pathology subtype classification, with the results validated in independent cohorts (area under the receiver operating characteristic curve ≥ 0.950). Using only visual information, our models achieved strong predictive performance in estrogen/progesterone/HER2 receptor status, PAM50 status, and TP53 mutation status. We demonstrated that these models learned lymphocyte-specific morphological signals to identify estrogen receptor status. Examination of the PAM50 cohort revealed a subset of PAM50 genes whose expression reflects cancer morphology. This work demonstrates the utility of deep learning-based image models in both clinical and research regimes, through its ability to uncover connections between visual morphology and genetic statuses.

Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides

Objectives

To develop and validate a deep learning (DL)-based primary tumor biopsy signature for predicting axillary lymph node (ALN) metastasis preoperatively in early breast cancer (EBC) patients with clinically negative ALN.

Methods

A total of 1,058 EBC patients with pathologically confirmed ALN status were enrolled from May 2010 to August 2020. A DL core-needle biopsy (DL-CNB) model was built on the attention-based multiple instance-learning (AMIL) framework to predict ALN status utilizing the DL features, which were extracted from the cancer areas of digitized whole-slide images (WSIs) of breast CNB specimens annotated by two pathologists. Accuracy, sensitivity, specificity, receiver operating characteristic (ROC) curves, and areas under the ROC curve (AUCs) were analyzed to evaluate our model.

Results

The best-performing DL-CNB model with VGG16_BN as the feature extractor achieved an AUC of 0.816 (95% confidence interval (CI): 0.758, 0.865) in predicting positive ALN metastasis in the independent test cohort. Furthermore, our model incorporating the clinical data, which was called DL-CNB+C, yielded the best accuracy of 0.831 (95%CI: 0.775, 0.878), especially for patients younger than 50 years (AUC: 0.918, 95%CI: 0.825, 0.971). The interpretation of DL-CNB model showed that the top signatures most predictive of ALN metastasis were characterized by the nucleus features including density (p = 0.015), circumference (p = 0.009), circularity (p = 0.010), and orientation (p = 0.012).

Conclusion

Our study provides a novel DL-based biomarker on primary tumor CNB slides to predict the metastatic status of ALN preoperatively for patients with EBC.

Machine Learning Techniques for Personalised Medicine Approaches in Immune-Mediated Chronic Inflammatory Diseases: Applications and Challenges

In the past decade, the emergence of machine learning (ML) applications has led to significant advances towards implementation of personalised medicine approaches for improved health care, due to the exceptional performance of ML models when utilising complex big data. The immune-mediated chronic inflammatory diseases are a group of complex disorders associated with dysregulated immune responses resulting in inflammation affecting various organs and systems. The heterogeneous nature of these diseases poses great challenges for tailored disease management and addressing unmet patient needs. Applying novel ML techniques to the clinical study of chronic inflammatory diseases shows promising results and great potential for precision medicine applications in clinical research and practice. In this review, we highlight the clinical applications of various ML techniques for prediction, diagnosis and prognosis of autoimmune rheumatic diseases, inflammatory bowel disease, autoimmune chronic kidney disease, and multiple sclerosis, as well as ML applications for patient stratification and treatment selection. We highlight the use of ML in drug development, including target identification, validation and drug repurposing, as well as challenges related to data interpretation and validation, and ethical concerns related to the use of artificial intelligence in clinical research.

Deep Learning of Histopathological Features for the Prediction of Tumour Molecular Genetics

Advanced diagnostics are enabling cancer treatments to become increasingly tailored to the individual through developments in immunotherapies and targeted therapies. However, long turnaround times and high costs of molecular testing hinder the widespread implementation of targeted cancer treatments. Meanwhile, gold-standard histopathological assessment carried out by a trained pathologist is widely regarded as routine and mandatory in most cancers. Recently, methods have been developed to mine hidden information from histopathological slides using deep learning applied to scanned and digitized slides; deep learning comprises a collection of computational methods which learn patterns in data in order to make predictions. Such methods have been reported to be successful in a variety of cancers for predicting the presence of biomarkers such as driver mutations, tumour mutational burden, and microsatellite instability. This information could prove valuable to pathologists and oncologists in clinical decision making for cancer treatment and triage for in-depth sequencing. In addition to identifying molecular features, deep learning has been applied to predict prognosis and treatment response in certain cancers. Despite reported successes, many challenges remain before the clinical implementation of such diagnostic strategies in the clinical setting is possible. This review aims to outline recent developments in the field of deep learning for predicting molecular genetics from histopathological slides, as well as to highlight limitations and pitfalls of working with histopathology slides in deep learning.

The impact of site-specific digital histology signatures on deep learning model accuracy and bias

The Cancer Genome Atlas (TCGA) is one of the largest biorepositories of digital histology. Deep learning (DL) models have been trained on TCGA to predict numerous features directly from histology, including survival, gene expression patterns, and driver mutations. However, we demonstrate that these features vary substantially across tissue submitting sites in TCGA for over 3,000 patients with six cancer subtypes. Additionally, we show that histologic image differences between submitting sites can easily be identified with DL. Site detection remains possible despite commonly used color normalization and augmentation methods, and we quantify the image characteristics constituting this site-specific digital histology signature. We demonstrate that these site-specific signatures lead to biased accuracy for prediction of features including survival, genomic mutations, and tumor stage. Furthermore, ethnicity can also be inferred from site-specific signatures, which must be accounted for to ensure equitable application of DL. These site-specific signatures can lead to overoptimistic estimates of model performance, and we propose a quadratic programming method that abrogates this bias by ensuring models are not trained and validated on samples from the same site.

On the stability of log-rank test under labeling errors

Motivation

Log-rank test is a widely used test that serves to assess the statistical significance of observed differences in survival, when comparing two or more groups. The log-rank test is based on several assumptions that support the validity of the calculations. It is naturally assumed, implicitly, that no errors occur in the labeling of the samples. That is, the mapping between samples and groups is perfectly correct. In this work, we investigate how test results may be affected when considering some errors in the original labeling.

Results

We introduce and define the uncertainty that arises from labeling errors in log-rank test. In order to deal with this uncertainty, we develop a novel algorithm for efficiently calculating a stability interval around the original log-rank P-value and prove its correctness. We demonstrate our algorithm on several datasets.

Availability and implementation

We provide a Python implementation, called LoRSI, for calculating the stability interval using our algorithm https://github.com/YakhiniGroup/LoRSI.

Supplementary information

Supplementary data are available at Bioinformatics online.

Current Triple-Negative Breast Cancer Subtypes: Dissecting the Most Aggressive Form of Breast Cancer

Triple-negative breast cancer (TNBC) is a highly heterogeneous disease defined by the absence of estrogen receptor (ER) and progesterone receptor (PR) expression, and human epidermal growth factor receptor 2 (HER2) overexpression that lacks targeted treatments, leading to dismal clinical outcomes. Thus, better stratification systems that reflect intrinsic and clinically useful differences between TNBC tumors will sharpen the treatment approaches and improve clinical outcomes. The lack of a rational classification system for TNBC also impacts current and emerging therapeutic alternatives. In the past years, several new methodologies to stratify TNBC have arisen thanks to the implementation of microarray technology, high-throughput sequencing, and bioinformatic methods, exponentially increasing the amount of genomic, epigenomic, transcriptomic, and proteomic information available. Thus, new TNBC subtypes are being characterized with the promise to advance the treatment of this challenging disease. However, the diverse nature of the molecular data, the poor integration between the various methods, and the lack of cost-effective methods for systematic classification have hampered the widespread implementation of these promising developments. However, the advent of artificial intelligence applied to translational oncology promises to bring light into definitive TNBC subtypes. This review provides a comprehensive summary of the available classification strategies. It includes evaluating the overlap between the molecular, immunohistochemical, and clinical characteristics between these approaches and a perspective about the increasing applications of artificial intelligence to identify definitive and clinically relevant TNBC subtypes.

Artificial intelligence in early drug discovery enabling precision medicine

Introduction: Precision medicine is the concept of treating diseases based on environmental factors, lifestyles, and molecular profiles of patients. This approach has been found to increase success rates of clinical trials and accelerate drug approvals. However, current precision medicine applications in early drug discovery use only a handful of molecular biomarkers to make decisions, whilst clinics gear up to capture the full molecular landscape of patients in the near future. This deep multi-omics characterization demands new analysis strategies to identify appropriate treatment regimens, which we envision will be pioneered by artificial intelligence.Areas covered: In this review, the authors discuss the current state of drug discovery in precision medicine and present our vision of how artificial intelligence will impact biomarker discovery and drug design.Expert opinion: Precision medicine is expected to revolutionize modern medicine; however, its traditional form is focusing on a few biomarkers, thus not equipped to leverage the full power of molecular landscapes. For learning how the development of drugs can be tailored to the heterogeneity of patients across their molecular profiles, artificial intelligence algorithms are the next frontier in precision medicine and will enable a fully personalized approach in drug design, and thus ultimately impacting clinical practice.

Deep Learning-Based Prediction Model for Breast Cancer Recurrence Using Adjuvant Breast Cancer Cohort in Tertiary Cancer Center Registry

Several prognosis prediction models have been developed for breast cancer (BC) patients with curative surgery, but there is still an unmet need to precisely determine BC prognosis for individual BC patients in real time. This is a retrospectively collected data analysis from adjuvant BC registry at Samsung Medical Center between January 2000 and December 2016. The initial data set contained 325 clinical data elements: baseline characteristics with demographics, clinical and pathologic information, and follow-up clinical information including laboratory and imaging data during surveillance. Weibull Time To Event Recurrent Neural Network (WTTE-RNN) by Martinsson was implemented for machine learning. We searched for the optimal window size as time-stamped inputs. To develop the prediction model, data from 13,117 patients were split into training (60%), validation (20%), and test (20%) sets. The median follow-up duration was 4.7 years and the median number of visits was 8.4. We identified 32 features related to BC recurrence and considered them in further analyses. Performance at a point of statistics was calculated using Harrell's C-index and area under the curve (AUC) at each 2-, 5-, and 7-year points. After 200 training epochs with a batch size of 100, the C-index reached 0.92 for the training data set and 0.89 for the validation and test data sets. The AUC values were 0.90 at 2-year point, 0.91 at 5-year point, and 0.91 at 7-year point. The deep learning-based final model outperformed three other machine learning-based models. In terms of pathologic characteristics, the median absolute error (MAE) and weighted mean absolute error (wMAE) showed great results of as little as 3.5%. This BC prognosis model to determine the probability of BC recurrence in real time was developed using information from the time of BC diagnosis and the follow-up period in RNN machine learning model.

Bevacizumab Plays a double-edged role in Neoadjuvant Therapy for Non-metastatic Breast Cancer: A Systemic Review and Meta-Analysis

The anti-angiogenic drug Bevacizumab (Bev) is engaged in neoadjuvant therapy for non-metastatic breast cancer (NMBC). However, whether neoadjuvant Bev providing a greater benefit to patients is debatable. Our study aimed to review Bev's role in Neoadjuvant therapy (NAT) in NMBC and identify predictive markers associated with its efficacy by systemic review and meta-analysis. Eligible trials were retrieved from the Pubmed, Embase, and Cochrane Library, and random or fixed effects models were applied to synthesize data. Power of pCR to predict DFS or OS was evaluated by nonlinear mixed effect model. In NMBC, Bev significantly improved the rate of patients achieving pCR, but this benefit discontinued in DFS or OS. Biomarkers such as PAM50 intrinsic subtype, VEGF overexpression, regulation of VEGF signaling pathway, hypoxia-related genes, BRCA1/2 mutation, P53 mutation and immune phenotype can be used to predict Bev-inducing pCR and/or DFS/OS. Unfortunately, although patients with pCR survived longer than those without pCR when ignoring the use of Bev, but patients achieving pCR with Bev might survive shorter than those achieving pCR without Bev. Subgroup analyses found Bev prolonged patients' OS when given pre- and post-surgery. Lastly, adding Bev increased adverse effects. Overall, Bev offered limited effect for patients with NMBC in an unscreened population. However, in biomarkers - identified subgroup, Bev could be promising to ameliorate the prognosis of specific patients with NMBC.

Intratumoral heterogeneity of second-harmonic generation scattering from tumor collagen and its effects on metastatic risk prediction

Background

Metastases are the leading cause of breast cancer-related deaths. The tumor microenvironment impacts cancer progression and metastatic ability. Fibrillar collagen, a major extracellular matrix component, can be studied using the light scattering phenomenon known as second-harmonic generation (SHG). The ratio of forward- to backward-scattered SHG photons (F/B) is sensitive to collagen fiber internal structure and has been shown to be an independent prognostic indicator of metastasis-free survival time (MFS). Here we assess the effects of heterogeneity in the tumor matrix on the possible use of F/B as a prognostic tool.

Methods

SHG imaging was performed on sectioned primary tumor excisions from 95 untreated, estrogen receptor-positive, lymph node negative invasive ductal carcinoma patients. We identified two distinct regions whose collagen displayed different average F/B values, indicative of spatial heterogeneity: the cellular tumor bulk and surrounding tumor-stroma interface. To evaluate the impact of heterogeneity on F/B’s prognostic ability, we performed SHG imaging in the tumor bulk and tumor-stroma interface, calculated a 21-gene recurrence score (surrogate for OncotypeDX®, or S-ODX) for each patient and evaluated their combined prognostic ability.

Results

We found that F/B measured in tumor-stroma interface, but not tumor bulk, is prognostic of MFS using three methods to select pixels for analysis: an intensity threshold selected by a blinded observer, a histogram-based thresholding method, and an adaptive thresholding method. Using both regression trees and Random Survival Forests for MFS outcome, we obtained data-driven prediction rules that show F/B from tumor-stroma interface, but not tumor bulk, and S-ODX both contribute to predicting MFS in this patient cohort. We also separated patients into low-intermediate (S-ODX < 26) and high risk (S-ODX ≥26) groups. In the low-intermediate risk group, comprised of patients not typically recommended for adjuvant chemotherapy, we find that F/B from the tumor-stroma interface is prognostic of MFS and can identify a patient cohort with poor outcomes.

Conclusions

These data demonstrate that intratumoral heterogeneity in F/B values can play an important role in its possible use as a prognostic marker, and that F/B from tumor-stroma interface of primary tumor excisions may provide useful information to stratify patients by metastatic risk.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-020-07713-4.

Deep Learning-Based Pixel-Wise Lesion Segmentation on Oral Squamous Cell Carcinoma Images

Oral squamous cell carcinoma is the most common oral cancer. In this paper, we present a performance analysis of four different deep learning-based pixel-wise methods for lesion segmentation on oral carcinoma images. Two diverse image datasets, one for training and another one for testing, are used to generate and evaluate the models used for segmenting the images, thus allowing to assess the generalization capability of the considered deep network architectures. An important contribution of this work is the creation of the Oral Cancer Annotated (ORCA) dataset, containing ground-truth data derived from the well-known Cancer Genome Atlas (TCGA) dataset.

Prediction of breast cancer molecular subtypes on DCE-MRI using convolutional neural network with transfer learning between two centers

OBJECTIVES

To apply deep learning algorithms using a conventional convolutional neural network (CNN) and a recurrent CNN to differentiate three breast cancer molecular subtypes on MRI.

METHODS

A total of 244 patients were analyzed, 99 in training dataset scanned at 1.5 T and 83 in testing-1 and 62 in testing-2 scanned at 3 T. Patients were classified into 3 subtypes based on hormonal receptor (HR) and HER2 receptor: (HR+/HER2-), HER2+, and triple negative (TN). Only images acquired in the DCE sequence were used in the analysis. The smallest bounding box covering tumor ROI was used as the input for deep learning to develop the model in the training dataset, by using a conventional CNN and the convolutional long short-term memory (CLSTM). Then, transfer learning was applied to re-tune the model using testing-1(2) and evaluated in testing-2(1).

RESULTS

In the training dataset, the mean accuracy evaluated using tenfold cross-validation was higher by using CLSTM (0.91) than by using CNN (0.79). When the developed model was applied to the independent testing datasets, the accuracy was 0.4-0.5. With transfer learning by re-tuning parameters in testing-1, the mean accuracy reached 0.91 by CNN and 0.83 by CLSTM, and improved accuracy in testing-2 from 0.47 to 0.78 by CNN and from 0.39 to 0.74 by CLSTM. Overall, transfer learning could improve the classification accuracy by greater than 30%.

CONCLUSIONS

The recurrent network using CLSTM could track changes in signal intensity during DCE acquisition, and achieved a higher accuracy compared with conventional CNN during training. For datasets acquired using different settings, transfer learning can be applied to re-tune the model and improve accuracy.

KEY POINTS

• Deep learning can be applied to differentiate breast cancer molecular subtypes. • The recurrent neural network using CLSTM could track the change of signal intensity in DCE images, and achieved a higher accuracy compared with conventional CNN during training. • For datasets acquired using different scanners with different imaging protocols, transfer learning provided an efficient method to re-tune the classification model and improve accuracy.

Deep learned tissue "fingerprints" classify breast cancers by ER/PR/Her2 status from H&E images

Because histologic types are subjective and difficult to reproduce between pathologists, tissue morphology often takes a back seat to molecular testing for the selection of breast cancer treatments. This work explores whether a deep-learning algorithm can learn objective histologic H&E features that predict the clinical subtypes of breast cancer, as assessed by immunostaining for estrogen, progesterone, and Her2 receptors (ER/PR/Her2). Translating deep learning to this and related problems in histopathology presents a challenge due to the lack of large, well-annotated data sets, which are typically required for the algorithms to learn statistically significant discriminatory patterns. To overcome this limitation, we introduce the concept of “tissue fingerprints,” which leverages large, unannotated datasets in a label-free manner to learn H&E features that can distinguish one patient from another. The hypothesis is that training the algorithm to learn the morphological differences between patients will implicitly teach it about the biologic variation between them. Following this training internship, we used the features the network learned, which we call “fingerprints,” to predict ER, PR, and Her2 status in two datasets. Despite the discovery dataset being relatively small by the standards of the machine learning community (n = 939), fingerprints enabled the determination of ER, PR, and Her2 status from whole slide H&E images with 0.89 AUC (ER), 0.81 AUC (PR), and 0.79 AUC (Her2) on a large, independent test set (n = 2531). Tissue fingerprints are concise but meaningful histopathologic image representations that capture biological information and may enable machine learning algorithms that go beyond the traditional ER/PR/Her2 clinical groupings by directly predicting theragnosis.