An integrative deep learning framework for classifying molecular subtypes of breast cancer

AI in Breast Cancer Imaging: An Update and Future Trends

Breast cancer is one of the most common types of cancer affecting women worldwide. Artificial intelligence (AI) is transforming breast cancer imaging by enhancing diagnostic capabilities across multiple imaging modalities including mammography, digital breast tomosynthesis, ultrasound, magnetic resonance imaging, and nuclear medicines techniques. AI is being applied to diverse tasks such as breast lesion detection and classification, risk stratification, molecular subtyping, gene mutation status prediction, and treatment response assessment, with emerging research demonstrating performance levels comparable to or potentially exceeding those of radiologists. The large foundation models are showing remarkable potential in different breast cancer imaging tasks. Self-supervised learning gives an insight into data inherent correlation, and federated learning is an alternative way to maintain data privacy. While promising results have been obtained so far, data standardization from source, large-scale annotated multimodal datasets, and extensive prospective clinical trials are still needed to fully explore and validate deep learning's clinical utility and address the legal and ethical considerations, which will ultimately determine its widespread adoption in breast cancer care. We hereby provide a review of the most up-to-date knowledge on AI in breast cancer imaging.

Cancer type and survival prediction based on transcriptomic feature map

This study achieved cancer type and survival time prediction by transforming transcriptomic features into feature maps and employing deep learning models. Using transcriptomic data from 27 cancer types and survival data from 10 types in the TCGA database, a pan-cancer transcriptomic feature map was constructed through data cleaning, feature extraction, and visualization. Using Inception network and gated convolutional modules yielded a pan-cancer classification accuracy of 91.8 %. Additionally, by extracting 31 differential genes from different cancer feature maps, an interaction network diagram was drawn, identifying two key genes, ANXA5 and ACTB. These genes are potential biomarkers related to cancer progression, angiogenesis, metastasis, and treatment resistance. Survival prediction analysis on 10 cancer types, combined with feature maps and data amplification, cancer survival prediction accuracy reached from 0.75 to 0.91. This transcriptomic feature map provides a novel approach for cancer omics analysis, to facilitate personalized treatments and reflecting individual differences.

Nongenerative Artificial Intelligence in Medicine: Advancements and Applications in Supervised and Unsupervised Machine Learning

The use of artificial intelligence (AI) within pathology and health care has advanced extensively. We have accordingly witnessed an increased adoption of various AI tools that are transforming our approach to clinical decision support, personalized medicine, predictive analytics, automation, and discovery. The familiar and more reliable AI tools that have been incorporated within health care thus far fall mostly under the nongenerative AI domain, which includes supervised and unsupervised machine learning (ML) techniques. This review article explores how such nongenerative AI methods, rooted in traditional rules-based systems, enhance diagnostic accuracy, efficiency, and consistency within medicine. Key concepts and the application of supervised learning models (ie, classification and regression) such as decision trees, support vector machines, linear and logistic regression, K-nearest neighbor, and neural networks are explained along with the newer landscape of neural network-based nongenerative foundation models. Unsupervised learning techniques, including clustering, dimensionality reduction, and anomaly detection, are also discussed for their roles in uncovering novel disease subtypes or identifying outliers. Technical details related to the application of nongenerative AI algorithms for analyzing whole slide images are also highlighted. The performance, explainability, and reliability of nongenerative AI models essential for clinical decision-making is also reviewed, as well as challenges related to data quality, model interpretability, and risk of data drift. An understanding of which AI-ML models to employ and which shortcomings need to be addressed is imperative to safely and efficiently leverage, integrate, and monitor these traditional AI tools in clinical practice and research.

MOCapsNet: Multiomics Data Integration for Cancer Subtype Analysis Based on Dynamic Self-Attention Learning and Capsule Networks

Background and Objective: With the rapid development of the accumulation of large-scale multiomics data sets, integrating various omics data to provide a thorough study from multiple perspectives can significantly provide stronger support for precise treatment of diseases. However, due to the complexity of multiomics data, issues of feature redundancy and noise often do not receive sufficient attention when processing high-dimensional data. Moreover, simple concatenation strategies may overlook the correlations between different omics data, failing to effectively capture the unique information inherent in multiomics data. Meanwhile, deep neural networks often rely on complex structures and numerous parameters for training and inference, making their internal feature representations and decision-making processes difficult to interpret. Methods: We propose an interpretable multiomics data integration method for cancer subtype classification, named MOCapsNet, based on self-attention and capsule networks. Specifically, the self-attention confidence learning module is implemented to assess the feature information within each omic and to assign weights to the embedded representations of various groups, resulting in more targeted integrated information. Furthermore, the capsule network structure is employed for the final cancer classification task. Results: The model achieved strong performance on both tasks: 87.8% accuracy on the BRCA multiclassification data set and 83.6% accuracy with an AUC of 88.8% on the LGG data set. Conclusions: The proposed framework has undergone extensive testing on omics data sets, consistently proving its effectiveness in integrating multiomics data. It improves classification accuracy while enhancing the interpretability of results by fully utilizing the feature information.

Open challenges and opportunities in federated foundation models towards biomedical healthcare

This survey explores the transformative impact of foundation models (FMs) in artificial intelligence, focusing on their integration with federated learning (FL) in biomedical research. Foundation models such as ChatGPT, LLaMa, and CLIP, which are trained on vast datasets through methods including unsupervised pretraining, self-supervised learning, instructed fine-tuning, and reinforcement learning from human feedback, represent significant advancements in machine learning. These models, with their ability to generate coherent text and realistic images, are crucial for biomedical applications that require processing diverse data forms such as clinical reports, diagnostic images, and multimodal patient interactions. The incorporation of FL with these sophisticated models presents a promising strategy to harness their analytical power while safeguarding the privacy of sensitive medical data. This approach not only enhances the capabilities of FMs in medical diagnostics and personalized treatment but also addresses critical concerns about data privacy and security in healthcare. This survey reviews the current applications of FMs in federated settings, underscores the challenges, and identifies future research directions including scaling FMs, managing data diversity, and enhancing communication efficiency within FL frameworks. The objective is to encourage further research into the combined potential of FMs and FL, laying the groundwork for healthcare innovations.

A review on trends in development and translation of omics signatures in cancer

The field of cancer genomics and transcriptomics has evolved from targeted profiling to swift sequencing of individual tumor genome and transcriptome. The steady growth in genome, epigenome, and transcriptome datasets on a genome-wide scale has significantly increased our capability in capturing signatures that represent both the intrinsic and extrinsic biological features of tumors. These biological differences can help in precise molecular subtyping of cancer, predicting tumor progression, metastatic potential, and resistance to therapeutic agents. In this review, we summarized the current development of genomic, methylomic, transcriptomic, proteomic and metabolic signatures in the field of cancer research and highlighted their potentials in clinical applications to improve diagnosis, prognosis, and treatment decision in cancer patients.

GREMI: An Explainable Multi-Omics Integration Framework for Enhanced Disease Prediction and Module Identification

Multi-omics integration has demonstrated promising performance in complex disease prediction. However, existing research typically focuses on maximizing prediction accuracy, while often neglecting the essential task of discovering meaningful biomarkers. This issue is particularly important in biomedicine, as molecules often interact rather than function individually to influence disease outcomes. To this end, we propose a two-phase framework named GREMI to assist multi-omics classification and explanation. In the prediction phase, we propose to improve prediction performance by employing a graph attention architecture on sample-wise co-functional networks to incorporate biomolecular interaction information for enhanced feature representation, followed by the integration of a joint-late mixed strategy and the true-class-probability block to adaptively evaluate classification confidence at both feature and omics levels. In the interpretation phase, we propose a multi-view approach to explain disease outcomes from the interaction module perspective, providing a more intuitive understanding and biomedical rationale. We incorporate Monte Carlo tree search (MCTS) to explore local-view subgraphs and pinpoint modules that highly contribute to disease characterization from the global-view. Extensive experiments demonstrate that the proposed framework outperforms state-of-the-art methods in seven different classification tasks, and our model effectively addresses data mutual interference when the number of omics types increases. We further illustrate the functional- and disease-relevance of the identified modules, as well as validate the classification performance of discovered modules using an independent cohort.

OmicPredict: a framework for omics data prediction using ANOVA-Firefly algorithm for feature selection

High-throughput technologies and machine learning (ML), when applied to a huge pool of medical data such as omics data, result in efficient analysis. Recent research aims to apply and develop ML models to predict a disease well in time using available omics datasets. The present work proposed a framework, 'OmicPredict', deploying a hybrid feature selection method and deep neural network (DNN) model to predict multiple diseases using omics data. The hybrid feature selection method is developed using the Analysis of Variance (ANOVA) technique and firefly algorithm. The OmicPredict framework is applied to three case studies, Alzheimer's disease, Breast cancer, and Coronavirus disease 2019 (COVID-19). In the case study of Alzheimer's disease, the framework predicts patients using GSE33000 and GSE44770 dataset. In the case study of Breast cancer, the framework predicts human epidermal growth factor receptor 2 (HER2) subtype status using Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) dataset. In the case study of COVID-19, the framework performs patients' classification using GSE157103 dataset. The experimental results show that DNN model achieved an Area Under Curve (AUC) score of 0.949 for the Alzheimer's (GSE33000 and GSE44770) dataset. Furthermore, it achieved an AUC score of 0.987 and 0.989 for breast cancer (METABRIC) and COVID-19 (GSE157103) datasets, respectively, outperforming Random Forest, Naïve Bayes models, and the existing research.

MOSGAT: Uniting Specificity-Aware GATs and Cross Modal-Attention to Integrate Multi-Omics Data for Disease Diagnosis

With the advancement of sequencing methodologies, the acquisition of vast amounts of multi-omics data presents a significant opportunity for comprehending the intricate biological mechanisms underlying diseases and achieving precise diagnosis and treatment for complex disorders. However, as diverse omics data are integrated, extracting sample-specific features within each omics modality and exploring potential correlations among different modalities while avoiding mutual interference becomes a critical challenge in multi-omics data integration research. In the context of this study, we proposed a framework that unites specificity-aware GATs and cross-modal attention to integrate different omics data (MOSGAT). To be specific, we devise Graph Attention Networks (GATs) tailored for each omics modality data to perform feature extraction on samples. Additionally, an adaptive confidence attention weighting technique is incorporated to enhance the confidence in the extracted features. Finally, a cross-modal attention mechanism was devised based on multi-head self-attention, thoroughly uncovering potential correlations between different omics data. Extensive experiments were conducted on four publicly available medical datasets, highlighting the superiority of the proposed framework when compared to state-of-the-art methodologies, particularly in the realm of classification tasks. The experimental results underscore MOSGAT's effectiveness in extracting features and exploring potential inter-omics associations.

Comparison of data fusion strategies for automated prostate lesion detection using mpMRI correlated with whole mount histology

BACKGROUND

In this work, we compare input level, feature level and decision level data fusion techniques for automatic detection of clinically significant prostate lesions (csPCa).

METHODS

Multiple deep learning CNN architectures were developed using the Unet as the baseline. The CNNs use both multiparametric MRI images (T2W, ADC, and High b-value) and quantitative clinical data (prostate specific antigen (PSA), PSA density (PSAD), prostate gland volume & gross tumor volume (GTV)), and only mp-MRI images (n = 118), as input. In addition, co-registered ground truth data from whole mount histopathology images (n = 22) were used as a test set for evaluation.

RESULTS

The CNNs achieved for early/intermediate / late level fusion a precision of 0.41/0.51/0.61, recall value of 0.18/0.22/0.25, an average precision of 0.13 / 0.19 / 0.27, and F scores of 0.55/0.67/ 0.76. Dice Sorensen Coefficient (DSC) was used to evaluate the influence of combining mpMRI with parametric clinical data for the detection of csPCa. We compared the DSC between the predictions of CNN's trained with mpMRI and parametric clinical and the CNN's trained with only mpMRI images as input with the ground truth. We obtained a DSC of data 0.30/0.34/0.36 and 0.26/0.33/0.34 respectively. Additionally, we evaluated the influence of each mpMRI input channel for the task of csPCa detection and obtained a DSC of 0.14 / 0.25 / 0.28.

CONCLUSION

The results show that the decision level fusion network performs better for the task of prostate lesion detection. Combining mpMRI data with quantitative clinical data does not show significant differences between these networks (p = 0.26/0.62/0.85). The results show that CNNs trained with all mpMRI data outperform CNNs with less input channels which is consistent with current clinical protocols where the same input is used for PI-RADS lesion scoring.

TRIAL REGISTRATION

The trial was registered retrospectively at the German Register for Clinical Studies (DRKS) under proposal number Nr. 476/14 & 476/19.

Pathformer: a biological pathway informed transformer for disease diagnosis and prognosis using multi-omics data

MOTIVATION

Multi-omics data provide a comprehensive view of gene regulation at multiple levels, which is helpful in achieving accurate diagnosis of complex diseases like cancer. However, conventional integration methods rarely utilize prior biological knowledge and lack interpretability.

RESULTS

To integrate various multi-omics data of tissue and liquid biopsies for disease diagnosis and prognosis, we developed a biological pathway informed Transformer, Pathformer. It embeds multi-omics input with a compacted multi-modal vector and a pathway-based sparse neural network. Pathformer also leverages criss-cross attention mechanism to capture the crosstalk between different pathways and modalities. We first benchmarked Pathformer with 18 comparable methods on multiple cancer datasets, where Pathformer outperformed all the other methods, with an average improvement of 6.3%-14.7% in F1 score for cancer survival prediction, 5.1%-12% for cancer stage prediction, and 8.1%-13.6% for cancer drug response prediction. Subsequently, for cancer prognosis prediction based on tissue multi-omics data, we used a case study to demonstrate the biological interpretability of Pathformer by identifying key pathways and their biological crosstalk. Then, for cancer early diagnosis based on liquid biopsy data, we used plasma and platelet datasets to demonstrate Pathformer's potential of clinical applications in cancer screening. Moreover, we revealed deregulation of interesting pathways (e.g. scavenger receptor pathway) and their crosstalk in cancer patients' blood, providing potential candidate targets for cancer microenvironment study.

AVAILABILITY AND IMPLEMENTATION

Pathformer is implemented and freely available at https://github.com/lulab/Pathformer.

HyperTMO: a trusted multi-omics integration framework based on hypergraph convolutional network for patient classification

MOTIVATION

The rapid development of high-throughput biomedical technologies can provide researchers with detailed multi-omics data. The multi-omics integrated analysis approach based on machine learning contributes a more comprehensive perspective to human disease research. However, there are still significant challenges in representing single-omics data and integrating multi-omics information.

RESULTS

This article presents HyperTMO, a Trusted Multi-Omics integration framework based on Hypergraph convolutional network for patient classification. HyperTMO constructs hypergraph structures to represent the association between samples in single-omics data, then evidence extraction is performed by hypergraph convolutional network, and multi-omics information is integrated at an evidence level. Last, we experimentally demonstrate that HyperTMO outperforms other state-of-the-art methods in breast cancer subtype classification and Alzheimer's disease classification tasks using multi-omics data from TCGA (BRCA) and ROSMAP datasets. Importantly, HyperTMO is the first attempt to integrate hypergraph structure, evidence theory, and multi-omics integration for patient classification. Its accurate and robust properties bring great potential for applications in clinical diagnosis.

AVAILABILITY AND IMPLEMENTATION

HyperTMO and datasets are publicly available at https://github.com/ippousyuga/HyperTMO.

A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis

Disease diagnosis represents a critical and arduous endeavor within the medical field. Artificial intelligence (AI) techniques, spanning from machine learning and deep learning to large model paradigms, stand poised to significantly augment physicians in rendering more evidence-based decisions, thus presenting a pioneering solution for clinical practice. Traditionally, the amalgamation of diverse medical data modalities (e.g., image, text, speech, genetic data, physiological signals) is imperative to facilitate a comprehensive disease analysis, a topic of burgeoning interest among both researchers and clinicians in recent times. Hence, there exists a pressing need to synthesize the latest strides in multi-modal data and AI technologies in the realm of medical diagnosis. In this paper, we narrow our focus to five specific disorders (Alzheimer's disease, breast cancer, depression, heart disease, epilepsy), elucidating advanced endeavors in their diagnosis and treatment through the lens of artificial intelligence. Our survey not only delineates detailed diagnostic methodologies across varying modalities but also underscores commonly utilized public datasets, the intricacies of feature engineering, prevalent classification models, and envisaged challenges for future endeavors. In essence, our research endeavors to contribute to the advancement of diagnostic methodologies, furnishing invaluable insights for clinical decision making.

[Identification of breast cancer subtypes based on graph convolutional network]

Identification of molecular subtypes of malignant tumors plays a vital role in individualized diagnosis, personalized treatment, and prognosis prediction of cancer patients. The continuous improvement of comprehensive tumor genomics database and the ongoing breakthroughs in deep learning technology have driven further advancements in computer-aided tumor classification. Although the existing classification methods based on gene expression omnibus database take the complexity of cancer molecular classification into account, they ignore the internal correlation and synergism of genes. To solve this problem, we propose a multi-layer graph convolutional network model for breast cancer subtype classification combined with hierarchical attention network. This model constructs the graph embedding datasets of patients' genes, and develops a new end-to-end multi-classification model, which can effectively recognize molecular subtypes of breast cancer. A large number of test data prove the good performance of this new model in the classification of breast cancer subtypes. Compared to the original graph convolutional neural networks and two mainstream graph neural network classification algorithms, the new model has remarkable advantages. The accuracy, weight-F1-score, weight-recall, and weight-precision of our model in seven-category classification has reached 0.851 7, 0.823 5, 0.851 7 and 0.793 6 respectively. In the four-category classification, the results are 0.928 5, 0.894 9, 0.928 5 and 0.865 0 respectively. In addition, compared with the latest breast cancer subtype classification algorithms, the method proposed in this paper also achieved the highest classification accuracy. In summary, the model proposed in this paper may serve as an auxiliary diagnostic technology, providing a reliable option for precise classification of breast cancer subtypes in the future and laying the theoretical foundation for computer-aided tumor classification.

Object detection: A novel AI technology for the diagnosis of hepatocyte ballooning

Metabolic dysfunction-associated fatty liver disease (MAFLD) has reached epidemic proportions worldwide and is the most frequent cause of chronic liver disease in developed countries. Within the spectrum of liver disease in MAFLD, steatohepatitis is a progressive form of liver disease and hepatocyte ballooning (HB) is a cardinal pathological feature of steatohepatitis. The accurate and reproducible diagnosis of HB is therefore critical for the early detection and treatment of steatohepatitis. Currently, a diagnosis of HB relies on pathological examination by expert pathologists, which may be a time-consuming and subjective process. Hence, there has been interest in developing automated methods for diagnosing HB. This narrative review briefly discusses the development of artificial intelligence (AI) technology for diagnosing fatty liver disease pathology over the last 30 years and provides an overview of the current research status of AI algorithms for the identification of HB, including published articles on traditional machine learning algorithms and deep learning algorithms. This narrative review also provides a summary of object detection algorithms, including the principles, historical developments, and applications in the medical image analysis. The potential benefits of object detection algorithms for HB diagnosis (specifically those combined with a transformer architecture) are discussed, along with the future directions of object detection algorithms in HB diagnosis and the potential applications of generative AI on transformer architecture in this field. In conclusion, object detection algorithms have huge potential for the identification of HB and could make the diagnosis of MAFLD more accurate and efficient in the near future.

Historical perspective and future directions: computational science in immuno-oncology

Immuno-oncology holds promise for transforming patient care having achieved durable clinical response rates across a variety of advanced and metastatic cancers. Despite these achievements, only a minority of patients respond to immunotherapy, underscoring the importance of elucidating molecular mechanisms responsible for response and resistance to inform the development and selection of treatments. Breakthroughs in molecular sequencing technologies have led to the generation of an immense amount of genomic and transcriptomic sequencing data that can be mined to uncover complex tumor-immune interactions using computational tools. In this review, we discuss existing and emerging computational methods that contextualize the composition and functional state of the tumor microenvironment, infer the reactivity and clonal dynamics from reconstructed immune cell receptor repertoires, and predict the antigenic landscape for immune cell recognition. We further describe the advantage of multi-omics analyses for capturing multidimensional relationships and artificial intelligence techniques for integrating omics data with histopathological and radiological images to encapsulate patterns of treatment response and tumor-immune biology. Finally, we discuss key challenges impeding their widespread use and clinical application and conclude with future perspectives. We are hopeful that this review will both serve as a guide for prospective researchers seeking to use existing tools for scientific discoveries and inspire the optimization or development of novel tools to enhance precision, ultimately expediting advancements in immunotherapy that improve patient survival and quality of life.

Automated quantification of stromal tumour infiltrating lymphocytes is associated with prognosis in breast cancer

Stromal tumour infiltrating lymphocytes (sTIL) in haematoxylin and eosin (H&E) stained sections has been linked to better outcomes and better responses to neoadjuvant therapy in triple-negative and HER2-positive breast cancer (TNBC and HER2 +). However, the infiltrate includes different cell populations that have specific roles in the tumour immune microenvironment. Various studies have found high concordance between sTIL visual quantification and computational assessment, but specific data on the individual prognostic impact of plasma cells or lymphocytes within sTIL on patient prognosis is still unknown. In this study, we validated a deep-learning breast cancer sTIL scoring model (smsTIL) based on the segmentation of tumour cells, benign ductal cells, lymphocytes, plasma cells, necrosis, and 'other' cells in whole slide images (WSI). Focusing on HER2 + and TNBC patient samples, we assessed the concordance between sTIL visual scoring and the smsTIL in 130 WSI. Furthermore, we analysed 175 WSI to correlate smsTIL with clinical data and patient outcomes. We found a high correlation between sTIL values scored visually and semi-automatically (R = 0.76; P = 2.2e-16). Patients with higher smsTIL had better overall survival (OS) in TNBC (P = 0.0021). In the TNBC cohort, smsTIL was as an independent prognostic factor for OS. As part of this work, we introduce a new segmentation dataset of H&E-stained WSI.

Multimodal deep learning approaches for single-cell multi-omics data integration

Integrating single-cell multi-omics data is a challenging task that has led to new insights into complex cellular systems. Various computational methods have been proposed to effectively integrate these rapidly accumulating datasets, including deep learning. However, despite the proven success of deep learning in integrating multi-omics data and its better performance over classical computational methods, there has been no systematic study of its application to single-cell multi-omics data integration. To fill this gap, we conducted a literature review to explore the use of multimodal deep learning techniques in single-cell multi-omics data integration, taking into account recent studies from multiple perspectives. Specifically, we first summarized different modalities found in single-cell multi-omics data. We then reviewed current deep learning techniques for processing multimodal data and categorized deep learning-based integration methods for single-cell multi-omics data according to data modality, deep learning architecture, fusion strategy, key tasks and downstream analysis. Finally, we provided insights into using these deep learning models to integrate multi-omics data and better understand single-cell biological mechanisms.

The role of tumor microenvironment in drug resistance: emerging technologies to unravel breast cancer heterogeneity

Breast cancer is a highly heterogeneous disease, at both inter- and intra-tumor levels, and this heterogeneity is a crucial determinant of malignant progression and response to treatments. In addition to genetic diversity and plasticity of cancer cells, the tumor microenvironment contributes to tumor heterogeneity shaping the physical and biological surroundings of the tumor. The activity of certain types of immune, endothelial or mesenchymal cells in the microenvironment can change the effectiveness of cancer therapies via a plethora of different mechanisms. Therefore, deciphering the interactions between the distinct cell types, their spatial organization and their specific contribution to tumor growth and drug sensitivity is still a major challenge. Dissecting intra-tumor heterogeneity is currently an urgent need to better define breast cancer biology and to develop therapeutic strategies targeting the microenvironment as helpful tools for combined and personalized treatment. In this review, we analyze the mechanisms by which the tumor microenvironment affects the characteristics of tumor heterogeneity that ultimately result in drug resistance, and we outline state of the art preclinical models and emerging technologies that will be instrumental in unraveling the impact of the tumor microenvironment on resistance to therapies.

A survey on multi-omics-based cancer diagnosis using machine learning with the potential application in gastrointestinal cancer

Gastrointestinal cancer is becoming increasingly common, which leads to over 3 million deaths every year. No typical symptoms appear in the early stage of gastrointestinal cancer, posing a significant challenge in the diagnosis and treatment of patients with gastrointestinal cancer. Many patients are in the middle and late stages of gastrointestinal cancer when they feel uncomfortable, unfortunately, most of them will die of gastrointestinal cancer. Recently, various artificial intelligence techniques like machine learning based on multi-omics have been presented for cancer diagnosis and treatment in the era of precision medicine. This paper provides a survey on multi-omics-based cancer diagnosis using machine learning with potential application in gastrointestinal cancer. Particularly, we make a comprehensive summary and analysis from the perspective of multi-omics datasets, task types, and multi-omics-based integration methods. Furthermore, this paper points out the remaining challenges of multi-omics-based cancer diagnosis using machine learning and discusses future topics.

Value of genomics- and radiomics-based machine learning models in the identification of breast cancer molecular subtypes: a systematic review and meta-analysis

Background

In the era of precision therapy, early classification of breast cancer (BRCA) molecular subtypes has clinical significance for disease management and prognosis. We explored the accuracy of machine learning (ML) models for early classification of BRCA molecular subtypes through a systematic review of the literature currently available.

Methods

We retrieved relevant studies published in PubMed, EMBASE, Cochrane, and Web of Science until 15 April 2022. A prediction model risk of bias assessment tool (PROBAST) was applied for the assessment of risk of bias of a genomics-based ML model, and the Radiomics Quality Score (RQS) was simultaneously used to evaluate the quality of this radiomics-based ML model. A random effects model was adopted to analyze the predictive accuracy of genomics-based ML and radiomics-based ML for Luminal A, Luminal B, Basal-like or triple-negative breast cancer (TNBC), and human epidermal growth factor receptor 2 (HER2). The PROSPERO of our study was prospectively registered (CRD42022333611).

Results

Of the 38 studies were selected for analysis, 14 ML models were based on gene-transcriptomic, with only 4 external validations; and 43 ML models were based on radiomics, with only 14 external validations. Meta-analysis results showed that c-statistic values of the ML based on radiomics for the identification of BRCA molecular subtypes Luminal A, Luminal B, Basal-like or TNBC, and HER2 were 0.76 [95% confidence interval (CI): 0.60-0.96], 0.78 (95% CI: 0.69-0.87), 0.89 (95% CI: 0.83-0.91), and 0.83 (95% CI: 0.81-0.86), respectively. The c-statistic values of ML based on the gene-transcriptomic analysis cohort for the identification of the previously described BRCA molecular subtypes were 0.96 (95% CI: 0.93-0.99), 0.96 (95% CI: 0.93-0.99), 0.98 (95% CI: 0.95-1.00), and 0.97 (95% CI: 0.96-0.98) respectively. Additionally, the sensitivity of the ML model based on radiomics for each molecular subtype ranged from 0.79 to 0.85, while the sensitivity of the ML model based on gene-transcriptomic was between 0.92 and 0.99.

Conclusions

Both radiomics and gene transcriptomics produced ideal effects on BRCA molecular subtype prediction. Compared with radiomics, gene transcriptomics yielded better prediction results, but radiomics was simpler and more convenient from a clinical point of view.

Omics Data and Data Representations for Deep Learning-Based Predictive Modeling

Medical discoveries mainly depend on the capability to process and analyze biological datasets, which inundate the scientific community and are still expanding as the cost of next-generation sequencing technologies is decreasing. Deep learning (DL) is a viable method to exploit this massive data stream since it has advanced quickly with there being successive innovations. However, an obstacle to scientific progress emerges: the difficulty of applying DL to biology, and this because both fields are evolving at a breakneck pace, thus making it hard for an individual to occupy the front lines of both of them. This paper aims to bridge the gap and help computer scientists bring their valuable expertise into the life sciences. This work provides an overview of the most common types of biological data and data representations that are used to train DL models, with additional information on the models themselves and the various tasks that are being tackled. This is the essential information a DL expert with no background in biology needs in order to participate in DL-based research projects in biomedicine, biotechnology, and drug discovery. Alternatively, this study could be also useful to researchers in biology to understand and utilize the power of DL to gain better insights into and extract important information from the omics data.

A benchmark study of deep learning-based multi-omics data fusion methods for cancer

BACKGROUND

A fused method using a combination of multi-omics data enables a comprehensive study of complex biological processes and highlights the interrelationship of relevant biomolecules and their functions. Driven by high-throughput sequencing technologies, several promising deep learning methods have been proposed for fusing multi-omics data generated from a large number of samples.

RESULTS

In this study, 16 representative deep learning methods are comprehensively evaluated on simulated, single-cell, and cancer multi-omics datasets. For each of the datasets, two tasks are designed: classification and clustering. The classification performance is evaluated by using three benchmarking metrics including accuracy, F1 macro, and F1 weighted. Meanwhile, the clustering performance is evaluated by using four benchmarking metrics including the Jaccard index (JI), C-index, silhouette score, and Davies Bouldin score. For the cancer multi-omics datasets, the methods' strength in capturing the association of multi-omics dimensionality reduction results with survival and clinical annotations is further evaluated. The benchmarking results indicate that moGAT achieves the best classification performance. Meanwhile, efmmdVAE, efVAE, and lfmmdVAE show the most promising performance across all complementary contexts in clustering tasks.

CONCLUSIONS

Our benchmarking results not only provide a reference for biomedical researchers to choose appropriate deep learning-based multi-omics data fusion methods, but also suggest the future directions for the development of more effective multi-omics data fusion methods. The deep learning frameworks are available at https://github.com/zhenglinyi/DL-mo .

With or without human interference for precise age estimation based on machine learning?

Age estimation can aid in forensic medicine applications, diagnosis, and treatment planning for orthodontics and pediatrics. Existing dental age estimation methods rely heavily on specialized knowledge and are highly subjective, wasting time, and energy, which can be perfectly solved by machine learning techniques. As the key factor affecting the performance of machine learning models, there are usually two methods for feature extraction: human interference and autonomous extraction without human interference. However, previous studies have rarely applied these two methods for feature extraction in the same image analysis task. Herein, we present two types of convolutional neural networks (CNNs) for dental age estimation. One is an automated dental stage evaluation model (ADSE model) based on specified manually defined features, and the other is an automated end-to-end dental age estimation model (ADAE model), which autonomously extracts potential features for dental age estimation. Although the mean absolute error (MAE) of the ADSE model for stage classification is 0.17 stages, its accuracy in dental age estimation is unsatisfactory, with the MAE (1.63 years) being only 0.04 years lower than the manual dental age estimation method (MDAE model). However, the MAE of the ADAE model is 0.83 years, being reduced by half that of the MDAE model. The results show that fully automated feature extraction in a deep learning model without human interference performs better in dental age estimation, prominently increasing the accuracy and objectivity. This indicates that without human interference, machine learning may perform better in the application of medical imaging.

Multimodal deep learning for biomedical data fusion: a review

Biomedical data are becoming increasingly multimodal and thereby capture the underlying complex relationships among biological processes. Deep learning (DL)-based data fusion strategies are a popular approach for modeling these nonlinear relationships. Therefore, we review the current state-of-the-art of such methods and propose a detailed taxonomy that facilitates more informed choices of fusion strategies for biomedical applications, as well as research on novel methods. By doing so, we find that deep fusion strategies often outperform unimodal and shallow approaches. Additionally, the proposed subcategories of fusion strategies show different advantages and drawbacks. The review of current methods has shown that, especially for intermediate fusion strategies, joint representation learning is the preferred approach as it effectively models the complex interactions of different levels of biological organization. Finally, we note that gradual fusion, based on prior biological knowledge or on search strategies, is a promising future research path. Similarly, utilizing transfer learning might overcome sample size limitations of multimodal data sets. As these data sets become increasingly available, multimodal DL approaches present the opportunity to train holistic models that can learn the complex regulatory dynamics behind health and disease.

L’intelligence artificielle pourrait-elle intervenir dans l’aide au diagnostic des cancers du sein ? – L’exemple de HER2

HER2 is an important prognostic and predictive biomarker in breast cancer. Its detection makes it possible to define which patients will benefit from a targeted treatment. While assessment of HER2 status by immunohistochemistry in positive vs negative categories is well implemented and reproducible, the introduction of a new "HER2-low" category could raise some concerns about its scoring and reproducibility. We herein described the current HER2 testing methods and the application of innovative machine learning techniques to improve these determinations, as well as the main challenges and opportunities related to the implementation of digital pathology in the up-and-coming AI era.

Application of preoperative ultrasound features combined with clinical factors in predicting HER2-positive subtype (non-luminal) breast cancer

BACKGROUND

Human epidermal growth factor receptor2+ subtype breast cancer has a high degree of malignancy and a poor prognosis. The aim of this study is to develop a prediction model for the human epidermal growth factor receptor2+ subtype (non-luminal) of breast cancer based on the clinical and ultrasound features related with estrogen receptor, progesterone receptor, and human epidermal growth factor receptor2.

METHODS

We collected clinical data and reviewed preoperative ultrasound images of enrolled breast cancers from September 2017 to August 2020. We divided the data into in three groups as follows. Group I: estrogen receptor ± , Group II: progesterone receptor ± and Group III: human epidermal growth factor receptor2 ± . Univariate and multivariate logistic regression analyses were used to analyze the clinical and ultrasound features related with biomarkers among these groups. A model to predict human epidermal growth factor receptor2+ subtype was then developed based on the results of multivariate regression analyses, and the efficacy was evaluated using the area under receiver operating characteristic curve, accuracy, sensitivity, specificity.

RESULTS

The human epidermal growth factor receptor2+ subtype accounted for 138 cases (11.8%) in the training set and 51 cases (10.1%) in the test set. In the multivariate regression analysis, age ≤ 50 years was an independent predictor of progesterone receptor + (p = 0.007), and posterior enhancement was a negative predictor of progesterone receptor + (p = 0.013) in Group II; palpable axillary lymph node, round, irregular shape and calcifications were independent predictors of the positivity for human epidermal growth factor receptor-2 in Group III (p = 0.001, p = 0.007, p = 0.010, p < 0.001, respectively). In Group I, shape was the only factor related to estrogen receptor status in the univariate analysis (p < 0.05). The area under receiver operating characteristic curve, accuracy, sensitivity, specificity of the model to predict human epidermal growth factor receptor2+ subtype breast cancer was 0.697, 60.14%, 72.46%, 58.49% and 0.725, 72.06%, 64.71%, 72.89% in the training and test sets, respectively.

CONCLUSIONS

Our study established a model to predict the human epidermal growth factor receptor2-positive subtype with moderate performance. And the results demonstrated that clinical and ultrasound features were significantly associated with biomarkers.

Deep learning in cancer diagnosis, prognosis and treatment selection

Deep learning is a subdiscipline of artificial intelligence that uses a machine learning technique called artificial neural networks to extract patterns and make predictions from large data sets. The increasing adoption of deep learning across healthcare domains together with the availability of highly characterised cancer datasets has accelerated research into the utility of deep learning in the analysis of the complex biology of cancer. While early results are promising, this is a rapidly evolving field with new knowledge emerging in both cancer biology and deep learning. In this review, we provide an overview of emerging deep learning techniques and how they are being applied to oncology. We focus on the deep learning applications for omics data types, including genomic, methylation and transcriptomic data, as well as histopathology-based genomic inference, and provide perspectives on how the different data types can be integrated to develop decision support tools. We provide specific examples of how deep learning may be applied in cancer diagnosis, prognosis and treatment management. We also assess the current limitations and challenges for the application of deep learning in precision oncology, including the lack of phenotypically rich data and the need for more explainable deep learning models. Finally, we conclude with a discussion of how current obstacles can be overcome to enable future clinical utilisation of deep learning.

Will Artificial Intelligence Replace the Movement Disorders Specialist for Diagnosing and Managing Parkinson's Disease?

The use of artificial intelligence (AI) to help diagnose and manage disease is of increasing interest to researchers and clinicians. Volumes of health data are generated from smartphones and ubiquitous inexpensive sensors. By using these data, AI can offer otherwise unobtainable insights about disease burden and patient status in a free-living environment. Moreover, from clinical datasets AI can improve patient symptom monitoring and global epidemiologic efforts. While these applications are exciting, it is necessary to examine both the utility and limitations of these novel analytic methods. The most promising uses of AI remain aspirational. For example, defining the molecular subtypes of Parkinson's disease will be assisted by future applications of AI to relevant datasets. This will allow clinicians to match patients to molecular therapies and will thus help launch precision medicine. Until AI proves its potential in pushing the frontier of precision medicine, its utility will primarily remain in individualized monitoring, complementing but not replacing movement disorders specialists.

On the stability of log-rank test under labeling errors

Motivation

Log-rank test is a widely used test that serves to assess the statistical significance of observed differences in survival, when comparing two or more groups. The log-rank test is based on several assumptions that support the validity of the calculations. It is naturally assumed, implicitly, that no errors occur in the labeling of the samples. That is, the mapping between samples and groups is perfectly correct. In this work, we investigate how test results may be affected when considering some errors in the original labeling.

Results

We introduce and define the uncertainty that arises from labeling errors in log-rank test. In order to deal with this uncertainty, we develop a novel algorithm for efficiently calculating a stability interval around the original log-rank P-value and prove its correctness. We demonstrate our algorithm on several datasets.

Availability and implementation

We provide a Python implementation, called LoRSI, for calculating the stability interval using our algorithm https://github.com/YakhiniGroup/LoRSI.

Supplementary information

Supplementary data are available at Bioinformatics online.

A Cascade Deep Forest Model for Breast Cancer Subtype Classification Using Multi-Omics Data

Automated diagnosis systems aim to reduce the cost of diagnosis while maintaining the same efficiency. Many methods have been used for breast cancer subtype classification. Some use single data source, while others integrate many data sources, the case that results in reduced computational performance as opposed to accuracy. Breast cancer data, especially biological data, is known for its imbalance, with lack of extensive amounts of histopathological images as biological data. Recent studies have shown that cascade Deep Forest ensemble model achieves a competitive classification accuracy compared with other alternatives, such as the general ensemble learning methods and the conventional deep neural networks (DNNs), especially for imbalanced training sets, through learning hyper-representations through using cascade ensemble decision trees. In this work, a cascade Deep Forest is employed to classify breast cancer subtypes, IntClust and Pam50, using multi-omics datasets and different configurations. The results obtained recorded an accuracy of 83.45% for 5 subtypes and 77.55% for 10 subtypes. The significance of this work is that it is shown that using gene expression data alone with the cascade Deep Forest classifier achieves comparable accuracy to other techniques with higher computational performance, where the time recorded is about 5 s for 10 subtypes, and 7 s for 5 subtypes.

Integration strategies of multi-omics data for machine learning analysis

Increased availability of high-throughput technologies has generated an ever-growing number of omics data that seek to portray many different but complementary biological layers including genomics, epigenomics, transcriptomics, proteomics, and metabolomics. New insight from these data have been obtained by machine learning algorithms that have produced diagnostic and classification biomarkers. Most biomarkers obtained to date however only include one omic measurement at a time and thus do not take full advantage of recent multi-omics experiments that now capture the entire complexity of biological systems.

Multi-omics data integration strategies are needed to combine the complementary knowledge brought by each omics layer. We have summarized the most recent data integration methods/ frameworks into five different integration strategies: early, mixed, intermediate, late and hierarchical. In this mini-review, we focus on challenges and existing multi-omics integration strategies by paying special attention to machine learning applications.

Artificial Intelligence in Bulk and Single-Cell RNA-Sequencing Data to Foster Precision Oncology

Artificial intelligence, or the discipline of developing computational algorithms able to perform tasks that requires human intelligence, offers the opportunity to improve our idea and delivery of precision medicine. Here, we provide an overview of artificial intelligence approaches for the analysis of large-scale RNA-sequencing datasets in cancer. We present the major solutions to disentangle inter- and intra-tumor heterogeneity of transcriptome profiles for an effective improvement of patient management. We outline the contributions of learning algorithms to the needs of cancer genomics, from identifying rare cancer subtypes to personalizing therapeutic treatments.