A Hybrid Deep Learning Model for Predicting Molecular Subtypes of Human Breast Cancer Using Multimodal Data

MOB-CBAM: A dual-channel attention-based deep learning generalizable model for breast cancer molecular subtypes prediction using mammograms

BACKGROUND AND OBJECTIVE

Deep Learning models have emerged as a significant tool in generating efficient solutions for complex problems including cancer detection, as they can analyze large amounts of data with high efficiency and performance. Recent medical studies highlight the significance of molecular subtype detection in breast cancer, aiding the development of personalized treatment plans as different subtypes of cancer respond better to different therapies.

METHODS

In this work, we propose a novel lightweight dual-channel attention-based deep learning model MOB-CBAM that utilizes the backbone of MobileNet-V3 architecture with a Convolutional Block Attention Module to make highly accurate and precise predictions about breast cancer. We used the CMMD mammogram dataset to evaluate the proposed model in our study. Nine distinct data subsets were created from the original dataset to perform coarse and fine-grained predictions, enabling it to identify masses, calcifications, benign, malignant tumors and molecular subtypes of cancer, including Luminal A, Luminal B, HER-2 Positive, and Triple Negative. The pipeline incorporates several image pre-processing techniques, including filtering, enhancement, and normalization, for enhancing the model's generalization ability.

RESULTS

While identifying benign versus malignant tumors, i.e., coarse-grained classification, the MOB-CBAM model produced exceptional results with 99 % accuracy, precision, recall, and F1-score values of 0.99 and MCC of 0.98. In terms of fine-grained classification, the MOB-CBAM model has proven to be highly efficient in accurately identifying mass with (benign/malignant) and calcification with (benign/malignant) classification tasks with an impressive accuracy rate of 98 %. We have also cross-validated the efficiency of the proposed MOB-CBAM deep learning architecture on two datasets: MIAS and CBIS-DDSM. On the MIAS dataset, an accuracy of 97 % was reported for the task of classifying benign, malignant, and normal images, while on the CBIS-DDSM dataset, an accuracy of 98 % was achieved for the classification of mass with either benign or malignant, and calcification with benign and malignant tumors.

CONCLUSION

This study presents lightweight MOB-CBAM, a novel deep learning framework, to address breast cancer diagnosis and subtype prediction. The model's innovative incorporation of the CBAM enhances precise predictions. The extensive evaluation of the CMMD dataset and cross-validation on other datasets affirm the model's efficacy.

Breast cancer prognosis through the use of multi-modal classifiers: current state of the art and the way forward

We present a survey of the current state-of-the-art in breast cancer detection and prognosis. We analyze the evolution of Artificial Intelligence-based approaches from using just uni-modal information to multi-modality for detection and how such paradigm shift facilitates the efficacy of detection, consistent with clinical observations. We conclude that interpretable AI-based predictions and ability to handle class imbalance should be considered priority.

Assessing the Influence of B-US, CDFI, SE, and Patient Age on Predicting Molecular Subtypes in Breast Lesions Using Deep Learning Algorithms

OBJECTIVES

Our study aims to investigate the impact of B-mode ultrasound (B-US) imaging, color Doppler flow imaging (CDFI), strain elastography (SE), and patient age on the prediction of molecular subtypes in breast lesions.

METHODS

Totally 2272 multimodal ultrasound imaging was collected from 198 patients. The ResNet-18 network was employed to predict four molecular subtypes from B-US imaging, CDFI, and SE of patients with different ages. All the images were split into training and testing datasets by the ratio of 80%:20%. The predictive performance on testing dataset was evaluated through 5 metrics including mean accuracy, precision, recall, F1-scores, and confusion matrix.

RESULTS

Based on B-US imaging, the test mean accuracy is 74.50%, the precision is 74.84%, the recall is 72.48%, and the F1-scores is 0.73. By combining B-US imaging with CDFI, the results were increased to 85.41%, 85.03%, 85.05%, and 0.84, respectively. With the integration of B-US imaging and SE, the results were changed to 75.64%, 74.69%, 73.86%, and 0.74, respectively. Using images from patients under 40 years old, the results were 90.48%, 90.88%, 88.47%, and 0.89. When images from patients who are above 40 years old, they were changed to 81.96%, 83.12%, 80.5%, and 0.81, respectively.

CONCLUSION

Multimodal ultrasound imaging can be used to accurately predict the molecular subtypes of breast lesions. In addition to B-US imaging, CDFI rather than SE contribute further to improve predictive performance. The predictive performance is notably better for patients under 40 years old compared with those who are 40 years old and above.

Graph-based multi-modality integration for prediction of cancer subtype and severity

Personalised cancer screening before therapy paves the way toward improving diagnostic accuracy and treatment outcomes. Most approaches are limited to a single data type and do not consider interactions between features, leaving aside the complementary insights that multimodality and systems biology can provide. In this project, we demonstrate the use of graph theory for data integration via individual networks where nodes and edges are individual-specific. We showcase the consequences of early, intermediate, and late graph-based fusion of RNA-Seq data and histopathology whole-slide images for predicting cancer subtypes and severity. The methodology developed is as follows: (1) we create individual networks; (2) we compute the similarity between individuals from these graphs; (3) we train our model on the similarity matrices; (4) we evaluate the performance using the macro F1 score. Pros and cons of elements of the pipeline are evaluated on publicly available real-life datasets. We find that graph-based methods can increase performance over methods that do not study interactions. Additionally, merging multiple data sources often improves classification compared to models based on single data, especially through intermediate fusion. The proposed workflow can easily be adapted to other disease contexts to accelerate and enhance personalized healthcare.

ForestSubtype: a cancer subtype identifying approach based on high-dimensional genomic data and a parallel random forest

BACKGROUND

Cancer subtype classification is helpful for personalized cancer treatment. Although, some approaches have been developed to classifying caner subtype based on high dimensional gene expression data, it is difficult to obtain satisfactory classification results. Meanwhile, some cancers have been well studied and classified to some subtypes, which are adopt by most researchers. Hence, this priori knowledge is significant for further identifying new meaningful subtypes.

RESULTS

In this paper, we present a combined parallel random forest and autoencoder approach for cancer subtype identification based on high dimensional gene expression data, ForestSubtype. ForestSubtype first adopts the parallel RF and the priori knowledge of cancer subtype to train a module and extract significant candidate features. Second, ForestSubtype uses a random forest as the base module and ten parallel random forests to compute each feature weight and rank them separately. Then, the intersection of the features with the larger weights output by the ten parallel random forests is taken as our subsequent candidate features. Third, ForestSubtype uses an autoencoder to condenses the selected features into a two-dimensional data. Fourth, ForestSubtype utilizes k-means++ to obtain new cancer subtype identification results. In this paper, the breast cancer gene expression data obtained from The Cancer Genome Atlas are used for training and validation, and an independent breast cancer dataset from the Molecular Taxonomy of Breast Cancer International Consortium is used for testing. Additionally, we use two other cancer datasets for validating the generalizability of ForestSubtype. ForestSubtype outperforms the other two methods in terms of the distribution of clusters, internal and external metric results. The open-source code is available at https://github.com/lffyd/ForestSubtype .

CONCLUSIONS

Our work shows that the combination of high-dimensional gene expression data and parallel random forests and autoencoder, guided by a priori knowledge, can identify new subtypes more effectively than existing methods of cancer subtype classification.

Computational Intelligence in Cancer Diagnostics: A Contemporary Review of Smart Phone Apps, Current Problems, and Future Research Potentials

Cancer is a dangerous and sometimes life-threatening disease that can have several negative consequences for the body, is a leading cause of mortality, and is becoming increasingly difficult to detect. Each form of cancer has its own set of traits, symptoms, and therapies, and early identification and management are important for a positive prognosis. Doctors utilize a variety of approaches to detect cancer, depending on the kind and location of the tumor. Imaging tests such as X-rays, Computed Tomography scans, Magnetic Resonance Imaging scans, and Positron Emission Tomography (PET) scans, which may provide precise pictures of the body's interior structures to spot any abnormalities, are some of the tools that doctors use to diagnose cancer. This article evaluates computational-intelligence approaches and provides a means to impact future work by focusing on the relevance of machine learning and deep learning models such as K Nearest Neighbour (KNN), Support Vector Machine (SVM), Naïve Bayes, Decision Tree, Deep Neural Network, Deep Boltzmann machine, and so on. It evaluates information from 114 studies using Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). This article explores the advantages and disadvantages of each model and provides an outline of how they are used in cancer diagnosis. In conclusion, artificial intelligence shows significant potential to enhance cancer imaging and diagnosis, despite the fact that there are a number of clinical issues that need to be addressed.

FDTrans: Frequency Domain Transformer Model for predicting subtypes of lung cancer using multimodal data

BACKGROUND AND PURPOSE

Accurate identification of lung cancer subtypes in medical images is of great significance for the diagnosis and treatment of lung cancer. Despite substantial progress in existing methods, they remain challenging due to limited annotated datasets, large intra-class differences, and high inter-class similarities.

METHODS

To address these challenges, we propose a Frequency Domain Transformer Model (FDTrans) to identify patients' lung cancer subtypes using the TCGA lung cancer dataset. We add a pre-processing process to transfer histopathological images to the frequency domain using a block-based discrete cosine transform and design a coordinate Coordinate-Spatial Attention Module (CSAM) to obtain critical detail information by reassigning weights to the location information and channel information of different frequency vectors. Then, a Cross-Domain Transformer Block (CDTB) is designed for Y, Cb, and Cr channel features, capturing the long-term dependencies and global contextual connections between different component features. At the same time, feature extraction is performed on the genomic data to obtain specific features. Finally, the image branch and the gene branch are fused, and the classification result is output through the fully connected layer.

RESULTS

In 10-fold cross-validation, the method achieves an AUC of 93.16% and overall accuracy of 92.33%, which is better than similar current lung cancer subtypes classification detection methods.

CONCLUSION

This method can help physicians diagnose the subtypes classification of lung cancer in patients and can benefit from both spatial and frequency domain information.

Predictive Biomarkers for Response to Immunotherapy in Triple Negative Breast Cancer: Promises and Challenges

Triple negative breast cancer (TNBC) is a highly heterogeneous disease with a poor prognosis and a paucity of therapeutic options. In recent years, immunotherapy has emerged as a new treatment option for patients with TNBC. However, this therapeutic evolution is paralleled by a growing need for biomarkers which allow for a better selection of patients who are most likely to benefit from this immune checkpoint inhibitor (ICI)-based regimen. These biomarkers will not only facilitate a better optimization of treatment strategies, but they will also avoid unnecessary side effects in non-responders, and limit the increasing financial toxicity linked to the use of these agents. Huge efforts have been deployed to identify predictive biomarkers for the ICI, but until now, the fruits of this labor remained largely unsatisfactory. Among clinically validated biomarkers, only programmed death-ligand 1 protein (PD-L1) expression has been prospectively assessed in TNBC trials. In addition to this, microsatellite instability and a high tumor mutational burden are approved as tumor agnostic biomarkers, but only a small percentage of TNBC fits this category. Furthermore, TNBC should no longer be approached as a single biological entity, but rather as a complex disease with different molecular, clinicopathological, and tumor microenvironment subgroups. This review provides an overview of the validated and evolving predictive biomarkers for a response to ICI in TNBC.

Breast cancer prediction with transcriptome profiling using feature selection and machine learning methods

BACKGROUND

We used a hybrid machine learning systems (HMLS) strategy that includes the extensive search for the discovery of the most optimal HMLSs, including feature selection algorithms, a feature extraction algorithm, and classifiers for diagnosing breast cancer. Hence, this study aims to obtain a high-importance transcriptome profile linked with classification procedures that can facilitate the early detection of breast cancer.

METHODS

In the present study, 762 breast cancer patients and 138 solid tissue normal subjects were included. Three groups of machine learning (ML) algorithms were employed: (i) four feature selection procedures are employed and compared to select the most valuable feature: (1) ANOVA; (2) Mutual Information; (3) Extra Trees Classifier; and (4) Logistic Regression (LGR), (ii) a feature extraction algorithm (Principal Component Analysis), iii) we utilized 13 classification algorithms accompanied with automated ML hyperparameter tuning, including (1) LGR; (2) Support Vector Machine; (3) Bagging; (4) Gaussian Naive Bayes; (5) Decision Tree; (6) Gradient Boosting Decision Tree; (7) K Nearest Neighborhood; (8) Bernoulli Naive Bayes; (9) Random Forest; (10) AdaBoost, (11) ExtraTrees; (12) Linear Discriminant Analysis; and (13) Multilayer Perceptron (MLP). For evaluating the proposed models' performance, balance accuracy and area under the curve (AUC) were used.

RESULTS

Feature selection procedure LGR + MLP classifier achieved the highest prediction accuracy and AUC (balanced accuracy: 0.86, AUC = 0.94), followed by an LGR + LGR classifier (balanced accuracy: 0.84, AUC = 0.94). The results showed that achieved AUC for the LGR + LGR classifier belonged to the 20 biomarkers as follows: TMEM212, SNORD115-13, ATP1A4, FRG2, CFHR4, ZCCHC13, FLJ46361, LY6G6E, ZNF323, KRT28, KRT25, LPPR5, C10orf99, PRKACG, SULT2A1, GRIN2C, EN2, GBA2, CUX2, and SNORA66.

CONCLUSIONS

The best performance was achieved using the LGR feature selection procedure and MLP classifier. Results show that the 20 biomarkers had the highest score or ranking in breast cancer detection.

Automatic classification of brain magnetic resonance images with hypercolumn deep features and machine learning

Brain tumours are life-threatening and their early detection is very important in a patient's life. At the present time, magnetic resonance imaging is one of the methods used for detecting brain tumours. Expert decision support systems serve specialist physicians to make more accurate diagnoses by minimizing the errors arising from their subjective opinions in real clinical settings. The model proposed in this study detects important keypoints and then extracts hypercolumn deep features of these keypoints from some convolutional layers of VGG16. Finally, Random Forest and Logistic Regression classifiers are fed with a set of these features. Random Forest classifier offered the best performance with 94.51% accuracy, 91.61% sensitivity, 8.39% false-negative rate, 97.42% specificity, and 97.29% precision using fivefold cross-validation in this study. Consequently, it is thought that the proposed model could contribute to field experts by integrating it into computer-aided brain magnetic resonance imaging diagnosis systems.

Multiclass Cancer Prediction Based on Copy Number Variation Using Deep Learning

DNA copy number variation (CNV) is the type of DNA variation which is associated with various human diseases. CNV ranges in size from 1 kilobase to several megabases on a chromosome. Most of the computational research for cancer classification is traditional machine learning based, which relies on handcrafted extraction and selection of features. To the best of our knowledge, the deep learning-based research also uses the step of feature extraction and selection. To understand the difference between multiple human cancers, we developed three end-to-end deep learning models, i.e., DNN (fully connected), CNN (convolution neural network), and RNN (recurrent neural network), to classify six cancer types using the CNV data of 24,174 genes. The strength of an end-to-end deep learning model lies in representation learning (automatic feature extraction). The purpose of proposing more than one model is to find which architecture among them performs better for CNV data. Our best model achieved 92% accuracy with an ROC of 0.99, and we compared the performances of our proposed models with state-of-the-art techniques. Our models have outperformed the state-of-the-art techniques in terms of accuracy, precision, and ROC. In the future, we aim to work on other types of cancers as well.

Multimodal deep learning for biomedical data fusion: a review

Biomedical data are becoming increasingly multimodal and thereby capture the underlying complex relationships among biological processes. Deep learning (DL)-based data fusion strategies are a popular approach for modeling these nonlinear relationships. Therefore, we review the current state-of-the-art of such methods and propose a detailed taxonomy that facilitates more informed choices of fusion strategies for biomedical applications, as well as research on novel methods. By doing so, we find that deep fusion strategies often outperform unimodal and shallow approaches. Additionally, the proposed subcategories of fusion strategies show different advantages and drawbacks. The review of current methods has shown that, especially for intermediate fusion strategies, joint representation learning is the preferred approach as it effectively models the complex interactions of different levels of biological organization. Finally, we note that gradual fusion, based on prior biological knowledge or on search strategies, is a promising future research path. Similarly, utilizing transfer learning might overcome sample size limitations of multimodal data sets. As these data sets become increasingly available, multimodal DL approaches present the opportunity to train holistic models that can learn the complex regulatory dynamics behind health and disease.

Computational Methods for Single-Cell Imaging and Omics Data Integration

Integrating single cell omics and single cell imaging allows for a more effective characterisation of the underlying mechanisms that drive a phenotype at the tissue level, creating a comprehensive profile at the cellular level. Although the use of imaging data is well established in biomedical research, its primary application has been to observe phenotypes at the tissue or organ level, often using medical imaging techniques such as MRI, CT, and PET. These imaging technologies complement omics-based data in biomedical research because they are helpful for identifying associations between genotype and phenotype, along with functional changes occurring at the tissue level. Single cell imaging can act as an intermediary between these levels. Meanwhile new technologies continue to arrive that can be used to interrogate the genome of single cells and its related omics datasets. As these two areas, single cell imaging and single cell omics, each advance independently with the development of novel techniques, the opportunity to integrate these data types becomes more and more attractive. This review outlines some of the technologies and methods currently available for generating, processing, and analysing single-cell omics- and imaging data, and how they could be integrated to further our understanding of complex biological phenomena like ageing. We include an emphasis on machine learning algorithms because of their ability to identify complex patterns in large multidimensional data.

Automatic Detection of Covid-19 with Bidirectional LSTM Network Using Deep Features Extracted from Chest X-ray Images

Coronavirus disease, which comes up in China at the end of 2019 and showed different symptoms in people infected, affected millions of people. Computer-aided expert systems are needed due to the inadequacy of the reverse transcription-polymerase chain reaction kit, which is widely used in the diagnosis of this disease. Undoubtedly, expert systems that provide effective solutions to many problems will be very useful in the detection of Covid-19 disease, especially when unskilled personnel and financial deficiencies in underdeveloped countries are taken into consideration. In the literature, there are numerous machine learning approaches built with different classifiers in the detection of this disease. This paper proposes an approach based on deep learning which detects Covid-19 and no-finding cases using chest X-ray images. Here, the classification performance of the Bi-LSTM network on the deep features was compared with the Deep Neural Network within the frame of the fivefold cross-validation technique. Accuracy, sensitivity, specificity and precision metrics were used to evaluate the classification performance of the trained models. Bi-LSTM network presented better performance compare to DNN with 97.6% value of high accuracy despite the few numbers of Covid-19 images in the dataset. In addition, it is understood that concatenated deep features more meaningful than deep features obtained with pre-trained networks by one by, as well. Consequently, it is thought that the proposed study based on the Bi-LSTM network and concatenated deep features will be noteworthy in the design of highly sensitive automated Covid-19 monitoring systems.

Multi-Modal Evolutionary Deep Learning Model for Ovarian Cancer Diagnosis

Ovarian cancer (OC) is a common reason for mortality among women. Deep learning has recently proven better performance in predicting OC stages and subtypes. However, most of the state-of-the-art deep learning models employ single modality data, which may afford low-level performance due to insufficient representation of important OC characteristics. Furthermore, these deep learning models still lack to the optimization of the model construction, which requires high computational cost to train and deploy them. In this work, a hybrid evolutionary deep learning model, using multi-modal data, is proposed. The established multi-modal fusion framework amalgamates gene modality alongside with histopathological image modality. Based on the different states and forms of each modality, we set up deep feature extraction network, respectively. This includes a predictive antlion-optimized long-short-term-memory model to process gene longitudinal data. Another predictive antlion-optimized convolutional neural network model is included to process histopathology images. The topology of each customized feature network is automatically set by the antlion optimization algorithm to make it realize better performance. After that the output from the two improved networks is fused based upon weighted linear aggregation. The deep fused features are finally used to predict OC stage. A number of assessment indicators was used to compare the proposed model to other nine multi-modal fusion models constructed using distinct evolutionary algorithms. This was conducted using a benchmark for OC and two benchmarks for breast and lung cancers. The results reveal that the proposed model is more precise and accurate in diagnosing OC and the other cancers.