Evaluating the Microsatellite Instability of Colorectal Cancer Based on Multimodal Deep Learning Integrating Histopathological and Molecular Data

Prognostic and predictive molecular biomarkers in colorectal cancer

Precision medicine has brought revolutionary changes to the diagnosis and treatment of cancer patients, and is currently a hot and challenging research topic. Currently, the treatment regimens for most colorectal cancer (CRC) patients are mainly determined by several biomakers, including Microsatellite Instability (MSI), RAS, and BRAF. However, the roles of promising biomarkers such as HER-2, consensus molecular subtypes (CMS), and circulating tumor DNA (ctDNA) in CRC are not yet fully clear. Therefore, it is urgent to explore the potential of these emerging biomarkers in the diagnosis and treatment of CRC patients. In this paper, we discuss recent advances in CRC biomarkers, especially clinical data, and focus on the roles of biomarkers in prognosis, prediction, treatment strategies, and the intrinsic connections with clinical pathological features, hoping to promote better precision medicine for colorectal cancer.

HCTTI: High-Performance Heterogeneous Computing Toolkit for Tissue Image Stain Normalization

Whole slide imaging (WSI) has transformed diagnostic medicine, particularly in the field of cancer diagnosis and treatment. The use of deep learning algorithms for predicting WSIs has opened up new avenues for advanced medical diagnostics. Additionally, stain normalization can reduce the color and intensity variations present in WSI from different hospitals. As a result, deep learning classification accuracy improves. However, WSI reading and color normalization are still largely performed by using CPUs, leading to sub-optimal performance. We proposed a High-Performance Heterogeneous Computing Toolkit for Tissue Image (HCTTI) that integrates multiple computer system-level optimizations and encompasses WSI reading, tile normalization, and tile saving. We explored the potential advantages and limitations of different WSI readers and color normalization techniques in WSI analysis and the performance of different tile serialization formats for saving tiles. We found that HCTTI is 7 × faster than OpenSlide for reading WSIs, GPU implementation of the Macenko normalization algorithm is 9 × faster than TIAToolbox implementation, and HDF5 is faster than png and Zarr for storing normalized images in both writing (13 × acceleration compared to png) and reading (2 × acceleration compared to png), Specifically, HDF5 provides superior performance in handling large, complex datasets due to its efficient chunking and compression capabilities, as well as its broad support for hierarchical data management, making it very suitable for workloads like deep learning training I/O pattern that involves randomly reading large amount of small files in each training epoch. We also achieved linear acceleration in our multi-node distributed GPU implementation. To our knowledge, HCTTI is the first comprehensive toolkit that comprises distributed WSI reading, normalization, and serialization. It is 13 × speedup compared to TIAToolbox implementation for normalizing a single WSI. Our findings could help pave the way for more effective and efficient deep learning-based approaches to WSI analysis, with the potential to transform medical diagnosis and treatment for a wide range of conditions. The source code of HCTTI is available at https://github.com/wangbo00129/HCTTI .

Advancing healthcare through multimodal data fusion: a comprehensive review of techniques and applications

With the increasing availability of diverse healthcare data sources, such as medical images and electronic health records, there is a growing need to effectively integrate and fuse this multimodal data for comprehensive analysis and decision-making. However, despite its potential, multimodal data fusion in healthcare remains limited. This review paper provides an overview of existing literature on multimodal data fusion in healthcare, covering 69 relevant works published between 2018 and 2024. It focuses on methodologies that integrate different data types to enhance medical analysis, including techniques for integrating medical images with structured and unstructured data, combining multiple image modalities, and other features. Additionally, the paper reviews various approaches to multimodal data fusion, such as early, intermediate, and late fusion methods, and examines the challenges and limitations associated with these techniques. The potential benefits and applications of multimodal data fusion in various diseases are highlighted, illustrating specific strategies employed in healthcare artificial intelligence (AI) model development. This research synthesizes existing information to facilitate progress in using multimodal data for improved medical diagnosis and treatment planning.

Deep learning application in prediction of cancer molecular alterations based on pathological images: a bibliographic analysis via CiteSpace

BACKGROUND

The advancements in artificial intelligence (AI) technology for image recognition were propelling molecular pathology research into a new era.

OBJECTIVE

To summarize the hot spots and research trends in the field of molecular pathology image recognition.

METHODS

Relevant articles from January 1st, 2010, to August 25th, 2023, were retrieved from the Web of Science Core Collection. Subsequently, CiteSpace was employed for bibliometric and visual analysis, generating diverse network diagrams illustrating keywords, highly cited references, hot topics, and research trends.

RESULTS

A total of 110 relevant articles were extracted from a pool of 10,205 articles. The overall publication count exhibited a rising trend each year. The leading contributors in terms of institutions, countries, and authors were Maastricht University (11 articles), the United States (38 articles), and Kather Jacob Nicholas (9 articles), respectively. Half of the top ten research institutions, based on publication volume, were affiliated with Germany. The most frequently cited article was authored by Nicolas Coudray et al. accumulating 703 citations. The keyword "Deep learning" had the highest frequency in 2019. Notably, the highlighted keywords from 2022 to 2023 included "microsatellite instability", and there were 21 articles focusing on utilizing algorithms to recognize microsatellite instability (MSI) in colorectal cancer (CRC) pathological images.

CONCLUSION

The use of DL is expected to provide a new strategy to effectively solve the current problem of time-consuming and expensive molecular pathology detection. Therefore, further research is needed to address issues, such as data quality and standardization, model interpretability, and resource and infrastructure requirements.

Precision prognosis of colorectal cancer: a multi-tiered model integrating microsatellite instability genes and clinical parameters

Background

Prognostic assessment for colorectal cancer (CRC) displays substantial heterogeneity, as reliance solely on traditional TNM staging falls short of achieving precise individualized predictions. The integration of diverse biological information sources holds the potential to enhance prognostic accuracy.

Objective

To establish a comprehensive multi-tiered precision prognostic evaluation system for CRC by amalgamating gene expression profiles, clinical characteristics, and tumor microsatellite instability (MSI) status in CRC patients.

Methods

We integrated genomic data, clinical information, and survival follow-up data from 483 CRC patients obtained from The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) databases. MSI-related gene modules were identified using differential expression analysis and Weighted Gene Co-expression Network Analysis (WGCNA). Three prognostic models were constructed: MSI-Related Gene Prognostic Model (Model I), Clinical Prognostic Model (Model II), and Integrated Multi-Layered Prognostic Model (Model III) by combining clinical features. Model performance was assessed and compared using Receiver Operating Characteristic (ROC) curves, Kaplan-Meier analysis, and other methods.

Results

Six MSI-related genes were selected for constructing Model I (AUC = 0.724); Model II used two clinical features (AUC = 0.684). Compared to individual models, the integrated Model III exhibited superior performance (AUC = 0.825) and demonstrated good stability in an independent dataset (AUC = 0.767).

Conclusion

This study successfully developed and validated a comprehensive multi-tiered precision prognostic assessment model for CRC, providing an effective tool for personalized medical management of CRC.

Predicting rectal cancer prognosis from histopathological images and clinical information using multi-modal deep learning

Objective

The objective of this study was to provide a multi-modal deep learning framework for forecasting the survival of rectal cancer patients by utilizing both digital pathological images data and non-imaging clinical data.

Materials and methods

The research included patients diagnosed with rectal cancer by pathological confirmation from January 2015 to December 2016. Patients were allocated to training and testing sets in a randomized manner, with a ratio of 4:1. The tissue microarrays (TMAs) and clinical indicators were obtained. Subsequently, we selected distinct deep learning models to individually forecast patient survival. We conducted a scanning procedure on the TMAs in order to transform them into digital pathology pictures. Additionally, we performed pre-processing on the clinical data of the patients. Subsequently, we selected distinct deep learning algorithms to conduct survival prediction analysis using patients' pathological images and clinical data, respectively.

Results

A total of 292 patients with rectal cancer were randomly allocated into two groups: a training set consisting of 234 cases, and a testing set consisting of 58 instances. Initially, we make direct predictions about the survival status by using pre-processed Hematoxylin and Eosin (H&E) pathological images of rectal cancer. We utilized the ResNest model to extract data from histopathological images of patients, resulting in a survival status prediction with an AUC (Area Under the Curve) of 0.797. Furthermore, we employ a multi-head attention fusion (MHAF) model to combine image features and clinical features in order to accurately forecast the survival rate of rectal cancer patients. The findings of our experiment show that the multi-modal structure works better than directly predicting from histopathological images. It achieves an AUC of 0.837 in predicting overall survival (OS).

Conclusions

Our study highlights the potential of multi-modal deep learning models in predicting survival status from histopathological images and clinical information, thus offering valuable insights for clinical applications.

Detecting microsatellite instability in colorectal cancer using Transformer-based colonoscopy image classification and retrieval

Colorectal cancer (CRC) is a major global health concern, with microsatellite instability-high (MSI-H) being a defining characteristic of hereditary nonpolyposis colorectal cancer syndrome and affecting 15% of sporadic CRCs. Tumors with MSI-H have unique features and better prognosis compared to MSI-L and microsatellite stable (MSS) tumors. This study proposed establishing a MSI prediction model using more available and low-cost colonoscopy images instead of histopathology. The experiment utilized a database of 427 MSI-H and 1590 MSS colonoscopy images and vision Transformer (ViT) with different feature training approaches to establish the MSI prediction model. The accuracy of combining pre-trained ViT features was 84% with an area under the receiver operating characteristic curve of 0.86, which was better than that of DenseNet201 (80%, 0.80) in the experiment with support vector machine. The content-based image retrieval (CBIR) approach showed that ViT features can obtain a mean average precision of 0.81 compared to 0.79 of DenseNet201. ViT reduced the issues that occur in convolutional neural networks, including limited receptive field and gradient disappearance, and may be better at interpreting diagnostic information around tumors and surrounding tissues. By using CBIR, the presentation of similar images with the same MSI status would provide more convincing deep learning suggestions for clinical use.

Long Non-Coding RNAs in Colorectal Cancer: Navigating the Intersections of Immunity, Intercellular Communication, and Therapeutic Potential

This comprehensive review elucidates the intricate roles of long non-coding RNAs (lncRNAs) within the colorectal cancer (CRC) microenvironment, intersecting the domains of immunity, intercellular communication, and therapeutic potential. lncRNAs, which are significantly involved in the pathogenesis of CRC, immune evasion, and the treatment response to CRC, have crucial implications in inflammation and serve as promising candidates for novel therapeutic strategies and biomarkers. This review scrutinizes the interaction of lncRNAs with the Consensus Molecular Subtypes (CMSs) of CRC, their complex interplay with the tumor stroma affecting immunity and inflammation, and their conveyance via extracellular vesicles, particularly exosomes. Furthermore, we delve into the intricate relationship between lncRNAs and other non-coding RNAs, including microRNAs and circular RNAs, in mediating cell-to-cell communication within the CRC microenvironment. Lastly, we propose potential strategies to manipulate lncRNAs to enhance anti-tumor immunity, thereby underlining the significance of lncRNAs in devising innovative therapeutic interventions in CRC.

The role of deep learning in diagnosing colorectal cancer

Colon cancer is a major public health issue, affecting a growing number of individuals worldwide. Proper and early diagnosis of colon cancer is the necessary first step toward effective treatment and/or prevention of future disease relapse. Artificial intelligence and its subtypes, deep learning in particular, tend nowadays to have an expanding role in all fields of medicine, and diagnosing colon cancer is no exception. This report aims to summarize the entire application spectrum of deep learning in all diagnostic tests regarding colon cancer, from endoscopy and histologic examination to medical imaging and screening serologic tests.

Long non-coding RNA signature in colorectal cancer: research progression and clinical application

Colorectal cancer is one of the top-ranked human malignancies. The development and progression of colorectal cancer are associated with aberrant expression of multiple coding and non-coding genes. Long non-coding RNAs (lncRNAs) have an important role in regulating gene stability as well as gene expression. Numerous current studies have shown that lncRNAs are promising biomarkers and therapeutic targets for colorectal cancer. In this review, we have searched the available literature to list lncRNAs involved in the pathogenesis and regulation of colorectal cancer. We focus on the role of lncRNAs in cancer promotion or suppression, their value in tumor diagnosis, and their role in treatment response and prognosis prediction. In addition, we will discuss the signaling pathways that these lncRNAs are mainly associated with in colorectal cancer. We also summarize the role of lncRNAs in colorectal precancerous lesions and colorectal cancer consensus molecular subgroups. We hope this review article will bring you the latest research progress and outlook on lncRNAs in colorectal cancer.