Clinical-Grade Detection of Microsatellite Instability in Colorectal Tumors by Deep Learning

Artificial intelligence and whole slide imaging, a new tool for the microsatellite instability prediction in colorectal cancer: Friend or foe?

Colorectal cancer (CRC) is the third most common and second most deadly cancer worldwide. Despite advances in screening and treatment, CRC is heterogeneous and the response to therapy varies significantly, limiting personalized treatment options. Certain molecular biomarkers, including microsatellite instability (MSI), are critical in planning personalized treatment, although only a subset of patients may benefit. Currently, the primary methods for assessing MSI status include immunohistochemistry (IHC) for DNA mismatch repair proteins (MMRs), polymerase chain reaction (PCR)-based molecular testing, or next-generation sequencing (NGS). However, these techniques have limitations, are expensive and time-consuming, and often result in inter-method inconsistencies. Deficient mismatch repair (dMMR) or high microsatellite instability (MSI-H) are critical predictive biomarkers of response to immune checkpoint inhibitor (ICI) therapy and MSI testing is recommended to identify patients who may benefit. There is a pressing need for a more robust, reliable, and cost-effective approach that accurately assesses MSI status. Recent advances in computational pathology, in particular the development of technologies that digitally scan whole slide images (WSI) at high resolution, as well as new approaches to artificial intelligence (AI) in medicine, are increasingly gaining ground. This review aims to provide an overview of the latest findings on WSI and advances in AI methods for predicting MSI status, summarize their applications in CRC, and discuss their strengths and limitations in daily clinical practice.

Deep Gaussian process with uncertainty estimation for microsatellite instability and immunotherapy response prediction from histology

Determining tumor microsatellite status has significant clinical value because tumors that are microsatellite instability-high (MSI-H) or mismatch repair deficient (dMMR) respond well to immune checkpoint inhibitors (ICIs) and oftentimes not to chemotherapeutics. We propose MSI-SEER, a deep Gaussian process-based Bayesian model that analyzes H&E whole-slide images in weakly-supervised-learning to predict microsatellite status in gastric and colorectal cancers. We performed extensive validation using multiple large datasets comprised of patients from diverse racial backgrounds. MSI-SEER achieved state-of-the-art performance with MSI prediction by integrating uncertainty prediction. We achieved high accuracy for predicting ICI responsiveness by combining tumor MSI status with stroma-to-tumor ratio. Finally, MSI-SEER's tile-level predictions revealed novel insights into the role of spatial distribution of MSI-H regions in the tumor microenvironment and ICI response.

Artificial intelligence entering the pathology arena in oncology: current applications and future perspectives

BACKGROUND

Artificial intelligence (AI) is rapidly transforming the fields of pathology and oncology, offering novel opportunities for advancing diagnosis, prognosis, and treatment of cancer.

METHODS

Through a systematic review-based approach, the representatives from the European Society for Medical Oncology (ESMO) Precision Oncology Working Group (POWG) and international experts identified studies in pathology and oncology that applied AI-based algorithms for tumour diagnosis, molecular biomarker detection, and cancer prognosis assessment. These findings were synthesised to provide a comprehensive overview of current AI applications and future directions in cancer pathology.

RESULTS

The integration of AI tools in digital pathology is markedly improving the accuracy and efficiency of image analysis, allowing for automated tumour detection and classification, identification of prognostic molecular biomarkers, and prediction of treatment response and patient outcomes. Several barriers for the adoption of AI in clinical workflows, such as data availability, explainability, and regulatory considerations, still persist. There are currently no prognostic or predictive AI-based biomarkers supported by level IA or IB evidence. The ongoing advancements in AI algorithms, particularly foundation models, generalist models and transformer-based deep learning, offer immense promise for the future of cancer research and care. AI is also facilitating the integration of multi-omics data, leading to more precise patient stratification and personalised treatment strategies.

CONCLUSIONS

The application of AI in pathology is poised to not only enhance the accuracy and efficiency of cancer diagnosis and prognosis but also facilitate the development of personalised treatment strategies. Although barriers to implementation remain, ongoing research and development in this field coupled with addressing ethical and regulatory considerations will likely lead to a future where AI plays an integral role in cancer management and precision medicine. The continued evolution and adoption of AI in pathology and oncology are anticipated to reshape the landscape of cancer care, heralding a new era of precision medicine and improved patient outcomes.

Defining a 'cells to society' research framework for appendiceal tumours

Tumours of the appendix - a vestigial digestive organ attached to the colon - are rare. Although we estimate that around 3,000 new appendiceal cancer cases are diagnosed annually in the USA, the challenges of accurately diagnosing and identifying this tumour type suggest that this number may underestimate true population incidence. In the current absence of disease-specific screening and diagnostic imaging modalities, or well-established risk factors, the incidental discovery of appendix tumours is often prompted by acute presentations mimicking appendicitis or when the tumour has already spread into the abdominal cavity - wherein the potential misclassification of appendiceal tumours as malignancies of the colon and ovaries also increases. Notwithstanding these diagnostic difficulties, our understanding of appendix carcinogenesis has advanced in recent years. However, there persist considerable challenges to accelerating the pace of research discoveries towards the path to improved treatments and cures for patients with this group of orphan malignancies. The premise of this Expert Recommendation article is to discuss the current state of the field, to delineate unique challenges for the study of appendiceal tumours, and to propose key priority research areas that will deliver a more complete picture of appendix carcinogenesis and metastasis. The Appendix Cancer Pseudomyxoma Peritonei (ACPMP) Research Foundation Scientific Think Tank delivered a consensus of core research priorities for appendiceal tumours that are poised to be ground-breaking and transformative for scientific discovery and innovation. On the basis of these six research areas, here, we define the first 'cells to society' research framework for appendix tumours.

The KMeansGraphMIL Model: A Weakly Supervised Multiple Instance Learning Model for Predicting Colorectal Cancer Tumor Mutational Burden

Colorectal cancer (CRC) is one of the top three most lethal malignancies worldwide, posing a significant threat to human health. Recently proposed immunotherapy checkpoint blockade treatments have proven effective for CRC, but their use depends on measuring specific biomarkers in patients. Among these biomarkers, tumor mutational burden (TMB) has emerged as a novel indicator, traditionally requiring next-generation sequencing for measurement, which is time-consuming, labor intensive, and costly. To provide an economical and rapid way to predict patients' TMB, the KMeansGraphMIL model was proposed based on weakly supervised multiple-instance learning. Compared with previous weakly supervised multiple-instance learning models, KMeansGraphMIL leveraged both the similarity of image patch feature vectors and the spatial relationships between patches. This approach improved the model's area under the receiver operating characteristic curve to 0.8334 and significantly increased the recall to 0.7556. Thus, this study presents an economical and rapid framework for predicting CRC TMB, offering the potential for physicians to quickly develop treatment plans and saving patients substantial time and money.

Assessing Genotype-Phenotype Correlations with Deep Learning in Colorectal Cancer: A Multi-Centric Study

Background

Deep Learning (DL) has emerged as a powerful tool to predict genetic biomarkers directly from digitized Hematoxylin and Eosin (H&E) slides in colorectal cancer (CRC). However, few studies have systematically investigated the predictability of biomarkers beyond routinely available alterations such as microsatellite instability (MSI), and BRAF and KRAS mutations.

Methods

Our primary dataset comprised H&E slides of CRC tumors across five cohorts totaling 1,376 patients who underwent comprehensive panel sequencing, with an additional 536 patients from two public datasets for validation. We developed a DL model using a single transformer model to predict multiple genetic alterations directly from the slides. The model's performance was compared against conventional single-target models, and potential confounders were analyzed.

Findings

The multi-target model was able to predict numerous biomarkers from pathology slides, matching and partly exceeding single-target transformers. The Area Under the Receiver Operating Characteristic curve (AUROC, mean ± std) on the primary external validation cohorts was: BRAF (0·78 ± 0·01), hypermutation (0·88 ± 0·01), MSI (0·93 ± 0·01), RNF43 (0·86 ± 0·01); this biomarker predictability was mirrored across metrics and co-occurrence analyses. However, biomarkers with high AUROCs largely correlated with MSI, with model predictions depending considerably on MSI-associated morphology upon pathological examination.

Interpretation

Our study demonstrates that multi-target transformers can predict the biomarker status for numerous genetic alterations in CRC directly from H&E slides. However, their predictability is mainly associated with MSI phenotype, despite indications of slight biomarker-inherent contributions to a phenotype. Our findings underscore the need to analyze confounders in AI-based oncology biomarkers. To enable this, we developed a validated model applicable to other cancers and larger, diverse datasets.

Funding

The German Federal Ministry of Health, the Max-Eder-Programme of German Cancer Aid, the German Federal Ministry of Education and Research, the German Academic Exchange Service, and the EU.

Deep learning-assisted colonoscopy images for prediction of mismatch repair deficiency in colorectal cancer

BACKGROUND

Deficient mismatch repair or microsatellite instability is a major predictive biomarker for the efficacy of immune checkpoint inhibitors of colorectal cancer. However, routine testing has not been uniformly implemented due to cost and resource constraints.

METHODS

We developed and validated a deep learning-based classifiers to detect mismatch repair-deficient status from routine colonoscopy images. We obtained the colonoscopy images from the imaging database at Endoscopic Center of the Sixth Affiliated Hospital, Sun Yat-sen University. Colonoscopy images from a prospective trial (Neoadjuvant PD-1 blockade by toripalimab with or without celecoxib in mismatch repair-deficient or microsatellite instability-high locally advanced colorectal cancer) were used to test the model.

RESULTS

A total of 5226 eligible images from 892 tumors from the consecutive patients were utilized to develop and validate the deep learning model. 2105 colorectal cancer images from 306 tumors were randomly selected to form model development dataset with a class-balanced approach. 3121 images of 488 proficient mismatch repair tumors and 98 deficient mismatch repair tumors were used to form the independent dataset. The model achieved an AUROC of 0.948 (95% CI 0.919-0.977) on the test dataset. On the independent validation dataset, the AUROC was 0.807 (0.760-0.854), and the NPV in was 94.2% (95% CI 0.918-0.967). On the prospective trial dataset, the model identified 29 tumors among the 33 deficient mismatch repair tumors (87.88%).

CONCLUSIONS

The model achieved a high NPV in detecting deficient mismatch repair colorectal cancers. This model might serve as an automatic screening tool.

A CT-based deep learning for segmenting tumors and predicting microsatellite instability in patients with colorectal cancers: a multicenter cohort study

PURPOSE

To develop and validate deep learning (DL) models using preoperative contrast-enhanced CT images for tumor auto-segmentation and microsatellite instability (MSI) prediction in colorectal cancer (CRC).

MATERIALS AND METHODS

Patients with CRC who underwent surgery or biopsy between January 2018 and April 2023 were retrospectively enrolled. Mismatch repair protein expression was determined via immunohistochemistry or fluorescence multiplex polymerase chain reaction-capillary electrophoresis. Manually delineated tumor contours using arterial and venous phase CT images by three abdominal radiologists are served as ground truth. Tumor auto-segmentation used nnU-Net. MSI prediction employed ViT or convolutional neural networks models, trained and validated with arterial and venous phase images (image model) or combined clinical-pathological factors (combined model). The segmentation model was evaluated using patch coverage ratio, Dice coefficient, recall, precision, and F1-score. The predictive models' efficacy was assessed using areas under the curves and decision curve analysis.

RESULTS

Overall, 2180 patients (median age: 61 years ± 17 [SD]; 1285 males) were divided into training (n = 1159), validation (n = 289), and independent external test (n = 732) groups. High-level MSI status was present in 435 patients (20%). In the external test set, the segmentation model performed well in the arterial phase, with patch coverage ratio, Dice coefficient, recall, precision, and F1-score values of 0.87, 0.71, 0.72, 0.74, and 0.71, respectively. For MSI prediction, the combined models outperformed the clinical model (AUC = 0.83 and 0.82 vs 0.67, p < 0.001) and two image models (AUC = 0.75 and 0.77, p < 0.001). Decision curve analysis confirmed the higher net benefit of the combined model compared to the other models across probability thresholds ranging from 0.1 to 0.45.

CONCLUSION

DL enhances tumor segmentation efficiency and, when integrated with contrast-enhanced CT and clinicopathological factors, exhibits good diagnostic performance in predicting MSI in CRC.

CIMIL-CRC: A clinically-informed multiple instance learning framework for patient-level colorectal cancer molecular subtypes classification from H&E stained images

BACKGROUND AND OBJECTIVE

Treatment approaches for colorectal cancer (CRC) are highly dependent on the molecular subtype, as immunotherapy has shown efficacy in cases with microsatellite instability (MSI) but is ineffective for the microsatellite stable (MSS) subtype. There is promising potential in utilizing deep neural networks (DNNs) to automate the differentiation of CRC subtypes by analyzing hematoxylin and eosin (H&E) stained whole-slide images (WSIs). Due to the extensive size of WSIs, multiple instance learning (MIL) techniques are typically explored. However, existing MIL methods focus on identifying the most representative image patches for classification, which may result in the loss of critical information. Additionally, these methods often overlook clinically relevant information, like the tendency for MSI class tumors to predominantly occur on the proximal (right side) colon.

METHODS

We introduce 'CIMIL-CRC', a DNN framework that: (1) solves the MSI/MSS MIL problem by efficiently combining a pre-trained feature extraction model with principal component analysis (PCA) to aggregate information from all patches, and (2) integrates clinical priors, particularly the tumor location within the colon, into the model to enhance patient-level classification accuracy. We assessed our CIMIL-CRC method using the average area under the receiver operating characteristic curve (AUROC) from a 5-fold cross-validation experimental setup for model development on the TCGA-CRC-DX cohort, contrasting it with a baseline patch-level classification, a MIL-only approach, and a clinically-informed patch-level classification approach.

RESULTS

Our CIMIL-CRC outperformed all methods (AUROC: 0.92±0.002 (95% CI 0.91-0.92), vs. 0.79±0.02 (95% CI 0.76-0.82), 0.86±0.01 (95% CI 0.85-0.88), and 0.87±0.01 (95% CI 0.86-0.88), respectively). The improvement was statistically significant. To the best of our knowledge, this is the best result achieved for MSI/MSS classification on this dataset.

CONCLUSION

Our CIMIL-CRC method holds promise for offering insights into the key representations of histopathological images and suggests a straightforward implementation.

Computational pathology applied to clinical colorectal cancer cohorts identifies immune and endothelial cell spatial patterns predictive of outcome

Colorectal cancer (CRC) is a histologically heterogeneous disease with variable clinical outcome. The role the tumour microenvironment (TME) plays in determining tumour progression is complex and not fully understood. To improve our understanding, it is critical that the TME is studied systematically within clinically annotated patient cohorts with long-term follow-up. Here we studied the TME in three clinical cohorts of metastatic CRC with diverse molecular subtype and treatment history. The MISSONI cohort included cases with microsatellite instability that received immunotherapy (n = 59, 24 months median follow-up). The BRAF cohort included BRAF V600E mutant microsatellite stable (MSS) cancers (n = 141, 24 months median follow-up). The VALENTINO cohort included RAS/RAF WT MSS cases who received chemotherapy and anti-EGFR therapy (n = 175, 32 months median follow-up). Using a Deep learning cell classifier, trained upon >38,000 pathologist annotations, to detect eight cell types within H&E-stained sections of CRC, we quantified the spatial tissue organisation and colocalisation of cell types across these cohorts. We found that the ratio of infiltrating endothelial cells to cancer cells, a possible marker of vascular invasion, was an independent predictor of progression-free survival (PFS) in the BRAF+MISSONI cohort (p = 0.033, HR = 1.44, CI = 1.029-2.01). In the VALENTINO cohort, this pattern was also an independent PFS predictor in TP53 mutant patients (p = 0.009, HR = 0.59, CI = 0.40-0.88). Tumour-infiltrating lymphocytes were an independent predictor of PFS in BRAF+MISSONI (p = 0.016, HR = 0.36, CI = 0.153-0.83). Elevated tumour-infiltrating macrophages were predictive of improved PFS in the MISSONI cohort (p = 0.031). We validated our cell classification using highly multiplexed immunofluorescence for 17 markers applied to the same sections that were analysed by the classifier (n = 26 cases). These findings uncovered important microenvironmental factors that underpin treatment response across and within CRC molecular subtypes, while providing an atlas of the distribution of 180 million cells in 375 clinically annotated CRC patients. © 2025 The Author(s). The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Convergence of evolving artificial intelligence and machine learning techniques in precision oncology

The confluence of new technologies with artificial intelligence (AI) and machine learning (ML) analytical techniques is rapidly advancing the field of precision oncology, promising to improve diagnostic approaches and therapeutic strategies for patients with cancer. By analyzing multi-dimensional, multiomic, spatial pathology, and radiomic data, these technologies enable a deeper understanding of the intricate molecular pathways, aiding in the identification of critical nodes within the tumor's biology to optimize treatment selection. The applications of AI/ML in precision oncology are extensive and include the generation of synthetic data, e.g., digital twins, in order to provide the necessary information to design or expedite the conduct of clinical trials. Currently, many operational and technical challenges exist related to data technology, engineering, and storage; algorithm development and structures; quality and quantity of the data and the analytical pipeline; data sharing and generalizability; and the incorporation of these technologies into the current clinical workflow and reimbursement models.

Partial-Label Contrastive Representation Learning for Fine-Grained Biomarkers Prediction From Histopathology Whole Slide Images

In the domain of histopathology analysis, existing representation learning methods for biomarkers prediction from whole slide images (WSIs) face challenges due to the complexity of tissue subtypes and label noise problems. This paper proposed a novel partial-label contrastive representation learning approach to enhance the discrimination of histopathology image representations for fine-grained biomarkers prediction. We designed a partial-label contrastive clustering (PLCC) module for partial-label disambiguation and a dynamic clustering algorithm to sample the most representative features of each category to the clustering queue during the contrastive learning process. We conducted comprehensive experiments on three gene mutation prediction datasets, including USTC-EGFR, BRCA-HER2, and TCGA-EGFR. The results show that our method outperforms 9 existing methods in terms of Accuracy, AUC, and F1 Score. Specifically, our method achieved an AUC of 0.950 in EGFR mutation subtyping of TCGA-EGFR and an AUC of 0.853 in HER2 0/1+/2+/3+ grading of BRCA-HER2, which demonstrates its superiority in fine-grained biomarkers prediction from histopathology whole slide images.

Ensemble transformer-based multiple instance learning to predict pathological subtypes and tumor mutational burden from histopathological whole slide images of endometrial and colorectal cancer

In endometrial cancer (EC) and colorectal cancer (CRC), in addition to microsatellite instability, tumor mutational burden (TMB) has gradually gained attention as a genomic biomarker that can be used clinically to determine which patients may benefit from immune checkpoint inhibitors. High TMB is characterized by a large number of mutated genes, which encode aberrant tumor neoantigens, and implies a better response to immunotherapy. Hence, a part of EC and CRC patients associated with high TMB may have higher chances to receive immunotherapy. TMB measurement was mainly evaluated by whole-exome sequencing or next-generation sequencing, which was costly and difficult to be widely applied in all clinical cases. Therefore, an effective, efficient, low-cost and easily accessible tool is urgently needed to distinguish the TMB status of EC and CRC patients. In this study, we present a deep learning framework, namely Ensemble Transformer-based Multiple Instance Learning with Self-Supervised Learning Vision Transformer feature encoder (ETMIL-SSLViT), to predict pathological subtype and TMB status directly from the H&E stained whole slide images (WSIs) in EC and CRC patients, which is helpful for both pathological classification and cancer treatment planning. Our framework was evaluated on two different cancer cohorts, including an EC cohort with 918 histopathology WSIs from 529 patients and a CRC cohort with 1495 WSIs from 594 patients from The Cancer Genome Atlas. The experimental results show that the proposed methods achieved excellent performance and outperforming seven state-of-the-art (SOTA) methods in cancer subtype classification and TMB prediction on both cancer datasets. Fisher's exact test further validated that the associations between the predictions of the proposed models and the actual cancer subtype or TMB status are both extremely strong (p<0.001). These promising findings show the potential of our proposed methods to guide personalized treatment decisions by accurately predicting the EC and CRC subtype and the TMB status for effective immunotherapy planning for EC and CRC patients.

Role of artificial intelligence in haematolymphoid diagnostics

The advent of digital pathology and the deployment of high-throughput molecular techniques are generating an unprecedented mass of data. Thanks to advances in computational sciences, artificial intelligence (AI) approaches represent a promising avenue for extracting relevant information from complex data structures. From diagnostic assistance to powerful research tools, the potential fields of application of machine learning techniques in pathology are vast and constitute the subject of considerable research work. The aim of this article is to provide an overview of the potential applications of AI in the field of haematopathology and to define the role that these emerging technologies could play in our laboratories in the short to medium term.

Artificial intelligence in drug development

Drug development is a complex and time-consuming endeavor that traditionally relies on the experience of drug developers and trial-and-error experimentation. The advent of artificial intelligence (AI) technologies, particularly emerging large language models and generative AI, is poised to redefine this paradigm. The integration of AI-driven methodologies into the drug development pipeline has already heralded subtle yet meaningful enhancements in both the efficiency and effectiveness of this process. Here we present an overview of recent advancements in AI applications across the entire drug development workflow, encompassing the identification of disease targets, drug discovery, preclinical and clinical studies, and post-market surveillance. Lastly, we critically examine the prevailing challenges to highlight promising future research directions in AI-augmented drug development.

From whole-slide image to biomarker prediction: end-to-end weakly supervised deep learning in computational pathology

Hematoxylin- and eosin-stained whole-slide images (WSIs) are the foundation of diagnosis of cancer. In recent years, development of deep learning-based methods in computational pathology has enabled the prediction of biomarkers directly from WSIs. However, accurately linking tissue phenotype to biomarkers at scale remains a crucial challenge for democratizing complex biomarkers in precision oncology. This protocol describes a practical workflow for solid tumor associative modeling in pathology (STAMP), enabling prediction of biomarkers directly from WSIs by using deep learning. The STAMP workflow is biomarker agnostic and allows for genetic and clinicopathologic tabular data to be included as an additional input, together with histopathology images. The protocol consists of five main stages that have been successfully applied to various research problems: formal problem definition, data preprocessing, modeling, evaluation and clinical translation. The STAMP workflow differentiates itself through its focus on serving as a collaborative framework that can be used by clinicians and engineers alike for setting up research projects in the field of computational pathology. As an example task, we applied STAMP to the prediction of microsatellite instability (MSI) status in colorectal cancer, showing accurate performance for the identification of tumors high in MSI. Moreover, we provide an open-source code base, which has been deployed at several hospitals across the globe to set up computational pathology workflows. The STAMP workflow requires one workday of hands-on computational execution and basic command line knowledge.

Pathomics-based machine learning models for predicting pathological complete response and prognosis in locally advanced rectal cancer patients post-neoadjuvant chemoradiotherapy: insights from two independent institutional studies

BACKGROUND

Accurate prediction of pathological complete response (pCR) and disease-free survival (DFS) in locally advanced rectal cancer (LARC) patients undergoing neoadjuvant chemoradiotherapy (NCRT) is essential for formulating effective treatment plans. This study aimed to construct and validate the machine learning (ML) models to predict pCR and DFS using pathomics.

METHOD

A retrospective analysis was conducted on 294 patients who received NCRT from two independent institutions. Pathomics from pre-NCRT H&E stains were extracted, and five ML models were developed and validated across two centers using ROC, Kaplan-Meier, time-dependent ROC, and nomogram analyses.

RESULT

Among the five ML models, the Xgboost (XGB) model demonstrated superior performance in predicting pCR, achieving an AUC of 1.000 (p < 0.001) on the internal data-set and an AUC of 0.950 (p = 0.001) on the external data-set.The XGB model effectively differentiated between high-risk and low-risk prognosis patients across all five centers: internal dataset (DFS, p = 0.002; OS, p = 0.004) and external dataset (DFS, p = 0.074; OS, p = 0.224).Furthermore, the COX regression demonstrated that the tumor length (HR = 1.230, 95%CI: 1.050-1.440, p = 0.010), post-NCRT CEA (HR = 1.716, 95%CI: 1.031- 2.858, p = 0.038), and XGB model score (HR = 0.128, 95%CI: 0.026-0.636, p = 0.012) were independent predictors of DFS after NCRT in the internal data-set.Using COX regression, the nomogram model and time-dependent AUC analysis demonstrated strong predictive discrimination for DFS in LARC patients across two independent institutions.

CONCLUSION

The ML model based on pathomics demonstrated effective prediction of pCR and prognosis in LARC patients. Further validation in larger cohorts is warranted to confirm the findings of this study.

Editorial for Special Issue "Cellular Senescence: Recent Cellular Advances and Discoveries"

Cellular senescence has emerged as a fascinating frontier in biological research and now presents profound implications across diverse fields, from aging research to cancer therapy [...].

Deep feature batch correction using ComBat for machine learning applications in computational pathology

Background

Developing artificial intelligence (AI) models for digital pathology requires large datasets from multiple sources. However, without careful implementation, AI models risk learning confounding site-specific features in datasets instead of clinically relevant information, leading to overestimated performance, poor generalizability to real-world data, and potential misdiagnosis.

Methods

Whole-slide images (WSIs) from The Cancer Genome Atlas (TCGA) colon (COAD), and stomach adenocarcinoma datasets were selected for inclusion in this study. Patch embeddings were obtained using three feature extraction models, followed by ComBat harmonization. Attention-based multiple instance learning models were trained to predict tissue-source site (TSS), as well as clinical and genetic attributes, using raw, Macenko normalized, and Combat-harmonized patch embeddings.

Results

TSS prediction achieved high accuracy (AUROC > 0.95) with all three feature extraction models. ComBat harmonization significantly reduced the AUROC for TSS prediction, with mean AUROCs dropping to approximately 0.5 for most models, indicating successful mitigation of batch effects (e.g., CCL-ResNet50 in TCGA-COAD: Pre-ComBat AUROC = 0.960, Post-ComBat AUROC = 0.506, p < 0.001). Clinical attributes associated with TSS, such as race and treatment response, showed decreased predictability post-harmonization. Notably, the prediction of genetic features like MSI status remained robust after harmonization (e.g., MSI in TCGA-COAD: Pre-ComBat AUROC = 0.667, Post-ComBat AUROC = 0.669, p=0.952), indicating the preservation of true histological signals.

Conclusion

ComBat harmonization of deep learning-derived histology features effectively reduces the risk of AI models learning confounding features in WSIs, ensuring more reliable performance estimates. This approach is promising for the integration of large-scale digital pathology datasets.

Federated attention consistent learning models for prostate cancer diagnosis and Gleason grading

Artificial intelligence (AI) holds significant promise in transforming medical imaging, enhancing diagnostics, and refining treatment strategies. However, the reliance on extensive multicenter datasets for training AI models poses challenges due to privacy concerns. Federated learning provides a solution by facilitating collaborative model training across multiple centers without sharing raw data. This study introduces a federated attention-consistent learning (FACL) framework to address challenges associated with large-scale pathological images and data heterogeneity. FACL enhances model generalization by maximizing attention consistency between local clients and the server model. To ensure privacy and validate robustness, we incorporated differential privacy by introducing noise during parameter transfer. We assessed the effectiveness of FACL in cancer diagnosis and Gleason grading tasks using 19,461 whole-slide images of prostate cancer from multiple centers. In the diagnosis task, FACL achieved an area under the curve (AUC) of 0.9718, outperforming seven centers with an average AUC of 0.9499 when categories are relatively balanced. For the Gleason grading task, FACL attained a Kappa score of 0.8463, surpassing the average Kappa score of 0.7379 from six centers. In conclusion, FACL offers a robust, accurate, and cost-effective AI training model for prostate cancer pathology while maintaining effective data safeguards.

The utility of artificial intelligence platforms for patient-generated questions in Mohs micrographic surgery: a multi-national, blinded expert panel evaluation

BACKGROUND

Artificial intelligence (AI) and large language models (LLMs) transform how patients inform themselves. LLMs offer potential as educational tools, but their quality depends upon the information generated. Current literature examining AI as an informational tool in dermatology has been limited in evaluating AI's multifaceted roles and diversity of opinions. Here, we evaluate LLMs as a patient-educational tool for Mohs micrographic surgery (MMS) in and out of the clinic utilizing an international expert panel.

METHODS

The most common patient MMS questions were extracted from Google and transposed into two LLMs and Google's search engine. 15 MMS surgeons evaluated the generated responses, examining their appropriateness as a patient-facing informational platform, sufficiency of response in a clinical environment, and accuracy of content generated. Validated scales were employed to assess the comprehensibility of each response.

RESULTS

The majority of reviewers deemed all LLM responses appropriate. 75% of responses were rated as mostly accurate or higher. ChatGPT had the highest mean accuracy. The majority of the panel deemed 33% of responses sufficient for clinical practice. The mean comprehensibility scores for all platforms indicated a required 10th-grade reading level.

CONCLUSIONS

LLM-generated responses were rated as appropriate patient informational sources and mostly accurate in their content. However, these platforms may not provide sufficient information to function in a clinical environment, and complex comprehensibility may represent a barrier to utilization. As the popularity of these platforms increases, it is important for dermatologists to be aware of these limitations.

Recent Advances in Artificial Intelligence to Improve Immunotherapy and the Use of Digital Twins to Identify Prognosis of Patients with Solid Tumors

To date, the public health system has been impacted by the increasing costs of many diagnostic and therapeutic pathways due to limited resources. At the same time, we are constantly seeking to improve these paths through approaches aimed at personalized medicine. To achieve the required levels of diagnostic and therapeutic precision, it is necessary to integrate data from different sources and simulation platforms. Today, artificial intelligence (AI), machine learning (ML), and predictive computer models are more efficient at guiding decisions regarding better therapies and medical procedures. The evolution of these multiparametric and multimodal systems has led to the creation of digital twins (DTs). The goal of our review is to summarize AI applications in discovering new immunotherapies and developing predictive models for more precise immunotherapeutic decision-making. The findings from this literature review highlight that DTs, particularly predictive mathematical models, will be pivotal in advancing healthcare outcomes. Over time, DTs will indeed bring the benefits of diagnostic precision and personalized treatment to a broader spectrum of patients.

Applications of artificial intelligence in digital pathology for gastric cancer

Gastric cancer is one of the most common cancers and is one of the leading causes of cancer-related deaths in worldwide. Early diagnosis and treatment are essential for a positive outcome. The integration of artificial intelligence in the pathology field is increasingly widespread, including histopathological images analysis. In recent years, the application of digital pathology technology emerged as a potential solution to enhance the understanding and management of gastric cancer. Through sophisticated image analysis algorithms, artificial intelligence technologies facilitate the accuracy and sensitivity of gastric cancer diagnosis and treatment and personalized therapeutic strategies. This review aims to evaluate the current landscape and future potential of artificial intelligence in transforming gastric cancer pathology, so as to provide ideas for future research.

Deep learning application in prediction of cancer molecular alterations based on pathological images: a bibliographic analysis via CiteSpace

BACKGROUND

The advancements in artificial intelligence (AI) technology for image recognition were propelling molecular pathology research into a new era.

OBJECTIVE

To summarize the hot spots and research trends in the field of molecular pathology image recognition.

METHODS

Relevant articles from January 1st, 2010, to August 25th, 2023, were retrieved from the Web of Science Core Collection. Subsequently, CiteSpace was employed for bibliometric and visual analysis, generating diverse network diagrams illustrating keywords, highly cited references, hot topics, and research trends.

RESULTS

A total of 110 relevant articles were extracted from a pool of 10,205 articles. The overall publication count exhibited a rising trend each year. The leading contributors in terms of institutions, countries, and authors were Maastricht University (11 articles), the United States (38 articles), and Kather Jacob Nicholas (9 articles), respectively. Half of the top ten research institutions, based on publication volume, were affiliated with Germany. The most frequently cited article was authored by Nicolas Coudray et al. accumulating 703 citations. The keyword "Deep learning" had the highest frequency in 2019. Notably, the highlighted keywords from 2022 to 2023 included "microsatellite instability", and there were 21 articles focusing on utilizing algorithms to recognize microsatellite instability (MSI) in colorectal cancer (CRC) pathological images.

CONCLUSION

The use of DL is expected to provide a new strategy to effectively solve the current problem of time-consuming and expensive molecular pathology detection. Therefore, further research is needed to address issues, such as data quality and standardization, model interpretability, and resource and infrastructure requirements.

Application of Artificial Intelligence in the diagnosis and treatment of colorectal cancer: a bibliometric analysis, 2004-2023

BACKGROUND

An increasing number of studies have turned their lens to the application of Artificial Intelligence (AI) in the diagnosis and treatment of colorectal cancer (CRC).

OBJECTIVE

To clarify and visualize the basic situation, research hotspots, and development trends of AI in the diagnosis and treatment of CRC, and provide clues for research in the future.

METHODS

On January 31, 2024, the Web of Science Core Collection (WoSCC) database was searched to screen and export the relevant research published during 2004-2023, and Cite Space, VoSviewer, Bibliometrix were used to visualize the number of publications, countries (regions), institutions, journals, authors, citations, keywords, etc.

RESULTS

A total of 2715 pieces of literature were included. The number of publications grew slowly until the end of 2016, but rapidly after 2017, till to the peak of 798 in 2023. A total of 92 countries, 3997 organizations, and 15,667 authors were involved in this research. Chinese scholars released the highest number of publications, and the U.S. contributed the highest number of total citations. As to authors, MORI, YUICHI had the highest number of publications, and WANG, PU had the highest number of total citations. According to the analysis of citations and keywords, the current research hotspots are mainly related to "Colonoscopy", "Polyp Segmentation", "Digital Pathology", "Radiomics", "prognosis".

CONCLUSION

Research on the application of AI in the diagnosis and treatment of CRC has made significant progress and is flourishing across the world. Current research hotspots include AI-assisted early screening and diagnosis, pathology, and staging, and prognosis assessment, and future research is predicted to put weight on multimodal data fusion, personalized treatment, and drug development.

Signaling pathways involved in colorectal cancer: pathogenesis and targeted therapy

Colorectal cancer (CRC) remains one of the leading causes of cancer-related mortality worldwide. Its complexity is influenced by various signal transduction networks that govern cellular proliferation, survival, differentiation, and apoptosis. The pathogenesis of CRC is a testament to the dysregulation of these signaling cascades, which culminates in the malignant transformation of colonic epithelium. This review aims to dissect the foundational signaling mechanisms implicated in CRC, to elucidate the generalized principles underpinning neoplastic evolution and progression. We discuss the molecular hallmarks of CRC, including the genomic, epigenomic and microbial features of CRC to highlight the role of signal transduction in the orchestration of the tumorigenic process. Concurrently, we review the advent of targeted and immune therapies in CRC, assessing their impact on the current clinical landscape. The development of these therapies has been informed by a deepening understanding of oncogenic signaling, leading to the identification of key nodes within these networks that can be exploited pharmacologically. Furthermore, we explore the potential of integrating AI to enhance the precision of therapeutic targeting and patient stratification, emphasizing their role in personalized medicine. In summary, our review captures the dynamic interplay between aberrant signaling in CRC pathogenesis and the concerted efforts to counteract these changes through targeted therapeutic strategies, ultimately aiming to pave the way for improved prognosis and personalized treatment modalities in colorectal cancer.

The digital revolution in veterinary pathology

For the past two centuries, the use of traditional light microscopy to examine tissues to make diagnoses has remained relatively unchanged. While the fundamental concept of tissue slide analysis has stayed the same, our interaction with the microscope is undergoing significant changes. Digital pathology (DP) has gained momentum in veterinary science and is on the verge of becoming a vital tool in diagnostics, research and education. Many diagnostic laboratories have incorporated DP as a critical part of their workflows. Innovations in DP and whole slide image technology have made telediagnosis (the process of transmitting digital clinical data using telecommunication networks for distant diagnosis) more accessible, leading to improved patient care through streamlining of workflows and greater accessibility of second opinions. The integration of machine learning and artificial intelligence and human-in-the-loop protocols for DP workflows will further the development of computer-aided diagnosis and prognostic tools. Despite its present weaknesses, DP will progressively aid veterinary clinicians and pathologists in delivering more accurate and reliable diagnoses. Consistent incorporation of DP frontline advancements into routine veterinary diagnostic pipelines will assist in improving current tools and help prepare pathologists for the progression of digitalization in the field.

Leveraging State-of-the-Art AI Algorithms in Personalized Oncology: From Transcriptomics to Treatment

BACKGROUND

Continuous breakthroughs in computational algorithms have positioned AI-based models as some of the most sophisticated technologies in the healthcare system. AI shows dynamic contributions in advancing various medical fields involving data interpretation and monitoring, imaging screening and diagnosis, and treatment response and survival prediction. Despite advances in clinical oncology, more effort must be employed to tailor therapeutic plans based on each patient's unique transcriptomic profile within the precision/personalized oncology frame. Furthermore, the standard analysis method is not compatible with the comprehensive deciphering of significant data streams, thus precluding the prediction of accurate treatment options.

METHODOLOGY

We proposed a novel approach that includes obtaining different tumour tissues and preparing RNA samples for comprehensive transcriptomic interpretation using specifically trained, programmed, and optimized AI-based models for extracting large data volumes, refining, and analyzing them. Next, the transcriptomic results will be scanned against an expansive drug library to predict the response of each target to the tested drugs. The obtained target-drug combination/s will be then validated using in vitro and in vivo experimental models. Finally, the best treatment combination option/s will be introduced to the patient. We also provided a comprehensive review discussing AI models' recent innovations and implementations to aid in molecular diagnosis and treatment planning.

RESULTS

The expected transcriptomic analysis generated by the AI-based algorithms will provide an inclusive genomic profile for each patient, containing statistical and bioinformatics analyses, identification of the dysregulated pathways, detection of the targeted genes, and recognition of molecular biomarkers. Subjecting these results to the prediction and pairing AI-based processes will result in statistical graphs presenting each target's likely response rate to various treatment options. Different in vitro and in vivo investigations will further validate the selection of the target drug/s pairs.

CONCLUSIONS

Leveraging AI models will provide more rigorous manipulation of large-scale datasets on specific cancer care paths. Such a strategy would shape treatment according to each patient's demand, thus fortifying the avenue of personalized/precision medicine. Undoubtedly, this will assist in improving the oncology domain and alleviate the burden of clinicians in the coming decade.

Single-cell AI-based detection and prognostic and predictive value of DNA mismatch repair deficiency in colorectal cancer

Testing for DNA mismatch repair deficiency (MMRd) is recommended for all colorectal cancers (CRCs). Automating this would enable precision medicine, particularly if providing information on etiology not captured by deep learning (DL) methods. We present AIMMeR, an AI-based method for determination of mismatch repair (MMR) protein expression at a single-cell level in routine pathology samples. AIMMeR shows an area under the receiver-operator curve (AUROC) of 0.98, and specificity of ≥75% at 98% sensitivity against pathologist ground truth in stage II/III in two trial cohorts, with positive predictive value of ≥98% for the commonest pattern of somatic MMRd. Lower agreement with microsatellite instability (MSI) testing (AUROC 0.86) reflects discordance between MMR and MSI PCR rather than AIMMeR misclassification. Analysis of the SCOT trial confirms MMRd prognostic value in oxaliplatin-treated patients; while MMRd does not predict differential benefit of chemotherapy duration, it correlates with difference in relapse by regimen (PInteraction = 0.04). AIMMeR may help reduce pathologist workload and streamline diagnostics in CRC.

Deep learning-based histological predictions of chromosomal instability in colorectal cancer

Colorectal cancer (CRC) is a lethal malignancy and a leading cause of cancer-related mortality worldwide. Chromosomal instability (CIN) is a key driver of genomic instability in CRC and is characterized by aneuploidy and somatic copy-number alterations. This study aimed to predict CIN in CRC using histological data from whole slide images (WSIs). CRC samples from TCGA were analyzed, with tumor regions segmented into tiles and nuclei for feature extraction using convolutional neural network (CNN) and morphologic analysis. Binary classification models were developed to distinguish high and low aneuploidy scores (AS) based on slide-level features. The analysis included 313 patients with 315 WSIs, resulting in over 350,000 tumor tiles and nearly 2.7 million tumor cell nuclei. The ResNet18-SSL model, pre-trained on histopathological images, demonstrated superior accuracy in tile-based AS prediction, while DenseNet121 excelled in nucleus-based prediction. Combining CNN-based and morphological features enhanced the classification accuracy of nucleus-based predictions. Additionally, significant correlations were observed between morphological features and copy-number signatures. Unsupervised clustering of nuclear features revealed that distinct groups are significantly correlated with CIN and TP53 mutations. This study underscores the potential of histological features from WSIs to predict CIN in CRC samples. Nuclear feature analysis, combined with deep-learning techniques, offers a robust method for CIN prediction, highlighting the importance of further research into the relationships between histological and molecular phenotypes.

MSIsensor-RNA: Microsatellite Instability Detection for Bulk and Single-cell Gene Expression Data

Microsatellite instability (MSI) is an indispensable biomarker in cancer immunotherapy. Currently, MSI scoring methods by high-throughput omics methods have gained popularity and demonstrated better performance than the gold standard method for MSI detection. However, the MSI detection method on expression data, especially single-cell expression data, is still lacking, limiting the scope of clinical application and prohibiting the investigation of MSI at a single-cell level. Herein, we developed MSIsensor-RNA, an accurate, robust, adaptable, and standalone software to detect MSI status based on expression values of MSI-associated genes. We demonstrated the favorable performance and promise of MSIsensor-RNA in both bulk and single-cell gene expression data in multiplatform technologies including RNA sequencing (RNA-seq), microarray, and single-cell RNA-seq. MSIsensor-RNA is a versatile, efficient, and robust method for MSI status detection from both bulk and single-cell gene expression data in clinical studies and applications. MSIsensor-RNA is available at https://github.com/xjtu-omics/msisensor-rna.

Exploring the interplay between colorectal cancer subtypes genomic variants and cellular morphology: A deep-learning approach

Molecular subtypes of colorectal cancer (CRC) significantly influence treatment decisions. While convolutional neural networks (CNNs) have recently been introduced for automated CRC subtype identification using H&E stained histopathological images, the correlation between CRC subtype genomic variants and their corresponding cellular morphology expressed by their imaging phenotypes is yet to be fully explored. The goal of this study was to determine such correlations by incorporating genomic variants in CNN models for CRC subtype classification from H&E images. We utilized the publicly available TCGA-CRC-DX dataset, which comprises whole slide images from 360 CRC-diagnosed patients (260 for training and 100 for testing). This dataset also provides information on CRC subtype classifications and genomic variations. We trained CNN models for CRC subtype classification that account for potential correlation between genomic variations within CRC subtypes and their corresponding cellular morphology patterns. We assessed the interplay between CRC subtypes' genomic variations and cellular morphology patterns by evaluating the CRC subtype classification accuracy of the different models in a stratified 5-fold cross-validation experimental setup using the area under the ROC curve (AUROC) and average precision (AP) as the performance metrics. The CNN models that account for potential correlation between genomic variations within CRC subtypes and their cellular morphology pattern achieved superior accuracy compared to the baseline CNN classification model that does not account for genomic variations when using either single-nucleotide-polymorphism (SNP) molecular features (AUROC: 0.824±0.02 vs. 0.761±0.04, p<0.05, AP: 0.652±0.06 vs. 0.58±0.08) or CpG-Island methylation phenotype (CIMP) molecular features (AUROC: 0.834±0.01 vs. 0.787±0.03, p<0.05, AP: 0.687±0.02 vs. 0.64±0.05). Combining the CNN models account for variations in CIMP and SNP further improved classification accuracy (AUROC: 0.847±0.01 vs. 0.787±0.03, p = 0.01, AP: 0.68±0.02 vs. 0.64±0.05). The improved accuracy of CNN models for CRC subtype classification that account for potential correlation between genomic variations within CRC subtypes and their corresponding cellular morphology as expressed by H&E imaging phenotypes may elucidate the biological cues impacting cancer histopathological imaging phenotypes. Moreover, considering CRC subtypes genomic variations has the potential to improve the accuracy of deep-learning models in discerning cancer subtype from histopathological imaging data.

Expanding role and scope of artificial intelligence in the field of gastrointestinal pathology

Digital pathology (DP) and its subsidiaries including artificial intelligence (AI) are rapidly making inroads into the area of diagnostic anatomic pathology (AP) including gastrointestinal (GI) pathology. It is poised to revolutionize the field of diagnostic AP. Historically, AP has been slow to adopt digital technology, but this is changing rapidly, with many centers worldwide transitioning to DP. Coupled with advanced techniques of AI such as deep learning and machine learning, DP is likely to transform histopathology from a subjective field to an objective, efficient, and transparent discipline. AI is increasingly integrated into GI pathology, offering numerous advancements and improvements in overall diagnostic accuracy, efficiency, and patient care. Specifically, AI in GI pathology enhances diagnostic accuracy, streamlines workflows, provides predictive insights, integrates multimodal data, supports research, and aids in education and training, ultimately improving patient care and outcomes. This review summarized the latest developments in the role and scope of AI in AP with a focus on GI pathology. The main aim was to provide updates and create awareness among the pathology community.

Predicting Mismatch Repair Deficiency Status in Endometrial Cancer through Multi-Resolution Ensemble Learning in Digital Pathology

For molecular classification of endometrial carcinoma, testing for mismatch repair (MMR) status is becoming a routine process. Mismatch repair deficiency (MMR-D) is caused by loss of expression in one or more of the 4 major MMR proteins: MLH1, MSH2, MSH6, PHS2. Over 30% of patients with endometrial cancer have MMR-D. Determining the MMR status holds significance as individuals with MMR-D are potential candidates for immunotherapy. Pathological whole slide image (WSI) of endometrial cancer with immunohistochemistry results of MMR proteins were gathered. Color normalization was applied to the tiles using a CycleGAN-based network. The WSI was divided into tiles at three different magnifications (2.5 × , 5 × , and 10 ×). Three distinct networks of the same architecture were employed to include features from all three magnification levels and were stacked for ensemble learning. Three architectures, InceptionResNetV2, EfficientNetB2, and EfficientNetB3 were employed and subjected to comparison. The per-tile results were gathered to classify MMR status in the WSI, and prediction accuracy was evaluated using the following performance metrics: AUC, accuracy, sensitivity, and specificity. The EfficientNetB2 was able to make predictions with an AUC of 0.821, highest among the three architectures, and an overall AUC range of 0.767 - 0.821 was reported across the three architectures. In summary, our study successfully predicted MMR classification from pathological WSIs in endometrial cancer through a multi-resolution ensemble learning approach, which holds the potential to facilitate swift decisions on tailored treatment, such as immunotherapy, in clinical settings.

Artificial Intelligence Models for the Detection of Microsatellite Instability from Whole-Slide Imaging of Colorectal Cancer

With the advent of whole-slide imaging (WSI), a technology that can digitally scan whole slides in high resolution, pathology is undergoing a digital revolution. Detecting microsatellite instability (MSI) in colorectal cancer is crucial for proper treatment, as it identifies patients responsible for immunotherapy. Even though universal testing for MSI is recommended, particularly in patients affected by colorectal cancer (CRC), many patients remain untested, and they reside mainly in low-income countries. A critical need exists for accessible, low-cost tools to perform MSI pre-screening. Here, the potential predictive role of the most relevant artificial intelligence-driven models in predicting microsatellite instability directly from histology alone is discussed, focusing on CRC. The role of deep learning (DL) models in identifying the MSI status is here analyzed in the most relevant studies reporting the development of algorithms trained to this end. The most important performance and the most relevant deficiencies are discussed for every AI method. The models proposed for algorithm sharing among multiple research and clinical centers, including federal learning (FL) and swarm learning (SL), are reported. According to all the studies reported here, AI models are valuable tools for predicting MSI status on WSI alone in CRC. The use of digitized H&E-stained sections and a trained algorithm allow the extraction of relevant molecular information, such as MSI status, in a short time and at a low cost. The possible advantages related to introducing DL methods in routine surgical pathology are underlined here, and the acceleration of the digital transformation of pathology departments and services is recommended.

Contrastive Pre-Training and Multiple Instance Learning for Predicting Tumor Microsatellite Instability

Accurate classification between tumor MicroSatellite Stability (MSS) and Instability (MSI) is crucial in gastrointestinal (GI) cancer prognosis and treatment. In this paper, we present a novel two-stage weakly supervised methodology, leveraging the synergy of Multiple Instance Learning (MIL) and a unique Contrastive Clustering Network (CCNet), aimed at enhancing MSI prediction in Whole Slide Image (WSI) analysis of GI cancers. In our framework, we utilize a contrastive learning-based feature extractor, coupled with MIL's efficient labeling. Our approach shows notable improvement in MSI classification, outperforming existing methods. Experiments using colorectal cancer and stomach adenocarcinoma datasets demonstrate the model's efficacy and generalizability, marking an advance in computational pathology and cancer diagnostics. Furthermore, we explored the efficacy of transfer learning using our model, examining the performance of pretrained feature extractors from ImageNet and STAD datasets. Our framework outperforms existing methods when pretrained on STAD and transferred to CRC data.Clinical relevance- The potential to enhance the accuracy of gastrointestinal cancer diagnosis and prognosis through advanced machine learning techniques. By leveraging transfer learning and weakly supervised frameworks clinicians can benefit from improved MSI prediction in histopathological images, aiding in personalized treatment strategies and patient outcomes.

Artificial Intelligence in Colorectal Cancer: From Patient Screening over Tailoring Treatment Decisions to Identification of Novel Biomarkers

BACKGROUND

Artificial intelligence (AI) is increasingly entering and transforming not only medical research but also clinical practice. In the last 10 years, new AI methods have enabled computers to perform visual tasks, reaching high performance and thereby potentially supporting and even outperforming human experts. This is in particular relevant for colorectal cancer (CRC), which is the 3rd most common cancer type in general, as along the CRC patient journey many complex visual tasks need to be performed: from endoscopy over imaging to histopathology; the screening, diagnosis, and treatment of CRC involve visual image analysis tasks.

SUMMARY

In all these clinical areas, AI models have shown promising results by supporting physicians, improving accuracy, and providing new biological insights and biomarkers. By predicting prognostic and predictive biomarkers from routine images/slides, AI models could lead to an improved patient stratification for precision oncology approaches in the near future. Moreover, it is conceivable that AI models, in particular together with innovative techniques such as single-cell or spatial profiling, could help identify novel clinically as well as biologically meaningful biomarkers that could pave the way to new therapeutic approaches.

KEY MESSAGES

Here, we give a comprehensive overview of AI in colorectal cancer, describing and discussing these developments as well as the next steps which need to be taken to incorporate AI methods more broadly into the clinical care of CRC.

A guide to artificial intelligence for cancer researchers

Artificial intelligence (AI) has been commoditized. It has evolved from a specialty resource to a readily accessible tool for cancer researchers. AI-based tools can boost research productivity in daily workflows, but can also extract hidden information from existing data, thereby enabling new scientific discoveries. Building a basic literacy in these tools is useful for every cancer researcher. Researchers with a traditional biological science focus can use AI-based tools through off-the-shelf software, whereas those who are more computationally inclined can develop their own AI-based software pipelines. In this article, we provide a practical guide for non-computational cancer researchers to understand how AI-based tools can benefit them. We convey general principles of AI for applications in image analysis, natural language processing and drug discovery. In addition, we give examples of how non-computational researchers can get started on the journey to productively use AI in their own work.

Unraveling the influence of metabolic signatures on immune dynamics for predicting immunotherapy response and survival in cancer

Metabolic reprogramming in cancer significantly impacts immune responses within the tumor microenvironment, but its influence on cancer immunotherapy effectiveness remains uncertain. This study aims to elucidate the prognostic significance of metabolic genes in cancer immunotherapy through a comprehensive analytical approach. Utilizing data from the IMvigor210 trial (n = 348) and validated by retrospective datasets, we performed patient clustering using non‐negative matrix factorization based on metabolism‐related genes. A metabiotic score was developed using a “DeepSurv” neural network to assess correlations with overall survival (OS), progression‐free survival, and immunotherapy response. Validation of the metabolic score and key genes was achieved via comparative gene expression analysis using qPCR. Our analysis identified four distinct metabolic classes with significant variations in OS. Notably, the metabolism‐inactive and hypoxia‐low class demonstrated the most pronounced benefit in terms of OS. The metabolic score predicted immunotherapeutic benefits with high accuracy (AUC: 0.93 at 12 months). SETD3 emerged as a crucial gene, showing strong correlations with improved OS outcomes. This study underscores the importance of metabolic profiling in predicting cancer immunotherapy success. Specifically, patients classified as metabolism‐inactive and hypoxia‐low appear to derive substantial benefits. SETD3 is established as a promising prognostic marker, linking metabolic activity with patient outcomes, advocating for the integration of metabolic profiling into immunotherapy strategies to enhance treatment precision and efficacy.

Automated curation of large-scale cancer histopathology image datasets using deep learning

BACKGROUND

Artificial intelligence (AI) has numerous applications in pathology, supporting diagnosis and prognostication in cancer. However, most AI models are trained on highly selected data, typically one tissue slide per patient. In reality, especially for large surgical resection specimens, dozens of slides can be available for each patient. Manually sorting and labelling whole-slide images (WSIs) is a very time-consuming process, hindering the direct application of AI on the collected tissue samples from large cohorts. In this study we addressed this issue by developing a deep-learning (DL)-based method for automatic curation of large pathology datasets with several slides per patient.

METHODS

We collected multiple large multicentric datasets of colorectal cancer histopathological slides from the United Kingdom (FOXTROT, N = 21,384 slides; CR07, N = 7985 slides) and Germany (DACHS, N = 3606 slides). These datasets contained multiple types of tissue slides, including bowel resection specimens, endoscopic biopsies, lymph node resections, immunohistochemistry-stained slides, and tissue microarrays. We developed, trained, and tested a deep convolutional neural network model to predict the type of slide from the slide overview (thumbnail) image. The primary statistical endpoint was the macro-averaged area under the receiver operating curve (AUROCs) for detection of the type of slide.

RESULTS

In the primary dataset (FOXTROT), with an AUROC of 0.995 [95% confidence interval [CI]: 0.994-0.996] the algorithm achieved a high classification performance and was able to accurately predict the type of slide from the thumbnail image alone. In the two external test cohorts (CR07, DACHS) AUROCs of 0.982 [95% CI: 0.979-0.985] and 0.875 [95% CI: 0.864-0.887] were observed, which indicates the generalizability of the trained model on unseen datasets. With a confidence threshold of 0.95, the model reached an accuracy of 94.6% (7331 classified cases) in CR07 and 85.1% (2752 classified cases) for the DACHS cohort.

CONCLUSION

Our findings show that using the low-resolution thumbnail image is sufficient to accurately classify the type of slide in digital pathology. This can support researchers to make the vast resource of existing pathology archives accessible to modern AI models with only minimal manual annotations.

Deep learning for colorectal cancer detection in contrast-enhanced CT without bowel preparation: a retrospective, multicentre study

BACKGROUND

Contrast-enhanced CT scans provide a means to detect unsuspected colorectal cancer. However, colorectal cancers in contrast-enhanced CT without bowel preparation may elude detection by radiologists. We aimed to develop a deep learning (DL) model for accurate detection of colorectal cancer, and evaluate whether it could improve the detection performance of radiologists.

METHODS

We developed a DL model using a manually annotated dataset (1196 cancer vs 1034 normal). The DL model was tested using an internal test set (98 vs 115), two external test sets (202 vs 265 in 1, and 252 vs 481 in 2), and a real-world test set (53 vs 1524). We compared the detection performance of the DL model with radiologists, and evaluated its capacity to enhance radiologists' detection performance.

FINDINGS

In the four test sets, the DL model had the area under the receiver operating characteristic curves (AUCs) ranging between 0.957 and 0.994. In both the internal test set and external test set 1, the DL model yielded higher accuracy than that of radiologists (97.2% vs 86.0%, p < 0.0001; 94.9% vs 85.3%, p < 0.0001), and significantly improved the accuracy of radiologists (93.4% vs 86.0%, p < 0.0001; 93.6% vs 85.3%, p < 0.0001). In the real-world test set, the DL model delivered sensitivity comparable to that of radiologists who had been informed about clinical indications for most cancer cases (94.3% vs 96.2%, p > 0.99), and it detected 2 cases that had been missed by radiologists.

INTERPRETATION

The developed DL model can accurately detect colorectal cancer and improve radiologists' detection performance, showing its potential as an effective computer-aided detection tool.

FUNDING

This study was supported by National Science Fund for Distinguished Young Scholars of China (No. 81925023); Regional Innovation and Development Joint Fund of National Natural Science Foundation of China (No. U22A20345); National Natural Science Foundation of China (No. 82072090 and No. 82371954); Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application (No. 2022B1212010011); High-level Hospital Construction Project (No. DFJHBF202105).

Challenges and perspectives in use of artificial intelligence to support treatment recommendations in clinical oncology

Artificial intelligence (AI) promises to be the next revolutionary step in modern society. Yet, its role in all fields of industry and science need to be determined. One very promising field is represented by AI-based decision-making tools in clinical oncology leading to more comprehensive, personalized therapy approaches. In this review, the authors provide an overview on all relevant technical applications of AI in oncology, which are required to understand the future challenges and realistic perspectives for decision-making tools. In recent years, various applications of AI in medicine have been developed focusing on the analysis of radiological and pathological images. AI applications encompass large amounts of complex data supporting clinical decision-making and reducing errors by objectively quantifying all aspects of the data collected. In clinical oncology, almost all patients receive a treatment recommendation in a multidisciplinary cancer conference at the beginning and during their treatment periods. These highly complex decisions are based on a large amount of information (of the patients and of the various treatment options), which need to be analyzed and correctly classified in a short time. In this review, the authors describe the technical and medical requirements of AI to address these scientific challenges in a multidisciplinary manner. Major challenges in the use of AI in oncology and decision-making tools are data security, data representation, and explainability of AI-based outcome predictions, in particular for decision-making processes in multidisciplinary cancer conferences. Finally, limitations and potential solutions are described and compared for current and future research attempts.

Deep learning for dual detection of microsatellite instability and POLE mutations in colorectal cancer histopathology

In the spectrum of colorectal tumors, microsatellite-stable (MSS) tumors with DNA polymerase ε (POLE) mutations exhibit a hypermutated profile, holding the potential to respond to immunotherapy similarly to their microsatellite-instable (MSI) counterparts. Yet, due to their rarity and the associated testing costs, systematic screening for these mutations is not commonly pursued. Notably, the histopathological phenotype resulting from POLE mutations is theorized to resemble that of MSI. This resemblance not only could facilitate their detection by a transformer-based Deep Learning (DL) system trained on MSI pathology slides, but also indicates the possibility for MSS patients with POLE mutations to access enhanced treatment options, which might otherwise be overlooked. To harness this potential, we trained a Deep Learning classifier on a large dataset with the ground truth for microsatellite status and subsequently validated its capabilities for MSI and POLE detection across three external cohorts. Our model accurately identified MSI status in both the internal and external resection cohorts using pathology images alone. Notably, with a classification threshold of 0.5, over 75% of POLE driver mutant patients in the external resection cohorts were flagged as "positive" by a DL system trained on MSI status. In a clinical setting, deploying this DL model as a preliminary screening tool could facilitate the efficient identification of clinically relevant MSI and POLE mutations in colorectal tumors, in one go.

[Artificial intelligence for pathology-how, where, and why?]

Artificial intelligence promises many innovations and simplifications in pathology, but also raises just as many questions and uncertainties. In this article, we provide a brief overview of the current status, the goals already achieved by existing algorithms, and the remaining challenges.

Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions

Artificial intelligence (AI) in oncology is advancing beyond algorithm development to integration into clinical practice. This review describes the current state of the field, with a specific focus on clinical integration. AI applications are structured according to cancer type and clinical domain, focusing on the four most common cancers and tasks of detection, diagnosis, and treatment. These applications encompass various data modalities, including imaging, genomics, and medical records. We conclude with a summary of existing challenges, evolving solutions, and potential future directions for the field.

SIGNIFICANCE

AI is increasingly being applied to all aspects of oncology, where several applications are maturing beyond research and development to direct clinical integration. This review summarizes the current state of the field through the lens of clinical translation along the clinical care continuum. Emerging areas are also highlighted, along with common challenges, evolving solutions, and potential future directions for the field.

Pros and cons of artificial intelligence implementation in diagnostic pathology

The rapid introduction of digital pathology has greatly facilitated development of artificial intelligence (AI) models in pathology that have shown great promise in assisting morphological diagnostics and quantitation of therapeutic targets. We are now at a tipping point where companies have started to bring algorithms to the market, and questions arise whether the pathology community is ready to implement AI in routine workflow. However, concerns also arise about the use of AI in pathology. This article reviews the pros and cons of introducing AI in diagnostic pathology.

Deep learning framework for comprehensive molecular and prognostic stratifications of triple-negative breast cancer

Triple-negative breast cancer (TNBC) is the most challenging breast cancer subtype. Molecular stratification and target therapy bring clinical benefit for TNBC patients, but it is difficult to implement comprehensive molecular testing in clinical practice. Here, using our multi-omics TNBC cohort (N = 425), a deep learning-based framework was devised and validated for comprehensive predictions of molecular features, subtypes and prognosis from pathological whole slide images. The framework first incorporated a neural network to decompose the tissue on WSIs, followed by a second one which was trained based on certain tissue types for predicting different targets. Multi-omics molecular features were analyzed including somatic mutations, copy number alterations, germline mutations, biological pathway activities, metabolomics features and immunotherapy biomarkers. It was shown that the molecular features with therapeutic implications can be predicted including the somatic PIK3CA mutation, germline BRCA2 mutation and PD-L1 protein expression (area under the curve [AUC]: 0.78, 0.79 and 0.74 respectively). The molecular subtypes of TNBC can be identified (AUC: 0.84, 0.85, 0.93 and 0.73 for the basal-like immune-suppressed, immunomodulatory, luminal androgen receptor, and mesenchymal-like subtypes respectively) and their distinctive morphological patterns were revealed, which provided novel insights into the heterogeneity of TNBC. A neural network integrating image features and clinical covariates stratified patients into groups with different survival outcomes (log-rank P < 0.001). Our prediction framework and neural network models were externally validated on the TNBC cases from TCGA (N = 143) and appeared robust to the changes in patient population. For potential clinical translation, we built a novel online platform, where we modularized and deployed our framework along with the validated models. It can realize real-time one-stop prediction for new cases. In summary, using only pathological WSIs, our proposed framework can enable comprehensive stratifications of TNBC patients and provide valuable information for therapeutic decision-making. It had the potential to be clinically implemented and promote the personalized management of TNBC.