Deep learning can predict lymph node status directly from histology in colorectal cancer

PEDRA-EFB0: colorectal cancer prognostication using deep learning with patch embeddings and dual residual attention

In computer-aided diagnosis systems, precise feature extraction from CT scans of colorectal cancer using deep learning is essential for effective prognosis. However, existing convolutional neural networks struggle to capture long-range dependencies and contextual information, resulting in incomplete CT feature extraction. To address this, the PEDRA-EFB0 architecture integrates patch embeddings and a dual residual attention mechanism for enhanced feature extraction and survival prediction in colorectal cancer CT scans. A patch embedding method processes CT scans into patches, creating positional features for global representation and guiding spatial attention computation. Additionally, a dual residual attention mechanism during the upsampling stage selectively combines local and global features, enhancing CT data utilization. Furthermore, this paper proposes a feature selection algorithm that combines autoencoders and entropy technology, encoding and compressing high-dimensional data to reduce redundant information and using entropy to assess the importance of features, thereby achieving precise feature selection. Experimental results indicate the PEDRA-EFB0 model outperforms traditional methods on colorectal cancer CT metrics, notably in C-index, BS, MCC, and AUC, enhancing survival prediction accuracy. Our code is freely available at https://github.com/smile0208z/PEDRA .

Leveraging Deep Learning for Immune Cell Quantification and Prognostic Evaluation in Radiotherapy-Treated Oropharyngeal Squamous Cell Carcinomas

The tumor microenvironment plays a critical role in cancer progression and therapeutic responsiveness, with the tumor immune microenvironment (TIME) being a key modulator. In head and neck squamous cell carcinomas (HNSCCs), immune cell infiltration significantly influences the response to radiotherapy (RT). A better understanding of the TIME in HNSCCs could help identify patients most likely to benefit from combining RT with immunotherapy. Standardized, cost-effective methods for studying TIME in HNSCCs are currently lacking. This study aims to leverage deep learning (DL) to quantify immune cell densities using immunohistochemistry in untreated oropharyngeal squamous cell carcinoma (OPSCC) biopsies of patients scheduled for curative RT and assess their prognostic value. We analyzed 84 pretreatment formalin-fixed paraffin-embedded tumor biopsies from OPSCC patients. Immunohistochemistry was performed for CD3, CD8, CD20, CD163, and FOXP3, and whole slide images were digitized for analysis using a U-Net-based DL model. Two quantification approaches were applied: a cell-counting method and an area-based method. These methods were applied to stained regions. The DL model achieved high accuracy in detecting stained cells across all biomarkers. Strong correlations were found between our DL pipeline, the HALO Image Analysis Platform, and the open-source QuPath software for estimating immune cell densities. Our DL pipeline provided an accurate and reproducible approach for quantifying immune cells in OPSCC. The area-based method demonstrated superior prognostic value for recurrence-free survival, when compared with the cell-counting method. Elevated densities of CD3, CD8, CD20, and FOXP3 were associated with improved recurrence-free survival, whereas CD163 showed no significant prognostic association. These results highlight the potential of DL in digital pathology for assessing TIME and predicting patient outcomes.

A pathologist-AI collaboration framework for enhancing diagnostic accuracies and efficiencies

In pathology, the deployment of artificial intelligence (AI) in clinical settings is constrained by limitations in data collection and in model transparency and interpretability. Here we describe a digital pathology framework, nuclei.io, that incorporates active learning and human-in-the-loop real-time feedback for the rapid creation of diverse datasets and models. We validate the effectiveness of the framework via two crossover user studies that leveraged collaboration between the AI and the pathologist, including the identification of plasma cells in endometrial biopsies and the detection of colorectal cancer metastasis in lymph nodes. In both studies, nuclei.io yielded considerable diagnostic performance improvements. Collaboration between clinicians and AI will aid digital pathology by enhancing accuracies and efficiencies.

Artificial intelligence in gastrointestinal cancer research: Image learning advances and applications

With the rapid advancement of artificial intelligence (AI) technologies, including deep learning, large language models, and neural networks, these methodologies are increasingly being developed and integrated into cancer research. Gastrointestinal tumors are characterized by complexity and heterogeneity, posing significant challenges for early detection, diagnostic accuracy, and the development of personalized treatment strategies. The application of AI in digestive oncology has demonstrated its transformative potential. AI not only alleviates the diagnostic burden on clinicians, but it improves tumor screening sensitivity, specificity, and accuracy. Additionally, AI aids the detection of biomarkers such as microsatellite instability and mismatch repair, supports intraoperative assessments of tumor invasion depth, predicts treatment responses, and facilitates the design of personalized treatment plans to potentially significantly enhance patient outcomes. Moreover, the integration of AI with multiomics analyses and imaging technologies has led to substantial advancements in foundational research on the tumor microenvironment. This review highlights the progress of AI in gastrointestinal oncology over the past 5 years with focus on early tumor screening, diagnosis, molecular marker identification, treatment planning, and prognosis predictions. We also explored the potential of AI to enhance medical imaging analyses to aid tumor detection and characterization as well as its role in automating and refining histopathological assessments.

Prediction of Lymph Node Metastasis in Colorectal Cancer Using Intraoperative Fluorescence Multi-Modal Imaging

The diagnosis of lymph node metastasis (LNM) is essential for colorectal cancer (CRC) treatment. The primary method of identifying LNM is to perform frozen sections and pathologic analysis, but this method is labor-intensive and time-consuming. Therefore, combining intraoperative fluorescence imaging with deep learning (DL) methods can improve efficiency. The majority of recent studies only analyze uni-modal fluorescence imaging, which provides less semantic information. In this work, we mainly established a multi-modal fluorescence imaging feature fusion prediction (MFI-FFP) model combining white light, fluorescence, and pseudo-color imaging of lymph nodes for LNM prediction. Firstly, based on the properties of various modal imaging, distinct feature extraction networks are chosen for feature extraction, which could significantly enhance the complementarity of various modal information. Secondly, the multi-modal feature fusion (MFF) module, which combines global and local information, is designed to fuse the extracted features. Furthermore, a novel loss function is formulated to tackle the issue of imbalanced samples, challenges in differentiating samples, and enhancing sample variety. Lastly, the experiments show that the model has a higher area under the receiver operating characteristic (ROC) curve (AUC), accuracy (ACC), and F1 score than the uni-modal and bi-modal models and has a better performance compared to other efficient image classification networks. Our study demonstrates that the MFI-FFP model has the potential to help doctors predict LNM and shows its promise in medical image analysis.

Advancements in the application of artificial intelligence in the field of colorectal cancer

Colorectal cancer (CRC) is a prevalent malignant tumor in the digestive system. As reported in the 2020 global cancer statistics, CRC accounted for more than 1.9 million new cases and 935,000 deaths, making it the third most common cancer worldwide in terms of incidence and the second leading cause of cancer-related deaths globally. This poses a significant threat to global public health. Early screening methods, such as fecal occult blood tests, colonoscopies, and imaging techniques, are crucial for detecting early lesions and enabling timely intervention before cancer becomes invasive. Early detection greatly enhances treatment possibilities, such as surgery, radiation therapy, and chemotherapy, with surgery being the main approach for treating early-stage CRC. In this context, artificial intelligence (AI) has shown immense potential in revolutionizing CRC management, serving as one of the most effective screening tools. AI, utilizing machine learning (ML) and deep learning (DL) algorithms, improves early detection, diagnosis, and treatment by processing large volumes of medical data, uncovering hidden patterns, and forecasting disease development. DL, a more advanced form of ML, simulates the brain's processing power, enhancing the accuracy of tumor detection, differentiation, and prognosis predictions. These innovations offer the potential to revolutionize cancer care by boosting diagnostic accuracy, refining treatment approaches, and ultimately enhancing patient outcomes.

Predicting benefit from PARP inhibitors using deep learning on H&E-stained ovarian cancer slides

PURPOSE

Ovarian cancer patients with a Homologous Recombination Deficiency (HRD) often benefit from polyadenosine diphosphate-ribose polymerase (PARP) inhibitor maintenance therapy after response to platinum-based chemotherapy. HR status is currently analyzed via complex molecular tests. Predicting benefit from PARP inhibitors directly on histological whole slide images (WSIs) could be a fast and cheap alternative.

PATIENTS AND METHODS

We trained a Deep Learning (DL) model on H&E stained WSIs with "shrunken centroid" (SC) based HRD ground truth using the AGO-TR1 cohort (n = 208: 108 training, 100 test) and tested its ability to predict HRD as evaluated by the Myriad classifier and the benefit from olaparib in the PAOLA-1 cohort (n = 447) in a blinded manner.

RESULTS

In contrast to the HRD prediction AUROC of 72 % on hold-out, our model only yielded an AUROC of 57 % external. Kaplan-Meier analysis showed that progression free survival (PFS) in the PARP inhibitor treated PAOLA-1 patients was significantly improved in the HRD positive group as defined by our model, but not in the HRD negative group. PFS improvement in PARP inhibitor-treated patients was substantially longer in our HRD positive group, hinting at a biologically meaningful prediction of benefit from PARP inhibitors.

CONCLUSION

Together, our results indicate that it might be possible to generate a predictor of benefit from PARP inhibitors based on the DL-mediated analysis of WSIs. However, further studies with larger cohorts and further methodological improvements will be necessary to generate a predictor with clinically useful accuracy across independent patient cohorts.

End-to-end prediction of clinical outcomes in head and neck squamous cell carcinoma with foundation model-based multiple instance learning

Background

Foundation models (FMs) show promise in medical AI by learning flexible features from large datasets, potentially surpassing handcrafted radiomics. Outcome prediction of head and neck squamous cell carcinomas (HNSCC) with FMs using routine imaging remains unexplored.

Purpose

To evaluate end-to-end FM-based multiple instance learning (MIL) for 2-year overall survival (OS), locoregional control (LRC), and freedom from distant metastasis (FFDM) prediction and risk group stratification using pretreatment CT scans in HNSCC.

Materials and Methods

We analyzed data of 2485 patients from three retrospective HNSCC cohorts (RADCURE, HN1, HN-PET-CT), treated between 2004 and 2017 with available pre-treatment CTs and primary gross tumor volume (GTVp) segmentations. The RADCURE cohort was split into training (n=1464) and test (N=606), with HN1 (n=131) and HN-PET-CT (n=284) as additional test cohorts. FM-based MIL models (2D, multiview and 3D) for 2-year endpoint prediction and risk stratification wre evaluated based on area under the receiver operator curve (AUROC) and Kaplan-Meier (KM) with hazard ratios (HR), compared with radiomics and assessed for multimodal enhancement with clinical baselines.

Results

2D MIL models achieved 2-year test AUROCs of 0.75-0.84 (OS), 0.66-0.75 (LRC) and 0.71-0.78 (FFDM), outperforming multiview and 3D MIL (AUROCs: 0.50-0.77, p≥0.15) and comparable or superior to radiomics (AUROCs: 0.64-0.74, p≥0.012). Significant stratification was observed (HRs: 2.14-4.77, p≤0.039). Multimodal enhancement of 2-year OS/FFDM (AUROCs: 0.82-0.87, p≤0.018) was observed for patients without human papilloma virus positive (HPV+) tumors.

Conclusion

FM-based MIL demonstrates promise in HNSCC risk prediction, showing similar or superior performance to radiomics and enhancing clinical baselines in non-HPV+ patients.

Development of a prediction model based on Hemoglobin, Albumin, Lymphocyte count, and Platelet-score for lymph node metastasis in rectal cancer

This study aimed to evaluate the ability of the preoperative Hemoglobin, Albumin, Lymphocyte count, and Platelet (HALP) score to predict lymph node metastasis (LNM) in patients with rectal cancer (RC) and improve prediction accuracy by incorporating clinical parameters. Data from 263 patients with RC were analyzed. The receiver operating characteristic (ROC) curve was used to determine the optimal cutoff value (OCV) for the HALP score in predicting LNM. Based on this cutoff value, patients were divided into two groups. A baseline analysis was conducted to identify independent factors linked to LNM. A support vector machine (SVM) prediction model was developed, and its performance was evaluated using ROC, calibration curves, decision curve analysis, and Kolmogorov-Smirnov curve. The OCV for HALP score was 45.979. Patients were then classified into a low HALP group (n = 182) and a high HALP group (n = 81). The analysis found 21 clinical factors significantly associated with LNM. Among them, the key risk factors included high inflammatory status, poor nutritional condition, and a low HALP score. The SVM model incorporated these factors and showed robust predictive performance, with area under the curve values of 0.897, 0.813, and 0.750 for the training, validation, and testing datasets, respectively. The HALP score was significantly associated with LNM in RC patients. A machine learning model integrating the HALP score and inflammatory markers may be an effective tool for predicting LNM in RC.

Deep learning-based lymph node metastasis status predicts prognosis from muscle-invasive bladder cancer histopathology

PURPOSE

To develop a deep learning (DL) model based on primary tumor tissue to predict the lymph node metastasis (LNM) status of muscle invasive bladder cancer (MIBC), while validating the prognostic value of the predicted aiN score in MIBC patients.

METHODS

A total of 323 patients from The Cancer Genome Atlas (TCGA) were used as the training and internal validation set, with image features extracted using a visual encoder called UNI. We investigated the ability to predict LNM status while assessing the prognostic value of aiN score. External validation was conducted on 139 patients from Renmin Hospital of Wuhan University (RHWU; Wuhan, China).

RESULTS

The DL model achieved area under the receiver operating characteristic curves of 0.79 (95% confidence interval [CI], 0.69-0.88) in the internal validation set for predicting LNM status, and 0.72 (95% CI, 0.68-0.75) in the external validation set. In multivariable Cox analysis, the model-predicted aiN score emerged as an independent predictor of survival for MIBC patients, with a hazard ratio of 1.608 (95% CI, 1.128-2.291; p = 0.008) in the TCGA cohort and 2.746 (95% CI, 1.486-5.076; p < 0.001) in the RHWU cohort. Additionally, the aiN score maintained prognostic value across different subgroups.

CONCLUSION

In this study, DL-based image analysis showed promising results by directly extracting relevant prognostic information from H&E-stained histology to predict the LNM status of MIBC patients. It might be used for personalized management of MIBC patients following prospective validation in the future.

Accurate colorectal cancer detection using a random hinge exponential distribution coupled attention network on pathological images

Colorectal cancer (CRC) is one of the most common and deadly forms of cancer worldwide, necessitating accurate and early detection to improve treatment outcomes. Traditional diagnostic methods often rely on manual examination of pathological images, which can be time-consuming and prone to human error. This study presents an advanced approach for colorectal cancer detection using a Random Hinge Exponential Distribution coupled Attention Network (RHED-CANet) on pathological images. The input dataset is sourced from the TCGA-CRC-DX cohort and the CRC dataset, both widely recognized for their comprehensive coverage of colorectal cancer cases. Pre-processing and feature extraction are performed using a Modified Square Root Sage-Husa Adaptive Kalman Filter combined with a Spike-Driven Transformer, enhancing noise reduction and feature clarity. Segmentation is achieved through an EfficientNetV2L Inception Transformer, ensuring precise delineation of cancerous regions. The final classification utilizes the RHED-CANet, a network tailored to handle the complexities of pathological data with high accuracy. This methodology achieved remarkable results, with an accuracy of 99.9% and a precision of 99.7%. These performance metrics underscore the method's ability to minimize false positives and enhance diagnostic accuracy. The proposed approach offers significant advantages, including a reduction in diagnostic time and a substantial improvement in detection accuracy, making it a promising tool for clinical applications. Despite its excellent accuracy, the suggested RHED-CANet technique has drawbacks, such as overfitting the TCGA-CRC-DX and CRC datasets by reducing generalizability on other datasets comprising other cancer types or image qualities. The actual application of the techniques in real-time clinical applications may be hampered by this computational load, especially in settings with limited resources, and the model's potential computational complexity due to multiple advanced processing steps. Additionally, the efficiency of training may be impacted by biased inputs, particularly for minor CRC subtypes.

Preoperative prediction of lymph node metastasis in colorectal cancer using 18F-FDG PET/CT peritumoral radiomics analysis

BACKGROUND

Radiomics has been used in the diagnosis of tumor lymph node metastasis (LNM). However, to date, most studies have been based on intratumoral radiomics. Few studies have focused on the use of 18F-fluorodeoxyglucose positron emission computed tomography (18F-FDG PET/CT) peritumoral radiomics for the diagnosis of LNM in colorectal cancer (CRC).

PURPOSE

Determining the value of radiomics features extracted from 18F-FDG PET/CT images of the peritumoral region in predicting LNM in patients with CRC.

METHODS

The clinical data and preoperative 18F-FDG PET/CT images of 244 CRC patients were retrospectively analyzed. Intratumoral and peritumoral radiomics features were screened using the mutual information method, and least absolute shrinkage and selection operator regression. Based on the selected radiomics features, a radiomics score (Rad-score) was calculated, and independent risk factors obtained from univariate and multivariate logistic regression analyses were used to construct clinical and combined (Radiomics + Clinical) models. The performance of these models was evaluated using the DeLong test, while their clinical utility was assessed by decision curve analysis. Finally, a nomogram was constructed to visualize the predictive model.

RESULTS

The most optimal set of features retained by the feature filtering process were all peritumoral radiomic features. Carcinoembryonic antigen levels, PET/CT-reported lymph node status and Rad-score were found to be independent risk factors for LNM. All three LNM risk assessment models exhibited good predictive performance, with the combined model showing the best classification results, with areas under the curve of 0.85 and 0.76 in the training and validation groups, respectively. The DeLong test revealed that the performance of the combined model was superior to that of the clinical and radiomics models in both the training and validation groups, although this difference was only statistically significant in the training group. DCA indicated that the combined model displayed better clinical utility.

CONCLUSIONS

18F-FDG PET/CT peritumoral radiomics is uniquely suited to predict the presence of LNM in patients with CRC. In particular, the predictive efficacy of LNM for precision therapy and individualized patient management can be improved by using a combination of clinical risk factors.

Deep Learning Radiomics Analysis of CT Imaging for Differentiating Between Crohn's Disease and Intestinal Tuberculosis

This study aimed to develop and evaluate a CT-based deep learning radiomics model for differentiating between Crohn's disease (CD) and intestinal tuberculosis (ITB). A total of 330 patients with pathologically confirmed as CD or ITB from the First Affiliated Hospital of Zhengzhou University were divided into the validation dataset one (CD: 167; ITB: 57) and validation dataset two (CD: 78; ITB: 28). Based on the validation dataset one, the synthetic minority oversampling technique (SMOTE) was adopted to create balanced dataset as training data for feature selection and model construction. The handcrafted and deep learning (DL) radiomics features were extracted from the arterial and venous phases images, respectively. The interobserver consistency analysis, Spearman's correlation, univariate analysis, and the least absolute shrinkage and selection operator (LASSO) regression were used to select features. Based on extracted multi-phase radiomics features, six logistic regression models were finally constructed. The diagnostic performances of different models were compared using ROC analysis and Delong test. The arterial-venous combined deep learning radiomics model for differentiating between CD and ITB showed a high prediction quality with AUCs of 0.885, 0.877, and 0.800 in SMOTE dataset, validation dataset one, and validation dataset two, respectively. Moreover, the deep learning radiomics model outperformed the handcrafted radiomics model in same phase images. In validation dataset one, the Delong test results indicated that there was a significant difference in the AUC of the arterial models (p = 0.037), while not in venous and arterial-venous combined models (p = 0.398 and p = 0.265) as comparing deep learning radiomics models and handcrafted radiomics models. In our study, the arterial-venous combined model based on deep learning radiomics analysis exhibited good performance in differentiating between CD and ITB.

Interpretable deep learning model to predict lymph node metastasis in early gastric cancer using whole slide images

In early gastric cancer (EGC), the presence of lymph node metastasis (LNM) is a crucial factor for determining the treatment options. Endoscopic resection is used for treatment of EGC with minimal risk of LNM. However, owing to the lack of definitive criteria for identifying patients who require additional surgery, some patients undergo unnecessary additional surgery. Considering that histopathologic patterns are significant factor for predicting lymph node metastasis in gastric cancer, we aimed to develop a machine learning algorithm which can predict LNM status using hematoxylin and eosin (H&E)-stained images. The images were obtained from several institutions. Our pipeline comprised two sequential approaches including a feature extractor and a risk classifier. For the feature extractor, a segmentation network (DeepLabV3+) was trained on 243 WSIs across three datasets to differentiate each histological subtype. The risk classifier was trained with XGBoost using 70 morphological features inferred from the trained feature extractor. The trained segmentation network, the feature extractor, achieved high performance, with pixel accuracies of 0.9348 and 0.8939 for the internal and external datasets in patch level, respectively. The risk classifier achieved an overall AUC of 0.75 in predicting LNM status. Remarkably, one of the datasets also showed a promising result with an AUC of 0.92. This is the first multi-institution study to develop machine learning algorithm for predicting LNM status in patients with EGC using H&E-stained histopathology images. Our findings have the potential to improve the selection of patients who require surgery among those with EGC showing high-risk histological features.

Magnetic resonance imaging-based radiomics and deep learning models for predicting lymph node metastasis of squamous cell carcinoma of the tongue

OBJECTIVE

This study aimed to establish a combined method of radiomics and deep learning (DL) in magnetic resonance imaging (MRI) to predict lymph node metastasis (LNM) preoperatively in patients with squamous cell carcinoma of the tongue.

STUDY DESIGN

In total, MR images of 196 patients with lingual squamous cell carcinoma were divided into training (n = 156) and test (n = 40) cohorts. Radiomics and DL features were extracted from MR images and selected to construct machine learning models. A DL radiomics nomogram was established via multivariate logistic regression by incorporating the radiomics signature, the DL signature, and MRI-reported LN status.

RESULTS

Nine radiomics and 3 DL features were selected. In the radiomics test cohort, the multilayer perceptron model performed best with an area under the receiver operating characteristic curve (AUC) of 0.747, but in the DL cohort, the best model (logistic regression) performed less well (AUC = 0.655). The DL radiomics nomogram showed good calibration and performance with an AUC of 0.934 (outstanding discrimination ability) in the training cohort and 0.757 (acceptable discrimination ability) in the test cohort. The decision curve analysis demonstrated that the nomogram could offer more net benefit than a single radiomics or DL signature.

CONCLUSION

The DL radiomics nomogram exhibited promising performance in predicting LNM, which facilitates personalized treatment of tongue cancer.

3-Dimensional Reconstruction From Histopathological Sections: A Systematic Review

Although pathological tissue analysis is typically performed on single 2-dimensional (2D) histologic reference slides, 3-dimensional (3D) reconstruction from a sequence of histologic sections could provide novel opportunities for spatial analysis of the extracted tissue. In this review, we analyze recent works published after 2018 and report information on the extracted tissue types, the section thickness, and the number of sections used for reconstruction. By analyzing the technological requirements for 3D reconstruction, we observe that software tools exist, both free and commercial, which include the functionality to perform 3D reconstruction from a sequence of histologic images. Through the analysis of the most recent works, we provide an overview of the workflows and tools that are currently used for 3D reconstruction from histologic sections and address points for future work, such as a missing common file format or computer-aided analysis of the reconstructed model.

Multi-domain stain normalization for digital pathology: A cycle-consistent adversarial network for whole slide images

The variation in histologic staining between different medical centers is one of the most profound challenges in the field of computer-aided diagnosis. The appearance disparity of pathological whole slide images causes algorithms to become less reliable, which in turn impedes the wide-spread applicability of downstream tasks like cancer diagnosis. Furthermore, different stainings lead to biases in the training which in case of domain shifts negatively affect the test performance. Therefore, in this paper we propose MultiStain-CycleGAN, a multi-domain approach to stain normalization based on CycleGAN. Our modifications to CycleGAN allow us to normalize images of different origins without retraining or using different models. We perform an extensive evaluation of our method using various metrics and compare it to commonly used methods that are multi-domain capable. First, we evaluate how well our method fools a domain classifier that tries to assign a medical center to an image. Then, we test our normalization on the tumor classification performance of a downstream classifier. Furthermore, we evaluate the image quality of the normalized images using the Structural similarity index and the ability to reduce the domain shift using the Fréchet inception distance. We show that our method proves to be multi-domain capable, provides a very high image quality among the compared methods, and can most reliably fool the domain classifier while keeping the tumor classifier performance high. By reducing the domain influence, biases in the data can be removed on the one hand and the origin of the whole slide image can be disguised on the other, thus enhancing patient data privacy.

Predicting rectal cancer prognosis from histopathological images and clinical information using multi-modal deep learning

Objective

The objective of this study was to provide a multi-modal deep learning framework for forecasting the survival of rectal cancer patients by utilizing both digital pathological images data and non-imaging clinical data.

Materials and methods

The research included patients diagnosed with rectal cancer by pathological confirmation from January 2015 to December 2016. Patients were allocated to training and testing sets in a randomized manner, with a ratio of 4:1. The tissue microarrays (TMAs) and clinical indicators were obtained. Subsequently, we selected distinct deep learning models to individually forecast patient survival. We conducted a scanning procedure on the TMAs in order to transform them into digital pathology pictures. Additionally, we performed pre-processing on the clinical data of the patients. Subsequently, we selected distinct deep learning algorithms to conduct survival prediction analysis using patients' pathological images and clinical data, respectively.

Results

A total of 292 patients with rectal cancer were randomly allocated into two groups: a training set consisting of 234 cases, and a testing set consisting of 58 instances. Initially, we make direct predictions about the survival status by using pre-processed Hematoxylin and Eosin (H&E) pathological images of rectal cancer. We utilized the ResNest model to extract data from histopathological images of patients, resulting in a survival status prediction with an AUC (Area Under the Curve) of 0.797. Furthermore, we employ a multi-head attention fusion (MHAF) model to combine image features and clinical features in order to accurately forecast the survival rate of rectal cancer patients. The findings of our experiment show that the multi-modal structure works better than directly predicting from histopathological images. It achieves an AUC of 0.837 in predicting overall survival (OS).

Conclusions

Our study highlights the potential of multi-modal deep learning models in predicting survival status from histopathological images and clinical information, thus offering valuable insights for clinical applications.

Overexpression of circulating CD38+ NK cells in colorectal cancer was associated with lymph node metastasis and poor prognosis

Introduction

Lymph node metastasis (LNM) is a critical prognostic factor for colorectal cancer (CRC). Due to the potential influence of immune system on CRC progression, investigation into lymphocyte subsets as clinical markers has gained attention. The objective of this study was to assess the capability of lymphocyte subsets in evaluating the lymph node status and prognosis of CRC.

Methods

Lymphocyte subsets, including T cells (CD3+), natural killer cells (NK, CD3- CD56+), natural killer-like T cells (NK-like T, CD3+ CD56+), CD38+ NK cells (CD3- CD56+ CD38+) and CD38+ NK-like T cells (CD3+ CD56+ CD38+), were detected by flow cytometry. Univariate and multivariate analyses were used to assess the risk factors of LNM. The prognostic role of parameters was evaluated by survival analysis.

Results

The proportion of CD38+ NK cells within the NK cell population was significantly higher in LNM-positive patients (p <0.0001). However, no significant differences were observed in the proportions of other lymphocyte subsets. Poorer histologic grade (odds ratio [OR] =4.76, p =0.03), lymphovascular invasion (LVI) (OR =22.38, p <0.01), and CD38+ NK cells (high) (OR =4.54, p <0.01) were identified as independent risk factors for LNM. Furthermore, high proportion of CD38+ NK cells was associated with poor prognosis of CRC patients (HR=2.37, p =0.03).

Conclusions

It was demonstrated that the proportion of CD38+ NK cells was a marker overexpressed in LNM-positive patients compared with LNM-negative patients. Moreover, an elevated proportion of CD38+ NK cells is a risk factor for LNM and poor prognosis in CRC.

Association of the pathomics-collagen signature with lymph node metastasis in colorectal cancer: a retrospective multicenter study

BACKGROUND

Lymph node metastasis (LNM) is a prognostic biomarker and affects therapeutic selection in colorectal cancer (CRC). Current evaluation methods are not adequate for estimating LNM in CRC. H&E images contain much pathological information, and collagen also affects the biological behavior of tumor cells. Hence, the objective of the study is to investigate whether a fully quantitative pathomics-collagen signature (PCS) in the tumor microenvironment can be used to predict LNM.

METHODS

Patients with histologically confirmed stage I-III CRC who underwent radical surgery were included in the training cohort (n = 329), the internal validation cohort (n = 329), and the external validation cohort (n = 315). Fully quantitative pathomics features and collagen features were extracted from digital H&E images and multiphoton images of specimens, respectively. LASSO regression was utilized to develop the PCS. Then, a PCS-nomogram was constructed incorporating the PCS and clinicopathological predictors for estimating LNM in the training cohort. The performance of the PCS-nomogram was evaluated via calibration, discrimination, and clinical usefulness. Furthermore, the PCS-nomogram was tested in internal and external validation cohorts.

RESULTS

By LASSO regression, the PCS was developed based on 11 pathomics and 9 collagen features. A significant association was found between the PCS and LNM in the three cohorts (P < 0.001). Then, the PCS-nomogram based on PCS, preoperative CEA level, lymphadenectasis on CT, venous emboli and/or lymphatic invasion and/or perineural invasion (VELIPI), and pT stage achieved AUROCs of 0.939, 0.895, and 0.893 in the three cohorts. The calibration curves identified good agreement between the nomogram-predicted and actual outcomes. Decision curve analysis indicated that the PCS-nomogram was clinically useful. Moreover, the PCS was still an independent predictor of LNM at station Nos. 1, 2, and 3. The PCS nomogram displayed AUROCs of 0.849-0.939 for the training cohort, 0.837-0.902 for the internal validation cohort, and 0.851-0.895 for the external validation cohorts in the three nodal stations.

CONCLUSIONS

This study proposed that PCS integrating pathomics and collagen features was significantly associated with LNM, and the PCS-nomogram has the potential to be a useful tool for predicting individual LNM in CRC patients.

[Artificial intelligence in pathological anatomy]

The review presents key concepts and global developments in the field of artificial intelligence used in pathological anatomy. The work examines two types of artificial intelligence (AI): weak and strong ones. A review of experimental algorithms using both deep machine learning and computer vision technologies to work with WSI images of preparations, diagnose and make a prognosis for various malignant neoplasms is carried out. It has been established that weak artificial intelligence at this stage of development of computer (digital) pathological anatomy shows significantly better results in speeding up and refining diagnostic procedures than strong artificial intelligence having signs of general intelligence. The article also discusses three options for the further development of AI assistants for pathologists based on the technologies of large language models (strong AI) ChatGPT (PathAsst), Flan-PaLM2 and LIMA. As a result of the analysis of the literature, key problems in the field were identified: the equipment of pathology institutions, the lack of experts in training neural networks, the lack of strict criteria for the clinical viability of AI diagnostic technologies.

Deep learning to predict breast cancer sentinel lymph node status on INSEMA histological images

BACKGROUND

Sentinel lymph node (SLN) status is a clinically important prognostic biomarker in breast cancer and is used to guide therapy, especially for hormone receptor-positive, HER2-negative cases. However, invasive lymph node staging is increasingly omitted before therapy, and studies such as the randomised Intergroup Sentinel Mamma (INSEMA) trial address the potential for further de-escalation of axillary surgery. Therefore, it would be helpful to accurately predict the pretherapeutic sentinel status using medical images.

METHODS

Using a ResNet 50 architecture pretrained on ImageNet and a previously successful strategy, we trained deep learning (DL)-based image analysis algorithms to predict sentinel status on hematoxylin/eosin-stained images of predominantly luminal, primary breast tumours from the INSEMA trial and three additional, independent cohorts (The Cancer Genome Atlas (TCGA) and cohorts from the University hospitals of Mannheim and Regensburg), and compared their performance with that of a logistic regression using clinical data only. Performance on an INSEMA hold-out set was investigated in a blinded manner.

RESULTS

None of the generated image analysis algorithms yielded significantly better than random areas under the receiver operating characteristic curves on the test sets, including the hold-out test set from INSEMA. In contrast, the logistic regression fitted on the Mannheim cohort retained a better than random performance on INSEMA and Regensburg. Including the image analysis model output in the logistic regression did not improve performance further on INSEMA.

CONCLUSIONS

Employing DL-based image analysis on histological slides, we could not predict SLN status for unseen cases in the INSEMA trial and other predominantly luminal cohorts.

Prediction of lymph node metastasis of lung squamous cell carcinoma by machine learning algorithm classifiers

BACKGROUND

Lymph node metastasis (LNM) is an essential factor affecting the prognosis of patients with lung squamous cell carcinoma (LUSC), as well as a critical consideration for the choice of treatment strategy. Exploring effective methods for predicting LNM in LUSC may benefit clinical decision making.

MATERIALS AND METHODS

We used data collected from the Surveillance, Epidemiology, and End Results (SEER) database to develop machine learning algorithm classifiers, including boosted trees (BTs), based on the primary clinical parameters of patients to predict LNM in LUSC. Training on a large-sample training cohort (n = 8,063) allowed for the construction of several concise classifiers for LNM prediction in LUSC, which were then validated using test and in-house cohorts (n = 2,017 and 57, respectively).

RESULTS

The six classifiers established in this research enabled distinction between patients with and without LNM. Among these classifiers, the BT classifier was the top performer, with accuracy, F1 scores, precision, recall, sensitivity, and specificity values of 0.654, 0.621, 0.654, 0.592, 0.592, and 0.711, respectively. The precision recall (PR) and receiver operating characteristic (ROC) (with area under the curve = 0.714) curves also supported this result, which was validated by the in-house cohort. Notably, the tumor stage was a critical factor in determining LNM in patients with LUSC.

CONCLUSIONS

The use of classifiers, especially the BT classifier, may serve as a useful tool for improving clinical precision and individualized treatment of patients with LUSC.

A 3-dimensional histology computer model of malignant melanoma and its implications for digital pathology

BACKGROUND

Historically, cancer diagnoses have been made by pathologists using two-dimensional histological slides. However, with the advent of digital pathology and artificial intelligence, slides are being digitised, providing new opportunities to integrate their information. Since nature is 3-dimensional (3D), it seems intuitive to digitally reassemble the 3D structure for diagnosis.

OBJECTIVE

To develop the first human-3D-melanoma-histology-model with full data and code availability. Further, to evaluate the 3D-simulation together with experienced pathologists in the field and discuss the implications of digital 3D-models for the future of digital pathology.

METHODS

A malignant melanoma of the skin was digitised via 3 µm cuts by a slide scanner; an open-source software was then leveraged to construct the 3D model. A total of nine pathologists from four different countries with at least 10 years of experience in the histologic diagnosis of melanoma tested the model and discussed their experiences as well as implications for future pathology.

RESULTS

We successfully constructed and tested the first 3D-model of human melanoma. Based on testing, 88.9% of pathologists believe that the technology is likely to enter routine pathology within the next 10 years; advantages include a better reflectance of anatomy, 3D assessment of symmetry and the opportunity to simultaneously evaluate different tissue levels at the same time; limitations include the high consumption of tissue and a yet inferior resolution due to computational limitations.

CONCLUSIONS

3D-histology-models are promising for digital pathology of cancer and melanoma specifically, however, there are yet limitations which need to be carefully addressed.

Deep learning trained on lymph node status predicts outcome from gastric cancer histopathology: a retrospective multicentric study

AIM

Gastric cancer (GC) is a tumour entity with highly variant outcomes. Lymph node metastasis is a prognostically adverse biomarker. We hypothesised that GC primary tissue contains information that is predictive of lymph node status and patient prognosis and that this information can be extracted using deep learning (DL).

METHODS

Using three patient cohorts comprising 1146 patients, we trained and validated a DL system to predict lymph node status directly from haematoxylin and eosin-stained GC tissue sections. We investigated the concordance between the DL-based prediction from the primary tumour slides (aiN score) and the histopathological lymph node status (pN). Furthermore, we assessed the prognostic value of the aiN score alone and when combined with the pN status.

RESULTS

The aiN score predicted the pN status reaching area under the receiver operating characteristic curves of 0.71 in the training cohort and 0.69 and 0.65 in the two test cohorts. In a multivariate Cox analysis, the aiN score was an independent predictor of patient survival with hazard ratios of 1.5 in the training cohort and of 1.3 and 2.2 in the two test cohorts. A combination of the aiN score and the pN status prognostically stratified patients by survival with p-values <0.05 in logrank tests.

CONCLUSION

GC primary tumour tissue contains additional prognostic information that is accessible using the aiN score. In combination with the pN status, this can be used for personalised management of GC patients after prospective validation.

Radiomics in colorectal cancer

Colorectal cancer (CRC) is a global health challenge with high morbidity and mortality. Radiomics, an emerging field, utilizes quantitative imaging features extracted from medical images for CRC diagnosis, staging, treatment response assessment, and prognostication. This review highlights the potential of radiomics for personalized CRC management. Radiomics enables noninvasive tumor characterization, aiding in early detection and accurate diagnosis, and it can be used to predict tumor stage, lymph node involvement, and prognosis. Furthermore, radiomics guides personalized therapies by assessing the treatment response and identifying patients who could benefit. Challenges include standardizing imaging protocols and analysis techniques. Robust validation frameworks and user‐friendly software are needed for the integration of radiomics into clinical practice. Despite challenges, radiomics offers valuable insights into tumor biology, treatment response, and prognosis in CRC. Overcoming technical and clinical hurdles will unlock its full potential in CRC management.

High serum mannose in colorectal cancer: a novel biomarker of lymph node metastasis and poor prognosis

Background

Lymph node status is an important prognostic indicator and it significantly influences treatment decisions for colorectal cancer (CRC). The objective of this study was to evaluate the ability of serum monosaccharides in predicting lymph node metastasis (LNM) and prognosis.

Methods

High performance anion exchange chromatography coupled with pulsed amperometric detector (HPAEC-PAD) was used to quantify serum monosaccharides from 252 CRC patients. Receiver operating characteristic (ROC) curves were used to evaluate predictive performance of parameters. Predictors of LNM were evaluated by univariate and multivariate analyses. The prognostic role of the factors was evaluated by survival analysis.

Results

The levels of serum mannose (Man) and galactose (Gal) were significantly increased in patients with LNM (p <0.0001, p =0.0017, respectively). The area under the curves (AUCs) of Man was 0.8140, which was higher than carcinoembryonic antigen (CEA) (AUC =0.6523). Univariate and multivariate analyses demonstrated histologic grade (G3) (odds ratio [OR] =2.60, p =0.043), histologic grade (mucin-producing subtype) (odds ratio [OR] =3.38, p =0.032), lymphovascular invasion (LVI) (OR =2.42, p <0.01), CEA (>5ng/ml) (OR =1.85, p =0.042) and high Man (OR =2.65, p =0.006) to be independent risk factors of LNM. The survival analysis showed that the high serum Man was independent risk factor for poor prognosis in CRC patients (HR=1.75, p =0.004).

Conclusions

The Man is superior to CEA in prediction of LNM for CRC patients. Man is expected to be a predictor for LNM in CRC. High serum Man is associated with poor prognosis of CRC patients.

Spatial cellular architecture predicts prognosis in glioblastoma

Intra-tumoral heterogeneity and cell-state plasticity are key drivers for the therapeutic resistance of glioblastoma. Here, we investigate the association between spatial cellular organization and glioblastoma prognosis. Leveraging single-cell RNA-seq and spatial transcriptomics data, we develop a deep learning model to predict transcriptional subtypes of glioblastoma cells from histology images. Employing this model, we phenotypically analyze 40 million tissue spots from 410 patients and identify consistent associations between tumor architecture and prognosis across two independent cohorts. Patients with poor prognosis exhibit higher proportions of tumor cells expressing a hypoxia-induced transcriptional program. Furthermore, a clustering pattern of astrocyte-like tumor cells is associated with worse prognosis, while dispersion and connection of the astrocytes with other transcriptional subtypes correlate with decreased risk. To validate these results, we develop a separate deep learning model that utilizes histology images to predict prognosis. Applying this model to spatial transcriptomics data reveal survival-associated regional gene expression programs. Overall, our study presents a scalable approach to unravel the transcriptional heterogeneity of glioblastoma and establishes a critical connection between spatial cellular architecture and clinical outcomes.

Predicting Lymph Node Metastasis Status from Primary Muscle-Invasive Bladder Cancer Histology Slides Using Deep Learning: A Retrospective Multicenter Study

BACKGROUND

Accurate prediction of lymph node metastasis (LNM) status in patients with muscle-invasive bladder cancer (MIBC) before radical cystectomy can guide the use of neoadjuvant chemotherapy and the extent of pelvic lymph node dissection. We aimed to develop and validate a weakly-supervised deep learning model to predict LNM status from digitized histopathological slides in MIBC.

METHODS

We trained a multiple instance learning model with an attention mechanism (namely SBLNP) from a cohort of 323 patients in the TCGA cohort. In parallel, we collected corresponding clinical information to construct a logistic regression model. Subsequently, the score predicted by the SBLNP was incorporated into the logistic regression model. In total, 417 WSIs from 139 patients in the RHWU cohort and 230 WSIs from 78 patients in the PHHC cohort were used as independent external validation sets.

RESULTS

In the TCGA cohort, the SBLNP achieved an AUROC of 0.811 (95% confidence interval [CI], 0.771-0.855), the clinical classifier achieved an AUROC of 0.697 (95% CI, 0.661-0.728) and the combined classifier yielded an improvement to 0.864 (95% CI, 0.827-0.906). Encouragingly, the SBLNP still maintained high performance in the RHWU cohort and PHHC cohort, with an AUROC of 0.762 (95% CI, 0.725-0.801) and 0.746 (95% CI, 0.687-0.799), respectively. Moreover, the interpretability of SBLNP identified stroma with lymphocytic inflammation as a key feature of predicting LNM presence.

CONCLUSIONS

Our proposed weakly-supervised deep learning model can predict the LNM status of MIBC patients from routine WSIs, demonstrating decent generalization performance and holding promise for clinical implementation.

Multimodal AI for prediction of distant metastasis in carcinoma patients

Metastasis of cancer is directly related to death in almost all cases, however a lot is yet to be understood about this process. Despite advancements in the available radiological investigation techniques, not all cases of Distant Metastasis (DM) are diagnosed at initial clinical presentation. Also, there are currently no standard biomarkers of metastasis. Early, accurate diagnosis of DM is however crucial for clinical decision making, and planning of appropriate management strategies. Previous works have achieved little success in attempts to predict DM from either clinical, genomic, radiology, or histopathology data. In this work we attempt a multimodal approach to predict the presence of DM in cancer patients by combining gene expression data, clinical data and histopathology images. We tested a novel combination of Random Forest (RF) algorithm with an optimization technique for gene selection, and investigated if gene expression pattern in the primary tissues of three cancer types (Bladder Carcinoma, Pancreatic Adenocarcinoma, and Head and Neck Squamous Carcinoma) with DM are similar or different. Gene expression biomarkers of DM identified by our proposed method outperformed Differentially Expressed Genes (DEGs) identified by the DESeq2 software package in the task of predicting presence or absence of DM. Genes involved in DM tend to be more cancer type specific rather than general across all cancers. Our results also indicate that multimodal data is more predictive of metastasis than either of the three unimodal data tested, and genomic data provides the highest contribution by a wide margin. The results re-emphasize the importance for availability of sufficient image data when a weakly supervised training technique is used. Code is made available at: https://github.com/rit-cui-lab/Multimodal-AI-for-Prediction-of-Distant-Metastasis-in-Carcinoma-Patients.

Application of artificial neural network algorithm in pathological diagnosis and prognosis prediction of digestive tract malignant tumors

The application of artificial neural network algorithm in pathological diagnosis of gastrointestinal malignant tumors has become a research hotspot. In the previous studies, the algorithm research mainly focused on the model development based on convolutional neural networks, while only a few studies used the combination of convolutional neural networks and recurrent neural networks. The research contents included classical histopathological diagnosis and molecular typing of malignant tumors, and the prediction of patient prognosis by utilizing artificial neural networks. This article reviews the research progress on artificial neural network algorithm in the pathological diagnosis and prognosis prediction of digestive tract malignant tumors.

Predicting lymph node metastasis from primary tumor histology and clinicopathologic factors in colorectal cancer using deep learning

BACKGROUND

Presence of lymph node metastasis (LNM) influences prognosis and clinical decision-making in colorectal cancer. However, detection of LNM is variable and depends on a number of external factors. Deep learning has shown success in computational pathology, but has struggled to boost performance when combined with known predictors.

METHODS

Machine-learned features are created by clustering deep learning embeddings of small patches of tumor in colorectal cancer via k-means, and then selecting the top clusters that add predictive value to a logistic regression model when combined with known baseline clinicopathological variables. We then analyze performance of logistic regression models trained with and without these machine-learned features in combination with the baseline variables.

RESULTS

The machine-learned extracted features provide independent signal for the presence of LNM (AUROC: 0.638, 95% CI: [0.590, 0.683]). Furthermore, the machine-learned features add predictive value to the set of 6 clinicopathologic variables in an external validation set (likelihood ratio test, p < 0.00032; AUROC: 0.740, 95% CI: [0.701, 0.780]). A model incorporating these features can also further risk-stratify patients with and without identified metastasis (p < 0.001 for both stage II and stage III).

CONCLUSION

This work demonstrates an effective approach to combine deep learning with established clinicopathologic factors in order to identify independently informative features associated with LNM. Further work building on these specific results may have important impact in prognostication and therapeutic decision making for LNM. Additionally, this general computational approach may prove useful in other contexts.

Colorectal cancer lymph node metastasis prediction with weakly supervised transformer-based multi-instance learning

Lymph node metastasis examined by the resected lymph nodes is considered one of the most important prognostic factors for colorectal cancer (CRC). However, it requires careful and comprehensive inspection by expert pathologists. To relieve the pathologists' burden and speed up the diagnostic process, in this paper, we develop a deep learning system with the binary positive/negative labels of the lymph nodes to solve the CRC lymph node classification task. The multi-instance learning (MIL) framework is adopted in our method to handle the whole slide images (WSIs) of gigapixels in size at once and get rid of the labor-intensive and time-consuming detailed annotations. First, a transformer-based MIL model, DT-DSMIL, is proposed in this paper based on the deformable transformer backbone and the dual-stream MIL (DSMIL) framework. The local-level image features are extracted and aggregated with the deformable transformer, and the global-level image features are obtained with the DSMIL aggregator. The final classification decision is made based on both the local and the global-level features. After the effectiveness of our proposed DT-DSMIL model is demonstrated by comparing its performance with its predecessors, a diagnostic system is developed to detect, crop, and finally identify the single lymph nodes within the slides based on the DT-DSMIL and the Faster R-CNN model. The developed diagnostic model is trained and tested on a clinically collected CRC lymph node metastasis dataset composed of 843 slides (864 metastasis lymph nodes and 1415 non-metastatic lymph nodes), achieving the accuracy of 95.3% and the area under the receiver operating characteristic curve (AUC) of 0.9762 (95% confidence interval [CI]: 0.9607-0.9891) for the single lymph node classification. As for the lymph nodes with micro-metastasis and macro-metastasis, our diagnostic system achieves the AUC of 0.9816 (95% CI: 0.9659-0.9935) and 0.9902 (95% CI: 0.9787-0.9983), respectively. Moreover, the system shows reliable diagnostic region localizing performance: the model can always identify the most likely metastases, no matter the model's predictions or manual labels, showing great potential in avoiding false negatives and discovering incorrectly labeled slides in actual clinical use.

A Deep Neural Network-Based Decision Support Tool for the Detection of Lymph Node Metastases in Colorectal Cancer Specimens

The identification of lymph node metastases in colorectal cancer (CRC) specimens is crucial for the planning of postoperative treatment and can be a time-consuming task for pathologists. In this study, we developed a deep neural network (DNN) algorithm for the detection of metastatic CRC in digitized histologic sections of lymph nodes and evaluated its performance as a diagnostic support tool. First, the DNN algorithm was trained using pixel-level annotations of cancerous areas on 758 whole slide images (360 with cancerous areas). The algorithm's performance was evaluated on 74 whole slide images (43 with cancerous areas). Second, the algorithm was evaluated as a decision support tool on 288 whole slide images covering 1517 lymph node sections, randomized in 16 batches. Two senior pathologists (C.K. and C.O.) assessed each batch with and without the help of the algorithm in a 2 × 2 crossover design, with a washout period of 1 month in between. The time needed for the evaluation of each node section was recorded. The DNN algorithm achieved a median pixel-level accuracy of 0.952 on slides with cancerous areas and 0.996 on slides with benign samples. N+ disease (metastases, micrometastases, or tumor deposits) was present in 103 of the 1517 sections. The algorithm highlighted cancerous areas in 102 of these sections, with a sensitivity of 0.990. Assisted by the algorithm, the median time needed for evaluation was significantly shortened for both pathologists (median time for pathologist 1, 26 vs 14 seconds; P < .001; 95% CI, 11.0-12.0; median time for pathologist 2, 25 vs 23 seconds; P < .001; 95% CI, 2.0-4.0). Our DNN showed high accuracy for detecting metastatic CRC in digitized histologic sections of lymph nodes. This decision support tool has the potential to improve the diagnostic workflow by shortening the time needed for the evaluation of lymph nodes in CRC specimens without impairing diagnostic accuracy.

Prognostic value of log odds of positive lymph nodes, lymph node ratio, and N stage in patients with colorectal signet ring cell carcinoma: A retrospective cohort study

Aim

Little attention has been paid in the prognosis of colorectal signet ring cell carcinoma (SRCC). This study aims to explore the predictive capacity of log odds of positive lymph nodes (LODDS), lymph node ratio (LNR), and pN stage in the prognosis of patients with colorectal SRCC.

Methods

A retrospective cohort study was designed, and data were extracted from the Surveillance, Epidemiology and End Results (SEER) database. Data on demographic characteristics, clinicopathological features, and treatment were extracted. Outcomes were overall survival (OS) and cancer-specific survival (CSS). Association of LODDS, LNR, and pN stage with OS and CSS were explored using Cox proportional hazard model and Cox competing risk model, respectively, with results showing as hazard ratio and 95% confidence interval (CI). Predictive performance of LODDS, LNR, and pN stage in OS and CSS was assessed by calculating C-index.

Results

A total of 2,198 patients were included in this study. LODDS, LNR, and pN stage were associated with the OS and CSS of colorectal SRCC patients (all P < 0.05). LODDS showed a good performance in the OS (C-index: 0.704, 95% CI: 0.690-0.718), which was superior to LNR (C-index: 0.657, 95% CI: 0.643-0.671) and pN stage (C-index: 0.643, 95% CI: 0.629-0.657). The C-index of LODDS, LNR, and pN stage for CSS was 0.733 (95% CI: 0.719-0.747), 0.713 (95% CI: 0.697-0.729), and 0.667 (95% CI: 0.651-0.683), respectively.

Conclusions

LODDS displayed a better predictive capacity in the OS and CSS than LNR and pN stage, indicating that LODDS may be effective to predict the prognosis of colorectal SRCC in the clinic.

Artificial intelligence applications in predicting the behavior of gastrointestinal cancers in pathology

Recent research has provided a wealth of data supporting the application of artificial intelligence (AI)-based applications in routine pathology practice. Indeed, it is clear that these methods can significantly support an accurate and rapid diagnosis by eliminating errors, increasing reliability, and improving workflow. In addition, the effectiveness of AI in the pathological evaluation of prognostic parameters associated with behavior, course, and treatment in many types of tumors has also been noted. Regarding gastrointestinal system (GIS) cancers, the contribution of AI methods to pathological diagnosis has been investigated in many studies. On the other hand, studies focusing on AI applications in evaluating parameters to determine tumor behavior are relatively few. For this purpose, the potential of AI models has been studied over a broad spectrum, from tumor subtyping to the identification of new digital biomarkers. The capacity of AI to infer genetic alterations of cancer tissues from digital slides has been demonstrated. Although current data suggest the merit of AI-based approaches in assessing tumor behavior in GIS cancers, a wide range of challenges still need to be solved, from laboratory infrastructure to improving the robustness of algorithms, before incorporating AI applications into real-life GIS pathology practice. This review aims to present data from AI applications in evaluating pathological parameters related to the behavior of GIS cancer with an overview of the opportunities and challenges encountered in implementing AI in pathology.

Deep-Learning to Predict BRCA Mutation and Survival from Digital H&E Slides of Epithelial Ovarian Cancer

BRCA 1/2 genes mutation status can already determine the therapeutic algorithm of high grade serous ovarian cancer patients. Nevertheless, its assessment is not sufficient to identify all patients with genomic instability, since BRCA 1/2 mutations are only the most well-known mechanisms of homologous recombination deficiency (HR-d) pathway, and patients displaying HR-d behave similarly to BRCA mutated patients. HRd assessment can be challenging and is progressively overcoming BRCA testing not only for prognostic information but more importantly for drugs prescriptions. However, HR testing is not already integrated in clinical practice, it is quite expensive and it is not refundable in many countries. Selecting patients who are more likely to benefit from this assessment (BRCA 1/2 WT patients) at an early stage of the diagnostic process, would allow an optimization of genomic profiling resources. In this study, we sought to explore whether somatic BRCA1/2 genes status can be predicted using computational pathology from standard hematoxylin and eosin histology. In detail, we adopted a publicly available, deep-learning-based weakly supervised method that uses attention-based learning to automatically identify sub regions of high diagnostic value to accurately classify the whole slide (CLAM). The same model was also tested for progression free survival (PFS) prediction. The model was tested on a cohort of 664 (training set: n = 464, testing set: n = 132) ovarian cancer patients, of whom 233 (35.1%) had a somatic BRCA 1/2 mutation. An area under the curve of 0.7 and 0.55 was achieved in the training and testing set respectively. The model was then further refined by manually identifying areas of interest in half of the cases. 198 images were used for training (126/72) and 87 images for validation (55/32). The model reached a zero classification error on the training set, but the performance was 0.59 in terms of validation ROC AUC, with a 0.57 validation accuracy. Finally, when applied to predict PFS, the model achieved an AUC of 0.71, with a negative predictive value of 0.69, and a positive predictive value of 0.75. Based on these analyses, we have planned further steps of development such as proving a reference classification performance, exploring the hyperparameters space for training optimization, eventually tweaking the learning algorithms and the neural networks architecture for better suiting this specific task. These actions may allow the model to improve performances for all the considered outcomes.

Deep Learning on Histopathological Images for Colorectal Cancer Diagnosis: A Systematic Review

Colorectal cancer (CRC) is the second most common cancer in women and the third most common in men, with an increasing incidence. Pathology diagnosis complemented with prognostic and predictive biomarker information is the first step for personalized treatment. The increased diagnostic load in the pathology laboratory, combined with the reported intra- and inter-variability in the assessment of biomarkers, has prompted the quest for reliable machine-based methods to be incorporated into the routine practice. Recently, Artificial Intelligence (AI) has made significant progress in the medical field, showing potential for clinical applications. Herein, we aim to systematically review the current research on AI in CRC image analysis. In histopathology, algorithms based on Deep Learning (DL) have the potential to assist in diagnosis, predict clinically relevant molecular phenotypes and microsatellite instability, identify histological features related to prognosis and correlated to metastasis, and assess the specific components of the tumor microenvironment.

Artificial intelligence in liver diseases: improving diagnostics, prognostics and response prediction

Clinical routine in hepatology involves the diagnosis and treatment of a wide spectrum of metabolic, infectious, autoimmune and neoplastic diseases. Clinicians integrate qualitative and quantitative information from multiple data sources to make a diagnosis, prognosticate the disease course, and recommend a treatment. In the last 5 years, advances in artificial intelligence (AI), particularly in deep learning, have made it possible to extract clinically relevant information from complex and diverse clinical datasets. In particular, histopathology and radiology image data contain diagnostic, prognostic and predictive information which AI can extract. Ultimately, such AI systems could be implemented in clinical routine as decision support tools. However, in the context of hepatology, this requires further large-scale clinical validation and regulatory approval. Herein, we summarise the state of the art in AI in hepatology with a particular focus on histopathology and radiology data. We present a roadmap for the further development of novel biomarkers in hepatology and outline critical obstacles which need to be overcome.

Integration of deep learning-based image analysis and genomic data in cancer pathology: A systematic review

BACKGROUND

Over the past decade, the development of molecular high-throughput methods (omics) increased rapidly and provided new insights for cancer research. In parallel, deep learning approaches revealed the enormous potential for medical image analysis, especially in digital pathology. Combining image and omics data with deep learning tools may enable the discovery of new cancer biomarkers and a more precise prediction of patient prognosis. This systematic review addresses different multimodal fusion methods of convolutional neural network-based image analyses with omics data, focussing on the impact of data combination on the classification performance.

METHODS

PubMed was screened for peer-reviewed articles published in English between January 2015 and June 2021 by two independent researchers. Search terms related to deep learning, digital pathology, omics, and multimodal fusion were combined.

RESULTS

We identified a total of 11 studies meeting the inclusion criteria, namely studies that used convolutional neural networks for haematoxylin and eosin image analysis of patients with cancer in combination with integrated omics data. Publications were categorised according to their endpoints: 7 studies focused on survival analysis and 4 studies on prediction of cancer subtypes, malignancy or microsatellite instability with spatial analysis.

CONCLUSIONS

Image-based classifiers already show high performances in prognostic and predictive cancer diagnostics. The integration of omics data led to improved performance in all studies described here. However, these are very early studies that still require external validation to demonstrate their generalisability and robustness. Further and more comprehensive studies with larger sample sizes are needed to evaluate performance and determine clinical benefits.