Deep learning with whole slide images can improve the prognostic risk stratification with stage III colorectal cancer

A multi-view contrastive learning and semi-supervised self-distillation framework for early recurrence prediction in ovarian cancer

OBJECTIVE

This study presents a novel framework that integrates contrastive learning and knowledge distillation to improve early ovarian cancer (OC) recurrence prediction, addressing the challenges posed by limited labeled data and tumor heterogeneity.

METHODS

The research utilized CT imaging data from 585 OC patients, including 142 cases with complete follow-up information and 125 cases with unknown recurrence status. To pre-train the teacher network, 318 unlabeled images were sourced from public datasets (TCGA-OV and PLAGH-202-OC). Multi-view contrastive learning (MVCL) was employed to generate multi-view 2D tumor slices, enhancing the teacher network's ability to extract features from complex, heterogeneous tumors with high intra-class variability. Building on this foundation, the proposed semi-supervised multi-task self-distillation (Semi-MTSD) framework integrated OC subtyping as an auxiliary task using multi-task learning (MTL). This approach allowed the co-training of a student network for recurrence prediction, leveraging both labeled and unlabeled data to improve predictive performance in data-limited settings. The student network's performance was assessed using preoperative CT images with known recurrence outcomes. Evaluation metrics included area under the receiver operating characteristic curve (AUC), accuracy (ACC), sensitivity (SEN), specificity (SPE), F1 score, floating-point operations (FLOPs), parameter count, training time, inference time, and mean corruption error (mCE).

RESULTS

The proposed framework achieved an ACC of 0.862, an AUC of 0.916, a SPE of 0.895, and an F1 score of 0.831, surpassing existing methods for OC recurrence prediction. Comparative and ablation studies validated the model's robustness, particularly in scenarios characterized by data scarcity and tumor heterogeneity.

CONCLUSION

The MVCL and Semi-MTSD framework demonstrates significant advancements in OC recurrence prediction, showcasing strong generalization capabilities in complex, data-constrained environments. This approach offers a promising pathway toward more personalized treatment strategies for OC patients.

Automatic ovarian follicle detection using object detection models

Ovaries are of paramount importance in reproduction as they produce female gametes through a complex developmental process known as folliculogenesis. In the prospect of better understanding the mechanisms of folliculogenesis and of developing novel pharmacological approaches to control it, it is important to accurately and quantitatively assess the later stages of ovarian folliculogenesis (i.e. the formation of antral follicles and corpus lutea). Manual counting from histological sections is commonly employed to determine the number of these follicular structures, however it is a laborious and error prone task. In this work, we show the benefits of deep learning models for counting antral follicles and corpus lutea in ovarian histology sections. Here, we use various backbone architectures to build two one-stage object detection models, i.e. YOLO and RetinaNet. We employ transfer learning, early stopping, and data augmentation approaches to improve the generalizability of the object detectors. Furthermore, we use sampling strategy to mitigate the foreground-foreground class imbalance and focal loss to reduce the imbalance between the foreground-background classes. Our models were trained and validated using a dataset containing only 1000 images. With RetinaNet, we achieved a mean average precision of 83% whereas with YOLO of 75% on the testing dataset. Our results demonstrate that deep learning methods are useful to speed up the follicle counting process and improve accuracy by correcting manual counting errors.

Whole slide image based prognosis prediction in rectal cancer using unsupervised artificial intelligence

BACKGROUND

Rectal cancer is a common cancer worldwide and lacks effective prognostic markers. The development of prognostic markers by computational pathology methods has attracted increasing attention. This paper aims to construct a prognostic signature from whole slide images for predicting progression-free survival (PFS) of rectal cancer through an unsupervised artificial intelligence algorithm.

METHODS

A total of 238 patients with rectal cancer from two datasets were collected for the development and validation of the prognostic signature. A tumor detection model was built by transfer learning. Then, on the basis of the tumor patches recognized by the tumor detection model, a convolutional autoencoder model was built for decoding the tumor patches into deep latent features. Next, on the basis of the deep latent features, the tumor patches were divided into different clusters. The cluster number and other hyperparameters were optimized by a nested cross-validation method. The percentage of each cluster from the patient's tumor patches, which is hereafter called PCF, was calculated for prognostic signature construction. The prognostic signature was constructed by Cox proportional hazard regression with L2 regularization. Finally, bioinformatic analysis was performed to explore the underlying biological mechanisms of the PCFs.

RESULTS

The accuracy of the tumor detection model in distinguishing tumor patches from non-tumor patches achieved 99.3%. The optimal cluster number was determined to be 9. Therfore, 9 PCFs were calculated to construct the prognostic signature. The prognostic signature achieved a concordance index of 0.701 in the validation cohort. The Kaplan-Meier survival curves showed the prognostic signature had good risk stratification ability. Through the bioinformatic analysis, several PCF-associated genes were identified. These genes were enriched in various gene ontology terms.

CONCLUSION

The developed prognostic signature can effectively predict PFS in patients with rectal cancer and exploration of the underlying biological mechanisms may help to promote its clinical translation.

From Pixels to Prognosis: A Survey on AI-Driven Cancer Patient Survival Prediction Using Digital Histology Images

Survival analysis is an integral part of medical statistics that is extensively utilized to establish prognostic indices for mortality or disease recurrence, assess treatment efficacy, and tailor effective treatment plans. The identification of prognostic biomarkers capable of predicting patient survival is a primary objective in the field of cancer research. With the recent integration of digital histology images into routine clinical practice, a plethora of Artificial Intelligence (AI)-based methods for digital pathology has emerged in scholarly literature, facilitating patient survival prediction. These methods have demonstrated remarkable proficiency in analyzing and interpreting whole slide images, yielding results comparable to those of expert pathologists. The complexity of AI-driven techniques is magnified by the distinctive characteristics of digital histology images, including their gigapixel size and diverse tissue appearances. Consequently, advanced patch-based methods are employed to effectively extract features that correlate with patient survival. These computational methods significantly enhance survival prediction accuracy and augment prognostic capabilities in cancer patients. The review discusses the methodologies employed in the literature, their performance metrics, ongoing challenges, and potential solutions for future advancements. This paper explains survival analysis and feature extraction methods for analyzing cancer patients. It also compiles essential acronyms related to cancer precision medicine. Furthermore, it is noteworthy that this is the inaugural review paper in the field. The target audience for this interdisciplinary review comprises AI practitioners, medical statisticians, and progressive oncologists who are enthusiastic about translating AI-driven solutions into clinical practice. We expect this comprehensive review article to guide future research directions in the field of cancer research.

ProDiv: Prototype-driven consistent pseudo-bag division for whole-slide image classification

BACKGROUND AND OBJECTIVE

Pathology image classification is one of the most essential auxiliary processes in cancer diagnosis. To overcome the problem of inadequate Whole-Slide Image (WSI) samples with weak labels, pseudo-bag-based multiple instance learning (MIL) methods have attracted wide attention in pathology image classification. In this type of method, the division scheme of pseudo-bags is usually a primary factor affecting classification performance. In order to improve the division of WSI pseudo-bags on existing random/clustering approaches, this paper proposes a new Prototype-driven Division (ProDiv) scheme for the pseudo-bag-based MIL classification framework on pathology images.

METHODS

This scheme first designs an attention-based method to generate a bag prototype for each slide. On this basis, it further groups WSI patch instances into a series of instance clusters according to the feature similarities between the prototype and patches. Finally, pseudo-bags are obtained by randomly combining the non-overlapping patch instances of different instance clusters. Moreover, the design scheme of our ProDiv considers practicality, and it could be smoothly assembled with almost all the MIL-based WSI classification methods in recent years.

RESULTS

Empirical results show that our ProDiv, when integrated with several existing methods, can deliver classification AUC improvements of up to 7.3% and 10.3%, respectively on two public WSI datasets.

CONCLUSIONS

ProDiv could almost always bring obvious performance improvements to compared MIL models on typical metrics, which suggests the effectiveness of our scheme. Experimental visualization also visually interprets the correctness of the proposed ProDiv.

SAGL: A self-attention-based graph learning framework for predicting survival of colorectal cancer patients

BACKGROUND AND OBJECTIVE

Colorectal cancer (CRC) is one of the most commonly diagnosed cancers worldwide. The accurate survival prediction for CRC patients plays a significant role in the formulation of treatment strategies. Recently, machine learning and deep learning approaches have been increasingly applied in cancer survival prediction. However, most existing methods inadequately represent and leverage the dependencies among features and fail to sufficiently mine and utilize the comorbidity patterns of CRC. To address these issues, we propose a self-attention-based graph learning (SAGL) framework to improve the postoperative cancer-specific survival prediction for CRC patients.

METHODS

We present a novel method for constructing dependency graph (DG) to reflect two types of dependencies including comorbidity-comorbidity dependencies and the dependencies between features related to patient characteristics and cancer treatments. This graph is subsequently refined by a disease comorbidity network, which offers a holistic view of comorbidity patterns of CRC. A DG-guided self-attention mechanism is proposed to unearth novel dependencies beyond what DG offers, thus augmenting CRC survival prediction. Finally, each patient will be represented, and these representations will be used for survival prediction.

RESULTS

The experimental results show that SAGL outperforms state-of-the-art methods on a real-world dataset, with the receiver operating characteristic curve for 3- and 5-year survival prediction achieving 0.849±0.002 and 0.895±0.005, respectively. In addition, the comparison results with different graph neural network-based variants demonstrate the advantages of our DG-guided self-attention graph learning framework.

CONCLUSIONS

Our study reveals that the potential of the DG-guided self-attention in optimizing feature graph learning which can improve the performance of CRC survival prediction.

Diagnosing and grading gastric atrophy and intestinal metaplasia using semi-supervised deep learning on pathological images: development and validation study

OBJECTIVE

Patients with gastric atrophy and intestinal metaplasia (IM) were at risk for gastric cancer, necessitating an accurate risk assessment. We aimed to establish and validate a diagnostic approach for gastric biopsy specimens using deep learning and OLGA/OLGIM for individual gastric cancer risk classification.

METHODS

In this study, we prospectively enrolled 545 patients suspected of atrophic gastritis during endoscopy from 13 tertiary hospitals between December 22, 2017, to September 25, 2020, with a total of 2725 whole-slide images (WSIs). Patients were randomly divided into a training set (n = 349), an internal validation set (n = 87), and an external validation set (n = 109). Sixty patients from the external validation set were randomly selected and divided into two groups for an observer study, one with the assistance of algorithm results and the other without. We proposed a semi-supervised deep learning algorithm to diagnose and grade IM and atrophy, and we compared it with the assessments of 10 pathologists. The model's performance was evaluated based on the area under the curve (AUC), sensitivity, specificity, and weighted kappa value.

RESULTS

The algorithm, named GasMIL, was established and demonstrated encouraging performance in diagnosing IM (AUC 0.884, 95% CI 0.862-0.902) and atrophy (AUC 0.877, 95% CI 0.855-0.897) in the external test set. In the observer study, GasMIL achieved an 80% sensitivity, 85% specificity, a weighted kappa value of 0.61, and an AUC of 0.953, surpassing the performance of all ten pathologists in diagnosing atrophy. Among the 10 pathologists, GasMIL's AUC ranked second in OLGA (0.729, 95% CI 0.625-0.833) and fifth in OLGIM (0.792, 95% CI 0.688-0.896). With the assistance of GasMIL, pathologists demonstrated improved AUC (p = 0.013), sensitivity (p = 0.014), and weighted kappa (p = 0.016) in diagnosing IM, and improved specificity (p = 0.007) in diagnosing atrophy compared to pathologists working alone.

CONCLUSION

GasMIL shows the best overall performance in diagnosing IM and atrophy when compared to pathologists, significantly enhancing their diagnostic capabilities.

Masked autoencoders with handcrafted feature predictions: Transformer for weakly supervised esophageal cancer classification

BACKGROUND AND OBJECTIVE

Esophageal cancer is a serious disease with a high prevalence in Eastern Asia. Histopathology tissue analysis stands as the gold standard in diagnosing esophageal cancer. In recent years, there has been a shift towards digitizing histopathological images into whole slide images (WSIs), progressively integrating them into cancer diagnostics. However, the gigapixel sizes of WSIs present significant storage and processing challenges, and they often lack localized annotations. To address this issue, multi-instance learning (MIL) has been introduced for WSI classification, utilizing weakly supervised learning for diagnosis analysis. By applying the principles of MIL to WSI analysis, it is possible to reduce the workload of pathologists by facilitating the generation of localized annotations. Nevertheless, the approach's effectiveness is hindered by the traditional simple aggregation operation and the domain shift resulting from the prevalent use of convolutional feature extractors pretrained on ImageNet.

METHODS

We propose a MIL-based framework for WSI analysis and cancer classification. Concurrently, we introduce employing self-supervised learning, which obviates the need for manual annotation and demonstrates versatility in various tasks, to pretrain feature extractors. This method enhances the extraction of representative features from esophageal WSI for MIL, ensuring more robust and accurate performance.

RESULTS

We build a comprehensive dataset of whole esophageal slide images and conduct extensive experiments utilizing this dataset. The performance on our dataset demonstrates the efficiency of our proposed MIL framework and the pretraining process, with our framework outperforming existing methods, achieving an accuracy of 93.07% and AUC (area under the curve) of 95.31%.

CONCLUSION

This work proposes an effective MIL method to classify WSI of esophageal cancer. The promising results indicate that our cancer classification framework holds great potential in promoting the automatic whole esophageal slide image analysis.

Artificial Intelligence Applications in the Treatment of Colorectal Cancer: A Narrative Review

Colorectal cancer is the third most prevalent cancer worldwide, and its treatment has been a demanding clinical problem. Beyond traditional surgical therapy and chemotherapy, newly revealed molecular mechanisms diversify therapeutic approaches for colorectal cancer. However, the selection of personalized treatment among multiple treatment options has become another challenge in the era of precision medicine. Artificial intelligence has recently been increasingly investigated in the treatment of colorectal cancer. This narrative review mainly discusses the applications of artificial intelligence in the treatment of colorectal cancer patients. A comprehensive literature search was conducted in MEDLINE, EMBASE, and Web of Science to identify relevant papers, resulting in 49 articles being included. The results showed that, based on different categories of data, artificial intelligence can predict treatment outcomes and essential guidance information of traditional and novel therapies, thus enabling individualized treatment strategy selection for colorectal cancer patients. Some frequently implemented machine learning algorithms and deep learning frameworks have also been employed for long-term prognosis prediction in patients with colorectal cancer. Overall, artificial intelligence shows encouraging results in treatment strategy selection and prognosis evaluation for colorectal cancer patients.

Tumor Mutation Burden-Related Histopathologic Features for Predicting Overall Survival in Gliomas Using Graph Deep Learning

Tumor mutation burden (TMB) is a potential biomarker for evaluating the prognosis and response to immune checkpoint inhibitors, but its costly and time-consuming method of measurement limits its widespread application. This study aimed to identify the TMB-related histopathologic features from hematoxylin and eosin slides and explore their prognostic value in gliomas. TMB-related features were detected using a graph convolutional neural network from whole-slide images of patients from The Cancer Genome Atlas data set (619 patients), and the correlation between features and TMB was evaluated in an external validation set (237 patients). TMB-related features were used for predicting overall survival (OS) of patients to investigate whether these features have potential for prognostic prediction. Moreover, biological pathways underlying the prognostic value of the features were further explored. Histopathologic features derived from whole-slide images were significantly associated with patient TMB (P = 0.007 in the external validation set). TMB-related features showed excellent performance for OS prediction, and patients with lower-grade gliomas could be further stratified into different risk groups according to the features (P = 0.00013; hazard ratio, 4.004). Pathways involved in the cell cycle and execution of immune response were enriched in patients with higher OS risk. The TMB-related features could be used to estimate TMB and aid in prognostic risk stratification of patients with glioma with dysregulated biological pathways.

Improving prognosis and assessing adjuvant chemotherapy benefit in locally advanced rectal cancer with deep learning for MRI: A retrospective, multi-cohort study

PURPOSE

Adjuvant therapy is recommended to minimize the risk of distant metastasis (DM) and local recurrence (LR) in patients with locally advanced rectal cancer (LARC). However, its role is controversial. We aimed to develop a pretreatment MRI-based deep learning model to predict LR, DM, and overall survival (OS) over 5 years after surgery and to identify patients benefitting from adjuvant chemotherapy (AC).

MATERIALS AND METHODS

The multi-survival tasks network (MuST) model was developed in a primary cohort (n = 308) and validated using two external cohorts (n = 247, 245). An AC decision tree integrating the MuST-DM score, perineural invasion (PNI), and preoperative carbohydrate antigen 19-9 (CA19-9) was constructed to assess chemotherapy benefits and aid personalized treatment of patients. We also quantified the prognostic improvement of the decision tree.

RESULTS

The MuST network demonstrated high prognostic accuracy in the primary and two external cohorts for the prediction of three different survival tasks. Within the stratified analysis and decision tree, patients with CA19-9 levels > 37 U/mL and high MuST-DM scores exhibited favorable chemotherapy efficacy. Similar results were observed in PNI-positive patients with low MuST-DM scores. PNI-negative patients with low MuST-DM scores exhibited poor chemotherapy efficacy. Based on the decision tree, 14 additional patients benefiting from AC and 391 patients who received over-treatment were identified in this retrospective study.

CONCLUSION

The MuST model accurately and non-invasively predicted OS, DM, and LR. A specific and direct tool linking chemotherapy decisions and benefit quantification has also been provided.

Applications of discriminative and deep learning feature extraction methods for whole slide image analysis: A survey

Digital pathology technologies, including whole slide imaging (WSI), have significantly improved modern clinical practices by facilitating storing, viewing, processing, and sharing digital scans of tissue glass slides. Researchers have proposed various artificial intelligence (AI) solutions for digital pathology applications, such as automated image analysis, to extract diagnostic information from WSI for improving pathology productivity, accuracy, and reproducibility. Feature extraction methods play a crucial role in transforming raw image data into meaningful representations for analysis, facilitating the characterization of tissue structures, cellular properties, and pathological patterns. These features have diverse applications in several digital pathology applications, such as cancer prognosis and diagnosis. Deep learning-based feature extraction methods have emerged as a promising approach to accurately represent WSI contents and have demonstrated superior performance in histology-related tasks. In this survey, we provide a comprehensive overview of feature extraction methods, including both manual and deep learning-based techniques, for the analysis of WSIs. We review relevant literature, analyze the discriminative and geometric features of WSIs (i.e., features suited to support the diagnostic process and extracted by "engineered" methods as opposed to AI), and explore predictive modeling techniques using AI and deep learning. This survey examines the advances, challenges, and opportunities in this rapidly evolving field, emphasizing the potential for accurate diagnosis, prognosis, and decision-making in digital pathology.

Recent Advancements in Deep Learning Using Whole Slide Imaging for Cancer Prognosis

This review furnishes an exhaustive analysis of the latest advancements in deep learning techniques applied to whole slide images (WSIs) in the context of cancer prognosis, focusing specifically on publications from 2019 through 2023. The swiftly maturing field of deep learning, in combination with the burgeoning availability of WSIs, manifests significant potential in revolutionizing the predictive modeling of cancer prognosis. In light of the swift evolution and profound complexity of the field, it is essential to systematically review contemporary methodologies and critically appraise their ramifications. This review elucidates the prevailing landscape of this intersection, cataloging major developments, evaluating their strengths and weaknesses, and providing discerning insights into prospective directions. In this paper, a comprehensive overview of the field aims to be presented, which can serve as a critical resource for researchers and clinicians, ultimately enhancing the quality of cancer care outcomes. This review's findings accentuate the need for ongoing scrutiny of recent studies in this rapidly progressing field to discern patterns, understand breakthroughs, and navigate future research trajectories.

CoADS: Cross attention based dual-space graph network for survival prediction of lung cancer using whole slide images

BACKGROUND AND OBJECTIVE

Accurate overall survival (OS) prediction for lung cancer patients is of great significance, which can help classify patients into different risk groups to benefit from personalized treatment. Histopathology slides are considered the gold standard for cancer diagnosis and prognosis, and many algorithms have been proposed to predict the OS risk. Most methods rely on selecting key patches or morphological phenotypes from whole slide images (WSIs). However, OS prediction using the existing methods exhibits limited accuracy and remains challenging.

METHODS

In this paper, we propose a novel cross-attention-based dual-space graph convolutional neural network model (CoADS). To facilitate the improvement of survival prediction, we fully take into account the heterogeneity of tumor sections from different perspectives. CoADS utilizes the information from both physical and latent spaces. With the guidance of cross-attention, both the spatial proximity in physical space and the feature similarity in latent space between different patches from WSIs are integrated effectively.

RESULTS

We evaluated our approach on two large lung cancer datasets of 1044 patients. The extensive experimental results demonstrated that the proposed model outperforms state-of-the-art methods with the highest concordance index.

CONCLUSIONS

The qualitative and quantitative results show that the proposed method is more powerful for identifying the pathology features associated with prognosis. Furthermore, the proposed framework can be extended to other pathological images for predicting OS or other prognosis indicators, and thus delivering individualized treatment.

Development of Prognostic Biomarkers by TMB-Guided WSI Analysis: A Two-Step Approach

The rapid development of computational pathology has brought new opportunities for prognosis prediction using histopathological images. However, the existing deep learning frameworks lack exploration of the relationship between images and other prognostic information, resulting in poor interpretability. Tumor mutation burden (TMB) is a promising biomarker for predicting the survival outcomes of cancer patients, but its measurement is costly. Its heterogeneity may be reflected in histopathological images. Here, we report a two-step framework for prognostic prediction using whole-slide images (WSIs). First, the framework adopts a deep residual network to encode the phenotype of WSIs and classifies patient-level TMB by the deep features after aggregation and dimensionality reduction. Then, the patients' prognosis is stratified by the TMB-related information obtained during the classification model development. Deep learning feature extraction and TMB classification model construction are performed on an in-house dataset of 295 Haematoxylin & Eosin stained WSIs of clear cell renal cell carcinoma (ccRCC). The development and evaluation of prognostic biomarkers are performed on The Cancer Genome Atlas-Kidney ccRCC (TCGA-KIRC) project with 304 WSIs. Our framework achieves good performance for TMB classification with an area under the receiver operating characteristic curve (AUC) of 0.813 on the validation set. Through survival analysis, our proposed prognostic biomarkers can achieve significant stratification of patients' overall survival (P $< $ 0.05) and outperform the original TMB signature in risk stratification of patients with advanced disease. The results indicate the feasibility of mining TMB-related information from WSI to achieve stepwise prognosis prediction.

GraphLSurv: A scalable survival prediction network with adaptive and sparse structure learning for histopathological whole-slide images

BACKGROUND AND OBJECTIVE

Predicting patients' survival from gigapixel Whole-Slide Images (WSIs) has always been a challenging task. To learn effective WSI representations for survival prediction, existing deep learning methods have explored utilizing graphs to describe the complex structure inner WSIs, where graph node is respective to WSI patch. However, these graphs are often densely-connected or static, leading to some redundant or missing patch correlations. Moreover, these methods cannot be directly scaled to the very-large WSI with more than 10,000 patches. To address these, this paper proposes a scalable graph convolution network, GraphLSurv, which can efficiently learn adaptive and sparse structures to better characterize WSIs for survival prediction.

METHODS

GraphLSurv has three highlights in methodology: (1) it generates adaptive and sparse structures for patches so that latent patch correlations could be captured and adjusted dynamically according to prediction tasks; (2) based on the generated structure and a given graph, GraphLSurv further aggregates local microenvironmental cues into a non-local embedding using the proposed hybrid message passing network; (3) to make this network suitable for very large-scale graphs, it adopts an anchor-based technique to reduce theorical computation complexity.

RESULTS

The experiments on 2268 WSIs show that GraphLSurv achieves a concordance-index of 0.66132 and 0.68348, with an improvement of 3.79% and 3.41% compared to existing methods, on NLST and TCGA-BRCA, respectively.

CONCLUSIONS

GraphLSurv could often perform better than previous methods, which suggests that GraphLSurv could provide an important and effective means for WSI survival prediction. Moreover, this work empirically shows that adaptive and sparse structures could be more suitable than static or dense ones for modeling WSIs.

The Applications of Artificial Intelligence in Digestive System Neoplasms: A Review

Importance

Digestive system neoplasms (DSNs) are the leading cause of cancer-related mortality with a 5-year survival rate of less than 20%. Subjective evaluation of medical images including endoscopic images, whole slide images, computed tomography images, and magnetic resonance images plays a vital role in the clinical practice of DSNs, but with limited performance and increased workload of radiologists or pathologists. The application of artificial intelligence (AI) in medical image analysis holds promise to augment the visual interpretation of medical images, which could not only automate the complicated evaluation process but also convert medical images into quantitative imaging features that associated with tumor heterogeneity.

Highlights

We briefly introduce the methodology of AI for medical image analysis and then review its clinical applications including clinical auxiliary diagnosis, assessment of treatment response, and prognosis prediction on 4 typical DSNs including esophageal cancer, gastric cancer, colorectal cancer, and hepatocellular carcinoma.

Conclusion

AI technology has great potential in supporting the clinical diagnosis and treatment decision-making of DSNs. Several technical issues should be overcome before its application into clinical practice of DSNs.

Interpretable classification of pathology whole-slide images using attention based context-aware graph convolutional neural network

BACKGROUND AND OBJECTIVE

Whole slide image (WSI) classification and lesion localization within giga-pixel slide are challenging tasks in computational pathology that requires context-aware representations of histological features to adequately infer nidus. The existing weakly supervised learning methods mainly treat different locations in the slide as independent regions and cannot learn potential nonlinear interactions between instances based on i.i.d assumption, resulting in the model unable to effectively utilize context-ware information to predict the labels of WSIs and locate the region of interest (ROI).

METHODS

Here, we propose an interpretable classification model named bidirectional Attention-based Multiple Instance Learning Graph Convolutional Network (ABMIL-GCN), which hierarchically aggregates context-aware features of instances into a global representation in a topology fashion to predict the slide labels and localize the region of lymph node metastasis in WSIs.

RESULTS

We verified the superiority of this method on the Camelyon16 dataset, and the results show that the average predicted ACC and AUC of the proposed model after flooding optimization can reach 90.89% and 0.9149, respectively. The average accuracy and ACC score are improved by more than 7% and 4% compared with the existing state-of-the-art algorithms.

CONCLUSIONS

The results demonstrate that context-aware GCN outperforms existing weakly supervised learning methods by introducing spatial correlations between the neighbor image patches, which also addresses the 'accuracy-interpretability trade-off' problem. The framework provides a novel paradigm for the clinical application of computer-aided diagnosis and intelligent systems.

A clinical decision support system optimising adjuvant chemotherapy for colorectal cancers by integrating deep learning and pathological staging markers: a development and validation study

BACKGROUND

The DoMore-v1-CRC marker was recently developed using deep learning and conventional haematoxylin and eosin-stained tissue sections, and was observed to outperform established molecular and morphological markers of patient outcome after primary colorectal cancer resection. The aim of the present study was to develop a clinical decision support system based on DoMore-v1-CRC and pathological staging markers to facilitate individualised selection of adjuvant treatment.

METHODS

We estimated cancer-specific survival in subgroups formed by pathological tumour stage (pT<4 or pT4), pathological nodal stage (pN0, pN1, or pN2), number of lymph nodes sampled (≤12 or >12) if not pN2, and DoMore-v1-CRC classification (good, uncertain, or poor prognosis) in 997 patients with stage II or III colorectal cancer considered to have no residual tumour (R0) from two community-based cohorts in Norway and the UK, and used these data to define three risk groups. An external cohort of 1075 patients with stage II or III R0 colorectal cancer from the QUASAR 2 trial was used for validation; these patients were treated with single-agent capecitabine. The proposed risk stratification system was evaluated using Cox regression analysis. We similarly evaluated a risk stratification system intended to reflect current guidelines and clinical practice. The primary outcome was cancer-specific survival.

FINDINGS

The new risk stratification system provided a hazard ratio of 10·71 (95% CI 6·39-17·93; p<0·0001) for high-risk versus low-risk patients and 3·06 (1·73-5·42; p=0·0001) for intermediate versus low risk in the primary analysis of the validation cohort. Estimated 3-year cancer-specific survival was 97·2% (95% CI 95·1-98·4; n=445 [41%]) for the low-risk group, 94·8% (91·7-96·7; n=339 [32%]) for the intermediate-risk group, and 77·6% (72·1-82·1; n=291 [27%]) for the high-risk group. The guideline-based risk grouping was observed to be less prognostic and informative (the low-risk group comprised only 142 [13%] of the 1075 patients).

INTERPRETATION

Integrating DoMore-v1-CRC and pathological staging markers provided a clinical decision support system that risk stratifies more accurately than its constituent elements, and identifies substantially more patients with stage II and III colorectal cancer with similarly good prognosis as the low-risk group in current guidelines. Avoiding adjuvant chemotherapy in these patients might be safe, and could reduce morbidity, mortality, and treatment costs.

FUNDING

The Research Council of Norway.

From Dukes-MAC Staging System to Molecular Classification: Evolving Concepts in Colorectal Cancer

This historical review aimed to summarize the main changes that colorectal carcinoma (CRC) staging systems suffered over time, starting from the creation of the classical Duke's classification, modified Astler-Coller staging, internationally used TNM (T-primary tumor, N-regional lymph nodes' status, M-distant metastases) staging system, and ending with molecular classifications and epithelial-mesenchymal transition (EMT) concept. Besides currently used staging parameters, this paper briefly presents the author's contribution in creating an immunohistochemical (IHC)-based molecular classification of CRC. It refers to the identification of three molecular groups of CRCs (epithelial, mesenchymal and hybrid) based on the IHC markers E-cadherin, β-catenin, maspin, and vimentin. Maspin is a novel IHC antibody helpful for tumor budding assessment, which role depends on its subcellular localization (cytoplasm vs. nuclei). The long road of updating the staging criteria for CRC has not come to an end. The newest prognostic biomarkers, aimed to be included in the molecular classifications, exert predictive roles, and become more and more important for targeted therapy decisions.