CoADS: Cross attention based dual-space graph network for survival prediction of lung cancer using whole slide images

Multi-positive contrastive learning-based cross-attention model for T cell receptor-antigen binding prediction

BACKGROUND AND OBJECTIVE

T cells play a vital role in the immune system by recognizing and eliminating infected or cancerous cells, thus driving adaptive immune responses. Their activation is triggered by the binding of T cell receptors (TCRs) to epitopes presented on Major Histocompatibility Complex (MHC) molecules. However, experimentally identifying antigens that could be recognizable by T cells and possess immunogenic properties is resource-intensive, with most candidates proving non-immunogenic, underscoring the need for computational tools to predict peptide-MHC (pMHC) and TCR binding. Despite extensive efforts, accurately predicting TCR-antigen binding pairs remains challenging due to the vast diversity of TCRs.

METHODS

In this study, we propose a Contrastive Cross-attention model for TCR (ConTCR) and pMHC binding prediction. Firstly, the pMHC and TCR sequences are transformed into high-level embedding by pretrained encoders as feature representations. Then, we employ the multi-modal cross-attention to combine the features between pMHC sequences and TCR sequences. Next, based on the contrastive learning strategy, we pretrained the backbone of ConTCR to boost the model's feature extraction ability for pMHC and TCR sequences. Finally, the model is fine-tuned for classification between positive and negative samples.

RESULTS

Based on this advanced strategy, our proposed model could effectively capture the critical information on TCR-pMHC interactions, and the model is visualized by the attention score heatmap for interpretability. ConTCR demonstrates strong generalization in predicting binding specificity for unseen epitopes and diverse TCR repertoires. On independent non-zero-shot test sets, the model achieved AUC-ROC scores of 0.849 and 0.950; on zero-shot test sets, it obtained AUC-ROC scores of 0.830 and 0.938.

CONCLUSION

Our framework offers a promising solution for improving pMHC-TCR binding prediction and model interpretability. By leveraging the ConTCR model and pMHC-TCR features, we achieve more precise precision than recently advanced models. Overall, ConTCR is a robust tool for predicting pMHC-TCR binding and holds significant promise to advance TCR-based immunotherapies as a valuable artificial intelligence tool. The codes and data used in this study are available at this website.

Dual-stream multi-dependency graph neural network enables precise cancer survival analysis

Histopathology image-based survival prediction aims to provide a precise assessment of cancer prognosis and can inform personalized treatment decision-making in order to improve patient outcomes. However, existing methods cannot automatically model the complex correlations between numerous morphologically diverse patches in each whole slide image (WSI), thereby preventing them from achieving a more profound understanding and inference of the patient status. To address this, here we propose a novel deep learning framework, termed dual-stream multi-dependency graph neural network (DM-GNN), to enable precise cancer patient survival analysis. Specifically, DM-GNN is structured with the feature updating and global analysis branches to better model each WSI as two graphs based on morphological affinity and global co-activating dependencies. As these two dependencies depict each WSI from distinct but complementary perspectives, the two designed branches of DM-GNN can jointly achieve the multi-view modeling of complex correlations between the patches. Moreover, DM-GNN is also capable of boosting the utilization of dependency information during graph construction by introducing the affinity-guided attention recalibration module as the readout function. This novel module offers increased robustness against feature perturbation, thereby ensuring more reliable and stable predictions. Extensive benchmarking experiments on five TCGA datasets demonstrate that DM-GNN outperforms other state-of-the-art methods and offers interpretable prediction insights based on the morphological depiction of high-attention patches. Overall, DM-GNN represents a powerful and auxiliary tool for personalized cancer prognosis from histopathology images and has great potential to assist clinicians in making personalized treatment decisions and improving patient outcomes.

Transformaer-based model for lung adenocarcinoma subtypes

BACKGROUND

Lung cancer has the highest morbidity and mortality rate among all types of cancer. Histological subtypes serve as crucial markers for the development of lung cancer and possess significant clinical values for cancer diagnosis, prognosis, and prediction of treatment responses. However, existing studies only dichotomize normal and cancerous tissues, failing to capture the unique characteristics of tissue sections and cancer types.

PURPOSE

Therefore, we have pioneered the classification of lung adenocarcinoma (LAD) cancer tissues into five subtypes (acinar, lepidic, micropapillary, papillary, and solid) based on section data in whole-slide image sections. In addition, a novel model called HybridNet was designed to improve the classification performance.

METHODS

HybridNet primarily consists of two interactive streams: a Transformer and a convolutional neural network (CNN). The Transformer stream captures rich global representations using a self-attention mechanism, while the CNN stream extracts local semantic features to optimize image details. Specifically, during the dual-stream parallelism, the feature maps of the Transformer stream as weights are weighted and summed with those of the CNN stream backbone; at the end of the parallelism, the respective final features are concatenated to obtain more discriminative semantic information.

RESULTS

Experimental results on a private dataset of LAD showed that HybridNet achieved 95.12% classification accuracy, and the accuracy of five histological subtypes (acinar, lepidic, micropapillary, papillary, and solid) reached 94.5%, 97.1%, 94%, 91%, and 99% respectively; the experimental results on the public BreakHis dataset show that HybridNet achieves the best results in three evaluation metrics: accuracy, recall and F1-score, with 92.40%, 90.63%, and 91.43%, respectively.

CONCLUSIONS

The process of classifying LAD into five subtypes assists pathologists in selecting appropriate treatments and enables them to predict tumor mutation burden (TMB) and analyze the spatial distribution of immune checkpoint proteins based on this and other clinical data. In addition, the proposed HybridNet fuses CNN and Transformer information several times and is able to improve the accuracy of subtype classification, and also shows satisfactory performance on public datasets with some generalization ability.

Masked hypergraph learning for weakly supervised histopathology whole slide image classification

BACKGROUND AND OBJECTIVES

Graph neural network (GNN) has been extensively used in histopathology whole slide image (WSI) analysis due to the efficiency and flexibility in modelling relationships among entities. However, most existing GNN-based WSI analysis methods only consider the pairwise correlation of patches from one single perspective (e.g. spatial affinity or embedding similarity) yet ignore the intrinsic non-pairwise relationships present in gigapixel WSI, which are likely to contribute to feature learning and downstream tasks. The objective of this study is therefore to explore the non-pairwise relationships in histopathology WSI and exploit them to guide the learning of slide-level representations for better classification performance.

METHODS

In this paper, we propose a novel Masked HyperGraph Learning (MaskHGL) framework for weakly supervised histopathology WSI classification. Compared with most GNN-based WSI classification methods, MaskHGL exploits the non-pairwise correlations between patches with hypergraph and global message passing conducted by hypergraph convolution. Concretely, multi-perspective hypergraphs are first built for each WSI, then hypergraph attention is introduced into the jointed hypergraph to propagate the non-pairwise relationships and thus yield more discriminative node representation. More importantly, a masked hypergraph reconstruction module is devised to guide the hypergraph learning which can generate more powerful robustness and generalization than the method only using hypergraph modelling. Additionally, a self-attention-based node aggregator is also applied to explore the global correlation of patches in WSI and produce the slide-level representation for classification.

RESULTS

The proposed method is evaluated on two public TCGA benchmark datasets and one in-house dataset. On the public TCGA-LUNG (1494 WSIs) and TCGA-EGFR (696 WSIs) test set, the area under receiver operating characteristic (ROC) curve (AUC) were 0.9752±0.0024 and 0.7421±0.0380, respectively. On the USTC-EGFR (754 WSIs) dataset, MaskHGL achieved significantly better performance with an AUC of 0.8745±0.0100, which surpassed the second-best state-of-the-art method SlideGraph+ 2.64%.

CONCLUSIONS

MaskHGL shows a great improvement, brought by considering the intrinsic non-pairwise relationships within WSI, in multiple downstream WSI classification tasks. In particular, the designed masked hypergraph reconstruction module promisingly alleviates the data scarcity and greatly enhances the robustness and classification ability of our MaskHGL. Notably, it has shown great potential in cancer subtyping and fine-grained lung cancer gene mutation prediction from hematoxylin and eosin (H&E) stained WSIs.

Global bibliometric mapping of the research trends in artificial intelligence-based digital pathology for lung cancer over the past two decades

Background and Objective

The rapid development of computer technology has led to a revolutionary transformation in artificial intelligence (AI)-assisted healthcare. The integration of whole-slide imaging technology with AI algorithms has facilitated the development of digital pathology for lung cancer (LC). However, there is a lack of comprehensive scientometric analysis in this field.

Methods

A bibliometric analysis was conducted on 197 publications related to digital pathology in LC from 502 institutions across 39 countries, published in 97 academic journals in the Web of Science Core Collection between 2004 and 2023.

Results

Our analysis has identified the United States and China as the primary research nations in the field of digital pathology in LC. However, it is important to note that the current research primarily consists of independent studies among countries, emphasizing the necessity of strengthening academic collaboration and data sharing between nations. The current focus and challenge of research related to digital pathology in LC lie in enhancing the accuracy of classification and prediction through improved deep learning algorithms. The integration of multi-omics studies presents a promising future research direction. Additionally, researchers are increasingly exploring the application of digital pathology in immunotherapy for LC patients.

Conclusions

In conclusion, this study provides a comprehensive knowledge framework for digital pathology in LC, highlighting research trends, hotspots, and gaps in this field. It also provides a theoretical basis for the application of AI in clinical decision-making for LC patients.