Inferring spatial transcriptomics markers from whole slide images to characterize metastasis-related spatial heterogeneity of colorectal tumors: A pilot study

Histology-Based Virtual RNA Inference Identifies Pathways Associated with Metastasis Risk in Colorectal Cancer

Colorectal cancer (CRC) remains a major health concern, with over 150,000 new diagnoses and more than 50,000 deaths annually in the United States, underscoring an urgent need for improved screening, prognostication, disease management, and therapeutic approaches. The tumor microenvironment (TME)-comprising cancerous and immune cells interacting within the tumor's spatial architecture-plays a critical role in disease progression and treatment outcomes, reinforcing its importance as a prognostic marker for metastasis and recurrence risk. However, traditional methods for TME characterization, such as bulk transcriptomics and multiplex protein assays, lack sufficient spatial resolution. Although spatial transcriptomics (ST) allows for the high-resolution mapping of whole transcriptomes at near-cellular resolution, current ST technologies (e.g., Visium, Xenium) are limited by high costs, low throughput, and issues with reproducibility, preventing their widespread application in large-scale molecular epidemiology studies. In this study, we refined and implemented Virtual RNA Inference (VRI) to derive ST-level molecular information directly from hematoxylin and eosin (H&E)-stained tissue images. Our VRI models were trained on the largest matched CRC ST dataset to date, comprising 45 patients and more than 300,000 Visium spots from primary tumors. Using state-of-the-art architectures (UNI, ResNet-50, ViT, and VMamba), we achieved a median Spearman's correlation coefficient of 0.546 between predicted and measured spot-level expression. As validation, VRI-derived gene signatures linked to specific tissue regions (tumor, interface, submucosa, stroma, serosa, muscularis, inflammation) showed strong concordance with signatures generated via direct ST, and VRI performed accurately in estimating cell-type proportions spatially from H&E slides. In an expanded CRC cohort controlling for tumor invasiveness and clinical factors, we further identified VRI-derived gene signatures significantly associated with key prognostic outcomes, including metastasis status. Although certain tumor-related pathways are not fully captured by histology alone, our findings highlight the ability of VRI to infer a wide range of "histology-associated" biological pathways at near-cellular resolution without requiring ST profiling. Future efforts will extend this framework to expand TME phenotyping from standard H&E tissue images, with the potential to accelerate translational CRC research at scale.

Graph neural networks in histopathology: Emerging trends and future directions

Histopathological analysis of whole slide images (WSIs) has seen a surge in the utilization of deep learning methods, particularly Convolutional Neural Networks (CNNs). However, CNNs often fail to capture the intricate spatial dependencies inherent in WSIs. Graph Neural Networks (GNNs) present a promising alternative, adept at directly modeling pairwise interactions and effectively discerning the topological tissue and cellular structures within WSIs. Recognizing the pressing need for deep learning techniques that harness the topological structure of WSIs, the application of GNNs in histopathology has experienced rapid growth. In this comprehensive review, we survey GNNs in histopathology, discuss their applications, and explore emerging trends that pave the way for future advancements in the field. We begin by elucidating the fundamentals of GNNs and their potential applications in histopathology. Leveraging quantitative literature analysis, we explore four emerging trends: Hierarchical GNNs, Adaptive Graph Structure Learning, Multimodal GNNs, and Higher-order GNNs. Through an in-depth exploration of these trends, we offer insights into the evolving landscape of GNNs in histopathological analysis. Based on our findings, we propose future directions to propel the field forward. Our analysis serves to guide researchers and practitioners towards innovative approaches and methodologies, fostering advancements in histopathological analysis through the lens of graph neural networks.

Integrative co-registration of elemental imaging and histopathology for enhanced spatial multimodal analysis of tissue sections through TRACE

Summary

Elemental imaging provides detailed profiling of metal bioaccumulation, offering more precision than bulk analysis by targeting specific tissue areas. However, accurately identifying comparable tissue regions from elemental maps is challenging, requiring the integration of hematoxylin and eosin (H&E) slides for effective comparison. Facilitating the streamlined co-registration of whole slide images (WSI) and elemental maps, TRACE enhances the analysis of tissue regions and elemental abundance in various pathological conditions. Through an interactive containerized web application, TRACE features real-time annotation editing, advanced statistical tools, and data export, supporting comprehensive spatial analysis. Notably, it allows for comparison of elemental abundances across annotated tissue structures and enables integration with other spatial data types through WSI co-registration.

Availability and implementation

Available on the following platforms-GitHub: jlevy44/trace_app, PyPI: trace_app, Docker: joshualevy44/trace_app, Singularity: docker://joshualevy44/trace_app.

Identification of novel diagnostic biomarkers associated with liver metastasis in colon adenocarcinoma by machine learning

BACKGROUND

Liver metastasis is one of the primary causes of poor prognosis in colon adenocarcinoma (COAD) patients, but there are few studies on its biomarkers.

METHODS

The Cancer Genome Atlas (TCGA)-COAD, GSE41258, and GSE49355 datasets were acquired from the public database. Differentially expressed genes (DEGs) between liver metastasis and primary tumor samples in COAD were identified by limma, and functional enrichment analysis were performed. MuTect2 and maftools were used to measure somatic mutation rates, while ADTEx was used to measure copy number variations (CNVs). The intersection of three machine learning methods, support vector machine (SVM), Random Forest, and least absolute shrinkage and selection operator (LASSO), is utilized to screen biomarkers, and their diagnostic performance is subsequently validated. The correlation between biomarkers and immune cells infiltration was analyzed by Spearman method.

RESULTS

47 DEGs between liver metastasis and primary tumor samples in COAD were obtained, which were mainly enriched in the complement and coagulation, extracellular matrix (ECM), and peptidase regulator activity, etc. 38 out of 47 DEGs had mutations and exhibited a high frequency of CNV amplification or deletion. Furthermore, 3 biomarkers (MMP3, MAB21L2, and COLEC11) were screened, which showed good diagnostic performance. The proportion of multiple immune cells, such as B cells naive, T cells CD4 naive, Monocytes, and Dendritic cells resting, was higher in liver metastasis samples than that in primary tumor samples. Meanwhile, MMP3, MAB21L2, and COLEC11 exhibited an outstanding correlation with immune cells infiltration.

CONCLUSION

In short, 3 biomarkers with good diagnostic efficacy were identified, providing a new perspective of therapeutic targets for liver metastasis in COAD.

An initial game-theoretic assessment of enhanced tissue preparation and imaging protocols for improved deep learning inference of spatial transcriptomics from tissue morphology

The application of deep learning to spatial transcriptomics (ST) can reveal relationships between gene expression and tissue architecture. Prior work has demonstrated that inferring gene expression from tissue histomorphology can discern these spatial molecular markers to enable population scale studies, reducing the fiscal barriers associated with large-scale spatial profiling. However, while most improvements in algorithmic performance have focused on improving model architectures, little is known about how the quality of tissue preparation and imaging can affect deep learning model training for spatial inference from morphology and its potential for widespread clinical adoption. Prior studies for ST inference from histology typically utilize manually stained frozen sections with imaging on non-clinical grade scanners. Training such models on ST cohorts is also costly. We hypothesize that adopting tissue processing and imaging practices that mirror standards for clinical implementation (permanent sections, automated tissue staining, and clinical grade scanning) can significantly improve model performance. An enhanced specimen processing and imaging protocol was developed for deep learning-based ST inference from morphology. This protocol featured the Visium CytAssist assay to permit automated hematoxylin and eosin staining (e.g. Leica Bond), 40×-resolution imaging, and joining of multiple patients' tissue sections per capture area prior to ST profiling. Using a cohort of 13 pathologic T Stage-III stage colorectal cancer patients, we compared the performance of models trained on slide prepared using enhanced versus traditional (i.e. manual staining and low-resolution imaging) protocols. Leveraging Inceptionv3 neural networks, we predicted gene expression across serial, histologically-matched tissue sections using whole slide images (WSI) from both protocols. The data Shapley was used to quantify and compare marginal performance gains on a patient-by-patient basis attributed to using the enhanced protocol versus the actual costs of spatial profiling. Findings indicate that training and validating on WSI acquired through the enhanced protocol as opposed to the traditional method resulted in improved performance at lower fiscal cost. In the realm of ST, the enhancement of deep learning architectures frequently captures the spotlight; however, the significance of specimen processing and imaging is often understated. This research, informed through a game-theoretic lens, underscores the substantial impact that specimen preparation/imaging can have on spatial transcriptomic inference from morphology. It is essential to integrate such optimized processing protocols to facilitate the identification of prognostic markers at a larger scale.

Integrative Co-Registration of Elemental Imaging and Histopathology for Enhanced Spatial Multimodal Analysis of Tissue Sections through TRACE

Summary

Elemental imaging provides detailed profiling of metal bioaccumulation, offering more precision than bulk analysis by targeting specific tissue areas. However, accurately identifying comparable tissue regions from elemental maps is challenging, requiring the integration of hematoxylin and eosin (H&E) slides for effective comparison. Facilitating the streamlined co-registration of Whole Slide Images (WSI) and elemental maps, TRACE enhances the analysis of tissue regions and elemental abundance in various pathological conditions. Through an interactive containerized web application, TRACE features real-time annotation editing, advanced statistical tools, and data export, supporting comprehensive spatial analysis. Notably, it allows for comparison of elemental abundances across annotated tissue structures and enables integration with other spatial data types through WSI co-registration.

Availability and Implementation

Available on the following platforms- GitHub: jlevy44/trace_app , PyPI: trace_app , Docker: joshualevy44/trace_app , Singularity: joshualevy44/trace_app .

Contact

joshua.levy@cshs.org.

Supplementary information

Supplementary data are available.

Potential to Enhance Large Scale Molecular Assessments of Skin Photoaging through Virtual Inference of Spatial Transcriptomics from Routine Staining

The advent of spatial transcriptomics technologies has heralded a renaissance in research to advance our understanding of the spatial cellular and transcriptional heterogeneity within tissues. Spatial transcriptomics allows investigation of the interplay between cells, molecular pathways, and the surrounding tissue architecture and can help elucidate developmental trajectories, disease pathogenesis, and various niches in the tumor microenvironment. Photoaging is the histological and molecular skin damage resulting from chronic/acute sun exposure and is a major risk factor for skin cancer. Spatial transcriptomics technologies hold promise for improving the reliability of evaluating photoaging and developing new therapeutics. Challenges to current methods include limited focus on dermal elastosis variations and reliance on self-reported measures, which can introduce subjectivity and inconsistency. Spatial transcriptomics offers an opportunity to assess photoaging objectively and reproducibly in studies of carcinogenesis and discern the effectiveness of therapies that intervene in photoaging and preventing cancer. Evaluation of distinct histological architectures using highly-multiplexed spatial technologies can identify specific cell lineages that have been understudied due to their location beyond the depth of UV penetration. However, the cost and interpatient variability using state-of-the-art assays such as the 10x Genomics Spatial Transcriptomics assays limits the scope and scale of large-scale molecular epidemiologic studies. Here, we investigate the inference of spatial transcriptomics information from routine hematoxylin and eosin-stained (H&E) tissue slides. We employed the Visium CytAssist spatial transcriptomics assay to analyze over 18,000 genes at a 50-micron resolution for four patients from a cohort of 261 skin specimens collected adjacent to surgical resection sites for basal cell and squamous cell keratinocyte tumors. The spatial transcriptomics data was co-registered with 40x resolution whole slide imaging (WSI) information. We developed machine learning models that achieved a macro-averaged median AUC and F1 score of 0.80 and 0.61 and Spearman coefficient of 0.60 in inferring transcriptomic profiles across the slides, and accurately captured biological pathways across various tissue architectures.

Deep topographic proteomics of a human brain tumour

The spatial organisation of cellular protein expression profiles within tissue determines cellular function and is key to understanding disease pathology. To define molecular phenotypes in the spatial context of tissue, there is a need for unbiased, quantitative technology capable of mapping proteomes within tissue structures. Here, we present a workflow for spatially-resolved, quantitative proteomics of tissue that generates maps of protein abundance across tissue slices derived from a human atypical teratoid-rhabdoid tumour at three spatial resolutions, the highest being 40 µm, to reveal distinct abundance patterns of thousands of proteins. We employ spatially-aware algorithms that do not require prior knowledge of the fine tissue structure to detect proteins and pathways with spatial abundance patterns and correlate proteins in the context of tissue heterogeneity and cellular features such as extracellular matrix or proximity to blood vessels. We identify PYGL, ASPH and CD45 as spatial markers for tumour boundary and reveal immune response-driven, spatially-organised protein networks of the extracellular tumour matrix. Overall, we demonstrate spatially-aware deep proteo-phenotyping of tissue heterogeneity, to re-define understanding tissue biology and pathology at the molecular level.

Feasibility of Inferring Spatial Transcriptomics from Single-Cell Histological Patterns for Studying Colon Cancer Tumor Heterogeneity

Background

Spatial transcriptomics involves studying the spatial organization of gene expression within tissues, offering insights into the molecular diversity of tumors. While spatial gene expression is commonly amalgamated from 1-10 cells across 50-micron spots, recent methods have demonstrated the capability to disaggregate this information at subspot resolution by leveraging both expression and histological patterns. However, elucidating such information from histology alone presents a significant challenge but if solved can better permit spatial molecular analysis at cellular resolution for instances where Visium data is not available, reducing study costs. This study explores integrating single-cell histological and transcriptomic data to infer spatial mRNA expression patterns in whole slide images collected from a cohort of stage pT3 colorectal cancer patients. A cell graph neural network algorithm was developed to align histological information extracted from detected cells with single cell RNA patterns through optimal transport methods, facilitating the analysis of cellular groupings and gene relationships. This approach leveraged spot-level expression as an intermediary to co-map histological and transcriptomic information at the single-cell level.

Results

Our study demonstrated that single-cell transcriptional heterogeneity within a spot could be predicted from histological markers extracted from cells detected within a spot. Furthermore, our model exhibited proficiency in delineating overarching gene expression patterns across whole-slide images. This approach compared favorably to traditional patch-based computer vision methods as well as other methods which did not incorporate single cell expression during the model fitting procedures. Topological nuances of single-cell expression within a Visium spot were preserved using the developed methodology.

Conclusion

This innovative approach augments the resolution of spatial molecular assays utilizing histology as a sole input through synergistic co-mapping of histological and transcriptomic datasets at the single-cell level, anchored by spatial transcriptomics. While initial results are promising, they warrant rigorous validation. This includes collaborating with pathologists for precise spatial identification of distinct cell types and utilizing sophisticated assays, such as Xenium, to attain deeper subcellular insights.

The Overlooked Role of Specimen Preparation in Bolstering Deep Learning-Enhanced Spatial Transcriptomics Workflows

The application of deep learning methods to spatial transcriptomics has shown promise in unraveling the complex relationships between gene expression patterns and tissue architecture as they pertain to various pathological conditions. Deep learning methods that can infer gene expression patterns directly from tissue histomorphology can expand the capability to discern spatial molecular markers within tissue slides. However, current methods utilizing these techniques are plagued by substantial variability in tissue preparation and characteristics, which can hinder the broader adoption of these tools. Furthermore, training deep learning models using spatial transcriptomics on small study cohorts remains a costly endeavor. Necessitating novel tissue preparation processes enhance assay reliability, resolution, and scalability. This study investigated the impact of an enhanced specimen processing workflow for facilitating a deep learning-based spatial transcriptomics assessment. The enhanced workflow leveraged the flexibility of the Visium CytAssist assay to permit automated H&E staining (e.g., Leica Bond) of tissue slides, whole-slide imaging at 40x-resolution, and multiplexing of tissue sections from multiple patients within individual capture areas for spatial transcriptomics profiling. Using a cohort of thirteen pT3 stage colorectal cancer (CRC) patients, we compared the efficacy of deep learning models trained on slide prepared using an enhanced workflow as compared to the traditional workflow which leverages manual tissue staining and standard imaging of tissue slides. Leveraging Inceptionv3 neural networks, we aimed to predict gene expression patterns across matched serial tissue sections, each stemming from a distinct workflow but aligned based on persistent histological structures. Findings indicate that the enhanced workflow considerably outperformed the traditional spatial transcriptomics workflow. Gene expression profiles predicted from enhanced tissue slides also yielded expression patterns more topologically consistent with the ground truth. This led to enhanced statistical precision in pinpointing biomarkers associated with distinct spatial structures. These insights can potentially elevate diagnostic and prognostic biomarker detection by broadening the range of spatial molecular markers linked to metastasis and recurrence. Future endeavors will further explore these findings to enrich our comprehension of various diseases and uncover molecular pathways with greater nuance. Combining deep learning with spatial transcriptomics provides a compelling avenue to enrich our understanding of tumor biology and improve clinical outcomes. For results of the highest fidelity, however, effective specimen processing is crucial, and fostering collaboration between histotechnicians, pathologists, and genomics specialists is essential to herald this new era in spatial transcriptomics-driven cancer research.

Potential to Enhance Large Scale Molecular Assessments of Skin Photoaging through Virtual Inference of Spatial Transcriptomics from Routine Staining

The advent of spatial transcriptomics technologies has heralded a renaissance in research to advance our understanding of the spatial cellular and transcriptional heterogeneity within tissues. Spatial transcriptomics allows investigation of the interplay between cells, molecular pathways and the surrounding tissue architecture and can help elucidate developmental trajectories, disease pathogenesis, and various niches in the tumor microenvironment. Photoaging is the histological and molecular skin damage resulting from chronic/acute sun exposure and is a major risk factor for skin cancer. Spatial transcriptomics technologies hold promise for improving the reliability of evaluating photoaging and developing new therapeutics. Current challenges, including limited focus on dermal elastosis variations and reliance on self-reported measures, can introduce subjectivity and inconsistency. Spatial transcriptomics offer an opportunity to assess photoaging objectively and reproducibly in studies of carcinogenesis and discern the effectiveness of therapies that intervene on photoaging and prevent cancer. Evaluation of distinct histological architectures using highly-multiplexed spatial technologies can identify specific cell lineages that have been understudied due to their location beyond the depth of UV penetration. However, the cost and inter-patient variability using state-of-the-art assays such as the 10x Genomics Spatial Transcriptomics assays limits the scope and scale of large-scale molecular epidemiologic studies. Here, we investigate the inference of spatial transcriptomics information from routine hematoxylin and eosin-stained (H&E) tissue slides. We employed the Visium CytAssist spatial transcriptomics assay to analyze over 18,000 genes at a 50-micron resolution for four patients from a cohort of 261 skin specimens collected adjacent to surgical resection sites for basal and squamous keratinocyte tumors. The spatial transcriptomics data was co-registered with 40x resolution whole slide imaging (WSI) information. We developed machine learning models that achieved a macro-averaged median AUC and F1 score of 0.80 and 0.61 and Spearman coefficient of 0.60 in inferring transcriptomic profiles across the slides, and accurately captured biological pathways across various tissue architectures.