Image Analysis of the Tumor Microenvironment

Whole slide imaging (WSI) scanner differences influence optical and computed properties of digitized prostate cancer histology

Purpose

Digital pathology is becoming an increasingly popular area of advancement in both research and clinically. Pathologists are now able to manage and interpret slides digitally, as well as collaborate with external pathologists with digital copies of slides. Differences in slide scanners include variation in resolution, image contrast, and optical properties, which may influence downstream image processing. This study tested the hypothesis that varying slide scanners would result in differences in computed pathomic features on prostate cancer whole mount slides.

Design

This study collected 192 unique tissue slides from 30 patients following prostatectomy. Tissue samples were paraffin-embedded, stained for hematoxylin and eosin (H&E), and digitized using 3 different scanning microscopes at the highest available magnification rate, for a total of 3 digitized slides per tissue slide. These scanners included a (S1) Nikon microscope equipped with an automated sliding stage, an (S2) Olympus VS120 slide scanner, and a (S3) Huron TissueScope LE scanner. A color deconvolution algorithm was then used to optimize contrast by projecting the RGB image into color channels representing optical stain density. The resulting intensity standardized images were then computationally processed to segment tissue and calculate pathomic features including lumen, stroma, epithelium, and epithelial cell density, as well as second-order features including lumen area and roundness; epithelial area, roundness, and wall thickness; and cell fraction. For each tested feature, mean values of that feature per digitized slide were collected and compared across slide scanners using mixed effect models, fit to compare differences in the tested feature associated with all slide scanners for each slide, including a random effect of subject with a nested random effect of slide to account for repeated measures. Similar models were also computed for tissue densities to examine how differences in scanner impact downstream processing.

Results

Each mean color channel intensity (i.e., Red, Green, Blue) differed between slide scanners (all P<.001). Of the color deconvolved images, only the hematoxylin channel was similar in all 3 scanners (all P>.05). Lumen and stroma densities between S3 and S1 slides, and epithelial cell density between S3 and S2 (P>.05) were comparable but all other comparisons were significantly different (P<.05). The second-order features were found to be comparable for all scanner comparisons, except for lumen area and epithelium area.

Conclusion

This study demonstrates that both optical and computed properties of digitized histological samples are impacted by slide scanner differences. Future research is warranted to better understand which scanner properties influence the tissue segmentation process and to develop harmonization techniques for comparing data across multiple slide scanners.

Deep-Tissue Fluorescence Imaging Study of Reactive Oxygen Species in a Tumor Microenvironment

Tumor microenvironment (TME) is the survival environment for tumor cells to proliferate and metastasize in deep tissue. TME contains tumor cells, immune cells, stromal cells and a variety of active molecules including reactive oxygen species (ROS). Inside the TME, ROS regulate the oxidation-reduction (redox) homeostasis and promote oxidative stress. Due to the rapid proliferation ability and specific metabolic patterns of the TME, ROS pervade virtually all complex physiological processes and play irreplaceable roles in protein modification, signal transduction, metabolism, and energy production in various tumors. Therefore, measurements of the dynamically, multicomponent simultaneous changes of ROS in the TME are of great significance to reveal the detailed proliferation and metastasis mechanisms of the tumor. Near-infrared (NIR) and two-photon (TP) fluorescence imaging techniques possess real-time, dynamic, highly sensitive, and highly signal-to-noise ratios with deep tissue penetration abilities. With the rationally designed probes, the NIR and TP fluorescence imaging techniques have been widely used to reveal the mechanisms of how ROS regulates and constructs complex signals and metabolic networks in TME. Therefore, we summarize the design principles and performances of NIR and TP fluorescence imaging of ROS in the TME in the last four years, as well as discuss the advantages and potentials of these works. This Review can provide guidance and prospects for future research work on TME and facilitate the development of antitumor drugs.

Perspectives in pathomics in head and neck cancer

PURPOSE OF REVIEW

Pathology is the cornerstone of cancer care. Pathomics, which represents the use of artificial intelligence in digital pathology, is an emerging and promising field that will revolutionize medical and surgical pathology in the coming years. This review provides an overview of pathomics, its current and future applications and its most relevant applications in Head and Neck cancer care.

RECENT FINDINGS

The number of studies investigating the use of artificial intelligence in pathology is rapidly growing, especially as the utilization of deep learning has shown great potential with Whole Slide Images. Even though numerous steps still remain before its clinical use, Pathomics has been used for varied applications comprising of computer-assisted diagnosis, molecular anomalies prediction, tumor microenvironment and biomarker identification as well as prognosis evaluation. The majority of studies were performed on the most frequent cancers, notably breast, prostate, and lung. Interesting results were also found in Head and Neck cancers.

SUMMARY

Even if its use in Head and Neck cancer care is still low, Pathomics is a powerful tool to improve diagnosis, identify prognostic factors and new biomarkers. Important challenges lie ahead before its use in a clinical practice, notably the lack of information on how AI makes its decisions, the slow deployment of digital pathology, and the need for extensively validated data in order to obtain authorities approval. Regardless, pathomics will most likely improve pathology in general, including Head and Neck cancer care in the coming years.

Interrogating the microenvironmental landscape of tumors with computational image analysis approaches

The tumor microenvironment is an interacting heterogeneous collection of cancer cells, resident as well as infiltrating host cells, secreted factors, and extracellular matrix proteins. With the growing importance of immunotherapies, it has become crucial to be able to characterize the composition and the functional orientation of the microenvironment. The development of novel computational image analysis methodologies may enable the robust quantification and localization of immune and related biomarker-expressing cells within the microenvironment. The aim of the review is to concisely highlight a selection of current and significant contributions pertinent to methodological advances coupled with biomedical or translational applications. A further aim is to concisely present computational advances that, to our knowledge, have currently very limited use for the assessment of the microenvironment but have the potential to enhance image analysis pipelines; on this basis, an example is shown for the detection and segmentation of cells of the microenvironment using a published pipeline and a public dataset. Finally, a general proposal is presented on the conceptual design of automation-optimized computational image analysis workflows in the biomedical and clinical domain.

Histoecology: Applying Ecological Principles and Approaches to Describe and Predict Tumor Ecosystem Dynamics Across Space and Time

Cancer cells exist within a complex spatially structured ecosystem composed of resources and different cell types. As the selective pressures imposed by this environment determine the fate of cancer cells, an improved understanding of how this ecosystem evolves will better elucidate how tumors grow and respond to therapy. State of the art imaging methods can now provide highly resolved descriptions of the microenvironment, yielding the data required for a thorough study of its role in tumor growth and treatment resistance. The field of landscape ecology has been studying such species-environment relationship for decades, and offers many tools and perspectives that cancer researchers could greatly benefit from. Here, we discuss one such tool, species distribution modeling (SDM), that has the potential to, among other things, identify critical environmental factors that drive tumor evolution and predict response to therapy. SDMs only scratch the surface of how ecological theory and methods can be applied to cancer, and we believe further integration will take cancer research in exciting new and productive directions. Significance: Here we describe how species distribution modeling can be used to quantitatively describe the complex relationship between tumor cells and their microenvironment. Such a description facilitates a deeper understanding of cancers eco-evolutionary dynamics, which in turn sheds light on the factors that drive tumor growth and response to treatment.

Automated quantitative analysis of Ki-67 staining and HE images recognition and registration based on whole tissue sections in breast carcinoma

Background

The scoring of Ki-67 is highly relevant for the diagnosis, classification, prognosis, and treatment in breast invasive ductal carcinoma (IDC). Traditional scoring method of Ki-67 staining followed by manual counting, is time-consumption and inter−/intra observer variability, which may limit its clinical value. Although more and more algorithms and individual platforms have been developed for the assessment of Ki-67 stained images to improve its accuracy level, most of them lack of accurate registration of immunohistochemical (IHC) images and their matched hematoxylin-eosin (HE) images, or did not accurately labelled each positive and negative cell with Ki-67 staining based on whole tissue sections (WTS). In view of this, we introduce an accurate image registration method and an automatic identification and counting software of Ki-67 based on WTS by deep learning.

Methods

We marked 1017 breast IDC whole slide imaging (WSI), established a research workflow based on the (i) identification of IDC area, (ii) registration of HE and IHC slides from the same anatomical region, and (iii) counting of positive Ki-67 staining.

Results

The accuracy, sensitivity, and specificity levels of identifying breast IDC regions were 89.44, 85.05, and 95.23%, respectively, and the contiguous HE and Ki-67 stained slides perfectly registered. We counted and labelled each cell of 10 Ki-67 slides as standard for testing on WTS, the accuracy by automatic calculation of Ki-67 positive rate in attained IDC was 90.2%. In the human-machine competition of Ki-67 scoring, the average time of 1 slide was 2.3 min with 1 GPU by using this software, and the accuracy was 99.4%, which was over 90% of the results provided by participating doctors.

Conclusions

Our study demonstrates the enormous potential of automated quantitative analysis of Ki-67 staining and HE images recognition and registration based on WTS, and the automated scoring of Ki67 can thus successfully address issues of consistency, reproducibility and accuracy. We will provide those labelled images as an open-free platform for researchers to assess the performance of computer algorithms for automated Ki-67 scoring on IHC stained slides.

The Immune Response in Nonmetastatic Axillary Lymph Nodes Is Associated with the Presence of Axillary Metastasis and Breast Cancer Patient Outcome

Tumor cells can modify the immune response in primary tumors and in the axillary lymph nodes with metastasis (ALN⁺) in breast cancer (BC), influencing patient outcome. We investigated whether patterns of immune cells in the primary tumor and in the axillary lymph nodes without metastasis (ALN^-) differed between patients diagnosed without ALN⁺ (diagnosed-ALN^-) and with ALN⁺ (diagnosed-ALN⁺) and the implications for clinical outcome. Eleven immune markers were studied using immunohistochemistry, tissue microarray, and digital image analysis in 141 BC patient samples (75 diagnosed-ALN⁺ and 66 diagnosed-ALN^-). Two logistic regression models were derived to identify the clinical, pathologic, and immunologic variables associated with the presence of ALN⁺ at diagnosis. There are immune patterns in the ALN^- associated with the presence of ALN⁺ at diagnosis. The regression models revealed a small subgroup of diagnosed-ALN⁺ with ALN^- immune patterns that were more similar to those of the ALN^- of the diagnosed-ALN^-. This small subgroup also showed similar clinical behavior to that of the diagnosed-ALN^-. Another small subgroup of diagnosed-ALN^- with ALN^- immune patterns was found whose members were more similar to those of the ALN^- of the diagnosed-ALN⁺. This small subgroup had similar clinical behavior to the diagnosed-ALN⁺. These data suggest that the immune response present in ALN^- at diagnosis could influence the clinical outcome of BC patients.

Clinical significance of CD73 in triple-negative breast cancer: multiplex analysis of a phase III clinical trial

Background

CD73 is an ecto-enzyme that promotes tumor immune escape through the production of immunosuppressive extracellular adenosine in the tumor microenvironment. Several CD73 inhibitors and adenosine receptor antagonists are being evaluated in phase I clinical trials.

Patients and methods

Full-face sections from formalin-fixed paraffin-embedded primary breast tumors from 122 samples of triple-negative breast cancer (TNBC) from the BIG 02-98 adjuvant phase III clinical trial were included in our analysis. Using multiplex immunofluorescence and image analysis, we assessed CD73 protein expression on tumor cells, tumor-infiltrating leukocytes and stromal cells. We investigated the associations between CD73 protein expression with disease-free survival (DFS), overall survival (OS) and the extent of tumor immune infiltration.

Results

Our results demonstrated that high levels of CD73 expression on epithelial tumor cells were significantly associated with reduced DFS, OS and negatively correlated with tumor immune infiltration (Spearman's R= -0.50, P < 0.0001). Patients with high levels of CD73 and low levels of tumor-infiltrating leukocytes had the worse clinical outcome.

Conclusions

Taken together, our study provides further support that CD73 expression is associated with a poor prognosis and reduced anti-tumor immunity in human TNBC and that targeting CD73 could be a promising strategy to reprogram the tumor microenvironment in this BC subtype.

Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology

In the past decade, advances in precision oncology have resulted in an increased demand for predictive assays that enable the selection and stratification of patients for treatment. The enormous divergence of signalling and transcriptional networks mediating the crosstalk between cancer, stromal and immune cells complicates the development of functionally relevant biomarkers based on a single gene or protein. However, the result of these complex processes can be uniquely captured in the morphometric features of stained tissue specimens. The possibility of digitizing whole-slide images of tissue has led to the advent of artificial intelligence (AI) and machine learning tools in digital pathology, which enable mining of subvisual morphometric phenotypes and might, ultimately, improve patient management. In this Perspective, we critically evaluate various AI-based computational approaches for digital pathology, focusing on deep neural networks and 'hand-crafted' feature-based methodologies. We aim to provide a broad framework for incorporating AI and machine learning tools into clinical oncology, with an emphasis on biomarker development. We discuss some of the challenges relating to the use of AI, including the need for well-curated validation datasets, regulatory approval and fair reimbursement strategies. Finally, we present potential future opportunities for precision oncology.

Digital Microscopy, Image Analysis, and Virtual Slide Repository

Advancements in technology and digitization have ushered in novel ways of enhancing tissue-based research via digital microscopy and image analysis. Whole slide imaging scanners enable digitization of histology slides to be stored in virtual slide repositories and to be viewed via computers instead of microscopes. Easier and faster sharing of histologic images for teaching and consultation, improved storage and preservation of quality of stained slides, and annotation of features of interest in the digital slides are just a few of the advantages of this technology. Combined with the development of software for digital image analysis, digital slides further pave the way for the development of tools that extract quantitative data from tissue-based studies. This review introduces digital microscopy and pathology, and addresses technical and scientific considerations in slide scanning, quantitative image analysis, and slide repositories. It also highlights the current state of the technology and factors that need to be taken into account to insure optimal utility, including preanalytical considerations and the importance of involving a pathologist in all major steps along the digital microscopy and pathology workflow.

Towards personalized computational oncology: from spatial models of tumour spheroids, to organoids, to tissues

A main goal of mathematical and computational oncology is to develop quantitative tools to determine the most effective therapies for each individual patient. This involves predicting the right drug to be administered at the right time and at the right dose. Such an approach is known as precision medicine. Mathematical modelling can play an invaluable role in the development of such therapeutic strategies, since it allows for relatively fast, efficient and inexpensive simulations of a large number of treatment schedules in order to find the most effective. This review is a survey of mathematical models that explicitly take into account the spatial architecture of three-dimensional tumours and address tumour development, progression and response to treatments. In particular, we discuss models of epithelial acini, multicellular spheroids, normal and tumour spheroids and organoids, and multi-component tissues. Our intent is to showcase how these in silico models can be applied to patient-specific data to assess which therapeutic strategies will be the most efficient. We also present the concept of virtual clinical trials that integrate standard-of-care patient data, medical imaging, organ-on-chip experiments and computational models to determine personalized medical treatment strategies.

Introduction to Digital Image Analysis in Whole-slide Imaging: A White Paper from the Digital Pathology Association

The advent of whole-slide imaging in digital pathology has brought about the advancement of computer-aided examination of tissue via digital image analysis. Digitized slides can now be easily annotated and analyzed via a variety of algorithms. This study reviews the fundamentals of tissue image analysis and aims to provide pathologists with basic information regarding the features, applications, and general workflow of these new tools. The review gives an overview of the basic categories of software solutions available, potential analysis strategies, technical considerations, and general algorithm readouts. Advantages and limitations of tissue image analysis are discussed, and emerging concepts, such as artificial intelligence and machine learning, are introduced. Finally, examples of how digital image analysis tools are currently being used in diagnostic laboratories, translational research, and drug development are discussed.

Quantitative Characterization of CD8+ T Cell Clustering and Spatial Heterogeneity in Solid Tumors

Quantitative characterization of the tumor microenvironment, including its immuno-architecture, is important for developing quantitative diagnostic and predictive biomarkers, matching patients to the most appropriate treatments for precision medicine, and for providing quantitative data for building systems biology computational models able to predict tumor dynamics in the context of immune checkpoint blockade therapies. The intra- and inter-tumoral spatial heterogeneities are potentially key to the understanding of the dose-response relationships, but they also bring challenges to properly parameterizing and validating such models. In this study, we developed a workflow to detect CD8+ T cells from whole slide imaging data, and quantify the spatial heterogeneity using multiple metrics by applying spatial point pattern analysis and morphometric analysis. The results indicate a higher intra-tumoral heterogeneity compared with the heterogeneity across patients. By comparing the baseline metrics with PD-1 blockade treatment outcome, our results indicate that the number of high-density T cell clusters of both circular and elongated shapes are higher in patients who responded to the treatment. This methodology can be applied to quantitatively characterize the tumor microenvironment, including immuno-architecture, and its heterogeneity for different cancer types.

Utility of CD8 score by automated quantitative image analysis in head and neck squamous cell carcinoma

INTRODUCTION

In head and neck squamous cell carcinoma (HNSCC) high numbers of tumor infiltrating CD8 T cells in the tumor microenvironment are associated with better outcome. However, no investigators have employed automated image analysis on whole slide images to permit CD8 scores for use in clinical practice. The aim of this study was to develop and validate an image analysis algorithm to automatically quantify CD8 T cells in patients with oropharyngeal HNSCC.

MATERIALS AND METHODS

Using brightfield image analysis results were cross-validated with fluorescence based quantification (AQUA™). A nuclear image algorithm designed to run on whole slide images was optimized to manual count. The algorithm was locked down and used on a cohort of whole tissue sections from HNSCC patients. Multivariate clinicopathologic parameters and outcomes were statistically correlated with image analysis results.

RESULTS

Linear correlation between manual counts and the customized CD8 algorithm was 0.943. A total of 74 oropharyngeal HNSCC cases were analyzed for CD8 immune cell infiltrate using this image analysis algorithm. A CD8 immune cell density above 136 cells/mm² was associated with median survival of 18 years compared to 5 years. When multivariate modeling was performed, HPV infection was the only predictor of survival; however, when HPV was excluded only CD8 cell density predicts survival.

CONCLUSIONS

We report the successful technical development and clinical validation of an image algorithm to automate CD8 immune cell density for oropharyngeal HNSCC. Employing brightfield image analysis on entire tumor sections instead of tumor subcompartments permits this strategy to be widely implemented.