Optimization of an automated tumor-infiltrating lymphocyte algorithm for improved prognostication in primary melanoma

Artificial intelligence entering the pathology arena in oncology: current applications and future perspectives

BACKGROUND

Artificial intelligence (AI) is rapidly transforming the fields of pathology and oncology, offering novel opportunities for advancing diagnosis, prognosis, and treatment of cancer.

METHODS

Through a systematic review-based approach, the representatives from the European Society for Medical Oncology (ESMO) Precision Oncology Working Group (POWG) and international experts identified studies in pathology and oncology that applied AI-based algorithms for tumour diagnosis, molecular biomarker detection, and cancer prognosis assessment. These findings were synthesised to provide a comprehensive overview of current AI applications and future directions in cancer pathology.

RESULTS

The integration of AI tools in digital pathology is markedly improving the accuracy and efficiency of image analysis, allowing for automated tumour detection and classification, identification of prognostic molecular biomarkers, and prediction of treatment response and patient outcomes. Several barriers for the adoption of AI in clinical workflows, such as data availability, explainability, and regulatory considerations, still persist. There are currently no prognostic or predictive AI-based biomarkers supported by level IA or IB evidence. The ongoing advancements in AI algorithms, particularly foundation models, generalist models and transformer-based deep learning, offer immense promise for the future of cancer research and care. AI is also facilitating the integration of multi-omics data, leading to more precise patient stratification and personalised treatment strategies.

CONCLUSIONS

The application of AI in pathology is poised to not only enhance the accuracy and efficiency of cancer diagnosis and prognosis but also facilitate the development of personalised treatment strategies. Although barriers to implementation remain, ongoing research and development in this field coupled with addressing ethical and regulatory considerations will likely lead to a future where AI plays an integral role in cancer management and precision medicine. The continued evolution and adoption of AI in pathology and oncology are anticipated to reshape the landscape of cancer care, heralding a new era of precision medicine and improved patient outcomes.

Quantitative Digital Image Analysis for Assessing Cutaneous Fibrosis Correlates With Histopathological Scoring in Patients With Systemic Sclerosis

Systemic sclerosis (SSc) is a multisystem autoimmune disease characterized by progressive fibrosis of the skin, blood vessels, and internal organs. Accurate assessment of skin fibrosis is essential for disease monitoring and treatment evaluation, yet reliable quantitative methods are lacking. We compared computer-assisted digital image analysis with traditional semiquantitative histopathological scoring. Patients with SSc and healthy controls were identified from a hospital database. Clinical data, modified Rodnan skin score, and hematoxylin and eosin-stained skin biopsies were collected. Biopsies were scored using modified Keyser and Farge criteria. Whole-slide images were analyzed using QuPath open source digital pathology software (v0.5.1) for area annotation, normalized cellularity, and collagen density, whereas collagen alignment was evaluated using CurveAlign (MATLAB) software. Seven SSc and 7 healthy controls (no significant demographic differences) were studied. SSc samples showed significant differences in hyalinized collagen (P < .001), dermal fibroblast cellularity (P = .009), and total histopathological score (P = .007). Quantitative analysis confirmed decreased dermal cellularity (P = .024), increased collagen density (P = .009), and higher collagen alignment, with reduced fiber orientation variability, reflecting extracellular matrix restructuring in SSc. Tools such as QuPath and CurveAlign improve objectivity and reproducibility in SSc skin assessment, correlating well with traditional scores. These findings support the integration of computer-aided quantitative analysis into clinical practice and trials, advancing personalized fibrosis evaluation. Larger, longitudinal studies are needed for further validation.

Hepatocellular Carcinoma Immune Microenvironment Analysis: A Comprehensive Assessment with Computational and Classical Pathology

PURPOSE

The spatial variability and clinical relevance of the tumor immune microenvironment (TIME) are still poorly understood for hepatocellular carcinoma (HCC). In this study, we aim to develop a deep learning (DL)-based image analysis model for the spatial analysis of immune cell biomarkers and microscopically evaluate the distribution of immune infiltration.

EXPERIMENTAL DESIGN

Ninety-two HCC surgical liver resections and 51 matched needle biopsies were histologically classified according to their immunophenotypes: inflamed, immune-excluded, and immune-desert. To characterize the TIME on immunohistochemistry (IHC)-stained slides, we designed a multistage DL algorithm, IHC-TIME, to automatically detect immune cells and their localization in the TIME in tumor-stroma and center-border segments.

RESULTS

Two models were trained to detect and localize the immune cells on IHC-stained slides. The framework models (i.e., immune cell detection models and tumor-stroma segmentation) reached 98% and 91% accuracy, respectively. Patients with inflamed tumors showed better recurrence-free survival than those with immune-excluded or immune-desert tumors. Needle biopsies were found to be 75% accurate in representing the immunophenotypes of the main tumor. Finally, we developed an algorithm that defines immunophenotypes automatically based on the IHC-TIME analysis, achieving an accuracy of 80%.

CONCLUSIONS

Our DL-based tool can accurately analyze and quantify immune cells on IHC-stained slides of HCC. Microscopic classification of the TIME can stratify HCC according to the patient prognosis. Needle biopsies can provide valuable insights for TIME-related prognostic prediction, albeit with specific constraints. The computational pathology tool provides a new way to study the HCC TIME.

Advancements in computer vision and pathology: Unraveling the potential of artificial intelligence for precision diagnosis and beyond

The integration of computer vision into pathology through slide digitalization represents a transformative leap in the field's evolution. Traditional pathology methods, while reliable, are often time-consuming and susceptible to intra- and interobserver variability. In contrast, computer vision, empowered by artificial intelligence (AI) and machine learning (ML), promises revolutionary changes, offering consistent, reproducible, and objective results with ever-increasing speed and scalability. The applications of advanced algorithms and deep learning architectures like CNNs and U-Nets augment pathologists' diagnostic capabilities, opening new frontiers in automated image analysis. As these technologies mature and integrate into digital pathology workflows, they are poised to provide deeper insights into disease processes, quantify and standardize biomarkers, enhance patient outcomes, and automate routine tasks, reducing pathologists' workload. However, this transformative force calls for cross-disciplinary collaboration between pathologists, computer scientists, and industry innovators to drive research and development. While acknowledging its potential, this chapter addresses the limitations of AI in pathology, encompassing technical, practical, and ethical considerations during development and implementation.

Current status of artificial intelligence methods for skin cancer survival analysis: a scoping review

Skin cancer mortality rates continue to rise, and survival analysis is increasingly needed to understand who is at risk and what interventions improve outcomes. However, current statistical methods are limited by inability to synthesize multiple data types, such as patient genetics, clinical history, demographics, and pathology and reveal significant multimodal relationships through predictive algorithms. Advances in computing power and data science enabled the rise of artificial intelligence (AI), which synthesizes vast amounts of data and applies algorithms that enable personalized diagnostic approaches. Here, we analyze AI methods used in skin cancer survival analysis, focusing on supervised learning, unsupervised learning, deep learning, and natural language processing. We illustrate strengths and weaknesses of these approaches with examples. Our PubMed search yielded 14 publications meeting inclusion criteria for this scoping review. Most publications focused on melanoma, particularly histopathologic interpretation with deep learning. Such concentration on a single type of skin cancer amid increasing focus on deep learning highlight growing areas for innovation; however, it also demonstrates opportunity for additional analysis that addresses other types of cutaneous malignancies and expands the scope of prognostication to combine both genetic, histopathologic, and clinical data. Moreover, researchers may leverage multiple AI methods for enhanced benefit in analyses. Expanding AI to this arena may enable improved survival analysis, targeted treatments, and outcomes.

Tumor-Infiltrating Lymphocyte Recognition in Primary Melanoma by Deep Learning Convolutional Neural Network

The presence of tumor-infiltrating lymphocytes (TILs) is associated with a favorable prognosis of primary melanoma (PM). Recently, artificial intelligence (AI)-based approach in digital pathology was proposed for the standardized assessment of TILs on hematoxylin and eosin-stained whole slide images (WSIs). Herein, the study applied a new convolution neural network (CNN) analysis of PM WSIs to automatically assess the infiltration of TILs and extract a TIL score. A CNN was trained and validated in a retrospective cohort of 307 PMs including a training set (237 WSIs, 57,758 patches) and an independent testing set (70 WSIs, 29,533 patches). An AI-based TIL density index (AI-TIL) was identified after the classification of tumor patches by the presence or absence of TILs. The proposed CNN showed high performance in recognizing TILs in PM WSIs, showing 100% specificity and sensitivity on the testing set. The AI-based TIL index correlated with conventional TIL evaluation and clinical outcome. The AI-TIL index was an independent prognostic marker associated directly with a favorable prognosis. A fully automated and standardized AI-TIL appeared to be superior to conventional methods at differentiating the PM clinical outcome. Further studies are required to develop an easy-to-use tool to assist pathologists to assess TILs in the clinical evaluation of solid tumors.

Enhancing personalized immune checkpoint therapy by immune archetyping and pharmacological targeting

Immune checkpoint inhibitors (ICIs) are an expanding class of immunotherapeutic agents with the potential to cure cancer. Despite the outstanding clinical response in patient subsets, most individuals become refractory or develop resistance. Patient stratification and personalized immunotherapies are limited by the absence of predictive response markers. Recent findings show that dominant patterns of immune cell composition, T-cell status and heterogeneity, and spatiotemporal distribution of immune cells within the tumor microenvironment (TME) are becoming essential determinants of prognosis and therapeutic response. In this context, ICIs also function as investigational tools and proof of concept, allowing the validation of the identified mechanisms. After reviewing the current state of ICIs, this article will explore new comprehensive predictive markers for ICIs based on recent discoveries. We will discuss the recent establishment of a classification of TMEs into immune archetypes as a tool for personalized immune profiling, allowing patient stratification before ICI treatment. We will discuss the developing comprehension of T-cell diversity and its role in shaping the immune profile of patients. We describe the potential of strategies that score the mutual spatiotemporal modulation between T-cells and other cellular components of the TME. Additionally, we will provide an overview of a range of synthetic and naturally occurring or derived small molecules. We will compare compounds that were recently identified by in silico prediction to wet lab-validated drug candidates with the potential to function as ICIs and/or modulators of the cellular components of the TME.

Artificial intelligence-assisted dermatology diagnosis: From unimodal to multimodal

Artificial Intelligence (AI) is progressively permeating medicine, notably in the realm of assisted diagnosis. However, the traditional unimodal AI models, reliant on large volumes of accurately labeled data and single data type usage, prove insufficient to assist dermatological diagnosis. Augmenting these models with text data from patient narratives, laboratory reports, and image data from skin lesions, dermoscopy, and pathologies could significantly enhance their diagnostic capacity. Large-scale pre-training multimodal models offer a promising solution, exploiting the burgeoning reservoir of clinical data and amalgamating various data types. This paper delves into unimodal models' methodologies, applications, and shortcomings while exploring how multimodal models can enhance accuracy and reliability. Furthermore, integrating cutting-edge technologies like federated learning and multi-party privacy computing with AI can substantially mitigate patient privacy concerns in dermatological datasets and further fosters a move towards high-precision self-diagnosis. Diagnostic systems underpinned by large-scale pre-training multimodal models can facilitate dermatology physicians in formulating effective diagnostic and treatment strategies and herald a transformative era in healthcare.

Deep learning in computational dermatopathology of melanoma: A technical systematic literature review

Deep learning (DL) has become one of the major approaches in computational dermatopathology, evidenced by a significant increase in this topic in the current literature. We aim to provide a structured and comprehensive overview of peer-reviewed publications on DL applied to dermatopathology focused on melanoma. In comparison to well-published DL methods on non-medical images (e.g., classification on ImageNet), this field of application comprises a specific set of challenges, such as staining artifacts, large gigapixel images, and various magnification levels. Thus, we are particularly interested in the pathology-specific technical state-of-the-art. We also aim to summarize the best performances achieved thus far with respect to accuracy, along with an overview of self-reported limitations. Accordingly, we conducted a systematic literature review of peer-reviewed journal and conference articles published between 2012 and 2022 in the databases ACM Digital Library, Embase, IEEE Xplore, PubMed, and Scopus, expanded by forward and backward searches to identify 495 potentially eligible studies. After screening for relevance and quality, a total of 54 studies were included. We qualitatively summarized and analyzed these studies from technical, problem-oriented, and task-oriented perspectives. Our findings suggest that the technical aspects of DL for histopathology in melanoma can be further improved. The DL methodology was adopted later in this field, and still lacks the wider adoption of DL methods already shown to be effective for other applications. We also discuss upcoming trends toward ImageNet-based feature extraction and larger models. While DL has achieved human-competitive accuracy in routine pathological tasks, its performance on advanced tasks is still inferior to wet-lab testing (for example). Finally, we discuss the challenges impeding the translation of DL methods to clinical practice and provide insight into future research directions.

Spatial analyses of immune cell infiltration in cancer: current methods and future directions: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer

Modern histologic imaging platforms coupled with machine learning methods have provided new opportunities to map the spatial distribution of immune cells in the tumor microenvironment. However, there exists no standardized method for describing or analyzing spatial immune cell data, and most reported spatial analyses are rudimentary. In this review, we provide an overview of two approaches for reporting and analyzing spatial data (raster versus vector-based). We then provide a compendium of spatial immune cell metrics that have been reported in the literature, summarizing prognostic associations in the context of a variety of cancers. We conclude by discussing two well-described clinical biomarkers, the breast cancer stromal tumor infiltrating lymphocytes score and the colon cancer Immunoscore, and describe investigative opportunities to improve clinical utility of these spatial biomarkers. © 2023 The Pathological Society of Great Britain and Ireland.

Deep learning-based scoring of tumour-infiltrating lymphocytes is prognostic in primary melanoma and predictive to PD-1 checkpoint inhibition in melanoma metastases

BACKGROUND

Recent advances in digital pathology have enabled accurate and standardised enumeration of tumour-infiltrating lymphocytes (TILs). Here, we aim to evaluate TILs as a percentage electronic TIL score (eTILs) and investigate its prognostic and predictive relevance in cutaneous melanoma.

METHODS

We included stage I to IV cutaneous melanoma patients and used hematoxylin-eosin-stained slides for TIL analysis. We assessed eTILs as a continuous and categorical variable using the published cut-off of 16.6% and applied Cox regression models to evaluate associations of eTILs with relapse-free, distant metastasis-free, and overall survival. We compared eTILs of the primaries with matched metastasis. Moreover, we assessed the predictive relevance of eTILs in therapy-naïve metastases according to the first-line therapy.

FINDINGS

We analysed 321 primary cutaneous melanomas and 191 metastatic samples. In simple Cox regression, tumour thickness (p < 0.0001), presence of ulceration (p = 0.0001) and eTILs ≤16.6% (p = 0.0012) were found to be significant unfavourable prognostic factors for RFS. In multiple Cox regression, eTILs ≤16.6% (p = 0.0161) remained significant and downgraded the current staging. Lower eTILs in the primary tissue was associated with unfavourable relapse-free (p = 0.0014) and distant metastasis-free survival (p = 0.0056). In multiple Cox regression adjusted for tumour thickness and ulceration, eTILs as continuous remained significant (p = 0.019). When comparing TILs in primary tissue and corresponding metastasis of the same patient, eTILs in metastases was lower than in primary melanomas (p < 0.0001). In therapy-naïve metastases, an eTILs >12.2% was associated with longer progression-free survival (p = 0.037) and melanoma-specific survival (p = 0.0038) in patients treated with anti-PD-1-based immunotherapy. In multiple Cox regression, lactate dehydrogenase (p < 0.0001) and eTILs ≤12.2% (p = 0.0130) were significantly associated with unfavourable melanoma-specific survival.

INTERPRETATION

Assessment of TILs is prognostic in primary melanoma samples, and the eTILs complements staging. In therapy-naïve metastases, eTILs ≤12.2% is predictive of unfavourable survival outcomes in patients receiving anti-PD-1-based therapy.

FUNDING

See a detailed list of funding bodies in the Acknowledgements section at the end of the manuscript.

Diagnostic and Prognostic Deep Learning Applications for Histological Assessment of Cutaneous Melanoma

Melanoma is among the most devastating human malignancies. Accurate diagnosis and prognosis are essential to offer optimal treatment. Histopathology is the gold standard for establishing melanoma diagnosis and prognostic features. However, discrepancies often exist between pathologists, and analysis is costly and time-consuming. Deep-learning algorithms are deployed to improve melanoma diagnosis and prognostication from histological images of melanoma. In recent years, the development of these machine-learning tools has accelerated, and machine learning is poised to become a clinical tool to aid melanoma histology. Nevertheless, a review of the advances in machine learning in melanoma histology was lacking. We performed a comprehensive literature search to provide a complete overview of the recent advances in machine learning in the assessment of melanoma based on hematoxylin eosin digital pathology images. In our work, we review 37 recent publications, compare the methods and performance of the reviewed studies, and highlight the variety of promising machine-learning applications in melanoma histology.

Quantitative Assessment of Tumor-Infiltrating Lymphocytes Using Machine Learning Predicts Survival in Muscle-Invasive Bladder Cancer

(1) Purpose: Although assessment of tumor-infiltrating lymphocytes (TILs) has been acknowledged to have important predictive prognostic value in muscle-invasive bladder cancer (MIBC), it is limited by inter- and intra-observer variability, hampering widespread clinical application. We aimed to evaluate the prognostic value of quantitative TILs score based on a machine learning (ML) algorithm to identify MIBC patients who might benefit from immunotherapy or the de-escalation of therapy. (2) Methods: We constructed an artificial neural network classifier for tumor cells, lymphocytes, stromal cells, and “ignore” cells from hematoxylin-and-eosin-stained slide images using the QuPath open source software. We defined four unique TILs variables based on ML to analyze TILs measurements. Pathological slide images from 133 MIBC patients were retrospectively collected as the discovery set to determine the optimal association of ML-read TILs variables with patient survival outcomes. For validation, we evaluated an independent external validation set consisting of 247 MIBC patients. (3) Results: We found that all four TILs variables had significant prognostic associations with survival outcomes in MIBC patients (p < 0.001 for all comparisons), with higher TILs score being associated with better prognosis. Univariate and multivariate Cox regression analyses demonstrated that electronic TILs (eTILs) variables were independently associated with overall survival after adjustment for clinicopathological factors including age, sex, and pathological stage (p < 0.001 for all analyses). Results analyzed in different subgroups showed that the eTILs variable was a strong prognostic factor that was not redundant with pre-existing clinicopathological features (p < 0.05 for all analyses). (4) Conclusion: ML-driven cell classifier-defined TILs variables were robust and independent prognostic factors in two independent cohorts of MIBC patients. eTILs have the potential to identify a subset of high-risk stage II or stage III-IV MIBC patients who might benefit from adjuvant immunotherapy.

Multi-modality artificial intelligence in digital pathology

In common medical procedures, the time-consuming and expensive nature of obtaining test results plagues doctors and patients. Digital pathology research allows using computational technologies to manage data, presenting an opportunity to improve the efficiency of diagnosis and treatment. Artificial intelligence (AI) has a great advantage in the data analytics phase. Extensive research has shown that AI algorithms can produce more up-to-date and standardized conclusions for whole slide images. In conjunction with the development of high-throughput sequencing technologies, algorithms can integrate and analyze data from multiple modalities to explore the correspondence between morphological features and gene expression. This review investigates using the most popular image data, hematoxylin-eosin stained tissue slide images, to find a strategic solution for the imbalance of healthcare resources. The article focuses on the role that the development of deep learning technology has in assisting doctors' work and discusses the opportunities and challenges of AI.

Multiplexed Immunohistochemistry and Digital Pathology as the Foundation for Next-Generation Pathology in Melanoma: Methodological Comparison and Future Clinical Applications

The state-of-the-art for melanoma treatment has recently witnessed an enormous revolution, evolving from a chemotherapeutic, "one-drug-for-all" approach, to a tailored molecular- and immunological-based approach with the potential to make personalized therapy a reality. Nevertheless, methods still have to improve a lot before these can reliably characterize all the tumoral features that make each patient unique. While the clinical introduction of next-generation sequencing has made it possible to match mutational profiles to specific targeted therapies, improving response rates to immunotherapy will similarly require a deep understanding of the immune microenvironment and the specific contribution of each component in a patient-specific way. Recent advancements in artificial intelligence and single-cell profiling of resected tumor samples are paving the way for this challenging task. In this review, we provide an overview of the state-of-the-art in artificial intelligence and multiplexed immunohistochemistry in pathology, and how these bear the potential to improve diagnostics and therapy matching in melanoma. A major asset of in-situ single-cell profiling methods is that these preserve the spatial distribution of the cells in the tissue, allowing researchers to not only determine the cellular composition of the tumoral microenvironment, but also study tissue sociology, making inferences about specific cell-cell interactions and visualizing distinctive cellular architectures - all features that have an impact on anti-tumoral response rates. Despite the many advantages, the introduction of these approaches requires the digitization of tissue slides and the development of standardized analysis pipelines which pose substantial challenges that need to be addressed before these can enter clinical routine.