The Digital Slide Archive: A Software Platform for Management, Integration, and Analysis of Histology for Cancer Research

Current Advancements in Digital Neuropathology and Machine Learning for the Study of Neurodegenerative Diseases

Computational neurodegenerative neuropathology represents a transformative approach in the analysis and understanding of neurodegenerative diseases through utilization of whole slide images (WSIs) and advanced machine learning/artificial intelligence (ML/AI) techniques. This review explores the emerging field of computational neurodegenerative neuropathology, emphasizing its potential to enhance neuropathologic assessment, diagnosis, and research. Recent advancements in ML/AI technologies have significantly affected image-based medical fields, including anatomic pathology, by automating disease staging, identifying novel morphologic biomarkers, and uncovering new clinical insights via multi-modal AI approaches. Despite its promise, the field faces several challenges, including limited expert annotations, slide scanning inaccessibility, inter-institutional variability, and the complexities of sharing large WSI data sets. This review discusses the importance of improving deep learning model accuracy and efficiency for better interpretation of neuropathologic data. It highlights the potential of unsupervised learning to identify patterns in unannotated data. Furthermore, the development of explainable AI models is crucial for experimental neuropathology. By addressing these challenges and leveraging cutting-edge AI techniques, computational neurodegenerative neuropathology has the potential to revolutionize the field and significantly advance our understanding of disease.

An update on applications of digital pathology: primary diagnosis; telepathology, education and research

Digital Pathology or whole slide imaging (WSI) is a diagnostic evaluation technique that produces digital images of high quality from tissue fragments. These images are formed on glass slides and evaluated by pathologist with the aid of microscope. As the concept of digital pathology is introduced, these high quality images are digitized and produced on-screen whole slide images in the form of digital files. This has paved the way for pathologists to collaborate with other pathology professionals in case of any additional recommendations and also provides remote working opportunities. The application of digital pathology in clinical practice is glazed with several advantages and adopted by pathologists and researchers for clinical, educational and research purposes. Moreover, digital pathology system integration requires an intensive effort from multiple stakeholders. All pathology departments have different needs, case usage, and blueprints, even though the framework elements and variables for effective clinical integration can be applied to any institution aiming for digital transformation. This article reviews the background and developmental phases of digital pathology and its application in clinical services, educational and research activities.

Summary of the National Cancer Institute 2023 Virtual Workshop on Medical Image De-identification-Part 2: Pathology Whole Slide Image De-identification, De-facing, the Role of AI in Image De-identification, and the NCI MIDI Datasets and Pipeline

De-identification of medical images intended for research is a core requirement for data sharing initiatives, particularly as the demand for data for artificial intelligence (AI) applications grows. The Center for Biomedical Informatics and Information Technology (CBIIT) of the United States National Cancer Institute (NCI) convened a two half-day virtual workshop with the intent of summarizing the state of the art in de-identification technology and processes and exploring interesting aspects of the subject. This paper summarizes the highlights of the second day of the workshop, the recordings and presentations of which are publicly available for review. The topics covered included pathology whole slide image de-identification, de-facing, the role of AI in image de-identification, and the NCI Medical Image De-Identification Initiative (MIDI) datasets and pipeline.

GNCnn: A QuPath extension for glomerulosclerosis and glomerulonephritis characterization based on deep learning

The digitalization of traditional glass slide microscopy into whole slide images has opened up new opportunities for pathology, such as the application of artificial intelligence techniques. Specialized software is necessary to visualize and analyze these images. One of these applications is QuPath, a popular bioimage analysis tool. This study proposes GNCnn, the first open-source QuPath extension specifically designed for nephropathology. It integrates deep learning models to provide nephropathologists with an accessible, automatic detector and classifier of glomeruli, the basic filtering units of the kidneys. The aim is to offer nephropathologists a freely available application to measure and analyze glomeruli to identify conditions such as glomerulosclerosis and glomerulonephritis. GNCnn offers a user-friendly interface that enables nephropathologists to detect glomeruli with high accuracy (Dice coefficient of 0.807) and categorize them as either sclerotic or non-sclerotic, achieving a balanced accuracy of 98.46%. Furthermore, it facilitates the classification of non-sclerotic glomeruli into 12 commonly diagnosed types of glomerulonephritis, with a top-3 balanced accuracy of 84.41%. GNCnn provides real-time updates of results, which are available at both the glomerulus and slide levels. This allows users to complete a typical analysis task without leaving the main application, QuPath. This tool is the first to integrate the entire workflow for the assessment of glomerulonephritis directly into the nephropathologists' workspace, accelerating and supporting their diagnosis.

SlideTiler: A dataset creator software for boosting deep learning on histological whole slide images

The introduction of deep learning caused a significant breakthrough in digital pathology. Thanks to its capability of mining hidden data patterns in digitised histological slides to resolve diagnostic tasks and extract prognostic and predictive information. However, the high performance achieved in classification tasks depends on the availability of large datasets, whose collection and preprocessing are still time-consuming processes. Therefore, strategies to make these steps more efficient are worth investigation. This work introduces SlideTiler, an open-source software with a user-friendly graphical interface. SlideTiler can manage several image preprocessing phases through an intuitive workflow that does not require specific coding skills. The software was designed to provide direct access to virtual slides, allowing custom tiling of specific regions of interest drawn by the user, tile labelling, quality assessment, and direct export to dataset directories. To illustrate the functions and the scalability of SlideTiler, a deep learning-based classifier was implemented to classify 4 different tumour histotypes available in the TCGA repository. The results demonstrate the effectiveness of SlideTiler in facilitating data preprocessing and promoting accessibility to digitised pathology images for research purposes. Considering the increasing interest in deep learning applications of digital pathology, SlideTiler has a positive impact on this field. Moreover, SlideTiler has been conceived as a dynamic tool in constant evolution, and more updated and efficient versions will be released in the future.

Machine learning approaches for spatial omics data analysis in digital pathology: tools and applications in genitourinary oncology

Recent advances in spatial omics technologies have enabled new approaches for analyzing tissue morphology, cell composition, and biomolecule expression patterns in situ. These advances are promoting the development of new computational tools and quantitative techniques in the emerging field of digital pathology. In this review, we survey current trends in the development of computational methods for spatially mapped omics data analysis using digitized histopathology slides and supplementary materials, with an emphasis on tools and applications relevant to genitourinary oncological research. The review contains three sections: 1) an overview of image processing approaches for histopathology slide analysis; 2) machine learning integration with spatially resolved omics data analysis; 3) a discussion of current limitations and future directions for integration of machine learning in the clinical decision-making process.

MOTH: Memory-Efficient On-the-Fly Tiling of Histological Image Annotations Using QuPath

The emerging usage of digitalized histopathological images is leading to a novel possibility for data analysis. With the help of artificial intelligence algorithms, it is now possible to detect certain structures and morphological features on whole slide images automatically. This enables algorithms to count, measure, or evaluate those areas when trained properly. To achieve suitable training, datasets must be annotated and curated by users in programs like QuPath. The extraction of this data for artificial intelligence algorithms is still rather tedious and needs to be saved on a local hard drive. We developed a toolkit for integration into existing pipelines and tools, like U-net, for the on-the-fly extraction of annotation tiles from existing QuPath projects. The tiles can be directly used as input for artificial intelligence algorithms, and the results are directly transferred back to QuPath for visual inspection. With the toolkit, we created a convenient way to incorporate QuPath into existing AI workflows.

Learning precise segmentation of neurofibrillary tangles from rapid manual point annotations

Accumulation of abnormal tau protein into neurofibrillary tangles (NFTs) is a pathologic hallmark of Alzheimer disease (AD). Accurate detection of NFTs in tissue samples can reveal relationships with clinical, demographic, and genetic features through deep phenotyping. However, expert manual analysis is time-consuming, subject to observer variability, and cannot handle the data amounts generated by modern imaging. We present a scalable, open-source, deep-learning approach to quantify NFT burden in digital whole slide images (WSIs) of post-mortem human brain tissue. To achieve this, we developed a method to generate detailed NFT boundaries directly from single-point-per-NFT annotations. We then trained a semantic segmentation model on 45 annotated 2400μm by 1200μm regions of interest (ROIs) selected from 15 unique temporal cortex WSIs of AD cases from three institutions (University of California (UC)-Davis, UC-San Diego, and Columbia University). Segmenting NFTs at the single-pixel level, the model achieved an area under the receiver operating characteristic of 0.832 and an F1 of 0.527 (196-fold over random) on a held-out test set of 664 NFTs from 20 ROIs (7 WSIs). We compared this to deep object detection, which achieved comparable but coarser-grained performance that was 60% faster. The segmentation and object detection models correlated well with expert semi-quantitative scores at the whole-slide level (Spearman's rho ρ=0.654 (p=6.50e-5) and ρ=0.513 (p=3.18e-3), respectively). We openly release this multi-institution deep-learning pipeline to provide detailed NFT spatial distribution and morphology analysis capability at a scale otherwise infeasible by manual assessment.

Automated deep learning segmentation of neuritic plaques and neurofibrillary tangles in Alzheimer disease brain sections using a proprietary software

Neuropathological diagnosis of Alzheimer disease (AD) relies on semiquantitative analysis of phosphorylated tau-positive neurofibrillary tangles (NFTs) and neuritic plaques (NPs), without consideration of lesion heterogeneity in individual cases. We developed a deep learning workflow for automated annotation and segmentation of NPs and NFTs from AT8-immunostained whole slide images (WSIs) of AD brain sections. Fifteen WSIs of frontal cortex from 4 biobanks with varying tissue quality, staining intensity, and scanning formats were analyzed. We established an artificial intelligence (AI)-driven iterative procedure to improve the generation of expert-validated annotation datasets for NPs and NFTs thereby increasing annotation quality by >50%. This strategy yielded an expert-validated annotation database with 5013 NPs and 5143 NFTs. We next trained two U-Net convolutional neural networks for detection and segmentation of NPs or NFTs, achieving high accuracy and consistency (mean Dice similarity coefficient: NPs, 0.77; NFTs, 0.81). The workflow showed high generalization performance across different cases. This study serves as a proof-of-concept for the utilization of proprietary image analysis software (Visiopharm) in the automated deep learning segmentation of NPs and NFTs, demonstrating that AI can significantly improve the annotation quality of complex neuropathological features and enable the creation of highly precise models for identifying these markers in AD brain sections.

FUSION: A web-based application for in-depth exploration of multi-omics data with brightfield histology

Spatial -OMICS technologies facilitate the interrogation of molecular profiles in the context of the underlying histopathology and tissue microenvironment. Paired analysis of histopathology and molecular data can provide pathologists with otherwise unobtainable insights into biological mechanisms. To connect the disparate molecular and histopathologic features into a single workspace, we developed FUSION (Functional Unit State IdentificatiON in WSIs [Whole Slide Images]), a web-based tool that provides users with a broad array of visualization and analytical tools including deep learning-based algorithms for in-depth interrogation of spatial -OMICS datasets and their associated high-resolution histology images. FUSION enables end-to-end analysis of functional tissue units (FTUs), automatically aggregating underlying molecular data to provide a histopathology-based medium for analyzing healthy and altered cell states and driving new discoveries using "pathomic" features. We demonstrate FUSION using 10x Visium spatial transcriptomics (ST) data from both formalin-fixed paraffin embedded (FFPE) and frozen prepared datasets consisting of healthy and diseased tissue. Through several use-cases, we demonstrate how users can identify spatial linkages between quantitative pathomics, qualitative image characteristics, and spatial --omics.

Integrating frontiers: a holistic, quantum and evolutionary approach to conquering cancer through systems biology and multidisciplinary synergy

Cancer therapy is facing increasingly significant challenges, marked by a wide range of techniques and research efforts centered around somatic mutations, precision oncology, and the vast amount of big data. Despite this abundance of information, the quest to cure cancer often seems more elusive, with the "war on cancer" yet to deliver a definitive victory. A particularly pressing issue is the development of tumor treatment resistance, highlighting the urgent need for innovative approaches. Evolutionary, Quantum Biology and System Biology offer a promising framework for advancing experimental cancer research. By integrating theoretical studies, translational methods, and flexible multidisciplinary clinical research, there's potential to enhance current treatment strategies and improve outcomes for cancer patients. Establishing stronger links between evolutionary, quantum, entropy and chaos principles and oncology could lead to more effective treatments that leverage an understanding of the tumor's evolutionary dynamics, paving the way for novel methods to control and mitigate cancer. Achieving these objectives necessitates a commitment to multidisciplinary and interprofessional collaboration at the heart of both research and clinical endeavors in oncology. This entails dismantling silos between disciplines, encouraging open communication and data sharing, and integrating diverse viewpoints and expertise from the outset of research projects. Being receptive to new scientific discoveries and responsive to how patients react to treatments is also crucial. Such strategies are key to keeping the field of oncology at the forefront of effective cancer management, ensuring patients receive the most personalized and effective care. Ultimately, this approach aims to push the boundaries of cancer understanding, treating it as a manageable chronic condition, aiming to extend life expectancy and enhance patient quality of life.

Computational methods and biomarker discovery strategies for spatial proteomics: a review in immuno-oncology

Advancements in imaging technologies have revolutionized our ability to deeply profile pathological tissue architectures, generating large volumes of imaging data with unparalleled spatial resolution. This type of data collection, namely, spatial proteomics, offers invaluable insights into various human diseases. Simultaneously, computational algorithms have evolved to manage the increasing dimensionality of spatial proteomics inherent in this progress. Numerous imaging-based computational frameworks, such as computational pathology, have been proposed for research and clinical applications. However, the development of these fields demands diverse domain expertise, creating barriers to their integration and further application. This review seeks to bridge this divide by presenting a comprehensive guideline. We consolidate prevailing computational methods and outline a roadmap from image processing to data-driven, statistics-informed biomarker discovery. Additionally, we explore future perspectives as the field moves toward interfacing with other quantitative domains, holding significant promise for precision care in immuno-oncology.

A panoptic segmentation dataset and deep-learning approach for explainable scoring of tumor-infiltrating lymphocytes

Tumor-Infiltrating Lymphocytes (TILs) have strong prognostic and predictive value in breast cancer, but their visual assessment is subjective. To improve reproducibility, the International Immuno-oncology Working Group recently released recommendations for the computational assessment of TILs that build on visual scoring guidelines. However, existing resources do not adequately address these recommendations due to the lack of annotation datasets that enable joint, panoptic segmentation of tissue regions and cells. Moreover, existing deep-learning methods focus entirely on either tissue segmentation or cell nuclei detection, which complicates the process of TILs assessment by necessitating the use of multiple models and reconciling inconsistent predictions. We introduce PanopTILs, a region and cell-level annotation dataset containing 814,886 nuclei from 151 patients, openly accessible at: sites.google.com/view/panoptils . Using PanopTILs we developed MuTILs, a neural network optimized for assessing TILs in accordance with clinical recommendations. MuTILs is a concept bottleneck model designed to be interpretable and to encourage sensible predictions at multiple resolutions. Using a rigorous internal-external cross-validation procedure, MuTILs achieves an AUROC of 0.93 for lymphocyte detection and a DICE coefficient of 0.81 for tumor-associated stroma segmentation. Our computational score closely matched visual scores from 2 pathologists (Spearman R = 0.58-0.61, p < 0.001). Moreover, computational TILs scores had a higher prognostic value than visual scores, independent of TNM stage and patient age. In conclusion, we introduce a comprehensive open data resource and a modeling approach for detailed mapping of the breast tumor microenvironment.

A fully automated and explainable algorithm for predicting malignant transformation in oral epithelial dysplasia

Oral epithelial dysplasia (OED) is a premalignant histopathological diagnosis given to lesions of the oral cavity. Its grading suffers from significant inter-/intra-observer variability, and does not reliably predict malignancy progression, potentially leading to suboptimal treatment decisions. To address this, we developed an artificial intelligence (AI) algorithm, that assigns an Oral Malignant Transformation (OMT) risk score based on the Haematoxylin and Eosin (H&E) stained whole slide images (WSIs). Our AI pipeline leverages an in-house segmentation model to detect and segment both nuclei and epithelium. Subsequently, a shallow neural network utilises interpretable morphological and spatial features, emulating histological markers, to predict progression. We conducted internal cross-validation on our development cohort (Sheffield; n = 193 cases) and independent validation on two external cohorts (Birmingham and Belfast; n = 89 cases). On external validation, the proposed OMTscore achieved an AUROC = 0.75 (Recall = 0.92) in predicting OED progression, outperforming other grading systems (Binary: AUROC = 0.72, Recall = 0.85). Survival analyses showed the prognostic value of our OMTscore (C-index = 0.60, p = 0.02), compared to WHO (C-index = 0.64, p = 0.003) and binary grades (C-index = 0.65, p < 0.001). Nuclear analyses elucidated the presence of peri-epithelial and intra-epithelial lymphocytes in highly predictive patches of transforming cases (p < 0.001). This is the first study to propose a completely automated, explainable, and externally validated algorithm for predicting OED transformation. Our algorithm shows comparable-to-human-level performance, offering a promising solution to the challenges of grading OED in routine clinical practice.

Exploring the association of POSTN+ cancer-associated fibroblasts with triple-negative breast cancer

Triple-negative breast cancer (TNBC) is a subtype of breast cancer with a poor prognosis. Cancer-associated fibroblasts (CAFs) play a critical role in regulating TNBC tumor development. This study aimed to identify and characterize a specific subtype of CAFs associated with TNBC. Initially, using high-throughput bulk transcriptomic data in two cohorts, we identified three CAF-related subtypes (CS1, CS2, CS3) in TNBC samples. These three CAFs subtypes were closely linked to the tumor microenvironment. The CS1 subtype exhibited a relatively immune-rich microenvironment and a favourable prognosis, whereas the CS3 subtype displayed an immune-deprived tumor microenvironment and an unfavourable prognosis. Through WGCNA analysis, POSTN was identified as a key biomarker for CAFs associated with TNBC. Then, POSTN+CAFs was identified and characterized. Both POSTN and POSTN+CAFs showed significant positive correlations with stromal molecules HGF and MET at both the transcriptional and protein levels. Specifically co-localized with CAFs in the tumor stromal area, POSTN, produced by POSTN+CAFs, could modulate the HGF-MET axis, serving as a bypass activation pathway to regulate tumor cell proliferation in response to EGFR inhibitor and MET inhibitor. This study underscores the significance of POSTN and POSTN+CAFs as crucial targets for the diagnosis and treatment of TNBC.

tRigon: an R package and Shiny App for integrative (path-)omics data analysis

BACKGROUND

Pathomics facilitates automated, reproducible and precise histopathology analysis and morphological phenotyping. Similar to molecular omics, pathomics datasets are high-dimensional, but also face large outlier variability and inherent data missingness, making quick and comprehensible data analysis challenging. To facilitate pathomics data analysis and interpretation as well as support a broad implementation we developed tRigon (Toolbox foR InteGrative (path-)Omics data aNalysis), a Shiny application for fast, comprehensive and reproducible pathomics analysis.

RESULTS

tRigon is available via the CRAN repository ( https://cran.r-project.org/web/packages/tRigon ) with its source code available on GitLab ( https://git-ce.rwth-aachen.de/labooratory-ai/trigon ). The tRigon package can be installed locally and its application can be executed from the R console via the command 'tRigon::run_tRigon()'. Alternatively, the application is hosted online and can be accessed at https://labooratory.shinyapps.io/tRigon . We show fast computation of small, medium and large datasets in a low- and high-performance hardware setting, indicating broad applicability of tRigon.

CONCLUSIONS

tRigon allows researchers without coding abilities to perform exploratory feature analyses of pathomics and non-pathomics datasets on their own using a variety of hardware.

A population-level digital histologic biomarker for enhanced prognosis of invasive breast cancer

Breast cancer is a heterogeneous disease with variable survival outcomes. Pathologists grade the microscopic appearance of breast tissue using the Nottingham criteria, which are qualitative and do not account for noncancerous elements within the tumor microenvironment. Here we present the Histomic Prognostic Signature (HiPS), a comprehensive, interpretable scoring of the survival risk incurred by breast tumor microenvironment morphology. HiPS uses deep learning to accurately map cellular and tissue structures to measure epithelial, stromal, immune, and spatial interaction features. It was developed using a population-level cohort from the Cancer Prevention Study-II and validated using data from three independent cohorts, including the Prostate, Lung, Colorectal, and Ovarian Cancer trial, Cancer Prevention Study-3, and The Cancer Genome Atlas. HiPS consistently outperformed pathologists in predicting survival outcomes, independent of tumor-node-metastasis stage and pertinent variables. This was largely driven by stromal and immune features. In conclusion, HiPS is a robustly validated biomarker to support pathologists and improve patient prognosis.

AI-based intra-tumor heterogeneity score of Ki67 expression as a prognostic marker for early-stage ER+/HER2- breast cancer

Early-stage estrogen receptor positive and human epidermal growth factor receptor negative (ER+/HER2-) luminal breast cancer (BC) is quite heterogeneous and accounts for about 70% of all BCs. Ki67 is a proliferation marker that has a significant prognostic value in luminal BC despite the challenges in its assessment. There is increasing evidence that spatial colocalization, which measures the evenness of different types of cells, is clinically important in several types of cancer. However, reproducible quantification of intra-tumor spatial heterogeneity remains largely unexplored. We propose an automated pipeline for prognostication of luminal BC based on the analysis of spatial distribution of Ki67 expression in tumor cells using a large well-characterized cohort (n = 2,081). The proposed Ki67 colocalization (Ki67CL) score can stratify ER+/HER2- BC patients with high significance in terms of BC-specific survival (p < 0.00001) and distant metastasis-free survival (p = 0.0048). Ki67CL score is shown to be highly significant compared with the standard Ki67 index. In addition, we show that the proposed Ki67CL score can help identify luminal BC patients who can potentially benefit from adjuvant chemotherapy.

Toward a generalizable machine learning workflow for neurodegenerative disease staging with focus on neurofibrillary tangles

Machine learning (ML) has increasingly been used to assist and expand current practices in neuropathology. However, generating large imaging datasets with quality labels is challenging in fields which demand high levels of expertise. Further complicating matters is the often seen disagreement between experts in neuropathology-related tasks, both at the case level and at a more granular level. Neurofibrillary tangles (NFTs) are a hallmark pathological feature of Alzheimer disease, and are associated with disease progression which warrants further investigation and granular quantification at a scale not currently accessible in routine human assessment. In this work, we first provide a baseline of annotator/rater agreement for the tasks of Braak NFT staging between experts and NFT detection using both experts and novices in neuropathology. We use a whole-slide-image (WSI) cohort of neuropathology cases from Emory University Hospital immunohistochemically stained for Tau. We develop a workflow for gathering annotations of the early stage formation of NFTs (Pre-NFTs) and mature intracellular (iNFTs) and show ML models can be trained to learn annotator nuances for the task of NFT detection in WSIs. We utilize a model-assisted-labeling approach and demonstrate ML models can be used to aid in labeling large datasets efficiently. We also show these models can be used to extract case-level features, which predict Braak NFT stages comparable to expert human raters, and do so at scale. This study provides a generalizable workflow for various pathology and related fields, and also provides a technique for accomplishing a high-level neuropathology task with limited human annotations.

Personalizable AI platform for universal access to research and diagnosis in digital pathology

BACKGROUND AND MOTIVATION

Digital pathology has been evolving over the last years, proposing significant workflow advantages that have fostered its adoption in professional environments. Patient clinical and image data are readily available in remote data banks that can be consumed efficiently over standard communication technologies. The appearance of new imaging techniques and advanced artificial intelligence algorithms has significantly reduced the burden on medical professionals by speeding up the screening process. Despite these advancements, the usage of digital pathology in professional environments has been slowed down by poor interoperability between services resulting from a lack of standard interfaces and integrative solutions. This work addresses this issue by proposing a cloud-based digital pathology platform built on standard and open interfaces.

METHODS

The work proposes and describes a vendor-neutral platform that provides interfaces for managing digital slides, and medical reports, and integrating digital image analysis services compatible with existing standards. The solution integrates the open-source plugin-based Dicoogle PACS for interoperability and extensibility, which grants the proposed solution great feature customization.

RESULTS

The solution was developed in collaboration with iPATH research project partners, including the validation by medical pathologists. The result is a pure Web collaborative framework that supports both research and production environments. A total of 566 digital slides from different pathologies were successfully uploaded to the platform. Using the integration interfaces, a mitosis detection algorithm was successfully installed into the platform, and it was trained with 2400 annotations collected from breast carcinoma images.

CONCLUSION

Interoperability is a key factor when discussing digital pathology solutions, as it facilitates their integration into existing institutions' information systems. Moreover, it improves data sharing and integration of third-party services such as image analysis services, which have become relevant in today's digital pathology workflow. The proposed solution fully embraces the DICOM standard for digital pathology, presenting an interoperable cloud-based solution that provides great feature customization thanks to its extensible architecture.

Correlating Deep Learning-Based Automated Reference Kidney Histomorphometry with Patient Demographics and Creatinine

Key Points

The authors leverage the unique benefits of panoptic segmentation to perform the largest ever quantitation of reference kidney morphometry. Kidney features vary with age and sex; and glomeruli size may intricately link to creatinine, defying prior notions.

Background

Reference histomorphometric data of healthy human kidneys are largely lacking because of laborious quantitation requirements. Correlating histomorphometric features with clinical parameters through machine learning approaches can provide valuable information about natural population variance. To this end, we leveraged deep learning (DL), computational image analysis, and feature analysis to associate the relationship of histomorphometry with patient age, sex, serum creatinine (SCr), and eGFR in a multinational set of reference kidney tissue sections.

Methods

A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in the digitized images of 79 periodic acid–Schiff-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g. , area, radius, density) were quantified from the segmented classes. Regression analysis aided in determining the association of histomorphometric parameters with age, sex, SCr, and eGFR.

Results

Our DL model achieved high segmentation performance for all test compartments. The size and density of glomeruli, tubules, and arteries/arterioles varied significantly among healthy humans, with potentially large differences between geographically diverse patients. Glomerular size was significantly correlated with SCr and eGFR. Slight, albeit significant, differences in renal vasculature were observed between sexes. Glomerulosclerosis percentage increased, and cortical density of arteries/arterioles decreased, as a function of increasing age.

Conclusions

Using DL, we automated precise measurements of kidney histomorphometric features. In the reference kidney tissue, several histomorphometric features demonstrated significant correlation to patient demographics, SCr, and eGFR. DL tools can increase the efficiency and rigor of histomorphometric analysis.

Natural Language Processing to extract SNOMED-CT codes from pathological reports

OBJECTIVE

The use of standardized structured reports (SSR) and suitable terminologies like SNOMED-CT can enhance data retrieval and analysis, fostering large-scale studies and collaboration. However, the still large prevalence of narrative reports in our laboratories warrants alternative and automated labeling approaches. In this project, natural language processing (NLP) methods were used to associate SNOMED-CT codes to structured and unstructured reports from an Italian Digital Pathology Department.

METHODS

Two NLP-based automatic coding systems (support vector machine, SVM, and long-short term memory, LSTM) were trained and applied to a series of narrative reports.

RESULTS

The 1163 cases were tested with both algorithms, showing good performances in terms of accuracy, precision, recall, and F1 score, with SVM showing slightly better performances as compared to LSTM (0.84, 0.87, 0.83, 0.82 vs 0.83, 0.85, 0.83, 0.82, respectively). The integration of an explainability allowed identification of terms and groups of words of importance, enabling fine-tuning, balancing semantic meaning and model performance.

CONCLUSIONS

AI tools allow the automatic SNOMED-CT labeling of the pathology archives, providing a retrospective fix to the large lack of organization of narrative reports.

Accelerating pharmaceutical R&D with a user-friendly AI system for histopathology image analysis

A system for analysis of histopathology data within a pharmaceutical R&D environment has been developed with the intention of enabling interdisciplinary collaboration. State-of-the-art AI tools have been deployed as easy-to-use self-service modules within an open-source whole slide image viewing platform, so that non-data scientist users (e.g., clinicians) can utilize and evaluate pre-trained algorithms and retrieve quantitative results. The outputs of analysis are automatically cataloged in the database to track data provenance and can be viewed interactively on the slide as annotations or heatmaps. Commonly used models for analysis of whole slide images including segmentation, extraction of hand-engineered features for segmented regions, and slide-level classification using multi-instance learning are included and new models can be added as needed. The source code that supports running inference with these models internally is backed up by a robust CI/CD pipeline to ensure model versioning, robust testing, and seamless deployment of the latest models. Examples of the use of this system in a pharmaceutical development workflow include glomeruli segmentation, enumeration of podocyte count from WT-1 immuno-histochemistry, measurement of beta-1 integrin target engagement from immunofluorescence, digital glomerular phenotyping from periodic acid-Schiff histology, PD-L1 score prediction using multi-instance learning, and the deployment of the open-source Segment Anything model to speed up annotation.

Deep learning in computational dermatopathology of melanoma: A technical systematic literature review

Deep learning (DL) has become one of the major approaches in computational dermatopathology, evidenced by a significant increase in this topic in the current literature. We aim to provide a structured and comprehensive overview of peer-reviewed publications on DL applied to dermatopathology focused on melanoma. In comparison to well-published DL methods on non-medical images (e.g., classification on ImageNet), this field of application comprises a specific set of challenges, such as staining artifacts, large gigapixel images, and various magnification levels. Thus, we are particularly interested in the pathology-specific technical state-of-the-art. We also aim to summarize the best performances achieved thus far with respect to accuracy, along with an overview of self-reported limitations. Accordingly, we conducted a systematic literature review of peer-reviewed journal and conference articles published between 2012 and 2022 in the databases ACM Digital Library, Embase, IEEE Xplore, PubMed, and Scopus, expanded by forward and backward searches to identify 495 potentially eligible studies. After screening for relevance and quality, a total of 54 studies were included. We qualitatively summarized and analyzed these studies from technical, problem-oriented, and task-oriented perspectives. Our findings suggest that the technical aspects of DL for histopathology in melanoma can be further improved. The DL methodology was adopted later in this field, and still lacks the wider adoption of DL methods already shown to be effective for other applications. We also discuss upcoming trends toward ImageNet-based feature extraction and larger models. While DL has achieved human-competitive accuracy in routine pathological tasks, its performance on advanced tasks is still inferior to wet-lab testing (for example). Finally, we discuss the challenges impeding the translation of DL methods to clinical practice and provide insight into future research directions.

The Comparative Pathology Workbench: Interactive visual analytics for biomedical data

Pathologists need to compare histopathological images of normal and diseased tissues between different samples, cases, and species. We have designed an interactive system, termed Comparative Pathology Workbench (CPW), which allows direct and dynamic comparison of images at a variety of magnifications, selected regions of interest, as well as the results of image analysis or other data analyses such as scRNA-seq. This allows pathologists to indicate key diagnostic features, with a mechanism to allow discussion threads amongst expert groups of pathologists and other disciplines. The data and associated discussions can be accessed online from anywhere in the world. The Comparative Pathology Workbench (CPW) is a web-browser-based visual analytics platform providing shared access to an interactive "spreadsheet" style presentation of image and associated analysis data. The CPW provides a grid layout of rows and columns so that images that correspond to matching data can be organised in the form of an image-enabled "spreadsheet". An individual workbench can be shared with other users with read-only or full edit access as required. In addition, each workbench element or the whole bench itself has an associated discussion thread to allow collaborative analysis and consensual interpretation of the data. The CPW is a Django-based web-application that hosts the workbench data, manages users, and user-preferences. All image data are hosted by other resource applications such as OMERO or the Digital Slide Archive. Further resources can be added as required. The discussion threads are managed using WordPress and include additional graphical and image data. The CPW has been developed to allow integration of image analysis outputs from systems such as QuPath or ImageJ. All software is open-source and available from a GitHub repository.

SliDL: A toolbox for processing whole-slide images in deep learning

The inspection of stained tissue slides by pathologists is essential for the early detection, diagnosis and monitoring of disease. Recently, deep learning methods for the analysis of whole-slide images (WSIs) have shown excellent performance on these tasks, and have the potential to substantially reduce the workload of pathologists. However, WSIs present a number of unique challenges for analysis, requiring special consideration of image annotations, slide and image artefacts, and evaluation of WSI-trained model performance. Here we introduce SliDL, a Python library for performing pre- and post-processing of WSIs. SliDL makes WSI data handling easy, allowing users to perform essential processing tasks in a few simple lines of code, bridging the gap between standard image analysis and WSI analysis. We introduce each of the main functionalities within SliDL: from annotation and tile extraction to tissue detection and model evaluation. We also provide 'code snippets' to guide the user in running SliDL. SliDL has been designed to interact with PyTorch, one of the most widely used deep learning libraries, allowing seamless integration into deep learning workflows. By providing a framework in which deep learning methods for WSI analysis can be developed and applied, SliDL aims to increase the accessibility of an important application of deep learning.

Artificial Intelligence Predictive Model for Hormone Therapy Use in Prostate Cancer

BACKGROUND: Androgen deprivation therapy (ADT) with radiotherapy can benefit patients with localized prostate cancer. However, ADT can negatively impact quality of life, and there remain no validated predictive models to guide its use. METHODS: We used digital pathology images from pretreatment prostate tissue and clinical data from 5727 patients enrolled in five phase 3 randomized trials, in which treatment was radiotherapy with or without ADT, as our data source to develop and validate an artificial intelligence (AI)–derived predictive patient-specific model that would determine which patients would develop the primary end point of distant metastasis. The model used baseline data to provide a binary output that a given patient will likely benefit from ADT or not. After the model was locked, validation was performed using data from NRG Oncology/Radiation Therapy Oncology Group (RTOG) 9408 (n=1594), a trial that randomly assigned men to radiotherapy plus or minus 4 months of ADT. Fine–Gray regression and restricted mean survival times were used to assess the interaction between treatment and the predictive model and within predictive model–positive, i.e., benefited from ADT, and –negative subgroup treatment effects. RESULTS: Overall, in the NRG/RTOG 9408 validation cohort (14.9 years of median follow-up), ADT significantly improved time to distant metastasis. Of these enrolled patients, 543 (34%) were model positive, and ADT significantly reduced the risk of distant metastasis compared with radiotherapy alone. Of 1051 patients who were model negative, ADT did not provide benefit. CONCLUSIONS: Our AI-based predictive model was able to identify patients with a predominantly intermediate risk for prostate cancer likely to benefit from short-term ADT. (Supported by a grant [U10CA180822] from NRG Oncology Statistical and Data Management Center, a grant [UG1CA189867] from NCI Community Oncology Research Program, a grant [U10CA180868] from NRG Oncology Operations, and a grant [U24CA196067] from NRG Specimen Bank from the National Cancer Institute and by Artera, Inc. ClinicalTrials.gov numbers NCT00767286, NCT00002597, NCT00769548, NCT00005044, and NCT00033631.)

Leveraging the Academic Artificial Intelligence Silecosystem to Advance the Community Oncology Enterprise

Over the last 75 years, artificial intelligence has evolved from a theoretical concept and novel paradigm describing the role that computers might play in our society to a tool with which we daily engage. In this review, we describe AI in terms of its constituent elements, the synthesis of which we refer to as the AI Silecosystem. Herein, we provide an historical perspective of the evolution of the AI Silecosystem, conceptualized and summarized as a Kuhnian paradigm. This manuscript focuses on the role that the AI Silecosystem plays in oncology and its emerging importance in the care of the community oncology patient. We observe that this important role arises out of a unique alliance between the academic oncology enterprise and community oncology practices. We provide evidence of this alliance by illustrating the practical establishment of the AI Silecosystem at the City of Hope Comprehensive Cancer Center and its team utilization by community oncology providers.

A correlation graph attention network for classifying chromosomal instabilities from histopathology whole-slide images

The chromosome instability (CIN) is one of the hallmarks of cancer and is closely related to tumor metastasis. However, the sheer size and resolution of histopathology whole-slide images (WSIs) already challenges the capabilities of computational pathology. In this study, we propose a correlation graph attention network (MLP-GAT) that can construct graphs for classifying multi-type CINs from the WSIs of breast cancer. We construct a WSIs dataset of breast cancer from the Cancer Genome Atlas Breast Invasive Carcinoma (TCGA-BRCA). Extensive experiments show that MLP-GAT far outperforms accepted state-of-the-art methods and demonstrate the advantages of the constructed graph networks for analyzing WSI data. The visualization shows the difference among the tiles in a WSI. Furthermore, the generalization performance of the proposed method was verified on the stomach cancer. This study provides guidance for studying the relationship between CIN and cancer from the perspective of image phenotype.

Automated Curation and AI Workflow Management System for Digital Pathology

Digital pathology applications present several challenges, including the processing, storage, and distribution of gigapixel images across distributed computational resources and viewing stations. Individual slides must be available for interactive review, and large repositories must be programmatically accessible for dataset and model building. We present a platform to manage and process multi-modal pathology data (images and case information) across multiple locations. Using an agent-based system coupled with open-source automated machine learning and review tools allows not only dynamic load-balancing and cross-network operation but also the development of research and clinical AI models using the data managed by the platform. The platform presented covers end-to-end AI workflow from data acquisition and curation through model training and evaluation allowing for sharing and review. We conclude with a case study of colon and prostate cancer model development utilizing the presented system.

Correlating Deep Learning-Based Automated Reference Kidney Histomorphometry with Patient Demographics and Creatinine

Background

Reference histomorphometric data of healthy human kidneys are largely lacking due to laborious quantitation requirements. Correlating histomorphometric features with clinical parameters through machine learning approaches can provide valuable information about natural population variance. To this end, we leveraged deep learning, computational image analysis, and feature analysis to investigate the relationship of histomorphometry with patient age, sex, and serum creatinine (SCr) in a multinational set of reference kidney tissue sections.

Methods

A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in the digitized images of 79 periodic acid-Schiff-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g., area, radius, density) were quantified from the segmented classes. Regression analysis aided in determining the relationship of histomorphometric parameters with age, sex, and SCr.

Results

Our deep-learning model achieved high segmentation performance for all test compartments. The size and density of nephrons and arteries/arterioles varied significantly among healthy humans, with potentially large differences between geographically diverse patients. Nephron size was significantly dependent on SCr. Slight, albeit significant, differences in renal vasculature were observed between sexes. Glomerulosclerosis percentage increased, and cortical density of arteries/arterioles decreased, as a function of age.

Conclusions

Using deep learning, we automated precise measurements of kidney histomorphometric features. In the reference kidney tissue, several histomorphometric features demonstrated significant correlation to patient demographics and SCr. Deep learning tools can increase the efficiency and rigor of histomorphometric analysis.

Artificial Intelligence Predictive Model for Hormone Therapy Use in Prostate Cancer

Background

Androgen deprivation therapy (ADT) with radiotherapy can benefit patients with localized prostate cancer. However, ADT can negatively impact quality of life and there remain no validated predictive models to guide its use.

Methods

Digital pathology image and clinical data from pre-treatment prostate tissue from 5,727 patients enrolled on five phase III randomized trials treated with radiotherapy +/- ADT were used to develop and validate an artificial intelligence (AI)-derived predictive model to assess ADT benefit with the primary endpoint of distant metastasis. After the model was locked, validation was performed on NRG/RTOG 9408 (n = 1,594) that randomized men to radiotherapy +/- 4 months of ADT. Fine-Gray regression and restricted mean survival times were used to assess the interaction between treatment and predictive model and within predictive model positive and negative subgroup treatment effects.

Results

In the NRG/RTOG 9408 validation cohort (14.9 years of median follow-up), ADT significantly improved time to distant metastasis (subdistribution hazard ratio [sHR] = 0.64, 95%CI [0.45-0.90], p = 0.01). The predictive model-treatment interaction was significant (p-interaction = 0.01). In predictive model positive patients (n = 543, 34%), ADT significantly reduced the risk of distant metastasis compared to radiotherapy alone (sHR = 0.34, 95%CI [0.19-0.63], p < 0.001). There were no significant differences between treatment arms in the predictive model negative subgroup (n = 1,051, 66%; sHR = 0.92, 95%CI [0.59-1.43], p = 0.71).

Conclusions

Our data, derived and validated from completed randomized phase III trials, show that an AI-based predictive model was able to identify prostate cancer patients, with predominately intermediate-risk disease, who are likely to benefit from short-term ADT.

Machine learning classification of placental villous infarction, perivillous fibrin deposition, and intervillous thrombus

INTRODUCTION

Placental parenchymal lesions are commonly encountered and carry significant clinical associations. However, they are frequently missed or misclassified by general practice pathologists. Interpretation of pathology slides has emerged as one of the most successful applications of machine learning (ML) in medicine with applications ranging from cancer detection and prognostication to transplant medicine. The goal of this study was to use a whole-slide learning model to identify and classify placental parenchymal lesions including villous infarctions, intervillous thrombi (IVT), and perivillous fibrin deposition (PVFD).

METHODS

We generated whole slide images from placental discs examined at our institution with infarct, IVT, PVFD, or no macroscopic lesion. Slides were analyzed as a set of overlapping patches. We extracted feature vectors from each patch using a pretrained convolutional neural network (EfficientNetV2L). We trained a model to assign attention to each vector and used the attentions as weights to produce a pooled feature vector. The pooled vector was classified as normal or 1 of 3 lesions using a fully connected network. Patch attention was plotted to highlight informative areas of the slide.

RESULTS

Overall balanced accuracy in a test set of held-out slides was 0.86 with receiver-operator characteristic areas under the curve of 0.917-0.993. Cases of PVFD were frequently miscalled as normal or infarcts, the latter possibly due to the perivillous fibrin found at the periphery of infarctions. We used attention maps to further understand some errors, including one most likely due to poor tissue fixation and processing.

DISCUSSION

We used a whole-slide learning paradigm to train models to recognize three of the most common placental parenchymal lesions. We used attention maps to gain insight into model function, which differed from intuitive explanations.

Computational Approaches Drive Developments in Immune-Oncology Therapies for PD-1/PD-L1 Immune Checkpoint Inhibitors

Computational approaches in immune-oncology therapies focus on using data-driven methods to identify potential immune targets and develop novel drug candidates. In particular, the search for PD-1/PD-L1 immune checkpoint inhibitors (ICIs) has enlivened the field, leveraging the use of cheminformatics and bioinformatics tools to analyze large datasets of molecules, gene expression and protein-protein interactions. Up to now, there is still an unmet clinical need for improved ICIs and reliable predictive biomarkers. In this review, we highlight the computational methodologies applied to discovering and developing PD-1/PD-L1 ICIs for improved cancer immunotherapies with a greater focus in the last five years. The use of computer-aided drug design structure- and ligand-based virtual screening processes, molecular docking, homology modeling and molecular dynamics simulations methodologies essential for successful drug discovery campaigns focusing on antibodies, peptides or small-molecule ICIs are addressed. A list of recent databases and web tools used in the context of cancer and immunotherapy has been compilated and made available, namely regarding a general scope, cancer and immunology. In summary, computational approaches have become valuable tools for discovering and developing ICIs. Despite significant progress, there is still a need for improved ICIs and biomarkers, and recent databases and web tools have been compiled to aid in this pursuit.

Artificial intelligence and digital biomarker in precision pathology guiding immune therapy selection and precision oncology

BACKGROUND

The currently available immunotherapies already changed the strategy how many cancers are treated from first to last line. Understanding even the most complex heterogeneity in tumor tissue and mapping the spatial cartography of the tumor immunity allows the best and optimized selection of immune modulating agents to (re-)activate the patient's immune system and direct it against the individual cancer in the most effective way.

RECENT FINDINGS

Primary cancer and metastases maintain a high degree of plasticity to escape any immune surveillance and continue to evolve depending on many intrinsic and extrinsic factors In the field of immune-oncology (IO) immune modulating agents are recognized as practice changing therapeutic modalities. Recent studies have shown that an optimal and lasting efficacy of IO therapeutics depends on the understanding of the spatial communication network and functional context of immune and cancer cells within the tumor microenvironment. Artificial intelligence (AI) provides an insight into the immune-cancer-network through the visualization of very complex tumor and immune interactions in cancer tissue specimens and allows the computer-assisted development and clinical validation of such digital biomarker.

CONCLUSIONS

The successful implementation of AI-supported digital biomarker solutions guides the clinical selection of effective immune therapeutics based on the retrieval and visualization of spatial and contextual information from cancer tissue images and standardized data. As such, computational pathology (CP) turns into "precision pathology" delivering individual therapy response prediction. Precision Pathology does not only include digital and computational solutions but also high levels of standardized processes in the routine histopathology workflow and the use of mathematical tools to support clinical and diagnostic decisions as the basic principle of a "precision oncology".

Histology Image Viewer and Converter (HIVC): A High-Speed Freeware Software to View and Convert Whole Slide Histology Images

Histology images are widely used to assess the microstructure of biological tissues, but scanners often save images in bulky SVS and multi-layered TIFF formats. These formats were designed to archive image blocks and high-resolution textual information and are not compatible with conventional image analysis software. Our goal was to create a freeware Histology Image Viewer and Converter (HIVC) with a graphical user interface that allows viewing and converting whole-slide images in batch. HIVC was developed using C# Language for Windows x64 operating system. HIVC's performance was assessed by converting 20 whole-slide images to a JPG format at 20x and 40x resolution and comparing the results to ImageJ, Cell Profiler, QuPath, Nanoborb, and Aperio ImageScope. HIVC was more than 8-times faster in converting images than other software packages. This software allows high-speed batch conversion of histology images to traditional formats, permitting platform-independent secondary analyses.

Hardware-software co-design of an open-source automatic multimodal whole slide histopathology imaging system

Significance

Advanced digital control of microscopes and programmable data acquisition workflows have become increasingly important for improving the throughput and reproducibility of optical imaging experiments. Combinations of imaging modalities have enabled a more comprehensive understanding of tissue biology and tumor microenvironments in histopathological studies. However, insufficient imaging throughput and complicated workflows still limit the scalability of multimodal histopathology imaging.

Aim

We present a hardware-software co-design of a whole slide scanning system for high-throughput multimodal tissue imaging, including brightfield (BF) and laser scanning microscopy.

Approach

The system can automatically detect regions of interest using deep neural networks in a low-magnification rapid BF scan of the tissue slide and then conduct high-resolution BF scanning and laser scanning imaging on targeted regions with deep learning-based run-time denoising and resolution enhancement. The acquisition workflow is built using Pycro-Manager, a Python package that bridges hardware control libraries of the Java-based open-source microscopy software Micro-Manager in a Python environment.

Results

The system can achieve optimized imaging settings for both modalities with minimized human intervention and speed up the laser scanning by an order of magnitude with run-time image processing.

Conclusions

The system integrates the acquisition pipeline and data analysis pipeline into a single workflow that improves the throughput and reproducibility of multimodal histopathological imaging.

An Automated Pipeline for Differential Cell Counts on Whole-Slide Bone Marrow Aspirate Smears

The pathologic diagnosis of bone marrow disorders relies in part on the microscopic analysis of bone marrow aspirate (BMA) smears and the manual counting of marrow nucleated cells to obtain a differential cell count (DCC). This manual process has significant limitations, including the analysis of only a small subset of optimal slide areas and nucleated cells, as well as interobserver variability due to differences in cell selection and classification. To address these shortcomings, we developed an automated machine learning-based pipeline for obtaining 11-component DCCs on whole-slide BMAs. This pipeline uses a sequential process of identifying optimal BMA regions with high proportions of marrow nucleated cells, detecting individual cells within these optimal areas, and classifying these cells into 1 of 11 DCC components. Convolutional neural network models were trained on 396,048 BMA region, 28,914 cell boundary, and 1,510,976 cell class images from manual annotations. The resulting automated pipeline produced 11-component DCCs that demonstrated a high statistical and diagnostic concordance with manual DCCs among a heterogeneous group of testing BMA slides with varying pathologies and cellularities. Additionally, we demonstrated that an automated analysis can reduce the intraslide variance in DCCs by analyzing the whole slide and marrow nucleated cells within all optimal regions. Finally, the pipeline outputs of region classification, cell detection, and cell classification can be visualized using whole-slide image analysis software. This study demonstrates the feasibility of a fully automated pipeline for generating DCCs on scanned whole-slide BMA images, with the potential for improving the current standard of practice for utilizing BMA smears in the laboratory analysis of hematologic disorders.

Automated Reference Kidney Histomorphometry using a Panoptic Segmentation Neural Network Correlates to Patient Demographics and Creatinine

Reference histomorphometric data of healthy human kidneys are lacking due to laborious quantitation requirements. We leveraged deep learning to investigate the relationship of histomorphometry with patient age, sex, and serum creatinine in a multinational set of reference kidney tissue sections. A panoptic segmentation neural network was developed and used to segment viable and sclerotic glomeruli, cortical and medullary interstitia, tubules, and arteries/arterioles in digitized images of 79 periodic acid-Schiff (PAS)-stained human nephrectomy sections showing minimal pathologic changes. Simple morphometrics (e.g., area, radius, density) were measured from the segmented classes. Regression analysis was used to determine the relationship of histomorphometric parameters with age, sex, and serum creatinine. The model achieved high segmentation performance for all test compartments. We found that the size and density of nephrons, arteries/arterioles, and the baseline level of interstitium vary significantly among healthy humans, with potentially large differences between subjects from different geographic locations. Nephron size in any region of the kidney was significantly dependent on patient creatinine. Slight differences in renal vasculature and interstitium were observed between sexes. Finally, glomerulosclerosis percentage increased and cortical density of arteries/arterioles decreased as a function of age. We show that precise measurements of kidney histomorphometric parameters can be automated. Even in reference kidney tissue sections with minimal pathologic changes, several histomorphometric parameters demonstrated significant correlation to patient demographics and serum creatinine. These robust tools support the feasibility of deep learning to increase efficiency and rigor in histomorphometric analysis and pave the way for future large-scale studies.

Annotating for Artificial Intelligence Applications in Digital Pathology: A Practical Guide for Pathologists and Researchers

Training machine learning models for artificial intelligence (AI) applications in pathology often requires extensive annotation by human experts, but there is little guidance on the subject. In this work, we aimed to describe our experience and provide a simple, useful, and practical guide addressing annotation strategies for AI development in computational pathology. Annotation methodology will vary significantly depending on the specific study's objectives, but common difficulties will be present across different settings. We summarize key aspects and issue guiding principles regarding team interaction, ground-truth quality assessment, different annotation types, and available software and hardware options and address common difficulties while annotating. This guide was specifically designed for pathology annotation, intending to help pathologists, other researchers, and AI developers with this process.

A Review of Fusion Methods for Omics and Imaging Data

The development of omics data and biomedical images has greatly advanced the progress of precision medicine in diagnosis, treatment, and prognosis. The fusion of omics and imaging data, i.e., omics-imaging fusion, offers a new strategy for understanding complex diseases. However, due to a variety of issues such as the limited number of samples, high dimensionality of features, and heterogeneity of different data types, efficiently learning complementary or associated discriminative fusion information from omics and imaging data remains a challenge. Recently, numerous machine learning methods have been proposed to alleviate these problems. In this review, from the perspective of fusion levels and fusion methods, we first provide an overview of preprocessing and feature extraction methods for omics and imaging data, and comprehensively analyze and summarize the basic forms and variations of commonly used and newly emerging fusion methods, along with their advantages, disadvantages and the applicable scope. We then describe public datasets and compare experimental results of various fusion methods on the ADNI and TCGA datasets. Finally, we discuss future prospects and highlight remaining challenges in the field.

Fibroblast inflammatory priming determines regenerative versus fibrotic skin repair in reindeer

Adult mammalian skin wounds heal by forming fibrotic scars. We report that full-thickness injuries of reindeer antler skin (velvet) regenerate, whereas back skin forms fibrotic scar. Single-cell multi-omics reveal that uninjured velvet fibroblasts resemble human fetal fibroblasts, whereas back skin fibroblasts express inflammatory mediators mimicking pro-fibrotic adult human and rodent fibroblasts. Consequently, injury elicits site-specific immune responses: back skin fibroblasts amplify myeloid infiltration and maturation during repair, whereas velvet fibroblasts adopt an immunosuppressive phenotype that restricts leukocyte recruitment and hastens immune resolution. Ectopic transplantation of velvet to scar-forming back skin is initially regenerative, but progressively transitions to a fibrotic phenotype akin to the scarless fetal-to-scar-forming transition reported in humans. Skin regeneration is diminished by intensifying, or enhanced by neutralizing, these pathologic fibroblast-immune interactions. Reindeer represent a powerful comparative model for interrogating divergent wound healing outcomes, and our results nominate decoupling of fibroblast-immune interactions as a promising approach to mitigate scar.

TIAToolbox as an end-to-end library for advanced tissue image analytics

Background

Computational pathology has seen rapid growth in recent years, driven by advanced deep-learning algorithms. Due to the sheer size and complexity of multi-gigapixel whole-slide images, to the best of our knowledge, there is no open-source software library providing a generic end-to-end API for pathology image analysis using best practices. Most researchers have designed custom pipelines from the bottom up, restricting the development of advanced algorithms to specialist users. To help overcome this bottleneck, we present TIAToolbox, a Python toolbox designed to make computational pathology accessible to computational, biomedical, and clinical researchers.

Methods

By creating modular and configurable components, we enable the implementation of computational pathology algorithms in a way that is easy to use, flexible and extensible. We consider common sub-tasks including reading whole slide image data, patch extraction, stain normalization and augmentation, model inference, and visualization. For each of these steps, we provide a user-friendly application programming interface for commonly used methods and models.

Results

We demonstrate the use of the interface to construct a full computational pathology deep-learning pipeline. We show, with the help of examples, how state-of-the-art deep-learning algorithms can be reimplemented in a streamlined manner using our library with minimal effort.

Conclusions

We provide a usable and adaptable library with efficient, cutting-edge, and unit-tested tools for data loading, pre-processing, model inference, post-processing, and visualization. This enables a range of users to easily build upon recent deep-learning developments in the computational pathology literature.

Interpretable deep learning of myelin histopathology in age-related cognitive impairment

Age-related cognitive impairment is multifactorial, with numerous underlying and frequently co-morbid pathological correlates. Amyloid beta (Aβ) plays a major role in Alzheimer's type age-related cognitive impairment, in addition to other etiopathologies such as Aβ-independent hyperphosphorylated tau, cerebrovascular disease, and myelin damage, which also warrant further investigation. Classical methods, even in the setting of the gold standard of postmortem brain assessment, involve semi-quantitative ordinal staging systems that often correlate poorly with clinical outcomes, due to imperfect cognitive measurements and preconceived notions regarding the neuropathologic features that should be chosen for study. Improved approaches are needed to identify histopathological changes correlated with cognition in an unbiased way. We used a weakly supervised multiple instance learning algorithm on whole slide images of human brain autopsy tissue sections from a group of elderly donors to predict the presence or absence of cognitive impairment (n = 367 with cognitive impairment, n = 349 without). Attention analysis allowed us to pinpoint the underlying subregional architecture and cellular features that the models used for the prediction in both brain regions studied, the medial temporal lobe and frontal cortex. Despite noisy labels of cognition, our trained models were able to predict the presence of cognitive impairment with a modest accuracy that was significantly greater than chance. Attention-based interpretation studies of the features most associated with cognitive impairment in the top performing models suggest that they identified myelin pallor in the white matter. Our results demonstrate a scalable platform with interpretable deep learning to identify unexpected aspects of pathology in cognitive impairment that can be translated to the study of other neurobiological disorders.

A user-friendly tool for cloud-based whole slide image segmentation with examples from renal histopathology

BACKGROUND

Image-based machine learning tools hold great promise for clinical applications in pathology research. However, the ideal end-users of these computational tools (e.g., pathologists and biological scientists) often lack the programming experience required for the setup and use of these tools which often rely on the use of command line interfaces.

METHODS

We have developed Histo-Cloud, a tool for segmentation of whole slide images (WSIs) that has an easy-to-use graphical user interface. This tool runs a state-of-the-art convolutional neural network (CNN) for segmentation of WSIs in the cloud and allows the extraction of features from segmented regions for further analysis.

RESULTS

By segmenting glomeruli, interstitial fibrosis and tubular atrophy, and vascular structures from renal and non-renal WSIs, we demonstrate the scalability, best practices for transfer learning, and effects of dataset variability. Finally, we demonstrate an application for animal model research, analyzing glomerular features in three murine models.

CONCLUSIONS

Histo-Cloud is open source, accessible over the internet, and adaptable for segmentation of any histological structure regardless of stain.

A machine learning model for separating epithelial and stromal regions in oral cavity squamous cell carcinomas using H&E-stained histology images: A multi-center, retrospective study

OBJECTIVE

Tissue slides from Oral cavity squamous cell carcinoma (OC-SCC), particularly the epithelial regions, hold morphologic features that are both diagnostic and prognostic. Yet, previously developed approaches for automated epithelium segmentation in OC-SCC have not been independently tested in a multi-center setting. In this study, we aimed to investigate the effectiveness and applicability of a convolutional neural network (CNN) model to perform epithelial segmentation using digitized H&E-stained diagnostic slides from OC-SCC patients in a multi-center setting.

METHODS

A CNN model was developed to segment the epithelial regions of digitized slides (n = 810), retrospectively collected from five different centers. Deep learning models were trained and validated using well-annotated tissue microarray (TMA) images (n = 212) at various magnifications. The best performing model was locked down and used for independent testing with a total of 478 whole-slide images (WSIs). Manually annotated epithelial regions were used as the reference standard for evaluation. We also compared the model generated results with IHC-stained epithelium (n = 120) as the reference.

RESULTS

The locked-down CNN model trained on the TMA image training cohorts with 10x magnification achieved the best segmentation performance. The locked-down model performed consistently and yielded Pixel Accuracy, Recall Rate, Precision Rate, and Dice Coefficient that ranged from 95.8% to 96.6%, 79.1% to 93.8%, 85.7% to 89.3%, and 82.3% to 89.0%, respectively for the three independent testing WSI cohorts.

CONCLUSION

The automated model achieved a consistently accurate performance for automated epithelial region segmentation compared to manual annotations. This model could be integrated into a computer-aided diagnosis or prognosis system.

Quantitative digital histopathology and machine learning to predict pathological complete response to chemotherapy in breast cancer patients using pre-treatment tumor biopsies

Complete pathological response (pCR) to neoadjuvant chemotherapy (NAC) is a prognostic factor for breast cancer (BC) patients and is correlated with improved survival. However, pCR rates are variable to standard NAC, depending on BC subtype. This study investigates quantitative digital histopathology coupled with machine learning (ML) to predict NAC response a priori. Clinicopathologic data and digitized slides of BC core needle biopsies were collected from 149 patients treated with NAC. The nuclei within the tumor regions were segmented on the histology images of biopsy samples using a weighted U-Net model. Five pathomic feature subsets were extracted from segmented digitized samples, including the morphological, intensity-based, texture, graph-based and wavelet features. Seven ML experiments were conducted with different feature sets to develop a prediction model of therapy response using a gradient boosting machine with decision trees. The models were trained and optimized using a five-fold cross validation on the training data and evaluated using an unseen independent test set. The prediction model developed with the best clinical features (tumor size, tumor grade, age, and ER, PR, HER2 status) demonstrated an area under the ROC curve (AUC) of 0.73. Various pathomic feature subsets resulted in models with AUCs in the range of 0.67 and 0.87, with the best results associated with the graph-based and wavelet features. The selected features among all subsets of the pathomic and clinicopathologic features included four wavelet and three graph-based features and no clinical features. The predictive model developed with these features outperformed the other models, with an AUC of 0.90, a sensitivity of 85% and a specificity of 82% on the independent test set. The results demonstrated the potential of quantitative digital histopathology features integrated with ML methods in predicting BC response to NAC. This study is a step forward towards precision oncology for BC patients to potentially guide future therapies.

Software tools and platforms in Digital Pathology: a review for clinicians and computer scientists

At the end of the twentieth century, a new technology was developed that allowed an entire tissue section to be scanned on an objective slide. Originally called virtual microscopy, this technology is now known as Whole Slide Imaging (WSI). WSI presents new challenges for reading, visualization, storage, and analysis. For this reason, several technologies have been developed to facilitate the handling of these images. In this paper, we analyze the most widely used technologies in the field of digital pathology, ranging from specialized libraries for the reading of these images to complete platforms that allow reading, visualization, and analysis. Our aim is to provide the reader, whether a pathologist or a computational scientist, with the knowledge to choose the technologies to use for new studies, development, or research.

PodoCount: A Robust, Fully Automated, Whole-Slide Podocyte Quantification Tool

Introduction

Podocyte depletion is a histomorphologic indicator of glomerular injury and predicts clinical outcomes. Podocyte estimation methods or podometrics are semiquantitative, technically involved, and laborious. Implementation of high-throughput podometrics in experimental and clinical workflows necessitates an automated podometrics pipeline. Recognizing that computational image analysis offers a robust approach to study cell and tissue structure, we developed and validated PodoCount (a computational tool for automated podocyte quantification in immunohistochemically labeled tissues) using a diverse data set.

Methods

Whole-slide images (WSIs) of tissues immunostained with a podocyte nuclear marker and periodic acid–Schiff counterstain were acquired. The data set consisted of murine whole kidney sections (n = 135) from 6 disease models and human kidney biopsy specimens from patients with diabetic nephropathy (DN) (n = 45). Within segmented glomeruli, podocytes were extracted and image analysis was applied to compute measures of podocyte depletion and nuclear morphometry. Computational performance evaluation and statistical testing were performed to validate podometric and associated image features. PodoCount was disbursed as an open-source, cloud-based computational tool.

Results

PodoCount produced highly accurate podocyte quantification when benchmarked against existing methods. Podocyte nuclear profiles were identified with 0.98 accuracy and segmented with 0.85 sensitivity and 0.99 specificity. Errors in podocyte count were bounded by 1 podocyte per glomerulus. Podocyte-specific image features were found to be significant predictors of disease state, proteinuria, and clinical outcome.

Conclusion

PodoCount offers high-performance podocyte quantitation in diverse murine disease models and in human kidney biopsy specimens. Resultant features offer significant correlation with associated metadata and outcome. Our cloud-based tool will provide end users with a standardized approach for automated podometrics from gigapixel-sized WSIs.

A tool for federated training of segmentation models on whole slide images

The largest bottleneck to the development of convolutional neural network (CNN) models in the computational pathology domain is the collection and curation of diverse training datasets. Training CNNs requires large cohorts of image data, and model generalizability is dependent on training data heterogeneity. Including data from multiple centers enhances the generalizability of CNN-based models, but this is hindered by the logistical challenges of sharing medical data. In this paper, we explore the feasibility of training our recently developed cloud-based segmentation tool (Histo-Cloud) using federated learning. Using a dataset of renal tissue biopsies we show that federated training to segment interstitial fibrosis and tubular atrophy (IFTA) using datasets from three institutions is not found to be different from a training by pooling the data on one server when tested on a fourth (holdout) institution's data. Further, training a model to segment glomeruli for a federated dataset (split by staining) demonstrates similar performance.