Whole-slide imaging and automated image analysis: considerations and opportunities in the practice of pathology

The use of telepathology in veterinary medicine: a scoping review

Telepathology, as a subset of teleconsulting, is pathology interpretation performed at a distance. Telepathology is not a new phenomenon, but since ~2015, significant advances in information technology and telecommunications coupled with the pandemic have led to unprecedented sophistication, accessibility, and use of telepathology in human and veterinary medicine. Furthermore, telepathology can connect veterinary practices to distant laboratories and provide support for underserved animals and communities. Through our scoping review, we provide an overview of how telepathology is being used in veterinary medicine, identify gaps in the literature, and highlight future areas of research and service development. We searched MEDLINE, CAB Abstracts, and the gray literature, and included all relevant literature. Despite the widespread use of digital microscopy in large veterinary diagnostic laboratories, we identified a paucity of literature describing the use of telepathology in veterinary medicine, with a significant gap in studies addressing the validation of whole-slide imaging for primary diagnosis. Underutilization of telepathology to support postmortem examinations conducted in the field was also identified, which indicates a potential area for service development. The use of telepathology is increasing in veterinary medicine, and pathologists must keep pace with the changing technology, ensure the validation of innovative technologies, and identify novel uses to advance the profession.

Deep learning based digital pathology for predicting treatment response to first-line PD-1 blockade in advanced gastric cancer

BACKGROUND

Advanced unresectable gastric cancer (GC) patients were previously treated with chemotherapy alone as the first-line therapy. However, with the Food and Drug Administration's (FDA) 2022 approval of programmed cell death protein 1 (PD-1) inhibitor combined with chemotherapy as the first-li ne treatment for advanced unresectable GC, patients have significantly benefited. However, the significant costs and potential adverse effects necessitate precise patient selection. In recent years, the advent of deep learning (DL) has revolutionized the medical field, particularly in predicting tumor treatment responses. Our study utilizes DL to analyze pathological images, aiming to predict first-line PD-1 combined chemotherapy response for advanced-stage GC.

METHODS

In this multicenter retrospective analysis, Hematoxylin and Eosin (H&E)-stained slides were collected from advanced GC patients across four medical centers. Treatment response was evaluated according to iRECIST 1.1 criteria after a comprehensive first-line PD-1 immunotherapy combined with chemotherapy. Three DL models were employed in an ensemble approach to create the immune checkpoint inhibitors Response Score (ICIsRS) as a novel histopathological biomarker derived from Whole Slide Images (WSIs).

RESULTS

Analyzing 148,181 patches from 313 WSIs of 264 advanced GC patients, the ensemble model exhibited superior predictive accuracy, leading to the creation of ICIsNet. The model demonstrated robust performance across four testing datasets, achieving AUC values of 0.92, 0.95, 0.96, and 1 respectively. The boxplot, constructed from the ICIsRS, reveals statistically significant disparities between the well response and poor response (all p-values < = 0.001).

CONCLUSION

ICIsRS, a DL-derived biomarker from WSIs, effectively predicts advanced GC patients' responses to PD-1 combined chemotherapy, offering a novel approach for personalized treatment planning and allowing for more individualized and potentially effective treatment strategies based on a patient's unique response situations.

Invisible for a few but essential for many: the role of Histotechnologists in the establishment of digital pathology

Digital pathology (DP) is indisputably the future for histopathology laboratories. The process of digital implementation requires deep workflow reorganisation which involves an interdisciplinary team. This transformation may have the greatest impact on the Histotechnologist (HTL) profession. Our review of the literature has clearly revealed that the role of HTLs in the establishment of DP is being unnoticed and guidance is limited. This article aims to bring HTLs from behind-the-scenes into the spotlight. Our objective is to provide them guidance and practical recommendations to successfully contribute to the implementation of a new digital workflow. Furthermore, it also intends to contribute for improvement of study programs, ensuring the role of HTL in DP is addressed as part of graduate and post-graduate education. In our review, we report on the differences encountered between workflow schemes and the limitations observed in this process. The authors propose a digital workflow to achieve its limitless potential, focusing on the HTL's role. This article explores the novel responsibilities of HTLs during specimen gross dissection, embedding, microtomy, staining, digital scanning, and whole slide image quality control. Furthermore, we highlight the benefits and challenges that DP implementation might bring the HTLs career. HTLs have an important role in the digital workflow: the responsibility of achieving the perfect glass slide.

Quantitative and large-format histochemistry to characterize peripheral artery compositional gradients

The femoropopliteal artery (FPA) is a long, flexible vessel that travels down the anteromedial compartment of the thigh as the femoral artery and then behind the kneecap as the popliteal artery. This artery undergoes various degrees of flexion, extension, and torsion during normal walking movements. The FPA is also the most susceptible peripheral artery to atherosclerosis and is where peripheral artery disease manifests in 80% of cases. The connection between peripheral artery location, its mechanical flexion, and its physiological or pathological biochemistry has been investigated for decades; however, histochemical methods remain poorly leveraged in their ability to spatially correlate normal or abnormal extracellular matrix and cells with regions of mechanical flexion. This study generates new histological image processing pipelines to quantitate tissue composition across high-resolution FPA regions-of-interest or low-resolution whole-section cross-sections in relation to their anatomical locations and flexions during normal movement. Comparing healthy ovine femoral, popliteal, and cranial-tibial artery sections as a pilot, substantial arterial contortion was observed in the distal popliteal and cranial tibial regions of the FPA which correlated with increased vascular smooth muscle cells and decreased elastin content. These methods aim to aid in the quantitative characterization of the spatial distribution of extracellular matrix and cells in large heterogeneous tissue sections such as the FPA. RESEARCH HIGHLIGHTS: Large-format histology preserves artery architecture. Elastin and smooth muscle content is correlated with distance from heart and contortion during flexion. Cell and protein analyses are sensitive to sectioning plane and image magnification.

Recent developments in cervical cancer diagnosis using deep learning on whole slide images: An Overview of models, techniques, challenges and future directions

Integration of whole slide imaging (WSI) and deep learning technology has led to significant improvements in the screening and diagnosis of cervical cancer. WSI enables the examination of all cells on a slide simultaneously and deep learning algorithms can accurately label them as cancerous or non-cancerous. Although many studies have investigated the application of deep learning for diagnosing various diseases, there is a lack of research focusing on the evolution, limitations, and gaps of intelligent algorithms in conjunction with WSI for cervical cancer. This paper provides a comprehensive overview of the state-of-the-art deep learning algorithms used for the timely and precise analysis of cervical WSI images. A total of 115 relevant papers were reviewed, and 37 were selected after screening with specific inclusion and exclusion criteria. Methodological aspects including deep learning techniques, data sources, architectures, and classification techniques employed by the selected studies were analyzed. The review presents the most popular techniques and current trends in deep learning-based cervical classification systems, and categorizes the evolution of the domain based on deep learning techniques, citing an in-depth analysis of various models developed over time. The paper advocates for the implementation of transfer supervised learning when utilizing deep learning models such as ResNet, VGG19, and EfficientNet, and builds a solid foundation for applying relevant techniques in different fields. Although some progress has been made in developing novel models for the diagnosis of cervical cancer, substantial work remains to be done in creating standardized benchmark databases of WSI images for the research community. This paper serves as a comprehensive guide for understanding the fundamental concepts, benefits, and challenges related to various deep learning models on WSI, including their application for cervical system classification. Additionally, it provides valuable insights into future research directions in this area.

Applications of Artificial Intelligence in Breast Pathology

CONTEXT.—

Increasing implementation of whole slide imaging together with digital workflow and advances in computing capacity enable the use of artificial intelligence (AI) in pathology, including breast pathology. Breast pathologists often face a significant workload, with diagnosis complexity, tedious repetitive tasks, and semiquantitative evaluation of biomarkers. Recent advances in developing AI algorithms have provided promising approaches to meet the demand in breast pathology.

OBJECTIVE.—

To provide an updated review of AI in breast pathology. We examined the success and challenges of current and potential AI applications in diagnosing and grading breast carcinomas and other pathologic changes, detecting lymph node metastasis, quantifying breast cancer biomarkers, predicting prognosis and therapy response, and predicting potential molecular changes.

DATA SOURCES.—

We obtained data and information by searching and reviewing literature on AI in breast pathology from PubMed and based our own experience.

CONCLUSIONS.—

With the increasing application in breast pathology, AI not only assists in pathology diagnosis to improve accuracy and reduce pathologists' workload, but also provides new information in predicting prognosis and therapy response.

Development and validation of a deep learning algorithm for pattern-based classification system of cervical cancer from pathological sections

Background

Multi-center research has demonstrated that adopting Silva's pattern-based classification system (SPBC) enhances the clinical prognosis and facilitates hierarchical management of patients with endocervical adenocarcinomas (EAC). However, inconsistencies in SPBC can arise due to variations in pathologists' experience levels. Thus, the implementation of standardized decision-making tools becomes crucial to enhance the practicality of SPBC in clinical diagnosis and treatment.

Methods

We enrolled a total of 90 patients with EAC in this study, of which 63 were assigned to the training group, and the remaining 27 were allocated to the validation group. To create and validate the prediction models for SPBC, we utilized a deep learning system (DLS) and calculated the area under the receiver operating characteristic curve (AUC).

Results

In Silva pattern classification, ResNet50 achieved an average accuracy of 74.36% (63.64% for pattern A, 55.56% for pattern B, and 89.47% for pattern C respectively). Moreover, in test set, ResNet50 achieved an AUC of 0.69 for pattern A, 0.58 for pattern B, and 0.91 for pattern C.

Conclusions

We successfully established a DLS for SPBC, which holds the potential to aid pathologists in accurately classifying patients with EAC.

Unleashing the potential of AI for pathology: challenges and recommendations

Computational pathology is currently witnessing a surge in the development of AI techniques, offering promise for achieving breakthroughs and significantly impacting the practices of pathology and oncology. These AI methods bring with them the potential to revolutionize diagnostic pipelines as well as treatment planning and overall patient care. Numerous peer-reviewed studies reporting remarkable performance across diverse tasks serve as a testimony to the potential of AI in the field. However, widespread adoption of these methods in clinical and pre-clinical settings still remains a challenge. In this review article, we present a detailed analysis of the major obstacles encountered during the development of effective models and their deployment in practice. We aim to provide readers with an overview of the latest developments, assist them with insights into identifying some specific challenges that may require resolution, and suggest recommendations and potential future research directions. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Whole slide imaging (WSI) scanner differences influence optical and computed properties of digitized prostate cancer histology

Purpose

Digital pathology is becoming an increasingly popular area of advancement in both research and clinically. Pathologists are now able to manage and interpret slides digitally, as well as collaborate with external pathologists with digital copies of slides. Differences in slide scanners include variation in resolution, image contrast, and optical properties, which may influence downstream image processing. This study tested the hypothesis that varying slide scanners would result in differences in computed pathomic features on prostate cancer whole mount slides.

Design

This study collected 192 unique tissue slides from 30 patients following prostatectomy. Tissue samples were paraffin-embedded, stained for hematoxylin and eosin (H&E), and digitized using 3 different scanning microscopes at the highest available magnification rate, for a total of 3 digitized slides per tissue slide. These scanners included a (S1) Nikon microscope equipped with an automated sliding stage, an (S2) Olympus VS120 slide scanner, and a (S3) Huron TissueScope LE scanner. A color deconvolution algorithm was then used to optimize contrast by projecting the RGB image into color channels representing optical stain density. The resulting intensity standardized images were then computationally processed to segment tissue and calculate pathomic features including lumen, stroma, epithelium, and epithelial cell density, as well as second-order features including lumen area and roundness; epithelial area, roundness, and wall thickness; and cell fraction. For each tested feature, mean values of that feature per digitized slide were collected and compared across slide scanners using mixed effect models, fit to compare differences in the tested feature associated with all slide scanners for each slide, including a random effect of subject with a nested random effect of slide to account for repeated measures. Similar models were also computed for tissue densities to examine how differences in scanner impact downstream processing.

Results

Each mean color channel intensity (i.e., Red, Green, Blue) differed between slide scanners (all P<.001). Of the color deconvolved images, only the hematoxylin channel was similar in all 3 scanners (all P>.05). Lumen and stroma densities between S3 and S1 slides, and epithelial cell density between S3 and S2 (P>.05) were comparable but all other comparisons were significantly different (P<.05). The second-order features were found to be comparable for all scanner comparisons, except for lumen area and epithelium area.

Conclusion

This study demonstrates that both optical and computed properties of digitized histological samples are impacted by slide scanner differences. Future research is warranted to better understand which scanner properties influence the tissue segmentation process and to develop harmonization techniques for comparing data across multiple slide scanners.

Quantitative Analysis of Inflammatory Uterine Lesions of Pregnant Gilts with Digital Image Analysis Following Experimental PRRSV-1 Infection

Reproductive disorders caused by porcine reproductive and respiratory syndrome virus-1 are not yet fully characterized. We report QuPath-based digital image analysis to count inflammatory cells in 141 routinely, and 35 CD163 immunohistochemically stained endometrial slides of vaccinated or unvaccinated pregnant gilts inoculated with a high or low virulent PRRSV-1 strain. To illustrate the superior statistical feasibility of the numerical data determined by digital cell counting, we defined the association between the number of these cells and endometrial, placental, and fetal features. There was strong concordance between the two manual scorers. Distributions of total cell counts and endometrial and placental qPCR results differed significantly between examiner1's endometritis grades. Total counts' distribution differed significantly between groups, except for the two unvaccinated. Higher vasculitis scores were associated with higher endometritis scores, and higher total cell counts were expected with high vasculitis/endometritis scores. Cell number thresholds of endometritis grades were determined. A significant correlation between fetal weights and total counts was shown in unvaccinated groups, and a significant positive correlation was found between these counts and endometrial qPCR results. We revealed significant negative correlations between CD163+ counts and qPCR results of the unvaccinated group infected with the highly virulent strain. Digital image analysis was efficiently applied to assess endometrial inflammation objectively.

Histology Image Viewer and Converter (HIVC): A High-Speed Freeware Software to View and Convert Whole Slide Histology Images

Histology images are widely used to assess the microstructure of biological tissues, but scanners often save images in bulky SVS and multi-layered TIFF formats. These formats were designed to archive image blocks and high-resolution textual information and are not compatible with conventional image analysis software. Our goal was to create a freeware Histology Image Viewer and Converter (HIVC) with a graphical user interface that allows viewing and converting whole-slide images in batch. HIVC was developed using C# Language for Windows x64 operating system. HIVC's performance was assessed by converting 20 whole-slide images to a JPG format at 20x and 40x resolution and comparing the results to ImageJ, Cell Profiler, QuPath, Nanoborb, and Aperio ImageScope. HIVC was more than 8-times faster in converting images than other software packages. This software allows high-speed batch conversion of histology images to traditional formats, permitting platform-independent secondary analyses.

Short Training Significantly Improves Ganglion Cell Detection Using an Algorithm-Assisted Approach

CONTEXT.—

Medical education in pathology relies on the accumulation of experience gained through inspection of numerous samples from each entity. Acquiring sufficient teaching material for rare diseases, such as Hirschsprung disease (HSCR), may be difficult, especially in smaller institutes. The current study makes use of a previously developed decision support system using a decision support algorithm meant to aid pathologists in the diagnosis of HSCR.

OBJECTIVE.—

To assess the effect of a short training session on algorithm-assisted HSCR diagnosis.

DESIGN.—

Five pathologists reviewed a data set of 568 image sets (1704 images in total) selected from 50 cases by the decision support algorithm and were tasked with scoring the images for the presence or absence of ganglion cells. The task was repeated a total of 3 times. Each pathologist had to complete a short educational presentation between the second and third iterations.

RESULTS.—

The training resulted in a significantly increased rate of correct diagnoses (true positive/negative) and a decreased need for referrals for expert consultation. No statistically significant changes in the rate of false positives/negatives were detected.

CONCLUSIONS.—

A very short (<10 minutes) training session can greatly improve the pathologist's performance in the algorithm-assisted diagnosis of HSCR. The same approach may be feasible in training for the diagnosis of other rare diseases.

An artificial intelligence model for the pathological diagnosis of invasion depth and histologic grade in bladder cancer

BACKGROUND

Accurate pathological diagnosis of invasion depth and histologic grade is key for clinical management in patients with bladder cancer (BCa), but it is labour-intensive, experience-dependent and subject to interobserver variability. Here, we aimed to develop a pathological artificial intelligence diagnostic model (PAIDM) for BCa diagnosis.

METHODS

A total of 854 whole slide images (WSIs) from 692 patients were included and divided into training and validation sets. The PAIDM was developed using the training set based on the deep learning algorithm ScanNet, and the performance was verified at the patch level in validation set 1 and at the WSI level in validation set 2. An independent validation cohort (validation set 3) was employed to compare the PAIDM and pathologists. Model performance was evaluated using the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value and negative predictive value.

RESULTS

The AUCs of the PAIDM were 0.878 (95% CI 0.875-0.881) at the patch level in validation set 1 and 0.870 (95% CI 0.805-0.923) at the WSI level in validation set 2. In comparing the PAIDM and pathologists, the PAIDM achieved an AUC of 0.847 (95% CI 0.779-0.905), which was non-inferior to the average diagnostic level of pathologists. There was high consistency between the model-predicted and manually annotated areas, improving the PAIDM's interpretability.

CONCLUSIONS

We reported an artificial intelligence-based diagnostic model for BCa that performed well in identifying invasion depth and histologic grade. Importantly, the PAIDM performed admirably in patch-level recognition, with a promising application for transurethral resection specimens.

Implementing Digital Pathology into Veterinary Academics and Research

The advantages of digital pathology (DP) have been recognized as early as 1963, but only within the last decade or so have the advancements of slide scanners and viewing software made the use and implementation of DP feasible in the classroom and in research. Several factors must be considered prior to undertaking the project of implementing the DP workflow in any setting, but particularly in an academic environment. Sustained and open dialogue with information technology (IT) is critical to the success of this enterprise. In addition to IT, there is a multitude of criteria to consider when determining the best hardware and software to purchase to support the project. The goals and limitations of the laboratory and the requirements of its users (students, instructors, and researchers) will ultimately direct these decisions. The objectives of this article are to provide an overview of the opportunities and challenges associated with the integration of DP in education and research, to highlight some important IT considerations, and to discuss some of the requirements and functionalities of some hardware and software options.

The Role of Pathology-Based Methods in Qualitative and Quantitative Approaches to Cancer Immunotherapy

Simple Summary

Immunotherapy has become the filar of modern oncological treatment, and programmed death-ligand 1 expression is one of the primary immune markers assessed by pathologists. However, there are still some issues concerning the evaluation of the marker and limited information about the interaction between the tumour and associated immune cells. Recent studies have focused on cancer immunology to try to understand the complex tumour microenvironment, and multiplex imaging methods are more widely used for this purpose. The presented article aims to provide an overall review of a different multiplex in situ method using spectral imaging, supported by automated image-acquisition and software-assisted marker visualisation and interpretation. Multiplex imaging methods could improve the current understanding of complex tumour-microenvironment immunology and could probably help to better match patients to appropriate treatment regimens.

Abstract

Immune checkpoint inhibitors, including those concerning programmed cell death 1 (PD-1) and its ligand (PD-L1), have revolutionised the cancer therapy approach in the past decade. However, not all patients benefit from immunotherapy equally. The prediction of patient response to this type of therapy is mainly based on conventional immunohistochemistry, which is limited by intraobserver variability, semiquantitative assessment, or single-marker-per-slide evaluation. Multiplex imaging techniques and digital image analysis are powerful tools that could overcome some issues concerning tumour-microenvironment studies. This novel approach to biomarker assessment offers a better understanding of the complicated interactions between tumour cells and their environment. Multiplex labelling enables the detection of multiple markers simultaneously and the exploration of their spatial organisation. Evaluating a variety of immune cell phenotypes and differentiating their subpopulations is possible while preserving tissue histology in most cases. Multiplexing supported by digital pathology could allow pathologists to visualise and understand every cell in a single tissue slide and provide meaning in a complex tumour-microenvironment contexture. This review aims to provide an overview of the different multiplex imaging methods and their application in PD-L1 biomarker assessment. Moreover, we discuss digital imaging techniques, with a focus on slide scanners and software.

Deep Learning-Based Recognition of Different Thyroid Cancer Categories Using Whole Frozen-Slide Images

Introduction: The pathological rare category of thyroid is a type of lesion with a low incidence rate and is easily misdiagnosed in clinical practice, which directly affects a patient’s treatment decision. However, it has not been adequately investigated to recognize the rare, benign, and malignant categories of thyroid using the deep learning method and recommend the rare to pathologists.

Methods: We present an empirical decision tree based on the binary classification results of the patch-based UNet model to predict rare categories and recommend annotated lesion areas to be rereviewed by pathologists.

Results: Applying this framework to 1,374 whole-slide images (WSIs) of frozen sections from thyroid lesions, we obtained an area under a curve of 0.946 and 0.986 for the test datasets with and without WSIs, respectively, of rare types. However, the recognition error rate for the rare categories was significantly higher than that for the benign and malignant categories (p < 0.00001). For rare WSIs, the addition of the empirical decision tree obtained a recall rate and precision of 0.882 and 0.498, respectively; the rare types (only 33.4% of all WSIs) were further recommended to be rereviewed by pathologists. Additionally, we demonstrated that the performance of our framework was comparable to that of pathologists in clinical practice for the predicted benign and malignant sections.

Conclusion: Our study provides a baseline for the recommendation of the uncertain predicted rare category to pathologists, offering potential feasibility for the improvement of pathologists’ work efficiency.

Evaluation of a Probability-Based Predictive Tool on Pathologist Agreement Using Urinary Bladder as a Pilot Tissue

Simple Summary

There is a common joke in pathology—put three pathologists in a room and you will obtain three different answers. This saying comes from the fact that pathology can be subjective; pathologists’ diagnoses can be influenced by many different biases, and pathologists are also influenced by the presence or absence of animal information and medical history. Compared to pathology, statistics is a much more objective field. This study aimed to develop a probability-based tool using statistics obtained by analyzing 338 histopathology slides of canine and feline urinary bladders, then see if the tool affected agreement between the test pathologists. Four pathologists diagnosed 25 canine and feline bladder slides and they conducted this three times: without animal and clinical information, then with this information, and finally using the probability tool. Results showed large differences in the pathologists’ interpretation of bladder slides, with kappa agreement values (low value for digital slide images, high value for glass slides) of 7–37% without any animal or clinical information, 23–37% with animal signalment and history, and 31–42% when our probability tool was used. This study provides a starting point for the use of probability-based tools in standardizing pathologist agreement in veterinary pathology.

Abstract

Inter-pathologist variation is widely recognized across human and veterinary pathology and is often compounded by missing animal or clinical information on pathology submission forms. Variation in pathologist threshold levels of resident inflammatory cells in the tissue of interest can further decrease inter-pathologist agreement. This study applied a predictive modeling tool to bladder histology slides that were assessed by four pathologists: first without animal and clinical information, then with this information, and finally using the predictive tool. All three assessments were performed twice, using digital whole-slide images (WSI) and then glass slides. Results showed marked variation in pathologists’ interpretation of bladder slides, with kappa agreement values of 7–37% without any animal or clinical information, 23–37% with animal signalment and history, and 31–42% when our predictive tool was applied, for digital WSI and glass slides. The concurrence of test pathologists to the reference diagnosis was 60% overall. This study provides a starting point for the use of predictive modeling in standardizing pathologist agreement in veterinary pathology. It also highlights the importance of high-quality whole-slide imaging to limit the effect of digitization on inter-pathologist agreement and the benefit of continued standardization of tissue assessment in veterinary pathology.

Reduced SARS-CoV-2 disease outcomes in Syrian hamsters receiving immune sera: Quantitative image analysis in pathologic assessments

There is a need to standardize pathologic endpoints in animal models of SARS-CoV-2 infection to help benchmark study quality, improve cross-institutional comparison of data, and assess therapeutic efficacy so that potential drugs and vaccines for SARS-CoV-2 can rapidly advance. The Syrian hamster model is a tractable small animal model for COVID-19 that models clinical disease in humans. Using the hamster model, the authors used traditional pathologic assessment with quantitative image analysis to assess disease outcomes in hamsters administered polyclonal immune sera from previously challenged rhesus macaques. The authors then used quantitative image analysis to assess pathologic endpoints across studies performed at different institutions using different tissue processing protocols. The authors detail pathological features of SARS-CoV-2 infection longitudinally and use immunohistochemistry to quantify myeloid cells and T lymphocyte infiltrates during SARS-CoV-2 infection. High-dose immune sera protected hamsters from weight loss and diminished viral replication in tissues and reduced lung lesions. Cumulative pathology scoring correlated with weight loss and was robust in distinguishing IgG efficacy. In formalin-infused lungs, quantitative measurement of percent area affected also correlated with weight loss but was less robust in non-formalin-infused lungs. Longitudinal immunohistochemical assessment of interstitial macrophage infiltrates showed that peak infiltration corresponded to weight loss, yet quantitative assessment of macrophage, neutrophil, and CD3+ T lymphocyte numbers did not distinguish IgG treatment effects. Here, the authors show that quantitative image analysis was a useful adjunct tool for assessing SARS-CoV-2 treatment outcomes in the hamster model.

Deep Learning-Based Classification of Hepatocellular Nodular Lesions on Whole-Slide Histopathologic Images

BACKGROUND & AIMS

Hepatocellular nodular lesions (HNLs) constitute a heterogeneous group of disorders. Differential diagnosis among these lesions, especially high-grade dysplastic nodules (HGDNs) and well-differentiated hepatocellular carcinoma (WD-HCC), can be challenging, let alone biopsy specimens. We aimed to develop a deep learning system to solve these puzzles, improving the histopathologic diagnosis of HNLs (WD-HCC, HGDN, low-grade DN, focal nodular hyperplasia, hepatocellular adenoma), and background tissues (nodular cirrhosis, normal liver tissue).

METHODS

The samples consisting of surgical and biopsy specimens were collected from 6 hospitals. Each specimen was reviewed by 2 to 3 subspecialists. Four deep neural networks (ResNet50, InceptionV3, Xception, and the Ensemble) were used. Their performances were evaluated by confusion matrix, receiver operating characteristic curve, classification map, and heat map. The predictive efficiency of the optimal model was further verified by comparing with that of 9 pathologists.

RESULTS

We obtained 213,280 patches from 1115 whole-slide images of 738 patients. An optimal model was finally chosen based on F1 score and area under the curve value, named hepatocellular-nodular artificial intelligence model (HnAIM), with the overall 7-category area under the curve of 0.935 in the independent external validation cohort. For biopsy specimens, the agreement rate with subspecialists' majority opinion was higher for HnAIM than 9 pathologists on both patch level and whole-slide images level.

CONCLUSIONS

We first developed a deep learning diagnostic model for HNLs, which performed well and contributed to enhancing the diagnosis rate of early HCC and risk stratification of patients with HNLs. Furthermore, HnAIM had significant advantages in patch-level recognition, with important diagnostic implications for fragmentary or scarce biopsy specimens.

Multi-Scale Attention Convolutional Network for Masson Stained Bile Duct Segmentation from Liver Pathology Images

In clinical practice, the Ishak Score system would be adopted to perform the evaluation of the grading and staging of hepatitis according to whether portal areas have fibrous expansion, bridging with other portal areas, or bridging with central veins. Based on these staging criteria, it is necessary to identify portal areas and central veins when performing the Ishak Score staging. The bile ducts have variant types and are very difficult to be detected under a single magnification, hence pathologists must observe bile ducts at different magnifications to obtain sufficient information. This pathologic examinations in routine clinical practice, however, would result in the labor intensive and expensive examination process. Therefore, the automatic quantitative analysis for pathologic examinations has had an increased demand and attracted significant attention recently. A multi-scale inputs of attention convolutional network is proposed in this study to simulate pathologists' examination procedure for observing bile ducts under different magnifications in liver biopsy. The proposed multi-scale attention network integrates cell-level information and adjacent structural feature information for bile duct segmentation. In addition, the attention mechanism of proposed model enables the network to focus the segmentation task on the input of high magnification, reducing the influence from low magnification input, but still helps to provide wider field of surrounding information. In comparison with existing models, including FCN, U-Net, SegNet, DeepLabv3 and DeepLabv3-plus, the experimental results demonstrated that the proposed model improved the segmentation performance on Masson bile duct segmentation task with 72.5% IOU and 84.1% F1-score.

MMO-Net (Multi-Magnification Organ Network): A use case for Organ Identification using Multiple Magnifications in Preclinical Pathology Studies

Identifying organs within histology images is a fundamental and non-trivial step in toxicological digital pathology workflows as multiple organs often appear on the same whole slide image (WSI). Previous works in automated tissue classification have investigated the use of single magnifications, and demonstrated limitations when attempting to identify small and contiguous organs at low magnifications. In order to overcome these shortcomings, we present a multi-magnification convolutional neural network (CNN), called MMO-Net, which employs context and cellular detail from different magnifications to facilitate the recognition of complex organs. Across N=320 WSI from 3 contract research organization (CRO) laboratories, we demonstrate state-of-the-art organ detection and segmentation performance of 7 rat organs with and without lesions: liver, kidney, thyroid gland, parathyroid gland, urinary bladder, salivary gland, and mandibular lymph node (AUROC=0.99–1.0 for all organs, Dice≥0.9 except parathyroid (0.73)). Evaluation takes place at both inter- and intra CRO levels, suggesting strong generalizability performance. Results are qualitatively reviewed using visualization masks to ensure separation of organs in close proximity (e.g., thyroid vs parathyroid glands). MMO-Net thus offers organ localization that serves as a potential quality control tool to validate WSI metadata and as a preprocessing step for subsequent organ-specific artificial intelligence (AI) use cases. To facilitate research in this area, all associated WSI and metadata used for this study are being made freely available, forming a first of its kind dataset for public use.

Whole-slide imaging, tissue image analysis, and artificial intelligence in veterinary pathology: An updated introduction and review

Since whole-slide imaging has been commercially available for over 2 decades, digital pathology has become a constantly expanding aspect of the pathology profession that will continue to significantly impact how pathologists conduct their craft. While some aspects, such as whole-slide imaging for archiving, consulting, and teaching, have gained broader acceptance, other facets such as quantitative tissue image analysis and artificial intelligence-based assessments are still met with some reservations. While most vendors in this space have focused on diagnostic applications, that is, viewing one or few slides at a time, some are developing solutions tailored more specifically to the various aspects of veterinary pathology including updated diagnostic, discovery, and research applications. This has especially advanced the use of digital pathology in toxicologic pathology and drug development, for primary reads as well as peer reviews. It is crucial that pathologists gain a deeper understanding of digital pathology and tissue image analysis technology and their applications in order to fully use these tools in a way that enhances and improves the pathologist's assessment as well as work environment. This review focuses on an updated introduction to the basics of digital pathology and image analysis and introduces emerging topics around artificial intelligence and machine learning.

Characterization of in vitro 3D cultures

Over the past decade, 3D culture models of human and animal cells have found their way into tissue differentiation, drug development, personalized medicine and tumour behaviour studies. Embryoid bodies (EBs) are in vitro 3D cultures established from murine pluripotential stem cells, whereas tumoroids are patient-derived in vitro 3D cultures. This thesis aims to describe a new implication of an embryoid body model and to characterize the patient-specific microenvironment of the parental tumour in relation to tumoroid growth rate. In this thesis, we described a high-throughput monitoring method, where EBs are used as a dynamic angiogenesis model. In this model, digital image analysis (DIA) is implemented on immunohistochemistry (IHC) stained sections of the cultures over time. Furthermore, we have investigated the correlation between the genetic profile and inflammatory microenvironment of parental tumours on the in vitro growth rate of tumoroids. The EBs were cultured in spinner flasks. The samples were collected at days 4, 6, 9, 14, 18 and 21, dehydrated and embedded in paraffin. The histological sections were IHC stained for the endothelial marker CD31 and digitally scanned. The virtual whole-image slides were digitally analysed by Visiopharm® software. Histological evaluation showed vascular-like structures over time. The quantitative DIA was plausible to monitor significant increase in the total area of the EBs and an increase in endothelial differentiation. The tumoroids were established from 32 colorectal adenocarcinomas. The in vitro growth rate of the tumoroids was followed by automated microscopy over an 11-day period. The parental tumours were analysed by next-generation sequencing for KRAS, TP53, PIK3CA, SMAD4, MAP2K1, BRAF, FGFR3 and FBXW7 status. The tumoroids established from KRAS-mutated parental tumours showed a significantly higher growth rate compared to their wild-type counterparts. The density of CD3+ T lymphocytes and CD68+ macrophages was calculated in the centre of the tumours and at the invasive margin of the tumours. The high density of CD3+ cells and the low density of CD68+ cells showed a significant correlation with a higher growth rate of the tumoroids. In conclusion, a novel approach for histological monitoring of endothelial differentiation is presented in the stem cell-derived EBs. Furthermore, the KRAS status and density of CD3+ T cells and macrophages in the parental tumour influence the growth rate of the tumoroids. Our results indicate that these parameters should be included when tumoroids are to be implemented in personalized medicine.

Poor Endometrial Proliferation After Clomiphene is Associated With Altered Estrogen Action

CONTEXT

Suboptimal endometrial thickening is associated with lower pregnancy rates and occurs in some infertile women treated with clomiphene.

OBJECTIVE

To examine cellular and molecular differences in the endometrium of women with suboptimal vs optimal endometrial thickening following clomiphene.

METHODS

Translational prospective cohort study from 2018 to 2020 at a university-affiliated clinic. Reproductive age women with unexplained infertility treated with 100 mg of clomiphene on cycle days 3 to 7 who developed optimal (≥8mm; n = 6, controls) or suboptimal (<6mm; n = 7, subjects) endometrial thickness underwent preovulatory blood and endometrial sampling. The main outcome measures were endometrial tissue architecture, abundance and location of specific proteins, RNA expression, and estrogen receptor (ER) α binding.

RESULTS

The endometrium of suboptimal subjects compared with optimal controls was characterized by a reduced volume of glandular epithelium (16% vs 24%, P = .01), decreased immunostaining of markers of proliferation (PCNA, ki67) and angiogenesis (PECAM-1), increased immunostaining of pan-leukocyte marker CD45 and ERβ, but decreased ERα immunostaining (all P < .05). RNA-seq identified 398 differentially expressed genes between groups. Pathway analysis of differentially expressed genes indicated reduced proliferation (Z-score = -2.2, P < .01), decreased angiogenesis (Z-score = -2.87, P < .001), increased inflammation (Z-score = +2.2, P < .01), and ERβ activation (Z-score = +1.6, P < .001) in suboptimal subjects. ChIP-seq identified 6 genes bound by ERα that were differentially expressed between groups (P < .01), some of which may play a role in implantation.

CONCLUSION

Women with suboptimal endometrial thickness after clomiphene exhibit aberrant ER expression patterns, architectural changes, and altered gene and protein expression suggesting reduced proliferation and angiogenesis in the setting of increased inflammation.

Quantitative Image Analysis for Tissue Biomarker Use: A White Paper From the Digital Pathology Association

Tissue biomarkers have been of increasing utility for scientific research, diagnosing disease, and treatment response prediction. There has been a steady shift away from qualitative assessment toward providing more quantitative scores for these biomarkers. The application of quantitative image analysis has thus become an indispensable tool for in-depth tissue biomarker interrogation in these contexts. This white paper reviews current technologies being employed for quantitative image analysis, their application and pitfalls, regulatory framework demands, and guidelines established for promoting their safe adoption in clinical practice.

Tissue Multiplex Analyte Detection in Anatomic Pathology - Pathways to Clinical Implementation

Background: Multiplex tissue analysis has revolutionized our understanding of the tumor microenvironment (TME) with implications for biomarker development and diagnostic testing. Multiplex labeling is used for specific clinical situations, but there remain barriers to expanded use in anatomic pathology practice.

Methods: We review immunohistochemistry (IHC) and related assays used to localize molecules in tissues, with reference to United States regulatory and practice landscapes. We review multiplex methods and strategies used in clinical diagnosis and in research, particularly in immuno-oncology. Within the framework of assay design and testing phases, we examine the suitability of multiplex immunofluorescence (mIF) for clinical diagnostic workflows, considering its advantages and challenges to implementation.

Results: Multiplex labeling is poised to radically transform pathologic diagnosis because it can answer questions about tissue-level biology and single-cell phenotypes that cannot be addressed with traditional IHC biomarker panels. Widespread implementation will require improved detection chemistry, illustrated by InSituPlex technology (Ultivue, Inc., Cambridge, MA) that allows coregistration of hematoxylin and eosin (H&E) and mIF images, greater standardization and interoperability of workflow and data pipelines to facilitate consistent interpretation by pathologists, and integration of multichannel images into digital pathology whole slide imaging (WSI) systems, including interpretation aided by artificial intelligence (AI). Adoption will also be facilitated by evidence that justifies incorporation into clinical practice, an ability to navigate regulatory pathways, and adequate health care budgets and reimbursement. We expand the brightfield WSI system “pixel pathway” concept to multiplex workflows, suggesting that adoption might be accelerated by data standardization centered on cell phenotypes defined by coexpression of multiple molecules.

Conclusion: Multiplex labeling has the potential to complement next generation sequencing in cancer diagnosis by allowing pathologists to visualize and understand every cell in a tissue biopsy slide. Until mIF reagents, digital pathology systems including fluorescence scanners, and data pipelines are standardized, we propose that diagnostic labs will play a crucial role in driving adoption of multiplex tissue diagnostics by using retrospective data from tissue collections as a foundation for laboratory-developed test (LDT) implementation and use in prospective trials as companion diagnostics (CDx).

Quantitative digital image analysis of somatostatin receptor 2 immunohistochemistry in pancreatic neuroendocrine tumors

Immunohistochemical analysis of somatostatin receptor 2 (SSTR2) provides important information regarding the potential therapeutic efficacy of somatostatin analogues (SSAs) in patients with neuroendocrine tumors. HER2 scoring has been proposed to interpret SSTR2 immunoreactivity but their reproducibility was relatively low because of its intrinsic subjective nature. Digital image analysis (DIA) has recently been proposed as an objective and more precise method of evaluating immunoreactivity. Therefore, in this study, we used DIA for analyzing SSTR2 immunoreactivity in pancreatic neuroendocrine tumors (PanNETs) to obtain its H score and "(%) strong positive cells" and compared the results with those of manually obtained HER2 scores. Membranous SSTR2 immunoreactivity evaluated by DIA was calculated by two scales as: "Membrane Optical Density" and "Minimum Membrane Completeness". PanNETs with HER2 score of > 2 demonstrated the highest concordance with results of "(%) strong positive cells" obtained by DIA when "Minimum Membrane Completeness" was tentatively set at 80%. The SSTR2 immunoreactivity, evaluated based on all scoring systems, was different between grades G1 and G2 in insulinoma but not in non-functional PanNETs. DIA provided reproducible results of SSTR2 immunoreactivity in PanNETs and yielded important information as to the potential application of SSAs.

Yet Another Automated Gleason Grading System (YAAGGS) by weakly supervised deep learning

The Gleason score contributes significantly in predicting prostate cancer outcomes and selecting the appropriate treatment option, which is affected by well-known inter-observer variations. We present a novel deep learning-based automated Gleason grading system that does not require extensive region-level manual annotations by experts and/or complex algorithms for the automatic generation of region-level annotations. A total of 6664 and 936 prostate needle biopsy single-core slides (689 and 99 cases) from two institutions were used for system discovery and validation, respectively. Pathological diagnoses were converted into grade groups and used as the reference standard. The grade group prediction accuracy of the system was 77.5% (95% confidence interval (CI): 72.3-82.7%), the Cohen's kappa score (κ) was 0.650 (95% CI: 0.570-0.730), and the quadratic-weighted kappa score (κ_quad) was 0.897 (95% CI: 0.815-0.979). When trained on 621 cases from one institution and validated on 167 cases from the other institution, the system's accuracy reached 67.4% (95% CI: 63.2-71.6%), κ 0.553 (95% CI: 0.495-0.610), and the κ_quad 0.880 (95% CI: 0.822-0.938). In order to evaluate the impact of the proposed method, performance comparison with several baseline methods was also performed. While limited by case volume and a few more factors, the results of this study can contribute to the potential development of an artificial intelligence system to diagnose other cancers without extensive region-level annotations.

Whole slide imaging is a high-throughput method to assess Candida biofilm formation

New strategies that enable fast and accurate visualization of Candida biofilms are necessary to better study their structure and response to antifungals agents. Here, we applied whole slide imaging (WSI) to study biofilm formation of Candida species. Three relevant biofilm-forming Candida species (C. albicans ATCC 10231, C. glabrata ATCC 2001, and C. tropicalis ATCC 750) were cultivated on glass coverslips both in presence and absence of widely used antifungals. Accumulated biofilms were stained with fluorescent markers and scanned in both bright-field and fluorescence modes using a WSI digital scanner. WSI enabled clear assessment of both size and structural features of Candida biofilms. Quantitative analyses readily detected reductions in biofilm-covered surface area upon antifungal exposure. Furthermore, we show that the overall biofilm growth can be adequately assessed across both bright-field and fluorescence modes. At the single-cell level, WSI proved adequate, as morphometric parameters evaluated with WSI did not differ significantly from those obtained with scanning electron microscopy, considered as golden standard at single-cell resolution. Thus, WSI allows for reliable visualization of Candida biofilms enabling both large-scale growth assessment and morphometric characterization of single-cell features, making it an important addition to the available microscopic toolset to image and analyse fungal biofilm growth.

Ensemble of transfer learnt classifiers for recognition of cardiovascular tissues from histological images

Recognition of tissues and organs is a recurrent step performed by experts during analyses of histological images. With advancement in the field of machine learning, such steps can be automated using computer vision methods. This paper presents an ensemble-based approach for improved classification of non-pathological tissues and organs in histological images using convolutional neural networks (CNNs). With limited dataset size, we relied upon transfer learning where pre-trained CNNs are re-used for new classification problems. The transfer learning was done using eleven CNN architectures upon 6000 image patches constituting training and validation subsets of a public dataset containing six cardiovascular categories. The CNN models were fine-tuned upon a much larger dataset obtained by augmenting training subset to obtain agreeable performance on validation subset. Lastly, we created various ensembles of trained classifiers and evaluate them on testing subset of 7500 patches. The best ensemble classifier gives, precision, recall, and accuracy of 0.876, 0.869 and 0.869, respectively upon test images. With an overall F₁-score of 0.870, our ensemble-based approach outperforms previous approaches with single fine-tuned CNN, CNN trained from scratch, and traditional machine learning by 0.019, 0.064 and 0.183, respectively. Ensemble approach can perform better than individual classifier-based ones, provided the constituent classifiers are chosen wisely. The empirical choice of classifiers reinforces the intuition that models which are newer and outperformed in their native domain are more likely to outperform in transferred-domain, since the best ensemble dominantly consists of more lately proposed and better architectures.

Developing a Qualification and Verification Strategy for Digital Tissue Image Analysis in Toxicological Pathology

Digital tissue image analysis is a computational method for analyzing whole-slide images and extracting large, complex, and quantitative data sets. However, as with any analysis method, the quality of generated results is dependent on a well-designed quality control system for the entire digital pathology workflow. Such system requires clear procedural controls, appropriate user training, and involvement of specialists to oversee key steps of the workflow. The toxicologic pathologist is responsible for reporting data obtained by digital image analysis and therefore needs to ensure that it is correct. To accomplish that, they must understand the main parameters of the quality control system and should play an integral part in its conception and implementation. This manuscript describes the most common digital tissue image analysis end points and potential sources of analysis errors. In addition, it outlines recommended approaches for ensuring quality and correctness of results for both classical and machine-learning based image analysis solutions, as adapted from a recently proposed Food and Drug Administration regulatory framework for modifications to artificial intelligence/machine learning-based software as a medical device. These approaches are beneficial for any type of toxicopathologic study which uses the described end points and can be adjusted based on the intended use of the image analysis solution.

X-ray dark-field phase-contrast imaging: Origins of the concept to practical implementation and applications

The basic idea of X-ray dark-field imaging (XDFI), first presented in 2000, was based on the concepts used in an X-ray interferometer. In this article, we review 20 years of developments in our theoretical understanding, scientific instrumentation, and experimental demonstration of XDFI and its applications to medical imaging. We first describe the concepts underlying XDFI that are responsible for imparting phase contrast information in projection X-ray images. We then review the algorithms that can convert these projection phase images into three-dimensional tomographic slices. Various implementations of computed tomography reconstructions algorithms for XDFI data are discussed. The next four sections describe and illustrate potential applications of XDFI in pathology, musculoskeletal imaging, oncologic imaging, and neuroimaging. The sample applications that are presented illustrate potential use scenarios for XDFI in histopathology and other clinical applications. Finally, the last section presents future perspectives and potential technical developments that can make XDFI an even more powerful tool.

The Application, Challenges, and Advancement Toward Regulatory Acceptance of Digital Toxicologic Pathology: Results of the 7th ESTP International Expert Workshop (September 20-21, 2019)

With advancements in whole slide imaging technology and improved understanding of the features of pathologist workstations required for digital slide evaluation, many institutions are investigating broad digital pathology adoption. The benefits of digital pathology evaluation include remote access to study or diagnostic case materials and integration of analysis and reporting tools. Diagnosis based on whole slide images is established in human medical pathology, and the use of digital pathology in toxicologic pathology is increasing. However, there has not been broad adoption in toxicologic pathology, particularly in the context of regulatory studies, due to lack of precedence. To address this topic, as well as practical aspects, the European Society of Toxicologic Pathology coordinated an expert international workshop to assess current applications and challenges and outline a set of minimal requirements needed to gain future regulatory acceptance for the use of digital toxicologic pathology workflows in research and development, so that toxicologic pathologists can benefit from digital slide technology.

Impact of Preanalytical Factors During Histology Processing on Section Suitability for Digital Image Analysis

Digital image analysis (DIA) is impacted by the quality of tissue staining. This study examined the influence of preanalytical variables-staining protocol design, reagent quality, section attributes, and instrumentation-on the performance of automated DIA software. Our hypotheses were that (1) staining intensity is impacted by subtle differences in protocol design, reagent quality, and section composition and that (2) identically programmed and loaded stainers will produce equivalent immunohistochemical (IHC) staining. We tested these propositions by using 1 hematoxylin and eosin stainer to process 13 formalin-fixed, paraffin-embedded (FFPE) mouse tissues and by using 3 identically programmed and loaded immunostainers to process 5 FFPE mouse tissues for 4 cell biomarkers. Digital images of stained sections acquired with a commercial whole slide scanner were analyzed by customizable algorithms incorporated into commercially available DIA software. Staining intensity as viewed qualitatively by an observer and/or quantitatively by DIA was affected by staining conditions and tissue attributes. Intrarun and inter-run IHC staining intensities were equivalent for each tissue when processed on a given stainer but varied measurably across stainers. Our data indicate that staining quality must be monitored for each method and stainer to ensure that preanalytical factors do not impact digital pathology data quality.

Computerized Image Analysis of Tumor Cell Nuclear Morphology Can Improve Patient Selection for Clinical Trials in Localized Clear Cell Renal Cell Carcinoma

Background:

Clinicopathological scores are used to predict the likelihood of recurrence-free survival for patients with clear cell renal cell carcinoma (ccRCC) after surgery. These are fallible, particularly in the middle range. This inevitably means that a significant proportion of ccRCC patients who will not develop recurrent disease enroll into clinical trials. As an exemplar of using digital pathology, we sought to improve the predictive power of “recurrence free” designation in localized ccRCC patients, by precise measurement of ccRCC nuclear morphological features using computational image analysis, thereby replacing manual nuclear grade assessment.

Materials and Methods:

TNM 8 UICC pathological stage pT1-pT3 ccRCC cases were recruited in Scotland and in Singapore. A Leibovich score (LS) was calculated. Definiens Tissue studio® (Definiens GmbH, Munich) image analysis platform was used to measure tumor nuclear morphological features in digitized hematoxylin and eosin (H&E) images.

Results:

Replacing human-defined nuclear grade with computer-defined mean perimeter generated a modified Leibovich algorithm, improved overall specificity 0.86 from 0.76 in the training cohort. The greatest increase in specificity was seen in LS 5 and 6, which went from 0 to 0.57 and 0.40, respectively. The modified Leibovich algorithm increased the specificity from 0.84 to 0.94 in the validation cohort.

Conclusions:

CcRCC nuclear mean perimeter, measured by computational image analysis, together with tumor stage and size, node status and necrosis improved the accuracy of predicting recurrence-free in the localized ccRCC patients. This finding was validated in an ethnically different Singaporean cohort, despite the different H and E staining protocol and scanner used. This may be a useful patient selection tool for recruitment to multicenter studies, preventing some patients from receiving unnecessary additional treatment while reducing the number of patients required to achieve adequate power within neoadjuvant and adjuvant clinical studies.

Quantitative neurotoxicology: Potential role of artificial intelligence/deep learning approach

Neurotoxicity studies are important in the preclinical stages of drug development process, because exposure to certain compounds that may enter the brain across a permeable blood brain barrier damages neurons and other supporting cells such as astrocytes. This could, in turn, lead to various neurological disorders such as Parkinson's or Huntington's disease as well as various dementias. Toxicity assessment is often done by pathologists after these exposures by qualitatively or semiquantitatively grading the severity of neurotoxicity in histopathology slides. Quantification of the extent of neurotoxicity supports qualitative histopathological analysis and provides a better understanding of the global extent of brain damage. Stereological techniques such as the utilization of an optical fractionator provide an unbiased quantification of the neuronal damage; however, the process is time-consuming. Advent of whole slide imaging (WSI) introduced digital image analysis which made quantification of neurotoxicity automated, faster and with reduced bias, making statistical comparisons possible. Although automated to a certain level, simple digital image analysis requires manual efforts of experts which is time-consuming and limits analysis of large datasets. Digital image analysis coupled with a deep learning artificial intelligence model provides a good alternative solution to time-consuming stereological and simple digital analysis. Deep learning models could be trained to identify damaged or dead neurons in an automated fashion. This review has focused on and discusses studies demonstrating the role of deep learning in segmentation of brain regions, toxicity detection and quantification of degenerated neurons as well as the estimation of area/volume of degeneration.

Immune contexture analysis in immuno-oncology: applications and challenges of multiplex fluorescent immunohistochemistry

The tumor microenvironment is an integral player in cancer initiation, tumor progression, response and resistance to anti-cancer therapy. Understanding the complex interactions of tumor immune architecture (referred to as 'immune contexture') has therefore become increasingly desirable to guide our approach to patient selection, clinical trial design, combination therapies, and patient management. Quantitative image analysis based on multiplexed fluorescence immunohistochemistry and deep learning technologies are rapidly developing to enable researchers to interrogate complex information from the tumor microenvironment and find predictive insights into treatment response. Herein, we discuss current developments in multiplexed fluorescence immunohistochemistry for immune contexture analysis, and their application in immuno-oncology, and discuss challenges to effectively use this technology in clinical settings. We also present a multiplexed image analysis workflow to analyse fluorescence multiplexed stained tumor sections using the Vectra Automated Digital Pathology System together with FCS express flow cytometry software. The benefit of this strategy is that the spectral unmixing accurately generates and analyses complex arrays of multiple biomarkers, which can be helpful for diagnosis, risk stratification, and guiding clinical management of oncology patients.

OpSeF: Open Source Python Framework for Collaborative Instance Segmentation of Bioimages

Various pre-trained deep learning models for the segmentation of bioimages have been made available as developer-to-end-user solutions. They are optimized for ease of use and usually require neither knowledge of machine learning nor coding skills. However, individually testing these tools is tedious and success is uncertain. Here, we present the Open Segmentation Framework (OpSeF), a Python framework for deep learning-based instance segmentation. OpSeF aims at facilitating the collaboration of biomedical users with experienced image analysts. It builds on the analysts' knowledge in Python, machine learning, and workflow design to solve complex analysis tasks at any scale in a reproducible, well-documented way. OpSeF defines standard inputs and outputs, thereby facilitating modular workflow design and interoperability with other software. Users play an important role in problem definition, quality control, and manual refinement of results. OpSeF semi-automates preprocessing, convolutional neural network (CNN)-based segmentation in 2D or 3D, and postprocessing. It facilitates benchmarking of multiple models in parallel. OpSeF streamlines the optimization of parameters for pre- and postprocessing such, that an available model may frequently be used without retraining. Even if sufficiently good results are not achievable with this approach, intermediate results can inform the analysts in the selection of the most promising CNN-architecture in which the biomedical user might invest the effort of manually labeling training data. We provide Jupyter notebooks that document sample workflows based on various image collections. Analysts may find these notebooks useful to illustrate common segmentation challenges, as they prepare the advanced user for gradually taking over some of their tasks and completing their projects independently. The notebooks may also be used to explore the analysis options available within OpSeF in an interactive way and to document and share final workflows. Currently, three mechanistically distinct CNN-based segmentation methods, the U-Net implementation used in Cellprofiler 3.0, StarDist, and Cellpose have been integrated within OpSeF. The addition of new networks requires little; the addition of new models requires no coding skills. Thus, OpSeF might soon become both an interactive model repository, in which pre-trained models might be shared, evaluated, and reused with ease.

Application of digital image analysis on histological images of a murine embryoid body model for monitoring endothelial differentiation

The in vitro 3D model established from murine pluripotential stem cells (i.e., embryoid bodies (EBs)) is a dynamic model for endothelial differentiation. The aim of the present study was to investigate whether digital image analysis (DIA) can be applied on histological sections of EBs in order to quantify endothelial differentiation over time. The EBs were established in suspension cultures for 21 days in three independent replicate experiments. At day 4, 6, 9, 14, 18, and 21, the EBs were fixed in formaldehyde, embedded in paraffin and immunohistochemically (IHC) stained for CD31. The IHC-stained slides were digitally scanned and analysed using the Visiopharm® Quantitative Digital Pathology software Oncotopix™. The EBs developed CD31+ vascular-like structures during their differentiation. The quantitative DIA of the EBs showed that the log10 values of the relative CD31+ areas increased from -0.574 ± 0.470 (mean ± SD) at day 4 to 0.093 ± 0.688 (mean ± SD) at day 21 (p < 0.001). The approach presented in this study is a fast, quantitative and reproducible alternative method for an otherwise time-consuming and observer-dependent histological investigation. The future perspectives for such a system would be implementation of a modified version of the method on different 3D cultures and IHC markers.

Developing image analysis pipelines of whole-slide images: Pre- and post-processing

Deep learning has pushed the scope of digital pathology beyond simple digitization and telemedicine. The incorporation of these algorithms in routine workflow is on the horizon and maybe a disruptive technology, reducing processing time, and increasing detection of anomalies. While the newest computational methods enjoy much of the press, incorporating deep learning into standard laboratory workflow requires many more steps than simply training and testing a model. Image analysis using deep learning methods often requires substantial pre- and post-processing order to improve interpretation and prediction. Similar to any data processing pipeline, images must be prepared for modeling and the resultant predictions need further processing for interpretation. Examples include artifact detection, color normalization, image subsampling or tiling, removal of errant predictions, etc. Once processed, predictions are complicated by image file size - typically several gigabytes when unpacked. This forces images to be tiled, meaning that a series of subsamples from the whole-slide image (WSI) are used in modeling. Herein, we review many of these methods as they pertain to the analysis of biopsy slides and discuss the multitude of unique issues that are part of the analysis of very large images.

A comparison of imaging software and conventional cell counting in determining melanocyte density in photodamaged control sample and melanoma in situ biopsies

BACKGROUND

Objective methods for distinguishing melanoma in situ (MIS) from photodamaged skin (PS) are needed to guide treatment in patients with melanocytic proliferations. Melanocyte density (MD) could serve as an objective histopathological criterion in difficult cases. Calculating MD via manual cell counts (MCC) with immunohistochemical (IHC)-stained slides has been previously published. However, the clinical application of this method is questionable, as quantification of MD via MCC on difficult cases is time consuming, especially in high volume practices.

METHODS

ImageJ is an image processing software that uses scanned slide images to determine cell count. In this study, we compared MCC to ImageJ calculated MD in microphthalmia transcription factor-IHC stained MIS biopsies and control PS acquired from the same patients.

RESULTS

We found a statistically significant difference in MD between PS and MIS as measured by both MCC and ImageJ software (P < 0.01). Additionally, no statistically significant difference was found when comparing MD measurements recorded by ImageJ vs those determined by the MCC method.

CONCLUSION

MD as determined by ImageJ strongly correlates with the MD calculated by MCC. We propose the use of ImageJ as a time-efficient, objective, and reproducible tool to assess MD.

Cross-Platform Comparison of Computer-assisted Image Analysis Quantification of In Situ mRNA Hybridization in Investigative Pathology

Although availability of automated platforms has proliferated, there is no standard practice for computer-assisted generation of scores for mRNA in situ hybridization (ISH) visualized by brightfield microscopic imaging on tissue sections. To address this systematically, an ISH for peptidylprolyl isomerase B (PPIB) (cyclophilin B) mRNA was optimized and applied to a tissue microarray of archival non-small cell lung carcinoma cases, and then automated image analysis for PPIB was refined across 4 commercially available software platforms. Operator experience and scoring results from ImageScope, HALO, CellMap, and Developer XD were systematically compared with each other and to manual pathologist scoring. Markup images were compared and contrasted for accuracy, the ability of the platform to identify cells, and the ease of visual assessment to determine appropriate interpretation. Comparing weighted scoring approaches using H-scores (Developer XD, ImageScope, and manual scoring) a correlation was observed (R value=0.7955), and association between the remaining 2 approaches (HALO and CellMap) was of similar value. ImageScope showed the highest R value in comparison with manual scoring (0.7377). Mean-difference plots showed that HALO produced the highest relative normalized values, suggesting higher relative sensitivity. ImageScope overestimated PPIB ISH signal at the high end of the range scores; however, this tendency was not observed in other platforms. HALO emerged with the highest number of favorable observations, no apparent systematic bias in score generation compared with the other methods, and potentially higher sensitivity to detect ISH. HALO may serve as a tool to empower teams of investigative pathology laboratory scientists to assist pathologists readily with quantitative scoring of ISH.

Whole Slide Imaging and Its Applications to Histopathological Studies of Liver Disorders

Histological analysis of hepatic tissue specimens is essential for evaluating the pathology of several liver disorders such as chronic liver diseases, hepatocellular carcinomas, liver steatosis, and infectious liver diseases. Manual examination of histological slides on the microscope is a classically used method to study these disorders. However, it is considered time-consuming, limited, and associated with intra- and inter-observer variability. Emerging technologies such as whole slide imaging (WSI), also termed virtual microscopy, have increasingly been used to improve the assessment of histological features with applications in both clinical and research laboratories. WSI enables the acquisition of the tissue morphology/pathology from glass slides and translates it into a digital form comparable to a conventional microscope, but with several advantages such as easy image accessibility and storage, portability, sharing, annotation, qualitative and quantitative image analysis, and use for educational purposes. WSI-generated images simultaneously provide high resolution and a wide field of observation that can cover the entire section, extending any single field of view. In this review, we summarize current knowledge on the application of WSI to histopathological analyses of liver disorders as well as to understand liver biology. We address how WSI may improve the assessment and quantification of multiple histological parameters in the liver, and help diagnose several hepatic conditions with important clinical implications. The WSI technical limitations are also discussed.

Automated Osteosclerosis Grading of Clinical Biopsies Using Infrared Spectroscopic Imaging

Osteosclerosis and myefibrosis are complications of myeloproliferative neoplasms. These disorders result in excess growth of trabecular bone and collagen fibers that replace hematopoietic cells, resulting in abnormal bone marrow function. Treatments using imatinib and JAK2 pathway inhibitors can be effective on osteosclerosis and fibrosis; therefore, accurate grading is critical for tracking treatment effectiveness. Current grading standards use a four-class system based on analysis of biopsies stained with three histological stains: hematoxylin and eosin (H&E), Masson's trichrome, and reticulin. However, conventional grading can be subjective and imprecise, impacting the effectiveness of treatment. In this Article, we demonstrate that mid-infrared spectroscopic imaging may serve as a quantitative diagnostic tool for quantitatively tracking disease progression and response to treatment. The proposed approach is label-free and provides automated quantitative analysis of osteosclerosis and collagen fibrosis.

Automated Gleason Scoring and Tumor Quantification in Prostate Core Needle Biopsy Images Using Deep Neural Networks and Its Comparison with Pathologist-Based Assessment

The Gleason grading system, currently the most powerful prognostic predictor of prostate cancer, is based solely on the tumor’s histological architecture and has high inter-observer variability. We propose an automated Gleason scoring system based on deep neural networks for diagnosis of prostate core needle biopsy samples. To verify its efficacy, the system was trained using 1133 cases of prostate core needle biopsy samples and validated on 700 cases. Further, system-based diagnosis results were compared with reference standards derived from three certified pathologists. In addition, the system’s ability to quantify cancer in terms of tumor length was also evaluated via comparison with pathologist-based measurements. The results showed a substantial diagnostic concordance between the system-grade group classification and the reference standard (0.907 quadratic-weighted Cohen’s kappa coefficient). The system tumor length measurements were also notably closer to the reference standard (correlation coefficient, R = 0.97) than the original hospital diagnoses (R = 0.90). We expect this system to assist pathologists to reduce the probability of over- or under-diagnosis by providing pathologist-level second opinions on the Gleason score when diagnosing prostate biopsy, and to support research on prostate cancer treatment and prognosis by providing reproducible diagnosis based on the consistent standards.