Multimodal Gated Mixture of Experts Using Whole Slide Image and Flow Cytometry for Multiple Instance Learning Classification of Lymphoma

In this study, we present a deep-learning-based multimodal classification method for lymphoma diagnosis in digital pathology, which utilizes a whole slide image (WSI) as the primary image data and flow cytometry (FCM) data as auxiliary information. In pathological diagnosis of malignant lymphoma, FCM serves as valuable auxiliary information during the diagnosis process, offering useful insights into predicting the major class (superclass) of subtypes. By incorporating both images and FCM data into the classification process, we can develop a method that mimics the diagnostic process of pathologists, enhancing the explainability. In order to incorporate the hierarchical structure between superclasses and their subclasses, the proposed method utilizes a network structure that effectively combines the mixture of experts (MoE) and multiple instance learning (MIL) techniques, where MIL is widely recognized for its effectiveness in handling WSIs in digital pathology. The MoE network in the proposed method consists of a gating network for superclass classification and multiple expert networks for (sub)class classification, specialized for each superclass. To evaluate the effectiveness of our method, we conducted experiments involving a six-class classification task using 600 lymphoma cases. The proposed method achieved a classification accuracy of 72.3%, surpassing the 69.5% obtained through the straightforward combination of FCM and images, as well as the 70.2% achieved by the method using only images. Moreover, the combination of multiple weights in the MoE and MIL allows for the visualization of specific cellular and tumor regions, resulting in a highly explanatory model that cannot be attained with conventional methods. It is anticipated that by targeting a larger number of classes and increasing the number of expert networks, the proposed method could be effectively applied to the real problem of lymphoma diagnosis.

Improving generalization capability of deep learning-based nuclei instance segmentation by non-deterministic train time and deterministic test time stain normalization

With the advent of digital pathology and microscopic systems that can scan and save whole slide histological images automatically, there is a growing trend to use computerized methods to analyze acquired images. Among different histopathological image analysis tasks, nuclei instance segmentation plays a fundamental role in a wide range of clinical and research applications. While many semi- and fully-automatic computerized methods have been proposed for nuclei instance segmentation, deep learning (DL)-based approaches have been shown to deliver the best performances. However, the performance of such approaches usually degrades when tested on unseen datasets. In this work, we propose a novel method to improve the generalization capability of a DL-based automatic segmentation approach. Besides utilizing one of the state-of-the-art DL-based models as a baseline, our method incorporates non-deterministic train time and deterministic test time stain normalization, and ensembling to boost the segmentation performance. We trained the model with one single training set and evaluated its segmentation performance on seven test datasets. Our results show that the proposed method provides up to 4.9%, 5.4%, and 5.9% better average performance in segmenting nuclei based on Dice score, aggregated Jaccard index, and panoptic quality score, respectively, compared to the baseline segmentation model.

Mathematical modelling and deep learning algorithms to automate assessment of single and digitally multiplexed immunohistochemical stains in tumoural stroma

Whilst automated analysis of immunostains in pathology research has focused predominantly on the epithelial compartment, automated analysis of stains in the stromal compartment is challenging and therefore requires time-consuming pathological input and guidance to adjust to tissue morphometry as perceived by pathologists. This study aimed to develop a robust method to automate stromal stain analyses using 2 of the commonest stromal stains (SMA and desmin) employed in clinical pathology practice as examples. An effective computational method capable of automatically assessing and quantifying tumour-associated stromal stains was developed and applied on cores of colorectal cancer tissue microarrays. The methodology combines both mathematical models and deep learning techniques with the former requiring no training data and the latter as many inputs as possible. The novel mathematical model was used to produce a digital double marker overlay allowing for fast automated digital multiplex analysis of stromal stains. The results show that deep learning methodologies in combination with mathematical modelling allow for an accurate means of quantifying stromal stains whilst also opening up new possibilities of digital multiplex analyses.

True-T - Improving T-cell response quantification with holistic artificial intelligence based prediction in immunohistochemistry images

The immune response associated with oncogenesis and potential oncological ther- apeutic interventions has dominated the field of cancer research over the last decade. T-cell lymphocytes in the tumor microenvironment are a crucial aspect of cancer's adaptive immunity, and the quantification of T-cells in specific can- cer types has been suggested as a potential diagnostic aid. However, this is cur- rently not part of routine diagnostics. To address this challenge, we present a new method called True-T, which employs artificial intelligence-based techniques to quantify T-cells in colorectal cancer (CRC) using immunohistochemistry (IHC) images. True-T analyses the chromogenic tissue hybridization signal of three widely recognized T-cell markers (CD3, CD4, and CD8). Our method employs a pipeline consisting of three stages: T-cell segmentation, density estimation from the segmented mask, and prediction of individual five-year survival rates. In the first stage, we utilize the U-Net method, where a pre-trained ResNet-34 is em- ployed as an encoder to extract clinically relevant T-cell features. The segmenta- tion model is trained and evaluated individually, demonstrating its generalization in detecting the CD3, CD4, and CD8 biomarkers in IHC images. In the second stage, the density of T-cells is estimated using the predicted mask, which serves as a crucial indicator for patient survival statistics in the third stage. This ap- proach was developed and tested in 1041 patients from four reference diagnostic institutions, ensuring broad applicability. The clinical effectiveness of True-T is demonstrated in stages II-IV CRC by offering valuable prognostic information that surpasses previous quantitative gold standards, opening possibilities for po- tential clinical applications. Finally, to evaluate the robustness and broader ap- plicability of our approach without additional training, we assessed the universal accuracy of the CD3 component of the True-T algorithm across 13 distinct solid tumors.

Role of artificial intelligence in digital pathology for gynecological cancers

The diagnosis of cancer is typically based on histopathological sections or biopsies on glass slides. Artificial intelligence (AI) approaches have greatly enhanced our ability to extract quantitative information from digital histopathology images as a rapid growth in oncology data. Gynecological cancers are major diseases affecting women's health worldwide. They are characterized by high mortality and poor prognosis, underscoring the critical importance of early detection, treatment, and identification of prognostic factors. This review highlights the various clinical applications of AI in gynecological cancers using digitized histopathology slides. Particularly, deep learning models have shown promise in accurately diagnosing, classifying histopathological subtypes, and predicting treatment response and prognosis. Furthermore, the integration with transcriptomics, proteomics, and other multi-omics techniques can provide valuable insights into the molecular features of diseases. Despite the considerable potential of AI, substantial challenges remain. Further improvements in data acquisition and model optimization are required, and the exploration of broader clinical applications, such as the biomarker discovery, need to be explored.

SlideTiler: A dataset creator software for boosting deep learning on histological whole slide images

The introduction of deep learning caused a significant breakthrough in digital pathology. Thanks to its capability of mining hidden data patterns in digitised histological slides to resolve diagnostic tasks and extract prognostic and predictive information. However, the high performance achieved in classification tasks depends on the availability of large datasets, whose collection and preprocessing are still time-consuming processes. Therefore, strategies to make these steps more efficient are worth investigation. This work introduces SlideTiler, an open-source software with a user-friendly graphical interface. SlideTiler can manage several image preprocessing phases through an intuitive workflow that does not require specific coding skills. The software was designed to provide direct access to virtual slides, allowing custom tiling of specific regions of interest drawn by the user, tile labelling, quality assessment, and direct export to dataset directories. To illustrate the functions and the scalability of SlideTiler, a deep learning-based classifier was implemented to classify 4 different tumour histotypes available in the TCGA repository. The results demonstrate the effectiveness of SlideTiler in facilitating data preprocessing and promoting accessibility to digitised pathology images for research purposes. Considering the increasing interest in deep learning applications of digital pathology, SlideTiler has a positive impact on this field. Moreover, SlideTiler has been conceived as a dynamic tool in constant evolution, and more updated and efficient versions will be released in the future.

Utility of artificial intelligence in a binary classification of soft tissue tumors

Soft tissue tumors (STTs) pose diagnostic and therapeutic challenges due to their rarity, complexity, and morphological overlap. Accurate differentiation between benign and malignant STTs is important to set treatment directions, however, this task can be difficult. The integration of machine learning and artificial intelligence (AI) models can potentially be helpful in classifying these tumors. The aim of this study was to investigate AI and machine learning tools in the classification of STT into benign and malignant categories. This study consisted of three components: (1) Evaluation of whole-slide images (WSIs) to classify STT into benign and malignant entities. Five specialized soft tissue pathologists from different medical centers independently reviewed 100 WSIs, representing 100 different cases, with limited clinical information and no additional workup. The results showed an overall concordance rate of 70.4% compared to the reference diagnosis. (2) Identification of cell-specific parameters that can distinguish benign and malignant STT. Using an image analysis software (QuPath) and a cohort of 95 cases, several cell-specific parameters were found to be statistically significant, most notably cell count, nucleus/cell area ratio, nucleus hematoxylin density mean, and cell max caliper. (3) Evaluation of machine learning library (Scikit-learn) in differentiating benign and malignant STTs. A total of 195 STT cases (156 cases in the training group and 39 cases in the validation group) achieved approximately 70% sensitivity and specificity, and an AUC of 0.68. Our limited study suggests that the use of WSI and AI in soft tissue pathology has the potential to enhance diagnostic accuracy and identify parameters that can differentiate between benign and malignant STTs. We envision the integration of AI as a supportive tool to augment the pathologists' diagnostic capabilities.

Crossing the Andes: Challenges and opportunities for digital pathology in Latin America

The most widely accepted and used type of digital pathology (DP) is whole-slide imaging (WSI). The USFDA granted two WSI system approvals for primary diagnosis, the first in 2017. In Latin America, DP has the potential to reshape healthcare by enhancing diagnostic capabilities through artificial intelligence (AI) and standardizing pathology reports. Yet, we must tackle regulatory hurdles, training, resource availability, and unique challenges to the region. Collectively addressing these hurdles can enable the region to harness DP's advantages-enhancing disease diagnosis, medical research, and healthcare accessibility for its population. Americas Health Foundation assembled a panel of Latin American pathologists who are experts in DP to assess the hurdles to implementing it into pathologists' workflows in the region and provide recommendations for overcoming them. Some key steps recommended include creating a Latin American Society of Digital Pathology to provide continuing education, developing AI models trained on the Latin American population, establishing national regulatory frameworks for protecting the data, and standardizing formats for DP images to ensure that pathologists can collaborate and validate specimens across the various DP platforms.

External validation of a deep learning-based algorithm for detection of tall cells in papillary thyroid carcinoma: A multicenter study

The tall cell subtype (TC-PTC) is an aggressive subtype of papillary thyroid carcinoma (PTC). The TC-PTC is defined as a PTC comprising at least 30% epithelial cells that are three times as tall as they are wide. In practice, this definition is difficult to adhere to, resulting in high inter-observer variability. In this multicenter study, we validated a previously trained deep learning (DL)-based algorithm for detection of tall cells on 160 externally collected hematoxylin and eosin (HE)-stained PTC whole-slide images. In a test set of 360 manual annotations of regions of interest from 18 separate tissue sections in the external dataset, the DL-based algorithm detected TCs with a sensitivity of 90.6% and a specificity of 88.5%. The DL algorithm detected non-TC areas with a sensitivity of 81.6% and a specificity of 92.9%. In the validation datasets, 20% and 30% TC thresholds correlated with a significantly shorter relapse-free survival. In conclusion, the DL algorithm detected TCs in unseen, external scanned HE tissue slides with high sensitivity and specificity without any retraining.

Quantitative digital pathology enables automated and quantitative assessment of inflammatory activity in patients with autoimmune hepatitis

Background

Chronic liver disease diagnoses depend on liver biopsy histopathological assessment. However, due to the limitations associated with biopsy, there is growing interest in the use of quantitative digital pathology to support pathologists. We evaluated the performance of computational algorithms in the assessment of hepatic inflammation in an autoimmune hepatitis in which inflammation is a major component.

Methods

Whole-slide digital image analysis was used to quantitatively characterize the area of tissue covered by inflammation [Inflammation Density (ID)] and number of inflammatory foci per unit area [Focal Density (FD)] on tissue obtained from 50 patients with autoimmune hepatitis undergoing routine liver biopsy. Correlations between digital pathology outputs and traditional categorical histology scores, biochemical, and imaging markers were assessed. The ability of ID and FD to stratify between low-moderate (both portal and lobular inflammation ≤1) and moderate-severe disease activity was estimated using the area under the receiver operating characteristic curve (AUC).

Results

ID and FD scores increased significantly and linearly with both portal and lobular inflammation grading. Both ID and FD correlated moderately-to-strongly and significantly with histology (portal and lobular inflammation; 0.36≤R≤0.69) and biochemical markers (ALT, AST, GGT, IgG, and gamma globulins; 0.43≤R≤0.57). ID (AUC: 0.85) and FD (AUC: 0.79) had good performance for stratifying between low-moderate and moderate-severe inflammation.

Conclusion

Quantitative assessment of liver biopsy using quantitative digital pathology metrics correlates well with traditional pathology scores and key biochemical markers. Whole-slide quantification of disease can support stratification and identification of patients with more advanced inflammatory disease activity.

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Number of intraepithelial lymphocytes and presence of a subepithelial band in normal colonic mucosa differs according to stainings and evaluation method

Chronic watery diarrhea is a frequent symptom. In approximately 10% of the patients, a diagnosis of microscopic colitis (MC) is established. The diagnosis relies on specific, but sometimes subtle, histopathological findings. As the histology of normal intestinal mucosa vary, discriminating subtle features of MC from normal tissue can be challenging and therefore auxiliary stainings are increasingly used. The aim of this study was to determine the variance in number of intraepithelial lymphocytes (IELs) and presence of a subepithelial band in normal ileum and colonic mucosa, according to different stains and digital assessment. Sixty-one patients without diarrhea referred to screening colonoscopy due to a positive feacal blood test and presenting with endoscopically normal mucosa were included. Basic histological features, number of IELs, and thickness of a subepithelial band was manually evaluated and a deep learning-based algorithm was developed to digitally determine the number of IELs in each of the two compartments; surface epithelium and cryptal epithelium, and the density of lymphocytes in the lamina propria compartment. The number of IELs was significantly higher on CD3-stained slides compared with slides stained with Hematoxylin-and-Eosin (HE) (p<0.001), and even higher numbers were reached using digital analysis. No significant difference between right and left colon in IELs or density of CD3-positive lymphocytes in lamina propria was found. No subepithelial band was present in HE-stained slides while a thin band was visualized on special stains. Conclusively, in this cohort of prospectively collected ileum and colonic biopsies from asymptomatic patients, the range of IELs and detection of a subepithelial collagenous band varied depending on the stain and method used for assessment. As assessment of biopsies from patients with diarrhea constitute a considerable workload in the pathology departments digital image analysis is highly desired. Knowledge provided by the present study highlight important differences that should be considered before introducing this method in the clinic.

A diagnostic strategy for pulmonary fat embolism based on routine H&E staining using computational pathology

Pulmonary fat embolism (PFE) as a cause of death often occurs in trauma cases such as fractures and soft tissue contusions. Traditional PFE diagnosis relies on subjective methods and special stains like oil red O. This study utilizes computational pathology, combining digital pathology and deep learning algorithms, to precisely quantify fat emboli in whole slide images using conventional hematoxylin-eosin (H&E) staining. The results demonstrate deep learning's ability to identify fat droplet morphology in lung microvessels, achieving an area under the receiver operating characteristic (ROC) curve (AUC) of 0.98. The AI-quantified fat globules generally matched the Falzi scoring system with oil red O staining. The relative quantity of fat emboli against lung area was calculated by the algorithm, determining a diagnostic threshold of 8.275% for fatal PFE. A diagnostic strategy based on this threshold achieved a high AUC of 0.984, similar to manual identification with special stains but surpassing H&E staining. This demonstrates computational pathology's potential as an affordable, rapid, and precise method for fatal PFE diagnosis in forensic practice.

An artificial intelligence model for detecting pathological lymph node metastasis in prostate cancer using whole slide images: a retrospective, multicentre, diagnostic study

Background

The pathological examination of lymph node metastasis (LNM) is crucial for treating prostate cancer (PCa). However, the limitations with naked-eye detection and pathologist workload contribute to a high missed-diagnosis rate for nodal micrometastasis. We aimed to develop an artificial intelligence (AI)-based, time-efficient, and high-precision PCa LNM detector (ProCaLNMD) and evaluate its clinical application value.

Methods

In this multicentre, retrospective, diagnostic study, consecutive patients with PCa who underwent radical prostatectomy and pelvic lymph node dissection at five centres between Sep 2, 2013 and Apr 28, 2023 were included, and histopathological slides of resected lymph nodes were collected and digitised as whole-slide images for model development and validation. ProCaLNMD was trained at a dataset from a single centre (the Sun Yat-sen Memorial Hospital of Sun Yat-sen University [SYSMH]), and externally validated in the other four centres. A bladder cancer dataset from SYSMH was used to further validate ProCaLNMD, and an additional validation (human-AI comparison and collaboration study) containing consecutive patients with PCa from SYSMH was implemented to evaluate the application value of integrating ProCaLNMD into the clinical workflow. The primary endpoint was the area under the receiver operating characteristic curve (AUROC) of ProCaLNMD. In addition, the performance measures for pathologists with ProCaLNMD assistance was also assessed.

Findings

In total, 8225 slides from 1297 patients with PCa were collected and digitised. Overall, 8158 slides (18,761 lymph nodes) from 1297 patients with PCa (median age 68 years [interquartile range 64-73]; 331 [26%] with LNM) were used to train and validate ProCaLNMD. The AUROC of ProCaLNMD ranged from 0.975 (95% confidence interval 0.953-0.998) to 0.992 (0.982-1.000) in the training and validation datasets, with sensitivities > 0.955 and specificities > 0.921. ProCaLNMD also demonstrated an AUROC of 0.979 in the cross-cancer dataset. ProCaLNMD use triggered true reclassification in 43 (4.3%) slides in which micrometastatic tumour regions were initially missed by pathologists, thereby correcting 28 (8.5%) missed-diagnosed cases of previous routine pathological reports. In the human-AI comparison and collaboration study, the sensitivity of ProCaLNMD (0.983 [0.908-1.000]) surpassed that of two junior pathologists (0.862 [0.746-0.939], P = 0.023; 0.879 [0.767-0.950], P = 0.041) by 10-12% and showed no difference to that of two senior pathologists (both 0.983 [0.908-1.000], both P > 0.99). Furthermore, ProCaLNMD significantly boosted the diagnostic sensitivity of two junior pathologists (both P = 0.041) to the level of senior pathologists (both P > 0.99), and substantially reduced the four pathologists' slide reviewing time (-31%, P < 0.0001; -34%, P < 0.0001; -29%, P < 0.0001; and -27%, P = 0.00031).

Interpretation

ProCaLNMD demonstrated high diagnostic capabilities for identifying LNM in prostate cancer, reducing the likelihood of missed diagnoses by pathologists and decreasing the slide reviewing time, highlighting its potential for clinical application.

Funding

National Natural Science Foundation of China, the Science and Technology Planning Project of Guangdong Province, the National Key Research and Development Programme of China, the Guangdong Provincial Clinical Research Centre for Urological Diseases, and the Science and Technology Projects in Guangzhou.

CellViT: Vision Transformers for precise cell segmentation and classification

Nuclei detection and segmentation in hematoxylin and eosin-stained (H&E) tissue images are important clinical tasks and crucial for a wide range of applications. However, it is a challenging task due to nuclei variances in staining and size, overlapping boundaries, and nuclei clustering. While convolutional neural networks have been extensively used for this task, we explore the potential of Transformer-based networks in combination with large scale pre-training in this domain. Therefore, we introduce a new method for automated instance segmentation of cell nuclei in digitized tissue samples using a deep learning architecture based on Vision Transformer called CellViT. CellViT is trained and evaluated on the PanNuke dataset, which is one of the most challenging nuclei instance segmentation datasets, consisting of nearly 200,000 annotated nuclei into 5 clinically important classes in 19 tissue types. We demonstrate the superiority of large-scale in-domain and out-of-domain pre-trained Vision Transformers by leveraging the recently published Segment Anything Model and a ViT-encoder pre-trained on 104 million histological image patches - achieving state-of-the-art nuclei detection and instance segmentation performance on the PanNuke dataset with a mean panoptic quality of 0.50 and an F1-detection score of 0.83. The code is publicly available at https://github.com/TIO-IKIM/CellViT.

Clinical evaluation of deep learning-based risk profiling in breast cancer histopathology and comparison to an established multigene assay

PURPOSE

To evaluate the Stratipath Breast tool for image-based risk profiling and compare it with an established prognostic multigene assay for risk profiling in a real-world case series of estrogen receptor (ER)-positive and human epidermal growth factor receptor 2 (HER2)-negative early breast cancer patients categorized as intermediate risk based on classic clinicopathological variables and eligible for chemotherapy.

METHODS

In a case series comprising 234 invasive ER-positive/HER2-negative tumors, clinicopathological data including Prosigna results and corresponding HE-stained tissue slides were retrieved. The digitized HE slides were analysed by Stratipath Breast.

RESULTS

Our findings showed that the Stratipath Breast analysis identified 49.6% of the clinically intermediate tumors as low risk and 50.4% as high risk. The Prosigna assay classified 32.5%, 47.0% and 20.5% tumors as low, intermediate and high risk, respectively. Among Prosigna intermediate-risk tumors, 47.3% were stratified as Stratipath low risk and 52.7% as high risk. In addition, 89.7% of Stratipath low-risk cases were classified as Prosigna low/intermediate risk. The overall agreement between the two tests for low-risk and high-risk groups (N = 124) was 71.0%, with a Cohen's kappa of 0.42. For both risk profiling tests, grade and Ki67 differed significantly between risk groups.

CONCLUSION

The results from this clinical evaluation of image-based risk stratification shows a considerable agreement to an established gene expression assay in routine breast pathology.

Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval

Large-scale digital whole slide image (WSI) datasets analysis have gained significant attention in computer-aided cancer diagnosis. Content-based histopathological image retrieval (CBHIR) is a technique that searches a large database for data samples matching input objects in both details and semantics, offering relevant diagnostic information to pathologists. However, the current methods are limited by the difficulty of gigapixels, the variable size of WSIs, and the dependence on manual annotations. In this work, we propose a novel histopathology language-image representation learning framework for fine-grained digital pathology cross-modal retrieval, which utilizes paired diagnosis reports to learn fine-grained semantics from the WSI. An anchor-based WSI encoder is built to extract hierarchical region features and a prompt-based text encoder is introduced to learn fine-grained semantics from the diagnosis reports. The proposed framework is trained with a multivariate cross-modal loss function to learn semantic information from the diagnosis report at both the instance level and region level. After training, it can perform four types of retrieval tasks based on the multi-modal database to support diagnostic requirements. We conducted experiments on an in-house dataset and a public dataset to evaluate the proposed method. Extensive experiments have demonstrated the effectiveness of the proposed method and its advantages to the present histopathology retrieval methods. The code is available at https://github.com/hudingyi/FGCR.

EpiDiP/NanoDiP: a versatile unsupervised machine learning edge computing platform for epigenomic tumour diagnostics

DNA methylation analysis based on supervised machine learning algorithms with static reference data, allowing diagnostic tumour typing with unprecedented precision, has quickly become a new standard of care. Whereas genome-wide diagnostic methylation profiling is mostly performed on microarrays, an increasing number of institutions additionally employ nanopore sequencing as a faster alternative. In addition, methylation-specific parallel sequencing can generate methylation and genomic copy number data. Given these diverse approaches to methylation profiling, to date, there is no single tool that allows (1) classification and interpretation of microarray, nanopore and parallel sequencing data, (2) direct control of nanopore sequencers, and (3) the integration of microarray-based methylation reference data. Furthermore, no software capable of entirely running in routine diagnostic laboratory environments lacking high-performance computing and network infrastructure exists. To overcome these shortcomings, we present EpiDiP/NanoDiP as an open-source DNA methylation and copy number profiling suite, which has been benchmarked against an established supervised machine learning approach using in-house routine diagnostics data obtained between 2019 and 2021. Running locally on portable, cost- and energy-saving system-on-chip as well as gpGPU-augmented edge computing devices, NanoDiP works in offline mode, ensuring data privacy. It does not require the rigid training data annotation of supervised approaches. Furthermore, NanoDiP is the core of our public, free-of-charge EpiDiP web service which enables comparative methylation data analysis against an extensive reference data collection. We envision this versatile platform as a useful resource not only for neuropathologists and surgical pathologists but also for the tumour epigenetics research community. In daily diagnostic routine, analysis of native, unfixed biopsies by NanoDiP delivers molecular tumour classification in an intraoperative time frame.

Prognostic and predictive role of immune microenvironment in colorectal cancer

Colorectal cancer (CRC) represents a molecularly heterogeneous disease and one of the most frequent causes of cancer-related death worldwide. The traditional classification of CRC is based on pathomorphological and molecular characteristics of tumor cells (mucinous, ring-cell carcinomas, etc.), analysis of mechanisms of carcinogenesis involved (chromosomal instability, microsatellite instability, CpG island methylator phenotype) and mutational statuses of commonly altered genes (KRAS, NRAS, BRAF, APC, etc.), as well as expression signatures (CMS 1-4). It is also suggested that the tumor microenvironment is a key player in tumor progression and metastasis in CRC. According to the latest data, the immune microenvironment can also be predictive of the response to immune checkpoint inhibitors. In this review, we highlight how the immune environment influences CRC prognosis and sensitivity to systemic therapy.

Glioma subtype classification from histopathological images using in-domain and out-of-domain transfer learning: An experimental study

We provide in this paper a comprehensive comparison of various transfer learning strategies and deep learning architectures for computer-aided classification of adult-type diffuse gliomas. We evaluate the generalizability of out-of-domain ImageNet representations for a target domain of histopathological images, and study the impact of in-domain adaptation using self-supervised and multi-task learning approaches for pretraining the models using the medium-to-large scale datasets of histopathological images. A semi-supervised learning approach is furthermore proposed, where the fine-tuned models are utilized to predict the labels of unannotated regions of the whole slide images (WSI). The models are subsequently retrained using the ground-truth labels and weak labels determined in the previous step, providing superior performance in comparison to standard in-domain transfer learning with balanced accuracy of 96.91% and F1-score 97.07%, and minimizing the pathologist's efforts for annotation. Finally, we provide a visualization tool working at WSI level which generates heatmaps that highlight tumor areas; thus, providing insights to pathologists concerning the most informative parts of the WSI.

[Artificial intelligence for pathology-how, where, and why?]

Artificial intelligence promises many innovations and simplifications in pathology, but also raises just as many questions and uncertainties. In this article, we provide a brief overview of the current status, the goals already achieved by existing algorithms, and the remaining challenges.

Histological criteria for selecting patients who need clonality test for non-gastric MALT lymphoma diagnosis

The histological diagnosis of extranodal marginal zone lymphoma of the mucosa-associated lymphoid tissue (MALT lymphoma) is difficult for pathologists. Recently, digital pathology systems have been widely used to provide tools that can objectively measure lesions on slides. In this study, we measured the extent of marginal zone expansion in suspected MALT lymphoma cases and compared the results with those of a molecular clonality test. In total, 115 patients who underwent an IGH gene rearrangement test for suspected MALT lymphoma were included in this study. All cases were histologically classified into three patterns; "small lymphoid aggregates with no germinal center (Pattern 1)," "lymphoid follicles with germinal center (Pattern 2)" and " fused marginal zone or diffuse small lymphocytic proliferation (Pattern 3)." The proportions of monoclonality in Pattern 1, 2, and 3 were 25.0%, 55.0%, and 97.9%, respectively. The ratios of marginal zone thickness to germinal center diameter and entire lymphoid follicle area to germinal center area were measured in Pattern 2 cases using a digital pathology system. Combining the width cutoff of 1.5 and the areal cutoff of 3.5, the sensitivity, specificity, positive predictive value, and negative predictive value for MALT lymphoma were 96.97%, 70.37%, 80.00%, and 95.00%, respectively. In conclusion, through objective measurement of the marginal zone, suspected cases of MALT lymphoma requiring a molecular clonality test can be effectively selected.

[Fluorescence confocal microscopy-complete digitization of pathology]

BACKGROUND

Fluorescence-based confocal microscopy (FCM) can be used to create virtual H&E sections in real time. So far, FCM has been used in dermato-, uro-, and gynecopathology. FCM allows the creation of a completely digitized frozen section, which could potentially replace conventional frozen sections in the future.

OBJECTIVE

The aim of the current work is to implement FCM technology as a component of fully digitized processes in the pathological workflow. For this purpose, the current use of FCM in liver transplant pathology will be extended to other disciplines such as urology and otorhinolaryngology.

MATERIALS AND METHODS

The FCM technique continues to be used prospectively on native tissue samples from potential donor livers. Conventional frozen sections are used comparatively to virtual FCM scans.

RESULTS

The data show a nearly perfect agreement for the detection of cholangitis, fibrosis, and malignancy, and a high level of agreement for, e.g., macrovesicular steatosis, inflammation, steatohepatitis, and necrosis between virtual FCM scans and conventional routine diagnostic frozen sections.

CONCLUSION

Since the availability of time- and cost-intensive frozen section diagnostics in the context of transplant pathology in continuous operation (24/7) is now only established at very few university centers in Germany due to an increasing shortage of specialists, the use of FCM could be an important building block in the current process leading towards a fully digitized pathology workflow and should thus be extended to various disciplines.

MMIR: an open-source software for the registration of multimodal histological images

BACKGROUND

Multimodal histology image registration is a process that transforms into a common coordinate system two or more images obtained from different microscopy modalities. The combination of information from various modalities can contribute to a comprehensive understanding of tissue specimens, aiding in more accurate diagnoses, and improved research insights. Multimodal image registration in histology samples presents a significant challenge due to the inherent differences in characteristics and the need for tailored optimization algorithms for each modality.

RESULTS

We developed MMIR a cloud-based system for multimodal histological image registration, which consists of three main modules: a project manager, an algorithm manager, and an image visualization system.

CONCLUSION

Our software solution aims to simplify image registration tasks with a user-friendly approach. It facilitates effective algorithm management, responsive web interfaces, supports multi-resolution images, and facilitates batch image registration. Moreover, its adaptable architecture allows for the integration of custom algorithms, ensuring that it aligns with the specific requirements of each modality combination. Beyond image registration, our software enables the conversion of segmented annotations from one modality to another.

Computational pathology model to assess acute and chronic transformations of the tubulointerstitial compartment in renal allograft biopsies

Managing patients with kidney allografts largely depends on biopsy diagnosis which is based on semiquantitative assessments of rejection features and extent of acute and chronic changes within the renal parenchyma. Current methods lack reproducibility while digital image data-driven computational models enable comprehensive and quantitative assays. In this study we aimed to develop a computational method for automated assessment of histopathology transformations within the tubulointerstitial compartment of the renal cortex. Whole slide images of modified Picrosirius red-stained biopsy slides were used for the training (n = 852) and both internal (n = 172) and external (n = 94) tests datasets. The pipeline utilizes deep learning segmentations of renal tubules, interstitium, and peritubular capillaries from which morphometry features were extracted. Seven indicators were selected for exploring the intrinsic spatial interactions within the tubulointerstitial compartment. A principal component analysis revealed two independent factors which can be interpreted as representing chronic and acute tubulointerstitial injury. A K-means clustering classified biopsies according to potential phenotypes of combined acute and chronic transformations of various degrees. We conclude that multivariate analyses of tubulointerstitial morphometry transformations enable extraction of and quantification of acute and chronic components of injury. The method is developed for renal allograft biopsies; however, the principle can be applied more broadly for kidney pathology assessment.

[Digital transformation of a routine histopathology lab : Dos and don'ts!]

The implementation of digital histopathology in the laboratory marks a crucial milestone in the overall digital transformation of pathology. This shift offers a range of new possibilities, including access to extensive datasets for AI-assisted analyses, the flexibility of remote work and home office arrangements for specialists, and the expedited and simplified sharing of images and data for research, conferences, and tumor boards. However, the transition to a fully digital workflow involves significant technological and personnel-related efforts. It necessitates careful and adaptable change management to minimize disruptions, particularly in the personnel domain, and to prevent the loss of valuable potential from employees who may be resistant to change. This article consolidates our institute's experiences, highlighting technical and personnel-related challenges encountered during the transition to digital pathology. It also presents a comprehensive overview of potential difficulties at various interfaces when converting routine operations to a digital workflow.

Immunohistochemical double nuclear staining for cell-specific automated quantification of the proliferation index - A promising diagnostic aid for melanocytic lesions

AIMS

Pathologists often use immunohistochemical staining of the proliferation marker Ki67 in their diagnostic assessment of melanocytic lesions. However, the interpretation of Ki67 can be challenging. We propose a new workflow to improve the diagnostic utility of the Ki67-index. In this workflow, Ki67 is combined with the melanocytic tumour-cell marker SOX10 in a Ki67/SOX10 double nuclear stain. The Ki67-index is then quantified automatically using digital image analysis (DIA). The aim of this study was to optimise and test three different multiplexing methods for Ki67/SOX10 double nuclear staining.

METHODS

Multiplex immunofluorescence (mIF), multiplex immunohistochemistry (mIHC), and multiplexed immunohistochemical consecutive staining on single slide (MICSSS) were optimised for Ki67/SOX10 double nuclear staining. DIA applications were designed for automated quantification of the Ki67-index. The methods were tested on a pilot case-control cohort of benign and malignant melanocytic lesions (n = 23).

RESULTS

Using the Ki67/SOX10 double nuclear stain, malignant melanocytic lesions could be completely distinguished from benign lesions by the Ki67-index. The Ki67-index cut-offs were 1.8% (mIF) and 1.5% (mIHC and MICSSS). The AUC of the automatically quantified Ki67-index based on double nuclear staining was 1.0 (95% CI: 1.0;1.0), whereas the AUC of conventional Ki67 single-stains was 0.87 (95% CI: 0.71;1.00).

CONCLUSIONS

The novel Ki67/SOX10 double nuclear stain highly improved the diagnostic precision of Ki67 interpretation. Both mIHC and mIF were useful methods for Ki67/SOX10 double nuclear staining, whereas the MICSSS method had challenges in the current setting. The Ki67/SOX10 double nuclear stain shows potential as a valuable diagnostic aid for melanocytic lesions.

From Pixels to Prognosis: A Survey on AI-Driven Cancer Patient Survival Prediction Using Digital Histology Images

Survival analysis is an integral part of medical statistics that is extensively utilized to establish prognostic indices for mortality or disease recurrence, assess treatment efficacy, and tailor effective treatment plans. The identification of prognostic biomarkers capable of predicting patient survival is a primary objective in the field of cancer research. With the recent integration of digital histology images into routine clinical practice, a plethora of Artificial Intelligence (AI)-based methods for digital pathology has emerged in scholarly literature, facilitating patient survival prediction. These methods have demonstrated remarkable proficiency in analyzing and interpreting whole slide images, yielding results comparable to those of expert pathologists. The complexity of AI-driven techniques is magnified by the distinctive characteristics of digital histology images, including their gigapixel size and diverse tissue appearances. Consequently, advanced patch-based methods are employed to effectively extract features that correlate with patient survival. These computational methods significantly enhance survival prediction accuracy and augment prognostic capabilities in cancer patients. The review discusses the methodologies employed in the literature, their performance metrics, ongoing challenges, and potential solutions for future advancements. This paper explains survival analysis and feature extraction methods for analyzing cancer patients. It also compiles essential acronyms related to cancer precision medicine. Furthermore, it is noteworthy that this is the inaugural review paper in the field. The target audience for this interdisciplinary review comprises AI practitioners, medical statisticians, and progressive oncologists who are enthusiastic about translating AI-driven solutions into clinical practice. We expect this comprehensive review article to guide future research directions in the field of cancer research.

The influence of resolution on the predictive power of spatial heterogeneity measures as biomarkers of liver fibrosis

Spatial heterogeneity of cells in liver biopsies can be used as biomarker for disease severity of patients. This heterogeneity can be quantified by non-parametric statistics of point pattern data, which make use of an aggregation of the point locations. The method and scale of aggregation are usually chosen ad hoc, despite values of the aforementioned statistics being heavily dependent on them. Moreover, in the context of measuring heterogeneity, increasing spatial resolution will not endlessly provide more accuracy. The question then becomes how changes in resolution influence heterogeneity indicators, and subsequently how they influence their predictive abilities. In this paper, cell level data of liver biopsy tissue taken from chronic Hepatitis B patients is used to analyze this issue. Firstly, Morisita-Horn indices, Shannon indices and Getis-Ord statistics were evaluated as heterogeneity indicators of different types of cells, using multiple resolutions. Secondly, the effect of resolution on the predictive performance of the indices in an ordinal regression model was investigated, as well as their importance in the model. A simulation study was subsequently performed to validate the aforementioned methods. In general, for specific heterogeneity indicators, a downward trend in predictive performance could be observed. While for local measures of heterogeneity a smaller grid-size is outperforming, global measures have a better performance with medium-sized grids. In addition, the use of both local and global measures of heterogeneity is recommended to improve the predictive performance.

Multiple instance learning for digital pathology: A review of the state-of-the-art, limitations & future potential

Digital whole slides images contain an enormous amount of information providing a strong motivation for the development of automated image analysis tools. Particularly deep neural networks show high potential with respect to various tasks in the field of digital pathology. However, a limitation is given by the fact that typical deep learning algorithms require (manual) annotations in addition to the large amounts of image data, to enable effective training. Multiple instance learning exhibits a powerful tool for training deep neural networks in a scenario without fully annotated data. These methods are particularly effective in the domain of digital pathology, due to the fact that labels for whole slide images are often captured routinely, whereas labels for patches, regions, or pixels are not. This potential resulted in a considerable number of publications, with the vast majority published in the last four years. Besides the availability of digitized data and a high motivation from the medical perspective, the availability of powerful graphics processing units exhibits an accelerator in this field. In this paper, we provide an overview of widely and effectively used concepts of (deep) multiple instance learning approaches and recent advancements. We also critically discuss remaining challenges as well as future potential.

Challenges and barriers of using large language models (LLM) such as ChatGPT for diagnostic medicine with a focus on digital pathology - a recent scoping review

BACKGROUND

The integration of large language models (LLMs) like ChatGPT in diagnostic medicine, with a focus on digital pathology, has garnered significant attention. However, understanding the challenges and barriers associated with the use of LLMs in this context is crucial for their successful implementation.

METHODS

A scoping review was conducted to explore the challenges and barriers of using LLMs, in diagnostic medicine with a focus on digital pathology. A comprehensive search was conducted using electronic databases, including PubMed and Google Scholar, for relevant articles published within the past four years. The selected articles were critically analyzed to identify and summarize the challenges and barriers reported in the literature.

RESULTS

The scoping review identified several challenges and barriers associated with the use of LLMs in diagnostic medicine. These included limitations in contextual understanding and interpretability, biases in training data, ethical considerations, impact on healthcare professionals, and regulatory concerns. Contextual understanding and interpretability challenges arise due to the lack of true understanding of medical concepts and lack of these models being explicitly trained on medical records selected by trained professionals, and the black-box nature of LLMs. Biases in training data pose a risk of perpetuating disparities and inaccuracies in diagnoses. Ethical considerations include patient privacy, data security, and responsible AI use. The integration of LLMs may impact healthcare professionals' autonomy and decision-making abilities. Regulatory concerns surround the need for guidelines and frameworks to ensure safe and ethical implementation.

CONCLUSION

The scoping review highlights the challenges and barriers of using LLMs in diagnostic medicine with a focus on digital pathology. Understanding these challenges is essential for addressing the limitations and developing strategies to overcome barriers. It is critical for health professionals to be involved in the selection of data and fine tuning of the models. Further research, validation, and collaboration between AI developers, healthcare professionals, and regulatory bodies are necessary to ensure the responsible and effective integration of LLMs in diagnostic medicine.

Augmented interpretation of HER2, ER, and PR in breast cancer by artificial intelligence analyzer: enhancing interobserver agreement through a reader study of 201 cases

BACKGROUND

Accurate classification of breast cancer molecular subtypes is crucial in determining treatment strategies and predicting clinical outcomes. This classification largely depends on the assessment of human epidermal growth factor receptor 2 (HER2), estrogen receptor (ER), and progesterone receptor (PR) status. However, variability in interpretation among pathologists pose challenges to the accuracy of this classification. This study evaluates the role of artificial intelligence (AI) in enhancing the consistency of these evaluations.

METHODS

AI-powered HER2 and ER/PR analyzers, consisting of cell and tissue models, were developed using 1,259 HER2, 744 ER, and 466 PR-stained immunohistochemistry (IHC) whole-slide images of breast cancer. External validation cohort comprising HER2, ER, and PR IHCs of 201 breast cancer cases were analyzed with these AI-powered analyzers. Three board-certified pathologists independently assessed these cases without AI annotation. Then, cases with differing interpretations between pathologists and the AI analyzer were revisited with AI assistance, focusing on evaluating the influence of AI assistance on the concordance among pathologists during the revised evaluation compared to the initial assessment.

RESULTS

Reevaluation was required in 61 (30.3%), 42 (20.9%), and 80 (39.8%) of HER2, in 15 (7.5%), 17 (8.5%), and 11 (5.5%) of ER, and in 26 (12.9%), 24 (11.9%), and 28 (13.9%) of PR evaluations by the pathologists, respectively. Compared to initial interpretations, the assistance of AI led to a notable increase in the agreement among three pathologists on the status of HER2 (from 49.3 to 74.1%, p < 0.001), ER (from 93.0 to 96.5%, p = 0.096), and PR (from 84.6 to 91.5%, p = 0.006). This improvement was especially evident in cases of HER2 2+ and 1+, where the concordance significantly increased from 46.2 to 68.4% and from 26.5 to 70.7%, respectively. Consequently, a refinement in the classification of breast cancer molecular subtypes (from 58.2 to 78.6%, p < 0.001) was achieved with AI assistance.

CONCLUSIONS

This study underscores the significant role of AI analyzers in improving pathologists' concordance in the classification of breast cancer molecular subtypes.

Artificial intelligence's impact on breast cancer pathology: a literature review

This review discusses the profound impact of artificial intelligence (AI) on breast cancer (BC) diagnosis and management within the field of pathology. It examines the various applications of AI across diverse aspects of BC pathology, highlighting key findings from multiple studies. Integrating AI into routine pathology practice stands to improve diagnostic accuracy, thereby contributing to reducing avoidable errors. Additionally, AI has excelled in identifying invasive breast tumors and lymph node metastasis through its capacity to process large whole-slide images adeptly. Adaptive sampling techniques and powerful convolutional neural networks mark these achievements. The evaluation of hormonal status, which is imperative for BC treatment choices, has also been enhanced by AI quantitative analysis, aiding interobserver concordance and reliability. Breast cancer grading and mitotic count evaluation also benefit from AI intervention. AI-based frameworks effectively classify breast carcinomas, even for moderately graded cases that traditional methods struggle with. Moreover, AI-assisted mitotic figures quantification surpasses manual counting in precision and sensitivity, fostering improved prognosis. The assessment of tumor-infiltrating lymphocytes in triple-negative breast cancer using AI yields insights into patient survival prognosis. Furthermore, AI-powered predictions of neoadjuvant chemotherapy response demonstrate potential for streamlining treatment strategies. Addressing limitations, such as preanalytical variables, annotation demands, and differentiation challenges, is pivotal for realizing AI's full potential in BC pathology. Despite the existing hurdles, AI's multifaceted contributions to BC pathology hold great promise, providing enhanced accuracy, efficiency, and standardization. Continued research and innovation are crucial for overcoming obstacles and fully harnessing AI's transformative capabilities in breast cancer diagnosis and assessment.

Predicting Mismatch Repair Deficiency Status in Endometrial Cancer through Multi-Resolution Ensemble Learning in Digital Pathology

For molecular classification of endometrial carcinoma, testing for mismatch repair (MMR) status is becoming a routine process. Mismatch repair deficiency (MMR-D) is caused by loss of expression in one or more of the 4 major MMR proteins: MLH1, MSH2, MSH6, PHS2. Over 30% of patients with endometrial cancer have MMR-D. Determining the MMR status holds significance as individuals with MMR-D are potential candidates for immunotherapy. Pathological whole slide image (WSI) of endometrial cancer with immunohistochemistry results of MMR proteins were gathered. Color normalization was applied to the tiles using a CycleGAN-based network. The WSI was divided into tiles at three different magnifications (2.5 × , 5 × , and 10 ×). Three distinct networks of the same architecture were employed to include features from all three magnification levels and were stacked for ensemble learning. Three architectures, InceptionResNetV2, EfficientNetB2, and EfficientNetB3 were employed and subjected to comparison. The per-tile results were gathered to classify MMR status in the WSI, and prediction accuracy was evaluated using the following performance metrics: AUC, accuracy, sensitivity, and specificity. The EfficientNetB2 was able to make predictions with an AUC of 0.821, highest among the three architectures, and an overall AUC range of 0.767 - 0.821 was reported across the three architectures. In summary, our study successfully predicted MMR classification from pathological WSIs in endometrial cancer through a multi-resolution ensemble learning approach, which holds the potential to facilitate swift decisions on tailored treatment, such as immunotherapy, in clinical settings.

Extended correlation functions for spatial analysis of multiplex imaging data

Imaging platforms for generating highly multiplexed histological images are being continually developed and improved. Significant improvements have also been made in the accuracy of methods for automated cell segmentation and classification. However, less attention has focused on the quantification and analysis of the resulting point clouds, which describe the spatial coordinates of individual cells. We focus here on a particular spatial statistical method, the cross-pair correlation function (cross-PCF), which can identify positive and negative spatial correlation between cells across a range of length scales. However, limitations of the cross-PCF hinder its widespread application to multiplexed histology. For example, it can only consider relations between pairs of cells, and cells must be classified using discrete categorical labels (rather than labeling continuous labels such as stain intensity). In this paper, we present three extensions to the cross-PCF which address these limitations and permit more detailed analysis of multiplex images: topographical correlation maps can visualize local clustering and exclusion between cells; neighbourhood correlation functions can identify colocalization of two or more cell types; and weighted-PCFs describe spatial correlation between points with continuous (rather than discrete) labels. We apply the extended PCFs to synthetic and biological datasets in order to demonstrate the insight that they can generate.

Algorithm-assisted diagnosis of Hirschsprung's disease - evaluation of robustness and comparative image analysis on data from various labs and slide scanners

BACKGROUND

Differences in the preparation, staining and scanning of digital pathology slides create significant pre-analytic variability. Algorithm-assisted tools must be able to contend with this variability in order to be applicable in clinical practice. In a previous study, a decision support algorithm was developed to assist in the diagnosis of Hirschsprung's disease. In the current study, we tested the robustness of this algorithm while assessing for pre-analytic factors which may affect its performance.

METHODS

The decision support algorithm was used on digital pathology slides obtained from four different medical centers (A-D) and scanned by three different scanner models (by Philips, Hamamatsu and 3DHISTECH). A total of 192 cases and 1782 slides were used in this study. RGB histograms were constructed to compare images from the various medical centers and scanner models and highlight the differences in color and contrast.

RESULTS

The algorithm was able to correctly identify ganglion cells in 99.2% of cases, from all medical centers (All scanned by the Philips slide scanner) as well as 95.5% and 100% of the slides scanned by the 3DHISTECH and Hamamatsu brand slide scanners, respectively. The total error rate for center D was lower than the other medical centers (3.9% vs 7.1%, 10.8% and 6% for centers A-C, respectively), the vast majority of errors being false positives (3.45% vs 0.45% false negatives). The other medical centers showed a higher rate of false negatives in relation to false positives (6.81% vs 0.29%, 9.8% vs 1.2% and 5.37% vs 0.63% for centers A-C, respectively). The total error rates for the Philips, Hamamatsu and 3DHISTECH brand scanners were 3.9%, 3.2% and 9.8%, respectively. RGB histograms demonstrated significant differences in pixel value distribution between the four medical centers, as well as between the 3DHISTECH brand scanner when compared to the Philips and Hamamatsu brand scanners.

CONCLUSIONS

The results reported in this paper suggest that the algorithm-based decision support system has sufficient robustness to be applicable for clinical practice. In addition, the novel method used in its development - Hierarchial-Contexual Analysis (HCA) may be applicable to the development of algorithm-assisted tools in other diseases, for which available datasets are limited. Validation of any given algorithm-assisted support system should nonetheless include data from as many medical centers and scanner models as possible.

Artificial Intelligence and Pathomics: Prostate Cancer

Artificial intelligence (AI) has the potential to transform pathologic diagnosis and cancer patient management as a predictive and prognostic biomarker. AI-based systems can be used to examine digitally scanned histopathology slides and differentiate benign from malignant cells and low from high grade. Deep learning models can analyze patient data from individual or multimodal combinations and identify patterns to be used to predict the response to different therapeutic options, the risk of recurrence or progression, and the prognosis of the newly diagnosed patient. AI-based models will improve treatment planning for patients with prostate cancer and improve the efficiency and cost-effectiveness of the pathology laboratory.

Enhancing mitosis quantification and detection in meningiomas with computational digital pathology

Mitosis is a critical criterion for meningioma grading. However, pathologists' assessment of mitoses is subject to significant inter-observer variation due to challenges in locating mitosis hotspots and accurately detecting mitotic figures. To address this issue, we leverage digital pathology and propose a computational strategy to enhance pathologists' mitosis assessment. The strategy has two components: (1) A depth-first search algorithm that quantifies the mathematically maximum mitotic count in 10 consecutive high-power fields, which can enhance the preciseness, especially in cases with borderline mitotic count. (2) Implementing a collaborative sphere to group a set of pathologists to detect mitoses under each high-power field, which can mitigate subjective random errors in mitosis detection originating from individual detection errors. By depth-first search algorithm (1) , we analyzed 19 meningioma slides and discovered that the proposed algorithm upgraded two borderline cases verified at consensus conferences. This improvement is attributed to the algorithm's ability to quantify the mitotic count more comprehensively compared to other conventional methods of counting mitoses. In implementing a collaborative sphere (2) , we evaluated the correctness of mitosis detection from grouped pathologists and/or pathology residents, where each member of the group annotated a set of 48 high-power field images for mitotic figures independently. We report that groups with sizes of three can achieve an average precision of 0.897 and sensitivity of 0.699 in mitosis detection, which is higher than an average pathologist in this study (precision: 0.750, sensitivity: 0.667). The proposed computational strategy can be integrated with artificial intelligence workflow, which envisions the future of achieving a rapid and robust mitosis assessment by interactive assisting algorithms that can ultimately benefit patient management.

Digital pathology in cardiac transplant diagnostics: from biopsies to algorithms

In the field of heart transplantation, the ability to accurately and promptly diagnose cardiac allograft rejection is crucial. This comprehensive review explores the transformative role of digital pathology and computational pathology, especially through machine learning, in this critical domain. These methodologies harness large datasets to extract subtle patterns and valuable information that extend beyond human perceptual capabilities, potentially enhancing diagnostic outcomes. Current research indicates that these computer-based systems could offer accuracy and performance matching, or even exceeding, that of expert pathologists, thereby introducing more objectivity and reducing observer variability. Despite promising results, several challenges such as limited sample sizes, diverse data sources, and the absence of standardized protocols pose significant barriers to the widespread adoption of these techniques. The future of digital pathology in heart transplantation diagnostics depends on utilizing larger, more diverse patient cohorts, standardizing data collection, processing, and evaluation protocols, and fostering collaborative research efforts. The integration of various data types, including clinical, demographic, and imaging information, could further refine diagnostic precision. As researchers address these challenges and promote collaborative efforts, digital pathology has the potential to become an integral part of clinical practice, ultimately improving patient care in heart transplantation.

A novel technique to estimate intravillous fetal vasculature on routine placenta histologic sections

Placental histopathologic lesions are dichotomized into "present" or "absent" and have limited inter-rater reliability. Continuous metrics are needed to characterize placental health and function. Tissue sections (N = 64) of human placenta were stained with CD34 antibody and hematoxylin. Proportion of the villous space occupied by fetal vascular endothelium (%FVE; pixels positive for CD34/total pixels) was evaluated for effect sizes associated with pregnancy outcomes, smoking status, and subtypes of lesions (n = 30). Time to fixation>60 min significantly increased the quantification. Large effect sizes were found between %FVE and both preterm birth and intrauterine growth restriction. These results demonstrate proof-of-concept for this vascular estimation.

Machine learning and machine teaching in histopathology

An artificial intelligence (AI) platform was trained by a consultant histopathologist to classify whole slide images (WSIs) of large bowel biopsies. Six medical students viewed WSIs of five large bowel biopsy cases and assigned the WSIs to one of the nine diagnostic categories. Then the students compared their answers with those generated by the AI. This training was repeated for a total of six rounds of five cases, and the accuracy of the students was recorded for each round. Each case had one or more WSIs. The student with the best final accuracy was asked to describe the morphological features that they had deduced. All the students improved during their training, from a mean accuracy of 13.7% in the first round to a mean accuracy of 77.1% in the sixth round (p = 0.0011). The student-deduced diagnostic features were mainly accurate. Some students learned more quickly than others.

FFPE++: Improving the quality of formalin-fixed paraffin-embedded tissue imaging via contrastive unpaired image-to-image translation

Formalin-fixation and paraffin-embedding (FFPE) is a technique for preparing and preserving tissue specimens that has been utilized in histopathology since the late 19th century. This process is further complicated by FFPE preparation steps such as fixation, processing, embedding, microtomy, staining, and coverslipping, which often results in artifacts due to the complex histological and cytological characteristics of a tissue specimen. The term "artifacts" includes, but is not limited to, staining inconsistencies, tissue folds, chattering, pen marks, blurring, air bubbles, and contamination. The presence of artifacts may interfere with pathological diagnosis in disease detection, subtyping, grading, and choice of therapy. In this study, we propose FFPE++, an unpaired image-to-image translation method based on contrastive learning with a mixed channel-spatial attention module and self-regularization loss that drastically corrects the aforementioned artifacts in FFPE tissue sections. Turing tests were performed by 10 board-certified pathologists with more than 10 years of experience. These tests which were performed for ovarian carcinoma, lung adenocarcinoma, lung squamous cell carcinoma, and papillary thyroid carcinoma, demonstrate the clear superiority of the proposed method in many clinical aspects compared with standard FFPE images. Based on the qualitative experiments and feedback from the Turing tests, we believe that FFPE++ can contribute to substantial diagnostic and prognostic accuracy in clinical pathology in the future and can also improve the performance of AI tools in digital pathology. The code and dataset are publicly available at https://github.com/DeepMIALab/FFPEPlus.

Deep learning based histological classification of adnex tumors

BACKGROUND

Cutaneous adnexal tumors are a diverse group of tumors arising from structures of the hair appendages. Although often benign, malignant entities occur which can metastasize and lead to patients´ death. Correct diagnosis is critical to ensure optimal treatment and best possible patient outcome. Artificial intelligence (AI) in the form of deep neural networks has recently shown enormous potential in the field of medicine including pathology, where we and others have found common cutaneous tumors can be detected with high sensitivity and specificity. To become a widely applied tool, AI approaches will also need to reliably detect and distinguish less common tumor entities including the diverse group of cutaneous adnexal tumors.

METHODS

To assess the potential of AI to recognize cutaneous adnexal tumors, we selected a diverse set of these entities from five German centers. The algorithm was trained with samples from four centers and then tested on slides from the fifth center.

RESULTS

The neural network was able to differentiate 14 different cutaneous adnexal tumors and distinguish them from more common cutaneous tumors (i.e. basal cell carcinoma and seborrheic keratosis). The total accuracy on the test set for classifying 248 samples into these 16 diagnoses was 89.92 %. Our findings support AI can distinguish rare tumors, for morphologically distinct entities even with very limited case numbers (< 50) for training.

CONCLUSION

This study further underlines the enormous potential of AI in pathology which could become a standard tool to aid pathologists in routine diagnostics in the foreseeable future. The final diagnostic responsibility will remain with the pathologist.

Improvements in digital pathology equipment for renal biopsies: updating the standard model

INTRODUCTION

Digital pathology can improve the technical and interpretative workflows in nephropathology by creating hub-spoke networks and virtuous collaboration projects among centers in different geographical regions. New high-resolution fast-scanning instruments combined with currently existing equipment were tested in a nephropathology hub to evaluate possible upgrading in the routine processing phases.

METHODS

The scanning performance of two different instruments (Aperio vs hybrid MIDI II) was evaluated and a comparative quality control check was performed on obtained whole slide images.

RESULTS

Both with default and custom settings for light microscopy, MIDI II proved to be faster, with only slightly more time required to prepare the scan and larger final file size as compared to Aperio (p < 0.001). No differences were noted in the number of out-of-focus slides per case (p = 0.75). Regarding immunofluorescence, the new scanner required longer preparation time (p = 0.001) with comparable scanning times and final file size (p = 0.169 and p = 0.177, respectively). Quality control showed differences in 3 quality features related to white background and blurriness (p < 0.001). No major discordances in the final diagnosis were recorded after comparing the report obtained with slides scanned using the two instruments, with only one case (4%) showing minor disagreement.

CONCLUSION

The present report describes the experience of a hub nephropathology center adopting next generation digital pathology tools for the routine assessment of renal biopsies, highlighting the need for a complementary approach towards a philosophy of interoperability.

SAMPLER: unsupervised representations for rapid analysis of whole slide tissue images

BACKGROUND

Deep learning has revolutionized digital pathology, allowing automatic analysis of hematoxylin and eosin (H&E) stained whole slide images (WSIs) for diverse tasks. WSIs are broken into smaller images called tiles, and a neural network encodes each tile. Many recent works use supervised attention-based models to aggregate tile-level features into a slide-level representation, which is then used for downstream analysis. Training supervised attention-based models is computationally intensive, architecture optimization of the attention module is non-trivial, and labeled data are not always available. Therefore, we developed an unsupervised and fast approach called SAMPLER to generate slide-level representations.

METHODS

Slide-level representations of SAMPLER are generated by encoding the cumulative distribution functions of multiscale tile-level features. To assess effectiveness of SAMPLER, slide-level representations of breast carcinoma (BRCA), non-small cell lung carcinoma (NSCLC), and renal cell carcinoma (RCC) WSIs of The Cancer Genome Atlas (TCGA) were used to train separate classifiers distinguishing tumor subtypes in FFPE and frozen WSIs. In addition, BRCA and NSCLC classifiers were externally validated on frozen WSIs. Moreover, SAMPLER's attention maps identify regions of interest, which were evaluated by a pathologist. To determine time efficiency of SAMPLER, we compared runtime of SAMPLER with two attention-based models. SAMPLER concepts were used to improve the design of a context-aware multi-head attention model (context-MHA).

FINDINGS

SAMPLER-based classifiers were comparable to state-of-the-art attention deep learning models to distinguish subtypes of BRCA (AUC = 0.911 ± 0.029), NSCLC (AUC = 0.940 ± 0.018), and RCC (AUC = 0.987 ± 0.006) on FFPE WSIs (internal test sets). However, training SAMLER-based classifiers was >100 times faster. SAMPLER models successfully distinguished tumor subtypes on both internal and external test sets of frozen WSIs. Histopathological review confirmed that SAMPLER-identified high attention tiles contained subtype-specific morphological features. The improved context-MHA distinguished subtypes of BRCA and RCC (BRCA-AUC = 0.921 ± 0.027, RCC-AUC = 0.988 ± 0.010) with increased accuracy on internal test FFPE WSIs.

INTERPRETATION

Our unsupervised statistical approach is fast and effective for analyzing WSIs, with greatly improved scalability over attention-based deep learning methods. The high accuracy of SAMPLER-based classifiers and interpretable attention maps suggest that SAMPLER successfully encodes the distinct morphologies within WSIs and will be applicable to general histology image analysis problems.

FUNDING

This study was supported by the National Cancer Institute (Grant No. R01CA230031 and P30CA034196).

Complete digital pathology transition: A large multi-center experience

INTRODUCTION

Transitioning from glass slide pathology to digital pathology for primary diagnostics requires an appropriate laboratory information system, an image management system, and slide scanners; it also reinforces the need for sophisticated pathology informatics including synoptic reporting. Previous reports have discussed the transition itself and relevant considerations for it, but not the selection criteria and considerations for the infrastructure.

OBJECTIVE

To describe the process used to evaluate slide scanners, image management systems, and synoptic reporting systems for a large multisite institution.

METHODS

Six network hospitals evaluated six slide scanners, three image management systems, and three synoptic reporting systems. Scanners were evaluated based on the quality of image, speed, ease of operation, and special capabilities (including z-stacking, fluorescence and others). Image management and synoptic reporting systems were evaluated for their ease of use and capacity.

RESULTS

Among the scanners evaluated, the Leica GT450 produced the highest quality images, while the 3DHistech Pannoramic provided fluorescence and superior z-stacking. The newest generation of scanners, released relatively recently, performed better than slightly older scanners from major manufacturers Although the Olympus VS200 was not fully vetted due to not meeting all inclusion criteria, it is discussed herein due to its exceptional versatility. For Image Management Software, the authors believe that Sectra is, at the time of writing the best developed option, but this could change in the very near future as other systems improve their capabilities. All synoptic reporting systems performed impressively.

CONCLUSIONS

Specifics regarding quality and abilities of different components will change rapidly with time, but large pathology practices considering such a transition should be aware of the issues discussed and evaluate the most current generation to arrive at appropriate conclusions.

Diagnosis of interstitial lung diseases: from Averill A. Liebow to artificial intelligence

Histopathologic criteria of usual interstitial pneumonia (UIP)/idiopathic pulmonary fibrosis (IPF) were defined over the years and endorsed by leading organizations decades after Dr. Averill A. Liebow first coined the term UIP in the 1960s as a distinct pathologic pattern of fibrotic interstitial lung disease. Novel technology and recent research on interstitial lung diseases with genetic component shed light on molecular pathogenesis of UIP/IPF. Two antifibrotic agents introduced in the mid-2010s opened a new era of therapeutic approaches to UIP/IPF, albeit contentious issues regarding their efficacy, side effects, and costs. Recently, the concept of progressive pulmonary fibrosis was introduced to acknowledge additional types of progressive fibrosing interstitial lung diseases with the clinical and pathologic phenotypes comparable to those of UIP/IPF. Likewise, some authors have proposed a paradigm shift by considering UIP as a stand-alone diagnostic entity to encompass other fibrosing interstitial lung diseases that manifest a relentless progression as in IPF. These trends signal a pendulum moving toward the tendency of lumping diagnoses, which poses a risk of obscuring potentially important information crucial to both clinical and research purposes. Recent advances in whole slide imaging for digital pathology and artificial intelligence technology could offer an unprecedented opportunity to enhance histopathologic evaluation of interstitial lung diseases. However, current clinical practice trends of moving away from surgical lung biopsies in interstitial lung disease patients may become a limiting factor in this endeavor as it would be difficult to build a large histopathologic database with correlative clinical data required for artificial intelligence models.

Tumor-infiltrating T lymphocytes evaluated using digital image analysis predict the prognosis of patients with diffuse large B-cell lymphoma

BACKGROUND

The implication of the presence of tumor-infiltrating T lymphocytes (TIL-T) in diffuse large B-cell lymphoma (DLBCL) is yet to be elucidated. We aimed to investigate the effect of TIL-T levels on the prognosis of patients with DLBCL.

METHODS

Ninety-six patients with DLBCL were enrolled in the study. The TIL-T ratio was measured using QuPath, a digital pathology software package. The TIL-T ratio was investigated in three foci (highest, intermediate, and lowest) for each case, resulting in TIL-T-Max, TIL-T-Intermediate, and TIL-T-Min. The relationship between the TIL-T ratios and prognosis was investigated.

RESULTS

When 19% was used as the cutoff value for TIL-T-Max, 72 (75.0%) and 24 (25.0%) patients had high and low TIL-T-Max, respectively. A high TIL-T-Max was significantly associated with lower serum lactate dehydrogenase levels (p < .001), with patient group who achieved complete remission after RCHOP therapy (p < .001), and a low-risk revised International Prognostic Index score (p < .001). Univariate analysis showed that patients with a low TIL-T-Max had a significantly worse prognosis in overall survival compared to those with a high TIL-T-Max (p < .001); this difference remained significant in a multivariate analysis with Cox proportional hazards (hazard ratio, 7.55; 95% confidence interval, 2.54 to 22.42; p < .001).

CONCLUSIONS

Patients with DLBCL with a high TIL-T-Max showed significantly better prognosis than those with a low TIL-T-Max, and the TIL-T-Max was an independent indicator of overall survival. These results suggest that evaluating TIL-T ratios using a digital pathology system is useful in predicting the prognosis of patients with DLBCL.

Biomedical laboratory scientists and technicians in digital pathology - Is there a need for professional development?

Objective

Digital pathology (DP) is moving into Danish pathology departments at high pace. Conventionally, biomedical laboratory scientists (BLS) and technicians have prepared tissue sections for light microscopy, but workflow alterations are required for the new digital era with whole slide imaging (WSI); digitally assisted image analysis (DAIA) and artificial intelligence (AI). We aim to explore the role of BLS in DP and assess a potential need for professional development.

Methods

We investigated the roles of BLS in the new digital era through qualitative interviews at Danish Pathology Departments in 2019/2020 before DP implementation (supported by a questionnaire); and in 2022 after DP implementation. Additionally, senior lecturers from three Danish University Colleges reported on how DP was integrated into the 2023 bachelor's degree educational curricula for BLS students.

Results

At some Danish pathology departments, BLS were involved in the implementation process of DP and their greatest concerns were lack of physical laboratory requirements (69%) and implementation strategies (63%). BLS were generally positive towards working with DP, however, some expressed concern about extended working hours for scanning. Work-task transfers from pathologists were generally greeted positively from both management and pathologists; however, at follow-up interviews after DP implementation, job transfers had not been effectuated. At Danish university colleges, DP had been integrated systematically in the curricula for BLS students, especially WSI.

Conclusion

Involving BLS in DP implementation and development may benefit the process, as BLS have a hands-on workflow perspective with a focus on quality assurance. Several new work opportunities for BLS may occur with DP including WSI, DAIA and AI, and therefore new qualifications are warranted, which must be considered in future undergraduate programmes for BLS students or postgraduate programmes for BLS.

Computational Image and Molecular Analysis Reveal Unique Prognostic Features of Immune Architecture in African Versus European American Women with Endometrial Cancer

Endometrial cancer (EC) disproportionately affects African American (AA) women in terms of progression and death. In our study, we sought to employ computerized image and bioinformatic analysis to tease out morphologic and molecular differences in EC between AA and European-American (EA) populations. We identified the differences in immune cell spatial patterns between AA and EA populations with markers of tumor biology, including histologic and molecular subtypes. The models performed best when they were trained and validated using data from the same population. Unsupervised clustering revealed a distinct association between immune cell features and known molecular subtypes of endometrial cancer that varied between AA and EA populations. Our genomic analysis revealed two distinct and novel gene sets with mutations associated with improved prognosis in AA and EA patients. Our study findings suggest the need for population-specific risk prediction models for women with endometrial cancer.