The Effect of Quality Control on Accuracy of Digital Pathology Image Analysis

An international study of stain variability in histopathology using qualitative and quantitative analysis

Hematoxylin and eosin (H&E) staining accounts for over 80% of slides stained worldwide. Although routinely used, there are high levels of variation between labs due to different staining methods. Staining is a pivotal part of slide preparation, but quality control is largely subjective, with overall clinical assurance provided by external quality assessment (EQA) services, underpinned by expert assessment. Digital pathology offers the potential to provide objective quantification of stain, through color analysis, to augment EQA assessment. This large-scale study evaluated H&E staining in 247 international labs participating in the UK NEQAS CPT EQA programme. Tissue sections were circulated to each lab to stain using their routine H&E staining protocol. The slides were reviewed by independent expert UK NEQAS CPT assessors, and quantitative digital analysis was conducted, comprising of H&E color deconvolution and color difference determination (ΔE). Most labs (69%) achieved an EQA score indicating good or excellent staining, with high inter-observer concordance to support this (92.5% within one mark of each other). H&E color difference, ΔE, showed 60% of labs were within 2 ΔE of the mean, which is considered as only perceptible through close observation. There was little correlation found between H&E intensity and assessor score, however, the H&E intensity ratio indicated a trend with assessor score suggesting there may be an optimal stain relationship that should be investigated further. The presented hybrid analysis combines expert analysis with objective data. This has the potential to inform upon optimal tissue staining and allows us to consider quantitative standards of H&E staining in pathology practice.

Generalizable deep neural networks for image quality classification of cervical images

Successful translation of artificial intelligence (AI) models into clinical practice, across clinical domains, is frequently hindered by the lack of image quality control. Diagnostic models are often trained on images with no denotation of image quality in the training data; this, in turn, can lead to misclassifications by these models when implemented in the clinical setting. In the case of cervical images, quality classification is a crucial task to ensure accurate detection of precancerous lesions or cancer; this is true for both gynecologic-oncologists' (manual) and diagnostic AI models' (automated) predictions. Factors that impact the quality of a cervical image include but are not limited to blur, poor focus, poor light, noise, obscured view of the cervix due to mucus and/or blood, improper position, and over- and/or under-exposure. Utilizing a multi-level image quality ground truth denoted by providers, we generated an image quality classifier following a multi-stage model selection process that investigated several key design choices on a multi-heterogenous "SEED" dataset of 40,534 images. We subsequently validated the best model on an external dataset ("EXT"), comprising 1,340 images captured using a different device and acquired in different geographies from "SEED". We assessed the relative impact of various axes of data heterogeneity, including device, geography, and ground-truth rater on model performance. Our best performing model achieved an area under the receiver operating characteristics curve (AUROC) of 0.92 (low quality, LQ vs. rest) and 0.93 (high quality, HQ vs. rest), and a minimal total %extreme misclassification (%EM) of 2.8% on the internal validation set. Our model also generalized well externally, achieving corresponding AUROCs of 0.83 and 0.82, and %EM of 3.9% when tested out-of-the-box on the external validation ("EXT") set. Additionally, our model was geography agnostic with no meaningful difference in performance across geographies, did not exhibit catastrophic forgetting upon retraining with new data, and mimicked the overall/average ground truth rater behavior well. Our work represents one of the first efforts at generating and externally validating an image quality classifier across multiple axes of data heterogeneity to aid in visual diagnosis of cervical precancer and cancer. We hope that this will motivate the accompaniment of adequate guardrails for AI-based pipelines to account for image quality and generalizability concerns.

Exploring Multiplex Immunohistochemistry (mIHC) Techniques and Histopathology Image Analysis: Current Practice and Potential for Clinical Incorporation

BACKGROUND

By simultaneously staining multiple immunomarkers on a single tissue section, multiplexed immunohistochemistry (mIHC) enhances the amount of information that can be observed in a single tissue section and thus can be a powerful tool to visualise cellular interactions directly in the tumour microenvironment. Performing mIHC remains technically and practically challenging, and this technique has many limitations if not properly validated. However, with proper validation, heterogeneity between histopathological images can be avoided.

AIMS

This review aimed to summarize the currently used methods and to propose a standardised method for effective mIHC.

MATERIALS AND METHODS

An extensive literature review was conducted to identify different methods currently in use for mIHC.

RESULTS

Guidelines for antibody selection, panel design, antibody validation and analytical strategies are given. The advantages and disadvantages of each method are discussed.

CONCLUSION

This review summarizes widely used pathology imaging software and discusses the potential for automation of pathology image analysis so that mIHC technology can be a truly powerful tool for research as well as clinical use.

Object-based feedback attention in convolutional neural networks improves tumour detection in digital pathology

Human visual attention allows prior knowledge or expectations to influence visual processing, allocating limited computational resources to only that part of the image that are likely to behaviourally important. Here, we present an image recognition system based on biological vision that guides attention to more informative locations within a larger parent image, using a sequence of saccade-like motions. We demonstrate that at the end of the saccade sequence the system has an improved classification ability compared to the convolutional neural network (CNN) that represents the feedforward part of the model. Feedback activations highlight salient image features supporting the explainability of the classification. Our attention model deviates substantially from more common feedforward attention mechanisms, which linearly reweight part of the input. This model uses several passes of feedforward and backward activation, which interact non-linearly. We apply our feedback architecture to histopathology patch images, demonstrating a 3.5% improvement in accuracy (p < 0.001) when retrospectively processing 59,057 9-class patches from 689 colorectal cancer WSIs. In the saccade implementation, overall agreement between expert-labelled patches and model prediction reached 93.23% for tumour tissue, surpassing inter-pathologist agreement. Our method is adaptable to other areas of science which rely on the analysis of extremely large-scale images.

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Image Quality Assessment and Reliability Analysis of Artificial Intelligence-Based Tumor Classification of Stimulated Raman Histology of Tumor Biobank Samples

BACKGROUND

Stimulated Raman histology (SRH) is a label-free optical imaging method for rapid intraoperative analysis of fresh tissue samples. Analysis of SRH images using Convolutional Neural Networks (CNN) has shown promising results for predicting the main histopathological classes of neurooncological tumors. Due to the relatively low number of rare tumor representations in CNN training datasets, a valid prediction of rarer entities remains limited. To develop new reliable analysis tools, larger datasets and greater tumor variety are crucial. One way to accomplish this is through research biobanks storing frozen tumor tissue samples. However, there is currently no data available regarding the pertinency of previously frozen tissue samples for SRH analysis. The aim of this study was to assess image quality and perform a comparative reliability analysis of artificial intelligence-based tumor classification using SRH in fresh and frozen tissue samples.

METHODS

In a monocentric prospective study, tissue samples from 25 patients undergoing brain tumor resection were obtained. SRH was acquired in fresh and defrosted samples of the same specimen after varying storage durations at -80 °C. Image quality was rated by an experienced neuropathologist, and prediction of histopathological diagnosis was performed using two established CNNs.

RESULTS

The image quality of SRH in fresh and defrosted tissue samples was high, with a mean image quality score of 1.96 (range 1-5) for both groups. CNN analysis showed high internal consistency for histo-(Cα 0.95) and molecular (Cα 0.83) pathological tumor classification. The results were confirmed using a dataset with samples from the local tumor biobank (Cα 0.91 and 0.53).

CONCLUSIONS

Our results showed that SRH appears comparably reliable in fresh and frozen tissue samples, enabling the integration of tumor biobank specimens to potentially improve the diagnostic range and reliability of CNN prediction tools.

Public evidence on AI products for digital pathology

Novel products applying artificial intelligence (AI)-based methods to digital pathology images are touted to have many uses and benefits. However, publicly available information for products can be variable, with few sources of independent evidence. This review aimed to identify public evidence for AI-based products for digital pathology. Key features of products on the European Economic Area/Great Britain (EEA/GB) markets were examined, including their regulatory approval, intended use, and published validation studies. There were 26 AI-based products that met the inclusion criteria and, of these, 24 had received regulatory approval via the self-certification route as General in vitro diagnostic (IVD) medical devices. Only 10 of the products (38%) had peer-reviewed internal validation studies and 11 products (42%) had peer-reviewed external validation studies. To support transparency an online register was developed using identified public evidence ( https://osf.io/gb84r/ ), which we anticipate will provide an accessible resource on novel devices and support decision making.

Equipping computational pathology systems with artifact processing pipelines: a showcase for computation and performance trade-offs

BACKGROUND

Histopathology is a gold standard for cancer diagnosis. It involves extracting tissue specimens from suspicious areas to prepare a glass slide for a microscopic examination. However, histological tissue processing procedures result in the introduction of artifacts, which are ultimately transferred to the digitized version of glass slides, known as whole slide images (WSIs). Artifacts are diagnostically irrelevant areas and may result in wrong predictions from deep learning (DL) algorithms. Therefore, detecting and excluding artifacts in the computational pathology (CPATH) system is essential for reliable automated diagnosis.

METHODS

In this paper, we propose a mixture of experts (MoE) scheme for detecting five notable artifacts, including damaged tissue, blur, folded tissue, air bubbles, and histologically irrelevant blood from WSIs. First, we train independent binary DL models as experts to capture particular artifact morphology. Then, we ensemble their predictions using a fusion mechanism. We apply probabilistic thresholding over the final probability distribution to improve the sensitivity of the MoE. We developed four DL pipelines to evaluate computational and performance trade-offs. These include two MoEs and two multiclass models of state-of-the-art deep convolutional neural networks (DCNNs) and vision transformers (ViTs). These DL pipelines are quantitatively and qualitatively evaluated on external and out-of-distribution (OoD) data to assess generalizability and robustness for artifact detection application.

RESULTS

We extensively evaluated the proposed MoE and multiclass models. DCNNs-based MoE and ViTs-based MoE schemes outperformed simpler multiclass models and were tested on datasets from different hospitals and cancer types, where MoE using (MobileNet) DCNNs yielded the best results. The proposed MoE yields 86.15 % F1 and 97.93% sensitivity scores on unseen data, retaining less computational cost for inference than MoE using ViTs. This best performance of MoEs comes with relatively higher computational trade-offs than multiclass models. Furthermore, we apply post-processing to create an artifact segmentation mask, a potential artifact-free RoI map, a quality report, and an artifact-refined WSI for further computational analysis. During the qualitative evaluation, field experts assessed the predictive performance of MoEs over OoD WSIs. They rated artifact detection and artifact-free area preservation, where the highest agreement translated to a Cohen Kappa of 0.82, indicating substantial agreement for the overall diagnostic usability of the DCNN-based MoE scheme.

CONCLUSIONS

The proposed artifact detection pipeline will not only ensure reliable CPATH predictions but may also provide quality control. In this work, the best-performing pipeline for artifact detection is MoE with DCNNs. Our detailed experiments show that there is always a trade-off between performance and computational complexity, and no straightforward DL solution equally suits all types of data and applications. The code and HistoArtifacts dataset can be found online at Github and Zenodo , respectively.

Development of a multi-scanner facility for data acquisition for digital pathology artificial intelligence

Whole slide imaging (WSI) of pathology glass slides using high-resolution scanners has enabled the large-scale application of artificial intelligence (AI) in pathology, to support the detection and diagnosis of disease, potentially increasing efficiency and accuracy in tissue diagnosis. Despite the promise of AI, it has limitations. 'Brittleness' or sensitivity to variation in inputs necessitates that large amounts of data are used for training. AI is often trained on data from different scanners but not usually by replicating the same slide across scanners. The utilisation of multiple WSI instruments to produce digital replicas of the same slides will make more comprehensive datasets and may improve the robustness and generalisability of AI algorithms as well as reduce the overall data requirements of AI training. To this end, the National Pathology Imaging Cooperative (NPIC) has built the AI FORGE (Facilitating Opportunities for Robust Generalisable data Emulation), a unique multi-scanner facility embedded in a clinical site in the NHS to (1) compare scanner performance, (2) replicate digital pathology image datasets across WSI systems, and (3) support the evaluation of clinical AI algorithms. The NPIC AI FORGE currently comprises 15 scanners from nine manufacturers. It can generate approximately 4,000 WSI images per day (approximately 7 TB of image data). This paper describes the process followed to plan and build such a facility. © 2024 The Author(s). The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Advancements in computer vision and pathology: Unraveling the potential of artificial intelligence for precision diagnosis and beyond

The integration of computer vision into pathology through slide digitalization represents a transformative leap in the field's evolution. Traditional pathology methods, while reliable, are often time-consuming and susceptible to intra- and interobserver variability. In contrast, computer vision, empowered by artificial intelligence (AI) and machine learning (ML), promises revolutionary changes, offering consistent, reproducible, and objective results with ever-increasing speed and scalability. The applications of advanced algorithms and deep learning architectures like CNNs and U-Nets augment pathologists' diagnostic capabilities, opening new frontiers in automated image analysis. As these technologies mature and integrate into digital pathology workflows, they are poised to provide deeper insights into disease processes, quantify and standardize biomarkers, enhance patient outcomes, and automate routine tasks, reducing pathologists' workload. However, this transformative force calls for cross-disciplinary collaboration between pathologists, computer scientists, and industry innovators to drive research and development. While acknowledging its potential, this chapter addresses the limitations of AI in pathology, encompassing technical, practical, and ethical considerations during development and implementation.

Artificial Intelligence-Based Quality Assessment of Histopathology Whole-Slide Images within a Clinical Workflow: Assessment of 'PathProfiler' in a Diagnostic Pathology Setting

Digital pathology continues to gain momentum, with the promise of artificial intelligence to aid diagnosis and for assessment of features which may impact prognosis and clinical management. Successful adoption of these technologies depends upon the quality of digitised whole-slide images (WSI); however, current quality control largely depends upon manual assessment, which is inefficient and subjective. We previously developed PathProfiler, an automated image quality assessment tool, and in this feasibility study we investigate its potential for incorporation into a diagnostic clinical pathology setting in real-time. A total of 1254 genitourinary WSI were analysed by PathProfiler. PathProfiler was developed and trained on prostate tissue and, of the prostate biopsy WSI, representing 46% of the WSI analysed, 4.5% were flagged as potentially being of suboptimal quality for diagnosis. All had concordant subjective issues, mainly focus-related, 54% severe enough to warrant remedial action which resulted in improved image quality. PathProfiler was less reliable in assessment of non-prostate surgical resection-type cases, on which it had not been trained. PathProfiler shows potential for incorporation into a digitised clinical pathology workflow, with opportunity for image quality improvement. Whilst its reliability in the current form appears greatest for assessment of prostate specimens, other specimen types, particularly biopsies, also showed benefit.

Tissue Contamination Challenges the Credibility of Machine Learning Models in Real World Digital Pathology

Machine learning (ML) models are poised to transform surgical pathology practice. The most successful use attention mechanisms to examine whole slides, identify which areas of tissue are diagnostic, and use them to guide diagnosis. Tissue contaminants, such as floaters, represent unexpected tissue. Although human pathologists are extensively trained to consider and detect tissue contaminants, we examined their impact on ML models. We trained 4 whole-slide models. Three operate in placenta for the following functions: (1) detection of decidual arteriopathy, (2) estimation of gestational age, and (3) classification of macroscopic placental lesions. We also developed a model to detect prostate cancer in needle biopsies. We designed experiments wherein patches of contaminant tissue are randomly sampled from known slides and digitally added to patient slides and measured model performance. We measured the proportion of attention given to contaminants and examined the impact of contaminants in the t-distributed stochastic neighbor embedding feature space. Every model showed performance degradation in response to one or more tissue contaminants. Decidual arteriopathy detection--balanced accuracy decreased from 0.74 to 0.69 ± 0.01 with addition of 1 patch of prostate tissue for every 100 patches of placenta (1% contaminant). Bladder, added at 10% contaminant, raised the mean absolute error in estimating gestational age from 1.626 weeks to 2.371 ± 0.003 weeks. Blood, incorporated into placental sections, induced false-negative diagnoses of intervillous thrombi. Addition of bladder to prostate cancer needle biopsies induced false positives, a selection of high-attention patches, representing 0.033 mm2, and resulted in a 97% false-positive rate when added to needle biopsies. Contaminant patches received attention at or above the rate of the average patch of patient tissue. Tissue contaminants induce errors in modern ML models. The high level of attention given to contaminants indicates a failure to encode biological phenomena. Practitioners should move to quantify and ameliorate this problem.

Are you sure it's an artifact? Artifact detection and uncertainty quantification in histological images

Modern cancer diagnostics involves extracting tissue specimens from suspicious areas and conducting histotechnical procedures to prepare a digitized glass slide, called Whole Slide Image (WSI), for further examination. These procedures frequently introduce different types of artifacts in the obtained WSI, and histological artifacts might influence Computational Pathology (CPATH) systems further down to a diagnostic pipeline if not excluded or handled. Deep Convolutional Neural Networks (DCNNs) have achieved promising results for the detection of some WSI artifacts, however, they do not incorporate uncertainty in their predictions. This paper proposes an uncertainty-aware Deep Kernel Learning (DKL) model to detect blurry areas and folded tissues, two types of artifacts that can appear in WSIs. The proposed probabilistic model combines a CNN feature extractor and a sparse Gaussian Processes (GPs) classifier, which improves the performance of current state-of-the-art artifact detection DCNNs and provides uncertainty estimates. We achieved 0.996 and 0.938 F1 scores for blur and folded tissue detection on unseen data, respectively. In extensive experiments, we validated the DKL model on unseen data from external independent cohorts with different staining and tissue types, where it outperformed DCNNs. Interestingly, the DKL model is more confident in the correct predictions and less in the wrong ones. The proposed DKL model can be integrated into the preprocessing pipeline of CPATH systems to provide reliable predictions and possibly serve as a quality control tool.

Quantitative assessment of H&E staining for pathology: development and clinical evaluation of a novel system

BACKGROUND

Staining tissue samples to visualise cellular detail and tissue structure is at the core of pathology diagnosis, but variations in staining can result in significantly different appearances of the tissue sample. While the human visual system is adept at compensating for stain variation, with the growth of digital imaging in pathology, the impact of this variation can be more profound. Despite the ubiquity of haematoxylin and eosin staining in clinical practice worldwide, objective quantification is not yet available. We propose a method for quantitative haematoxylin and eosin stain assessment to facilitate quality assurance of histopathology staining, enabling truly quantitative quality control and improved standardisation.

METHODS

The stain quantification method comprises conventional microscope slides with a stain-responsive biopolymer film affixed to one side, called stain assessment slides. The stain assessment slides were characterised with haematoxylin and eosin, and implemented in one clinical laboratory to quantify variation levels.

RESULTS

Stain assessment slide stain uptake increased linearly with duration of haematoxylin and eosin staining (r = 0.99), and demonstrated linearly comparable staining to samples of human liver tissue (r values 0.98-0.99). Laboratory implementation of this technique quantified intra- and inter-instrument variation of staining instruments at one point in time and across a five-day period.

CONCLUSION

The proposed method has been shown to reliably quantify stain uptake, providing an effective laboratory quality control method for stain variation. This is especially important for whole slide imaging and the future development of artificial intelligence in digital pathology.

Digitization of Pathology Labs: A Review of Lessons Learned

Pathology laboratories are increasingly using digital workflows. This has the potential of increasing laboratory efficiency, but the digitization process also involves major challenges. Several reports have been published describing the individual experiences of specific laboratories with the digitization process. However, a comprehensive overview of the lessons learned is still lacking. We provide an overview of the lessons learned for different aspects of the digitization process, including digital case management, digital slide reading, and computer-aided slide reading. We also cover metrics used for monitoring performance and pitfalls and corresponding values observed in practice. The overview is intended to help pathologists, information technology decision makers, and administrators to benefit from the experiences of others and to implement the digitization process in an optimal way to make their own laboratory future-proof.

Tissue contamination challenges the credibility of machine learning models in real world digital pathology

Machine learning (ML) models are poised to transform surgical pathology practice. The most successful use attention mechanisms to examine whole slides, identify which areas of tissue are diagnostic, and use them to guide diagnosis. Tissue contaminants, such as floaters, represent unexpected tissue. While human pathologists are extensively trained to consider and detect tissue contaminants, we examined their impact on ML models. We trained 4 whole slide models. Three operate in placenta for 1) detection of decidual arteriopathy (DA), 2) estimation of gestational age (GA), and 3) classification of macroscopic placental lesions. We also developed a model to detect prostate cancer in needle biopsies. We designed experiments wherein patches of contaminant tissue are randomly sampled from known slides and digitally added to patient slides and measured model performance. We measured the proportion of attention given to contaminants and examined the impact of contaminants in T-distributed Stochastic Neighbor Embedding (tSNE) feature space. Every model showed performance degradation in response to one or more tissue contaminants. DA detection balanced accuracy decreased from 0.74 to 0.69 +/- 0.01 with addition of 1 patch of prostate tissue for every 100 patches of placenta (1% contaminant). Bladder, added at 10% contaminant raised the mean absolute error in estimating gestation age from 1.626 weeks to 2.371 +/ 0.003 weeks. Blood, incorporated into placental sections, induced false negative diagnoses of intervillous thrombi. Addition of bladder to prostate cancer needle biopsies induced false positives, a selection of high-attention patches, representing 0.033mm2, resulted in a 97% false positive rate when added to needle biopsies. Contaminant patches received attention at or above the rate of the average patch of patient tissue. Tissue contaminants induce errors in modern ML models. The high level of attention given to contaminants indicates a failure to encode biological phenomena. Practitioners should move to quantify and ameliorate this problem.

Artificial intelligence and machine learning overview in pathology & laboratory medicine: A general review of data preprocessing and basic supervised concepts

Machine learning (ML) is becoming an integral aspect of several domains in medicine. Yet, most pathologists and laboratory professionals remain unfamiliar with such tools and are unprepared for their inevitable integration. To bridge this knowledge gap, we present an overview of key elements within this emerging data science discipline. First, we will cover general, well-established concepts within ML, such as data type concepts, data preprocessing methods, and ML study design. We will describe common supervised and unsupervised learning algorithms and their associated common machine learning terms (provided within a comprehensive glossary of terms that are discussed within this review). Overall, this review will offer a broad overview of the key concepts and algorithms in machine learning, with a focus on pathology and laboratory medicine. The objective is to provide an updated useful reference for those new to this field or those who require a refresher.

Multi-site cross-organ calibrated deep learning (MuSClD): Automated diagnosis of non-melanoma skin cancer

Although deep learning (DL) has demonstrated impressive diagnostic performance for a variety of computational pathology tasks, this performance often markedly deteriorates on whole slide images (WSI) generated at external test sites. This phenomenon is due in part to domain shift, wherein differences in test-site pre-analytical variables (e.g., slide scanner, staining procedure) result in WSI with notably different visual presentations compared to training data. To ameliorate pre-analytic variances, approaches such as CycleGAN can be used to calibrate visual properties of images between sites, with the intent of improving DL classifier generalizability. In this work, we present a new approach termed Multi-Site Cross-Organ Calibration based Deep Learning (MuSClD) that employs WSIs of an off-target organ for calibration created at the same site as the on-target organ, based off the assumption that cross-organ slides are subjected to a common set of pre-analytical sources of variance. We demonstrate that by using an off-target organ from the test site to calibrate training data, the domain shift between training and testing data can be mitigated. Importantly, this strategy uniquely guards against potential data leakage introduced during calibration, wherein information only available in the testing data is imparted on the training data. We evaluate MuSClD in the context of the automated diagnosis of non-melanoma skin cancer (NMSC). Specifically, we evaluated MuSClD for identifying and distinguishing (a) basal cell carcinoma (BCC), (b) in-situ squamous cell carcinomas (SCC-In Situ), and (c) invasive squamous cell carcinomas (SCC-Invasive), using an Australian (training, n = 85) and a Swiss (held-out testing, n = 352) cohort. Our experiments reveal that MuSCID reduces the Wasserstein distances between sites in terms of color, contrast, and brightness metrics, without imparting noticeable artifacts to training data. The NMSC-subtyping performance is statistically improved as a result of MuSCID in terms of one-vs. rest AUC: BCC (0.92 vs 0.87, p = 0.01), SCC-In Situ (0.87 vs 0.73, p = 0.15) and SCC-Invasive (0.92 vs 0.82, p = 1e-5). Compared to baseline NMSC-subtyping with no calibration, the internal validation results of MuSClD (BCC (0.98), SCC-In Situ (0.92), and SCC-Invasive (0.97)) suggest that while domain shift indeed degrades classification performance, our on-target calibration using off-target tissue can safely compensate for pre-analytical variabilities, while improving the robustness of the model.

Efficient quality control of whole slide pathology images with human-in-the-loop training

Histopathology whole slide images (WSIs) are being widely used to develop deep learning-based diagnostic solutions, especially for precision oncology. Most of these diagnostic softwares are vulnerable to biases and impurities in the training and test data which can lead to inaccurate diagnoses. For instance, WSIs contain multiple types of tissue regions, at least some of which might not be relevant to the diagnosis. We introduce HistoROI, a robust yet lightweight deep learning-based classifier to segregate WSI into 6 broad tissue regions-epithelium, stroma, lymphocytes, adipose, artifacts, and miscellaneous. HistoROI is trained using a novel human in-the-loop and active learning paradigm that ensures variations in training data for labeling efficient generalization. HistoROI consistently performs well across multiple organs, despite being trained on only a single dataset, demonstrating strong generalization. Further, we have examined the utility of HistoROI in improving the performance of downstream deep learning-based tasks using the CAMELYON breast cancer lymph node and TCGA lung cancer datasets. For the former dataset, the area under the receiver operating characteristic curve (AUC) for metastasis versus normal tissue of a neural network trained using weakly supervised learning increased from 0.88 to 0.92 by filtering the data using HistoROI. Similarly, the AUC increased from 0.88 to 0.93 for the classification between adenocarcinoma and squamous cell carcinoma on the lung cancer dataset. We also found that the performance of the HistoROI improves upon HistoQC for artifact detection on a test dataset of 93 annotated WSIs. The limitations of the proposed model are analyzed, and potential extensions are also discussed.

Stain normalization in digital pathology: Clinical multi-center evaluation of image quality

In digital pathology, the final appearance of digitized images is affected by several factors, resulting in stain color and intensity variation. Stain normalization is an innovative solution to overcome stain variability. However, the validation of color normalization tools has been assessed only from a quantitative perspective, through the computation of similarity metrics between the original and normalized images. To the best of our knowledge, no works investigate the impact of normalization on the pathologist's evaluation. The objective of this paper is to propose a multi-tissue (i.e., breast, colon, liver, lung, and prostate) and multi-center qualitative analysis of a stain normalization tool with the involvement of pathologists with different years of experience. Two qualitative studies were carried out for this purpose: (i) a first study focused on the analysis of the perceived image quality and absence of significant image artifacts after the normalization process; (ii) a second study focused on the clinical score of the normalized image with respect to the original one. The results of the first study prove the high quality of the normalized image with a low impact artifact generation, while the second study demonstrates the superiority of the normalized image with respect to the original one in clinical practice. The normalization process can help both to reduce variability due to tissue staining procedures and facilitate the pathologist in the histological examination. The experimental results obtained in this work are encouraging and can justify the use of a stain normalization tool in clinical routine.

Attention-guided sampling for colorectal cancer analysis with digital pathology

Improvements to patient care through the development of automated image analysis in pathology are restricted by the small image patch size that can be processed by convolutional neural networks (CNNs), when compared to the whole-slide image (WSI). Tile-by-tile processing across the entire WSI is slow and inefficient. While this may improve with future computing power, the technique remains vulnerable to noise from uninformative image areas.

We propose a novel attention-inspired algorithm that selects image patches from informative parts of the WSI, first using a sparse randomised grid pattern, then iteratively re-sampling at higher density in regions where a CNN classifies patches as tumour. Subsequent uniform sampling across the enclosing region of interest (ROI) is used to mitigate sampling bias. Benchmarking tests informed the adoption of VGG19 as the main CNN architecture, with 79% classification accuracy. A further CNN was trained to separate false-positive normal epithelium from tumour epithelium, in a novel adaptation of a two-stage model used in brain imaging.

These subsystems were combined in a processing pipeline to generate spatial distributions of classified patches from unseen WSIs. The ROI was predicted with a mean F1 (Dice) score of 86.6% over 100 evaluation WSIs. Several algorithms for evaluating tumour–stroma ratio (TSR) within the ROI were compared, giving a lowest root mean square (RMS) error of 11.3% relative to pathologists’ annotations, against 13.5% for an equivalent tile-by-tile pipeline. Our pipeline processed WSIs between 3.3x and 6.3x faster than tile-by-tile processing.

We propose our attention-based sampling pipeline as a useful tool for pathology researchers, with the further potential for incorporating additional diagnostic calculations.

Semantic annotation for computational pathology: multidisciplinary experience and best practice recommendations

Recent advances in whole-slide imaging (WSI) technology have led to the development of a myriad of computer vision and artificial intelligence-based diagnostic, prognostic, and predictive algorithms. Computational Pathology (CPath) offers an integrated solution to utilise information embedded in pathology WSIs beyond what can be obtained through visual assessment. For automated analysis of WSIs and validation of machine learning (ML) models, annotations at the slide, tissue, and cellular levels are required. The annotation of important visual constructs in pathology images is an important component of CPath projects. Improper annotations can result in algorithms that are hard to interpret and can potentially produce inaccurate and inconsistent results. Despite the crucial role of annotations in CPath projects, there are no well-defined guidelines or best practices on how annotations should be carried out. In this paper, we address this shortcoming by presenting the experience and best practices acquired during the execution of a large-scale annotation exercise involving a multidisciplinary team of pathologists, ML experts, and researchers as part of the Pathology image data Lake for Analytics, Knowledge and Education (PathLAKE) consortium. We present a real-world case study along with examples of different types of annotations, diagnostic algorithm, annotation data dictionary, and annotation constructs. The analyses reported in this work highlight best practice recommendations that can be used as annotation guidelines over the lifecycle of a CPath project.

Potential quality pitfalls of digitalized whole slide image of breast pathology in routine practice

Using digitalized whole slide images (WSI) in routine histopathology practice is a revolutionary technology. This study aims to assess the clinical impacts of WSI quality and representation of the corresponding glass slides. 40,160 breast WSIs were examined and compared with their corresponding glass slides. The presence, frequency, location, tissue type, and the clinical impacts of missing tissue were assessed. Scanning time, type of the specimens, time to WSIs implementation, and quality control (QC) measures were also considered. The frequency of missing tissue ranged from 2% to 19%. The area size of the missed tissue ranged from 1-70%. In most cases (>75%), the missing tissue area size was <10% and peripherally located. In all cases the missed tissue was fat with or without small entrapped normal breast parenchyma. No missing tissue was identified in WSIs of the core biopsy specimens. QC measures improved images quality and reduced WSI failure rates by seven-fold. A negative linear correlation between the frequency of missing tissue and both the scanning time and the image file size was observed (p < 0.05). None of the WSI with missing tissues resulted in a change in the final diagnosis. Missing tissue on breast WSI is observed but with variable frequency and little diagnostic consequence. Balancing between WSI quality and scanning time/image file size should be considered and pathology laboratories should undertake their own assessments of risk and provide the relevant mitigations with the appropriate level of caution.

Types and frequency of whole slide imaging scan failures in a clinical high throughput digital pathology scanning laboratory

Digital workflow transformation continues to sweep throughout a diversity of pathology departments spanning the globe following catalyzation of whole slide imaging (WSI) adoption by the SARS-CoV-2 (COVID-19) pandemic. The utility of WSI for a litany of use cases including primary diagnosis has been emphasized during this period, with WSI scanning devices gaining the approval of healthcare regulatory bodies and practitioners alike for clinical applications following extensive validatory efforts. As successful validation for WSI is predicated upon pathologist diagnostic interpretability of digital images with high glass slide concordance, departmental adoption of WSI is tantamount to the reliability of such images often predicated upon quality assessment notwithstanding image interpretability but extending to quality of practice following WSI adoption. Metrics of importance within this context include failure rates inclusive of different scanning errors that result in poor image quality and the potential such errors may incur upon departmental turnaround time (TAT). We sought to evaluate the impact of WSI implementation through retrospective evaluation of scan failure frequency in archival versus newly prepared slides, types of scanning error, and impact upon TAT following commencement of live WSI operation in May 2017 until the present period within a fully digitized high-volume academic institution. A 1.19% scan failure incidence rate was recorded during this period, with re-scanning requested and successfully executed for 1.19% of cases during the reported period of January 2019 until present. No significant impact upon TAT was deduced, suggesting an outcome which may be encouraging for departments considering digital workflow adoption.

A Transfer Learning Architecture Based on a Support Vector Machine for Histopathology Image Classification

Recently, digital pathology is an essential application for clinical practice and medical research. Due to the lack of large annotated datasets, the deep transfer learning technique is often used to classify histopathology images. A softmax classifier is often used to perform classification tasks. Besides, a Support Vector Machine (SVM) classifier is also popularly employed, especially for binary classification problems. Accurately determining the category of the histopathology images is vital for the diagnosis of diseases. In this paper, the conventional softmax classifier and the SVM classifier-based transfer learning approach are evaluated to classify histopathology cancer images in a binary breast cancer dataset and a multiclass lung and colon cancer dataset. In order to achieve better classification accuracy, a methodology that attaches SVM classifier to the fully-connected (FC) layer of the softmax-based transfer learning model is proposed. The proposed architecture involves a first step training the newly added FC layer on the target dataset using the softmax-based model and a second step training the SVM classifier with the newly trained FC layer. Cross-validation is used to ensure no bias for the evaluation of the performance of the models. Experimental results reveal that the conventional SVM classifier-based model is the least accurate on either binary or multiclass cancer datasets. The conventional softmax-based model shows moderate classification accuracy, while the proposed synthetic architecture achieves the best classification accuracy.