Improving unsupervised stain-to-stain translation using self-supervision and meta-learning

Deep learning applications for kidney histology analysis

PURPOSE OF REVIEW

Nephropathology is increasingly incorporating computational methods to enhance research and diagnostic accuracy. The widespread adoption of digital pathology, coupled with advancements in deep learning, will likely transform our pathology practices. Here, we discuss basic concepts of deep learning, recent applications in nephropathology, current challenges in implementation and future perspectives.

RECENT FINDINGS

Deep learning models have been developed and tested in various areas of nephropathology, for example, predicting kidney disease progression or diagnosing diseases based on imaging and clinical data. Despite their promising potential, challenges remain that hinder a wider adoption, for example, the lack of prospective evidence and testing in real-world scenarios.

SUMMARY

Deep learning offers great opportunities to improve quantitative and qualitative kidney histology analysis for research and clinical nephropathology diagnostics. Although exciting approaches already exist, the potential of deep learning in nephropathology is only at its beginning and we can expect much more to come.

Noninvasive virtual biopsy using micro-registered optical coherence tomography (OCT) in human subjects

Histological hematoxylin and eosin-stained (H&E) tissue sections are used as the gold standard for pathologic detection of cancer, tumor margin detection, and disease diagnosis. Producing H&E sections, however, is invasive and time-consuming. While deep learning has shown promise in virtual staining of unstained tissue slides, true virtual biopsy requires staining of images taken from intact tissue. In this work, we developed a micron-accuracy coregistration method [micro-registered optical coherence tomography (OCT)] that can take a two-dimensional (2D) H&E slide and find the exact corresponding section in a 3D OCT image taken from the original fresh tissue. We trained a conditional generative adversarial network using the paired dataset and showed high-fidelity conversion of noninvasive OCT images to virtually stained H&E slices in both 2D and 3D. Applying these trained neural networks to in vivo OCT images should enable physicians to readily incorporate OCT imaging into their clinical practice, reducing the number of unnecessary biopsy procedures.

Time for a full digital approach in nephropathology: a systematic review of current artificial intelligence applications and future directions

INTRODUCTION

Artificial intelligence (AI) integration in nephropathology has been growing rapidly in recent years, facing several challenges including the wide range of histological techniques used, the low occurrence of certain diseases, and the need for data sharing. This narrative review retraces the history of AI in nephropathology and provides insights into potential future developments.

METHODS

Electronic searches in PubMed-MEDLINE and Embase were made to extract pertinent articles from the literature. Works about automated image analysis or the application of an AI algorithm on non-neoplastic kidney histological samples were included and analyzed to extract information such as publication year, AI task, and learning type. Prepublication servers and reviews were not included.

RESULTS

Seventy-six (76) original research articles were selected. Most of the studies were conducted in the United States in the last 7 years. To date, research has been mainly conducted on relatively easy tasks, like single-stain glomerular segmentation. However, there is a trend towards developing more complex tasks such as glomerular multi-stain classification.

CONCLUSION

Deep learning has been used to identify patterns in complex histopathology data and looks promising for the comprehensive assessment of renal biopsy, through the use of multiple stains and virtual staining techniques. Hybrid and collaborative learning approaches have also been explored to utilize large amounts of unlabeled data. A diverse team of experts, including nephropathologists, computer scientists, and clinicians, is crucial for the development of AI systems for nephropathology. Collaborative efforts among multidisciplinary experts result in clinically relevant and effective AI tools.

Operational greenhouse-gas emissions of deep learning in digital pathology: a modelling study

BACKGROUND

Deep learning is a promising way to improve health care. Image-processing medical disciplines, such as pathology, are expected to be transformed by deep learning. The first clinically applicable deep-learning diagnostic support tools are already available in cancer pathology, and their number is increasing. However, data on the environmental sustainability of these tools are scarce. We aimed to conduct an environmental-sustainability analysis of a theoretical implementation of deep learning in patient-care pathology.

METHODS

For this modelling study, we first assembled and calculated relevant data and parameters of a digital-pathology workflow. Data were breast and prostate specimens from the university clinic at the Institute of Pathology of the Rheinisch-Westfälische Technische Hochschule Aachen (Aachen, Germany), for which commercially available deep learning was already available. Only specimens collected between Jan 1 and Dec 31, 2019 were used, to omit potential biases due to the COVID-19 pandemic. Our final selection was based on 2 representative weeks outside holidays, covering different types of specimens. To calculate carbon dioxide (CO2) or CO2 equivalent (CO2 eq) emissions of deep learning in pathology, we gathered relevant data for exact numbers and sizes of whole-slide images (WSIs), which were generated by scanning histopathology samples of prostate and breast specimens. We also evaluated different data input scenarios (including all slide tiles, only tiles containing tissue, or only tiles containing regions of interest). To convert estimated energy consumption from kWh to CO2 eq, we used the internet protocol address of the computational server and the Electricity Maps database to obtain information on the sources of the local electricity grid (ie, renewable vs non-renewable), and estimated the number of trees and proportion of the local and world's forests needed to sequester the CO2 eq emissions. We calculated the computational requirements and CO2 eq emissions of 30 deep-learning models that varied in task and size. The first scenario represented the use of one commercially available deep-learning model for one task in one case (1-task), the second scenario considered two deep-learning models for two tasks per case (2-task), the third scenario represented a future, potentially automated workflow that could handle 7 tasks per case (7-task), and the fourth scenario represented the use of a single potential, large, computer-vision model that could conduct multiple tasks (multitask). We also compared the performance (ie, accuracy) and CO2 eq emissions of different deep-learning models for the classification of renal cell carcinoma on WSIs, also from Rheinisch-Westfälische Technische Hochschule Aachen. We also tested other approaches to reducing CO2 eq emissions, including model pruning and an alternative method for histopathology analysis (pathomics).

FINDINGS

The pathology database contained 35 552 specimens (237 179 slides), 6420 of which were prostate specimens (10 115 slides) and 11 801 of which were breast specimens (19 763 slides). We selected and subsequently digitised 140 slides from eight breast-cancer cases and 223 slides from five prostate-cancer cases. Applying large deep-learning models on all WSI tiles of prostate and breast pathology cases would result in yearly CO2 eq emissions of 7·65 metric tons (t; 95% CI 7·62-7·68) with the use of a single deep-learning model per case; yearly CO2 eq emissions were up to 100·56 t (100·21-100·99) with the use of seven deep-learning models per case. CO2 eq emissions for different deep-learning model scenarios, data inputs, and deep-learning model sizes for all slides varied from 3·61 t (3·59-3·63) to 2795·30 t (1177·51-6482·13. For the estimated number of overall pathology cases worldwide, the yearly CO2 eq emissions varied, reaching up to 16 megatons (Mt) of CO2 eq, requiring up to 86 590 km2 (0·22%) of world forest to sequester the CO2 eq emissions. Use of the 7-task scenario and small deep-learning models on slides containing tissue only could substantially reduce CO2 eq emissions worldwide by up to 141 times (0·1 Mt, 95% CI 0·1-0·1). Considering the local environment in Aachen, Germany, the maximum CO2 eq emission from the use of deep learning in digital pathology only would require 32·8% (95% CI 13·8-76·6) of the local forest to sequester the CO2 eq emissions. A single pathomics run on a tissue could provide information that was comparable to or even better than the output of multitask deep-learning models, but with 147 times reduced CO2 eq emissions.

INTERPRETATION

Our findings suggest that widespread use of deep learning in pathology might have considerable global-warming potential. The medical community, policy decision makers, and the public should be aware of this potential and encourage the use of CO2 eq emissions reduction strategies where possible.

FUNDING

German Research Foundation, European Research Council, German Federal Ministry of Education and Research, Health, Economic Affairs and Climate Action, and the Innovation Fund of the Federal Joint Committee.

Deep learning-enabled virtual histological staining of biological samples

Histological staining is the gold standard for tissue examination in clinical pathology and life-science research, which visualizes the tissue and cellular structures using chromatic dyes or fluorescence labels to aid the microscopic assessment of tissue. However, the current histological staining workflow requires tedious sample preparation steps, specialized laboratory infrastructure, and trained histotechnologists, making it expensive, time-consuming, and not accessible in resource-limited settings. Deep learning techniques created new opportunities to revolutionize staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, and accurate alternatives to standard chemical staining methods. These techniques, broadly referred to as virtual staining, were extensively explored by multiple research groups and demonstrated to be successful in generating various types of histological stains from label-free microscopic images of unstained samples; similar approaches were also used for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this Review, we provide a comprehensive overview of the recent research advances in deep learning-enabled virtual histological staining techniques. The basic concepts and the typical workflow of virtual staining are introduced, followed by a discussion of representative works and their technical innovations. We also share our perspectives on the future of this emerging field, aiming to inspire readers from diverse scientific fields to further expand the scope of deep learning-enabled virtual histological staining techniques and their applications.

Stain-Independent Deep Learning-Based Analysis of Digital Kidney Histopathology

Convolutional neural network (CNN)-based image analysis applications in digital pathology (eg, tissue segmentation) require a large amount of annotated data and are mostly trained and applicable on a single stain. Here, a novel concept based on stain augmentation is proposed to develop stain-independent CNNs requiring only one annotated stain. In this benchmark study on stain independence in digital pathology, this approach is comprehensively compared with state-of-the-art techniques including image registration and stain translation, and several modifications thereof. A previously developed CNN for segmentation of periodic acid-Schiff-stained kidney histology was used and applied to various immunohistochemical stainings. Stain augmentation showed very high performance in all evaluated stains and outperformed all other techniques in all structures and stains. Without the need for additional annotations, it enabled segmentation on immunohistochemical stainings with performance nearly comparable to that of the annotated periodic acid-Schiff stain and could further uphold performance on several held-out stains not seen during training. Herein, examples of how this framework can be applied for compartment-specific quantification of immunohistochemical stains for inflammation and fibrosis in animal models and patient biopsy specimens are presented. The results show that stain augmentation is a highly effective approach to enable stain-independent applications of deep-learning segmentation algorithms. This opens new possibilities for broad implementation in digital pathology.