Identification of glomerular lesions and intrinsic glomerular cell types in kidney diseases via deep learning

Advancing the application of the analytical renal pathology system in allograft IgA nephropathy patients

BACKGROUND

The analytical renal pathology system (ARPS) based on convolutional neural networks has been used successfully in native IgA nephropathy (IgAN) patients. Considering the similarity of pathologic features, we aim to evaluate the performance of the ARPS in allograft IgAN patients and broaden its implementation.

METHODS

Biopsy-proven allograft IgAN patients from two different centers were enrolled for internal and external validation. We implemented the ARPS to identify glomerular lesions and intrinsic glomerular cells, and then evaluated its performance. Consistency between the ARPS and pathologists was assessed using intraclass correlation coefficients. The association of digital pathological features with clinical and pathological data was measured. Kaplan-Meier survival curve and cox proportional hazards model were applied to investigate prognosis prediction.

RESULTS

A total of 56 biopsy-proven allograft IgAN patients from the internal center and 17 biopsy-proven allograft IgAN patients from the external center were enrolled in this study. The ARPS was successfully applied to identify the glomerular lesions (F1-score, 0.696-0.959) and quantify intrinsic glomerular cells (F1-score, 0.888-0.968) in allograft IgAN patients rapidly and precisely. Furthermore, the mesangial hypercellularity score was positively correlated with all mesangial metrics provided by ARPS [Spearman's correlation coefficient (r), 0.439-0.472, and all p values < 0.001]. Besides, a higher allograft survival was noticed among patients in the high-level groups of the maximum and ratio of endothelial cells, as well as the maximum and density of podocytes.

CONCLUSION

We propose that the ARPS could be implemented in future clinical practice with outstanding capability.

Artificial intelligence assists identification and pathologic classification of glomerular lesions in patients with diabetic nephropathy

BACKGROUND

Glomerular lesions are the main injuries of diabetic nephropathy (DN) and are used as a crucial index for pathologic classification. Manual quantification of these morphologic features currently used is semi-quantitative and time-consuming. Automatically quantifying glomerular morphologic features is urgently needed.

METHODS

A series of convolutional neural networks (CNN) were designed to identify and classify glomerular morphologic features in DN patients. Associations of these digital features with pathologic classification and prognosis were further analyzed.

RESULTS

Our CNN-based model achieved a 0.928 F1-score for global glomerulosclerosis and 0.953 F1-score for Kimmelstiel-Wilson lesion, further obtained a dice of 0.870 for the mesangial area and F1-score beyond 0.839 for three glomerular intrinsic cells. As the pathologic classes increased, mesangial cell numbers and mesangial area increased, and podocyte numbers decreased (p for all < 0.001), while endothelial cell numbers remained stable (p = 0.431). Glomeruli with Kimmelstiel-Wilson lesion showed more severe podocyte deletion compared to those without (p < 0.001). Furthermore, CNN-based classifications showed moderate agreement with pathologists-based classification, the kappa value between the CNN model 3 and pathologists reached 0.624 (ranging from 0.529 to 0.688, p < 0.001). Notably, CNN-based classifications obtained equivalent performance to pathologists-based classifications on predicting baseline and long-term renal function.

CONCLUSION

Our CNN-based model is promising in assisting the identification and pathologic classification of glomerular lesions in DN patients.

[Automatic classification of immune-mediated glomerular diseases based on multi-modal multi-instance learning]

OBJECTIVE

To develop a multi-modal deep learning method for automatic classification of immune-mediated glomerular diseases based on images of optical microscopy (OM), immunofluorescence microscopy (IM), and transmission electron microscopy (TEM).

METHODS

We retrospectively collected the pathological images from 273 patients and constructed a multi-modal multi- instance model for classification of 3 immune-mediated glomerular diseases, namely immunoglobulin A nephropathy (IgAN), membranous nephropathy (MN), and lupus nephritis (LN). This model adopts an instance-level multi-instance learning (I-MIL) method to select the TEM images for multi-modal feature fusion with the OM images and IM images of the same patient. By comparing this model with unimodal and bimodal models, we explored different combinations of the 3 modalities and the optimal methods for modal feature fusion.

RESULTS

The multi-modal multi-instance model combining OM, IM, and TEM images had a disease classification accuracy of (88.34±2.12)%, superior to that of the optimal unimodal model [(87.08±4.25)%] and that of the optimal bimodal model [(87.92±3.06)%].

CONCLUSION

This multi- modal multi- instance model based on OM, IM, and TEM images can achieve automatic classification of immune-mediated glomerular diseases with a good classification accuracy.

Artificial intelligence-assisted quantification and assessment of whole slide images for pediatric kidney disease diagnosis

MOTIVATION

Pediatric kidney disease is a widespread, progressive condition that severely impacts growth and development of children. Chronic kidney disease is often more insidious in children than in adults, usually requiring a renal biopsy for diagnosis. Biopsy evaluation requires copious examination by trained pathologists, which can be tedious and prone to human error. In this study, we propose an artificial intelligence (AI) method to assist pathologists in accurate segmentation and classification of pediatric kidney structures, named as AI-based Pediatric Kidney Diagnosis (APKD).

RESULTS

We collected 2935 pediatric patients diagnosed with kidney disease for the development of APKD. The dataset comprised 93 932 histological structures annotated manually by three skilled nephropathologists. APKD scored an average accuracy of 94% for each kidney structure category, including 99% in the glomerulus. We found strong correlation between the model and manual detection in detected glomeruli (Spearman correlation coefficient r = 0.98, P < .001; intraclass correlation coefficient ICC = 0.98, 95% CI = 0.96-0.98). Compared to manual detection, APKD was approximately 5.5 times faster in segmenting glomeruli. Finally, we show how the pathological features extracted by APKD can identify focal abnormalities of the glomerular capillary wall to aid in the early diagnosis of pediatric kidney disease.

AVAILABILITY AND IMPLEMENTATION

https://github.com/ChunyueFeng/Kidney-DataSet.

Digital pathology for nonalcoholic steatohepatitis assessment

Histological assessment of nonalcoholic fatty liver disease (NAFLD) has anchored knowledge development about the phenotypes of the condition, their natural history and their clinical course. This fact has led to the use of histological assessment as a reference standard for the evaluation of efficacy of drug interventions for nonalcoholic steatohepatitis (NASH) - the more histologically active form of NAFLD. However, certain limitations of conventional histological assessment systems pose challenges in drug development. These limitations have spurred intense scientific and commercial development of machine learning and digital approaches towards the assessment of liver histology in patients with NAFLD. This research field remains an area in rapid evolution. In this Perspective article, we summarize the current conventional assessment of NASH and its limitations, the use of specific digital approaches for histological assessment, and their application to the study of NASH and its response to therapy. Although this is not a comprehensive review, the leading tools currently used to assess therapeutic efficacy in drug development are specifically discussed. The potential translation of these approaches to support routine clinical assessment of NAFLD and an agenda for future research are also discussed.

Application of cloud server-based machine learning for assisting pathological structure recognition in IgA nephropathy

BACKGROUND

Machine learning (ML) models can help assisting diagnosis by rapidly localising and classifying regions of interest (ROIs) within whole slide images (WSIs). Effective ML models for clinical decision support require a substantial dataset of 'real' data, and in reality, it should be robust, user-friendly and universally applicable.

METHODS

WSIs of primary IgAN were collected and annotated. The H-AI-L algorithm which could facilitate direct WSI viewing and potential ROI detection for clinicians was built on the cloud server of matpool, a shared internet-based service platform. Model performance was evaluated using F1-score, precision, recall and Matthew's correlation coefficient (MCC).

RESULTS

The F1-score of glomerular localisation in WSIs was 0.85 and 0.89 for the initial and pretrained models, respectively, with corresponding recall values of 0.79 and 0.83, and precision scores of 0.92 and 0.97. Dichotomous differentiation between global sclerotic (GS) and other glomeruli revealed F1-scores of 0.70 and 0.91, and MCC values of 0.55 and 0.87, for the initial and pretrained models, respectively. The overall F1-score of multiclassification was 0.81 for the pretrained models. The total glomerular recall rate was 0.96, with F1-scores of 0.68, 0.56 and 0.26 for GS, segmental glomerulosclerosis and crescent (C), respectively. Interstitial fibrosis/tubular atrophy lesion similarity between the true label and model predictions was 0.75.

CONCLUSIONS

Our results underscore the efficacy of the ML integration algorithm in segmenting ROIs in IgAN WSIs, and the internet-based model deployment is in favour of widespread adoption and utilisation across multiple centres and increased volumes of WSIs.

Performance and limitations of a supervised deep learning approach for the histopathological Oxford Classification of glomeruli with IgA nephropathy

BACKGROUND AND OBJECTIVE

The Oxford Classification for IgA nephropathy is the most successful example of an evidence-based nephropathology classification system. The aim of our study was to replicate the glomerular components of Oxford scoring with an end-to-end deep learning pipeline that involves automatic glomerular segmentation followed by classification for mesangial hypercellularity (M), endocapillary hypercellularity (E), segmental sclerosis (S) and active crescents (C).

METHODS

A total number of 1056 periodic acid-Schiff (PAS) whole slide images (WSIs), coming from 386 kidney biopsies, were annotated. Several detection models for glomeruli, based on the Mask R-CNN architecture, were trained on 587 WSIs, validated on 161 WSIs, and tested on 127 WSIs. For the development of segmentation models, 20,529 glomeruli were annotated, of which 16,571 as training and 3958 as validation set. The test set of the segmentation module comprised of 2948 glomeruli. For the Oxford classification, 6206 expert-annotated glomeruli from 308 PAS WSIs were labelled for M, E, S, C and split into a training set of 4298 glomeruli from 207 WSIs, and a test set of 1908 glomeruli. We chose the best-performing models to construct an end-to-end pipeline, which we named MESCnn (MESC classification by neural network), for the glomerular Oxford classification of WSIs.

RESULTS

Instance segmentation yielded excellent results with an AP50 ranging between 78.2-80.1 % (79.4 ± 0.7 %) on the validation and 75.1-77.7 % (76.5 ± 0.9 %) on the test set. The aggregated Jaccard Index was between 73.4-75.9 % (75.0 ± 0.8 %) on the validation and 69.1-73.4 % (72.2 ± 1.4 %) on the test set. At granular glomerular level, Oxford Classification was best replicated for M with EfficientNetV2-L with a mean ROC-AUC of 90.2 % and a mean precision/recall area under the curve (PR-AUC) of 81.8 %, best for E with MobileNetV2 (ROC-AUC 94.7 %) and ResNet50 (PR-AUC 75.8 %), best for S with EfficientNetV2-M (mean ROC-AUC 92.7 %, mean PR-AUC 87.7 %), best for C with EfficientNetV2-L (ROC-AUC 92.3 %) and EfficientNetV2-S (PR-AUC 54.7 %). At biopsy-level, correlation between expert and deep learning labels fulfilled the demands of the Oxford Classification.

CONCLUSION

We designed an end-to-end pipeline for glomerular Oxford Classification on both a granular glomerular and an entire biopsy level. Both the glomerular segmentation and the classification modules are freely available for further development to the renal medicine community.

Diagnosis of diabetic kidney disease in whole slide images via AI-driven quantification of pathological indicators

Diagnosis of diabetic kidney disease (DKD) mainly relies on screening the morphological variations and internal lesions of glomeruli from pathological kidney biopsy. The prominent pathological alterations of glomeruli for DKD include glomerular hypertrophy and nodular mesangial sclerosis. However, the qualitative judgment of these alterations is inaccurate and inconstant due to the intra- and inter-subject variability of pathologists. It is necessary to design artificial intelligence (AI) methods for accurate quantification of these pathological alterations and outcome prediction of DKD. In this work, we present an AI-driven framework to quantify the volume of glomeruli and degree of nodular mesangial sclerosis, respectively, based on an instance segmentation module and a novel weakly supervised Macro-Micro Aggregation (MMA) module. Subsequently, we construct classic machine learning models to predict the degree of DKD based on three selected pathological indicators via factor analysis. These corresponding modules are trained and tested on a total of 281 whole slide images (WSIs) digitized from two hospitals with different scanners. Our designed AI framework achieved inspiring results with 0.926 mIoU for glomerulus segmentation, and 0.899 F1 score for glomerulus classification in the external testing dataset. Meantime, the visualized results of the MMA module could reflect the location of the lesions. The performance of predicting disease achieved the F1 score of 0.917, which further proved the effectiveness of our AI-driven quantification of pathological indicators. Additionally, the interpretation of the machine learning model with the SHAP method showed similar accordance with the development of DKD in pathology. In conclusion, the proposed auxiliary diagnostic technologies have the feasibility for quantitative analysis of glomerular pathological tissues and alterations in DKD. Pathological quantitative indicators will also make it more convenient to provide doctors with assistance in clinical practice.

A Spatially Guided Machine-Learning Method to Classify and Quantify Glomerular Patterns of Injury in Histology Images

Introduction The diagnosis of glomerular diseases is primarily based on visual assessment of histologic patterns. Semi-quantitative scoring of active and chronic lesions is often required to assess individual characteristics of the disease. Reproducibility of the visual scoring systems remains debatable, while digital and machine-learning technologies present opportunities to detect, classify and quantify glomerular lesions, also considering their inter- and intraglomerular heterogeneity.

MATERIALS AND METHODS

We performed a cross-validated comparison of three modifications of a convolutional neural network (CNN)-based approach for recognition and intraglomerular quantification of nine main glomerular patterns of injury. Reference values provided by two nephropathologists were used for validation. For each glomerular image, visual attention heatmaps were generated with a probability of class attribution for further intraglomerular quantification. The quality of classifier-produced heatmaps was evaluated by intersection over union metrics (IoU) between predicted and ground truth localization heatmaps.

RESULTS

A proposed spatially guided modification of the CNN classifier achieved the highest glomerular pattern classification accuracies, with area under curve (AUC) values up to 0.981. With regards to heatmap overlap area and intraglomerular pattern quantification, the spatially guided classifier achieved a significantly higher generalized mean IoU value compared to single-multiclass and multiple-binary classifiers.

CONCLUSIONS

We propose a spatially guided CNN classifier that in our experiments reveals the potential to achieve high accuracy for the localization of intraglomerular patterns.

Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model

Glomeruli are interconnected capillaries in the renal cortex that are responsible for blood filtration. Damage to these glomeruli often signifies the presence of kidney disorders like glomerulonephritis and glomerulosclerosis, which can ultimately lead to chronic kidney disease and kidney failure. The timely detection of such conditions is essential for effective treatment. This paper proposes a modified UNet model to accurately detect glomeruli in whole-slide images of kidney tissue. The UNet model was modified by changing the number of filters and feature map dimensions from the first to the last layer to enhance the model's capacity for feature extraction. Moreover, the depth of the UNet model was also improved by adding one more convolution block to both the encoder and decoder sections. The dataset used in the study comprised 20 large whole-side images. Due to their large size, the images were cropped into 512 × 512-pixel patches, resulting in a dataset comprising 50,486 images. The proposed model performed well, with 95.7% accuracy, 97.2% precision, 96.4% recall, and 96.7% F1-score. These results demonstrate the proposed model's superior performance compared to the original UNet model, the UNet model with EfficientNetb3, and the current state-of-the-art. Based on these experimental findings, it has been determined that the proposed model accurately identifies glomeruli in extracted kidney patches.

Artificial intelligence can accurately distinguish IgA nephropathy from diabetic nephropathy under Masson staining and becomes an important assistant for renal pathologists

Introduction

Hyperplasia of the mesangial area is common in IgA nephropathy (IgAN) and diabetic nephropathy (DN), and it is often difficult to distinguish them by light microscopy alone, especially in the absence of clinical data. At present, artificial intelligence (AI) is widely used in pathological diagnosis, but mainly in tumor pathology. The application of AI in renal pathological is still in its infancy.

Methods

Patients diagnosed as IgAN or DN by renal biopsy in First Affiliated Hospital of Zhejiang Chinese Medicine University from September 1, 2020 to April 30, 2022 were selected as the training set, and patients who diagnosed from May 1, 2022 to June 30, 2022 were selected as the test set. We focused on the glomerulus and captured the field of the glomerulus in Masson staining WSI at 200x magnification, all in 1,000 × 1,000 pixels JPEG format. We augmented the data from training set through minor affine transformation, and then randomly split the training set into training and adjustment data according to 8:2. The training data and the Yolov5 6.1 algorithm were used to train the AI model with constant adjustment of parameters according to the adjusted data. Finally, we obtained the optimal model, tested this model with test set and compared it with renal pathologists.

Results

AI can accurately detect the glomeruli. The overall accuracy of AI glomerulus detection was 98.67% and the omission rate was only 1.30%. No Intact glomerulus was missed. The overall accuracy of AI reached 73.24%, among which the accuracy of IgAN reached 77.27% and DN reached 69.59%. The AUC of IgAN was 0.733 and that of DN was 0.627. In addition, compared with renal pathologists, AI can distinguish IgAN from DN more quickly and accurately, and has higher consistency.

Discussion

We constructed an AI model based on Masson staining images of renal tissue to distinguish IgAN from DN. This model has also been successfully deployed in the work of renal pathologists to assist them in their daily diagnosis and teaching work.

Computational Pipeline for Glomerular Segmentation and Association of the Quantified Regions with Prognosis of Kidney Function in IgA Nephropathy

The histopathological findings of the glomeruli from whole slide images (WSIs) of a renal biopsy play an important role in diagnosing and grading kidney disease. This study aimed to develop an automated computational pipeline to detect glomeruli and to segment the histopathological regions inside of the glomerulus in a WSI. In order to assess the significance of this pipeline, we conducted a multivariate regression analysis to determine whether the quantified regions were associated with the prognosis of kidney function in 46 cases of immunoglobulin A nephropathy (IgAN). The developed pipelines showed a mean intersection over union (IoU) of 0.670 and 0.693 for five classes (i.e., background, Bowman's space, glomerular tuft, crescentic, and sclerotic regions) against the WSI of its facility, and 0.678 and 0.609 against the WSI of the external facility. The multivariate analysis revealed that the predicted sclerotic regions, even those that were predicted by the external model, had a significant negative impact on the slope of the estimated glomerular filtration rate after biopsy. This is the first study to demonstrate that the quantified sclerotic regions that are predicted by an automated computational pipeline for the segmentation of the histopathological glomerular components on WSIs impact the prognosis of kidney function in patients with IgAN.

Artificial intelligence in glomerular diseases

In this narrative review, we focus on the application of artificial intelligence in the clinical history of patients with glomerular disease, digital pathology in kidney biopsy, renal ultrasonography imaging, and prediction of chronic kidney disease (CKD). With the development of natural language processing, the clinical history of a patient can be used to identify a computable phenotype. In kidney pathology, digital imaging has adopted innovative deep learning algorithms (DLAs) that can improve the predictive capability of the examined lesions. However, at this time, these applications can only be used in research because there is no recognized validation to replace the conventional diagnostic applications. Kidney ultrasonography, used in the clinical examination of patients, provides information about the progression of kidney damage. Machine learning algorithms (MLAs) with promising results for the early detection of CKD have been proposed, but, still, they are not solid enough to be incorporated into the clinical practice. A few tools for glomerulonephritis, based on MLAs, are available in clinical practice. They can be downloaded on computers and cellular phones but can only be applied to uniracial cohorts of patients. To improve their performance, it is necessary to organize large consortia with multiracial cohorts. Finally, in many studies MLA development has been carried out using retrospective cohorts. The performance of the models might differ in retrospective cohorts compared to real-world data. Therefore, the models should be validated in prospective external large cohorts.

Boundary-aware glomerulus segmentation: Toward one-to-many stain generalization

The growing availability of scanned whole-slide images (WSIs) has allowed nephropathology to open new possibilities for medical decision-making over high-resolution images. Diagnosis of renal WSIs includes locating and identifying specific structures in the tissue. Considering the glomerulus as one of the first structures analyzed by pathologists, we propose here a novel convolutional neural network for glomerulus segmentation. Our end-to-end network, named DS-FNet, combines the strengths of semantic segmentation and semantic boundary detection networks via an attention-aware mechanism. Although we trained the proposed network on periodic acid-Schiff (PAS)-stained WSIs, we found that our network was capable to segment glomeruli on WSIs stained with different techniques, such as periodic acid-methenamine silver (PAMS), hematoxylin-eosin (HE), and Masson trichrome (TRI). To assess the performance of the proposed method, we used three public data sets: HuBMAP (available in a Kaggle competition), a subset of the NEPTUNE data set, and a novel challenging data set, called WSI_Fiocruz. We compared the DS-FNet with six other deep learning networks: original U-Net, our attention version of U-Net called AU-Net, U-Net++, U-Net3Plus, ResU-Net, and DeepLabV3+. Results showed that DS-FNet achieved equivalent or superior results on all data sets: On the HuBMAP data set, it reached a dice score (DSC) of 95.05%, very close to the first place (95.15%); on the NEPTUNE and WSI_Fiocruz data sets, DS-FNet obtained the highest average DSC, whether on PAS-stained images or images stained with other techniques. To the best we know, this is the first work to show consistently high performance in a one-to-many-stain glomerulus segmentation following a thorough protocol on data sets from different medical labs.

Glo-In-One: holistic glomerular detection, segmentation, and lesion characterization with large-scale web image mining

Purpose: The quantitative detection, segmentation, and characterization of glomeruli from high-resolution whole slide imaging (WSI) play essential roles in the computer-assisted diagnosis and scientific research in digital renal pathology. Historically, such comprehensive quantification requires extensive programming skills to be able to handle heterogeneous and customized computational tools. To bridge the gap of performing glomerular quantification for non-technical users, we develop the Glo-In-One toolkit to achieve holistic glomerular detection, segmentation, and characterization via a single line of command. Additionally, we release a large-scale collection of 30,000 unlabeled glomerular images to further facilitate the algorithmic development of self-supervised deep learning. Approach: The inputs of the Glo-In-One toolkit are WSIs, while the outputs are (1) WSI-level multi-class circle glomerular detection results (which can be directly manipulated with ImageScope), (2) glomerular image patches with segmentation masks, and (3) different lesion types. In the current version, the fine-grained global glomerulosclerosis (GGS) characterization is provided, including assessed-solidified-GSS (associated with hypertension-related injury), disappearing-GSS (a further end result of the SGGS becoming contiguous with fibrotic interstitium), and obsolescent-GSS (nonspecific GGS increasing with aging) glomeruli. To leverage the performance of the Glo-In-One toolkit, we introduce self-supervised deep learning to glomerular quantification via large-scale web image mining. Results: The GGS fine-grained classification model achieved a decent performance compared with baseline supervised methods while only using 10% of the annotated data. The glomerular detection achieved an average precision of 0.627 with circle representations, while the glomerular segmentation achieved a 0.955 patch-wise Dice dimilarity coefficient. Conclusion: We develop and release an open-source Glo-In-One toolkit, a software with holistic glomerular detection, segmentation, and lesion characterization. This toolkit is user-friendly to non-technical users via a single line of command. The toolbox and the 30,000 web mined glomerular images have been made publicly available at https://github.com/hrlblab/Glo-In-One.

Artificial Intelligence-Assisted Renal Pathology: Advances and Prospects

Digital imaging and advanced microscopy play a pivotal role in the diagnosis of kidney diseases. In recent years, great achievements have been made in digital imaging, providing novel approaches for precise quantitative assessments of nephropathology and relieving burdens of renal pathologists. Developing novel methods of artificial intelligence (AI)-assisted technology through multidisciplinary interaction among computer engineers, renal specialists, and nephropathologists could prove beneficial for renal pathology diagnoses. An increasing number of publications has demonstrated the rapid growth of AI-based technology in nephrology. In this review, we offer an overview of AI-assisted renal pathology, including AI concepts and the workflow of processing digital image data, focusing on the impressive advances of AI application in disease-specific backgrounds. In particular, this review describes the applied computer vision algorithms for the segmentation of kidney structures, diagnosis of specific pathological changes, and prognosis prediction based on images. Lastly, we discuss challenges and prospects to provide an objective view of this topic.

Evaluating tubulointerstitial compartments in renal biopsy specimens using a deep learning-based approach for classifying normal and abnormal tubules

Renal pathology is essential for diagnosing and assessing the severity and prognosis of kidney diseases. Deep learning-based approaches have developed rapidly and have been applied in renal pathology. However, methods for the automated classification of normal and abnormal renal tubules remain scarce. Using a deep learning-based method, we aimed to classify normal and abnormal renal tubules, thereby assisting renal pathologists in the evaluation of renal biopsy specimens. Consequently, we developed a U-Net-based segmentation model using randomly selected regions obtained from 21 renal biopsy specimens. Further, we verified its performance in multiclass segmentation by calculating the Dice coefficients (DCs). We used 15 cases of tubulointerstitial nephritis to assess its applicability in aiding routine diagnoses conducted by renal pathologists and calculated the agreement ratio between diagnoses conducted by two renal pathologists and the time taken for evaluation. We also determined whether such diagnoses were improved when the output of segmentation was considered. The glomeruli and interstitium had the highest DCs, whereas the normal and abnormal renal tubules had intermediate DCs. Following the detailed evaluation of the tubulointerstitial compartments, the proximal, distal, atrophied, and degenerated tubules had intermediate DCs, whereas the arteries and inflamed tubules had low DCs. The annotation and output areas involving normal and abnormal tubules were strongly correlated in each class. The pathological concordance for the glomerular count, t, ct, and ci scores of the Banff classification of renal allograft pathology remained high with or without the segmented images. However, in terms of time consumption, the quantitative assessment of tubulitis, tubular atrophy, degenerated tubules, and the interstitium was improved significantly when renal pathologists considered the segmentation output. Deep learning algorithms can assist renal pathologists in the classification of normal and abnormal tubules in renal biopsy specimens, thereby facilitating the enhancement of renal pathology and ensuring appropriate clinical decisions.

The potential of artificial intelligence-based applications in kidney pathology

PURPOSE OF REVIEW

The field of pathology is currently undergoing a significant transformation from traditional glass slides to a digital format dependent on whole slide imaging. Transitioning from glass to digital has opened the field to development and application of image analysis technology, commonly deep learning methods (artificial intelligence [AI]) to assist pathologists with tissue examination. Nephropathology is poised to leverage this technology to improve precision, accuracy, and efficiency in clinical practice.

RECENT FINDINGS

Through a multidisciplinary approach, nephropathologists, and computer scientists have made significant recent advances in developing AI technology to identify histological structures within whole slide images (segmentation), quantification of histologic structures, prediction of clinical outcomes, and classifying disease. Virtual staining of tissue and automation of electron microscopy imaging are emerging applications with particular significance for nephropathology.

SUMMARY

AI applied to image analysis in nephropathology has potential to transform the field by improving diagnostic accuracy and reproducibility, efficiency, and prognostic power. Reimbursement, demonstration of clinical utility, and seamless workflow integration are essential to widespread adoption.

Automatic Evaluation of Histological Prognostic Factors Using Two Consecutive Convolutional Neural Networks on Kidney Samples

BACKGROUND AND OBJECTIVES

The prognosis of patients undergoing kidney tumor resection or kidney donation is linked to many histologic criteria. These criteria notably include glomerular density, glomerular volume, vascular luminal stenosis, and severity of interstitial fibrosis/tubular atrophy. Automated measurements through a deep-learning approach could save time and provide more precise data. This work aimed to develop a free tool to automatically obtain kidney histologic prognostic features.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

In total, 241 samples of healthy kidney tissue were split into three independent cohorts. The "Training" cohort (n=65) was used to train two convolutional neural networks: one to detect the cortex and a second to segment the kidney structures. The "Test" cohort (n=50) assessed their performance by comparing manually outlined regions of interest to predicted ones. The "Application" cohort (n=126) compared prognostic histologic data obtained manually or through the algorithm on the basis of the combination of the two convolutional neural networks.

RESULTS

In the Test cohort, the networks isolated the cortex and segmented the elements of interest with good performances (>90% of the cortex, healthy tubules, glomeruli, and even globally sclerotic glomeruli were detected). In the Application cohort, the expected and predicted prognostic data were significantly correlated. The correlation coefficients r were 0.85 for glomerular volume, 0.51 for glomerular density, 0.75 for interstitial fibrosis, 0.71 for tubular atrophy, and 0.73 for vascular intimal thickness, respectively. The algorithm had a good ability to predict significant (>25%) tubular atrophy and interstitial fibrosis level (receiver operator characteristic curve with an area under the curve, 0.92 and 0.91, respectively) or a significant vascular luminal stenosis (>50%) (area under the curve, 0.85).

CONCLUSION

This freely available tool enables the automated segmentation of kidney tissue to obtain prognostic histologic data in a fast, objective, reliable, and reproducible way.

Assessment of glomerular morphological patterns by deep learning algorithms

Background

Compilation of different morphological lesion signatures is characteristic of renal pathology. Previous studies have documented the potential value of artificial intelligence (AI) in recognizing relatively clear-cut glomerular structures and patterns, such as segmental or global sclerosis or mesangial hypercellularity. This study aimed to test the capacity of deep learning algorithms to recognize complex glomerular structural changes that reflect common diagnostic dilemmas in nephropathology.

Methods

For this purpose, we defined nine classes of glomerular morphological patterns and trained twelve convolutional neuronal network (CNN) models on these. The two-step training process was done on a first dataset defined by an expert nephropathologist (12,253 images) and a second consensus dataset (11,142 images) defined by three experts in the field.

Results

The efficacy of CNN training was evaluated using another set with 180 consensus images, showing convincingly good classification results (kappa-values 0.838–0.938).

Furthermore, we elucidated the image areas decisive for CNN-based decision making by class activation maps. Finally, we demonstrated that the algorithm could decipher glomerular disease patterns coinciding in a single glomerulus (e.g. necrosis along with mesangial and endocapillary hypercellularity).

Conclusions

In summary, our model, focusing on glomerular lesions detectable by conventional microscopy, is the first sui generis to deploy deep learning as a reliable and promising tool in recognition of even discrete and/or overlapping morphological changes. Our results provide a stimulus for ongoing projects that integrate further input levels next to morphology (such as immunohistochemistry, electron microscopy, and clinical information) to develop a novel tool applicable for routine diagnostic nephropathology.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40620-021-01221-9.

Holistic fine-grained global glomerulosclerosis characterization: from detection to unbalanced classification

Purpose: Recent studies have demonstrated the diagnostic and prognostic values of global glomerulosclerosis (GGS) in IgA nephropathy, aging, and end-stage renal disease. However, the fine-grained quantitative analysis of multiple GGS subtypes (e.g., obsolescent, solidified, and disappearing glomerulosclerosis) is typically a resource extensive manual process. Very few automatic methods, if any, have been developed to bridge this gap for such analytics. We present a holistic pipeline to quantify GGS (with both detection and classification) from a whole slide image in a fully automatic manner. In addition, we conduct the fine-grained classification for the subtypes of GGS. Our study releases the open-source quantitative analytical tool for fine-grained GGS characterization while tackling the technical challenges in unbalanced classification and integrating detection and classification. Approach: We present a deep learning-based framework to perform fine-grained detection and classification of GGS, with a hierarchical two-stage design. Moreover, we incorporate the state-of-the-art transfer learning techniques to achieve a more generalizable deep learning model for tackling the imbalanced distribution of our dataset. This way, we build a highly efficient WSI-to-results GGS characterization pipeline. Meanwhile, we investigated the largest fine-grained GGS cohort as of yet with 11,462 glomeruli and 10,619 nonglomeruli, which include 7841 globally sclerotic glomeruli of three distinct categories. With these data, we apply deep learning techniques to achieve (1) fine-grained GGS characterization, (2) GGS versus non-GGS classification, and (3) improved glomeruli detection results. Results: For fine-grained GGS characterization, when pretrained on the larger dataset, our model can achieve a 0.778-macro- F 1 score, compared to a 0.746-macro- F 1 score when using the regular ImageNet-pretrained weights. On the external dataset, our best model achieves an area under the curve (AUC) score of 0.994 when tasked with differentiating GGS from normal glomeruli. Using our dataset, we are able to build algorithms that allow for fine-grained classification of glomeruli lesions and are robust to distribution shifts. Conclusion: Our study showed that the proposed methods consistently improve the detection and fine-grained classification performance through both cross validation and external validation. Our code and pretrained models have been released for public use at https://github.com/luyuzhe111/glomeruli.

Deep Learning-Based Model Significantly Improves Diagnostic Performance for Assessing Renal Histopathology in Lupus Glomerulonephritis

<b><i>Background:</i></b> Assessment of glomerular lesions and structures plays an essential role in understanding the pathological diagnosis of glomerulonephritis and prognostic evaluation of many kidney diseases. Renal pathophysiological assessment requires novel high-throughput tools to conduct quantitative, unbiased, and reproducible analyses representing a central readout. Deep learning may be an effective tool for glomerulonephritis pathological analysis. <b><i>Methods:</i></b> We developed a murine renal pathological system (MRPS) model to objectify the pathological evaluation via the deep learning method on whole-slide image (WSI) segmentation and feature extraction. A convolutional neural network model was used for accurate segmentation of glomeruli and glomerular cells of periodic acid-Schiff-stained kidney tissue from healthy and lupus nephritis mice. To achieve a quantitative evaluation, we subsequently filtered five independent predictors as image biomarkers from all features and developed a formula for the scoring model. <b><i>Results:</i></b> Perimeter, shape factor, minimum internal diameter, minimum caliper diameter, and number of objects were identified as independent predictors and were included in the establishment of the MRPS. The MRPS showed a positive correlation with renal score (<i>r</i> = 0.480, <i>p</i> &#x3c; 0.001) and obtained great diagnostic performance in discriminating different score bands (Obuchowski index, 0.842 [95% confidence interval: 0.759, 0.925]), with an area under the curve of 0.78–0.98, sensitivity of 58–93%, specificity of 72–100%, and accuracy of 74–94%. <b><i>Conclusion:</i></b> Our MRPS for quantitative assessment of renal WSIs from MRL/lpr lupus nephritis mice enables accurate histopathological analyses with high reproducibility, which may serve as a useful tool for glomerulonephritis diagnosis and prognosis evaluation.

Focal Segmental Glomerulosclerosis Superimposed on Transplant Glomerulopathy: Implications for Graft Survival

INTRODUCTION

Transplant glomerulopathy (TG) is a morphological lesion resulting from chronic glomerular endothelium injury, and it is strongly associated with poor graft survival. TG coexisting with focal segmental glomerulosclerosis (FSGS) can be found in renal allograft biopsies, but few related studies are available.

METHODS

Consecutive kidney transplant recipients with biopsy-proven TG were studied retrospectively. Patients concomitant with FSGS were identified and compared with those without FSGS. The influence of FSGS on allograft outcomes was assessed using univariate and multivariate Cox regression models.

RESULTS

Of the 66 patients with TG, 40 (60.6%) had concomitant FSGS. TG patients with FSGS had higher proteinuria (median, 2.6 vs. 0.8 g/24 h, p < 0.001) and serum creatinine levels (median, 2.5 vs. 2.1 mg/dL, p = 0.04), lower serum albumin levels, higher chronic glomerulopathy (cg) score, larger glomerular tuft area, lower number of podocytes, and higher incidences of podocyte hyperplasia, pseudotubule formation, and diffuse foot process effacement than those without FSGS (all p < 0.05). The kidney allograft loss rate of patients with FSGS was higher than that of patients without FSGS (65.7% vs. 37.5%, p = 0.03). The presence of FSGS was independently associated with allograft loss in TG (hazard ratio (HR) = 3.42, 95% confidence interval (CI): 1.30-8.98, p = 0.01). Other independent predictors were proteinuria (HR = 1.18, 95% CI: 1.02-1.37, p = 0.02), estimated glomerular filtration rate (HR = 0.94, 95% CI: 0.91-0.97, p < 0.001), and panel reactive antibody (HR = 3.99, 95% CI: 1.14-13.99, p = 0.03). Moreover, FSGS (odds ratio (OR) = 4.39, 95% CI: 1.29-14.92, p = 0.02) and cg (OR = 5.36, 95% CI: 1.56-18.40, p = 0.01) were independent risk factors for proteinuria.

CONCLUSION

In this cohort of patients with TG, the presence of FSGS was strongly associated with more severe clinicopathological features and worse allograft survival.

Multi-view features integrated 2D\3D Net for glomerulopathy histologic types classification using ultrasound images

BACKGROUND AND OBJECTIVE

Early diagnoses and rational therapeutics of glomerulopathy can control progression and improve prognosis. The gold standard for the diagnosis of glomerulopathy is pathology by renal biopsy, which is invasive and has many contraindications. We aim to use renal ultrasonography for histologic classification of glomerulopathy.

METHODS

Ultrasonography can present multi-view sections of kidney, thus we proposed a multi-view and cross-domain integration strategy (CD-ConcatNet) to obtain more effective features and improve diagnosis accuracy. We creatively apply 2D group convolution and 3D convolution to process multiple 2D ultrasound images and extract multi-view features of renal ultrasound images. Cross-domain concatenation in each spatial resolution of feature maps is applied for more informative feature learning.

RESULTS

A total of 76 adult patients were collected and divided into training dataset (56 cases with 515 images) and validation dataset (20 cases with 180 images). We obtained the best mean accuracy of 0.83 and AUC of 0.8667 in the validation dataset.

CONCLUSION

Comparison experiments demonstrate that our designed CD-ConcatNet achieves the best classification performance and has great superiority on histologic types diagnosis. Results also prove that the integration of multi-view ultrasound images is beneficial for histologic classification and ultrasound images can indeed provide discriminating information for histologic diagnosis.

Automated Diagnosis and Localization of Melanoma from Skin Histopathology Slides Using Deep Learning: A Multicenter Study

In traditional hospital systems, diagnosis and localization of melanoma are the critical challenges for pathological analysis, treatment instructions, and prognosis evaluation particularly in skin diseases. In literature, various studies have been reported to address these issues; however, a prominent smart diagnosis system is needed to be developed for the smart healthcare system. In this study, a deep learning-enabled diagnostic system is proposed and implemented that it has the capacity to automatically detect malignant melanoma in whole slide images (WSIs). In this system, the convolutional neural network (CNN), sophisticated statistical method, and image processing algorithms were integrated and implemented to locate benign and malignant lesions which are extremely useful in the diagnoses process of melanoma disease. To verify the exceptional performance of the proposed scheme, it is implemented in a multicenter database, which has 701 WSIs (641 WSIs from Central South University Xiangya Hospital (CSUXH) and 60 WSIs from the Cancer Genome Atlas (TCGA)). Experimental results have verified that the proposed system has achieved an area under the receiver operating characteristic curve (AUROC) of 0.971. Furthermore, the lesion area on the WSIs is represented by its degree of malignancy. These results show that the proposed system has the capacity to fully automate the diagnosis and localization problem of the melanoma in the smart healthcare systems.

Deep Learning-Based Artificial Intelligence System for Automatic Assessment of Glomerular Pathological Findings in Lupus Nephritis

Accurate assessment of renal histopathology is crucial for the clinical management of patients with lupus nephritis (LN). However, the current classification system has poor interpathologist agreement. This paper proposes a deep convolutional neural network (CNN)-based system that detects and classifies glomerular pathological findings in LN. A dataset of 349 renal biopsy whole-slide images (WSIs) (163 patients with LN, periodic acid-Schiff stain, 3906 glomeruli) annotated by three expert nephropathologists was used. The CNN models YOLOv4 and VGG16 were employed to localise the glomeruli and classify glomerular lesions (slight/severe impairments or sclerotic lesions). An additional 321 unannotated WSIs from 161 patients were used for performance evaluation at the per-patient kidney level. The proposed model achieved an accuracy of 0.951 and Cohen's kappa of 0.932 (95% CI 0.915-0.949) for the entire test set for classifying the glomerular lesions. For multiclass detection at the glomerular level, the mean average precision of the CNN was 0.807, with 'slight' and 'severe' glomerular lesions being easily identified (F1: 0.924 and 0.952, respectively). At the per-patient kidney level, the model achieved a high agreement with nephropathologist (linear weighted kappa: 0.855, 95% CI: 0.795-0.916, p < 0.001; quadratic weighted kappa: 0.906, 95% CI: 0.873-0.938, p < 0.001). The results suggest that deep learning is a feasible assistive tool for the objective and automatic assessment of pathological LN lesions.

A deep learning- and CT image-based prognostic model for the prediction of survival in non-small cell lung cancer

OBJECTIVE

To assist clinicians in arranging personalized treatment, planning follow-up programs and extending survival times for non-small cell lung cancer (NSCLC) patients, a method of deep learning combined with computed tomography (CT) imaging for survival prediction was designed.

METHODS

Data were collected from 484 patients from four research centers. The data from 344 patients were utilized to build the A_CNN survival prognosis model to classify 2-year overall survival time ranges (730 days cut-off). Data from 140 patients, including independent internal and external test sets, were utilized for model testing. First, a series of preprocessing techniques were used to process the original CT images and generate training and test data sets from the axial, coronal, and sagittal planes. Second, the structure of the A_CNN model was designed based on asymmetric convolution, bottleneck blocks, the uniform cross-entropy (UC) loss function, and other advanced techniques. After that, the A_CNN model was trained, and numerous comparative experiments were designed to obtain the best prognostic survival model. Last, the model performance was evaluated, and the predicted survival curves were analyzed.

RESULTS

The A_CNN survival prognosis model yielded a high patient-level accuracy of 88.8%, a patch-level accuracy of 82.9%, and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.932. When tested on an external data set, the maximum patient-level accuracy was 80.0%.

CONCLUSIONS

The results suggest that using a deep learning method can improve prognosis in patients with NSCLC and has important application value in establishing individualized prognostic models.

Multi-Task Learning-Based Immunofluorescence Classification of Kidney Disease

Chronic kidney disease is one of the most important causes of mortality worldwide, but a shortage of nephrology pathologists has led to delays or errors in its diagnosis and treatment. Immunofluorescence (IF) images of patients with IgA nephropathy (IgAN), membranous nephropathy (MN), diabetic nephropathy (DN), and lupus nephritis (LN) were obtained from the General Hospital of Chinese PLA. The data were divided into training and test data. To simulate the inaccurate focus of the fluorescence microscope, the Gaussian method was employed to blur the IF images. We proposed a novel multi-task learning (MTL) method for image quality assessment, de-blurring, and disease classification tasks. A total of 1608 patients’ IF images were included—1289 in the training set and 319 in the test set. For non-blurred IF images, the classification accuracy of the test set was 0.97, with an AUC of 1.000. For blurred IF images, the proposed MTL method had a higher accuracy (0.94 vs. 0.93, p < 0.01) and higher AUC (0.993 vs. 0.986) than the common MTL method. The novel MTL method not only diagnosed four types of kidney diseases through blurred IF images but also showed good performance in two auxiliary tasks: image quality assessment and de-blurring.

Glomerular Disease Classification and Lesion Identification by Machine Learning

Background

Classification of glomerular diseases and identification of glomerular lesions require careful morphological examination by experienced nephropathologists, which is labor-intensive, time-consuming, and prone to interobserver variability. In this regard, recent advance in machine learning-based image analysis is promising.

Methods

We combined Mask Region-based Convolutional Neural Networks (Mask R–CNN) with an additional classification step to build a glomerulus detection model using human kidney biopsy samples. A Long Short-Term Memory (LSTM) recurrent neural network was applied for glomerular disease classification, and another two-stage model using ResNeXt-101 was constructed for glomerular lesion identification in cases of lupus nephritis.

Results

The detection model showed state-of-the-art performance on variedly stained slides with F1 scores up to 0.944. The disease classification model showed good accuracies up to 0.940 on recognizing different glomerular diseases based on H&E whole slide images. The lesion identification model demonstrated high discriminating power with area under the receiver operating characteristic curve up to 0.947 for various glomerular lesions. Models showed good generalization on external testing datasets.

Conclusion

This study is the first-of-its-kind showing how each step of kidney biopsy interpretation carried out by nephropathologists can be captured and simulated by machine learning models. The models were integrated into a whole slide image viewing and annotating platform to enable nephropathologists to review, correct, and confirm the inference results. Further improvement on model performances and incorporating inputs from immunofluorescence, electron microscopy, and clinical data might realize actual clinical use.

Evaluation of Kidney Histological Images Using Unsupervised Deep Learning

INTRODUCTION

Evaluating histopathology via machine learning has gained research and clinical interest, and the performance of supervised learning tasks has been described in various areas of medicine. Unsupervised learning of histological images has the advantage of reproducibility for labeling; however, the relationship between unsupervised evaluation and clinical information remains unclear in nephrology.

METHODS

We propose an unsupervised approach combining convolutional neural networks (CNNs) and a visualization algorithm to cluster the histological images and calculate the score for patients. We applied the approach to the entire images or patched images of the glomerulus of kidney biopsy samples stained with hematoxylin and eosin obtained from 68 patients with immunoglobulin A nephropathy. We assessed the relationship between the obtained scores and clinical variables of urinary occult blood, urinary protein, serum creatinine (SCr), systolic blood pressure, and age.

RESULTS

The glomeruli of the patients were classified into 12 distinct classes and 10 patches. The output of the fine-tuned CNN, which we defined as the histological scores, had significant relationships with assessed clinical variables. In addition, the clustering and visualization results suggested that the defined clusters captured important findings when evaluating renal histopathology. For the score of the patch-based cluster containing crescentic glomeruli, SCr (coefficient = 0.09, P = 0.019) had a significant relationship.

CONCLUSION

The proposed approach could successfully extract features that were related to the clinical variables from the kidney biopsy images along with the visualization for interpretability. The approach could aid in the quantified evaluation of renal histopathology.

A Deep Learning-Based Approach for Glomeruli Instance Segmentation from Multistained Renal Biopsy Pathologic Images

Glomeruli instance segmentation from pathologic images is a fundamental step in the automatic analysis of renal biopsies. Glomerular histologic manifestations vary widely among diseases and cases, and several special staining methods are necessary for pathologic diagnosis. A robust model is needed to segment and classify glomeruli with different staining methods and apply in cases with various glomerular pathologic changes. Herein, pathologic images from renal biopsy slides stained with three basic special staining methods were used to build the data sets. The snapshot group included 1970 glomeruli from 516 patients, and the whole-slide image group included 8665 glomeruli from 148 patients. Cascade Mask region-based convolutional neural net architecture was trained to detect, classify, and segment glomeruli into three categories: i) GN, structural normal; ii) global sclerosis; and iii) glomerular with other lesions. In the snapshot group, total glomeruli, GN, global sclerosis, and glomerular with other lesions achieved an F1 score of 0.914, 0.896, 0.681, and 0.756, respectively, which were comparable with those in the whole-slide image group (0.940, 0.839, 0.806, and 0.753, respectively). Among the three categories, GN achieved the best instance segmentation effect in both groups, as determined by average precision, average recall, F1 score, and Mask mean Intersection over Union. The present model segments and classifies multistained glomeruli with efficiency and robustness. It can be applied as the first step for more detailed glomerular histologic analysis.

The evolution and future of diabetic kidney disease research: a bibliometric analysis

BACKGROUND

Diabetic kidney disease (DKD) is one of the most important complications of diabetic mellitus. It is essential for nephrologists to understand the evolution and development trends of DKD.

METHODS

Based on the total cited numbers in the Web of Science Core Collection, which was searched through September 28th, 2020, we performed a bibliometric analysis of the top 100 most cited full-length original articles on the subject of DKD. The timespans, authors, contributions, subcategories, and topics of those 100 articles were analysed. In addition, the evolution of topics in DKD research was investigated.

RESULTS

There were 23,968 items under the subject of DKD in the Web of Science Core Collection. The top 100 cited articles, published from 1999 to 2017, were cited 38,855 times in total. Researchers from the USA contributed the most publications. The number of articles included in 'Experimental studies (EG)', 'Clinical studies (CS)', 'Epidemiological studies (ES)', and 'Pathological and pathophysiological studies (PP)' were 65, 26, 7, and 2, respectively. Among the 15 topics, the most popular topic is the renin-angiotensin-aldosterone system (RAAS), occurring in 26 articles, including 6 of the top 10 most cited articles. The evolution of topics reveals that the role of RAAS inhibitor is a continuous hotspot, and sodium-glucose cotransporter 2 (SGLT-2) inhibitor and glucagon-like peptide 1 (GLP-1) agonist are two renoprotective agents which represent novel therapeutic methods in DKD. In addition, the 26 clinical studies among the top 100 most cited articles were highlighted, as they help guide clinical practice to better serve patients.

CONCLUSIONS

This bibliometric analysis of the top 100 most cited articles revealed important studies, popular topics, and trends in DKD research to assist researchers in further understanding the subject.