1
|
Cazzaniga G, Rossi M, Eccher A, Girolami I, L'Imperio V, Van Nguyen H, Becker JU, Bueno García MG, Sbaraglia M, Dei Tos AP, Gambaro G, Pagni F. Time for a full digital approach in nephropathology: a systematic review of current artificial intelligence applications and future directions. J Nephrol 2024; 37:65-76. [PMID: 37768550 PMCID: PMC10920416 DOI: 10.1007/s40620-023-01775-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Accepted: 08/22/2023] [Indexed: 09/29/2023]
Abstract
INTRODUCTION Artificial intelligence (AI) integration in nephropathology has been growing rapidly in recent years, facing several challenges including the wide range of histological techniques used, the low occurrence of certain diseases, and the need for data sharing. This narrative review retraces the history of AI in nephropathology and provides insights into potential future developments. METHODS Electronic searches in PubMed-MEDLINE and Embase were made to extract pertinent articles from the literature. Works about automated image analysis or the application of an AI algorithm on non-neoplastic kidney histological samples were included and analyzed to extract information such as publication year, AI task, and learning type. Prepublication servers and reviews were not included. RESULTS Seventy-six (76) original research articles were selected. Most of the studies were conducted in the United States in the last 7 years. To date, research has been mainly conducted on relatively easy tasks, like single-stain glomerular segmentation. However, there is a trend towards developing more complex tasks such as glomerular multi-stain classification. CONCLUSION Deep learning has been used to identify patterns in complex histopathology data and looks promising for the comprehensive assessment of renal biopsy, through the use of multiple stains and virtual staining techniques. Hybrid and collaborative learning approaches have also been explored to utilize large amounts of unlabeled data. A diverse team of experts, including nephropathologists, computer scientists, and clinicians, is crucial for the development of AI systems for nephropathology. Collaborative efforts among multidisciplinary experts result in clinically relevant and effective AI tools.
Collapse
Affiliation(s)
- Giorgio Cazzaniga
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy.
| | - Mattia Rossi
- Division of Nephrology, Department of Medicine, University of Verona, Piazzale Aristide Stefani, 1, 37126, Verona, Italy
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy
- Department of Medical and Surgical Sciences for Children and Adults, University of Modena and Reggio Emilia, University Hospital of Modena, Modena, Italy
| | - Ilaria Girolami
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy
| | - Vincenzo L'Imperio
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy
| | - Hien Van Nguyen
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77004, USA
| | - Jan Ulrich Becker
- Institute of Pathology, University Hospital of Cologne, Cologne, Germany
| | - María Gloria Bueno García
- VISILAB Research Group, E.T.S. Ingenieros Industriales, University of Castilla-La Mancha, Ciudad Real, Spain
| | - Marta Sbaraglia
- Department of Pathology, Azienda Ospedale-Università Padova, Padua, Italy
- Department of Medicine, University of Padua School of Medicine, Padua, Italy
| | - Angelo Paolo Dei Tos
- Department of Pathology, Azienda Ospedale-Università Padova, Padua, Italy
- Department of Medicine, University of Padua School of Medicine, Padua, Italy
| | - Giovanni Gambaro
- Division of Nephrology, Department of Medicine, University of Verona, Piazzale Aristide Stefani, 1, 37126, Verona, Italy
| | - Fabio Pagni
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy
| |
Collapse
|
2
|
Yao T, Lu Y, Long J, Jha A, Zhu Z, Asad Z, Yang H, Fogo AB, Huo Y. Glo-In-One: holistic glomerular detection, segmentation, and lesion characterization with large-scale web image mining. J Med Imaging (Bellingham) 2022; 9:052408. [PMID: 35747553 DOI: 10.1117/1.jmi.9.5.052408] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2021] [Accepted: 05/31/2022] [Indexed: 11/14/2022] Open
Abstract
Purpose: The quantitative detection, segmentation, and characterization of glomeruli from high-resolution whole slide imaging (WSI) play essential roles in the computer-assisted diagnosis and scientific research in digital renal pathology. Historically, such comprehensive quantification requires extensive programming skills to be able to handle heterogeneous and customized computational tools. To bridge the gap of performing glomerular quantification for non-technical users, we develop the Glo-In-One toolkit to achieve holistic glomerular detection, segmentation, and characterization via a single line of command. Additionally, we release a large-scale collection of 30,000 unlabeled glomerular images to further facilitate the algorithmic development of self-supervised deep learning. Approach: The inputs of the Glo-In-One toolkit are WSIs, while the outputs are (1) WSI-level multi-class circle glomerular detection results (which can be directly manipulated with ImageScope), (2) glomerular image patches with segmentation masks, and (3) different lesion types. In the current version, the fine-grained global glomerulosclerosis (GGS) characterization is provided, including assessed-solidified-GSS (associated with hypertension-related injury), disappearing-GSS (a further end result of the SGGS becoming contiguous with fibrotic interstitium), and obsolescent-GSS (nonspecific GGS increasing with aging) glomeruli. To leverage the performance of the Glo-In-One toolkit, we introduce self-supervised deep learning to glomerular quantification via large-scale web image mining. Results: The GGS fine-grained classification model achieved a decent performance compared with baseline supervised methods while only using 10% of the annotated data. The glomerular detection achieved an average precision of 0.627 with circle representations, while the glomerular segmentation achieved a 0.955 patch-wise Dice dimilarity coefficient. Conclusion: We develop and release an open-source Glo-In-One toolkit, a software with holistic glomerular detection, segmentation, and lesion characterization. This toolkit is user-friendly to non-technical users via a single line of command. The toolbox and the 30,000 web mined glomerular images have been made publicly available at https://github.com/hrlblab/Glo-In-One.
Collapse
Affiliation(s)
- Tianyuan Yao
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Yuzhe Lu
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Jun Long
- Central South University, Big Data Institute, Changsha, China
| | - Aadarsh Jha
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Zheyu Zhu
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Zuhayr Asad
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| | - Haichun Yang
- Vanderbilt University Medical Center, Department of Pathology, Microbiology and Immunology, Nashville, Tennessee, United States
| | - Agnes B Fogo
- Vanderbilt University Medical Center, Department of Pathology, Microbiology and Immunology, Nashville, Tennessee, United States
| | - Yuankai Huo
- Vanderbilt University, Department of Computer Science, Nashville, Tennessee, United States
| |
Collapse
|
3
|
Girolami I, Pantanowitz L, Marletta S, Hermsen M, van der Laak J, Munari E, Furian L, Vistoli F, Zaza G, Cardillo M, Gesualdo L, Gambaro G, Eccher A. Artificial intelligence applications for pre-implantation kidney biopsy pathology practice: a systematic review. J Nephrol 2022; 35:1801-1808. [PMID: 35441256 PMCID: PMC9458558 DOI: 10.1007/s40620-022-01327-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2022] [Accepted: 03/28/2022] [Indexed: 10/29/2022]
Abstract
BACKGROUND Transplant nephropathology is a highly specialized field of pathology comprising both the evaluation of organ donor biopsy for organ allocation and post-transplant graft biopsy for assessment of rejection or graft damage. The introduction of digital pathology with whole-slide imaging (WSI) in clinical research, trials and practice has catalyzed the application of artificial intelligence (AI) for histopathology, with development of novel machine-learning models for tissue interrogation and discovery. We aimed to review the literature for studies specifically applying AI algorithms to WSI-digitized pre-implantation kidney biopsy. METHODS A systematic search was carried out in the electronic databases PubMed-MEDLINE and Embase until 25th September, 2021 with a combination of the key terms "kidney", "biopsy", "transplantation" and "artificial intelligence" and their aliases. Studies dealing with the application of AI algorithms coupled with WSI in pre-implantation kidney biopsies were included. The main theme addressed was detection and quantification of tissue components. Extracted data were: author, year and country of the study, type of biopsy features investigated, number of cases, type of algorithm deployed, main results of the study in terms of diagnostic outcome, and the main limitations of the study. RESULTS Of 5761 retrieved articles, 7 met our inclusion criteria. All studies focused largely on AI-based detection and classification of glomerular structures and to a lesser extent on tubular and vascular structures. Performance of AI algorithms was excellent and promising. CONCLUSION All studies highlighted the importance of expert pathologist annotation to reliably train models and the need to acknowledge clinical nuances of the pre-implantation setting. Close cooperation between computer scientists and practicing as well as expert renal pathologists is needed, helping to refine the performance of AI-based models for routine pre-implantation kidney biopsy clinical practice.
Collapse
Affiliation(s)
- Ilaria Girolami
- Division of Pathology, Central Hospital Bolzano, Bolzano, Italy
| | - Liron Pantanowitz
- Department of Pathology and Clinical Labs, University of Michigan, Ann Arbor, MI, USA
| | - Stefano Marletta
- Department of Diagnostics and Public Health, University and Hospital Trust of Verona, Verona, Italy
| | - Meyke Hermsen
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Enrico Munari
- Pathology Unit, Department of Molecular and Translational Medicine, Spedali Civili-University of Brescia, Brescia, Italy
| | - Lucrezia Furian
- Department of Surgical, Oncological and Gastroenterological Sciences, Unit of Kidney and Pancreas Transplantation, University of Padua, Padua, Italy
| | - Fabio Vistoli
- Division of General and Transplant Surgery, University of Pisa, Pisa, Italy
| | - Gianluigi Zaza
- Department of Nephro-Urology, Nephrology, Dialysis and Transplant Unit, University of Foggia, Foggia, Italy
| | | | - Loreto Gesualdo
- Nephrology, Dialysis, and Transplantation Unit, Department of Emergency and Organ Transplantation, University of Bari Aldo Moro, Bari, Italy
| | - Giovanni Gambaro
- Department of General Medicine, Renal Unit, University and Hospital Trust of Verona, Verona, Italy
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy.
| |
Collapse
|
4
|
Nguyen EH, Yang H, Deng R, Lu Y, Zhu Z, Roland JT, Lu L, Landman BA, Fogo AB, Huo Y. Circle Representation for Medical Object Detection. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:746-754. [PMID: 34699352 PMCID: PMC8963364 DOI: 10.1109/tmi.2021.3122835] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Box representation has been extensively used for object detection in computer vision. Such representation is efficacious but not necessarily optimized for biomedical objects (e.g., glomeruli), which play an essential role in renal pathology. In this paper, we propose a simple circle representation for medical object detection and introduce CircleNet, an anchor-free detection framework. Compared with the conventional bounding box representation, the proposed bounding circle representation innovates in three-fold: (1) it is optimized for ball-shaped biomedical objects; (2) The circle representation reduced the degree of freedom compared with box representation; (3) It is naturally more rotation invariant. When detecting glomeruli and nuclei on pathological images, the proposed circle representation achieved superior detection performance and be more rotation-invariant, compared with the bounding box. The code has been made publicly available: https://github.com/hrlblab/CircleNet.
Collapse
|
5
|
Yang CK, Lee CY, Wang HS, Huang SC, Liang PI, Chen JS, Kuo CF, Tu KH, Yeh CY, Chen TD. Glomerular Disease Classification and Lesion Identification by Machine Learning. Biomed J 2021; 45:675-685. [PMID: 34506971 PMCID: PMC9486238 DOI: 10.1016/j.bj.2021.08.011] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Revised: 06/21/2021] [Accepted: 08/31/2021] [Indexed: 12/13/2022] Open
Abstract
Background Classification of glomerular diseases and identification of glomerular lesions require careful morphological examination by experienced nephropathologists, which is labor-intensive, time-consuming, and prone to interobserver variability. In this regard, recent advance in machine learning-based image analysis is promising. Methods We combined Mask Region-based Convolutional Neural Networks (Mask R–CNN) with an additional classification step to build a glomerulus detection model using human kidney biopsy samples. A Long Short-Term Memory (LSTM) recurrent neural network was applied for glomerular disease classification, and another two-stage model using ResNeXt-101 was constructed for glomerular lesion identification in cases of lupus nephritis. Results The detection model showed state-of-the-art performance on variedly stained slides with F1 scores up to 0.944. The disease classification model showed good accuracies up to 0.940 on recognizing different glomerular diseases based on H&E whole slide images. The lesion identification model demonstrated high discriminating power with area under the receiver operating characteristic curve up to 0.947 for various glomerular lesions. Models showed good generalization on external testing datasets. Conclusion This study is the first-of-its-kind showing how each step of kidney biopsy interpretation carried out by nephropathologists can be captured and simulated by machine learning models. The models were integrated into a whole slide image viewing and annotating platform to enable nephropathologists to review, correct, and confirm the inference results. Further improvement on model performances and incorporating inputs from immunofluorescence, electron microscopy, and clinical data might realize actual clinical use.
Collapse
Affiliation(s)
- Cheng-Kun Yang
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Ching-Yi Lee
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Hsiang-Sheng Wang
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Shun-Chen Huang
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Kaohsiung Branch, No. 123, Dapi Rd., Niaosong Dist., Kaohsiung City 833, Taiwan.
| | - Peir-In Liang
- Department of Pathology, Kaohsiung Medical University Hospital, No. 100, Ziyou 1st Rd., Sanmin Dist., Kaohsiung City 807, Taiwan.
| | - Jung-Sheng Chen
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Chang-Fu Kuo
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Kun-Hua Tu
- Department of Nephrology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Chao-Yuan Yeh
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Tai-Di Chen
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| |
Collapse
|
6
|
Shubham S, Jain N, Gupta V, Mohan S, Ariffin MM, Ahmadian A. Identify glomeruli in human kidney tissue images using a deep learning approach. Soft comput 2021. [DOI: 10.1007/s00500-021-06143-z] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
7
|
Huo Y, Deng R, Liu Q, Fogo AB, Yang H. AI applications in renal pathology. Kidney Int 2021; 99:1309-1320. [PMID: 33581198 PMCID: PMC8154730 DOI: 10.1016/j.kint.2021.01.015] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2020] [Revised: 01/09/2021] [Accepted: 01/13/2021] [Indexed: 12/20/2022]
Abstract
The explosive growth of artificial intelligence (AI) technologies, especially deep learning methods, has been translated at revolutionary speed to efforts in AI-assisted healthcare. New applications of AI to renal pathology have recently become available, driven by the successful AI deployments in digital pathology. However, synergetic developments of renal pathology and AI require close interdisciplinary collaborations between computer scientists and renal pathologists. Computer scientists should understand that not every AI innovation is translatable to renal pathology, while renal pathologists should capture high-level principles of the relevant AI technologies. Herein, we provide an integrated review on current and possible future applications in AI-assisted renal pathology, by including perspectives from computer scientists and renal pathologists. First, the standard stages, from data collection to analysis, in full-stack AI-assisted renal pathology studies are reviewed. Second, representative renal pathology-optimized AI techniques are introduced. Last, we review current clinical AI applications, as well as promising future applications with the recent advances in AI.
Collapse
Affiliation(s)
- Yuankai Huo
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| | - Ruining Deng
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| | - Quan Liu
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| | - Agnes B Fogo
- Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Haichun Yang
- Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center, Nashville, Tennessee, USA.
| |
Collapse
|
8
|
Lutnick B, Kammardi Shashiprakash A, Manthey D, Sarder P. User friendly, cloud based, whole slide image segmentation. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11603. [PMID: 34366542 DOI: 10.1117/12.2581383] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
Convolutional neural networks, the state of the art for image segmentation, have been successfully applied to histology images by many computational researchers. However, the translatability of this technology to clinicians and biological researchers is limited due to the complex and undeveloped user interface of the code, as well as the extensive computer setup required. We have developed a plugin for segmentation of whole slide images (WSIs) with an easy to use graphical user interface. This plugin runs a state-of-the-art convolutional neural network for segmentation of WSIs in the cloud. Our plugin is built on the open source tool HistomicsTK by Kitware Inc. (Clifton Park, NY), which provides remote data management and viewing abilities for WSI datasets. The ability to access this tool over the internet will facilitate widespread use by computational non-experts. Users can easily upload slides to a server where our plugin is installed and perform the segmentation analysis remotely. This plugin is open source and once trained, has the ability to be applied to the segmentation of any pathological structure. For a proof of concept, we have trained it to segment glomeruli from renal tissue images, demonstrating it on holdout tissue slides.
Collapse
Affiliation(s)
- Brendon Lutnick
- Department of Pathology and Anatomical Sciences, SUNY Buffalo
| | | | | | - Pinaki Sarder
- Department of Pathology and Anatomical Sciences, SUNY Buffalo
| |
Collapse
|
9
|
Abstract
PURPOSE OF REVIEW Successful integration of artificial intelligence into extant clinical workflows is contingent upon a number of factors including clinician comprehension and interpretation of computer vision. This article discusses how image analysis and machine learning have enabled comprehensive characterization of kidney morphology for development of automated diagnostic and prognostic renal pathology applications. RECENT FINDINGS The primordial digital pathology informatics work employed classical image analysis and machine learning to prognosticate renal disease. Although this classical approach demonstrated tremendous potential, subsequent advancements in hardware technology rendered artificial neural networks '(ANNs) the method of choice for machine vision in computational pathology'. Offering rapid and reproducible detection, characterization and classification of kidney morphology, ANNs have facilitated the development of diagnostic and prognostic applications. In addition, modern machine learning with ANNs has revealed novel biomarkers in kidney disease, demonstrating the potential for machine vision to elucidate novel pathologic mechanisms beyond extant clinical knowledge. SUMMARY Despite the revolutionary developments potentiated by modern machine learning, several challenges remain, including data quality control and curation, image annotation and ontology, integration of multimodal data and interpretation of machine vision or 'opening the black box'. Resolution of these challenges will not only revolutionize diagnostic pathology but also pave the way for precision medicine and integration of artificial intelligence in the process of care.
Collapse
|
10
|
Yang H, Deng R, Lu Y, Zhu Z, Chen Y, Roland JT, Lu L, Landman BA, Fogo AB, Huo Y. CircleNet: Anchor-free Glomerulus Detection with Circle Representation. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2020; 2020:35-44. [PMID: 34414404 PMCID: PMC8372751 DOI: 10.1007/978-3-030-59719-1_4] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/03/2022]
Abstract
Object detection networks are powerful in computer vision, but not necessarily optimized for biomedical object detection. In this work, we propose CircleNet, a simple anchor-free detection method with circle representation for detection of the ball-shaped glomerulus. Different from the traditional bounding box based detection method, the bounding circle (1) reduces the degrees of freedom of detection representation, (2) is naturally rotation invariant, (3) and optimized for ball-shaped objects. The key innovation to enable this representation is the anchor-free framework with the circle detection head. We evaluate CircleNet in the context of detection of glomerulus. CircleNet increases average precision of the glomerulus detection from 0.598 to 0.647. Another key advantage is that CircleNet achieves better rotation consistency compared with bounding box representations.
Collapse
Affiliation(s)
- Haichun Yang
- Vanderbilt University Medical Center, Nashville TN 37215, USA
| | | | - Yuzhe Lu
- Vanderbilt University, Nashville TN 37215, USA
| | - Zheyu Zhu
- Vanderbilt University, Nashville TN 37215, USA
| | - Ye Chen
- Vanderbilt University, Nashville TN 37215, USA
| | - Joseph T Roland
- Vanderbilt University Medical Center, Nashville TN 37215, USA
| | - Le Lu
- PAII Inc., Bethesda MD 20817, USA
| | | | - Agnes B Fogo
- Vanderbilt University Medical Center, Nashville TN 37215, USA
| | - Yuankai Huo
- Vanderbilt University, Nashville TN 37215, USA
| |
Collapse
|
11
|
Lutnick B, Ginley B, Jen KY, Dong W, Sarder P. Generative modeling for label-free glomerular modeling and classification. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2020; 11320:1132007. [PMID: 32362708 PMCID: PMC7194217] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Generative modeling using GANs has gained traction in machine learning literature, as training does not require labeled datasets. This is perfect for applications in biological datasets, where large labeled datasets are often difficult and expensive to acquire. However, generative models offer no easy way to encode real images into feature-sets, something that is desirable for network explainability and may yield potentially informative image features. For this reason, we test a VAE-GAN architecture for label-free modeling of glomerular structural features. We show that this network can generate realistic looking synthetic images, and be used to interpolate between images. To prove the biological relevance of the network encodings, we classify small-labeled sets of encoded glomeruli by biopsy Tervaert class and for the presence of sclerosis, obtaining a Cohen's kappa values of 0.87 and 0.78 respectfully.
Collapse
Affiliation(s)
- Brendon Lutnick
- Department of Pathology and Anatomical Sciences, SUNY
Buffalo
| | - Brandon Ginley
- Department of Pathology and Anatomical Sciences, SUNY
Buffalo
| | - Kuang-Yu Jen
- Department of Pathology, University of California at
Davis
| | - Wen Dong
- Department of Computer Science and Engineering, SUNY
Buffalo
| | - Pinaki Sarder
- Department of Pathology and Anatomical Sciences, SUNY
Buffalo
| |
Collapse
|
12
|
Hermsen M, de Bel T, den Boer M, Steenbergen EJ, Kers J, Florquin S, Roelofs JJTH, Stegall MD, Alexander MP, Smith BH, Smeets B, Hilbrands LB, van der Laak JAWM. Deep Learning-Based Histopathologic Assessment of Kidney Tissue. J Am Soc Nephrol 2019; 30:1968-1979. [PMID: 31488607 DOI: 10.1681/asn.2019020144] [Citation(s) in RCA: 192] [Impact Index Per Article: 38.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2019] [Accepted: 07/01/2019] [Indexed: 01/15/2023] Open
Abstract
BACKGROUND The development of deep neural networks is facilitating more advanced digital analysis of histopathologic images. We trained a convolutional neural network for multiclass segmentation of digitized kidney tissue sections stained with periodic acid-Schiff (PAS). METHODS We trained the network using multiclass annotations from 40 whole-slide images of stained kidney transplant biopsies and applied it to four independent data sets. We assessed multiclass segmentation performance by calculating Dice coefficients for ten tissue classes on ten transplant biopsies from the Radboud University Medical Center in Nijmegen, The Netherlands, and on ten transplant biopsies from an external center for validation. We also fully segmented 15 nephrectomy samples and calculated the network's glomerular detection rates and compared network-based measures with visually scored histologic components (Banff classification) in 82 kidney transplant biopsies. RESULTS The weighted mean Dice coefficients of all classes were 0.80 and 0.84 in ten kidney transplant biopsies from the Radboud center and the external center, respectively. The best segmented class was "glomeruli" in both data sets (Dice coefficients, 0.95 and 0.94, respectively), followed by "tubuli combined" and "interstitium." The network detected 92.7% of all glomeruli in nephrectomy samples, with 10.4% false positives. In whole transplant biopsies, the mean intraclass correlation coefficient for glomerular counting performed by pathologists versus the network was 0.94. We found significant correlations between visually scored histologic components and network-based measures. CONCLUSIONS This study presents the first convolutional neural network for multiclass segmentation of PAS-stained nephrectomy samples and transplant biopsies. Our network may have utility for quantitative studies involving kidney histopathology across centers and provide opportunities for deep learning applications in routine diagnostics.
Collapse
Affiliation(s)
| | | | | | | | - Jesper Kers
- Department of Pathology, Amsterdam Infection & Immunity, Amsterdam Cardiovascular Sciences, Amsterdam UMC, and.,Center for Analytical Sciences Amsterdam, Van 't Hoff Institute for Molecular Sciences, University of Amsterdam, Amsterdam, The Netherlands.,The Ragon Institute of the Massachusetts General Hospital, Massachusetts Institute of Technology and Harvard University, Cambridge, Massachusetts
| | - Sandrine Florquin
- Department of Pathology, Amsterdam Infection & Immunity, Amsterdam Cardiovascular Sciences, Amsterdam UMC, and
| | - Joris J T H Roelofs
- Department of Pathology, Amsterdam Infection & Immunity, Amsterdam Cardiovascular Sciences, Amsterdam UMC, and
| | - Mark D Stegall
- Divisions of Transplantation surgery.,William J. von Liebig Center for Transplantation and Clinical Regeneration, Mayo Clinic, Rochester, Minnesota; and
| | - Mariam P Alexander
- William J. von Liebig Center for Transplantation and Clinical Regeneration, Mayo Clinic, Rochester, Minnesota; and.,Pathology, and
| | - Byron H Smith
- William J. von Liebig Center for Transplantation and Clinical Regeneration, Mayo Clinic, Rochester, Minnesota; and.,Biomedical Statistics and Informatics, and
| | | | - Luuk B Hilbrands
- Nephrology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeroen A W M van der Laak
- Departments of Pathology and .,Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| |
Collapse
|
13
|
Majumdar A, Jen KY, Jain S, Tomaszewski JE, Sarder P. Examining Structural Patterns and Causality in Diabetic Nephropathy using inter-Glomerular Distance and Bayesian Graphical Models. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2019; 10956:1095608. [PMID: 31186597 PMCID: PMC6557453 DOI: 10.1117/12.2513598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
In diabetic nephropathy (DN), hyperglycemia drives a progressive thickening of glomerular filtration surfaces, increased cell proliferation as well as mesangial expansion and a constriction of capillary lumens. This leads to progressive structural changes inside the Glomeruli. In this work, we make a study of structural glomerular changes in DN from a graph-theoretic standpoint, using features extracted from Minimal Spanning Trees (MSTs) constructed over intercellular distances in order to classify the "packing signatures" of different DN stages. We further investigate the significance of the competing effects of Volume change measured here in 2Dimensional Pixel span area (Area) on one hand and increased cell proliferation on the other in determining the packing patterns. Towards that we formulate the problem as Dynamic Bayesian Network (DBN). From our preliminary results we do postulate that volume expansion caused by internal pressure as capillary lumens constriction has perhaps has a greater effect in the early stages.
Collapse
Affiliation(s)
- Aurijoy Majumdar
- Departments of Pathology and Anatomical Sciences, University at Buffalo
| | - Kuang-Yu Jen
- Departments of Pathology, University at California at Davis
| | - Sanjay Jain
- Department of Medicine, Washington University School of Medicine in St. Louis
| | | | - Pinaki Sarder
- Departments of Pathology and Anatomical Sciences, University at Buffalo
| |
Collapse
|
14
|
Marsh JN, Matlock MK, Kudose S, Liu TC, Stappenbeck TS, Gaut JP, Swamidass SJ. Deep Learning Global Glomerulosclerosis in Transplant Kidney Frozen Sections. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:2718-2728. [PMID: 29994669 PMCID: PMC6296264 DOI: 10.1109/tmi.2018.2851150] [Citation(s) in RCA: 91] [Impact Index Per Article: 15.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Transplantable kidneys are in very limited supply. Accurate viability assessment prior to transplantation could minimize organ discard. Rapid and accurate evaluation of intra-operative donor kidney biopsies is essential for determining which kidneys are eligible for transplantation. The criterion for accepting or rejecting donor kidneys relies heavily on pathologist determination of the percent of glomeruli (determined from a frozen section) that are normal and sclerotic. This percentage is a critical measurement that correlates with transplant outcome. Inter- and intra-observer variability in donor biopsy evaluation is, however, significant. An automated method for determination of percent global glomerulosclerosis could prove useful in decreasing evaluation variability, increasing throughput, and easing the burden on pathologists. Here, we describe the development of a deep learning model that identifies and classifies non-sclerosed and sclerosed glomeruli in whole-slide images of donor kidney frozen section biopsies. This model extends a convolutional neural network (CNN) pre-trained on a large database of digital images. The extended model, when trained on just 48 whole slide images, exhibits slide-level evaluation performance on par with expert renal pathologists. Encouragingly, the model's performance is robust to slide preparation artifacts associated with frozen section preparation. The model substantially outperforms a model trained on image patches of isolated glomeruli, in terms of both accuracy and speed. The methodology overcomes the technical challenge of applying a pretrained CNN bottleneck model to whole-slide image classification. The traditional patch-based approach, while exhibiting deceptively good performance classifying isolated patches, does not translate successfully to whole-slide image segmentation in this setting. As the first model reported that identifies and classifies normal and sclerotic glomeruli in frozen kidney sections, and thus the first model reported in the literature relevant to kidney transplantation, it may become an essential part of donor kidney biopsy evaluation in the clinical setting.
Collapse
|
15
|
Digital pathology in nephrology clinical trials, research, and pathology practice. Curr Opin Nephrol Hypertens 2018; 26:450-459. [PMID: 28858910 DOI: 10.1097/mnh.0000000000000360] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
PURPOSE OF REVIEW In this review, we will discuss (i) how the recent advancements in digital technology and computational engineering are currently applied to nephropathology in the setting of clinical research, trials, and practice; (ii) the benefits of the new digital environment; (iii) how recognizing its challenges provides opportunities for transformation; and (iv) nephropathology in the upcoming era of kidney precision and predictive medicine. RECENT FINDINGS Recent studies highlighted how new standardized protocols facilitate the harmonization of digital pathology database infrastructure and morphologic, morphometric, and computer-aided quantitative analyses. Digital pathology enables robust protocols for clinical trials and research, with the potential to identify previously underused or unrecognized clinically useful parameters. The integration of digital pathology with molecular signatures is leading the way to establishing clinically relevant morpho-omic taxonomies of renal diseases. SUMMARY The introduction of digital pathology in clinical research and trials, and the progressive implementation of the modern software ecosystem, opens opportunities for the development of new predictive diagnostic paradigms and computer-aided algorithms, transforming the practice of renal disease into a modern computational science.
Collapse
|
16
|
Simon O, Yacoub R, Jain S, Tomaszewski JE, Sarder P. Multi-radial LBP Features as a Tool for Rapid Glomerular Detection and Assessment in Whole Slide Histopathology Images. Sci Rep 2018; 8:2032. [PMID: 29391542 PMCID: PMC5795004 DOI: 10.1038/s41598-018-20453-7] [Citation(s) in RCA: 47] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2017] [Accepted: 01/18/2018] [Indexed: 12/03/2022] Open
Abstract
We demonstrate a simple and effective automated method for the localization of glomeruli in large (~1 gigapixel) histopathological whole-slide images (WSIs) of thin renal tissue sections and biopsies, using an adaptation of the well-known local binary patterns (LBP) image feature vector to train a support vector machine (SVM) model. Our method offers high precision (>90%) and reasonable recall (>70%) for glomeruli from WSIs, is readily adaptable to glomeruli from multiple species, including mouse, rat, and human, and is robust to diverse slide staining methods. Using 5 Intel(R) Core(TM) i7-4790 CPUs with 40 GB RAM, our method typically requires ~15 sec for training and ~2 min to extract glomeruli reproducibly from a WSI. Deploying a deep convolutional neural network trained for glomerular recognition in tandem with the SVM suffices to reduce false positives to below 3%. We also apply our LBP-based descriptor to successfully detect pathologic changes in a mouse model of diabetic nephropathy. We envision potential clinical and laboratory applications for this approach in the study and diagnosis of glomerular disease, and as a means of greatly accelerating the construction of feature sets to fuel deep learning studies into tissue structure and pathology.
Collapse
Affiliation(s)
- Olivier Simon
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, USA
| | - Rabi Yacoub
- Division of Nephrology, Department of Medicine, University at Buffalo, Buffalo, USA
| | - Sanjay Jain
- Division of Nephrology, Department of Medicine, Washington University School of Medicine, St. Louis, USA
| | - John E Tomaszewski
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, USA
| | - Pinaki Sarder
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, USA.
| |
Collapse
|
17
|
|