1
|
Hosseini MS, Bejnordi BE, Trinh VQH, Chan L, Hasan D, Li X, Yang S, Kim T, Zhang H, Wu T, Chinniah K, Maghsoudlou S, Zhang R, Zhu J, Khaki S, Buin A, Chaji F, Salehi A, Nguyen BN, Samaras D, Plataniotis KN. Computational pathology: A survey review and the way forward. J Pathol Inform 2024; 15:100357. [PMID: 38420608 PMCID: PMC10900832 DOI: 10.1016/j.jpi.2023.100357] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/21/2023] [Accepted: 12/23/2023] [Indexed: 03/02/2024] Open
Abstract
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Collapse
Affiliation(s)
- Mahdi S Hosseini
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | | | - Vincent Quoc-Huy Trinh
- Institute for Research in Immunology and Cancer of the University of Montreal, Montreal, QC H3T 1J4, Canada
| | - Lyndon Chan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Danial Hasan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Xingwen Li
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Stephen Yang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Taehyo Kim
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Haochen Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Theodore Wu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Kajanan Chinniah
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Sina Maghsoudlou
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ryan Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Jiadai Zhu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Samir Khaki
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Andrei Buin
- Huron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada
| | - Fatemeh Chaji
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ala Salehi
- Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada
| | - Bich Ngoc Nguyen
- University of Montreal Hospital Center, Montreal, QC H2X 0C2, Canada
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, United States
| | - Konstantinos N Plataniotis
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| |
Collapse
|
2
|
Pilva P, Bülow R, Boor P. Deep learning applications for kidney histology analysis. Curr Opin Nephrol Hypertens 2024; 33:291-297. [PMID: 38411024 DOI: 10.1097/mnh.0000000000000973] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/28/2024]
Abstract
PURPOSE OF REVIEW Nephropathology is increasingly incorporating computational methods to enhance research and diagnostic accuracy. The widespread adoption of digital pathology, coupled with advancements in deep learning, will likely transform our pathology practices. Here, we discuss basic concepts of deep learning, recent applications in nephropathology, current challenges in implementation and future perspectives. RECENT FINDINGS Deep learning models have been developed and tested in various areas of nephropathology, for example, predicting kidney disease progression or diagnosing diseases based on imaging and clinical data. Despite their promising potential, challenges remain that hinder a wider adoption, for example, the lack of prospective evidence and testing in real-world scenarios. SUMMARY Deep learning offers great opportunities to improve quantitative and qualitative kidney histology analysis for research and clinical nephropathology diagnostics. Although exciting approaches already exist, the potential of deep learning in nephropathology is only at its beginning and we can expect much more to come.
Collapse
Affiliation(s)
| | | | - Peter Boor
- Institute of Pathology
- Department of Nephrology and Clinical Immunology, RWTH Aachen University Hospital, Aachen, Germany
| |
Collapse
|
3
|
Nguyen TTU, Nguyen AT, Kim H, Jung YJ, Park W, Kim KM, Park I, Kim W. Deep-learning model for evaluating histopathology of acute renal tubular injury. Sci Rep 2024; 14:9010. [PMID: 38637573 PMCID: PMC11026462 DOI: 10.1038/s41598-024-58506-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 03/30/2024] [Indexed: 04/20/2024] Open
Abstract
Tubular injury is the most common cause of acute kidney injury. Histopathological diagnosis may help distinguish between the different types of acute kidney injury and aid in treatment. To date, a limited number of study has used deep-learning models to assist in the histopathological diagnosis of acute kidney injury. This study aimed to perform histopathological segmentation to identify the four structures of acute renal tubular injury using deep-learning models. A segmentation model was used to classify tubule-specific injuries following cisplatin treatment. A total of 45 whole-slide images with 400 generated patches were used in the segmentation model, and 27,478 annotations were created for four classes: glomerulus, healthy tubules, necrotic tubules, and tubules with casts. A segmentation model was developed using the DeepLabV3 architecture with a MobileNetv3-Large backbone to accurately identify the four histopathological structures associated with acute renal tubular injury in PAS-stained mouse samples. In the segmentation model for four structures, the highest Intersection over Union and the Dice coefficient were obtained for the segmentation of the "glomerulus" class, followed by "necrotic tubules," "healthy tubules," and "tubules with cast" classes. The overall performance of the segmentation algorithm for all classes in the test set included an Intersection over Union of 0.7968 and a Dice coefficient of 0.8772. The Dice scores for the glomerulus, healthy tubules, necrotic tubules, and tubules with cast are 91.78 ± 11.09, 87.37 ± 4.02, 88.08 ± 6.83, and 83.64 ± 20.39%, respectively. The utilization of deep learning in a predictive model has demonstrated promising performance in accurately identifying the degree of injured renal tubules. These results may provide new opportunities for the application of the proposed methods to evaluate renal pathology more effectively.
Collapse
Affiliation(s)
- Thi Thuy Uyen Nguyen
- Department of Histology, Embryology, Pathology and Forensic Medicine, Hue University of Medicine and Pharmacy, Hue University, Hue City, Vietnam
| | - Anh-Tien Nguyen
- Department of Radiology, Chonnam National University and Hospital, Gwangju, Korea
- Department of Medical Informatics, University Medical Center Göttingen, Göttingen, Germany
| | - Hyeongwan Kim
- Department of Internal Medicine, Jeonbuk National University Medical School, Jeonju, Republic of Korea
- Research Institute of Clinical Medicine of Jeonbuk National University-Biomedical Research Institute, Jeonju, Republic of Korea
| | - Yu Jin Jung
- Department of Internal Medicine, Jeonbuk National University Medical School, Jeonju, Republic of Korea
- Research Institute of Clinical Medicine of Jeonbuk National University-Biomedical Research Institute, Jeonju, Republic of Korea
| | - Woong Park
- Department of Internal Medicine, Jeonbuk National University Medical School, Jeonju, Republic of Korea
- Research Institute of Clinical Medicine of Jeonbuk National University-Biomedical Research Institute, Jeonju, Republic of Korea
| | - Kyoung Min Kim
- Department of Pathology, Jeonbuk National University Medical School, Jeonju, Korea
| | - Ilwoo Park
- Department of Radiology, Chonnam National University and Hospital, Gwangju, Korea.
- Department of Artificial Intelligence Convergence, Chonnam National University, Gwangju, Korea.
- Department of Data Science, Chonnam National University, Gwangju, Korea.
| | - Won Kim
- Department of Internal Medicine, Jeonbuk National University Medical School, Jeonju, Republic of Korea.
- Research Institute of Clinical Medicine of Jeonbuk National University-Biomedical Research Institute, Jeonju, Republic of Korea.
| |
Collapse
|
4
|
Augulis R, Rasmusson A, Laurinaviciene A, Jen KY, Laurinavicius A. Computational pathology model to assess acute and chronic transformations of the tubulointerstitial compartment in renal allograft biopsies. Sci Rep 2024; 14:5345. [PMID: 38438513 PMCID: PMC10912734 DOI: 10.1038/s41598-024-55936-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 02/29/2024] [Indexed: 03/06/2024] Open
Abstract
Managing patients with kidney allografts largely depends on biopsy diagnosis which is based on semiquantitative assessments of rejection features and extent of acute and chronic changes within the renal parenchyma. Current methods lack reproducibility while digital image data-driven computational models enable comprehensive and quantitative assays. In this study we aimed to develop a computational method for automated assessment of histopathology transformations within the tubulointerstitial compartment of the renal cortex. Whole slide images of modified Picrosirius red-stained biopsy slides were used for the training (n = 852) and both internal (n = 172) and external (n = 94) tests datasets. The pipeline utilizes deep learning segmentations of renal tubules, interstitium, and peritubular capillaries from which morphometry features were extracted. Seven indicators were selected for exploring the intrinsic spatial interactions within the tubulointerstitial compartment. A principal component analysis revealed two independent factors which can be interpreted as representing chronic and acute tubulointerstitial injury. A K-means clustering classified biopsies according to potential phenotypes of combined acute and chronic transformations of various degrees. We conclude that multivariate analyses of tubulointerstitial morphometry transformations enable extraction of and quantification of acute and chronic components of injury. The method is developed for renal allograft biopsies; however, the principle can be applied more broadly for kidney pathology assessment.
Collapse
Affiliation(s)
- Renaldas Augulis
- Department of Pathology and Forensic Medicine, Institute of Biomedical Sciences of the Faculty of Medicine, Vilnius University, M. K. Ciurlionio Str. 21, 03101, Vilnius, Lithuania.
- National Centre of Pathology, Vilnius University Hospital Santaros Klinikos, P. Baublio Str. 5, 08406, Vilnius, Lithuania.
| | - Allan Rasmusson
- Department of Pathology and Forensic Medicine, Institute of Biomedical Sciences of the Faculty of Medicine, Vilnius University, M. K. Ciurlionio Str. 21, 03101, Vilnius, Lithuania
- National Centre of Pathology, Vilnius University Hospital Santaros Klinikos, P. Baublio Str. 5, 08406, Vilnius, Lithuania
| | - Aida Laurinaviciene
- Department of Pathology and Forensic Medicine, Institute of Biomedical Sciences of the Faculty of Medicine, Vilnius University, M. K. Ciurlionio Str. 21, 03101, Vilnius, Lithuania
- National Centre of Pathology, Vilnius University Hospital Santaros Klinikos, P. Baublio Str. 5, 08406, Vilnius, Lithuania
| | - Kuang-Yu Jen
- Department of Pathology and Laboratory Medicine, University of California, Davis School of Medicine, Sacramento, CA, USA
| | - Arvydas Laurinavicius
- Department of Pathology and Forensic Medicine, Institute of Biomedical Sciences of the Faculty of Medicine, Vilnius University, M. K. Ciurlionio Str. 21, 03101, Vilnius, Lithuania
- National Centre of Pathology, Vilnius University Hospital Santaros Klinikos, P. Baublio Str. 5, 08406, Vilnius, Lithuania
| |
Collapse
|
5
|
Feng C, Ong K, Young DM, Chen B, Li L, Huo X, Lu H, Gu W, Liu F, Tang H, Zhao M, Yang M, Zhu K, Huang L, Wang Q, Marini GPL, Gui K, Han H, Sanders SJ, Li L, Yu W, Mao J. Artificial intelligence-assisted quantification and assessment of whole slide images for pediatric kidney disease diagnosis. Bioinformatics 2024; 40:btad740. [PMID: 38058211 PMCID: PMC10796177 DOI: 10.1093/bioinformatics/btad740] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Revised: 11/13/2023] [Accepted: 12/05/2023] [Indexed: 12/08/2023] Open
Abstract
MOTIVATION Pediatric kidney disease is a widespread, progressive condition that severely impacts growth and development of children. Chronic kidney disease is often more insidious in children than in adults, usually requiring a renal biopsy for diagnosis. Biopsy evaluation requires copious examination by trained pathologists, which can be tedious and prone to human error. In this study, we propose an artificial intelligence (AI) method to assist pathologists in accurate segmentation and classification of pediatric kidney structures, named as AI-based Pediatric Kidney Diagnosis (APKD). RESULTS We collected 2935 pediatric patients diagnosed with kidney disease for the development of APKD. The dataset comprised 93 932 histological structures annotated manually by three skilled nephropathologists. APKD scored an average accuracy of 94% for each kidney structure category, including 99% in the glomerulus. We found strong correlation between the model and manual detection in detected glomeruli (Spearman correlation coefficient r = 0.98, P < .001; intraclass correlation coefficient ICC = 0.98, 95% CI = 0.96-0.98). Compared to manual detection, APKD was approximately 5.5 times faster in segmenting glomeruli. Finally, we show how the pathological features extracted by APKD can identify focal abnormalities of the glomerular capillary wall to aid in the early diagnosis of pediatric kidney disease. AVAILABILITY AND IMPLEMENTATION https://github.com/ChunyueFeng/Kidney-DataSet.
Collapse
Affiliation(s)
- Chunyue Feng
- Department of Nephrology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou 310000, China
- National Clinical Research Center for Child Health, Hangzhou 310000, China
| | - Kokhaur Ong
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
| | - David M Young
- Institute of Molecular and Cell Biology, A*STAR, Singapore 138673, Singapore
- Department of Psychiatry and Behavioral Sciences, UCSF Weill Institute for Neurosciences, University of California, San Francisco, CA, 94143, United States
| | - Bingxian Chen
- Ningbo Konfoong Bioinformation Tech Co., Ltd., Ningbo 315000, China
| | - Longjie Li
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
| | - Xinmi Huo
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
| | - Haoda Lu
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
- Institute for AI in Medicine, Nanjing University of Information Science and Technology, Nanjing 210044, China
| | - Weizhong Gu
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Fei Liu
- Department of Nephrology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou 310000, China
- National Clinical Research Center for Child Health, Hangzhou 310000, China
| | - Hongfeng Tang
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Manli Zhao
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Min Yang
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Kun Zhu
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Limin Huang
- Department of Nephrology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou 310000, China
- National Clinical Research Center for Child Health, Hangzhou 310000, China
| | - Qiang Wang
- Ningbo Konfoong Bioinformation Tech Co., Ltd., Ningbo 315000, China
| | | | - Kun Gui
- Ningbo Konfoong Bioinformation Tech Co., Ltd., Ningbo 315000, China
| | - Hao Han
- Institute of Molecular and Cell Biology, A*STAR, Singapore 138673, Singapore
| | - Stephan J Sanders
- Department of Psychiatry and Behavioral Sciences, UCSF Weill Institute for Neurosciences, University of California, San Francisco, CA, 94143, United States
| | - Lin Li
- Department of Nephrology, Shanghai Changzheng Hospital, Naval Medical University, Shanghai 200003, China
| | - Weimiao Yu
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
- Institute of Molecular and Cell Biology, A*STAR, Singapore 138673, Singapore
- Institute for AI in Medicine, Nanjing University of Information Science and Technology, Nanjing 210044, China
| | - Jianhua Mao
- Department of Nephrology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou 310000, China
- National Clinical Research Center for Child Health, Hangzhou 310000, China
| |
Collapse
|
6
|
Cazzaniga G, Rossi M, Eccher A, Girolami I, L'Imperio V, Van Nguyen H, Becker JU, Bueno García MG, Sbaraglia M, Dei Tos AP, Gambaro G, Pagni F. Time for a full digital approach in nephropathology: a systematic review of current artificial intelligence applications and future directions. J Nephrol 2024; 37:65-76. [PMID: 37768550 PMCID: PMC10920416 DOI: 10.1007/s40620-023-01775-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Accepted: 08/22/2023] [Indexed: 09/29/2023]
Abstract
INTRODUCTION Artificial intelligence (AI) integration in nephropathology has been growing rapidly in recent years, facing several challenges including the wide range of histological techniques used, the low occurrence of certain diseases, and the need for data sharing. This narrative review retraces the history of AI in nephropathology and provides insights into potential future developments. METHODS Electronic searches in PubMed-MEDLINE and Embase were made to extract pertinent articles from the literature. Works about automated image analysis or the application of an AI algorithm on non-neoplastic kidney histological samples were included and analyzed to extract information such as publication year, AI task, and learning type. Prepublication servers and reviews were not included. RESULTS Seventy-six (76) original research articles were selected. Most of the studies were conducted in the United States in the last 7 years. To date, research has been mainly conducted on relatively easy tasks, like single-stain glomerular segmentation. However, there is a trend towards developing more complex tasks such as glomerular multi-stain classification. CONCLUSION Deep learning has been used to identify patterns in complex histopathology data and looks promising for the comprehensive assessment of renal biopsy, through the use of multiple stains and virtual staining techniques. Hybrid and collaborative learning approaches have also been explored to utilize large amounts of unlabeled data. A diverse team of experts, including nephropathologists, computer scientists, and clinicians, is crucial for the development of AI systems for nephropathology. Collaborative efforts among multidisciplinary experts result in clinically relevant and effective AI tools.
Collapse
Affiliation(s)
- Giorgio Cazzaniga
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy.
| | - Mattia Rossi
- Division of Nephrology, Department of Medicine, University of Verona, Piazzale Aristide Stefani, 1, 37126, Verona, Italy
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy
- Department of Medical and Surgical Sciences for Children and Adults, University of Modena and Reggio Emilia, University Hospital of Modena, Modena, Italy
| | - Ilaria Girolami
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy
| | - Vincenzo L'Imperio
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy
| | - Hien Van Nguyen
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77004, USA
| | - Jan Ulrich Becker
- Institute of Pathology, University Hospital of Cologne, Cologne, Germany
| | - María Gloria Bueno García
- VISILAB Research Group, E.T.S. Ingenieros Industriales, University of Castilla-La Mancha, Ciudad Real, Spain
| | - Marta Sbaraglia
- Department of Pathology, Azienda Ospedale-Università Padova, Padua, Italy
- Department of Medicine, University of Padua School of Medicine, Padua, Italy
| | - Angelo Paolo Dei Tos
- Department of Pathology, Azienda Ospedale-Università Padova, Padua, Italy
- Department of Medicine, University of Padua School of Medicine, Padua, Italy
| | - Giovanni Gambaro
- Division of Nephrology, Department of Medicine, University of Verona, Piazzale Aristide Stefani, 1, 37126, Verona, Italy
| | - Fabio Pagni
- Department of Medicine and Surgery, Pathology, Fondazione IRCCS San Gerardo dei Tintori, Università di Milano-Bicocca, Monza, Italy
| |
Collapse
|
7
|
Huang YL, Liu XQ, Huang Y, Jin FY, Zhao Q, Wu QY, Ma KL. Application of cloud server-based machine learning for assisting pathological structure recognition in IgA nephropathy. J Clin Pathol 2023:jcp-2023-209215. [PMID: 38123970 DOI: 10.1136/jcp-2023-209215] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Accepted: 11/26/2023] [Indexed: 12/23/2023]
Abstract
BACKGROUND Machine learning (ML) models can help assisting diagnosis by rapidly localising and classifying regions of interest (ROIs) within whole slide images (WSIs). Effective ML models for clinical decision support require a substantial dataset of 'real' data, and in reality, it should be robust, user-friendly and universally applicable. METHODS WSIs of primary IgAN were collected and annotated. The H-AI-L algorithm which could facilitate direct WSI viewing and potential ROI detection for clinicians was built on the cloud server of matpool, a shared internet-based service platform. Model performance was evaluated using F1-score, precision, recall and Matthew's correlation coefficient (MCC). RESULTS The F1-score of glomerular localisation in WSIs was 0.85 and 0.89 for the initial and pretrained models, respectively, with corresponding recall values of 0.79 and 0.83, and precision scores of 0.92 and 0.97. Dichotomous differentiation between global sclerotic (GS) and other glomeruli revealed F1-scores of 0.70 and 0.91, and MCC values of 0.55 and 0.87, for the initial and pretrained models, respectively. The overall F1-score of multiclassification was 0.81 for the pretrained models. The total glomerular recall rate was 0.96, with F1-scores of 0.68, 0.56 and 0.26 for GS, segmental glomerulosclerosis and crescent (C), respectively. Interstitial fibrosis/tubular atrophy lesion similarity between the true label and model predictions was 0.75. CONCLUSIONS Our results underscore the efficacy of the ML integration algorithm in segmenting ROIs in IgAN WSIs, and the internet-based model deployment is in favour of widespread adoption and utilisation across multiple centres and increased volumes of WSIs.
Collapse
Affiliation(s)
- Yu-Lin Huang
- Institute of Nephrology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Xiao Qi Liu
- Department of Nephrology, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Yang Huang
- Institute of Nephrology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Feng Yong Jin
- Institute of Nephrology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Qing Zhao
- Institute of Nephrology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Qin Yi Wu
- Institute of Nephrology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Kun Ling Ma
- Department of Nephrology, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| |
Collapse
|
8
|
Xia P, Lv Z, Wen Y, Zhang B, Zhao X, Zhang B, Wang Y, Cui H, Wang C, Zheng H, Qin Y, Sun L, Ye N, Cheng H, Yao L, Zhou H, Zhen J, Hu Z, Zhu W, Zhang F, Li X, Ren F, Chen L. Development of a multiple convolutional neural network-facilitated diagnostic screening program for immunofluorescence images of IgA nephropathy and idiopathic membranous nephropathy. Clin Kidney J 2023; 16:2503-2513. [PMID: 38046020 PMCID: PMC10689194 DOI: 10.1093/ckj/sfad153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Indexed: 12/05/2023] Open
Abstract
Background Immunoglobulin A nephropathy (IgAN) and idiopathic membranous nephropathy (IMN) are the most common glomerular diseases. Immunofluorescence (IF) tests of renal tissues are crucial for the diagnosis. We developed a multiple convolutional neural network (CNN)-facilitated diagnostic program to assist the IF diagnosis of IgAN and IMN. Methods The diagnostic program consisted of four parts: a CNN trained as a glomeruli detection module, an IF intensity comparator, dual-CNN (D-CNN) trained as a deposition appearance and location classifier and a post-processing module. A total of 1573 glomerular IF images from 1009 patients with glomerular diseases were used for the training and validation of the diagnostic program. A total of 1610 images of 426 patients from different hospitals were used as test datasets. The performance of the diagnostic program was compared with nephropathologists. Results In >90% of the tested images, the glomerulus location module achieved an intersection over union >0.8. The accuracy of the D-CNN in recognizing irregular granular mesangial deposition and fine granular deposition along the glomerular basement membrane was 96.1% and 93.3%, respectively. As for the diagnostic program, the accuracy, sensitivity and specificity of diagnosing suspected IgAN were 97.6%, 94.4% and 96.0%, respectively. The accuracy, sensitivity and specificity of diagnosing suspected IMN were 91.7%, 88.9% and 95.8%, respectively. The corresponding areas under the curve (AUCs) were 0.983 and 0.935. When tested with images from the outside hospital, the diagnostic program showed stable performance. The AUCs for diagnosing suspected IgAN and IMN were 0.972 and 0.948, respectively. Compared with inexperienced nephropathologists, the program showed better performance. Conclusion The proposed diagnostic program could assist the IF diagnosis of IgAN and IMN.
Collapse
Affiliation(s)
- Peng Xia
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Zhilong Lv
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
- School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China
| | - Yubing Wen
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | | | - Xuesong Zhao
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Boyao Zhang
- Beijing Zhijian Life Technology, Beijing, China
| | - Ying Wang
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Haoyuan Cui
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Chuanpeng Wang
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Hua Zheng
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yan Qin
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Lijun Sun
- Department of Nephrology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Nan Ye
- Department of Nephrology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Hong Cheng
- Department of Nephrology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Li Yao
- Department of Nephrology, First Hospital Affiliated to China Medical University, Shenyang, China
| | - Hua Zhou
- Department of Nephrology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Junhui Zhen
- Department of Pathology, Qilu Hospital of Shandong University, Jinan, China
| | - Zhao Hu
- Department of Nephrology, Qilu Hospital of Shandong University, Jinan, China
| | - Weiguo Zhu
- Department of Information, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Fa Zhang
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
| | - Xuemei Li
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Fei Ren
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
| | - Limeng Chen
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
9
|
Hou X, Tian C, Liu W, Li Y, Li W, Wang Z. Construction of artificial intelligence non-invasive diagnosis model for common glomerular diseases based on hyperspectral and urine analysis. Photodiagnosis Photodyn Ther 2023; 44:103736. [PMID: 37597684 DOI: 10.1016/j.pdpdt.2023.103736] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 07/26/2023] [Accepted: 08/04/2023] [Indexed: 08/21/2023]
Abstract
OBJECTIVE To develop a non-invasive fluid biopsy assisted diagnosis model for glomerular diseases based on hyperspectral, so as to solve the problem of poor compliance of patients with invasive examination and improve the early diagnosis rate of glomerular diseases. METHODS A total of 65 urine samples from patients who underwent renal biopsy from November 2020 to January 2022 in Qianfoshan Hospital of Shandong Province were collected.By simultaneously capturing spectral information of the above urine samples in the 400-1000 nm range, more obvious differences were found in the spectra of urine from patients with glomerular diseases between 650 nm and 680 nm. We obtained the original hyperspectral images in this wavelength range through digital scanning, and sampled pixel points at intervals on the original images. The two-dimensional digital image generated from each pixel point served as a member of the subsequent training and test sets. . After manually labeling the images according to different biopsy pathological types, they were randomly divided into training set (n = 58,800) and test set (n = 25,200). The training set was used for training learning and parameter iteration of artificial intelligence non-invasive liquid diagnosis model, and the test set for model recognition and interpretation. The evaluation indexes such as accuracy, sensitivity and specificity were calculated to evaluate the performance of the diagnosis model. RESULTS The model has an accuracy rate of 96% for early diagnosis of four glomerular diseases. CONCLUSION The auxiliary diagnosis model system has high accuracy. It is expected to be used as a non-invasive diagnostic method for glomerular diseases in clinic.
Collapse
Affiliation(s)
- Xiangyu Hou
- Department of Nephrology, Shandong Institute of Nephrology, The First Affiliated Hospital of Shandong First Medical University & Shandong Provincial Qianfoshan Hospital, No. 16766 Jingshi Road, Jinan, Shandong 250014, China
| | - Chongxuan Tian
- Department of biomedical Engineering Institute, School of Control Science and Engineering, Shandong University, Jinan, Shandong, 250016, China
| | - Wen Liu
- Department of Nephrology, Shandong Institute of Nephrology, The First Affiliated Hospital of Shandong First Medical University & Shandong Provincial Qianfoshan Hospital, No. 16766 Jingshi Road, Jinan, Shandong 250014, China
| | - Yang Li
- Department of Nephrology, Shandong Institute of Nephrology, The First Affiliated Hospital of Shandong First Medical University & Shandong Provincial Qianfoshan Hospital, No. 16766 Jingshi Road, Jinan, Shandong 250014, China
| | - Wei Li
- Department of biomedical Engineering Institute, School of Control Science and Engineering, Shandong University, Jinan, Shandong, 250016, China.
| | - Zunsong Wang
- Department of Nephrology, Shandong Institute of Nephrology, The First Affiliated Hospital of Shandong First Medical University & Shandong Provincial Qianfoshan Hospital, No. 16766 Jingshi Road, Jinan, Shandong 250014, China.
| |
Collapse
|
10
|
Gaupp C, Schmid B, Tripal P, Edwards A, Daniel C, Zimmermann S, Goppelt-Struebe M, Willam C, Rosen S, Schley G. Reconfiguration and loss of peritubular capillaries in chronic kidney disease. Sci Rep 2023; 13:19660. [PMID: 37952029 PMCID: PMC10640592 DOI: 10.1038/s41598-023-46146-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 10/27/2023] [Indexed: 11/14/2023] Open
Abstract
Functional and structural alterations of peritubular capillaries (PTCs) are a major determinant of chronic kidney disease (CKD). Using a software-based algorithm for semiautomatic segmentation and morphometric quantification, this study analyzes alterations of PTC shape associated with chronic tubulointerstitial injury in three mouse models and in human biopsies. In normal kidney tissue PTC shape was predominantly elongated, whereas the majority of PTCs associated with chronic tubulointerstitial injury had a rounder shape. This was reflected by significantly reduced PTC luminal area, perimeter and diameters as well as by significantly increased circularity and roundness. These morphological alterations were consistent in all mouse models and human kidney biopsies. The mean circularity of PTCs correlated significantly with categorized glomerular filtration rates and the degree of interstitial fibrosis and tubular atrophy (IFTA) and classified the presence of CKD or IFTA. 3D reconstruction of renal capillaries revealed not only a significant reduction, but more importantly a substantial simplification and reconfiguration of the renal microvasculature in mice with chronic tubulointerstitial injury. Computational modelling predicted that round PTCs can deliver oxygen more homogeneously to the surrounding tissue. Our findings indicate that alterations of PTC shape represent a common and uniform reaction to chronic tubulointerstitial injury independent of the underlying kidney disease.
Collapse
Affiliation(s)
- Charlotte Gaupp
- Department of Nephrology and Hypertension, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Ulmenweg 18, 91054, Erlangen, Germany
| | - Benjamin Schmid
- Optical Imaging Center Erlangen, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Philipp Tripal
- Optical Imaging Center Erlangen, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Aurélie Edwards
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Christoph Daniel
- Department of Nephropathology, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Erlangen, Germany
| | - Stefan Zimmermann
- Department of Computer Science, University of Applied Sciences Worms, Worms, Germany
| | - Margarete Goppelt-Struebe
- Department of Nephrology and Hypertension, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Ulmenweg 18, 91054, Erlangen, Germany
| | - Carsten Willam
- Department of Nephrology and Hypertension, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Ulmenweg 18, 91054, Erlangen, Germany
| | - Seymour Rosen
- Department of Pathology, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, MA, USA
| | - Gunnar Schley
- Department of Nephrology and Hypertension, Friedrich-Alexander University Erlangen-Nürnberg (FAU) and University Hospital Erlangen, Ulmenweg 18, 91054, Erlangen, Germany.
| |
Collapse
|
11
|
Liu X, Wu Y, Chen Y, Hui D, Zhang J, Hao F, Lu Y, Cheng H, Zeng Y, Han W, Wang C, Li M, Zhou X, Zheng W. Diagnosis of diabetic kidney disease in whole slide images via AI-driven quantification of pathological indicators. Comput Biol Med 2023; 166:107470. [PMID: 37722173 DOI: 10.1016/j.compbiomed.2023.107470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 08/29/2023] [Accepted: 09/04/2023] [Indexed: 09/20/2023]
Abstract
Diagnosis of diabetic kidney disease (DKD) mainly relies on screening the morphological variations and internal lesions of glomeruli from pathological kidney biopsy. The prominent pathological alterations of glomeruli for DKD include glomerular hypertrophy and nodular mesangial sclerosis. However, the qualitative judgment of these alterations is inaccurate and inconstant due to the intra- and inter-subject variability of pathologists. It is necessary to design artificial intelligence (AI) methods for accurate quantification of these pathological alterations and outcome prediction of DKD. In this work, we present an AI-driven framework to quantify the volume of glomeruli and degree of nodular mesangial sclerosis, respectively, based on an instance segmentation module and a novel weakly supervised Macro-Micro Aggregation (MMA) module. Subsequently, we construct classic machine learning models to predict the degree of DKD based on three selected pathological indicators via factor analysis. These corresponding modules are trained and tested on a total of 281 whole slide images (WSIs) digitized from two hospitals with different scanners. Our designed AI framework achieved inspiring results with 0.926 mIoU for glomerulus segmentation, and 0.899 F1 score for glomerulus classification in the external testing dataset. Meantime, the visualized results of the MMA module could reflect the location of the lesions. The performance of predicting disease achieved the F1 score of 0.917, which further proved the effectiveness of our AI-driven quantification of pathological indicators. Additionally, the interpretation of the machine learning model with the SHAP method showed similar accordance with the development of DKD in pathology. In conclusion, the proposed auxiliary diagnostic technologies have the feasibility for quantitative analysis of glomerular pathological tissues and alterations in DKD. Pathological quantitative indicators will also make it more convenient to provide doctors with assistance in clinical practice.
Collapse
Affiliation(s)
- Xueyu Liu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yongfei Wu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China.
| | - Yilin Chen
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Dongna Hui
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China
| | - Jianan Zhang
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Fang Hao
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yuanyue Lu
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China
| | - Hangbei Cheng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yue Zeng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Weixia Han
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Chen Wang
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Ming Li
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Xiaoshuang Zhou
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China.
| | - Wen Zheng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| |
Collapse
|
12
|
Besusparis J, Morkunas M, Laurinavicius A. A Spatially Guided Machine-Learning Method to Classify and Quantify Glomerular Patterns of Injury in Histology Images. J Imaging 2023; 9:220. [PMID: 37888327 PMCID: PMC10607091 DOI: 10.3390/jimaging9100220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 09/26/2023] [Accepted: 10/09/2023] [Indexed: 10/28/2023] Open
Abstract
Introduction The diagnosis of glomerular diseases is primarily based on visual assessment of histologic patterns. Semi-quantitative scoring of active and chronic lesions is often required to assess individual characteristics of the disease. Reproducibility of the visual scoring systems remains debatable, while digital and machine-learning technologies present opportunities to detect, classify and quantify glomerular lesions, also considering their inter- and intraglomerular heterogeneity. MATERIALS AND METHODS We performed a cross-validated comparison of three modifications of a convolutional neural network (CNN)-based approach for recognition and intraglomerular quantification of nine main glomerular patterns of injury. Reference values provided by two nephropathologists were used for validation. For each glomerular image, visual attention heatmaps were generated with a probability of class attribution for further intraglomerular quantification. The quality of classifier-produced heatmaps was evaluated by intersection over union metrics (IoU) between predicted and ground truth localization heatmaps. RESULTS A proposed spatially guided modification of the CNN classifier achieved the highest glomerular pattern classification accuracies, with area under curve (AUC) values up to 0.981. With regards to heatmap overlap area and intraglomerular pattern quantification, the spatially guided classifier achieved a significantly higher generalized mean IoU value compared to single-multiclass and multiple-binary classifiers. CONCLUSIONS We propose a spatially guided CNN classifier that in our experiments reveals the potential to achieve high accuracy for the localization of intraglomerular patterns.
Collapse
Affiliation(s)
- Justinas Besusparis
- Faculty of Medicine, Vilnius University, M.K.Ciurlionio 21, LT-03101 Vilnius, Lithuania; (M.M.); (A.L.)
- National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Clinics, P. Baublio 5, LT-08406 Vilnius, Lithuania
| | - Mindaugas Morkunas
- Faculty of Medicine, Vilnius University, M.K.Ciurlionio 21, LT-03101 Vilnius, Lithuania; (M.M.); (A.L.)
- National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Clinics, P. Baublio 5, LT-08406 Vilnius, Lithuania
| | - Arvydas Laurinavicius
- Faculty of Medicine, Vilnius University, M.K.Ciurlionio 21, LT-03101 Vilnius, Lithuania; (M.M.); (A.L.)
- National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Clinics, P. Baublio 5, LT-08406 Vilnius, Lithuania
| |
Collapse
|
13
|
Kaur G, Garg M, Gupta S, Juneja S, Rashid J, Gupta D, Shah A, Shaikh A. Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model. Diagnostics (Basel) 2023; 13:3152. [PMID: 37835895 PMCID: PMC10572820 DOI: 10.3390/diagnostics13193152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 09/23/2023] [Accepted: 09/25/2023] [Indexed: 10/15/2023] Open
Abstract
Glomeruli are interconnected capillaries in the renal cortex that are responsible for blood filtration. Damage to these glomeruli often signifies the presence of kidney disorders like glomerulonephritis and glomerulosclerosis, which can ultimately lead to chronic kidney disease and kidney failure. The timely detection of such conditions is essential for effective treatment. This paper proposes a modified UNet model to accurately detect glomeruli in whole-slide images of kidney tissue. The UNet model was modified by changing the number of filters and feature map dimensions from the first to the last layer to enhance the model's capacity for feature extraction. Moreover, the depth of the UNet model was also improved by adding one more convolution block to both the encoder and decoder sections. The dataset used in the study comprised 20 large whole-side images. Due to their large size, the images were cropped into 512 × 512-pixel patches, resulting in a dataset comprising 50,486 images. The proposed model performed well, with 95.7% accuracy, 97.2% precision, 96.4% recall, and 96.7% F1-score. These results demonstrate the proposed model's superior performance compared to the original UNet model, the UNet model with EfficientNetb3, and the current state-of-the-art. Based on these experimental findings, it has been determined that the proposed model accurately identifies glomeruli in extracted kidney patches.
Collapse
Affiliation(s)
- Gurjinder Kaur
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Meenu Garg
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Sheifali Gupta
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Sapna Juneja
- Kulliyyah of Information and Communication Technology, International Islamic University Malaysia, Kuala Lumpur 53100, Malaysia;
| | - Junaid Rashid
- Department of Data Science, Sejong University, Seoul 05006, Republic of Korea;
| | - Deepali Gupta
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Asadullah Shah
- Kulliyyah of Information and Communication Technology, International Islamic University Malaysia, Kuala Lumpur 53100, Malaysia;
| | - Asadullah Shaikh
- Department of Information Systems, College of Computer Science and Information Systems, Najran University, Najran 55461, Saudi Arabia;
| |
Collapse
|
14
|
Layton AT. "Hi, how can i help you?": embracing artificial intelligence in kidney research. Am J Physiol Renal Physiol 2023; 325:F395-F406. [PMID: 37589052 DOI: 10.1152/ajprenal.00177.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 08/08/2023] [Accepted: 08/09/2023] [Indexed: 08/18/2023] Open
Abstract
In recent years, biology and precision medicine have benefited from major advancements in generating large-scale molecular and biomedical datasets and in analyzing those data using advanced machine learning algorithms. Machine learning applications in kidney physiology and pathophysiology include segmenting kidney structures from imaging data and predicting conditions like acute kidney injury or chronic kidney disease using electronic health records. Despite the potential of machine learning to revolutionize nephrology by providing innovative diagnostic and therapeutic tools, its adoption in kidney research has been slower than in other organ systems. Several factors contribute to this underutilization. The complexity of the kidney as an organ, with intricate physiology and specialized cell populations, makes it challenging to extrapolate bulk omics data to specific processes. In addition, kidney diseases often present with overlapping manifestations and morphological changes, making diagnosis and treatment complex. Moreover, kidney diseases receive less funding compared with other pathologies, leading to lower awareness and limited public-private partnerships. To promote the use of machine learning in kidney research, this review provides an introduction to machine learning and reviews its notable applications in renal research, such as morphological analysis, omics data examination, and disease diagnosis and prognosis. Challenges and limitations associated with data-driven predictive techniques are also discussed. The goal of this review is to raise awareness and encourage the kidney research community to embrace machine learning as a powerful tool that can drive advancements in understanding kidney diseases and improving patient care.
Collapse
Affiliation(s)
- Anita T Layton
- Department of Applied Mathematics, University of Waterloo, Waterloo, Ontario, Canada
- Department of Biology, University of Waterloo, Waterloo, Ontario, Canada
- Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada
- School of Pharmacology, University of Waterloo, Waterloo, Ontario, Canada
| |
Collapse
|
15
|
Deng R, Liu Q, Cui C, Yao T, Long J, Asad Z, Womick RM, Zhu Z, Fogo AB, Zhao S, Yang H, Huo Y. Omni-Seg: A Scale-Aware Dynamic Network for Renal Pathological Image Segmentation. IEEE Trans Biomed Eng 2023; 70:2636-2644. [PMID: 37030838 PMCID: PMC10517077 DOI: 10.1109/tbme.2023.3260739] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/10/2023]
Abstract
Comprehensive semantic segmentation on renal pathological images is challenging due to the heterogeneous scales of the objects. For example, on a whole slide image (WSI), the cross-sectional areas of glomeruli can be 64 times larger than that of the peritubular capillaries, making it impractical to segment both objects on the same patch, at the same scale. To handle this scaling issue, prior studies have typically trained multiple segmentation networks in order to match the optimal pixel resolution of heterogeneous tissue types. This multi-network solution is resource-intensive and fails to model the spatial relationship between tissue types. In this article, we propose the Omni-Seg network, a scale-aware dynamic neural network that achieves multi-object (six tissue types) and multi-scale (5× to 40× scale) pathological image segmentation via a single neural network. The contribution of this article is three-fold: (1) a novel scale-aware controller is proposed to generalize the dynamic neural network from single-scale to multi-scale; (2) semi-supervised consistency regularization of pseudo-labels is introduced to model the inter-scale correlation of unannotated tissue types into a single end-to-end learning paradigm; and (3) superior scale-aware generalization is evidenced by directly applying a model trained on human kidney images to mouse kidney images, without retraining. By learning from 150,000 human pathological image patches from six tissue types at three different resolutions, our approach achieved superior segmentation performance according to human visual assessment and evaluation of image-omics (i.e., spatial transcriptomics).
Collapse
|
16
|
Lee J, Warner E, Shaikhouni S, Bitzer M, Kretzler M, Gipson D, Pennathur S, Bellovich K, Bhat Z, Gadegbeku C, Massengill S, Perumal K, Saha J, Yang Y, Luo J, Zhang X, Mariani L, Hodgin JB, Rao A. Clustering-based spatial analysis (CluSA) framework through graph neural network for chronic kidney disease prediction using histopathology images. Sci Rep 2023; 13:12701. [PMID: 37543648 PMCID: PMC10404289 DOI: 10.1038/s41598-023-39591-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 07/27/2023] [Indexed: 08/07/2023] Open
Abstract
Machine learning applied to digital pathology has been increasingly used to assess kidney function and diagnose the underlying cause of chronic kidney disease (CKD). We developed a novel computational framework, clustering-based spatial analysis (CluSA), that leverages unsupervised learning to learn spatial relationships between local visual patterns in kidney tissue. This framework minimizes the need for time-consuming and impractical expert annotations. 107,471 histopathology images obtained from 172 biopsy cores were used in the clustering and in the deep learning model. To incorporate spatial information over the clustered image patterns on the biopsy sample, we spatially encoded clustered patterns with colors and performed spatial analysis through graph neural network. A random forest classifier with various groups of features were used to predict CKD. For predicting eGFR at the biopsy, we achieved a sensitivity of 0.97, specificity of 0.90, and accuracy of 0.95. AUC was 0.96. For predicting eGFR changes in one-year, we achieved a sensitivity of 0.83, specificity of 0.85, and accuracy of 0.84. AUC was 0.85. This study presents the first spatial analysis based on unsupervised machine learning algorithms. Without expert annotation, CluSA framework can not only accurately classify and predict the degree of kidney function at the biopsy and in one year, but also identify novel predictors of kidney function and renal prognosis.
Collapse
Affiliation(s)
- Joonsang Lee
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA.
| | - Elisa Warner
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Salma Shaikhouni
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Markus Bitzer
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Matthias Kretzler
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Debbie Gipson
- Department of Pediatrics, Pediatric Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Subramaniam Pennathur
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Keith Bellovich
- Department of Internal Medicine, Nephrology, St. Clair Nephrology Research, Detroit, MI, USA
| | - Zeenat Bhat
- Department of Internal Medicine, Nephrology, Wayne State University, Detroit, MI, USA
| | - Crystal Gadegbeku
- Department of Internal Medicine, Nephrology, Cleveland Clinic, , Cleveland, OH, USA
| | - Susan Massengill
- Department of Pediatrics, Pediatric Nephrology, Levine Children's Hospital, Charlotte, NC, USA
| | - Kalyani Perumal
- Department of Internal Medicine, Nephrology, Department of JH Stroger Hospital, Chicago, IL, USA
| | - Jharna Saha
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - Yingbao Yang
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - Jinghui Luo
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - Xin Zhang
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Laura Mariani
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Jeffrey B Hodgin
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA.
| | - Arvind Rao
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA.
- Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA.
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI, USA.
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.
| |
Collapse
|
17
|
Jain Y, Godwin LL, Ju Y, Sood N, Quardokus EM, Bueckle A, Longacre T, Horning A, Lin Y, Esplin ED, Hickey JW, Snyder MP, Patterson NH, Spraggins JM, Börner K. Segmentation of human functional tissue units in support of a Human Reference Atlas. Commun Biol 2023; 6:717. [PMID: 37468557 PMCID: PMC10356924 DOI: 10.1038/s42003-023-04848-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 04/17/2023] [Indexed: 07/21/2023] Open
Abstract
The Human BioMolecular Atlas Program (HuBMAP) aims to compile a Human Reference Atlas (HRA) for the healthy adult body at the cellular level. Functional tissue units (FTUs), relevant for HRA construction, are of pathobiological significance. Manual segmentation of FTUs does not scale; highly accurate and performant, open-source machine-learning algorithms are needed. We designed and hosted a Kaggle competition that focused on development of such algorithms and 1200 teams from 60 countries participated. We present the competition outcomes and an expanded analysis of the winning algorithms on additional kidney and colon tissue data, and conduct a pilot study to understand spatial location and density of FTUs across the kidney. The top algorithm from the competition, Tom, outperforms other algorithms in the expanded study, while using fewer computational resources. Tom was added to the HuBMAP infrastructure to run kidney FTU segmentation at scale-showcasing the value of Kaggle competitions for advancing research.
Collapse
Affiliation(s)
- Yashvardhan Jain
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA.
| | - Leah L Godwin
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Yingnan Ju
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Naveksha Sood
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Ellen M Quardokus
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Andreas Bueckle
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Teri Longacre
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | - Aaron Horning
- Thermo Fisher Scientific, South San Francisco, CA, 94080, USA
| | - Yiing Lin
- Department of Surgery, Washington University School of Medicine, St. Louis, MO, 63110, USA
| | - Edward D Esplin
- Department of Genetics, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | - John W Hickey
- Department of Microbiology & Immunology, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | - Michael P Snyder
- Department of Genetics, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | | | - Jeffrey M Spraggins
- Mass Spectrometry Research Center, Vanderbilt University, Nashville, TN, 37232, USA
- Department of Cell and Developmental Biology, Vanderbilt University, Nashville, TN, 37232, USA
| | - Katy Börner
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA.
| |
Collapse
|
18
|
Shickel B, Lucarelli N, Rao AS, Yun D, Moon KC, Han SS, Sarder P. Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.02.20.23286044. [PMID: 36865174 PMCID: PMC9980230 DOI: 10.1101/2023.02.20.23286044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that sugest opportunities for future spatially aware WSI research using limited pathology datasets.
Collapse
Affiliation(s)
- Benjamin Shickel
- Dept. of Medicine—Quantitative Health, Univ. of Florida, Gainesville, FL, USA
- Univ. of Florida Intelligent Critical Care Center, Gainesville, FL, USA; Dept. of Electrical & Computer Engineering, Univ. of Florida, Gainesville, FL, USA
| | | | - Adish S. Rao
- Dept. of Computer and Information Science and Engineering, Univ. of Florida, Gainesville, FL
| | - Donghwan Yun
- Dept. of Internal Medicine, Seoul National Univ., Seoul, Korea
| | - Kyung Chul Moon
- Dept. of Internal Medicine, Seoul National Univ., Seoul, Korea
| | - Seung Seok Han
- Dept. of Internal Medicine, Seoul National Univ., Seoul, Korea
| | - Pinaki Sarder
- Dept. of Medicine—Quantitative Health, Univ. of Florida, Gainesville, FL, USA
- Dept. of Biomedical Engineering, Univ. of Florida, Gainesville, FL, USA
- Univ. of Florida Intelligent Critical Care Center, Gainesville, FL, USA; Dept. of Electrical & Computer Engineering, Univ. of Florida, Gainesville, FL, USA
| |
Collapse
|
19
|
Predict, diagnose, and treat chronic kidney disease with machine learning: a systematic literature review. J Nephrol 2023; 36:1101-1117. [PMID: 36786976 PMCID: PMC10227138 DOI: 10.1007/s40620-023-01573-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2022] [Accepted: 01/01/2023] [Indexed: 02/15/2023]
Abstract
OBJECTIVES In this systematic review we aimed at assessing how artificial intelligence (AI), including machine learning (ML) techniques have been deployed to predict, diagnose, and treat chronic kidney disease (CKD). We systematically reviewed the available evidence on these innovative techniques to improve CKD diagnosis and patient management. METHODS We included English language studies retrieved from PubMed. The review is therefore to be classified as a "rapid review", since it includes one database only, and has language restrictions; the novelty and importance of the issue make missing relevant papers unlikely. We extracted 16 variables, including: main aim, studied population, data source, sample size, problem type (regression, classification), predictors used, and performance metrics. We followed the Preferred Reporting Items for Systematic Reviews (PRISMA) approach; all main steps were done in duplicate. RESULTS From a total of 648 studies initially retrieved, 68 articles met the inclusion criteria. Models, as reported by authors, performed well, but the reported metrics were not homogeneous across articles and therefore direct comparison was not feasible. The most common aim was prediction of prognosis, followed by diagnosis of CKD. Algorithm generalizability, and testing on diverse populations was rarely taken into account. Furthermore, the clinical evaluation and validation of the models/algorithms was perused; only a fraction of the included studies, 6 out of 68, were performed in a clinical context. CONCLUSIONS Machine learning is a promising tool for the prediction of risk, diagnosis, and therapy management for CKD patients. Nonetheless, future work is needed to address the interpretability, generalizability, and fairness of the models to ensure the safe application of such technologies in routine clinical practice.
Collapse
|
20
|
Shickel B, Lucarelli N, Rao A, Yun D, Moon KC, Han SS, Sarder P. Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12471:124710K. [PMID: 37818350 PMCID: PMC10563813 DOI: 10.1117/12.2655266] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/12/2023]
Abstract
Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists' predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that suggest opportunities for future spatially aware WSI research using limited pathology datasets.
Collapse
Affiliation(s)
- Benjamin Shickel
- Dept. of Medicine, University of Florida, Gainesville, FL, USA
- Univ. of Florida Intelligent Critical Care Center, Gainesville, FL, USA
| | | | - Adish Rao
- Dept. of Computer and Information Science and Engineering, Univ. of Florida, Gainesville, FL
| | - Donghwan Yun
- Dept. of Internal Medicine, Seoul National Univ. College of Medicine, Seoul, Korea
| | - Kyung Chul Moon
- Dept. of Internal Medicine, Seoul National Univ. College of Medicine, Seoul, Korea
| | - Seung Seok Han
- Dept. of Internal Medicine, Seoul National Univ. College of Medicine, Seoul, Korea
| | - Pinaki Sarder
- Dept. of Medicine, University of Florida, Gainesville, FL, USA
- Univ. of Florida Intelligent Critical Care Center, Gainesville, FL, USA
| |
Collapse
|
21
|
Hao F, Liu X, Li M, Han W. Accurate Kidney Pathological Image Classification Method Based on Deep Learning and Multi-Modal Fusion Method with Application to Membranous Nephropathy. Life (Basel) 2023; 13:life13020399. [PMID: 36836756 PMCID: PMC9960995 DOI: 10.3390/life13020399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 01/25/2023] [Accepted: 01/27/2023] [Indexed: 02/04/2023] Open
Abstract
Membranous nephropathy is one of the most prevalent conditions responsible for nephrotic syndrome in adults. It is clinically nonspecific and mainly diagnosed by kidney biopsy pathology, with three prevalent techniques: light microscopy, electron microscopy, and immunofluorescence microscopy. Manual observation of glomeruli one by one under the microscope is very time-consuming, and there are certain differences in the observation results between physicians. This study makes use of whole-slide images scanned by a light microscope as well as immunofluorescence images to classify patients with membranous nephropathy. The framework mainly includes a glomerular segmentation module, a confidence coefficient extraction module, and a multi-modal fusion module. This framework first identifies and segments the glomerulus from whole-slide images and immunofluorescence images, and then a glomerular classifier is trained to extract the features of each glomerulus. The results are then combined to produce the final diagnosis. The results of the experiments show that the F1-score of image classification results obtained by combining two kinds of features, which can reach 97.32%, is higher than those obtained by using only light-microscopy-observed images or immunofluorescent images, which reach 92.76% and 93.20%, respectively. Experiments demonstrate that considering both WSIs and immunofluorescence images is effective in improving the diagnosis of membranous nephropathy.
Collapse
Affiliation(s)
- Fang Hao
- College of Data Science, Taiyuan University of Technology, Taiyuan 030024, China
- Correspondence:
| | - Xueyu Liu
- College of Data Science, Taiyuan University of Technology, Taiyuan 030024, China
| | - Ming Li
- College of Data Science, Taiyuan University of Technology, Taiyuan 030024, China
| | - Weixia Han
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan 030001, China
| |
Collapse
|
22
|
Glomerulosclerosis detection with pre-trained CNNs ensemble. Comput Stat 2022. [DOI: 10.1007/s00180-022-01307-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
23
|
Ayorinde JOO, Citterio F, Landrò M, Peruzzo E, Islam T, Tilley S, Taylor G, Bardsley V, Liò P, Samoshkin A, Pettigrew GJ. Artificial Intelligence You Can Trust: What Matters Beyond Performance When Applying Artificial Intelligence to Renal Histopathology? J Am Soc Nephrol 2022; 33:2133-2140. [PMID: 36351761 PMCID: PMC9731632 DOI: 10.1681/asn.2022010069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022] Open
Abstract
Although still in its infancy, artificial intelligence (AI) analysis of kidney biopsy images is anticipated to become an integral aspect of renal histopathology. As these systems are developed, the focus will understandably be on developing ever more accurate models, but successful translation to the clinic will also depend upon other characteristics of the system.In the extreme, deployment of highly performant but "black box" AI is fraught with risk, and high-profile errors could damage future trust in the technology. Furthermore, a major factor determining whether new systems are adopted in clinical settings is whether they are "trusted" by clinicians. Key to unlocking trust will be designing platforms optimized for intuitive human-AI interactions and ensuring that, where judgment is required to resolve ambiguous areas of assessment, the workings of the AI image classifier are understandable to the human observer. Therefore, determining the optimal design for AI systems depends on factors beyond performance, with considerations of goals, interpretability, and safety constraining many design and engineering choices.In this article, we explore challenges that arise in the application of AI to renal histopathology, and consider areas where choices around model architecture, training strategy, and workflow design may be influenced by factors beyond the final performance metrics of the system.
Collapse
Affiliation(s)
- John O O Ayorinde
- Department of Surgery, University of Cambridge, Addenbrooke's Hospital, Cambridge, United Kingdom
| | | | | | | | | | | | | | - Victoria Bardsley
- Department of Histopathology, Addenbrooke's Hospital, Cambridge, United Kingdom
| | - Pietro Liò
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Alex Samoshkin
- Office for Translational Research, School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom
| | - Gavin J Pettigrew
- Department of Surgery, University of Cambridge, Addenbrooke's Hospital, Cambridge, United Kingdom
| |
Collapse
|
24
|
Hermsen M, Ciompi F, Adefidipe A, Denic A, Dendooven A, Smith BH, van Midden D, Bräsen JH, Kers J, Stegall MD, Bándi P, Nguyen T, Swiderska-Chadaj Z, Smeets B, Hilbrands LB, van der Laak JAWM. Convolutional Neural Networks for the Evaluation of Chronic and Inflammatory Lesions in Kidney Transplant Biopsies. THE AMERICAN JOURNAL OF PATHOLOGY 2022; 192:1418-1432. [PMID: 35843265 DOI: 10.1016/j.ajpath.2022.06.009] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2021] [Revised: 06/13/2022] [Accepted: 06/22/2022] [Indexed: 06/15/2023]
Abstract
In kidney transplant biopsies, both inflammation and chronic changes are important features that predict long-term graft survival. Quantitative scoring of these features is important for transplant diagnostics and kidney research. However, visual scoring is poorly reproducible and labor intensive. The goal of this study was to investigate the potential of convolutional neural networks (CNNs) to quantify inflammation and chronic features in kidney transplant biopsies. A structure segmentation CNN and a lymphocyte detection CNN were applied on 125 whole-slide image pairs of periodic acid-Schiff- and CD3-stained slides. The CNN results were used to quantify healthy and sclerotic glomeruli, interstitial fibrosis, tubular atrophy, and inflammation within both nonatrophic and atrophic tubuli, and in areas of interstitial fibrosis. The computed tissue features showed high correlation with Banff lesion scores of five pathologists (A.A., A.Dend., J.H.B., J.K., and T.N.). Analyses on a small subset showed a moderate correlation toward higher CD3+ cell density within scarred regions and higher CD3+ cell count inside atrophic tubuli correlated with long-term change of estimated glomerular filtration rate. The presented CNNs are valid tools to yield objective quantitative information on glomeruli number, fibrotic tissue, and inflammation within scarred and non-scarred kidney parenchyma in a reproducible manner. CNNs have the potential to improve kidney transplant diagnostics and will benefit the community as a novel method to generate surrogate end points for large-scale clinical studies.
Collapse
Affiliation(s)
- Meyke Hermsen
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Adeyemi Adefidipe
- Department of Pathology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, the Netherlands
| | - Aleksandar Denic
- Division of Nephrology and Hypertension, Mayo Clinic, Rochester, Minnesota
| | - Amélie Dendooven
- Department of Pathology, Ghent University Hospital, Ghent, Belgium; Faculty of Medicine, University of Antwerp, Wilrijk, Antwerp, Belgium
| | - Byron H Smith
- William J. von Liebig Center for Transplantation and Clinical Regeneration, Mayo Clinic, Rochester, Minnesota; Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, Minnesota
| | - Dominique van Midden
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jan Hinrich Bräsen
- Nephropathology Unit, Institute of Pathology, Hannover Medical School, Hannover, Germany
| | - Jesper Kers
- Department of Pathology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, the Netherlands; Department of Pathology, Leiden University Medical Center, Leiden, the Netherlands; Center for Analytical Sciences Amsterdam, Van 't Hoff Institute for Molecular Sciences, University of Amsterdam, Amsterdam, the Netherlands
| | - Mark D Stegall
- Division of Transplantation Surgery, Mayo Clinic, Rochester, Minnesota
| | - Péter Bándi
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Tri Nguyen
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Zaneta Swiderska-Chadaj
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands; Faculty of Electrical Engineering, Warsaw University of Technology, Warsaw, Poland
| | - Bart Smeets
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Luuk B Hilbrands
- Department of Nephrology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jeroen A W M van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands; Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden.
| |
Collapse
|
25
|
Silva J, Souza L, Chagas P, Calumby R, Souza B, Pontes I, Duarte A, Pinheiro N, Santos W, Oliveira L. Boundary-aware glomerulus segmentation: Toward one-to-many stain generalization. Comput Med Imaging Graph 2022; 100:102104. [PMID: 36007483 DOI: 10.1016/j.compmedimag.2022.102104] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 05/28/2022] [Accepted: 07/08/2022] [Indexed: 11/30/2022]
Abstract
The growing availability of scanned whole-slide images (WSIs) has allowed nephropathology to open new possibilities for medical decision-making over high-resolution images. Diagnosis of renal WSIs includes locating and identifying specific structures in the tissue. Considering the glomerulus as one of the first structures analyzed by pathologists, we propose here a novel convolutional neural network for glomerulus segmentation. Our end-to-end network, named DS-FNet, combines the strengths of semantic segmentation and semantic boundary detection networks via an attention-aware mechanism. Although we trained the proposed network on periodic acid-Schiff (PAS)-stained WSIs, we found that our network was capable to segment glomeruli on WSIs stained with different techniques, such as periodic acid-methenamine silver (PAMS), hematoxylin-eosin (HE), and Masson trichrome (TRI). To assess the performance of the proposed method, we used three public data sets: HuBMAP (available in a Kaggle competition), a subset of the NEPTUNE data set, and a novel challenging data set, called WSI_Fiocruz. We compared the DS-FNet with six other deep learning networks: original U-Net, our attention version of U-Net called AU-Net, U-Net++, U-Net3Plus, ResU-Net, and DeepLabV3+. Results showed that DS-FNet achieved equivalent or superior results on all data sets: On the HuBMAP data set, it reached a dice score (DSC) of 95.05%, very close to the first place (95.15%); on the NEPTUNE and WSI_Fiocruz data sets, DS-FNet obtained the highest average DSC, whether on PAS-stained images or images stained with other techniques. To the best we know, this is the first work to show consistently high performance in a one-to-many-stain glomerulus segmentation following a thorough protocol on data sets from different medical labs.
Collapse
Affiliation(s)
- Jefferson Silva
- Universidade Federal do Maranhão, Brazil; Universidade Federal da Bahia, Brazil
| | | | | | | | - Bianca Souza
- Universidade Federal da Bahia, Brazil; Fundação Oswaldo Cruz, Brazil
| | | | | | | | - Washington Santos
- Universidade Federal da Bahia, Brazil; Fundação Oswaldo Cruz, Brazil
| | | |
Collapse
|
26
|
Artificial Intelligence-Assisted Renal Pathology: Advances and Prospects. J Clin Med 2022; 11:jcm11164918. [PMID: 36013157 PMCID: PMC9410196 DOI: 10.3390/jcm11164918] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 07/30/2022] [Accepted: 08/11/2022] [Indexed: 11/17/2022] Open
Abstract
Digital imaging and advanced microscopy play a pivotal role in the diagnosis of kidney diseases. In recent years, great achievements have been made in digital imaging, providing novel approaches for precise quantitative assessments of nephropathology and relieving burdens of renal pathologists. Developing novel methods of artificial intelligence (AI)-assisted technology through multidisciplinary interaction among computer engineers, renal specialists, and nephropathologists could prove beneficial for renal pathology diagnoses. An increasing number of publications has demonstrated the rapid growth of AI-based technology in nephrology. In this review, we offer an overview of AI-assisted renal pathology, including AI concepts and the workflow of processing digital image data, focusing on the impressive advances of AI application in disease-specific backgrounds. In particular, this review describes the applied computer vision algorithms for the segmentation of kidney structures, diagnosis of specific pathological changes, and prognosis prediction based on images. Lastly, we discuss challenges and prospects to provide an objective view of this topic.
Collapse
|
27
|
A user-friendly tool for cloud-based whole slide image segmentation with examples from renal histopathology. COMMUNICATIONS MEDICINE 2022; 2:105. [PMID: 35996627 PMCID: PMC9391340 DOI: 10.1038/s43856-022-00138-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2021] [Accepted: 06/09/2022] [Indexed: 01/21/2023] Open
Abstract
Background Image-based machine learning tools hold great promise for clinical applications in pathology research. However, the ideal end-users of these computational tools (e.g., pathologists and biological scientists) often lack the programming experience required for the setup and use of these tools which often rely on the use of command line interfaces. Methods We have developed Histo-Cloud, a tool for segmentation of whole slide images (WSIs) that has an easy-to-use graphical user interface. This tool runs a state-of-the-art convolutional neural network (CNN) for segmentation of WSIs in the cloud and allows the extraction of features from segmented regions for further analysis. Results By segmenting glomeruli, interstitial fibrosis and tubular atrophy, and vascular structures from renal and non-renal WSIs, we demonstrate the scalability, best practices for transfer learning, and effects of dataset variability. Finally, we demonstrate an application for animal model research, analyzing glomerular features in three murine models. Conclusions Histo-Cloud is open source, accessible over the internet, and adaptable for segmentation of any histological structure regardless of stain.
Collapse
|
28
|
Patel AU, Shaker N, Mohanty S, Sharma S, Gangal S, Eloy C, Parwani AV. Cultivating Clinical Clarity through Computer Vision: A Current Perspective on Whole Slide Imaging and Artificial Intelligence. Diagnostics (Basel) 2022; 12:diagnostics12081778. [PMID: 35892487 PMCID: PMC9332710 DOI: 10.3390/diagnostics12081778] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2022] [Revised: 07/10/2022] [Accepted: 07/11/2022] [Indexed: 11/17/2022] Open
Abstract
Diagnostic devices, methodological approaches, and traditional constructs of clinical pathology practice, cultivated throughout centuries, have transformed radically in the wake of explosive technological growth and other, e.g., environmental, catalysts of change. Ushered into the fray of modern laboratory medicine are digital imaging devices and machine-learning (ML) software fashioned to mitigate challenges, e.g., practitioner shortage while preparing clinicians for emerging interconnectivity of environments and diagnostic information in the era of big data. As computer vision shapes new constructs for the modern world and intertwines with clinical medicine, cultivating clarity of our new terrain through examining the trajectory and current scope of computational pathology and its pertinence to clinical practice is vital. Through review of numerous studies, we find developmental efforts for ML migrating from research to standardized clinical frameworks while overcoming obstacles that have formerly curtailed adoption of these tools, e.g., generalizability, data availability, and user-friendly accessibility. Groundbreaking validatory efforts have facilitated the clinical deployment of ML tools demonstrating the capacity to effectively aid in distinguishing tumor subtype and grade, classify early vs. advanced cancer stages, and assist in quality control and primary diagnosis applications. Case studies have demonstrated the benefits of streamlined, digitized workflows for practitioners alleviated by decreased burdens.
Collapse
Affiliation(s)
- Ankush U. Patel
- Mayo Clinic Department of Laboratory Medicine and Pathology, Rochester, MN 55905, USA
- Correspondence: ; Tel.: +1-206-451-3519
| | - Nada Shaker
- Department of Pathology, Wexner Medical Center, The Ohio State University, Columbus, OH 43210, USA; (N.S.); (S.G.); (A.V.P.)
| | - Sambit Mohanty
- CORE Diagnostics, Gurugram 122016, India; (S.M.); (S.S.)
- Advanced Medical Research Institute, Bareilly 243001, India
| | - Shivani Sharma
- CORE Diagnostics, Gurugram 122016, India; (S.M.); (S.S.)
| | - Shivam Gangal
- Department of Pathology, Wexner Medical Center, The Ohio State University, Columbus, OH 43210, USA; (N.S.); (S.G.); (A.V.P.)
- College of Engineering, Biomedical Engineering, The Ohio State University, Columbus, OH 43210, USA
| | - Catarina Eloy
- Institute of Molecular Pathology and Immunology of the University of Porto (IPATIMUP), Rua Júlio Amaral de Carvalho, 45, 4200-135 Porto, Portugal;
- Institute for Research and Innovation in Health (I3S Consortium), Rua Alfredo Allen, 208, 4200-135 Porto, Portugal
| | - Anil V. Parwani
- Department of Pathology, Wexner Medical Center, The Ohio State University, Columbus, OH 43210, USA; (N.S.); (S.G.); (A.V.P.)
- Cooperative Human Tissue Network (CHTN) Midwestern Division, Columbus, OH 43240, USA
| |
Collapse
|
29
|
Hara S, Haneda E, Kawakami M, Morita K, Nishioka R, Zoshima T, Kometani M, Yoneda T, Kawano M, Karashima S, Nambo H. Evaluating tubulointerstitial compartments in renal biopsy specimens using a deep learning-based approach for classifying normal and abnormal tubules. PLoS One 2022; 17:e0271161. [PMID: 35816495 PMCID: PMC9273082 DOI: 10.1371/journal.pone.0271161] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2022] [Accepted: 06/27/2022] [Indexed: 12/02/2022] Open
Abstract
Renal pathology is essential for diagnosing and assessing the severity and prognosis of kidney diseases. Deep learning-based approaches have developed rapidly and have been applied in renal pathology. However, methods for the automated classification of normal and abnormal renal tubules remain scarce. Using a deep learning-based method, we aimed to classify normal and abnormal renal tubules, thereby assisting renal pathologists in the evaluation of renal biopsy specimens. Consequently, we developed a U-Net-based segmentation model using randomly selected regions obtained from 21 renal biopsy specimens. Further, we verified its performance in multiclass segmentation by calculating the Dice coefficients (DCs). We used 15 cases of tubulointerstitial nephritis to assess its applicability in aiding routine diagnoses conducted by renal pathologists and calculated the agreement ratio between diagnoses conducted by two renal pathologists and the time taken for evaluation. We also determined whether such diagnoses were improved when the output of segmentation was considered. The glomeruli and interstitium had the highest DCs, whereas the normal and abnormal renal tubules had intermediate DCs. Following the detailed evaluation of the tubulointerstitial compartments, the proximal, distal, atrophied, and degenerated tubules had intermediate DCs, whereas the arteries and inflamed tubules had low DCs. The annotation and output areas involving normal and abnormal tubules were strongly correlated in each class. The pathological concordance for the glomerular count, t, ct, and ci scores of the Banff classification of renal allograft pathology remained high with or without the segmented images. However, in terms of time consumption, the quantitative assessment of tubulitis, tubular atrophy, degenerated tubules, and the interstitium was improved significantly when renal pathologists considered the segmentation output. Deep learning algorithms can assist renal pathologists in the classification of normal and abnormal tubules in renal biopsy specimens, thereby facilitating the enhancement of renal pathology and ensuring appropriate clinical decisions.
Collapse
Affiliation(s)
- Satoshi Hara
- Medical Education Research Center, Graduate School of Medical Sciences, Kanazawa University, Kanazawa, Japan
- Department of Rheumatology, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
| | - Emi Haneda
- School of Electrical Information Communication Engineering, College of Science and Engineering, Kanazawa University, Kanazawa, Japan
| | - Masaki Kawakami
- School of Electrical Information Communication Engineering, College of Science and Engineering, Kanazawa University, Kanazawa, Japan
| | - Kento Morita
- School of Electrical Information Communication Engineering, College of Science and Engineering, Kanazawa University, Kanazawa, Japan
| | - Ryo Nishioka
- Department of Rheumatology, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
| | - Takeshi Zoshima
- Department of Rheumatology, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
| | - Mitsuhiro Kometani
- Department of Endocrinology and Metabolism, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
| | - Takashi Yoneda
- Department of Endocrinology and Metabolism, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
- Department of Health Promotion and Medicine of the Future, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
- Faculty of Transdisciplinary Sciences, Institute of Transdisciplinary Sciences, Kanazawa University, Kanazawa, Japan
| | - Mitsuhiro Kawano
- Department of Rheumatology, Kanazawa University Graduate School of Medicine, Kanazawa, Japan
- * E-mail: (MK); (HN)
| | | | - Hidetaka Nambo
- School of Electrical Information Communication Engineering, College of Science and Engineering, Kanazawa University, Kanazawa, Japan
- * E-mail: (MK); (HN)
| |
Collapse
|
30
|
Kunishima N, Takeda Y, Hirose R, Kume S, Maeda M, Oguchi A, Yanagita M, Shibuya H, Tamura M, Kataoka Y, Murakawa Y, Ito K, Omote K. Compact laboratory-based X-ray microscope enabling nondestructive 3D structure acquisition of mouse nephron with high speed and better user accessibility. Microscopy (Oxf) 2022; 71:315-323. [PMID: 35778966 PMCID: PMC9731380 DOI: 10.1093/jmicro/dfac033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Revised: 06/20/2022] [Accepted: 07/01/2022] [Indexed: 12/15/2022] Open
Abstract
X-ray microscopes adopting computed tomography enable nondestructive 3D visualization of biological specimens at micron-level resolution without conventional 2D serial sectioning that is a destructive/laborious method and is routinely used for analyzing renal biopsy in clinical diagnosis of kidney diseases. Here we applied a compact commercial system of laboratory-based X-ray microscope to observe a resin-embedded osmium-stained 1-mm strip of a mouse kidney piece as a model of renal biopsy, toward a more efficient diagnosis of kidney diseases. A reconstructed computed tomography image from several hours of data collection using CCD detector allowed us to unambiguously segment a single nephron connected to a renal corpuscle, which was consistent with previous reports using serial sectioning. Histogram analysis on the segmented nephron confirmed that the proximal and distal tubules were distinguishable on the basis of their X-ray opacities. A 3D rendering model of the segmented nephron visualized a convoluted structure of renal tubules neighboring the renal corpuscle and a branched structure of efferent arterioles. Furthermore, another data collection using scientific complementary metal-oxide semiconductor detector with a much shorter data acquisition time of 15 min provided similar results from the same samples. These results suggest a potential application of the compact laboratory-based X-ray microscope to analyze mouse renal biopsy.
Collapse
Affiliation(s)
| | - Yoshihiro Takeda
- X-ray Research Laboratory, Rigaku Corporation, Akishima, Tokyo 196-8666, Japan
| | - Raita Hirose
- X-ray Research Laboratory, Rigaku Corporation, Akishima, Tokyo 196-8666, Japan
| | - Satoshi Kume
- Laboratory for Pathophysiological and Health Science, RIKEN Center for Biosystems Dynamics Research, Kobe, Hyogo 650-0047, Japan,Center for Health Science Innovation, Osaka City University, Osaka 530-0011, Japan
| | - Mitsuyo Maeda
- Multi-Modal Microstructure Analysis Unit, RIKEN-JEOL Collaboration Center, Kobe, Hyogo 650-0047, Japan
| | - Akiko Oguchi
- RIKEN Center for Integrative Medical Sciences, Yokohama, Kanagawa 230-0045, Japan,Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto 606-8501, Japan
| | - Motoko Yanagita
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto 606-8501, Japan,Institute for the Advanced Study of Human Biology (ASHBi), Kyoto University, Kyoto 606-8501, Japan
| | - Hirotoshi Shibuya
- Technology and Development Team for Mouse Phenotype Analysis, RIKEN BioResource Research Center, Tsukuba, Ibaraki 305-0074, Japan
| | - Masaru Tamura
- Technology and Development Team for Mouse Phenotype Analysis, RIKEN BioResource Research Center, Tsukuba, Ibaraki 305-0074, Japan
| | - Yosky Kataoka
- Multi-Modal Microstructure Analysis Unit, RIKEN-JEOL Collaboration Center, Kobe, Hyogo 650-0047, Japan,Laboratory for Cellular Function Imaging, RIKEN Center for Biosystems Dynamics Research, Kobe, Hyogo 650-0047, Japan
| | - Yasuhiro Murakawa
- RIKEN Center for Integrative Medical Sciences, Yokohama, Kanagawa 230-0045, Japan,Institute for the Advanced Study of Human Biology (ASHBi), Kyoto University, Kyoto 606-8501, Japan,IFOM―the FIRC Institute of Molecular Oncology, Milan 20139, Italy
| | - Koichiro Ito
- New Market Development Office, Rigaku Corporation, Akishima, Tokyo 196-8666, Japan
| | - Kazuhiko Omote
- X-ray Research Laboratory, Rigaku Corporation, Akishima, Tokyo 196-8666, Japan
| |
Collapse
|
31
|
Jiang J, Chan L, Nadkarni GN. The promise of artificial intelligence for kidney pathophysiology. Curr Opin Nephrol Hypertens 2022; 31:380-386. [PMID: 35703218 PMCID: PMC10309072 DOI: 10.1097/mnh.0000000000000808] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
PURPOSE OF REVIEW We seek to determine recent advances in kidney pathophysiology that have been enabled or enhanced by artificial intelligence. We describe some of the challenges in the field as well as future directions. RECENT FINDINGS We first provide an overview of artificial intelligence terminologies and methodologies. We then describe the use of artificial intelligence in kidney diseases to discover risk factors from clinical data for disease progression, annotate whole slide imaging and decipher multiomics data. We delineate key examples of risk stratification and prognostication in acute kidney injury (AKI) and chronic kidney disease (CKD). We contextualize these applications in kidney disease oncology, one of the subfields to benefit demonstrably from artificial intelligence using all if these approaches. We conclude by elucidating technical challenges and ethical considerations and briefly considering future directions. SUMMARY The integration of clinical data, patient derived data, histology and proteomics and genomics can enhance the work of clinicians in providing more accurate diagnoses and elevating understanding of disease progression. Implementation research needs to be performed to translate these algorithms to the clinical setting.
Collapse
Affiliation(s)
- Joy Jiang
- Division of Data Driven and Digital Medicine (D3M), Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Lili Chan
- Division of Data Driven and Digital Medicine (D3M), Icahn School of Medicine at Mount Sinai, New York, New York, USA
- Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Girish N. Nadkarni
- Division of Data Driven and Digital Medicine (D3M), Icahn School of Medicine at Mount Sinai, New York, New York, USA
- Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- Mount Sinai Clinical Intelligence Center, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- The Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| |
Collapse
|
32
|
|
33
|
Hodgin JB, Mariani LH, Zee J, Liu Q, Smith AR, Eddy S, Hartman J, Hamidi H, Gaut JP, Palmer MB, Nast CC, Chang A, Hewitt S, Gillespie BW, Kretzler M, Holzman LB, Barisoni L. Quantification of Glomerular Structural Lesions: Associations With Clinical Outcomes and Transcriptomic Profiles in Nephrotic Syndrome. Am J Kidney Dis 2022; 79:807-819.e1. [PMID: 34864148 DOI: 10.1053/j.ajkd.2021.10.004] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2021] [Accepted: 10/20/2021] [Indexed: 12/18/2022]
Abstract
RATIONALE & OBJECTIVE The current classification system for focal segmental glomerulosclerosis (FSGS) and minimal change disease (MCD) does not fully capture the complex structural changes in kidney biopsies nor the clinical and molecular heterogeneity of these diseases. STUDY DESIGN Prospective observational cohort study. SETTING & PARTICIPANTS 221 MCD and FSGS patients enrolled in the Nephrotic Syndrome Study Network (NEPTUNE). EXPOSURE The NEPTUNE Digital Pathology Scoring System (NDPSS) was applied to generate scores for 37 glomerular descriptors. OUTCOME Time from biopsy to complete proteinuria remission, time from biopsy to kidney disease progression (40% estimated glomerular filtration rate [eGFR] decline or kidney failure), and eGFR over time. ANALYTICAL APPROACH Cluster analysis was used to group patients with similar morphologic characteristics. Glomerular descriptors and patient clusters were assessed for associations with outcomes using adjusted Cox models and linear mixed models. Messenger RNA from glomerular tissue was used to assess differentially expressed genes between clusters and identify genes associated with individual descriptors driving cluster membership. RESULTS Three clusters were identified: X (n = 56), Y (n = 68), and Z (n = 97). Clusters Y and Z had higher probabilities of proteinuria remission (HRs of 1.95 [95% CI, 0.99-3.85] and 3.29 [95% CI, 1.52-7.13], respectively), lower hazards of disease progression (HRs of 0.22 [95% CI, 0.08-0.57] and 0.11 [95% CI, 0.03-0.45], respectively), and lower loss of eGFR over time compared with X. Cluster X had 1,920 genes that were differentially expressed compared with Y+Z; these reflected activation of pathways of immune response and inflammation. Six descriptors driving the clusters individually correlated with clinical outcomes and gene expression. LIMITATIONS Low prevalence of some descriptors and biopsy at a single time point. CONCLUSIONS The NDPSS allows for categorization of FSGS/MCD patients into clinically and biologically relevant subgroups, and uncovers histologic parameters associated with clinical outcomes and molecular signatures not included in current classification systems.
Collapse
Affiliation(s)
- Jeffrey B Hodgin
- Renal Pathology, Department of Pathology, University of Michigan, Ann Arbor, Michigan.
| | - Laura H Mariani
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - Jarcy Zee
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania, Philadelphia, Pennsylvania; Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | - Qian Liu
- Arbor Research Collaborative for Health, Ann Arbor, Michigan, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Abigail R Smith
- Arbor Research Collaborative for Health, Ann Arbor, Michigan, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Sean Eddy
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - John Hartman
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - Habib Hamidi
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - Joseph P Gaut
- Department of Pathology and Immunology, and Internal Medicine, Washington University, St. Louis, Missouri
| | - Matthew B Palmer
- Department of Pathology, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Cynthia C Nast
- Department of Pathology, Cedars-Sinai Medical Center, Los Angeles, California
| | - Anthony Chang
- Department of Pathology, University of Chicago Medicine, Chicago, Illinois
| | - Stephen Hewitt
- Laboratory of Pathology, National Cancer Institute, National Institutes of Health, Bethesda, Maryland
| | - Brenda W Gillespie
- Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, Michigan
| | - Matthias Kretzler
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - Lawrence B Holzman
- Renal-Electrolyte and Hypertension Division, Department of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Laura Barisoni
- Department of Pathology, Division of AI & Computational Pathology, Duke University, Durham, North Carolina; Department of Medicine, Division of Nephrology, Duke University, Durham, North Carolina.
| |
Collapse
|
34
|
Allender F, Allègre R, Wemmert C, Dischler JM. Data augmentation based on spatial deformations for histopathology: An evaluation in the context of glomeruli segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 221:106919. [PMID: 35701252 DOI: 10.1016/j.cmpb.2022.106919] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Revised: 05/18/2022] [Accepted: 05/26/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE The effective application of deep learning to digital histopathology is hampered by the shortage of high-quality annotated images. In this paper we focus on the supervised segmentation of glomerular structures in patches of whole slide images of renal histopathological slides. Considering a U-Net model employed for segmentation, our goal is to evaluate the impact of augmenting training data with random spatial deformations. METHODS The effective application of deep learning to digital histopathology is hampered by the shortage of high-quality annotated images. In this paper we focus on the supervised segmentation of glomerular structures in patches of whole slide images of renal histopathological slides. Considering a U-Net model employed for segmentation, our goal is to evaluate the impact of augmenting training data with random spatial deformations. RESULTS We show that augmenting training data with spatially deformed images yields an improvement of up to 0.23 in average Dice score, with respect to training with no augmentation. We demonstrate that deformations with relatively strong distortions yield the best performance increase, while previous work only report the use of deformations with low distortions. The selected deformation models yield similar performance increase, provided that their parameters are properly adjusted. We provide bounds on the optimal parameter values, obtained through parameter sampling, which is achieved in a lower computational complexity with our single-parameter method. The paper is accompanied by a framework for evaluating the impact of random spatial deformations on the performance of any U-Net segmentation model. CONCLUSION To our knowledge, this study is the first to evaluate the impact of random spatial deformations on the segmentation of histopathological images. Our study and framework provide tools to help practitioners and researchers to make a better usage of random spatial deformations when training deep models for segmentation.
Collapse
|
35
|
Gupta L, Klinkhammer BM, Seikrit C, Fan N, Bouteldja N, Gräbel P, Gadermayr M, Boor P, Merhof D. Large-scale extraction of interpretable features provides new insights into kidney histopathology – a proof-of-concept study. J Pathol Inform 2022; 13:100097. [PMID: 36268111 PMCID: PMC9576990 DOI: 10.1016/j.jpi.2022.100097] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 04/14/2022] [Accepted: 05/02/2022] [Indexed: 11/21/2022] Open
Abstract
Whole slide images contain a magnitude of quantitative information that may not be fully explored in qualitative visual assessments. We propose: (1) a novel pipeline for extracting a comprehensive set of visual features, which are detectable by a pathologist, as well as sub-visual features, which are not discernible by human experts and (2) perform detailed analyses on renal images from mice with experimental unilateral ureteral obstruction. An important criterion for these features is that they are easy to interpret, as opposed to features obtained from neural networks. We extract and compare features from pathological and healthy control kidneys to learn how the compartments (glomerulus, Bowman's capsule, tubule, interstitium, artery, and arterial lumen) are affected by the pathology. We define feature selection methods to extract the most informative and discriminative features. We perform statistical analyses to understand the relation of the extracted features, both individually, and in combinations, with tissue morphology and pathology. Particularly for the presented case-study, we highlight features that are affected in each compartment. With this, prior biological knowledge, such as the increase in interstitial nuclei, is confirmed and presented in a quantitative way, alongside with novel findings, like color and intensity changes in glomeruli and Bowman's capsule. The proposed approach is therefore an important step towards quantitative, reproducible, and rater-independent analysis in histopathology.
Collapse
Affiliation(s)
- Laxmi Gupta
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
- Corresponding author.
| | | | - Claudia Seikrit
- Institute of Pathology, University Hospital Aachen, RWTH Aachen University, Aachen, Germany
- Division of Nephrology and Clinical Immunology, RWTH Aachen University, Aachen, Germany
| | - Nina Fan
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
| | - Nassim Bouteldja
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
- Institute of Pathology, University Hospital Aachen, RWTH Aachen University, Aachen, Germany
| | - Philipp Gräbel
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
| | - Michael Gadermayr
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
- Salzburg University of Applied Sciences, Puch/Salzburg, Austria
| | - Peter Boor
- Institute of Pathology, University Hospital Aachen, RWTH Aachen University, Aachen, Germany
| | - Dorit Merhof
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
| |
Collapse
|
36
|
Lee J, Warner E, Shaikhouni S, Bitzer M, Kretzler M, Gipson D, Pennathur S, Bellovich K, Bhat Z, Gadegbeku C, Massengill S, Perumal K, Saha J, Yang Y, Luo J, Zhang X, Mariani L, Hodgin JB, Rao A. Unsupervised machine learning for identifying important visual features through bag-of-words using histopathology data from chronic kidney disease. Sci Rep 2022; 12:4832. [PMID: 35318420 PMCID: PMC8941143 DOI: 10.1038/s41598-022-08974-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Accepted: 03/14/2022] [Indexed: 12/22/2022] Open
Abstract
Pathologists use visual classification to assess patient kidney biopsy samples when diagnosing the underlying cause of kidney disease. However, the assessment is qualitative, or semi-quantitative at best, and reproducibility is challenging. To discover previously unknown features which predict patient outcomes and overcome substantial interobserver variability, we developed an unsupervised bag-of-words model. Our study applied to the C-PROBE cohort of patients with chronic kidney disease (CKD). 107,471 histopathology images were obtained from 161 biopsy cores and identified important morphological features in biopsy tissue that are highly predictive of the presence of CKD both at the time of biopsy and in one year. To evaluate the performance of our model, we estimated the AUC and its 95% confidence interval. We show that this method is reliable and reproducible and can achieve 0.93 AUC at predicting glomerular filtration rate at the time of biopsy as well as predicting a loss of function at one year. Additionally, with this method, we ranked the identified morphological features according to their importance as diagnostic markers for chronic kidney disease. In this study, we have demonstrated the feasibility of using an unsupervised machine learning method without human input in order to predict the level of kidney function in CKD. The results from our study indicate that the visual dictionary, or visual image pattern, obtained from unsupervised machine learning can predict outcomes using machine-derived values that correspond to both known and unknown clinically relevant features.
Collapse
Affiliation(s)
- Joonsang Lee
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Elisa Warner
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Salma Shaikhouni
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Markus Bitzer
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Matthias Kretzler
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Debbie Gipson
- Department of Pediatrics, Pediatric Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Subramaniam Pennathur
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Keith Bellovich
- Department of Internal Medicine, Nephrology, St. Clair Nephrology Research, Detroit, MI, USA
| | - Zeenat Bhat
- Department of Internal Medicine, Nephrology, Wayne State University, Detroit, MI, USA
| | - Crystal Gadegbeku
- Department of Internal Medicine, Nephrology, Cleveland Clinic, Cleveland, OH, USA
| | - Susan Massengill
- Department of Pediatrics, Pediatric Nephrology, Levine Children's Hospital, Charlotte, NC, USA
| | - Kalyani Perumal
- Department of Internal Medicine, Nephrology, Department of JH Stroger Hospital, Chicago, IL, USA
| | - Jharna Saha
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - Yingbao Yang
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - Jinghui Luo
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - Xin Zhang
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Laura Mariani
- Department of Internal Medicine, Nephrology, University of Michigan, Ann Arbor, MI, USA
| | - Jeffrey B Hodgin
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA.
| | - Arvind Rao
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA.
- Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA.
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI, USA.
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.
| |
Collapse
|
37
|
Ren J, Hu Q, Wang W, Zhang W, Hubbard CS, Zhang P, An N, Zhou Y, Dahmani L, Wang D, Fu X, Sun Z, Wang Y, Wang R, Li L, Liu H. Fast cortical surface reconstruction from MRI using deep learning. Brain Inform 2022; 9:6. [PMID: 35262808 PMCID: PMC8907118 DOI: 10.1186/s40708-022-00155-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2022] [Accepted: 02/25/2022] [Indexed: 11/23/2022] Open
Abstract
Reconstructing cortical surfaces from structural magnetic resonance imaging (MRI) is a prerequisite for surface-based functional and anatomical image analyses. Conventional algorithms for cortical surface reconstruction are computationally inefficient and typically take several hours for each subject, causing a bottleneck in applications when a fast turnaround time is needed. To address this challenge, we propose a fast cortical surface reconstruction (FastCSR) pipeline by leveraging deep machine learning. We trained our model to learn an implicit representation of the cortical surface in volumetric space, termed the “level set representation”. A fast volumetric topology correction method and a topology-preserving surface mesh extraction procedure were employed to reconstruct the cortical surface based on the level set representation. Using 1-mm isotropic T1-weighted images, the FastCSR pipeline was able to reconstruct a subject’s cortical surfaces within 5 min with comparable surface quality, which is approximately 47 times faster than the traditional FreeSurfer pipeline. The advantage of FastCSR becomes even more apparent when processing high-resolution images. Importantly, the model demonstrated good generalizability in previously unseen data and showed high test–retest reliability in cortical morphometrics and anatomical parcellations. Finally, FastCSR was robust to images with compromised quality or with distortions caused by lesions. This fast and robust pipeline for cortical surface reconstruction may facilitate large-scale neuroimaging studies and has potential in clinical applications wherein brain images may be compromised.
Collapse
Affiliation(s)
- Jianxun Ren
- National Engineering Laboratory for Neuromodulation, School of Aerospace Engineering, Tsinghua University, Beijing, 100084, China.,Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, 02129, USA
| | - Qingyu Hu
- School of Computer Science and Technology, University of Science and Technology of China, Hefei, 230027, China
| | | | - Wei Zhang
- Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, 100080, China
| | - Catherine S Hubbard
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC, 29425, USA
| | | | - Ning An
- Neural Galaxy, Beijing, 102206, China
| | - Ying Zhou
- Neural Galaxy, Beijing, 102206, China
| | - Louisa Dahmani
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, 02129, USA
| | - Danhong Wang
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, 02129, USA
| | - Xiaoxuan Fu
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, 02129, USA.,Department of Neuroscience, Medical University of South Carolina, Charleston, SC, 29425, USA.,State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin, 300401, China
| | | | | | - Ruiqi Wang
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC, 29425, USA
| | - Luming Li
- National Engineering Laboratory for Neuromodulation, School of Aerospace Engineering, Tsinghua University, Beijing, 100084, China. .,Precision Medicine and Healthcare Research Center, Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen, 518055, China. .,IDG/McGovern Institute for Brain Research at Tsinghua University, Beijing, 100084, China. .,Beijing Neurosurgical Institute, Capital Medical University, Beijing, China.
| | - Hesheng Liu
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, 02129, USA. .,Department of Neuroscience, Medical University of South Carolina, Charleston, SC, 29425, USA.
| |
Collapse
|
38
|
Chen X, Huang H, Heidari AA, Sun C, Lv Y, Gui W, Liang G, Gu Z, Chen H, Li C, Chen P. An efficient multilevel thresholding image segmentation method based on the slime mould algorithm with bee foraging mechanism: A real case with lupus nephritis images. Comput Biol Med 2022; 142:105179. [DOI: 10.1016/j.compbiomed.2021.105179] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2021] [Revised: 12/24/2021] [Accepted: 12/24/2021] [Indexed: 02/01/2023]
|
39
|
Nguyen EH, Yang H, Deng R, Lu Y, Zhu Z, Roland JT, Lu L, Landman BA, Fogo AB, Huo Y. Circle Representation for Medical Object Detection. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:746-754. [PMID: 34699352 PMCID: PMC8963364 DOI: 10.1109/tmi.2021.3122835] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Box representation has been extensively used for object detection in computer vision. Such representation is efficacious but not necessarily optimized for biomedical objects (e.g., glomeruli), which play an essential role in renal pathology. In this paper, we propose a simple circle representation for medical object detection and introduce CircleNet, an anchor-free detection framework. Compared with the conventional bounding box representation, the proposed bounding circle representation innovates in three-fold: (1) it is optimized for ball-shaped biomedical objects; (2) The circle representation reduced the degree of freedom compared with box representation; (3) It is naturally more rotation invariant. When detecting glomeruli and nuclei on pathological images, the proposed circle representation achieved superior detection performance and be more rotation-invariant, compared with the bounding box. The code has been made publicly available: https://github.com/hrlblab/CircleNet.
Collapse
|
40
|
Glomerulus Semantic Segmentation Using Ensemble of Deep Learning Models. ARABIAN JOURNAL FOR SCIENCE AND ENGINEERING 2022. [DOI: 10.1007/s13369-022-06608-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
|
41
|
Combination of Transfer Learning Methods for Kidney Glomeruli Image Classification. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12031040] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
The rising global incidence of chronic kidney disease necessitates the development of image categorization of renal glomeruli. COVID-19 has been shown to enter the glomerulus, a tissue structure in the kidney. This study observes the differences between focal-segmental, normal and sclerotic renal glomerular tissue diseases. The splitting and combining of allied and multivariate models was accomplished utilizing a combined technique using existing models. In this study, model combinations are created by using a high-accuracy accuracy-based model to improve other models. This research exhibits excellent accuracy and consistent classification results on the ResNet101V2 combination using a mix of transfer learning methods, with the combined model on ResNet101V2 showing an accuracy of up to 97 percent with an F1-score of 0.97, compared to other models. However, this study discovered that the anticipated time required was higher than the model employed in general, which was mitigated by the usage of high-performance computing in this study.
Collapse
|
42
|
Identification of glomerulosclerosis using IBM Watson and shallow neural networks. J Nephrol 2022; 35:1235-1242. [PMID: 35041197 PMCID: PMC8765108 DOI: 10.1007/s40620-021-01200-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2021] [Accepted: 11/03/2021] [Indexed: 11/26/2022]
Abstract
Background Advanced stages of different renal diseases feature glomerular sclerosis at a histological level which is observed by light microscopy on tissue samples obtained by performing a kidney biopsy. Computer-aided diagnosis (CAD) systems leverage the potential of artificial intelligence (AI) in healthcare to support physicians in the diagnostic process. Methods We propose a novel CAD system that processes histological images and discriminates between sclerotic and non-sclerotic glomeruli. To this goal, we designed, tested, and compared two artificial neural network (ANN) classifiers. The former implements a shallow ANN classifying hand-crafted features extracted from Regions of Interest (ROIs) by means of image-processing procedures. The latter, instead, employs the IBM Watson Visual Recognition System, which uses a deep artificial neural network making decisions taking the images as input, without the need to design any procedure for describing images with features. The input dataset consisted of 428 sclerotic glomeruli and 2344 non-sclerotic glomeruli derived from images of kidney biopsies scanned by the Aperio ScanScope System. Results Both AI approaches allowed to very accurately distinguish (mean MCC 0.95 and mean Accuracy 0.99) between sclerotic and non-sclerotic glomeruli. Although the systems may seem interchangeable, the approach based on feature extraction and classification would allow clinicians to gain information on the most discriminating features. In fact, further procedures could explain the classifier’s decision by analysing which subset of features impacted the most on the final decision. Conclusions We developed a customizable support system that can facilitate the work of renal pathologists both in clinical and research settings. Graphical abstract ![]()
Supplementary Information The online version contains supplementary material available at 10.1007/s40620-021-01200-0.
Collapse
|
43
|
Noriaki S, Eiichiro U, Yasushi O. Artificial Intelligence in Kidney Pathology. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_181] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
44
|
Arthurs C, Roufosse C. Forging the tools for a computer-aided workflow in transplant pathology. Lancet Digit Health 2021; 4:e2-e3. [PMID: 34794931 DOI: 10.1016/s2589-7500(21)00254-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 10/29/2021] [Indexed: 10/19/2022]
Affiliation(s)
- Callum Arthurs
- Centre for Inflammatory Diseases, Department of Immunology and Inflammation, Imperial College London, London, UK; Department of Cellular Pathology, North West London Pathology, London, UK.
| | - Candice Roufosse
- Centre for Inflammatory Diseases, Department of Immunology and Inflammation, Imperial College London, London, UK; Department of Cellular Pathology, North West London Pathology, London, UK
| |
Collapse
|
45
|
Lee HN, Seo HD, Kim EM, Han BS, Kang JS. Classification of Mouse Lung Metastatic Tumor with Deep Learning. Biomol Ther (Seoul) 2021; 30:179-183. [PMID: 34725310 PMCID: PMC8902456 DOI: 10.4062/biomolther.2021.130] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Revised: 08/31/2021] [Accepted: 09/13/2021] [Indexed: 11/26/2022] Open
Abstract
Traditionally, pathologists microscopically examine tissue sections to detect pathological lesions; the many slides that must be evaluated impose severe work burdens. Also, diagnostic accuracy varies by pathologist training and experience; better diagnostic tools are required. Given the rapid development of computer vision, automated deep learning is now used to classify microscopic images, including medical images. Here, we used a Inception-v3 deep learning model to detect mouse lung metastatic tumors via whole slide imaging (WSI); we cropped the images to 151 by 151 pixels. The images were divided into training (53.8%) and test (46.2%) sets (21,017 and 18,016 images, respectively). When images from lung tissue containing tumor tissues were evaluated, the model accuracy was 98.76%. When images from normal lung tissue were evaluated, the model accuracy (“no tumor”) was 99.87%. Thus, the deep learning model distinguished metastatic lesions from normal lung tissue. Our approach will allow the rapid and accurate analysis of various tissues.
Collapse
Affiliation(s)
- Ha Neul Lee
- Department of Biomedical, Laboratory Science, Namseoul University, Cheonan 31020, Republic of Korea
| | - Hong-Deok Seo
- Department of Industrial Promotion, Spatial Information Industry Promotion Agency, Seongnam 13487, Republic of Korea
| | - Eui-Myoung Kim
- Department of Spatial Information Engineering, Namseoul University, Cheonan 31020, Republic of Korea
| | - Beom Seok Han
- Department of Pharmaceutical Engineering, Hoseo University, Asan 31499, Republic of Korea
| | - Jin Seok Kang
- Department of Biomedical, Laboratory Science, Namseoul University, Cheonan 31020, Republic of Korea
| |
Collapse
|
46
|
Li X, Davis RC, Xu Y, Wang Z, Souma N, Sotolongo G, Bell J, Ellis M, Howell D, Shen X, Lafata KJ, Barisoni L. Deep learning segmentation of glomeruli on kidney donor frozen sections. J Med Imaging (Bellingham) 2021; 8:067501. [PMID: 34950750 PMCID: PMC8685284 DOI: 10.1117/1.jmi.8.6.067501] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Accepted: 11/08/2021] [Indexed: 10/15/2023] Open
Abstract
Purpose: Recent advances in computational image analysis offer the opportunity to develop automatic quantification of histologic parameters as aid tools for practicing pathologists. We aim to develop deep learning (DL) models to quantify nonsclerotic and sclerotic glomeruli on frozen sections from donor kidney biopsies. Approach: A total of 258 whole slide images (WSI) from cadaveric donor kidney biopsies performed at our institution ( n = 123 ) and at external institutions ( n = 135 ) were used in this study. WSIs from our institution were divided at the patient level into training and validation datasets (ratio: 0.8:0.2), and external WSIs were used as an independent testing dataset. Nonsclerotic ( n = 22767 ) and sclerotic ( n = 1366 ) glomeruli were manually annotated by study pathologists on all WSIs. A nine-layer convolutional neural network based on the common U-Net architecture was developed and tested for the segmentation of nonsclerotic and sclerotic glomeruli. DL-derived, manual segmentation, and reported glomerular count (standard of care) were compared. Results: The average Dice similarity coefficient testing was 0.90 and 0.83. And the F 1 , recall, and precision scores were 0.93, 0.96, and 0.90, and 0.87, 0.93, and 0.81, for nonsclerotic and sclerotic glomeruli, respectively. DL-derived and manual segmentation-derived glomerular counts were comparable, but statistically different from reported glomerular count. Conclusions: DL segmentation is a feasible and robust approach for automatic quantification of glomeruli. We represent the first step toward new protocols for the evaluation of donor kidney biopsies.
Collapse
Affiliation(s)
- Xiang Li
- Duke University, Department of Electrical and Computer Engineering, Durham, North Carolina, United States
| | - Richard C. Davis
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
| | - Yuemei Xu
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
- Nanjing Drum Tower Hospital, Department of Pathology, Nanjing, China
| | - Zehan Wang
- Duke University, Department of Biomedical Engineering, Durham, North Carolina, United States
| | - Nao Souma
- Duke University, Department of Medicine, Division of Nephrology, Durham, North Carolina, United States
| | - Gina Sotolongo
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
| | - Jonathan Bell
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
| | - Matthew Ellis
- Duke University, Department of Medicine, Division of Nephrology, Durham, North Carolina, United States
- Duke University, Department of Surgery, Durham, North Carolina, United States
| | - David Howell
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
| | - Xiling Shen
- Duke University, Department of Biomedical Engineering, Durham, North Carolina, United States
| | - Kyle J. Lafata
- Duke University, Department of Electrical and Computer Engineering, Durham, North Carolina, United States
- Duke University, Department of Radiation Oncology, Durham, North Carolina, United States
- Duke University, Department of Radiology, Durham, North Carolina, United States
| | - Laura Barisoni
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
- Duke University, Department of Medicine, Division of Nephrology, Durham, North Carolina, United States
| |
Collapse
|
47
|
Yi Z, Salem F, Menon MC, Keung K, Xi C, Hultin S, Haroon Al Rasheed MR, Li L, Su F, Sun Z, Wei C, Huang W, Fredericks S, Lin Q, Banu K, Wong G, Rogers NM, Farouk S, Cravedi P, Shingde M, Smith RN, Rosales IA, O'Connell PJ, Colvin RB, Murphy B, Zhang W. Deep learning identified pathological abnormalities predictive of graft loss in kidney transplant biopsies. Kidney Int 2021; 101:288-298. [PMID: 34757124 DOI: 10.1016/j.kint.2021.09.028] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Revised: 08/12/2021] [Accepted: 09/09/2021] [Indexed: 10/19/2022]
Abstract
Interstitial fibrosis, tubular atrophy, and inflammation are major contributors to kidney allograft failure. Here we sought an objective, quantitative pathological assessment of these lesions to improve predictive utility and constructed a deep-learning-based pipeline recognizing normal vs. abnormal kidney tissue compartments and mononuclear leukocyte infiltrates. Periodic acid- Schiff stained slides of transplant biopsies (60 training and 33 testing) were used to quantify pathological lesions specific for interstitium, tubules and mononuclear leukocyte infiltration. The pipeline was applied to the whole slide images from 789 transplant biopsies (478 baseline [pre-implantation] and 311 post-transplant 12-month protocol biopsies) in two independent cohorts (GoCAR: 404 patients, AUSCAD: 212 patients) of transplant recipients to correlate composite lesion features with graft loss. Our model accurately recognized kidney tissue compartments and mononuclear leukocytes. The digital features significantly correlated with revised Banff 2007 scores but were more sensitive to subtle pathological changes below the thresholds in the Banff scores. The Interstitial and Tubular Abnormality Score (ITAS) in baseline samples was highly predictive of one-year graft loss, while a Composite Damage Score in 12-month post-transplant protocol biopsies predicted later graft loss. ITASs and Composite Damage Scores outperformed Banff scores or clinical predictors with superior graft loss prediction accuracy. High/intermediate risk groups stratified by ITASs or Composite Damage Scores also demonstrated significantly higher incidence of estimated glomerular filtration rate decline and subsequent graft damage. Thus, our deep-learning approach accurately detected and quantified pathological lesions from baseline or post-transplant biopsies and demonstrated superior ability for prediction of post-transplant graft loss with potential application as a prevention, risk stratification or monitoring tool.
Collapse
Affiliation(s)
- Zhengzi Yi
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Fadi Salem
- Pathology Division, Department of Molecular and Cell Based Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Madhav C Menon
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA; Nephrology Division, Department of Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - Karen Keung
- Centre for Transplant and Renal Research, Westmead Institute for Medical Research, University of Sydney, Sydney, New South Wales, Australia; Department of Nephrology, Prince of Wales Hospital, Sydney, New South Wales, Australia
| | - Caixia Xi
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Sebastian Hultin
- Centre for Transplant and Renal Research, Westmead Institute for Medical Research, University of Sydney, Sydney, New South Wales, Australia
| | - M Rizwan Haroon Al Rasheed
- Pathology Division, Department of Molecular and Cell Based Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Li Li
- Pathology Division, Department of Molecular and Cell Based Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Fei Su
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Zeguo Sun
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Chengguo Wei
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Weiqing Huang
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Samuel Fredericks
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Qisheng Lin
- Nephrology Division, Department of Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - Khadija Banu
- Nephrology Division, Department of Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - Germaine Wong
- Centre for Transplant and Renal Research, Westmead Institute for Medical Research, University of Sydney, Sydney, New South Wales, Australia
| | - Natasha M Rogers
- Centre for Transplant and Renal Research, Westmead Institute for Medical Research, University of Sydney, Sydney, New South Wales, Australia
| | - Samira Farouk
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Paolo Cravedi
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Meena Shingde
- Centre for Transplant and Renal Research, Westmead Institute for Medical Research, University of Sydney, Sydney, New South Wales, Australia
| | - R Neal Smith
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, USA
| | - Ivy A Rosales
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, USA
| | - Philip J O'Connell
- Centre for Transplant and Renal Research, Westmead Institute for Medical Research, University of Sydney, Sydney, New South Wales, Australia; Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia; Department of Nephrology, Westmead Hospital, Sydney, New South Wales, Australia
| | - Robert B Colvin
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, USA
| | - Barbara Murphy
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Weijia Zhang
- Renal Division, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA.
| |
Collapse
|
48
|
Zheng Z, Zhang X, Ding J, Zhang D, Cui J, Fu X, Han J, Zhu P. Deep Learning-Based Artificial Intelligence System for Automatic Assessment of Glomerular Pathological Findings in Lupus Nephritis. Diagnostics (Basel) 2021; 11:diagnostics11111983. [PMID: 34829330 PMCID: PMC8621095 DOI: 10.3390/diagnostics11111983] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Revised: 10/19/2021] [Accepted: 10/21/2021] [Indexed: 12/14/2022] Open
Abstract
Accurate assessment of renal histopathology is crucial for the clinical management of patients with lupus nephritis (LN). However, the current classification system has poor interpathologist agreement. This paper proposes a deep convolutional neural network (CNN)-based system that detects and classifies glomerular pathological findings in LN. A dataset of 349 renal biopsy whole-slide images (WSIs) (163 patients with LN, periodic acid-Schiff stain, 3906 glomeruli) annotated by three expert nephropathologists was used. The CNN models YOLOv4 and VGG16 were employed to localise the glomeruli and classify glomerular lesions (slight/severe impairments or sclerotic lesions). An additional 321 unannotated WSIs from 161 patients were used for performance evaluation at the per-patient kidney level. The proposed model achieved an accuracy of 0.951 and Cohen's kappa of 0.932 (95% CI 0.915-0.949) for the entire test set for classifying the glomerular lesions. For multiclass detection at the glomerular level, the mean average precision of the CNN was 0.807, with 'slight' and 'severe' glomerular lesions being easily identified (F1: 0.924 and 0.952, respectively). At the per-patient kidney level, the model achieved a high agreement with nephropathologist (linear weighted kappa: 0.855, 95% CI: 0.795-0.916, p < 0.001; quadratic weighted kappa: 0.906, 95% CI: 0.873-0.938, p < 0.001). The results suggest that deep learning is a feasible assistive tool for the objective and automatic assessment of pathological LN lesions.
Collapse
Affiliation(s)
- Zhaohui Zheng
- Department of Clinical Immunology, Xijing Hospital, Fourth Military Medical University, Xi’an 710032, China; (Z.Z.); (J.D.); (X.F.)
| | - Xiangsen Zhang
- School of Automation, Northwestern Polytechnical University, Xi’an 710072, China; (X.Z.); (D.Z.)
| | - Jin Ding
- Department of Clinical Immunology, Xijing Hospital, Fourth Military Medical University, Xi’an 710032, China; (Z.Z.); (J.D.); (X.F.)
| | - Dingwen Zhang
- School of Automation, Northwestern Polytechnical University, Xi’an 710072, China; (X.Z.); (D.Z.)
| | - Jihong Cui
- Lab of Tissue Engineering, College of Life Sciences, Northwest University, Xi’an 710069, China;
| | - Xianghui Fu
- Department of Clinical Immunology, Xijing Hospital, Fourth Military Medical University, Xi’an 710032, China; (Z.Z.); (J.D.); (X.F.)
| | - Junwei Han
- School of Automation, Northwestern Polytechnical University, Xi’an 710072, China; (X.Z.); (D.Z.)
- Correspondence: (J.H.); (P.Z.)
| | - Ping Zhu
- Department of Clinical Immunology, Xijing Hospital, Fourth Military Medical University, Xi’an 710032, China; (Z.Z.); (J.D.); (X.F.)
- Correspondence: (J.H.); (P.Z.)
| |
Collapse
|
49
|
Yang CK, Lee CY, Wang HS, Huang SC, Liang PI, Chen JS, Kuo CF, Tu KH, Yeh CY, Chen TD. Glomerular Disease Classification and Lesion Identification by Machine Learning. Biomed J 2021; 45:675-685. [PMID: 34506971 PMCID: PMC9486238 DOI: 10.1016/j.bj.2021.08.011] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Revised: 06/21/2021] [Accepted: 08/31/2021] [Indexed: 12/13/2022] Open
Abstract
Background Classification of glomerular diseases and identification of glomerular lesions require careful morphological examination by experienced nephropathologists, which is labor-intensive, time-consuming, and prone to interobserver variability. In this regard, recent advance in machine learning-based image analysis is promising. Methods We combined Mask Region-based Convolutional Neural Networks (Mask R–CNN) with an additional classification step to build a glomerulus detection model using human kidney biopsy samples. A Long Short-Term Memory (LSTM) recurrent neural network was applied for glomerular disease classification, and another two-stage model using ResNeXt-101 was constructed for glomerular lesion identification in cases of lupus nephritis. Results The detection model showed state-of-the-art performance on variedly stained slides with F1 scores up to 0.944. The disease classification model showed good accuracies up to 0.940 on recognizing different glomerular diseases based on H&E whole slide images. The lesion identification model demonstrated high discriminating power with area under the receiver operating characteristic curve up to 0.947 for various glomerular lesions. Models showed good generalization on external testing datasets. Conclusion This study is the first-of-its-kind showing how each step of kidney biopsy interpretation carried out by nephropathologists can be captured and simulated by machine learning models. The models were integrated into a whole slide image viewing and annotating platform to enable nephropathologists to review, correct, and confirm the inference results. Further improvement on model performances and incorporating inputs from immunofluorescence, electron microscopy, and clinical data might realize actual clinical use.
Collapse
Affiliation(s)
- Cheng-Kun Yang
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Ching-Yi Lee
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Hsiang-Sheng Wang
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Shun-Chen Huang
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Kaohsiung Branch, No. 123, Dapi Rd., Niaosong Dist., Kaohsiung City 833, Taiwan.
| | - Peir-In Liang
- Department of Pathology, Kaohsiung Medical University Hospital, No. 100, Ziyou 1st Rd., Sanmin Dist., Kaohsiung City 807, Taiwan.
| | - Jung-Sheng Chen
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Chang-Fu Kuo
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Kun-Hua Tu
- Department of Nephrology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Chao-Yuan Yeh
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Tai-Di Chen
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| |
Collapse
|
50
|
Sato N, Uchino E, Kojima R, Sakuragi M, Hiragi S, Minamiguchi S, Haga H, Yokoi H, Yanagita M, Okuno Y. Evaluation of Kidney Histological Images Using Unsupervised Deep Learning. Kidney Int Rep 2021; 6:2445-2454. [PMID: 34514205 PMCID: PMC8418980 DOI: 10.1016/j.ekir.2021.06.008] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2021] [Revised: 05/22/2021] [Accepted: 06/07/2021] [Indexed: 12/18/2022] Open
Abstract
INTRODUCTION Evaluating histopathology via machine learning has gained research and clinical interest, and the performance of supervised learning tasks has been described in various areas of medicine. Unsupervised learning of histological images has the advantage of reproducibility for labeling; however, the relationship between unsupervised evaluation and clinical information remains unclear in nephrology. METHODS We propose an unsupervised approach combining convolutional neural networks (CNNs) and a visualization algorithm to cluster the histological images and calculate the score for patients. We applied the approach to the entire images or patched images of the glomerulus of kidney biopsy samples stained with hematoxylin and eosin obtained from 68 patients with immunoglobulin A nephropathy. We assessed the relationship between the obtained scores and clinical variables of urinary occult blood, urinary protein, serum creatinine (SCr), systolic blood pressure, and age. RESULTS The glomeruli of the patients were classified into 12 distinct classes and 10 patches. The output of the fine-tuned CNN, which we defined as the histological scores, had significant relationships with assessed clinical variables. In addition, the clustering and visualization results suggested that the defined clusters captured important findings when evaluating renal histopathology. For the score of the patch-based cluster containing crescentic glomeruli, SCr (coefficient = 0.09, P = 0.019) had a significant relationship. CONCLUSION The proposed approach could successfully extract features that were related to the clinical variables from the kidney biopsy images along with the visualization for interpretability. The approach could aid in the quantified evaluation of renal histopathology.
Collapse
Affiliation(s)
- Noriaki Sato
- Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Eiichiro Uchino
- Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Ryosuke Kojima
- Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Minoru Sakuragi
- Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Shusuke Hiragi
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
- Division of Medical Informatics and Administration Planning, Kyoto University Hospital, Kyoto, Japan
| | - Sachiko Minamiguchi
- Department of Diagnostic Pathology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Hironori Haga
- Department of Diagnostic Pathology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Hideki Yokoi
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Motoko Yanagita
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
- Institute for the Advanced Study of Human Biology (ASHBi), Kyoto University, Kyoto, Japan
| | - Yasushi Okuno
- Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| |
Collapse
|