1
|
Juang CF, Chuang YW, Lin GW, Chung IF, Lo YC. Deep learning-based glomerulus detection and classification with generative morphology augmentation in renal pathology images. Comput Med Imaging Graph 2024; 115:102375. [PMID: 38599040 DOI: 10.1016/j.compmedimag.2024.102375] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 02/01/2024] [Accepted: 03/19/2024] [Indexed: 04/12/2024]
Abstract
Glomerulus morphology on renal pathology images provides valuable diagnosis and outcome prediction information. To provide better care, an efficient, standardized, and scalable method is urgently needed to optimize the time-consuming and labor-intensive interpretation process by renal pathologists. This paper proposes a deep convolutional neural network (CNN)-based approach to automatically detect and classify glomeruli with different stains in renal pathology images. In the glomerulus detection stage, this paper proposes a flattened Xception with a feature pyramid network (FX-FPN). The FX-FPN is employed as a backbone in the framework of faster region-based CNN to improve glomerulus detection performance. In the classification stage, this paper considers classifications of five glomerulus morphologies using a flattened Xception classifier. To endow the classifier with higher discriminability, this paper proposes a generative data augmentation approach for patch-based glomerulus morphology augmentation. New glomerulus patches of different morphologies are generated for data augmentation through the cycle-consistent generative adversarial network (CycleGAN). The single detection model shows the F1 score up to 0.9524 in H&E and PAS stains. The classification result shows that the average sensitivity and specificity are 0.7077 and 0.9316, respectively, by using the flattened Xception with the original training data. The sensitivity and specificity increase to 0.7623 and 0.9443, respectively, by using the generative data augmentation. Comparisons with different deep CNN models show the effectiveness and superiority of the proposed approach.
Collapse
Affiliation(s)
- Chia-Feng Juang
- Department of Electrical Engineering, National Chung Hsing University, Taichung 40227, Taiwan, ROC
| | - Ya-Wen Chuang
- Section of Nephrology, Department of Medicine, Taichung Veterans General Hospital, Taichung 40705, Taiwan, ROC; Graduate Institute of Biomedical Sciences, College of Medicine, China Medical University, Taichung 406040, Taiwan, ROC; School of Medicine, College of Medicine, China Medical University, Taichung 406040, Taiwan, ROC; Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taichung 40227, Taiwan, ROC
| | - Guan-Wen Lin
- Department of Electrical Engineering, National Chung Hsing University, Taichung 40227, Taiwan, ROC
| | - I-Fang Chung
- Institute of Biomedical Informatics, National Yang Ming Chiao Tung University, Taipei 112304, Taiwan, ROC.
| | - Ying-Chih Lo
- Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, MA 02115, USA; Harvard Medical School, Boston, MA 02115, USA.
| |
Collapse
|
2
|
Feng C, Ong K, Young DM, Chen B, Li L, Huo X, Lu H, Gu W, Liu F, Tang H, Zhao M, Yang M, Zhu K, Huang L, Wang Q, Marini GPL, Gui K, Han H, Sanders SJ, Li L, Yu W, Mao J. Artificial intelligence-assisted quantification and assessment of whole slide images for pediatric kidney disease diagnosis. Bioinformatics 2024; 40:btad740. [PMID: 38058211 PMCID: PMC10796177 DOI: 10.1093/bioinformatics/btad740] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Revised: 11/13/2023] [Accepted: 12/05/2023] [Indexed: 12/08/2023] Open
Abstract
MOTIVATION Pediatric kidney disease is a widespread, progressive condition that severely impacts growth and development of children. Chronic kidney disease is often more insidious in children than in adults, usually requiring a renal biopsy for diagnosis. Biopsy evaluation requires copious examination by trained pathologists, which can be tedious and prone to human error. In this study, we propose an artificial intelligence (AI) method to assist pathologists in accurate segmentation and classification of pediatric kidney structures, named as AI-based Pediatric Kidney Diagnosis (APKD). RESULTS We collected 2935 pediatric patients diagnosed with kidney disease for the development of APKD. The dataset comprised 93 932 histological structures annotated manually by three skilled nephropathologists. APKD scored an average accuracy of 94% for each kidney structure category, including 99% in the glomerulus. We found strong correlation between the model and manual detection in detected glomeruli (Spearman correlation coefficient r = 0.98, P < .001; intraclass correlation coefficient ICC = 0.98, 95% CI = 0.96-0.98). Compared to manual detection, APKD was approximately 5.5 times faster in segmenting glomeruli. Finally, we show how the pathological features extracted by APKD can identify focal abnormalities of the glomerular capillary wall to aid in the early diagnosis of pediatric kidney disease. AVAILABILITY AND IMPLEMENTATION https://github.com/ChunyueFeng/Kidney-DataSet.
Collapse
Affiliation(s)
- Chunyue Feng
- Department of Nephrology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou 310000, China
- National Clinical Research Center for Child Health, Hangzhou 310000, China
| | - Kokhaur Ong
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
| | - David M Young
- Institute of Molecular and Cell Biology, A*STAR, Singapore 138673, Singapore
- Department of Psychiatry and Behavioral Sciences, UCSF Weill Institute for Neurosciences, University of California, San Francisco, CA, 94143, United States
| | - Bingxian Chen
- Ningbo Konfoong Bioinformation Tech Co., Ltd., Ningbo 315000, China
| | - Longjie Li
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
| | - Xinmi Huo
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
| | - Haoda Lu
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
- Institute for AI in Medicine, Nanjing University of Information Science and Technology, Nanjing 210044, China
| | - Weizhong Gu
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Fei Liu
- Department of Nephrology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou 310000, China
- National Clinical Research Center for Child Health, Hangzhou 310000, China
| | - Hongfeng Tang
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Manli Zhao
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Min Yang
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Kun Zhu
- National Clinical Research Center for Child Health, Hangzhou 310000, China
- Department of Pathology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310000, China
| | - Limin Huang
- Department of Nephrology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou 310000, China
- National Clinical Research Center for Child Health, Hangzhou 310000, China
| | - Qiang Wang
- Ningbo Konfoong Bioinformation Tech Co., Ltd., Ningbo 315000, China
| | | | - Kun Gui
- Ningbo Konfoong Bioinformation Tech Co., Ltd., Ningbo 315000, China
| | - Hao Han
- Institute of Molecular and Cell Biology, A*STAR, Singapore 138673, Singapore
| | - Stephan J Sanders
- Department of Psychiatry and Behavioral Sciences, UCSF Weill Institute for Neurosciences, University of California, San Francisco, CA, 94143, United States
| | - Lin Li
- Department of Nephrology, Shanghai Changzheng Hospital, Naval Medical University, Shanghai 200003, China
| | - Weimiao Yu
- Bioinformatics Institute, A*STAR, Singapore 138673, Singapore
- Institute of Molecular and Cell Biology, A*STAR, Singapore 138673, Singapore
- Institute for AI in Medicine, Nanjing University of Information Science and Technology, Nanjing 210044, China
| | - Jianhua Mao
- Department of Nephrology, Children’s Hospital, Zhejiang University School of Medicine, Hangzhou 310000, China
- National Clinical Research Center for Child Health, Hangzhou 310000, China
| |
Collapse
|
3
|
Xia P, Lv Z, Wen Y, Zhang B, Zhao X, Zhang B, Wang Y, Cui H, Wang C, Zheng H, Qin Y, Sun L, Ye N, Cheng H, Yao L, Zhou H, Zhen J, Hu Z, Zhu W, Zhang F, Li X, Ren F, Chen L. Development of a multiple convolutional neural network-facilitated diagnostic screening program for immunofluorescence images of IgA nephropathy and idiopathic membranous nephropathy. Clin Kidney J 2023; 16:2503-2513. [PMID: 38046020 PMCID: PMC10689194 DOI: 10.1093/ckj/sfad153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Indexed: 12/05/2023] Open
Abstract
Background Immunoglobulin A nephropathy (IgAN) and idiopathic membranous nephropathy (IMN) are the most common glomerular diseases. Immunofluorescence (IF) tests of renal tissues are crucial for the diagnosis. We developed a multiple convolutional neural network (CNN)-facilitated diagnostic program to assist the IF diagnosis of IgAN and IMN. Methods The diagnostic program consisted of four parts: a CNN trained as a glomeruli detection module, an IF intensity comparator, dual-CNN (D-CNN) trained as a deposition appearance and location classifier and a post-processing module. A total of 1573 glomerular IF images from 1009 patients with glomerular diseases were used for the training and validation of the diagnostic program. A total of 1610 images of 426 patients from different hospitals were used as test datasets. The performance of the diagnostic program was compared with nephropathologists. Results In >90% of the tested images, the glomerulus location module achieved an intersection over union >0.8. The accuracy of the D-CNN in recognizing irregular granular mesangial deposition and fine granular deposition along the glomerular basement membrane was 96.1% and 93.3%, respectively. As for the diagnostic program, the accuracy, sensitivity and specificity of diagnosing suspected IgAN were 97.6%, 94.4% and 96.0%, respectively. The accuracy, sensitivity and specificity of diagnosing suspected IMN were 91.7%, 88.9% and 95.8%, respectively. The corresponding areas under the curve (AUCs) were 0.983 and 0.935. When tested with images from the outside hospital, the diagnostic program showed stable performance. The AUCs for diagnosing suspected IgAN and IMN were 0.972 and 0.948, respectively. Compared with inexperienced nephropathologists, the program showed better performance. Conclusion The proposed diagnostic program could assist the IF diagnosis of IgAN and IMN.
Collapse
Affiliation(s)
- Peng Xia
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Zhilong Lv
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
- School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China
| | - Yubing Wen
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | | | - Xuesong Zhao
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Boyao Zhang
- Beijing Zhijian Life Technology, Beijing, China
| | - Ying Wang
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Haoyuan Cui
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Chuanpeng Wang
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Hua Zheng
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yan Qin
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Lijun Sun
- Department of Nephrology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Nan Ye
- Department of Nephrology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Hong Cheng
- Department of Nephrology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Li Yao
- Department of Nephrology, First Hospital Affiliated to China Medical University, Shenyang, China
| | - Hua Zhou
- Department of Nephrology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Junhui Zhen
- Department of Pathology, Qilu Hospital of Shandong University, Jinan, China
| | - Zhao Hu
- Department of Nephrology, Qilu Hospital of Shandong University, Jinan, China
| | - Weiguo Zhu
- Department of Information, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Fa Zhang
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
| | - Xuemei Li
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Fei Ren
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
| | - Limeng Chen
- Department of Nephrology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
4
|
Liu X, Wu Y, Chen Y, Hui D, Zhang J, Hao F, Lu Y, Cheng H, Zeng Y, Han W, Wang C, Li M, Zhou X, Zheng W. Diagnosis of diabetic kidney disease in whole slide images via AI-driven quantification of pathological indicators. Comput Biol Med 2023; 166:107470. [PMID: 37722173 DOI: 10.1016/j.compbiomed.2023.107470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 08/29/2023] [Accepted: 09/04/2023] [Indexed: 09/20/2023]
Abstract
Diagnosis of diabetic kidney disease (DKD) mainly relies on screening the morphological variations and internal lesions of glomeruli from pathological kidney biopsy. The prominent pathological alterations of glomeruli for DKD include glomerular hypertrophy and nodular mesangial sclerosis. However, the qualitative judgment of these alterations is inaccurate and inconstant due to the intra- and inter-subject variability of pathologists. It is necessary to design artificial intelligence (AI) methods for accurate quantification of these pathological alterations and outcome prediction of DKD. In this work, we present an AI-driven framework to quantify the volume of glomeruli and degree of nodular mesangial sclerosis, respectively, based on an instance segmentation module and a novel weakly supervised Macro-Micro Aggregation (MMA) module. Subsequently, we construct classic machine learning models to predict the degree of DKD based on three selected pathological indicators via factor analysis. These corresponding modules are trained and tested on a total of 281 whole slide images (WSIs) digitized from two hospitals with different scanners. Our designed AI framework achieved inspiring results with 0.926 mIoU for glomerulus segmentation, and 0.899 F1 score for glomerulus classification in the external testing dataset. Meantime, the visualized results of the MMA module could reflect the location of the lesions. The performance of predicting disease achieved the F1 score of 0.917, which further proved the effectiveness of our AI-driven quantification of pathological indicators. Additionally, the interpretation of the machine learning model with the SHAP method showed similar accordance with the development of DKD in pathology. In conclusion, the proposed auxiliary diagnostic technologies have the feasibility for quantitative analysis of glomerular pathological tissues and alterations in DKD. Pathological quantitative indicators will also make it more convenient to provide doctors with assistance in clinical practice.
Collapse
Affiliation(s)
- Xueyu Liu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yongfei Wu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China.
| | - Yilin Chen
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Dongna Hui
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China
| | - Jianan Zhang
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Fang Hao
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yuanyue Lu
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China
| | - Hangbei Cheng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yue Zeng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Weixia Han
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Chen Wang
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Ming Li
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Xiaoshuang Zhou
- Department of Nephrology, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China.
| | - Wen Zheng
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China
| |
Collapse
|
5
|
Kaur G, Garg M, Gupta S, Juneja S, Rashid J, Gupta D, Shah A, Shaikh A. Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model. Diagnostics (Basel) 2023; 13:3152. [PMID: 37835895 PMCID: PMC10572820 DOI: 10.3390/diagnostics13193152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 09/23/2023] [Accepted: 09/25/2023] [Indexed: 10/15/2023] Open
Abstract
Glomeruli are interconnected capillaries in the renal cortex that are responsible for blood filtration. Damage to these glomeruli often signifies the presence of kidney disorders like glomerulonephritis and glomerulosclerosis, which can ultimately lead to chronic kidney disease and kidney failure. The timely detection of such conditions is essential for effective treatment. This paper proposes a modified UNet model to accurately detect glomeruli in whole-slide images of kidney tissue. The UNet model was modified by changing the number of filters and feature map dimensions from the first to the last layer to enhance the model's capacity for feature extraction. Moreover, the depth of the UNet model was also improved by adding one more convolution block to both the encoder and decoder sections. The dataset used in the study comprised 20 large whole-side images. Due to their large size, the images were cropped into 512 × 512-pixel patches, resulting in a dataset comprising 50,486 images. The proposed model performed well, with 95.7% accuracy, 97.2% precision, 96.4% recall, and 96.7% F1-score. These results demonstrate the proposed model's superior performance compared to the original UNet model, the UNet model with EfficientNetb3, and the current state-of-the-art. Based on these experimental findings, it has been determined that the proposed model accurately identifies glomeruli in extracted kidney patches.
Collapse
Affiliation(s)
- Gurjinder Kaur
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Meenu Garg
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Sheifali Gupta
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Sapna Juneja
- Kulliyyah of Information and Communication Technology, International Islamic University Malaysia, Kuala Lumpur 53100, Malaysia;
| | - Junaid Rashid
- Department of Data Science, Sejong University, Seoul 05006, Republic of Korea;
| | - Deepali Gupta
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India; (G.K.); (M.G.); (S.G.); (D.G.)
| | - Asadullah Shah
- Kulliyyah of Information and Communication Technology, International Islamic University Malaysia, Kuala Lumpur 53100, Malaysia;
| | - Asadullah Shaikh
- Department of Information Systems, College of Computer Science and Information Systems, Najran University, Najran 55461, Saudi Arabia;
| |
Collapse
|
6
|
Al-Thelaya K, Gilal NU, Alzubaidi M, Majeed F, Agus M, Schneider J, Househ M. Applications of discriminative and deep learning feature extraction methods for whole slide image analysis: A survey. J Pathol Inform 2023; 14:100335. [PMID: 37928897 PMCID: PMC10622844 DOI: 10.1016/j.jpi.2023.100335] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Revised: 07/17/2023] [Accepted: 07/19/2023] [Indexed: 11/07/2023] Open
Abstract
Digital pathology technologies, including whole slide imaging (WSI), have significantly improved modern clinical practices by facilitating storing, viewing, processing, and sharing digital scans of tissue glass slides. Researchers have proposed various artificial intelligence (AI) solutions for digital pathology applications, such as automated image analysis, to extract diagnostic information from WSI for improving pathology productivity, accuracy, and reproducibility. Feature extraction methods play a crucial role in transforming raw image data into meaningful representations for analysis, facilitating the characterization of tissue structures, cellular properties, and pathological patterns. These features have diverse applications in several digital pathology applications, such as cancer prognosis and diagnosis. Deep learning-based feature extraction methods have emerged as a promising approach to accurately represent WSI contents and have demonstrated superior performance in histology-related tasks. In this survey, we provide a comprehensive overview of feature extraction methods, including both manual and deep learning-based techniques, for the analysis of WSIs. We review relevant literature, analyze the discriminative and geometric features of WSIs (i.e., features suited to support the diagnostic process and extracted by "engineered" methods as opposed to AI), and explore predictive modeling techniques using AI and deep learning. This survey examines the advances, challenges, and opportunities in this rapidly evolving field, emphasizing the potential for accurate diagnosis, prognosis, and decision-making in digital pathology.
Collapse
Affiliation(s)
- Khaled Al-Thelaya
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Nauman Ullah Gilal
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Mahmood Alzubaidi
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Fahad Majeed
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Marco Agus
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Jens Schneider
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| | - Mowafa Househ
- Department of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
| |
Collapse
|
7
|
Liu Y, Lawson BC, Huang X, Broom BM, Weinstein JN. Prediction of Ovarian Cancer Response to Therapy Based on Deep Learning Analysis of Histopathology Images. Cancers (Basel) 2023; 15:4044. [PMID: 37627071 PMCID: PMC10452505 DOI: 10.3390/cancers15164044] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 08/06/2023] [Accepted: 08/07/2023] [Indexed: 08/27/2023] Open
Abstract
BACKGROUND Ovarian cancer remains the leading gynecological cause of cancer mortality. Predicting the sensitivity of ovarian cancer to chemotherapy at the time of pathological diagnosis is a goal of precision medicine research that we have addressed in this study using a novel deep-learning neural network framework to analyze the histopathological images. METHODS We have developed a method based on the Inception V3 deep learning algorithm that complements other methods for predicting response to standard platinum-based therapy of the disease. For the study, we used histopathological H&E images (pre-treatment) of high-grade serous carcinoma from The Cancer Genome Atlas (TCGA) Genomic Data Commons portal to train the Inception V3 convolutional neural network system to predict whether cancers had independently been labeled as sensitive or resistant to subsequent platinum-based chemotherapy. The trained model was then tested using data from patients left out of the training process. We used receiver operating characteristic (ROC) and confusion matrix analyses to evaluate model performance and Kaplan-Meier survival analysis to correlate the predicted probability of resistance with patient outcome. Finally, occlusion sensitivity analysis was piloted as a start toward correlating histopathological features with a response. RESULTS The study dataset consisted of 248 patients with stage 2 to 4 serous ovarian cancer. For a held-out test set of forty patients, the trained deep learning network model distinguished sensitive from resistant cancers with an area under the curve (AUC) of 0.846 ± 0.009 (SE). The probability of resistance calculated from the deep-learning network was also significantly correlated with patient survival and progression-free survival. In confusion matrix analysis, the network classifier achieved an overall predictive accuracy of 85% with a sensitivity of 73% and specificity of 90% for this cohort based on the Youden-J cut-off. Stage, grade, and patient age were not statistically significant for this cohort size. Occlusion sensitivity analysis suggested histopathological features learned by the network that may be associated with sensitivity or resistance to the chemotherapy, but multiple marker studies will be necessary to follow up on those preliminary results. CONCLUSIONS This type of analysis has the potential, if further developed, to improve the prediction of response to therapy of high-grade serous ovarian cancer and perhaps be useful as a factor in deciding between platinum-based and other therapies. More broadly, it may increase our understanding of the histopathological variables that predict response and may be adaptable to other cancer types and imaging modalities.
Collapse
Affiliation(s)
- Yuexin Liu
- Department of Bioinformatics and Computational Biology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA;
| | - Barrett C. Lawson
- Department of Pathology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA;
| | - Xuelin Huang
- Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA;
| | - Bradley M. Broom
- Department of Bioinformatics and Computational Biology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA;
| | - John N. Weinstein
- Department of Bioinformatics and Computational Biology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA;
- Department of Systems Biology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| |
Collapse
|
8
|
Jain Y, Godwin LL, Ju Y, Sood N, Quardokus EM, Bueckle A, Longacre T, Horning A, Lin Y, Esplin ED, Hickey JW, Snyder MP, Patterson NH, Spraggins JM, Börner K. Segmentation of human functional tissue units in support of a Human Reference Atlas. Commun Biol 2023; 6:717. [PMID: 37468557 PMCID: PMC10356924 DOI: 10.1038/s42003-023-04848-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 04/17/2023] [Indexed: 07/21/2023] Open
Abstract
The Human BioMolecular Atlas Program (HuBMAP) aims to compile a Human Reference Atlas (HRA) for the healthy adult body at the cellular level. Functional tissue units (FTUs), relevant for HRA construction, are of pathobiological significance. Manual segmentation of FTUs does not scale; highly accurate and performant, open-source machine-learning algorithms are needed. We designed and hosted a Kaggle competition that focused on development of such algorithms and 1200 teams from 60 countries participated. We present the competition outcomes and an expanded analysis of the winning algorithms on additional kidney and colon tissue data, and conduct a pilot study to understand spatial location and density of FTUs across the kidney. The top algorithm from the competition, Tom, outperforms other algorithms in the expanded study, while using fewer computational resources. Tom was added to the HuBMAP infrastructure to run kidney FTU segmentation at scale-showcasing the value of Kaggle competitions for advancing research.
Collapse
Affiliation(s)
- Yashvardhan Jain
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA.
| | - Leah L Godwin
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Yingnan Ju
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Naveksha Sood
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Ellen M Quardokus
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Andreas Bueckle
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA
| | - Teri Longacre
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | - Aaron Horning
- Thermo Fisher Scientific, South San Francisco, CA, 94080, USA
| | - Yiing Lin
- Department of Surgery, Washington University School of Medicine, St. Louis, MO, 63110, USA
| | - Edward D Esplin
- Department of Genetics, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | - John W Hickey
- Department of Microbiology & Immunology, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | - Michael P Snyder
- Department of Genetics, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | | | - Jeffrey M Spraggins
- Mass Spectrometry Research Center, Vanderbilt University, Nashville, TN, 37232, USA
- Department of Cell and Developmental Biology, Vanderbilt University, Nashville, TN, 37232, USA
| | - Katy Börner
- Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, 47408, USA.
| |
Collapse
|
9
|
Ram S, Tang W, Bell AJ, Pal R, Spencer C, Buschhaus A, Hatt CR, diMagliano MP, Rehemtulla A, Rodríguez JJ, Galban S, Galban CJ. Lung cancer lesion detection in histopathology images using graph-based sparse PCA network. Neoplasia 2023; 42:100911. [PMID: 37269818 DOI: 10.1016/j.neo.2023.100911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Accepted: 05/17/2023] [Indexed: 06/05/2023]
Abstract
Early detection of lung cancer is critical for improvement of patient survival. To address the clinical need for efficacious treatments, genetically engineered mouse models (GEMM) have become integral in identifying and evaluating the molecular underpinnings of this complex disease that may be exploited as therapeutic targets. Assessment of GEMM tumor burden on histopathological sections performed by manual inspection is both time consuming and prone to subjective bias. Therefore, an interplay of needs and challenges exists for computer-aided diagnostic tools, for accurate and efficient analysis of these histopathology images. In this paper, we propose a simple machine learning approach called the graph-based sparse principal component analysis (GS-PCA) network, for automated detection of cancerous lesions on histological lung slides stained by hematoxylin and eosin (H&E). Our method comprises four steps: 1) cascaded graph-based sparse PCA, 2) PCA binary hashing, 3) block-wise histograms, and 4) support vector machine (SVM) classification. In our proposed architecture, graph-based sparse PCA is employed to learn the filter banks of the multiple stages of a convolutional network. This is followed by PCA hashing and block histograms for indexing and pooling. The meaningful features extracted from this GS-PCA are then fed to an SVM classifier. We evaluate the performance of the proposed algorithm on H&E slides obtained from an inducible K-rasG12D lung cancer mouse model using precision/recall rates, Fβ-score, Tanimoto coefficient, and area under the curve (AUC) of the receiver operator characteristic (ROC) and show that our algorithm is efficient and provides improved detection accuracy compared to existing algorithms.
Collapse
Affiliation(s)
- Sundaresh Ram
- Departments of Radiology, and Biomedical Engineering, University of Michigan, Ann Arbor, MI 48109, USA.
| | - Wenfei Tang
- Department of Computer Science and Engineering, University of Michigan, Ann Arbor, MI 48109, USA
| | - Alexander J Bell
- Departments of Radiology, and Biomedical Engineering, University of Michigan, Ann Arbor, MI 48109, USA
| | - Ravi Pal
- Department of Radiology, University of Michigan, Ann Arbor, MI 48109, USA
| | - Cara Spencer
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI 48109, USA
| | | | - Charles R Hatt
- Department of Radiology, University of Michigan, Ann Arbor, MI 48109, USA; Imbio LLC, Minneapolis, MN 55405, USA
| | - Marina Pasca diMagliano
- Departments of Surgery, and Cell and Developmental Biology, University of Michigan, Ann Arbor, MI 48109, USA
| | - Alnawaz Rehemtulla
- Departments of Radiology, and Radiation Oncology, University of Michigan, Ann Arbor, MI 48109, USA
| | - Jeffrey J Rodríguez
- Departments of Electrical and Computer Engineering, and Biomedical Engineering, The University of Arizona, Tucson, AZ 85721, USA
| | - Stefanie Galban
- Department of Radiology, University of Michigan, Ann Arbor, MI 48109, USA
| | - Craig J Galban
- Departments of Radiology, and Biomedical Engineering, University of Michigan, Ann Arbor, MI 48109, USA
| |
Collapse
|
10
|
Wu B, Moeckel G. Application of digital pathology and machine learning in the liver, kidney and lung diseases. J Pathol Inform 2023; 14:100184. [PMID: 36714454 PMCID: PMC9874068 DOI: 10.1016/j.jpi.2022.100184] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 11/28/2022] [Accepted: 12/28/2022] [Indexed: 01/05/2023] Open
Abstract
The development of rapid and accurate Whole Slide Imaging (WSI) has paved the way for the application of Artificial Intelligence (AI) to digital pathology. The availability of WSI in the recent years allowed the rapid development of various AI technologies to blossom. WSI-based digital pathology combined with neural networks can automate arduous and time-consuming tasks of slide evaluation. Machine Learning (ML)-based AI has been demonstrated to outperform pathologists by eliminating inter- and intra-observer subjectivity, obtaining quantitative data from slide images, and extracting hidden image patterns that are relevant to disease subtype and progression. In this review, we outline the functionality of different AI technologies such as neural networks and deep learning and discover how aspects of different diseases make them benefit from the implementation of AI. AI has proven to be valuable in many different organs, with this review focusing on the liver, kidney, and lungs. We also discuss how AI and image analysis not only can grade diseases objectively but also discover aspects of diseases that have prognostic value. In the end, we review the current status of the integration of AI in pathology and share our vision on the future of digital pathology.
Collapse
Affiliation(s)
- Benjamin Wu
- Horace Mann School, Bronx, NY, USA,Corresponding author at: 950 Post Rd., Scarsdale, NY 10583, USA.
| | - Gilbert Moeckel
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
| |
Collapse
|
11
|
Kawazoe Y, Shimamoto K, Yamaguchi R, Nakamura I, Yoneda K, Shinohara E, Shintani-Domoto Y, Ushiku T, Tsukamoto T, Ohe K. Computational Pipeline for Glomerular Segmentation and Association of the Quantified Regions with Prognosis of Kidney Function in IgA Nephropathy. Diagnostics (Basel) 2022; 12:diagnostics12122955. [PMID: 36552963 PMCID: PMC9776670 DOI: 10.3390/diagnostics12122955] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 11/20/2022] [Accepted: 11/20/2022] [Indexed: 11/29/2022] Open
Abstract
The histopathological findings of the glomeruli from whole slide images (WSIs) of a renal biopsy play an important role in diagnosing and grading kidney disease. This study aimed to develop an automated computational pipeline to detect glomeruli and to segment the histopathological regions inside of the glomerulus in a WSI. In order to assess the significance of this pipeline, we conducted a multivariate regression analysis to determine whether the quantified regions were associated with the prognosis of kidney function in 46 cases of immunoglobulin A nephropathy (IgAN). The developed pipelines showed a mean intersection over union (IoU) of 0.670 and 0.693 for five classes (i.e., background, Bowman's space, glomerular tuft, crescentic, and sclerotic regions) against the WSI of its facility, and 0.678 and 0.609 against the WSI of the external facility. The multivariate analysis revealed that the predicted sclerotic regions, even those that were predicted by the external model, had a significant negative impact on the slope of the estimated glomerular filtration rate after biopsy. This is the first study to demonstrate that the quantified sclerotic regions that are predicted by an automated computational pipeline for the segmentation of the histopathological glomerular components on WSIs impact the prognosis of kidney function in patients with IgAN.
Collapse
Affiliation(s)
- Yoshimasa Kawazoe
- Artificial Intelligence in Healthcare, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
- Correspondence: ; Tel.: +81-3-5800-9077
| | - Kiminori Shimamoto
- Artificial Intelligence in Healthcare, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Ryohei Yamaguchi
- Ohshima Memorial Kisen Hospital, 3-5-15, Misaki, Chiba 274-0812, Japan
| | - Issei Nakamura
- NTT DOCOMO, Inc., Sanno Park Tower, 2-11-1, Nagata-cho, Chiyoda-ku, Tokyo 100-6150, Japan
| | - Kota Yoneda
- Department of Reproductive, Developmental, and Aging Sciences, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Emiko Shinohara
- Artificial Intelligence in Healthcare, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Yukako Shintani-Domoto
- Department of Diagnostic Pathology, Nippon Medical School Hospital, 1-1-5, Sendagi, Bunkyo-ku, Tokyo 113-8602, Japan
| | - Tetsuo Ushiku
- Department of Pathology, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Tatsuo Tsukamoto
- Department of Nephrology and Dialysis, Tazuke Kofukai Medical Research Institute, Kitano Hospital, 2-4-20, Ohgimachi, Kita-ku, Osaka 530-8480, Japan
| | - Kazuhiko Ohe
- Department of Biomedical Informatics, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| |
Collapse
|
12
|
Artificial Intelligence-Assisted Renal Pathology: Advances and Prospects. J Clin Med 2022; 11:jcm11164918. [PMID: 36013157 PMCID: PMC9410196 DOI: 10.3390/jcm11164918] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 07/30/2022] [Accepted: 08/11/2022] [Indexed: 11/17/2022] Open
Abstract
Digital imaging and advanced microscopy play a pivotal role in the diagnosis of kidney diseases. In recent years, great achievements have been made in digital imaging, providing novel approaches for precise quantitative assessments of nephropathology and relieving burdens of renal pathologists. Developing novel methods of artificial intelligence (AI)-assisted technology through multidisciplinary interaction among computer engineers, renal specialists, and nephropathologists could prove beneficial for renal pathology diagnoses. An increasing number of publications has demonstrated the rapid growth of AI-based technology in nephrology. In this review, we offer an overview of AI-assisted renal pathology, including AI concepts and the workflow of processing digital image data, focusing on the impressive advances of AI application in disease-specific backgrounds. In particular, this review describes the applied computer vision algorithms for the segmentation of kidney structures, diagnosis of specific pathological changes, and prognosis prediction based on images. Lastly, we discuss challenges and prospects to provide an objective view of this topic.
Collapse
|
13
|
Lutnick B, Manthey D, Becker JU, Ginley B, Moos K, Zuckerman JE, Rodrigues L, Gallan AJ, Barisoni L, Alpers CE, Wang XX, Myakala K, Jones BA, Levi M, Kopp JB, Yoshida T, Zee J, Han SS, Jain S, Rosenberg AZ, Jen KY, Sarder P. A user-friendly tool for cloud-based whole slide image segmentation with examples from renal histopathology. COMMUNICATIONS MEDICINE 2022; 2:105. [PMID: 35996627 PMCID: PMC9391340 DOI: 10.1038/s43856-022-00138-z] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2021] [Accepted: 06/09/2022] [Indexed: 01/21/2023] Open
Abstract
Background Image-based machine learning tools hold great promise for clinical applications in pathology research. However, the ideal end-users of these computational tools (e.g., pathologists and biological scientists) often lack the programming experience required for the setup and use of these tools which often rely on the use of command line interfaces. Methods We have developed Histo-Cloud, a tool for segmentation of whole slide images (WSIs) that has an easy-to-use graphical user interface. This tool runs a state-of-the-art convolutional neural network (CNN) for segmentation of WSIs in the cloud and allows the extraction of features from segmented regions for further analysis. Results By segmenting glomeruli, interstitial fibrosis and tubular atrophy, and vascular structures from renal and non-renal WSIs, we demonstrate the scalability, best practices for transfer learning, and effects of dataset variability. Finally, we demonstrate an application for animal model research, analyzing glomerular features in three murine models. Conclusions Histo-Cloud is open source, accessible over the internet, and adaptable for segmentation of any histological structure regardless of stain.
Collapse
Affiliation(s)
- Brendon Lutnick
- Department of Pathology and Anatomical Sciences, SUNY Buffalo, Buffalo, USA
| | | | - Jan U. Becker
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Brandon Ginley
- Department of Pathology and Anatomical Sciences, SUNY Buffalo, Buffalo, USA
| | - Katharina Moos
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Jonathan E. Zuckerman
- Department of Pathology and Laboratory Medicine, University of California at Los Angeles, Los Angeles, USA
| | - Luis Rodrigues
- University Clinic of Nephrology, Faculty of Medicine, University of Coimbra, Coimbra, Portugal
| | | | - Laura Barisoni
- Departments of Pathology and Medicine, Duke University, Durham, USA
| | - Charles E. Alpers
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, USA
| | - Xiaoxin X. Wang
- Departments of Biochemistry and Molecular & Cellular Biology, Georgetown University, Washington, DC USA
| | - Komuraiah Myakala
- Departments of Biochemistry and Molecular & Cellular Biology, Georgetown University, Washington, DC USA
| | - Bryce A. Jones
- Department of Pharmacology and Physiology, Georgetown University, Washington, DC USA
| | - Moshe Levi
- Departments of Biochemistry and Molecular & Cellular Biology, Georgetown University, Washington, DC USA
| | | | | | - Jarcy Zee
- Department of Biostatistics, Epidemiology, & Informatics, University of Pennsylvania, Philadelphia, USA
| | - Seung Seok Han
- Department of Internal Medicine, Seoul National University College of Medicine, Seoul, South Korea
| | - Sanjay Jain
- Department of Medicine, Nephrology, Washington University School of Medicine, St. Louis, USA
| | - Avi Z. Rosenberg
- Department of Pathology, Johns Hopkins University, Baltimore, USA
| | - Kuang Yu. Jen
- Department of Pathology and Laboratory Medicine, University of California at Davis, Sacramento, USA
| | - Pinaki Sarder
- Department of Pathology and Anatomical Sciences, SUNY Buffalo, Buffalo, USA
| |
Collapse
|
14
|
Hodgin JB, Mariani LH, Zee J, Liu Q, Smith AR, Eddy S, Hartman J, Hamidi H, Gaut JP, Palmer MB, Nast CC, Chang A, Hewitt S, Gillespie BW, Kretzler M, Holzman LB, Barisoni L. Quantification of Glomerular Structural Lesions: Associations With Clinical Outcomes and Transcriptomic Profiles in Nephrotic Syndrome. Am J Kidney Dis 2022; 79:807-819.e1. [PMID: 34864148 DOI: 10.1053/j.ajkd.2021.10.004] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2021] [Accepted: 10/20/2021] [Indexed: 12/18/2022]
Abstract
RATIONALE & OBJECTIVE The current classification system for focal segmental glomerulosclerosis (FSGS) and minimal change disease (MCD) does not fully capture the complex structural changes in kidney biopsies nor the clinical and molecular heterogeneity of these diseases. STUDY DESIGN Prospective observational cohort study. SETTING & PARTICIPANTS 221 MCD and FSGS patients enrolled in the Nephrotic Syndrome Study Network (NEPTUNE). EXPOSURE The NEPTUNE Digital Pathology Scoring System (NDPSS) was applied to generate scores for 37 glomerular descriptors. OUTCOME Time from biopsy to complete proteinuria remission, time from biopsy to kidney disease progression (40% estimated glomerular filtration rate [eGFR] decline or kidney failure), and eGFR over time. ANALYTICAL APPROACH Cluster analysis was used to group patients with similar morphologic characteristics. Glomerular descriptors and patient clusters were assessed for associations with outcomes using adjusted Cox models and linear mixed models. Messenger RNA from glomerular tissue was used to assess differentially expressed genes between clusters and identify genes associated with individual descriptors driving cluster membership. RESULTS Three clusters were identified: X (n = 56), Y (n = 68), and Z (n = 97). Clusters Y and Z had higher probabilities of proteinuria remission (HRs of 1.95 [95% CI, 0.99-3.85] and 3.29 [95% CI, 1.52-7.13], respectively), lower hazards of disease progression (HRs of 0.22 [95% CI, 0.08-0.57] and 0.11 [95% CI, 0.03-0.45], respectively), and lower loss of eGFR over time compared with X. Cluster X had 1,920 genes that were differentially expressed compared with Y+Z; these reflected activation of pathways of immune response and inflammation. Six descriptors driving the clusters individually correlated with clinical outcomes and gene expression. LIMITATIONS Low prevalence of some descriptors and biopsy at a single time point. CONCLUSIONS The NDPSS allows for categorization of FSGS/MCD patients into clinically and biologically relevant subgroups, and uncovers histologic parameters associated with clinical outcomes and molecular signatures not included in current classification systems.
Collapse
Affiliation(s)
- Jeffrey B Hodgin
- Renal Pathology, Department of Pathology, University of Michigan, Ann Arbor, Michigan.
| | - Laura H Mariani
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - Jarcy Zee
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania, Philadelphia, Pennsylvania; Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | - Qian Liu
- Arbor Research Collaborative for Health, Ann Arbor, Michigan, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Abigail R Smith
- Arbor Research Collaborative for Health, Ann Arbor, Michigan, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Sean Eddy
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - John Hartman
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - Habib Hamidi
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - Joseph P Gaut
- Department of Pathology and Immunology, and Internal Medicine, Washington University, St. Louis, Missouri
| | - Matthew B Palmer
- Department of Pathology, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Cynthia C Nast
- Department of Pathology, Cedars-Sinai Medical Center, Los Angeles, California
| | - Anthony Chang
- Department of Pathology, University of Chicago Medicine, Chicago, Illinois
| | - Stephen Hewitt
- Laboratory of Pathology, National Cancer Institute, National Institutes of Health, Bethesda, Maryland
| | - Brenda W Gillespie
- Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, Michigan
| | - Matthias Kretzler
- Division of Nephrology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan
| | - Lawrence B Holzman
- Renal-Electrolyte and Hypertension Division, Department of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Laura Barisoni
- Department of Pathology, Division of AI & Computational Pathology, Duke University, Durham, North Carolina; Department of Medicine, Division of Nephrology, Duke University, Durham, North Carolina.
| |
Collapse
|
15
|
Gupta L, Klinkhammer BM, Seikrit C, Fan N, Bouteldja N, Gräbel P, Gadermayr M, Boor P, Merhof D. Large-scale extraction of interpretable features provides new insights into kidney histopathology – a proof-of-concept study. J Pathol Inform 2022; 13:100097. [PMID: 36268111 PMCID: PMC9576990 DOI: 10.1016/j.jpi.2022.100097] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 04/14/2022] [Accepted: 05/02/2022] [Indexed: 11/21/2022] Open
Abstract
Whole slide images contain a magnitude of quantitative information that may not be fully explored in qualitative visual assessments. We propose: (1) a novel pipeline for extracting a comprehensive set of visual features, which are detectable by a pathologist, as well as sub-visual features, which are not discernible by human experts and (2) perform detailed analyses on renal images from mice with experimental unilateral ureteral obstruction. An important criterion for these features is that they are easy to interpret, as opposed to features obtained from neural networks. We extract and compare features from pathological and healthy control kidneys to learn how the compartments (glomerulus, Bowman's capsule, tubule, interstitium, artery, and arterial lumen) are affected by the pathology. We define feature selection methods to extract the most informative and discriminative features. We perform statistical analyses to understand the relation of the extracted features, both individually, and in combinations, with tissue morphology and pathology. Particularly for the presented case-study, we highlight features that are affected in each compartment. With this, prior biological knowledge, such as the increase in interstitial nuclei, is confirmed and presented in a quantitative way, alongside with novel findings, like color and intensity changes in glomeruli and Bowman's capsule. The proposed approach is therefore an important step towards quantitative, reproducible, and rater-independent analysis in histopathology.
Collapse
Affiliation(s)
- Laxmi Gupta
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
- Corresponding author.
| | | | - Claudia Seikrit
- Institute of Pathology, University Hospital Aachen, RWTH Aachen University, Aachen, Germany
- Division of Nephrology and Clinical Immunology, RWTH Aachen University, Aachen, Germany
| | - Nina Fan
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
| | - Nassim Bouteldja
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
- Institute of Pathology, University Hospital Aachen, RWTH Aachen University, Aachen, Germany
| | - Philipp Gräbel
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
| | - Michael Gadermayr
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
- Salzburg University of Applied Sciences, Puch/Salzburg, Austria
| | - Peter Boor
- Institute of Pathology, University Hospital Aachen, RWTH Aachen University, Aachen, Germany
| | - Dorit Merhof
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
| |
Collapse
|
16
|
Girolami I, Pantanowitz L, Marletta S, Hermsen M, van der Laak J, Munari E, Furian L, Vistoli F, Zaza G, Cardillo M, Gesualdo L, Gambaro G, Eccher A. Artificial intelligence applications for pre-implantation kidney biopsy pathology practice: a systematic review. J Nephrol 2022; 35:1801-1808. [PMID: 35441256 PMCID: PMC9458558 DOI: 10.1007/s40620-022-01327-8] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2022] [Accepted: 03/28/2022] [Indexed: 10/29/2022]
Abstract
BACKGROUND Transplant nephropathology is a highly specialized field of pathology comprising both the evaluation of organ donor biopsy for organ allocation and post-transplant graft biopsy for assessment of rejection or graft damage. The introduction of digital pathology with whole-slide imaging (WSI) in clinical research, trials and practice has catalyzed the application of artificial intelligence (AI) for histopathology, with development of novel machine-learning models for tissue interrogation and discovery. We aimed to review the literature for studies specifically applying AI algorithms to WSI-digitized pre-implantation kidney biopsy. METHODS A systematic search was carried out in the electronic databases PubMed-MEDLINE and Embase until 25th September, 2021 with a combination of the key terms "kidney", "biopsy", "transplantation" and "artificial intelligence" and their aliases. Studies dealing with the application of AI algorithms coupled with WSI in pre-implantation kidney biopsies were included. The main theme addressed was detection and quantification of tissue components. Extracted data were: author, year and country of the study, type of biopsy features investigated, number of cases, type of algorithm deployed, main results of the study in terms of diagnostic outcome, and the main limitations of the study. RESULTS Of 5761 retrieved articles, 7 met our inclusion criteria. All studies focused largely on AI-based detection and classification of glomerular structures and to a lesser extent on tubular and vascular structures. Performance of AI algorithms was excellent and promising. CONCLUSION All studies highlighted the importance of expert pathologist annotation to reliably train models and the need to acknowledge clinical nuances of the pre-implantation setting. Close cooperation between computer scientists and practicing as well as expert renal pathologists is needed, helping to refine the performance of AI-based models for routine pre-implantation kidney biopsy clinical practice.
Collapse
Affiliation(s)
- Ilaria Girolami
- Division of Pathology, Central Hospital Bolzano, Bolzano, Italy
| | - Liron Pantanowitz
- Department of Pathology and Clinical Labs, University of Michigan, Ann Arbor, MI, USA
| | - Stefano Marletta
- Department of Diagnostics and Public Health, University and Hospital Trust of Verona, Verona, Italy
| | - Meyke Hermsen
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Enrico Munari
- Pathology Unit, Department of Molecular and Translational Medicine, Spedali Civili-University of Brescia, Brescia, Italy
| | - Lucrezia Furian
- Department of Surgical, Oncological and Gastroenterological Sciences, Unit of Kidney and Pancreas Transplantation, University of Padua, Padua, Italy
| | - Fabio Vistoli
- Division of General and Transplant Surgery, University of Pisa, Pisa, Italy
| | - Gianluigi Zaza
- Department of Nephro-Urology, Nephrology, Dialysis and Transplant Unit, University of Foggia, Foggia, Italy
| | | | - Loreto Gesualdo
- Nephrology, Dialysis, and Transplantation Unit, Department of Emergency and Organ Transplantation, University of Bari Aldo Moro, Bari, Italy
| | - Giovanni Gambaro
- Department of General Medicine, Renal Unit, University and Hospital Trust of Verona, Verona, Italy
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, P.le Stefani n. 1, 37126, Verona, Italy.
| |
Collapse
|
17
|
Noriaki S, Eiichiro U, Yasushi O. Artificial Intelligence in Kidney Pathology. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_181] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
18
|
Mewada H, Al-Asad JF, Patel A, Chaudhari J, Mahant K, Vala A. Multi-Channel Local Binary Pattern Guided Convolutional Neural Network for Breast Cancer Classification. Open Biomed Eng J 2021. [DOI: 10.2174/1874120702115010132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Background:
The advancement in convolutional neural network (CNN) has reduced the burden of experts using the computer-aided diagnosis of human breast cancer. However, most CNN networks use spatial features only. The inherent texture structure present in histopathological images plays an important role in distinguishing malignant tissues. This paper proposes an alternate CNN network that integrates Local Binary Pattern (LBP) based texture information with CNN features.
Methods:
The study propagates that LBP provides the most robust rotation, and translation-invariant features in comparison with other texture feature extractors. Therefore, a formulation of LBP in context of convolution operation is presented and used in the proposed CNN network. A non-trainable fixed set binary convolutional filters representing LBP features are combined with trainable convolution filters to approximate the response of the convolution layer. A CNN architecture guided by LBP features is used to classify the histopathological images.
Result:
The network is trained using BreKHis datasets. The use of a fixed set of LBP filters reduces the burden of CNN by minimizing training parameters by a factor of 9. This makes it suitable for the environment with fewer resources. The proposed network obtained 96.46% of maximum accuracy with 98.51% AUC and 97% F1-score.
Conclusion:
LBP based texture information plays a vital role in cancer image classification. A multi-channel LBP futures fusion is used in the CNN network. The experiment results propagate that the new structure of LBP-guided CNN requires fewer training parameters preserving the capability of the CNN network’s classification accuracy.
Collapse
|
19
|
Li X, Davis RC, Xu Y, Wang Z, Souma N, Sotolongo G, Bell J, Ellis M, Howell D, Shen X, Lafata KJ, Barisoni L. Deep learning segmentation of glomeruli on kidney donor frozen sections. J Med Imaging (Bellingham) 2021; 8:067501. [PMID: 34950750 PMCID: PMC8685284 DOI: 10.1117/1.jmi.8.6.067501] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Accepted: 11/08/2021] [Indexed: 10/15/2023] Open
Abstract
Purpose: Recent advances in computational image analysis offer the opportunity to develop automatic quantification of histologic parameters as aid tools for practicing pathologists. We aim to develop deep learning (DL) models to quantify nonsclerotic and sclerotic glomeruli on frozen sections from donor kidney biopsies. Approach: A total of 258 whole slide images (WSI) from cadaveric donor kidney biopsies performed at our institution ( n = 123 ) and at external institutions ( n = 135 ) were used in this study. WSIs from our institution were divided at the patient level into training and validation datasets (ratio: 0.8:0.2), and external WSIs were used as an independent testing dataset. Nonsclerotic ( n = 22767 ) and sclerotic ( n = 1366 ) glomeruli were manually annotated by study pathologists on all WSIs. A nine-layer convolutional neural network based on the common U-Net architecture was developed and tested for the segmentation of nonsclerotic and sclerotic glomeruli. DL-derived, manual segmentation, and reported glomerular count (standard of care) were compared. Results: The average Dice similarity coefficient testing was 0.90 and 0.83. And the F 1 , recall, and precision scores were 0.93, 0.96, and 0.90, and 0.87, 0.93, and 0.81, for nonsclerotic and sclerotic glomeruli, respectively. DL-derived and manual segmentation-derived glomerular counts were comparable, but statistically different from reported glomerular count. Conclusions: DL segmentation is a feasible and robust approach for automatic quantification of glomeruli. We represent the first step toward new protocols for the evaluation of donor kidney biopsies.
Collapse
Affiliation(s)
- Xiang Li
- Duke University, Department of Electrical and Computer Engineering, Durham, North Carolina, United States
| | - Richard C. Davis
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
| | - Yuemei Xu
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
- Nanjing Drum Tower Hospital, Department of Pathology, Nanjing, China
| | - Zehan Wang
- Duke University, Department of Biomedical Engineering, Durham, North Carolina, United States
| | - Nao Souma
- Duke University, Department of Medicine, Division of Nephrology, Durham, North Carolina, United States
| | - Gina Sotolongo
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
| | - Jonathan Bell
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
| | - Matthew Ellis
- Duke University, Department of Medicine, Division of Nephrology, Durham, North Carolina, United States
- Duke University, Department of Surgery, Durham, North Carolina, United States
| | - David Howell
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
| | - Xiling Shen
- Duke University, Department of Biomedical Engineering, Durham, North Carolina, United States
| | - Kyle J. Lafata
- Duke University, Department of Electrical and Computer Engineering, Durham, North Carolina, United States
- Duke University, Department of Radiation Oncology, Durham, North Carolina, United States
- Duke University, Department of Radiology, Durham, North Carolina, United States
| | - Laura Barisoni
- Duke University, Department of Pathology, Division of AI and Computational Pathology, Durham, North Carolina, United States
- Duke University, Department of Medicine, Division of Nephrology, Durham, North Carolina, United States
| |
Collapse
|
20
|
Cascarano GD, Debitonto FS, Lemma R, Brunetti A, Buongiorno D, De Feudis I, Guerriero A, Venere U, Matino S, Rocchetti MT, Rossini M, Pesce F, Gesualdo L, Bevilacqua V. A neural network for glomerulus classification based on histological images of kidney biopsy. BMC Med Inform Decis Mak 2021; 21:300. [PMID: 34724926 PMCID: PMC8559346 DOI: 10.1186/s12911-021-01650-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2021] [Accepted: 10/06/2021] [Indexed: 11/26/2022] Open
Abstract
Background Computer-aided diagnosis (CAD) systems based on medical images could support physicians in the decision-making process. During the last decades, researchers have proposed CAD systems in several medical domains achieving promising results.
CAD systems play an important role in digital pathology supporting pathologists in analyzing biopsy slides by means of standardized and objective workflows. In the proposed work, we designed and tested a novel CAD system module based on image processing techniques and machine learning, whose objective was to classify the condition affecting renal corpuscles (glomeruli) between sclerotic and non-sclerotic. Such discrimination is useful for the biopsy slides evaluation performed by pathologists. Results We collected 26 digital slides taken from the kidneys of 19 donors with Periodic Acid-Schiff staining. Expert pathologists have conducted the slides preparation, digital acquisition and glomeruli annotations. Before setting the classifiers, we evaluated several feature extraction techniques from the annotated regions. Then, a feature reduction procedure followed by a shallow artificial neural network allowed discriminating between the glomeruli classes.
We evaluated the workflow considering an independent dataset (i.e., processing images not used in the training procedure). Ten independent runs of the training algorithm, and evaluation, allowed achieving MCC and Accuracy of 0.95 (± 0.01) and 0.99 (standard deviation < 0.00), respectively. We also obtained good precision (0.9844 ± 0.0111) and recall (0.9310 ± 0.0153). Conclusions Results on the test set confirm that the proposed workflow is consistent and reliable for the investigated domain, and it can support the clinical practice of discriminating the two classes of glomeruli. Analyses on misclassifications show that the involved images are usually affected by staining artefacts or present partial sections due to slice preparation and staining processes. In clinical practice, however, pathologists discard images showing such artefacts.
Collapse
Affiliation(s)
- Giacomo Donato Cascarano
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Bary, Italy.,Apulian Bioengineering s.r.l., Modugno, BA, Italy
| | | | - Ruggero Lemma
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Bary, Italy
| | - Antonio Brunetti
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Bary, Italy.,Apulian Bioengineering s.r.l., Modugno, BA, Italy
| | - Domenico Buongiorno
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Bary, Italy.,Apulian Bioengineering s.r.l., Modugno, BA, Italy
| | - Irio De Feudis
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Bary, Italy.,Apulian Bioengineering s.r.l., Modugno, BA, Italy
| | - Andrea Guerriero
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Bary, Italy
| | - Umberto Venere
- Department of Emergency and Organ Transplantation, Nephrology Unit University of Bari Aldo Moro, Bari, Italy
| | - Silvia Matino
- Department of Emergency and Organ Transplantation, Nephrology Unit University of Bari Aldo Moro, Bari, Italy
| | - Maria Teresa Rocchetti
- Department of Emergency and Organ Transplantation, Nephrology Unit University of Bari Aldo Moro, Bari, Italy
| | - Michele Rossini
- Department of Emergency and Organ Transplantation, Nephrology Unit University of Bari Aldo Moro, Bari, Italy
| | - Francesco Pesce
- Department of Emergency and Organ Transplantation, Nephrology Unit University of Bari Aldo Moro, Bari, Italy
| | - Loreto Gesualdo
- Department of Emergency and Organ Transplantation, Nephrology Unit University of Bari Aldo Moro, Bari, Italy
| | - Vitoantonio Bevilacqua
- Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari, Bary, Italy. .,Apulian Bioengineering s.r.l., Modugno, BA, Italy.
| |
Collapse
|
21
|
Multi-Task Learning-Based Immunofluorescence Classification of Kidney Disease. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph182010798. [PMID: 34682567 PMCID: PMC8535636 DOI: 10.3390/ijerph182010798] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Revised: 09/24/2021] [Accepted: 09/26/2021] [Indexed: 12/14/2022]
Abstract
Chronic kidney disease is one of the most important causes of mortality worldwide, but a shortage of nephrology pathologists has led to delays or errors in its diagnosis and treatment. Immunofluorescence (IF) images of patients with IgA nephropathy (IgAN), membranous nephropathy (MN), diabetic nephropathy (DN), and lupus nephritis (LN) were obtained from the General Hospital of Chinese PLA. The data were divided into training and test data. To simulate the inaccurate focus of the fluorescence microscope, the Gaussian method was employed to blur the IF images. We proposed a novel multi-task learning (MTL) method for image quality assessment, de-blurring, and disease classification tasks. A total of 1608 patients’ IF images were included—1289 in the training set and 319 in the test set. For non-blurred IF images, the classification accuracy of the test set was 0.97, with an AUC of 1.000. For blurred IF images, the proposed MTL method had a higher accuracy (0.94 vs. 0.93, p < 0.01) and higher AUC (0.993 vs. 0.986) than the common MTL method. The novel MTL method not only diagnosed four types of kidney diseases through blurred IF images but also showed good performance in two auxiliary tasks: image quality assessment and de-blurring.
Collapse
|
22
|
Yang CK, Lee CY, Wang HS, Huang SC, Liang PI, Chen JS, Kuo CF, Tu KH, Yeh CY, Chen TD. Glomerular Disease Classification and Lesion Identification by Machine Learning. Biomed J 2021; 45:675-685. [PMID: 34506971 PMCID: PMC9486238 DOI: 10.1016/j.bj.2021.08.011] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Revised: 06/21/2021] [Accepted: 08/31/2021] [Indexed: 12/13/2022] Open
Abstract
Background Classification of glomerular diseases and identification of glomerular lesions require careful morphological examination by experienced nephropathologists, which is labor-intensive, time-consuming, and prone to interobserver variability. In this regard, recent advance in machine learning-based image analysis is promising. Methods We combined Mask Region-based Convolutional Neural Networks (Mask R–CNN) with an additional classification step to build a glomerulus detection model using human kidney biopsy samples. A Long Short-Term Memory (LSTM) recurrent neural network was applied for glomerular disease classification, and another two-stage model using ResNeXt-101 was constructed for glomerular lesion identification in cases of lupus nephritis. Results The detection model showed state-of-the-art performance on variedly stained slides with F1 scores up to 0.944. The disease classification model showed good accuracies up to 0.940 on recognizing different glomerular diseases based on H&E whole slide images. The lesion identification model demonstrated high discriminating power with area under the receiver operating characteristic curve up to 0.947 for various glomerular lesions. Models showed good generalization on external testing datasets. Conclusion This study is the first-of-its-kind showing how each step of kidney biopsy interpretation carried out by nephropathologists can be captured and simulated by machine learning models. The models were integrated into a whole slide image viewing and annotating platform to enable nephropathologists to review, correct, and confirm the inference results. Further improvement on model performances and incorporating inputs from immunofluorescence, electron microscopy, and clinical data might realize actual clinical use.
Collapse
Affiliation(s)
- Cheng-Kun Yang
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Ching-Yi Lee
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Hsiang-Sheng Wang
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Shun-Chen Huang
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Kaohsiung Branch, No. 123, Dapi Rd., Niaosong Dist., Kaohsiung City 833, Taiwan.
| | - Peir-In Liang
- Department of Pathology, Kaohsiung Medical University Hospital, No. 100, Ziyou 1st Rd., Sanmin Dist., Kaohsiung City 807, Taiwan.
| | - Jung-Sheng Chen
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Chang-Fu Kuo
- Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Kun-Hua Tu
- Department of Nephrology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| | - Chao-Yuan Yeh
- aetherAI, Co., Ltd., 9F., No.3-2, Park St., Nangang Dist., Taipei City 115, Taiwan.
| | - Tai-Di Chen
- Department of Anatomic Pathology, Chang Gung Memorial Hospital Linkou Main Branch, No. 5, Fuxing St., Guishan Dist., Taoyuan City 333, Taiwan.
| |
Collapse
|
23
|
Liu X, Li M, Wu Y, Chen Y, Hao F, Zhou D, Wang C, Ma C, Shi G, Zhou X. An efficient glomerular object locator for renal whole slide images using proposal-free network and dynamic scale evaluation method. AI COMMUN 2021. [DOI: 10.3233/aic-210073] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
In the diagnosis of chronic kidney disease, glomerulus as the blood filter provides important information for an accurate disease diagnosis. Thus automatic localization of the glomeruli is the necessary groundwork for future auxiliary kidney disease diagnosis, such as glomerular classification and area measurement. In this paper, we propose an efficient glomerular object locator in kidney whole slide image(WSI) based on proposal-free network and dynamic scale evaluation method. In the training phase, we construct an intensive proposal-free network which can learn efficiently the fine-grained features of the glomerulus. In the evaluation phase, a dynamic scale evaluation method is utilized to help the well-trained model find the most appropriate evaluation scale for each high-resolution WSI. We collect and digitalize 1204 renal biopsy microscope slides containing more than 41000 annotated glomeruli, which is the largest number of dataset to our best knowledge. We validate the each component of the proposed locator via the ablation study. Experimental results confirm that the proposed locator outperforms recently proposed approaches and pathologists by comparing F 1 and run time in localizing glomeruli from WSIs at a resolution of 0.25 μm/pixel and thus achieves state-of-the-art performance. Particularly, the proposed locator can be embedded into the renal intelligent auxiliary diagnosis system for renal clinical diagnosis by localizing glomeruli in high-resolution WSIs effectively.
Collapse
Affiliation(s)
- Xueyu Liu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Ming Li
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Yongfei Wu
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
- Faculty of Science and Technology, University of Macau, Taipa, Macau, China. E-mail:
| | - Yilin Chen
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Fang Hao
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Daoxiang Zhou
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Chen Wang
- Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, China. E-mail:
| | - Chuanfeng Ma
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Guangze Shi
- College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China. E-mail:
| | - Xiaoshuang Zhou
- Department of Nephrology, Shanxi Provincial People’s Hospital, Taiyuan, Shanxi, China. E-mail:
| |
Collapse
|
24
|
Jiang L, Chen W, Dong B, Mei K, Zhu C, Liu J, Cai M, Yan Y, Wang G, Zuo L, Shi H. A Deep Learning-Based Approach for Glomeruli Instance Segmentation from Multistained Renal Biopsy Pathologic Images. THE AMERICAN JOURNAL OF PATHOLOGY 2021; 191:1431-1441. [PMID: 34294192 DOI: 10.1016/j.ajpath.2021.05.004] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Revised: 04/02/2021] [Accepted: 05/07/2021] [Indexed: 12/20/2022]
Abstract
Glomeruli instance segmentation from pathologic images is a fundamental step in the automatic analysis of renal biopsies. Glomerular histologic manifestations vary widely among diseases and cases, and several special staining methods are necessary for pathologic diagnosis. A robust model is needed to segment and classify glomeruli with different staining methods and apply in cases with various glomerular pathologic changes. Herein, pathologic images from renal biopsy slides stained with three basic special staining methods were used to build the data sets. The snapshot group included 1970 glomeruli from 516 patients, and the whole-slide image group included 8665 glomeruli from 148 patients. Cascade Mask region-based convolutional neural net architecture was trained to detect, classify, and segment glomeruli into three categories: i) GN, structural normal; ii) global sclerosis; and iii) glomerular with other lesions. In the snapshot group, total glomeruli, GN, global sclerosis, and glomerular with other lesions achieved an F1 score of 0.914, 0.896, 0.681, and 0.756, respectively, which were comparable with those in the whole-slide image group (0.940, 0.839, 0.806, and 0.753, respectively). Among the three categories, GN achieved the best instance segmentation effect in both groups, as determined by average precision, average recall, F1 score, and Mask mean Intersection over Union. The present model segments and classifies multistained glomeruli with efficiency and robustness. It can be applied as the first step for more detailed glomerular histologic analysis.
Collapse
Affiliation(s)
- Lei Jiang
- Electron Microscope Lab, Peking University People's Hospital, Beijing, China
| | - Wenkai Chen
- School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing, China
| | - Bao Dong
- Department of Nephrology, Peking University People's Hospital, Beijing, China
| | - Ke Mei
- School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing, China
| | - Chuang Zhu
- School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing, China.
| | - Jun Liu
- School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing, China
| | - Meishun Cai
- Department of Nephrology, Peking University People's Hospital, Beijing, China
| | - Yu Yan
- Department of Nephrology, Peking University People's Hospital, Beijing, China
| | - Gongwei Wang
- Department of Pathology, Peking University People's Hospital, Beijing, China
| | - Li Zuo
- Department of Nephrology, Peking University People's Hospital, Beijing, China
| | - Hongxia Shi
- Electron Microscope Lab, Peking University People's Hospital, Beijing, China.
| |
Collapse
|
25
|
Deng R, Yang H, Jha A, Lu Y, Chu P, Fogo AB, Huo Y. Map3D: Registration-Based Multi-Object Tracking on 3D Serial Whole Slide Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:1924-1933. [PMID: 33780334 PMCID: PMC8249345 DOI: 10.1109/tmi.2021.3069154] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
There has been a long pursuit for precise and reproducible glomerular quantification on renal pathology to leverage both research and practice. When digitizing the biopsy tissue samples using whole slide imaging (WSI), a set of serial sections from the same tissue can be acquired as a stack of images, similar to frames in a video. In radiology, the stack of images (e.g., computed tomography) are naturally used to provide 3D context for organs, tissues, and tumors. In pathology, it is appealing to do a similar 3D assessment. However, the 3D identification and association of large-scale glomeruli on renal pathology is challenging due to large tissue deformation, missing tissues, and artifacts from WSI. In this paper, we propose a novel Multi-object Association for Pathology in 3D (Map3D) method for automatically identifying and associating large-scale cross-sections of 3D objects from routine serial sectioning and WSI. The innovations of the Multi-Object Association for Pathology in 3D (Map3D) method are three-fold: (1) the large-scale glomerular association is formed as a new multi-object tracking (MOT) perspective; (2) the quality-aware whole series registration is proposed to not only provide affinity estimation but also offer automatic kidney-wise quality assurance (QA) for registration; (3) a dual-path association method is proposed to tackle the large deformation, missing tissues, and artifacts during tracking. To the best of our knowledge, the Map3D method is the first approach that enables automatic and large-scale glomerular association across 3D serial sectioning using WSI. Our proposed method Map3D achieved MOTA = 44.6, which is 12.1% higher than the non-deep learning benchmarks.
Collapse
|
26
|
Huo Y, Deng R, Liu Q, Fogo AB, Yang H. AI applications in renal pathology. Kidney Int 2021; 99:1309-1320. [PMID: 33581198 PMCID: PMC8154730 DOI: 10.1016/j.kint.2021.01.015] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2020] [Revised: 01/09/2021] [Accepted: 01/13/2021] [Indexed: 12/20/2022]
Abstract
The explosive growth of artificial intelligence (AI) technologies, especially deep learning methods, has been translated at revolutionary speed to efforts in AI-assisted healthcare. New applications of AI to renal pathology have recently become available, driven by the successful AI deployments in digital pathology. However, synergetic developments of renal pathology and AI require close interdisciplinary collaborations between computer scientists and renal pathologists. Computer scientists should understand that not every AI innovation is translatable to renal pathology, while renal pathologists should capture high-level principles of the relevant AI technologies. Herein, we provide an integrated review on current and possible future applications in AI-assisted renal pathology, by including perspectives from computer scientists and renal pathologists. First, the standard stages, from data collection to analysis, in full-stack AI-assisted renal pathology studies are reviewed. Second, representative renal pathology-optimized AI techniques are introduced. Last, we review current clinical AI applications, as well as promising future applications with the recent advances in AI.
Collapse
Affiliation(s)
- Yuankai Huo
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| | - Ruining Deng
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| | - Quan Liu
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| | - Agnes B Fogo
- Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Haichun Yang
- Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center, Nashville, Tennessee, USA.
| |
Collapse
|
27
|
Wilbur DC, Smith ML, Cornell LD, Andryushkin A, Pettus JR. Automated identification of glomeruli and synchronised review of special stains in renal biopsies by machine learning and slide registration: a cross-institutional study. Histopathology 2021; 79:499-508. [PMID: 33813779 DOI: 10.1111/his.14376] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2021] [Revised: 03/11/2021] [Accepted: 03/29/2021] [Indexed: 11/30/2022]
Abstract
AIMS Machine learning in digital pathology can improve efficiency and accuracy via prescreening with automated feature identification. Studies using uniform histological material have shown promise. Generalised application requires validation on slides from multiple institutions. We used machine learning to identify glomeruli on renal biopsies and compared performance between single and multiple institutions. METHODS AND RESULTS Randomly selected, adequately sampled renal core biopsy cases (71) consisting of four stains each (haematoxylin and eosin, trichrome, silver, periodic acid Schiff) from three institutions were digitised at ×40. Glomeruli were manually annotated by three renal pathologists using a digital tool. Cases were divided into training/validation (n = 52) and evaluation (n = 19) cohorts. An algorithm was trained to develop three convolutional neural network (CNN) models which tested case cohorts intra- and inter-institutionally. Raw CNN search data from each of the four slides per case were merged into composite regions of interest containing putative glomeruli. The sensitivity and modified specificity of glomerulus detection (versus annotated truth) were calculated for each model/cohort. Intra-institutional (3) sensitivity ranged from 90 to 93%, with modified specificity from 86 to 98%. Interinstitutional (1) sensitivity was 77%, with modified specificity 97%. Combined intra- and inter-institutional (1) sensitivity was 86%, with modified specificity 92%. CONCLUSIONS Feature detection sensitivity degrades when training and test material originate from different sites. Training using a combined set of digital slides from three institutions improves performance. Differing histology methods probably account for algorithm performance contrasts. Our data highlight the need for diverse training sets for the development of generalisable machine learning histology algorithms.
Collapse
Affiliation(s)
| | - Maxwell L Smith
- Department of Laboratory Medicine and Pathology, Mayo Clinic - Scottsdale, Phoenix, AZ, USA
| | - Lynn D Cornell
- Department of Laboratory Medicine and Pathology, Mayo Clinic - Rochester, Rochester, MN, USA
| | | | - Jason R Pettus
- Department of Pathology and Laboratory Medicine, Dartmouth-Hitchcock Medical Center, Lebanon, MH, USA
| |
Collapse
|
28
|
Farris AB, Vizcarra J, Amgad M, Cooper LAD, Gutman D, Hogan J. Artificial intelligence and algorithmic computational pathology: an introduction with renal allograft examples. Histopathology 2021; 78:791-804. [PMID: 33211332 DOI: 10.1111/his.14304] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Whole slide imaging, which is an important technique in the field of digital pathology, has recently been the subject of increased interest and avenues for utilisation, and with more widespread whole slide image (WSI) utilisation, there will also be increased interest in and implementation of image analysis (IA) techniques. IA includes artificial intelligence (AI) and targeted or hypothesis-driven algorithms. In the overall pathology field, the number of citations related to these topics has increased in recent years. Renal pathology is one anatomical pathology subspecialty that has utilised WSIs and IA algorithms; it can be argued that renal transplant pathology could be particularly suited for whole slide imaging and IA, as renal transplant pathology is frequently classified by use of the semiquantitative Banff classification of renal allograft pathology. Hypothesis-driven/targeted algorithms have been used in the past for the assessment of a variety of features in the kidney (e.g. interstitial fibrosis, tubular atrophy, inflammation); in recent years, the amount of research has particularly increased in the area of AI/machine learning for the identification of glomeruli, for histological segmentation, and for other applications. Deep learning is the form of machine learning that is most often used for such AI approaches to the 'big data' of pathology WSIs, and deep learning methods such as artificial neural networks (ANNs)/convolutional neural networks (CNNs) are utilised. Unsupervised and supervised AI algorithms can be employed to accomplish image or semantic classification. In this review, AI and other IA algorithms applied to WSIs are discussed, and examples from renal pathology are covered, with an emphasis on renal transplant pathology.
Collapse
Affiliation(s)
- Alton B Farris
- Department of Pathology and Laboratory Medicine, Atlanta, GA, USA
| | - Juan Vizcarra
- Department of Bioinformatics, Emory University, Atlanta, GA, USA
| | - Mohamed Amgad
- Department of Pathology and Center for Computational Imaging and Signal Analytics, Northwestern University, Chicago, IL, USA
| | - Lee A D Cooper
- Department of Pathology and Center for Computational Imaging and Signal Analytics, Northwestern University, Chicago, IL, USA
| | - David Gutman
- Department of Bioinformatics, Emory University, Atlanta, GA, USA
| | - Julien Hogan
- Department of Surgery, Emory University, Atlanta, GA, USA
| |
Collapse
|
29
|
Govind D, Santo BA, Ginley B, Yacoub R, Rosenberg AZ, Jen KY, Walavalkar V, Wilding GE, Worral AM, Mohammad I, Sarder P. Automated detection and quantification of Wilms' Tumor 1-positive cells in murine diabetic kidney disease. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11603. [PMID: 34366543 DOI: 10.1117/12.2581387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
In diabetic kidney disease (DKD), podocyte depletion, and the subsequent migration of parietal epithelial cells (PECs) to the tuft, is a precursor to progressive glomerular damage, but the limitations of brightfield microscopy currently preclude direct pathological quantitation of these cells. Here we present an automated approach to podocyte and PEC detection developed using kidney sections from mouse model emulating DKD, stained first for Wilms' Tumor 1 (WT1) (podocyte and PEC marker) by immunofluorescence, then post-stained with periodic acid-Schiff (PAS). A generative adversarial network (GAN)-based pipeline was used to translate these PAS-stained sections into WT1-labeled IF images, enabling in silico label-free podocyte and PEC identification in brightfield images. Our method detected WT1-positive cells with high sensitivity/specificity (0.87/0.92). Additionally, our algorithm performed with a higher Cohen's kappa (0.85) than the average manual identification by three renal pathologists (0.78). We propose that this pipeline will enable accurate detection of WT1-positive cells in research applications.
Collapse
Affiliation(s)
- Darshana Govind
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, NY
| | - Briana A Santo
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, NY
| | - Brandon Ginley
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, NY
| | - Rabi Yacoub
- Department of Internal Medicine, University at Buffalo, Buffalo, NY
| | - Avi Z Rosenberg
- Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, MD
| | - Kuang-Yu Jen
- Department of Pathology and Laboratory Medicine, University of California at Davis, CA
| | - Vignesh Walavalkar
- Department of Pathology, University of California San Francisco, San Francisco, CA
| | | | - Amber M Worral
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, NY
| | - Imtiaz Mohammad
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, NY
| | - Pinaki Sarder
- Department of Pathology and Anatomical Sciences, University at Buffalo, Buffalo, NY
| |
Collapse
|
30
|
Cicalese PA, Mobiny A, Shahmoradi Z, Yi X, Mohan C, Van Nguyen H. Kidney Level Lupus Nephritis Classification Using Uncertainty Guided Bayesian Convolutional Neural Networks. IEEE J Biomed Health Inform 2021; 25:315-324. [PMID: 33206612 DOI: 10.1109/jbhi.2020.3039162] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The kidney biopsy based diagnosis of Lupus Nephritis (LN) is characterized by low inter-observer agreement, with misdiagnosis being associated with increased patient morbidity and mortality. Although various Computer Aided Diagnosis (CAD) systems have been developed for other nephrohistopathological applications, little has been done to accurately classify kidneys based on their kidney level Lupus Glomerulonephritis (LGN) scores. The successful implementation of CAD systems has also been hindered by the diagnosing physician's perceived classifier strengths and weaknesses, which has been shown to have a negative effect on patient outcomes. We propose an Uncertainty-Guided Bayesian Classification (UGBC) scheme that is designed to accurately classify control, class I/II, and class III/IV LGN (3 class) at both the glomerular-level classification task (26,634 segmented glomerulus images) and the kidney-level classification task (87 MRL/lpr mouse kidney sections). Data annotation was performed using a high throughput, bulk labeling scheme that is designed to take advantage of Deep Neural Network's (or DNNs) resistance to label noise. Our augmented UGBC scheme achieved a 94.5% weighted glomerular-level accuracy while achieving a weighted kidney-level accuracy of 96.6%, improving upon the standard Convolutional Neural Network (CNN) architecture by 11.8% and 3.5% respectively.
Collapse
|
31
|
Abstract
PURPOSE OF REVIEW Successful integration of artificial intelligence into extant clinical workflows is contingent upon a number of factors including clinician comprehension and interpretation of computer vision. This article discusses how image analysis and machine learning have enabled comprehensive characterization of kidney morphology for development of automated diagnostic and prognostic renal pathology applications. RECENT FINDINGS The primordial digital pathology informatics work employed classical image analysis and machine learning to prognosticate renal disease. Although this classical approach demonstrated tremendous potential, subsequent advancements in hardware technology rendered artificial neural networks '(ANNs) the method of choice for machine vision in computational pathology'. Offering rapid and reproducible detection, characterization and classification of kidney morphology, ANNs have facilitated the development of diagnostic and prognostic applications. In addition, modern machine learning with ANNs has revealed novel biomarkers in kidney disease, demonstrating the potential for machine vision to elucidate novel pathologic mechanisms beyond extant clinical knowledge. SUMMARY Despite the revolutionary developments potentiated by modern machine learning, several challenges remain, including data quality control and curation, image annotation and ontology, integration of multimodal data and interpretation of machine vision or 'opening the black box'. Resolution of these challenges will not only revolutionize diagnostic pathology but also pave the way for precision medicine and integration of artificial intelligence in the process of care.
Collapse
|
32
|
Applying Machine Learning for Integration of Multi-Modal Genomics Data and Imaging Data to Quantify Heterogeneity in Tumour Tissues. Methods Mol Biol 2021; 2190:209-228. [PMID: 32804368 DOI: 10.1007/978-1-0716-0826-5_10] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
With rapid advances in experimental instruments and protocols, imaging and sequencing data are being generated at an unprecedented rate contributing significantly to the current and coming big biomedical data. Meanwhile, unprecedented advances in computational infrastructure and analysis algorithms are realizing image-based digital diagnosis not only in radiology and cardiology but also oncology and other diseases. Machine learning methods, especially deep learning techniques, are already and broadly implemented in diverse technological and industrial sectors, but their applications in healthcare are just starting. Uniquely in biomedical research, a vast potential exists to integrate genomics data with histopathological imaging data. The integration has the potential to extend the pathologist's limits and boundaries, which may create breakthroughs in diagnosis, treatment, and monitoring at molecular and tissue levels. Moreover, the applications of genomics data are realizing the potential for personalized medicine, making diagnosis, treatment, monitoring, and prognosis more accurate. In this chapter, we discuss machine learning methods readily available for digital pathology applications, new prospects of integrating spatial genomics data on tissues with tissue morphology, and frontier approaches to combining genomics data with pathological imaging data. We present perspectives on how artificial intelligence can be synergized with molecular genomics and imaging to make breakthroughs in biomedical and translational research for computer-aided applications.
Collapse
|
33
|
Noriaki S, Eiichiro U, Yasushi O. Artificial Intelligence in Kidney Pathology. Artif Intell Med 2021. [DOI: 10.1007/978-3-030-58080-3_181-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
34
|
AF-SENet: Classification of Cancer in Cervical Tissue Pathological Images Based on Fusing Deep Convolution Features. SENSORS 2020; 21:s21010122. [PMID: 33375508 PMCID: PMC7795214 DOI: 10.3390/s21010122] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Revised: 12/03/2020] [Accepted: 12/24/2020] [Indexed: 11/26/2022]
Abstract
Cervical cancer is the fourth most common cancer in the world. Whole-slide images (WSIs) are an important standard for the diagnosis of cervical cancer. Missed diagnoses and misdiagnoses often occur due to the high similarity in pathological cervical images, the large number of readings, the long reading time, and the insufficient experience levels of pathologists. Existing models have insufficient feature extraction and representation capabilities, and they suffer from insufficient pathological classification. Therefore, this work first designs an image processing algorithm for data augmentation. Second, the deep convolutional features are extracted by fine-tuning pre-trained deep network models, including ResNet50 v2, DenseNet121, Inception v3, VGGNet19, and Inception-ResNet, and then local binary patterns and a histogram of the oriented gradient to extract traditional image features are used. Third, the features extracted by the fine-tuned models are serially fused according to the feature representation ability parameters and the accuracy of multiple experiments proposed in this paper, and spectral embedding is used for dimension reduction. Finally, the fused features are inputted into the Analysis of Variance-F value-Spectral Embedding Net (AF-SENet) for classification. There are four different pathological images of the dataset: normal, low-grade squamous intraepithelial lesion (LSIL), high-grade squamous intraepithelial lesion (HSIL), and cancer. The dataset is divided into a training set (90%) and a test set (10%). The serial fusion effect of the deep features extracted by Resnet50v2 and DenseNet121 (C5) is the best, with average classification accuracy reaching 95.33%, which is 1.07% higher than ResNet50 v2 and 1.05% higher than DenseNet121. The recognition ability is significantly improved, especially in LSIL, reaching 90.89%, which is 2.88% higher than ResNet50 v2 and 2.1% higher than DenseNet121. Thus, this method significantly improves the accuracy and generalization ability of pathological cervical WSI recognition by fusing deep features.
Collapse
|
35
|
Barisoni L, Lafata KJ, Hewitt SM, Madabhushi A, Balis UGJ. Digital pathology and computational image analysis in nephropathology. Nat Rev Nephrol 2020; 16:669-685. [PMID: 32848206 PMCID: PMC7447970 DOI: 10.1038/s41581-020-0321-6] [Citation(s) in RCA: 98] [Impact Index Per Article: 24.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/24/2020] [Indexed: 12/17/2022]
Abstract
The emergence of digital pathology - an image-based environment for the acquisition, management and interpretation of pathology information supported by computational techniques for data extraction and analysis - is changing the pathology ecosystem. In particular, by virtue of our new-found ability to generate and curate digital libraries, the field of machine vision can now be effectively applied to histopathological subject matter by individuals who do not have deep expertise in machine vision techniques. Although these novel approaches have already advanced the detection, classification, and prognostication of diseases in the fields of radiology and oncology, renal pathology is just entering the digital era, with the establishment of consortia and digital pathology repositories for the collection, analysis and integration of pathology data with other domains. The development of machine-learning approaches for the extraction of information from image data, allows for tissue interrogation in a way that was not previously possible. The application of these novel tools are placing pathology centre stage in the process of defining new, integrated, biologically and clinically homogeneous disease categories, to identify patients at risk of progression, and shifting current paradigms for the treatment and prevention of kidney diseases.
Collapse
Affiliation(s)
- Laura Barisoni
- Department of Pathology, Duke University, Durham, NC, USA.
- Department of Medicine, Division of Nephrology, Duke University, Durham, NC, USA.
| | - Kyle J Lafata
- Department of Radiology, Duke University, Durham, NC, USA
- Department of Radiation Oncology, Duke University, Durham, NC, USA
| | - Stephen M Hewitt
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, NIH, Bethesda, MD, USA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA
- Louis Stokes Veterans Administration Medical Center, Cleveland, OH, USA
| | | |
Collapse
|
36
|
A Deep Learning Instance Segmentation Approach for Global Glomerulosclerosis Assessment in Donor Kidney Biopsies. ELECTRONICS 2020. [DOI: 10.3390/electronics9111768] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The histological assessment of glomeruli is fundamental for determining if a kidney is suitable for transplantation. The Karpinski score is essential to evaluate the need for a single or dual kidney transplant and includes the ratio between the number of sclerotic glomeruli and the overall number of glomeruli in a kidney section. The manual evaluation of kidney biopsies performed by pathologists is time-consuming and error-prone, so an automatic framework to delineate all the glomeruli present in a kidney section can be very useful. Our experiments have been conducted on a dataset provided by the Department of Emergency and Organ Transplantations (DETO) of Bari University Hospital. This dataset is composed of 26 kidney biopsies coming from 19 donors. The rise of Convolutional Neural Networks (CNNs) has led to a realm of methods which are widely applied in Medical Imaging. Deep learning techniques are also very promising for the segmentation of glomeruli, with a variety of existing approaches. Many methods only focus on semantic segmentation—which consists in segmentation of individual pixels—or ignore the problem of discriminating between non-sclerotic and sclerotic glomeruli, so these approaches are not optimal or inadequate for transplantation assessment. In this work, we employed an end-to-end fully automatic approach based on Mask R-CNN for instance segmentation and classification of glomeruli. We also compared the results obtained with a baseline based on Faster R-CNN, which only allows detection at bounding boxes level. With respect to the existing literature, we improved the Mask R-CNN approach in sliding window contexts, by employing a variant of the Non-Maximum Suppression (NMS) algorithm, which we called Non-Maximum-Area Suppression (NMAS). The obtained results are very promising, leading to improvements over existing literature. The baseline Faster R-CNN-based approach obtained an F-Measure of 0.904 and 0.667 for non-sclerotic and sclerotic glomeruli, respectively. The Mask R-CNN approach has a significant improvement over the baseline, obtaining an F-Measure of 0.925 and 0.777 for non-sclerotic and sclerotic glomeruli, respectively. The proposed method is very promising for the instance segmentation and classification of glomeruli, and allows to make a robust evaluation of global glomerulosclerosis. We also compared Karpinski score obtained with our algorithm to that obtained with pathologists’ annotations to show the soundness of the proposed workflow from a clinical point of view.
Collapse
|
37
|
Wu Z, Wang L, Li C, Cai Y, Liang Y, Mo X, Lu Q, Dong L, Liu Y. DeepLRHE: A Deep Convolutional Neural Network Framework to Evaluate the Risk of Lung Cancer Recurrence and Metastasis From Histopathology Images. Front Genet 2020; 11:768. [PMID: 33193560 PMCID: PMC7477356 DOI: 10.3389/fgene.2020.00768] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2020] [Accepted: 06/29/2020] [Indexed: 12/21/2022] Open
Abstract
It is critical for patients who cannot undergo eradicable surgery to predict the risk of lung cancer recurrence and metastasis; therefore, the physicians can design the appropriate adjuvant therapy plan. However, traditional circulating tumor cell (CTC) detection or next-generation sequencing (NGS)-based methods are usually expensive and time-inefficient, which urge the need for more efficient computational models. In this study, we have established a convolutional neural network (CNN) framework called DeepLRHE to predict the recurrence risk of lung cancer by analyzing histopathological images of patients. The steps for using DeepLRHE include automatic tumor region identification, image normalization, biomarker identification, and sample classification. In practice, we used 110 lung cancer samples downloaded from The Cancer Genome Atlas (TCGA) database to train and validate our CNN model and 101 samples as independent test dataset. The area under the receiver operating characteristic (ROC) curve (AUC) for test dataset was 0.79, suggesting a relatively good prediction performance. Our study demonstrates that the features extracted from histopathological images could be well used to predict lung cancer recurrence after surgical resection and help classify patients who should receive additional adjuvant therapy.
Collapse
Affiliation(s)
- Zhijun Wu
- Department of Oncology, The First People's Hospital of Changde City, Changde, China
| | - Lin Wang
- Department of Oncology, Hainan General Hospital, Haikou, China
| | - Churong Li
- Sichuan Cancer Hospital and Institute, The Affiliated Cancer Hospital, School of Medicine, UESTC, Chengdu, China
| | | | | | - Xiaofei Mo
- Geneis (Beijing) Co., Ltd., Beijing, China
| | | | - Lixin Dong
- The First Hospital of Qinhuangdao, Qinhuangdao, China
| | | |
Collapse
|
38
|
Uchino E, Suzuki K, Sato N, Kojima R, Tamada Y, Hiragi S, Yokoi H, Yugami N, Minamiguchi S, Haga H, Yanagita M, Okuno Y. Classification of glomerular pathological findings using deep learning and nephrologist-AI collective intelligence approach. Int J Med Inform 2020; 141:104231. [PMID: 32682317 DOI: 10.1016/j.ijmedinf.2020.104231] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2019] [Revised: 05/30/2020] [Accepted: 07/06/2020] [Indexed: 12/20/2022]
Abstract
BACKGROUND Automated classification of glomerular pathological findings is potentially beneficial in establishing an efficient and objective diagnosis in renal pathology. While previous studies have verified the artificial intelligence (AI) models for the classification of global sclerosis and glomerular cell proliferation, there are several other glomerular pathological findings required for diagnosis, and the comprehensive models for the classification of these major findings have not yet been reported. Whether the cooperation between these AI models and clinicians improves diagnostic performance also remains unknown. Here, we developed AI models to classify glomerular images for major findings required for pathological diagnosis and investigated whether those models could improve the diagnostic performance of nephrologists. METHODS We used a dataset of 283 kidney biopsy cases comprising 15,888 glomerular images that were annotated by a total of 25 nephrologists. AI models to classify seven pathological findings: global sclerosis, segmental sclerosis, endocapillary proliferation, mesangial matrix accumulation, mesangial cell proliferation, crescent, and basement membrane structural changes, were constructed using deep learning by fine-tuning of InceptionV3 convolutional neural network. Subsequently, we compared the agreement to truth labels between majority decision among nephrologists with or without the AI model as a voter. RESULTS Our model for global sclerosis showed high performance (area under the curve: periodic acid-Schiff, 0.986; periodic acid methenamine silver, 0.983); the models for the other findings also showed performance close to those of nephrologists. By adding the AI model output to majority decision among nephrologists, out of the 14 constructed models, the results of the majority decision showed improvement in sensitivity for 10 models (four of them were statistically significant) and specificity for eight models (five significant). CONCLUSION Our study showed a proof-of-concept for the classification of multiple glomerular findings in a comprehensive method of deep learning and suggested its potential effectiveness in improving diagnostic accuracy of clinicians.
Collapse
Affiliation(s)
- Eiichiro Uchino
- Department of Medical Intelligent Systems, Graduate School of Medicine, Kyoto University, Kyoto, Japan; Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | | | - Noriaki Sato
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan; Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Ryosuke Kojima
- Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Yoshinori Tamada
- Department of Medical Intelligent Systems, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Shusuke Hiragi
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan; Division of Medical Informatics and Administration Planning, Kyoto University Hospital, Kyoto, Japan
| | - Hideki Yokoi
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | | | - Sachiko Minamiguchi
- Department of Diagnostic Pathology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Hironori Haga
- Department of Diagnostic Pathology, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Motoko Yanagita
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto, Japan; Institute for the Advanced Study of Human Biology (ASHBi), Kyoto University, Kyoto, Japan.
| | - Yasushi Okuno
- Department of Biomedical Data Intelligence, Graduate School of Medicine, Kyoto University, Kyoto, Japan; RIKEN, The Drug Development Data Intelligence Platform Group, Yokohama, Japan.
| |
Collapse
|
39
|
Zeng C, Nan Y, Xu F, Lei Q, Li F, Chen T, Liang S, Hou X, Lv B, Liang D, Luo W, Lv C, Li X, Xie G, Liu Z. Identification of glomerular lesions and intrinsic glomerular cell types in kidney diseases via deep learning. J Pathol 2020; 252:53-64. [PMID: 32542677 PMCID: PMC7496925 DOI: 10.1002/path.5491] [Citation(s) in RCA: 35] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2020] [Revised: 05/26/2020] [Accepted: 06/05/2020] [Indexed: 12/14/2022]
Abstract
Identification of glomerular lesions and structures is a key point for pathological diagnosis, treatment instructions, and prognosis evaluation in kidney diseases. These time‐consuming tasks require a more accurate and reproducible quantitative analysis method. We established derivation and validation cohorts composed of 400 Chinese patients with immunoglobulin A nephropathy (IgAN) retrospectively. Deep convolutional neural networks and biomedical image processing algorithms were implemented to locate glomeruli, identify glomerular lesions (global and segmental glomerular sclerosis, crescent, and none of the above), identify and quantify different intrinsic glomerular cells, and assess a network‐based mesangial hypercellularity score in periodic acid–Schiff (PAS)‐stained slides. Our framework achieved 93.1% average precision and 94.9% average recall for location of glomeruli, and a total Cohen's kappa of 0.912 [95% confidence interval (CI), 0.892–0.932] for glomerular lesion classification. The evaluation of global, segmental glomerular sclerosis, and crescents achieved Cohen's kappa values of 1.0, 0.776, 0.861, and 95% CI of (1.0, 1.0), (0.727, 0.825), (0.824, 0.898), respectively. The well‐designed neural network can identify three kinds of intrinsic glomerular cells with 92.2% accuracy, surpassing the about 5–11% average accuracy of junior pathologists. Statistical interpretation shows that there was a significant difference (P value < 0.0001) between this analytic renal pathology system (ARPS) and four junior pathologists for identifying mesangial and endothelial cells, while that for podocytes was similar, with P value = 0.0602. In addition, this study indicated that the ratio of mesangial cells, endothelial cells, and podocytes within glomeruli from IgAN was 0.41:0.36:0.23, and the performance of mesangial score assessment reached a Cohen's kappa of 0.42 and 95% CI (0.18, 0.69). The proposed computer‐aided diagnosis system has feasibility for quantitative analysis and auxiliary recognition of glomerular pathological features. © 2020 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Caihong Zeng
- National Clinical Research Center of Kidney Diseases, Jinling Hospital, Nanjing University School of Medicine, Nanjing, PR China
| | - Yang Nan
- Ping An Healthcare Technology, Shang Hai, PR China
| | - Feng Xu
- National Clinical Research Center of Kidney Diseases, Jinling Hospital, Nanjing University School of Medicine, Nanjing, PR China
| | - Qunjuan Lei
- National Clinical Research Center of Kidney Diseases, Jinling Hospital, Nanjing University School of Medicine, Nanjing, PR China
| | - Fengyi Li
- Ping An Healthcare Technology, Shang Hai, PR China
| | - Tingyu Chen
- National Clinical Research Center of Kidney Diseases, Jinling Hospital, Nanjing University School of Medicine, Nanjing, PR China
| | - Shaoshan Liang
- National Clinical Research Center of Kidney Diseases, Jinling Hospital, Nanjing University School of Medicine, Nanjing, PR China
| | | | - Bin Lv
- Ping An Healthcare Technology, Shang Hai, PR China
| | - Dandan Liang
- National Clinical Research Center of Kidney Diseases, Jinling Hospital, Nanjing University School of Medicine, Nanjing, PR China
| | - WeiLi Luo
- Ping An Healthcare Technology, Shang Hai, PR China
| | - Chuanfeng Lv
- Ping An Healthcare Technology, Shang Hai, PR China
| | - Xiang Li
- Ping An Healthcare Technology, Shang Hai, PR China
| | - Guotong Xie
- Ping An Healthcare Technology, Shang Hai, PR China
| | - Zhihong Liu
- National Clinical Research Center of Kidney Diseases, Jinling Hospital, Nanjing University School of Medicine, Nanjing, PR China
| |
Collapse
|
40
|
Becker JU, Mayerich D, Padmanabhan M, Barratt J, Ernst A, Boor P, Cicalese PA, Mohan C, Nguyen HV, Roysam B. Artificial intelligence and machine learning in nephropathology. Kidney Int 2020; 98:65-75. [PMID: 32475607 PMCID: PMC8906056 DOI: 10.1016/j.kint.2020.02.027] [Citation(s) in RCA: 44] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2019] [Revised: 01/03/2020] [Accepted: 02/12/2020] [Indexed: 12/15/2022]
Abstract
Artificial intelligence (AI) for the purpose of this review is an umbrella term for technologies emulating a nephropathologist's ability to extract information on diagnosis, prognosis, and therapy responsiveness from native or transplant kidney biopsies. Although AI can be used to analyze a wide variety of biopsy-related data, this review focuses on whole slide images traditionally used in nephropathology. AI applications in nephropathology have recently become available through several advancing technologies, including (i) widespread introduction of glass slide scanners, (ii) data servers in pathology departments worldwide, and (iii) through greatly improved computer hardware to enable AI training. In this review, we explain how AI can enhance the reproducibility of nephropathology results for certain parameters in the context of precision medicine using advanced architectures, such as convolutional neural networks, that are currently the state of the art in machine learning software for this task. Because AI applications in nephropathology are still in their infancy, we show the power and potential of AI applications mostly in the example of oncopathology. Moreover, we discuss the technological obstacles as well as the current stakeholder and regulatory concerns about developing AI applications in nephropathology from the perspective of nephropathologists and the wider nephrology community. We expect the gradual introduction of these technologies into routine diagnostics and research for selective tasks, suggesting that this technology will enhance the performance of nephropathologists rather than making them redundant.
Collapse
Affiliation(s)
- Jan U Becker
- Institute of Pathology, University Hospital of Cologne, Cologne, Germany.
| | - David Mayerich
- Department of Electrical and Computer Engineering, University of Houston, Houston, Texas, USA
| | - Meghana Padmanabhan
- Department of Electrical and Computer Engineering, University of Houston, Houston, Texas, USA
| | - Jonathan Barratt
- The Mayer IgA Nephropathy Laboratories, Department of Cardiovascular, University of Leicester, Leicester, UK
| | - Angela Ernst
- Faculty of Medicine, Institute of Medical Statistics and Computational Biology, University of Cologne, Cologne, Germany
| | - Peter Boor
- Institute of Pathology, RWTH Aachen, Germany; Department of Nephrology, RWTH Aachen, Germany
| | | | - Chandra Mohan
- College of Engineering, University of Houston, Houston, Texas, USA
| | - Hien V Nguyen
- Department of Electrical and Computer Engineering, University of Houston, Houston, Texas, USA
| | - Badrinath Roysam
- Department of Electrical and Computer Engineering, University of Houston, Houston, Texas, USA
| |
Collapse
|
41
|
Maraszek KE, Santo BA, Yacoub R, Tomaszewski JE, Mohammad I, Worral AM, Sarder P. The Presence and Location of Podocytes in Glomeruli as Affected by Diabetes Mellitus. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2020; 11320:1132018. [PMID: 32362706 PMCID: PMC7194214 DOI: 10.1117/12.2548904] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
The primary purpose of the kidney, specifically the glomerulus, is filtration. Filtration is accomplished through the glomerular filtration barrier, which consists of the fenestrated endothelium, glomerular basement membrane, and specialized epithelial cells called podocytes. In pathologic states, such as Diabetes Mellitus (DM) and diabetic kidney disease (DKD), variable glomerular conditions result in podocyte injury and depletion, followed by progressive glomerular injury and DKD progression. In this work we quantified glomerulus and podocyte structural changes in histopathology image data derived from a murine model of DM. Using a variety of image processing techniques, we studied changes in podocyte morphology and intra-glomerular distribution across healthy, mild DM, and DM glomeruli. Our feature analysis provided feature trends which we believe are reflective of DKD pathology; while glomerular area peaked in mild DM, average podocyte number and distance from the urinary pole continued to decrease and increase, respectively, throughout DM. Ultimately, this study aims to augment the set of quantifiable image biomarkers used for evaluation of DKD progression in digital pathology, as well as underscore the importance of engineering biologically-inspired image features.
Collapse
Affiliation(s)
- Kathryn E. Maraszek
- Department of Pathology and Anatomical Sciences, University
at Buffalo – The State University of New York
| | - Briana A. Santo
- Department of Pathology and Anatomical Sciences, University
at Buffalo – The State University of New York
| | - Rabi Yacoub
- Medicine – Nephrology, University at Buffalo
– The State University of New York
| | - John E. Tomaszewski
- Department of Pathology and Anatomical Sciences, University
at Buffalo – The State University of New York
| | - Imtiaz Mohammad
- Department of Pathology and Anatomical Sciences, University
at Buffalo – The State University of New York
| | - Amber M. Worral
- Department of Pathology and Anatomical Sciences, University
at Buffalo – The State University of New York
| | - Pinaki Sarder
- Department of Pathology and Anatomical Sciences, University
at Buffalo – The State University of New York
| |
Collapse
|
42
|
Chagas P, Souza L, Araújo I, Aldeman N, Duarte A, Angelo M, Dos-Santos WLC, Oliveira L. Classification of glomerular hypercellularity using convolutional features and support vector machine. Artif Intell Med 2020; 103:101808. [PMID: 32143802 DOI: 10.1016/j.artmed.2020.101808] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2019] [Revised: 11/19/2019] [Accepted: 01/13/2020] [Indexed: 02/06/2023]
Abstract
Glomeruli are histological structures of the kidney cortex formed by interwoven blood capillaries, and are responsible for blood filtration. Glomerular lesions impair kidney filtration capability, leading to protein loss and metabolic waste retention. An example of lesion is the glomerular hypercellularity, which is characterized by an increase in the number of cell nuclei in different areas of the glomeruli. Glomerular hypercellularity is a frequent lesion present in different kidney diseases. Automatic detection of glomerular hypercellularity would accelerate the screening of scanned histological slides for the lesion, enhancing clinical diagnosis. Having this in mind, we propose a new approach for classification of hypercellularity in human kidney images. Our proposed method introduces a novel architecture of a convolutional neural network (CNN) along with a support vector machine, achieving near perfect average results on FIOCRUZ data set in a binary classification (lesion or normal). Additionally, classification of hypercellularity sub-lesions was also evaluated, considering mesangial, endocapilar and both lesions, reaching an average accuracy of 82%. Either in binary task or in the multi-classification one, our proposed method outperformed Xception, ResNet50 and InceptionV3 networks, as well as a traditional handcrafted-based method. To the best of our knowledge, this is the first study on deep learning over a data set of glomerular hypercellularity images of human kidney.
Collapse
Affiliation(s)
- Paulo Chagas
- IVISION Lab, Universidade Federal da Bahia, Bahia, Brazil.
| | - Luiz Souza
- IVISION Lab, Universidade Federal da Bahia, Bahia, Brazil.
| | - Ikaro Araújo
- PPGM, Universidade Federal da Bahia, Bahia, Brazil.
| | - Nayze Aldeman
- Departamento de Medicina Especializada - Universidade Federal do Piauí, Piauí, Brazil.
| | - Angelo Duarte
- Universidade Estadual de Feira de Santana, Bahia, Brazil.
| | - Michele Angelo
- Universidade Estadual de Feira de Santana, Bahia, Brazil.
| | | | | |
Collapse
|
43
|
Kuo CC, Chang CM, Liu KT, Lin WK, Chiang HY, Chung CW, Ho MR, Sun PR, Yang RL, Chen KT. Automation of the kidney function prediction and classification through ultrasound-based kidney imaging using deep learning. NPJ Digit Med 2019; 2:29. [PMID: 31304376 PMCID: PMC6550224 DOI: 10.1038/s41746-019-0104-2] [Citation(s) in RCA: 93] [Impact Index Per Article: 18.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2018] [Accepted: 03/19/2019] [Indexed: 12/22/2022] Open
Abstract
Prediction of kidney function and chronic kidney disease (CKD) through kidney ultrasound imaging has long been considered desirable in clinical practice because of its safety, convenience, and affordability. However, this highly desirable approach is beyond the capability of human vision. We developed a deep learning approach for automatically determining the estimated glomerular filtration rate (eGFR) and CKD status. We exploited the transfer learning technique, integrating the powerful ResNet model pretrained on an ImageNet dataset in our neural network architecture, to predict kidney function based on 4,505 kidney ultrasound images labeled using eGFRs derived from serum creatinine concentrations. To further extract the information from ultrasound images, we leveraged kidney length annotations to remove the peripheral region of the kidneys and applied various data augmentation schemes to produce additional data with variations. Bootstrap aggregation was also applied to avoid overfitting and improve the model's generalization. Moreover, the kidney function features obtained by our deep neural network were used to identify the CKD status defined by an eGFR of <60 ml/min/1.73 m2. A Pearson correlation coefficient of 0.741 indicated the strong relationship between artificial intelligence (AI)- and creatinine-based GFR estimations. Overall CKD status classification accuracy of our model was 85.6% -higher than that of experienced nephrologists (60.3%-80.1%). Our model is the first fundamental step toward realizing the potential of transforming kidney ultrasound imaging into an effective, real-time, distant screening tool. AI-GFR estimation offers the possibility of noninvasive assessment of kidney function, a key goal of AI-powered functional automation in clinical practice.
Collapse
Affiliation(s)
- Chin-Chi Kuo
- Big Data Center, China Medical University Hospital, China Medical University, Taichung, Taiwan
- Kidney Institute and Division of Nephrology, Department of Internal Medicine, China Medical University Hospital and College of Medicine, China Medical University, Taichung, Taiwan
| | - Chun-Min Chang
- Institute of Information Science, Academia Sinica, Taichung, Taiwan
| | - Kuan-Ting Liu
- Institute of Information Science, Academia Sinica, Taichung, Taiwan
| | - Wei-Kai Lin
- Institute of Information Science, Academia Sinica, Taichung, Taiwan
| | - Hsiu-Yin Chiang
- Big Data Center, China Medical University Hospital, China Medical University, Taichung, Taiwan
| | - Chih-Wei Chung
- Big Data Center, China Medical University Hospital, China Medical University, Taichung, Taiwan
| | - Meng-Ru Ho
- Institute of Information Science, Academia Sinica, Taichung, Taiwan
| | - Pei-Ran Sun
- Information Office, China Medical University Hospital, China Medical University, Taichung, Taiwan
| | - Rong-Lin Yang
- Information Office, China Medical University Hospital, China Medical University, Taichung, Taiwan
| | - Kuan-Ta Chen
- Institute of Information Science, Academia Sinica, Taichung, Taiwan
| |
Collapse
|
44
|
Kannan S, Morgan LA, Liang B, Cheung MG, Lin CQ, Mun D, Nader RG, Belghasem ME, Henderson JM, Francis JM, Chitalia VC, Kolachalama VB. Segmentation of Glomeruli Within Trichrome Images Using Deep Learning. Kidney Int Rep 2019; 4:955-962. [PMID: 31317118 PMCID: PMC6612039 DOI: 10.1016/j.ekir.2019.04.008] [Citation(s) in RCA: 88] [Impact Index Per Article: 17.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2018] [Revised: 04/04/2019] [Accepted: 04/08/2019] [Indexed: 12/22/2022] Open
Abstract
Introduction The number of glomeruli and glomerulosclerosis evaluated on kidney biopsy slides constitute standard components of a renal pathology report. Prevailing methods for glomerular assessment remain manual, labor intensive, and nonstandardized. We developed a deep learning framework to accurately identify and segment glomeruli from digitized images of human kidney biopsies. Methods Trichrome-stained images (n = 275) from renal biopsies of 171 patients with chronic kidney disease treated at the Boston Medical Center from 2009 to 2012 were analyzed. A sliding window operation was defined to crop each original image to smaller images. Each cropped image was then evaluated by at least 3 experts into 3 categories: (i) no glomerulus, (ii) normal or partially sclerosed (NPS) glomerulus, and (iii) globally sclerosed (GS) glomerulus. This led to identification of 751 unique images representing nonglomerular regions, 611 images with NPS glomeruli, and 134 images with GS glomeruli. A convolutional neural network (CNN) was trained with cropped images as inputs and corresponding labels as output. Using this model, an image processing routine was developed to scan the test images to segment the GS glomeruli. Results The CNN model was able to accurately discriminate nonglomerular images from NPS and GS images (performance on test data: Accuracy: 92.67% ± 2.02% and Kappa: 0.8681 ± 0.0392). The segmentation model that was based on the CNN multilabel classifier accurately marked the GS glomeruli on the test data (Matthews correlation coefficient = 0.628). Conclusion This work demonstrates the power of deep learning for assessing complex histologic structures from digitized human kidney biopsies.
Collapse
Affiliation(s)
- Shruti Kannan
- Section of Computational Biomedicine, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
| | - Laura A Morgan
- College of Engineering, Boston University, Boston, Massachusetts, USA
| | - Benjamin Liang
- College of Engineering, Boston University, Boston, Massachusetts, USA
| | - McKenzie G Cheung
- College of Engineering, Boston University, Boston, Massachusetts, USA
| | - Christopher Q Lin
- College of Engineering, Boston University, Boston, Massachusetts, USA
| | - Dan Mun
- College of Health & Rehabilitation Sciences, Sargent College, Boston University, Boston, Massachusetts, USA
| | - Ralph G Nader
- Renal Section, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
| | - Mostafa E Belghasem
- Department of Pathology and Laboratory Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
| | - Joel M Henderson
- Department of Pathology and Laboratory Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
| | - Jean M Francis
- Renal Section, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
| | - Vipul C Chitalia
- Renal Section, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA.,Department of Pathology and Laboratory Medicine, Boston University School of Medicine, Boston, Massachusetts, USA.,Whitaker Cardiovascular Institute, Boston University School of Medicine, Boston, Massachusetts, USA.,Veterans Administration Boston Healthcare System, Boston, Massachusetts, USA
| | - Vijaya B Kolachalama
- Section of Computational Biomedicine, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA.,Whitaker Cardiovascular Institute, Boston University School of Medicine, Boston, Massachusetts, USA.,Hariri Institute for Computing and Computational Science & Engineering, Boston University, Boston, Massachusetts, USA
| |
Collapse
|
45
|
Lutnick B, Ginley B, Govind D, McGarry SD, LaViolette PS, Yacoub R, Jain S, Tomaszewski JE, Jen KY, Sarder P. An integrated iterative annotation technique for easing neural network training in medical image analysis. NAT MACH INTELL 2019; 1:112-119. [PMID: 31187088 PMCID: PMC6557463 DOI: 10.1038/s42256-019-0018-3] [Citation(s) in RCA: 68] [Impact Index Per Article: 13.6] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2018] [Accepted: 01/07/2019] [Indexed: 01/29/2023]
Abstract
Neural networks promise to bring robust, quantitative analysis to medical fields. However, their adoption is limited by the technicalities of training these networks and the required volume and quality of human-generated annotations. To address this gap in the field of pathology, we have created an intuitive interface for data annotation and the display of neural network predictions within a commonly used digital pathology whole-slide viewer. This strategy used a 'human-in-the-loop' to reduce the annotation burden. We demonstrate that segmentation of human and mouse renal micro compartments is repeatedly improved when humans interact with automatically generated annotations throughout the training process. Finally, to show the adaptability of this technique to other medical imaging fields, we demonstrate its ability to iteratively segment human prostate glands from radiology imaging data.
Collapse
Affiliation(s)
- Brendon Lutnick
- Department of Pathology & Anatomical Sciences, SUNY Buffalo, New York, NY, USA
| | - Brandon Ginley
- Department of Pathology & Anatomical Sciences, SUNY Buffalo, New York, NY, USA
| | - Darshana Govind
- Department of Pathology & Anatomical Sciences, SUNY Buffalo, New York, NY, USA
| | - Sean D. McGarry
- Department of Biophysics, Medical College of Wisconsin, Wauwatosa, WI, USA
| | - Peter S. LaViolette
- Department of Radiology and Biomedical Engineering, Medical College of Wisconsin, Wauwatosa, WI, USA
| | - Rabi Yacoub
- Department of Medicine, Nephrology, SUNY Buffalo, New York, NY, USA
| | - Sanjay Jain
- Department of Medicine, Nephrology, Washington University School of Medicine, St Louis, MO, USA
| | - John E. Tomaszewski
- Department of Pathology & Anatomical Sciences, SUNY Buffalo, New York, NY, USA
| | - Kuang-Yu Jen
- Department of Pathology, University of California, Davis Medical Center, Sacramento, CA, USA
| | - Pinaki Sarder
- Department of Pathology & Anatomical Sciences, SUNY Buffalo, New York, NY, USA
| |
Collapse
|
46
|
Marsh JN, Matlock MK, Kudose S, Liu TC, Stappenbeck TS, Gaut JP, Swamidass SJ. Deep Learning Global Glomerulosclerosis in Transplant Kidney Frozen Sections. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:2718-2728. [PMID: 29994669 PMCID: PMC6296264 DOI: 10.1109/tmi.2018.2851150] [Citation(s) in RCA: 92] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Transplantable kidneys are in very limited supply. Accurate viability assessment prior to transplantation could minimize organ discard. Rapid and accurate evaluation of intra-operative donor kidney biopsies is essential for determining which kidneys are eligible for transplantation. The criterion for accepting or rejecting donor kidneys relies heavily on pathologist determination of the percent of glomeruli (determined from a frozen section) that are normal and sclerotic. This percentage is a critical measurement that correlates with transplant outcome. Inter- and intra-observer variability in donor biopsy evaluation is, however, significant. An automated method for determination of percent global glomerulosclerosis could prove useful in decreasing evaluation variability, increasing throughput, and easing the burden on pathologists. Here, we describe the development of a deep learning model that identifies and classifies non-sclerosed and sclerosed glomeruli in whole-slide images of donor kidney frozen section biopsies. This model extends a convolutional neural network (CNN) pre-trained on a large database of digital images. The extended model, when trained on just 48 whole slide images, exhibits slide-level evaluation performance on par with expert renal pathologists. Encouragingly, the model's performance is robust to slide preparation artifacts associated with frozen section preparation. The model substantially outperforms a model trained on image patches of isolated glomeruli, in terms of both accuracy and speed. The methodology overcomes the technical challenge of applying a pretrained CNN bottleneck model to whole-slide image classification. The traditional patch-based approach, while exhibiting deceptively good performance classifying isolated patches, does not translate successfully to whole-slide image segmentation in this setting. As the first model reported that identifies and classifies normal and sclerotic glomeruli in frozen kidney sections, and thus the first model reported in the literature relevant to kidney transplantation, it may become an essential part of donor kidney biopsy evaluation in the clinical setting.
Collapse
|
47
|
Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, Moreira AL, Razavian N, Tsirigos A. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med 2018; 24:1559-1567. [PMID: 30224757 PMCID: PMC9847512 DOI: 10.1038/s41591-018-0177-5] [Citation(s) in RCA: 1331] [Impact Index Per Article: 221.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2017] [Accepted: 07/06/2018] [Indexed: 02/06/2023]
Abstract
Visual inspection of histopathology slides is one of the main methods used by pathologists to assess the stage, type and subtype of lung tumors. Adenocarcinoma (LUAD) and squamous cell carcinoma (LUSC) are the most prevalent subtypes of lung cancer, and their distinction requires visual inspection by an experienced pathologist. In this study, we trained a deep convolutional neural network (inception v3) on whole-slide images obtained from The Cancer Genome Atlas to accurately and automatically classify them into LUAD, LUSC or normal lung tissue. The performance of our method is comparable to that of pathologists, with an average area under the curve (AUC) of 0.97. Our model was validated on independent datasets of frozen tissues, formalin-fixed paraffin-embedded tissues and biopsies. Furthermore, we trained the network to predict the ten most commonly mutated genes in LUAD. We found that six of them-STK11, EGFR, FAT1, SETBP1, KRAS and TP53-can be predicted from pathology images, with AUCs from 0.733 to 0.856 as measured on a held-out population. These findings suggest that deep-learning models can assist pathologists in the detection of cancer subtype or gene mutations. Our approach can be applied to any cancer type, and the code is available at https://github.com/ncoudray/DeepPATH .
Collapse
Affiliation(s)
- Nicolas Coudray
- Applied Bioinformatics Laboratories, New York University School of Medicine, NY 10016, USA,Skirball Institute, Dept. of Cell Biology, New York University School of Medicine, NY 10016, USA
| | | | - Theodore Sakellaropoulos
- School of Mechanical Engineering, National Technical University of Athens, Zografou 15780, Greece
| | - Navneet Narula
- Department of Pathology, New York University School of Medicine, NY 10016, USA
| | - Matija Snuderl
- Department of Pathology, New York University School of Medicine, NY 10016, USA
| | - David Fenyö
- Institute for Systems Genetics, New York University School of Medicine, NY 10016, USA,Department of Biochemistry and molecular Pharmacology, New York University School of Medicine, NY 10016, USA
| | - Andre L. Moreira
- Department of Pathology, New York University School of Medicine, NY 10016, USA,Center for Biospecimen Research and Development, New York University, NY 10016, USA
| | - Narges Razavian
- Department of Population Health and the Center for Healthcare Innovation and Delivery Science, New York University School of Medicine, NY 10016, USA,To whom correspondence should be addressed. Tel: +1 646 501 2693; ; Correspondence may also be addressed to Narges Razavian. Tel: +1 212 263 2234,
| | - Aristotelis Tsirigos
- Applied Bioinformatics Laboratories, New York University School of Medicine, NY 10016, USA,Department of Pathology, New York University School of Medicine, NY 10016, USA,To whom correspondence should be addressed. Tel: +1 646 501 2693; ; Correspondence may also be addressed to Narges Razavian. Tel: +1 212 263 2234,
| |
Collapse
|
48
|
|