1
|
Egevad L, Camilloni A, Delahunt B, Samaratunga H, Eklund M, Kartasalo K. The Role of Artificial Intelligence in the Evaluation of Prostate Pathology. Pathol Int 2025; 75:213-220. [PMID: 40226937 PMCID: PMC12101047 DOI: 10.1111/pin.70015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2024] [Revised: 01/31/2025] [Accepted: 04/07/2025] [Indexed: 04/15/2025]
Abstract
Artificial intelligence (AI) is an emerging tool in diagnostic pathology, including prostate pathology. This review summarizes the possibilities offered by AI and also discusses the challenges and risks. AI has the potential to assist in the diagnosis and grading of prostate cancer. Diagnostic safety can be enhanced by avoiding the accidental underdiagnosis of small lesions. Another possible benefit is a greater degree of standardization of grading. AI for clinical use needs to be trained on large, high-quality data sets that have been assessed by experienced pathologists. A problem with the use of AI in prostate pathology is the plethora of benign mimics of prostate cancer and morphological variants of cancer that are too unusual to allow sufficient training of AI. AI systems need to be able to account for variations in local routines for cutting, staining, and scanning of slides. We also need to be aware of the risk that users will rely too much on the output of an AI system, leading to diagnostic errors and loss of clinical competence. The reporting pathologist must ultimately be responsible for accepting or rejecting the diagnosis proposed by AI.
Collapse
Affiliation(s)
- Lars Egevad
- Department of Oncology‐PathologyKarolinska InstitutetStockholmSweden
| | - Andrea Camilloni
- Department of Medical Epidemiology and BiostatisticsKarolinska InstitutetStockholmSweden
| | - Brett Delahunt
- Department of Oncology‐PathologyKarolinska InstitutetStockholmSweden
- Malaghan Institute of Medical ResearchWellingtonNew Zealand
| | - Hemamali Samaratunga
- Aquesta Pathology and University of Queensland School of MedicineBrisbaneQueenslandAustralia
| | - Martin Eklund
- Department of Medical Epidemiology and BiostatisticsKarolinska InstitutetStockholmSweden
| | - Kimmo Kartasalo
- SciLifeLab, Department of Medical Epidemiology and BiostatisticsKarolinska InstitutetStockholmSweden
| |
Collapse
|
2
|
Xu J, Wang H, Lu M, Bi H, Li D, Xue Z, Zhang Q. An accurate and trustworthy deep learning approach for bladder tumor segmentation with uncertainty estimation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2025; 263:108645. [PMID: 39954510 DOI: 10.1016/j.cmpb.2025.108645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/05/2024] [Revised: 01/19/2025] [Accepted: 02/02/2025] [Indexed: 02/17/2025]
Abstract
BACKGROUND AND OBJECTIVE Although deep learning-based intelligent diagnosis of bladder cancer has achieved excellent performance, the reliability of neural network predicted results may not be evaluated. This study aims to explore a trustworthy AI-based tumor segmentation model, which not only outputs predicted results but also provides confidence information about the predictions. METHODS This paper proposes a novel model for bladder tumor segmentation with uncertainty estimation (BSU), which is not merely able to effectively segment the lesion area but also yields an uncertainty map showing the confidence information of the segmentation results. In contrast to previous uncertainty estimation, we utilize test time augmentation (TTA) and test time dropout (TTD) to estimate aleatoric uncertainty and epistemic uncertainty in both internal and external datasets to explore the effects of both uncertainties on different datasets. RESULTS Our BSU model achieved the Dice coefficients of 0.766 and 0.848 on internal and external cystoscopy datasets, respectively, along with accuracy of 0.950 and 0.954. Compared to the state-of-the-art methods, our BSU model demonstrated superior performance, which was further validated by the statistically significance of the t-tests at the conventional level. Clinical experiments verified the practical value of uncertainty estimation in real-world bladder cancer diagnostics. CONCLUSIONS The proposed BSU model is able to visualize the confidence of the segmentation results, serving as a valuable addition for assisting urologists in enhancing both the precision and efficiency of bladder cancer diagnoses in clinical practice.
Collapse
Affiliation(s)
- Jie Xu
- School of Information Technology and Management, University of International Business and Economics, Beijing 100029, China
| | - Haixin Wang
- Cadre Medical Department, The 1st medical Center, Chinese PLA General Hospital, Beijing 100853, China
| | - Min Lu
- Department of Pathology, School of Basic Medical Sciences, Peking University Third Hospital, Peking University Health Science Center, Beijing 100191, China
| | - Hai Bi
- Department of Urology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 201620, China
| | - Deng Li
- Department of Urology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 201620, China
| | - Zixuan Xue
- Department of Urology, Peking University Third Hospital, Beijing 100191, China
| | - Qi Zhang
- School of Information Technology and Management, University of International Business and Economics, Beijing 100029, China.
| |
Collapse
|
3
|
Ji X, Salmon R, Mulliqi N, Khan U, Wang Y, Blilie A, Olsson H, Pedersen BG, Sørensen KD, Ulhøi BP, Kjosavik SR, Janssen EAM, Rantalainen M, Egevad L, Ruusuvuori P, Eklund M, Kartasalo K. Physical Color Calibration of Digital Pathology Scanners for Robust Artificial Intelligence-Assisted Cancer Diagnosis. Mod Pathol 2025; 38:100715. [PMID: 39826798 DOI: 10.1016/j.modpat.2025.100715] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2024] [Revised: 12/19/2024] [Accepted: 01/08/2025] [Indexed: 01/22/2025]
Abstract
The potential of artificial intelligence (AI) in digital pathology is limited by technical inconsistencies in the production of whole slide images (WSIs). This causes degraded AI performance and poses a challenge for widespread clinical application, as fine-tuning algorithms for each site is impractical. Changes in the imaging workflow can also compromise diagnostic accuracy and patient safety. Physical color calibration of scanners, relying on a biomaterial-based calibrant slide and a spectrophotometric reference measurement, has been proposed for standardizing WSI appearance, but its impact on AI performance has not been investigated. We evaluated whether physical color calibration can enable robust AI performance. We trained fully supervised and foundation model-based AI systems for detecting and Gleason grading prostate cancer using WSIs of prostate biopsies from the STHLM3 clinical trial (n = 3651) and evaluated their performance in 3 external cohorts (n = 1161) with and without calibration. With physical color calibration, the fully supervised system's concordance with pathologists' grading (Cohen linearly weighted κ) improved from 0.439 to 0.619 in the Stavanger University Hospital cohort (n = 860), from 0.354 to 0.738 in the Karolinska University Hospital cohort (n = 229), and from 0.423 to 0.452 in the Aarhus University Hospital cohort (n = 72). The foundation model's concordance improved as follows: from 0.739 to 0.760 (Karolinska), from 0.424 to 0.459 (Aarhus), and from 0.547 to 0.670 (Stavanger). This study demonstrated that physical color calibration provides a potential solution to the variation introduced by different scanners, making AI-based cancer diagnostics more reliable and applicable in diverse clinical settings.
Collapse
Affiliation(s)
- Xiaoyi Ji
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | | | - Nita Mulliqi
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Umair Khan
- Institute of Biomedicine, University of Turku, Turku, Finland
| | - Yinxi Wang
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Anders Blilie
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway; Faculty of Health Sciences, University of Stavanger, Stavanger, Norway
| | - Henrik Olsson
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Bodil Ginnerup Pedersen
- Department of Radiology, Aarhus University Hospital, Aarhus, Denmark; Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Karina Dalsgaard Sørensen
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark; Department of Molecular Medicine, Aarhus University Hospital, Aarhus, Denmark
| | | | - Svein R Kjosavik
- The General Practice and Care Coordination Research Group, Stavanger University Hospital, Stavanger, Norway; Department of Global Public Health and Primary Care, Faculty of Medicine, University of Bergen, Bergen, Norway
| | - Emilius A M Janssen
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway; Department of Chemistry, Bioscience and Environmental Engineering, University of Stavanger, Stavanger, Norway; Institute for Biomedicine and Glycomics, Griffith University, Queensland, Australia
| | - Mattias Rantalainen
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Lars Egevad
- Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden
| | - Pekka Ruusuvuori
- Institute of Biomedicine, University of Turku, Turku, Finland; InFLAMES Research Flagship, University of Turku, Turku, Finland; Faculty of Medicine and Health Technology, Tampere University, Tampere, Finland
| | - Martin Eklund
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Kimmo Kartasalo
- Department of Medical Epidemiology and Biostatistics, SciLifeLab, Karolinska Institutet, Stockholm, Sweden.
| |
Collapse
|
4
|
Sreenivasan AP, Vaivade A, Noui Y, Khoonsari PE, Burman J, Spjuth O, Kultima K. Conformal prediction enables disease course prediction and allows individualized diagnostic uncertainty in multiple sclerosis. NPJ Digit Med 2025; 8:224. [PMID: 40275055 PMCID: PMC12022056 DOI: 10.1038/s41746-025-01616-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2024] [Accepted: 04/04/2025] [Indexed: 04/26/2025] Open
Abstract
Accurate assessment of progression and disease course in multiple sclerosis (MS) is vital for timely and appropriate clinical intervention. The gradual transition from relapsing-remitting MS (RRMS) to secondary progressive MS (SPMS) is often diagnosed retrospectively with a typical delay of three years. To address this diagnostic delay, we developed a predictive model that uses electronic health records to distinguish between RRMS and SPMS at each individual visit. To enable reliable predictions, conformal prediction was implemented at the individual patient level with a confidence of 93%. Our model accurately predicted the change in diagnosis from RRMS to SPMS for patients who transitioned during the study period. Additionally, we identified new patients who, with high probability, are in the transition phase but have not yet received a clinical diagnosis. Our methodology aids in monitoring MS progression and proactively identifying transitioning patients. An anonymized model is available at https://msp-tracker.serve.scilifelab.se/ .
Collapse
Affiliation(s)
| | - Aina Vaivade
- Department of Medical Sciences, Uppsala University, Uppsala, 75185, Sweden
| | - Yassine Noui
- Department of Medical Sciences, Uppsala University, Uppsala, 75185, Sweden
| | - Payam Emami Khoonsari
- Department of Medical Sciences, Uppsala University, Uppsala, 75185, Sweden
- Department of Biochemistry and Biophysics, National Bioinformatics Infrastructure Sweden, Science for Life Laboratory, Stockholm University, Solna, 17121, Sweden
| | - Joachim Burman
- Department of Medical Sciences, Uppsala University, Uppsala, 75185, Sweden
| | - Ola Spjuth
- Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, Uppsala, 75124, Sweden
| | - Kim Kultima
- Department of Medical Sciences, Uppsala University, Uppsala, 75185, Sweden.
| |
Collapse
|
5
|
Zhang X, Wang T, Yan C, Najdawi F, Zhou K, Ma Y, Cheung YM, Malin BA. Implementing Trust in Non-Small Cell Lung Cancer Diagnosis with a Conformalized Uncertainty-Aware AI Framework in Whole-Slide Images. RESEARCH SQUARE 2025:rs.3.rs-5723270. [PMID: 40195980 PMCID: PMC11975025 DOI: 10.21203/rs.3.rs-5723270/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/09/2025]
Abstract
Ensuring trustworthiness is fundamental to the development of artificial intelligence (AI) that is considered societally responsible, particularly in cancer diagnostics, where a misdiagnosis can have dire consequences. Current digital pathology AI models lack systematic solutions to address trustworthiness concerns arising from model limitations and data discrepancies between model deployment and development environments. To address this issue, we developed TRUECAM, a framework designed to ensure both data and model trustworthiness in non-small cell lung cancer subtyping with whole-slide images. TRUECAM integrates 1) a spectral-normalized neural Gaussian process for identifying out-of-scope inputs and 2) an ambiguity-guided elimination of tiles to filter out highly ambiguous regions, addressing data trustworthiness, as well as 3) conformal prediction to ensure controlled error rates. We systematically evaluated the framework across multiple large-scale cancer datasets, leveraging both task-specific and foundation models, illustrate that an AI model wrapped with TRUECAM significantly outperforms models that lack such guidance, in terms of classification accuracy, robustness, interpretability, and data efficiency, while also achieving improvements in fairness. These findings highlight TRUECAM as a versatile wrapper framework for digital pathology AI models with diverse architectural designs, promoting their responsible and effective applications in real-world settings.
Collapse
Affiliation(s)
- Xiaoge Zhang
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong
| | - Tao Wang
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong
| | - Chao Yan
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Fedaa Najdawi
- Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Kai Zhou
- Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong
| | - Yuan Ma
- Department of Mechanical Engineering and Research Institute for Intelligent Wearable Systems, The Hong Kong Polytechnic University, Kowloon, Hong Kong
| | - Yiu-Ming Cheung
- Department of Computer Science, Hong Kong Baptist University, Kowloon Tong, Kowloon, Hong Kong
| | - Bradley A Malin
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
- Department of Computer Science, Vanderbilt University, Nashville, TN, USA
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA
| |
Collapse
|
6
|
He Q, Xu Y, Huang Q, Wang Y, Ye J, He Y, Li J, Zhu L, Wang Z, Guan T. Unveiling pathology-related predictive uncertainty of glomerular lesion recognition using prototype learning. J Biomed Inform 2025; 161:104745. [PMID: 39746430 DOI: 10.1016/j.jbi.2024.104745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2024] [Revised: 10/21/2024] [Accepted: 11/02/2024] [Indexed: 01/04/2025]
Abstract
OBJECTIVE Recognizing glomerular lesions is essential in diagnosing chronic kidney disease. However, deep learning faces challenges due to the lesion heterogeneity, superposition, progression, and tissue incompleteness, leading to uncertainty in model predictions. Therefore, it is crucial to analyze pathology-related predictive uncertainty in glomerular lesion recognition and unveil its relationship with pathological properties and its impact on model performance. METHODS This paper presents a novel framework for pathology-related predictive uncertainty analysis towards glomerular lesion recognition, including prototype learning based predictive uncertainty estimation, pathology-characterized correlation analysis and weight-redistributed prediction rectification. The prototype learning based predictive uncertainty estimation includes deep prototyping, affinity embedding, and multi-dimensional uncertainty fusion. The pathology-characterized correlation analysis is the first to use expert-based and learning- based approach to construct the pathology-related characterization of lesions and tissues. The weight-redistributed prediction rectification module performs reweighting- based lesion recognition. RESULTS To validate the performance, extensive experiments were conducted. Based on the Spearman and Pearson correlation analysis, the proposed framework enables more efficient correlation analysis, and strong correlation with pathology-related characterization can be achieved (c index > 0.6 and p < 0.01). Furthermore, the prediction rectification module demonstrated improved lesion recognition performance across most metrics, with enhancements of up to 6.36 %. CONCLUSION The proposed predictive uncertainty analysis in glomerular lesion recognition offers a valuable approach for assessing computational pathology's predictive uncertainty from a pathology-related perspective. SIGNIFICANCE The paper provides a solution for pathology-related predictive uncertainty estimation in algorithm development and clinical practice.
Collapse
Affiliation(s)
- Qiming He
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, China.
| | - Yingming Xu
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, China.
| | - Qiang Huang
- Shengqiang Technology Co., Ltd., Shenzhen, China.
| | - Yanxia Wang
- Department of Pathology, State Key Laboratory of Cancer Biology, Xijing Hospital, Fourth Military Medical University, School of Basic Medicine, Xi'an, China.
| | - Jing Ye
- Department of Pathology, State Key Laboratory of Cancer Biology, Xijing Hospital, Fourth Military Medical University, School of Basic Medicine, Xi'an, China.
| | - Yonghong He
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, China.
| | - Jing Li
- Department of Pathology, State Key Laboratory of Cancer Biology, Xijing Hospital, Fourth Military Medical University, School of Basic Medicine, Xi'an, China.
| | - Lianghui Zhu
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, China.
| | - Zhe Wang
- Department of Pathology, State Key Laboratory of Cancer Biology, Xijing Hospital, Fourth Military Medical University, School of Basic Medicine, Xi'an, China.
| | - Tian Guan
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, China
| |
Collapse
|
7
|
Szolnoky K, Nordström T, Eklund M. Tomorrow's patient management: LLMs empowered by external tools. Nat Rev Urol 2024:10.1038/s41585-024-00965-w. [PMID: 39567680 DOI: 10.1038/s41585-024-00965-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2024]
Affiliation(s)
- Kelvin Szolnoky
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Tobias Nordström
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
- Department of Clinical Sciences at Danderyd Hospital, Karolinska Institutet, Stockholm, Sweden
| | - Martin Eklund
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden.
| |
Collapse
|
8
|
Yang M, Zhuang J, Hu W, Li J, Wang Y, Zhang Z, Liu C, Chen H. Enhancing Patient Selection in Sepsis Clinical Trials Design Through an AI Enrichment Strategy: Algorithm Development and Validation. J Med Internet Res 2024; 26:e54621. [PMID: 39231425 PMCID: PMC11411223 DOI: 10.2196/54621] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 04/22/2024] [Accepted: 07/21/2024] [Indexed: 09/06/2024] Open
Abstract
BACKGROUND Sepsis is a heterogeneous syndrome, and enrollment of more homogeneous patients is essential to improve the efficiency of clinical trials. Artificial intelligence (AI) has facilitated the identification of homogeneous subgroups, but how to estimate the uncertainty of the model outputs when applying AI to clinical decision-making remains unknown. OBJECTIVE We aimed to design an AI-based model for purposeful patient enrollment, ensuring that a patient with sepsis recruited into a trial would still be persistently ill by the time the proposed therapy could impact patient outcome. We also expected that the model could provide interpretable factors and estimate the uncertainty of the model outputs at a customized confidence level. METHODS In this retrospective study, 9135 patients with sepsis requiring vasopressor treatment within 24 hours after sepsis onset were enrolled from Beth Israel Deaconess Medical Center. This cohort was used for model development, and 10-fold cross-validation with 50 repeats was used for internal validation. In total, 3743 patients with sepsis from the eICU Collaborative Research Database were used as the external validation cohort. All included patients with sepsis were stratified based on disease progression trajectories: rapid death, recovery, and persistent ill. A total of 148 variables were selected for predicting the 3 trajectories. Four machine learning algorithms with 3 different setups were used. We estimated the uncertainty of the model outputs using conformal prediction (CP). The Shapley Additive Explanations method was used to explain the model. RESULTS The multiclass gradient boosting machine was identified as the best-performing model with good discrimination and calibration performance in both validation cohorts. The mean area under the receiver operating characteristic curve with SD was 0.906 (0.018) for rapid death, 0.843 (0.008) for recovery, and 0.807 (0.010) for persistent ill in the internal validation cohort. In the external validation cohort, the mean area under the receiver operating characteristic curve (SD) was 0.878 (0.003) for rapid death, 0.764 (0.008) for recovery, and 0.696 (0.007) for persistent ill. The maximum norepinephrine equivalence, total urine output, Acute Physiology Score III, mean systolic blood pressure, and the coefficient of variation of oxygen saturation contributed the most. Compared to the model without CP, using the model with CP at a mixed confidence approach reduced overall prediction errors by 27.6% (n=62) and 30.7% (n=412) in the internal and external validation cohorts, respectively, as well as enabled the identification of more potentially persistent ill patients. CONCLUSIONS The implementation of our model has the potential to reduce heterogeneity and enroll more homogeneous patients in sepsis clinical trials. The use of CP for estimating the uncertainty of the model outputs allows for a more comprehensive understanding of the model's reliability and assists in making informed decisions based on the predicted outcomes.
Collapse
Affiliation(s)
- Meicheng Yang
- State Key Laboratory of Digital Medical Engineering, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Jinqiang Zhuang
- Emergency Intensive Care Unit (EICU), The Affiliated Hospital of Yangzhou University, Yangzhou University, Yangzhou, China
- Key Laboratory of Big Data Analysis and Knowledge Services of Yangzhou City, Yangzhou University, Yangzhou, China
| | - Wenhan Hu
- Jiangsu Provincial Key Laboratory of Critical Care Medicine, Department of Critical Care Medicine, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| | - Jianqing Li
- State Key Laboratory of Digital Medical Engineering, School of Instrument Science and Engineering, Southeast University, Nanjing, China
- School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, China
| | - Yu Wang
- Key Laboratory of Big Data Analysis and Knowledge Services of Yangzhou City, Yangzhou University, Yangzhou, China
| | - Zhongheng Zhang
- Department of Emergency Medicine, Key Laboratory of Precision Medicine in Diagnosis and Monitoring Research of Zhejiang Province, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Chengyu Liu
- State Key Laboratory of Digital Medical Engineering, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Hui Chen
- Jiangsu Provincial Key Laboratory of Critical Care Medicine, Department of Critical Care Medicine, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China
| |
Collapse
|
9
|
Liu F. Data Science Methods for Real-World Evidence Generation in Real-World Data. Annu Rev Biomed Data Sci 2024; 7:201-224. [PMID: 38748863 DOI: 10.1146/annurev-biodatasci-102423-113220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/25/2024]
Abstract
In the healthcare landscape, data science (DS) methods have emerged as indispensable tools to harness real-world data (RWD) from various data sources such as electronic health records, claim and registry data, and data gathered from digital health technologies. Real-world evidence (RWE) generated from RWD empowers researchers, clinicians, and policymakers with a more comprehensive understanding of real-world patient outcomes. Nevertheless, persistent challenges in RWD (e.g., messiness, voluminousness, heterogeneity, multimodality) and a growing awareness of the need for trustworthy and reliable RWE demand innovative, robust, and valid DS methods for analyzing RWD. In this article, I review some common current DS methods for extracting RWE and valuable insights from complex and diverse RWD. This article encompasses the entire RWE-generation pipeline, from study design with RWD to data preprocessing, exploratory analysis, methods for analyzing RWD, and trustworthiness and reliability guarantees, along with data ethics considerations and open-source tools. This review, tailored for an audience that may not be experts in DS, aspires to offer a systematic review of DS methods and assists readers in selecting suitable DS methods and enhancing the process of RWE generation for addressing their specific challenges.
Collapse
Affiliation(s)
- Fang Liu
- Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, Indiana, USA;
| |
Collapse
|
10
|
Harris C, Olshvang D, Chellappa R, Santhanam P. Obesity prediction: Novel machine learning insights into waist circumference accuracy. Diabetes Metab Syndr 2024; 18:103113. [PMID: 39243515 DOI: 10.1016/j.dsx.2024.103113] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/24/2024] [Revised: 08/27/2024] [Accepted: 08/28/2024] [Indexed: 09/09/2024]
Abstract
AIMS This study aims to enhance the precision of obesity risk assessments by improving the accuracy of waist circumference predictions using machine learning techniques. METHODS We utilized data from the NHANES and Look AHEAD studies, applying machine learning algorithms augmented with uncertainty quantification. Our approach centered on conformal prediction techniques, which provide a methodological basis for generating prediction intervals that reflect uncertainty levels. This method allows for constructing intervals expected to contain the true waist circumference values with a high degree of probability. RESULTS The application of conformal predictions yielded high coverage rates, achieving 0.955 for men and 0.954 for women in the NHANES dataset. These rates surpassed the expected performance benchmarks and demonstrated robustness when applied to the Look AHEAD dataset, maintaining coverage rates of 0.951 for men and 0.952 for women. Traditional point prediction models did not show such high consistency or reliability. CONCLUSIONS The findings support the integration of waist circumference into standard clinical practice for obesity-related risk assessments using machine learning approaches.
Collapse
Affiliation(s)
- Carl Harris
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, 21287, USA
| | - Daniel Olshvang
- Department of Electrical and Computer Engineering, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, 21287, USA
| | - Rama Chellappa
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, 21287, USA; Department of Electrical and Computer Engineering, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, 21287, USA
| | - Prasanna Santhanam
- Division of Endocrinology, Diabetes, and Metabolism, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, 21287, USA.
| |
Collapse
|
11
|
Eklund M. Artificial intelligence for scoring prostate MRI: ready for prospective evaluation. Lancet Oncol 2024; 25:827-828. [PMID: 38876122 DOI: 10.1016/s1470-2045(24)00284-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2024] [Revised: 05/10/2024] [Accepted: 05/15/2024] [Indexed: 06/16/2024]
Affiliation(s)
- Martin Eklund
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm 171 77, Sweden.
| |
Collapse
|
12
|
Arvidsson McShane S, Norinder U, Alvarsson J, Ahlberg E, Carlsson L, Spjuth O. CPSign: conformal prediction for cheminformatics modeling. J Cheminform 2024; 16:75. [PMID: 38943219 PMCID: PMC11214261 DOI: 10.1186/s13321-024-00870-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Accepted: 06/11/2024] [Indexed: 07/01/2024] Open
Abstract
Conformal prediction has seen many applications in pharmaceutical science, being able to calibrate outputs of machine learning models and producing valid prediction intervals. We here present the open source software CPSign that is a complete implementation of conformal prediction for cheminformatics modeling. CPSign implements inductive and transductive conformal prediction for classification and regression, and probabilistic prediction with the Venn-ABERS methodology. The main chemical representation is signatures but other types of descriptors are also supported. The main modeling methodology is support vector machines (SVMs), but additional modeling methods are supported via an extension mechanism, e.g. DeepLearning4J models. We also describe features for visualizing results from conformal models including calibration and efficiency plots, as well as features to publish predictive models as REST services. We compare CPSign against other common cheminformatics modeling approaches including random forest, and a directed message-passing neural network. The results show that CPSign produces robust predictive performance with comparative predictive efficiency, with superior runtime and lower hardware requirements compared to neural network based models. CPSign has been used in several studies and is in production-use in multiple organizations. The ability to work directly with chemical input files, perform descriptor calculation and modeling with SVM in the conformal prediction framework, with a single software package having a low footprint and fast execution time makes CPSign a convenient and yet flexible package for training, deploying, and predicting on chemical data. CPSign can be downloaded from GitHub at https://github.com/arosbio/cpsign .Scientific contribution CPSign provides a single software that allows users to perform data preprocessing, modeling and make predictions directly on chemical structures, using conformal and probabilistic prediction. Building and evaluating new models can be achieved at a high abstraction level, without sacrificing flexibility and predictive performance-showcased with a method evaluation against contemporary modeling approaches, where CPSign performs on par with a state-of-the-art deep learning based model.
Collapse
Affiliation(s)
- Staffan Arvidsson McShane
- Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, Uppsala, 75124, Sweden
| | - Ulf Norinder
- Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, Uppsala, 75124, Sweden
- Department of Computer and Systems Sciences, Stockholm University, Stockholm, 10587, Sweden
- MTM Research Centre, School of Science and Technology, Örebro University, Örebro, 70182, Sweden
| | - Jonathan Alvarsson
- Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, Uppsala, 75124, Sweden
| | - Ernst Ahlberg
- Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, Uppsala, 75124, Sweden
- Department of Computer Science, Royal Holloway University of London, Egham, TW20 0EX, UK
| | - Lars Carlsson
- Department of Computer Science, Royal Holloway University of London, Egham, TW20 0EX, UK
- Department of Computing, Jönköping University, Jönköping, 55111, Sweden
| | - Ola Spjuth
- Department of Pharmaceutical Biosciences and Science for Life Laboratory, Uppsala University, Uppsala, 75124, Sweden.
| |
Collapse
|
13
|
Williams DKA, Graifman G, Hussain N, Amiel M, Tran P, Reddy A, Haider A, Kavitesh BK, Li A, Alishahian L, Perera N, Efros C, Babu M, Tharakan M, Etienne M, Babu BA. Digital pathology, deep learning, and cancer: a narrative review. Transl Cancer Res 2024; 13:2544-2560. [PMID: 38881914 PMCID: PMC11170525 DOI: 10.21037/tcr-23-964] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Accepted: 03/24/2024] [Indexed: 06/18/2024]
Abstract
Background and Objective Cancer is a leading cause of morbidity and mortality worldwide. The emergence of digital pathology and deep learning technologies signifies a transformative era in healthcare. These technologies can enhance cancer detection, streamline operations, and bolster patient care. A substantial gap exists between the development phase of deep learning models in controlled laboratory environments and their translations into clinical practice. This narrative review evaluates the current landscape of deep learning and digital pathology, analyzing the factors influencing model development and implementation into clinical practice. Methods We searched multiple databases, including Web of Science, Arxiv, MedRxiv, BioRxiv, Embase, PubMed, DBLP, Google Scholar, IEEE Xplore, Semantic Scholar, and Cochrane, targeting articles on whole slide imaging and deep learning published from 2014 and 2023. Out of 776 articles identified based on inclusion criteria, we selected 36 papers for the analysis. Key Content and Findings Most articles in this review focus on the in-laboratory phase of deep learning model development, a critical stage in the deep learning lifecycle. Challenges arise during model development and their integration into clinical practice. Notably, lab performance metrics may not always match real-world clinical outcomes. As technology advances and regulations evolve, we expect more clinical trials to bridge this performance gap and validate deep learning models' effectiveness in clinical care. High clinical accuracy is vital for informed decision-making throughout a patient's cancer care. Conclusions Deep learning technology can enhance cancer detection, clinical workflows, and patient care. Challenges may arise during model development. The deep learning lifecycle involves data preprocessing, model development, and clinical implementation. Achieving health equity requires including diverse patient groups and eliminating bias during implementation. While model development is integral, most articles focus on the pre-deployment phase. Future longitudinal studies are crucial for validating models in real-world settings post-deployment. A collaborative approach among computational pathologists, technologists, industry, and healthcare providers is essential for driving adoption in clinical settings.
Collapse
Affiliation(s)
| | | | - Nowair Hussain
- Department of Internal Medicine, Overlook Medical Center, Summit, NJ, USA
| | | | | | - Arjun Reddy
- Applied Mathematics & Statistics Stony Brook University, Stony Brook, NY, USA
| | - Ali Haider
- Department of Artificial Intelligence, Yeshiva University, New York, NY, USA
| | - Bali Kumar Kavitesh
- Centre for Frontier AI Research (CFAR), Agency for Science, Technology, and Research (A*STAR), Singapore, Singapore
| | - Austin Li
- New York Medical College, Valhalla, NY, USA
| | | | | | | | - Myoungmee Babu
- Artificial Intelligence and Mathematics, New York City Department of Education, New York, NY, USA
| | | | - Mill Etienne
- Department of Neurology, New York Medical College, Valhalla, NY, USA
| | - Benson A Babu
- New York Medical College, Valhalla, NY, USA
- Department of Hospital Medicine, Wyckoff, Medical Center, New York, NY, USA
| |
Collapse
|
14
|
Wei ML, Tada M, So A, Torres R. Artificial intelligence and skin cancer. Front Med (Lausanne) 2024; 11:1331895. [PMID: 38566925 PMCID: PMC10985205 DOI: 10.3389/fmed.2024.1331895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 02/26/2024] [Indexed: 04/04/2024] Open
Abstract
Artificial intelligence is poised to rapidly reshape many fields, including that of skin cancer screening and diagnosis, both as a disruptive and assistive technology. Together with the collection and availability of large medical data sets, artificial intelligence will become a powerful tool that can be leveraged by physicians in their diagnoses and treatment plans for patients. This comprehensive review focuses on current progress toward AI applications for patients, primary care providers, dermatologists, and dermatopathologists, explores the diverse applications of image and molecular processing for skin cancer, and highlights AI's potential for patient self-screening and improving diagnostic accuracy for non-dermatologists. We additionally delve into the challenges and barriers to clinical implementation, paths forward for implementation and areas of active research.
Collapse
Affiliation(s)
- Maria L. Wei
- Department of Dermatology, University of California, San Francisco, San Francisco, CA, United States
- Dermatology Service, San Francisco VA Health Care System, San Francisco, CA, United States
| | - Mikio Tada
- Institute for Neurodegenerative Diseases, University of California, San Francisco, San Francisco, CA, United States
| | - Alexandra So
- School of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Rodrigo Torres
- Dermatology Service, San Francisco VA Health Care System, San Francisco, CA, United States
| |
Collapse
|
15
|
Yang M, Chen H, Hu W, Mischi M, Shan C, Li J, Long X, Liu C. Development and Validation of an Interpretable Conformal Predictor to Predict Sepsis Mortality Risk: Retrospective Cohort Study. J Med Internet Res 2024; 26:e50369. [PMID: 38498038 PMCID: PMC10985608 DOI: 10.2196/50369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 10/16/2023] [Accepted: 01/24/2024] [Indexed: 03/19/2024] Open
Abstract
BACKGROUND Early and reliable identification of patients with sepsis who are at high risk of mortality is important to improve clinical outcomes. However, 3 major barriers to artificial intelligence (AI) models, including the lack of interpretability, the difficulty in generalizability, and the risk of automation bias, hinder the widespread adoption of AI models for use in clinical practice. OBJECTIVE This study aimed to develop and validate (internally and externally) a conformal predictor of sepsis mortality risk in patients who are critically ill, leveraging AI-assisted prediction modeling. The proposed approach enables explaining the model output and assessing its confidence level. METHODS We retrospectively extracted data on adult patients with sepsis from a database collected in a teaching hospital at Beth Israel Deaconess Medical Center for model training and internal validation. A large multicenter critical care database from the Philips eICU Research Institute was used for external validation. A total of 103 clinical features were extracted from the first day after admission. We developed an AI model using gradient-boosting machines to predict the mortality risk of sepsis and used Mondrian conformal prediction to estimate the prediction uncertainty. The Shapley additive explanation method was used to explain the model. RESULTS A total of 16,746 (80%) patients from Beth Israel Deaconess Medical Center were used to train the model. When tested on the internal validation population of 4187 (20%) patients, the model achieved an area under the receiver operating characteristic curve of 0.858 (95% CI 0.845-0.871), which was reduced to 0.800 (95% CI 0.789-0.811) when externally validated on 10,362 patients from the Philips eICU database. At a specified confidence level of 90% for the internal validation cohort the percentage of error predictions (n=438) out of all predictions (n=4187) was 10.5%, with 1229 (29.4%) predictions requiring clinician review. In contrast, the AI model without conformal prediction made 1449 (34.6%) errors. When externally validated, more predictions (n=4004, 38.6%) were flagged for clinician review due to interdatabase heterogeneity. Nevertheless, the model still produced significantly lower error rates compared to the point predictions by AI (n=1221, 11.8% vs n=4540, 43.8%). The most important predictors identified in this predictive model were Acute Physiology Score III, age, urine output, vasopressors, and pulmonary infection. Clinically relevant risk factors contributing to a single patient were also examined to show how the risk arose. CONCLUSIONS By combining model explanation and conformal prediction, AI-based systems can be better translated into medical practice for clinical decision-making.
Collapse
Affiliation(s)
- Meicheng Yang
- State Key Laboratory of Digital Medical Engineering, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Hui Chen
- Department of Critical Care Medicine, Jiangsu Provincial Key Laboratory of Critical Care Medicine, Zhongda Hospital, Southeast University, Nanjing, China
| | - Wenhan Hu
- Department of Critical Care Medicine, Jiangsu Provincial Key Laboratory of Critical Care Medicine, Zhongda Hospital, Southeast University, Nanjing, China
| | - Massimo Mischi
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Caifeng Shan
- College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao, China
- School of Intelligence Science and Technology, Nanjing University, Nanjing, China
| | - Jianqing Li
- State Key Laboratory of Digital Medical Engineering, School of Instrument Science and Engineering, Southeast University, Nanjing, China
- School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, China
| | - Xi Long
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Chengyu Liu
- State Key Laboratory of Digital Medical Engineering, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| |
Collapse
|
16
|
Kanwal N, López-Pérez M, Kiraz U, Zuiverloon TCM, Molina R, Engan K. Are you sure it's an artifact? Artifact detection and uncertainty quantification in histological images. Comput Med Imaging Graph 2024; 112:102321. [PMID: 38199127 DOI: 10.1016/j.compmedimag.2023.102321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 11/08/2023] [Accepted: 12/12/2023] [Indexed: 01/12/2024]
Abstract
Modern cancer diagnostics involves extracting tissue specimens from suspicious areas and conducting histotechnical procedures to prepare a digitized glass slide, called Whole Slide Image (WSI), for further examination. These procedures frequently introduce different types of artifacts in the obtained WSI, and histological artifacts might influence Computational Pathology (CPATH) systems further down to a diagnostic pipeline if not excluded or handled. Deep Convolutional Neural Networks (DCNNs) have achieved promising results for the detection of some WSI artifacts, however, they do not incorporate uncertainty in their predictions. This paper proposes an uncertainty-aware Deep Kernel Learning (DKL) model to detect blurry areas and folded tissues, two types of artifacts that can appear in WSIs. The proposed probabilistic model combines a CNN feature extractor and a sparse Gaussian Processes (GPs) classifier, which improves the performance of current state-of-the-art artifact detection DCNNs and provides uncertainty estimates. We achieved 0.996 and 0.938 F1 scores for blur and folded tissue detection on unseen data, respectively. In extensive experiments, we validated the DKL model on unseen data from external independent cohorts with different staining and tissue types, where it outperformed DCNNs. Interestingly, the DKL model is more confident in the correct predictions and less in the wrong ones. The proposed DKL model can be integrated into the preprocessing pipeline of CPATH systems to provide reliable predictions and possibly serve as a quality control tool.
Collapse
Affiliation(s)
- Neel Kanwal
- Department of Electrical Engineering and Computer Science, University of Stavanger, 4021 Stavanger, Norway.
| | - Miguel López-Pérez
- Department of Computer Science and Artificial Intelligence, University of Granada, 18071 Granada, Spain
| | - Umay Kiraz
- Department of Pathology, Stavanger University Hospital, 4011 Stavanger, Norway; Department of Chemistry, Bioscience and Environmental Engineering, University of Stavanger, 4021 Stavanger, Norway
| | - Tahlita C M Zuiverloon
- Department of Urology, University Medical Center Rotterdam, Erasmus MC Cancer Institute, 1035 GD Rotterdam, The Netherlands
| | - Rafael Molina
- Department of Computer Science and Artificial Intelligence, University of Granada, 18071 Granada, Spain
| | - Kjersti Engan
- Department of Electrical Engineering and Computer Science, University of Stavanger, 4021 Stavanger, Norway
| |
Collapse
|
17
|
Wang M, Lin T, Wang L, Lin A, Zou K, Xu X, Zhou Y, Peng Y, Meng Q, Qian Y, Deng G, Wu Z, Chen J, Lin J, Zhang M, Zhu W, Zhang C, Zhang D, Goh RSM, Liu Y, Pang CP, Chen X, Chen H, Fu H. Uncertainty-inspired open set learning for retinal anomaly identification. Nat Commun 2023; 14:6757. [PMID: 37875484 PMCID: PMC10598011 DOI: 10.1038/s41467-023-42444-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Accepted: 10/11/2023] [Indexed: 10/26/2023] Open
Abstract
Failure to recognize samples from the classes unseen during training is a major limitation of artificial intelligence in the real-world implementation for recognition and classification of retinal anomalies. We establish an uncertainty-inspired open set (UIOS) model, which is trained with fundus images of 9 retinal conditions. Besides assessing the probability of each category, UIOS also calculates an uncertainty score to express its confidence. Our UIOS model with thresholding strategy achieves an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set, external target categories (TC)-JSIEC dataset and TC-unseen testing set, respectively, compared to the F1 score of 92.20%, 80.69% and 64.74% by the standard AI model. Furthermore, UIOS correctly predicts high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images. UIOS provides a robust method for real-world screening of retinal anomalies.
Collapse
Affiliation(s)
- Meng Wang
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Tian Lin
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China
| | - Lianyu Wang
- College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, 211100, Nanjing, Jiangsu, China
- Laboratory of Brain-Machine Intelligence Technology, Ministry of Education Nanjing University of Aeronautics and Astronautics, 211106, Nanjing, Jiangsu, China
| | - Aidi Lin
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China
| | - Ke Zou
- National Key Laboratory of Fundamental Science on Synthetic Vision and the College of Computer Science, Sichuan University, 610065, Chengdu, Sichuan, China
| | - Xinxing Xu
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Yi Zhou
- School of Electronics and Information Engineering, Soochow University, 215006, Suzhou, Jiangsu, China
| | - Yuanyuan Peng
- School of Biomedical Engineering, Anhui Medical University, 230032, Hefei, Anhui, China
| | - Qingquan Meng
- School of Electronics and Information Engineering, Soochow University, 215006, Suzhou, Jiangsu, China
| | - Yiming Qian
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Guoyao Deng
- National Key Laboratory of Fundamental Science on Synthetic Vision and the College of Computer Science, Sichuan University, 610065, Chengdu, Sichuan, China
| | - Zhiqun Wu
- Longchuan People's Hospital, 517300, Heyuan, Guangdong, China
| | - Junhong Chen
- Puning People's Hospital, 515300, Jieyang, Guangdong, China
| | - Jianhong Lin
- Haifeng PengPai Memory Hospital, 516400, Shanwei, Guangdong, China
| | - Mingzhi Zhang
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China
| | - Weifang Zhu
- School of Electronics and Information Engineering, Soochow University, 215006, Suzhou, Jiangsu, China
| | - Changqing Zhang
- College of Intelligence and Computing, Tianjin University, 300350, Tianjin, China
| | - Daoqiang Zhang
- College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, 211100, Nanjing, Jiangsu, China
- Laboratory of Brain-Machine Intelligence Technology, Ministry of Education Nanjing University of Aeronautics and Astronautics, 211106, Nanjing, Jiangsu, China
| | - Rick Siow Mong Goh
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Yong Liu
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore
| | - Chi Pui Pang
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, 999077, Hong Kong, China
| | - Xinjian Chen
- School of Electronics and Information Engineering, Soochow University, 215006, Suzhou, Jiangsu, China.
- State Key Laboratory of Radiation Medicine and Protection, Soochow University, 215006, Suzhou, China.
| | - Haoyu Chen
- Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong Kong, 515041, Shantou, Guangdong, China.
| | - Huazhu Fu
- Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Republic of Singapore.
| |
Collapse
|
18
|
Wong S, Simmons A, Villicana JR, Barnett S. Estimating Patient-Level Uncertainty in Seizure Detection Using Group-Specific Out-of-Distribution Detection Technique. SENSORS (BASEL, SWITZERLAND) 2023; 23:8375. [PMID: 37896469 PMCID: PMC10611125 DOI: 10.3390/s23208375] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 09/29/2023] [Accepted: 10/05/2023] [Indexed: 10/29/2023]
Abstract
Epilepsy is a chronic neurological disorder affecting around 1% of the global population, characterized by recurrent epileptic seizures. Accurate diagnosis and treatment are crucial for reducing mortality rates. Recent advancements in machine learning (ML) algorithms have shown potential in aiding clinicians with seizure detection in electroencephalography (EEG) data. However, these algorithms face significant challenges due to the patient-specific variability in seizure patterns and the limited availability of high-quality EEG data for training, causing erratic predictions. These erratic predictions are harmful, especially for high-stake domains in healthcare, negatively affecting patients. Therefore, ensuring safety in AI is of the utmost importance. In this study, we propose a novel ensemble method for uncertainty quantification to identify patients with low-confidence predictions in ML-based seizure detection algorithms. Our approach aims to mitigate high-risk predictions in previously unseen seizure patients, thereby enhancing the robustness of existing seizure detection algorithms. Additionally, our method can be implemented with most of the deep learning (DL) models. We evaluated the proposed method against established uncertainty detection techniques, demonstrating its effectiveness in identifying patients for whom the model's predictions are less certain. Our proposed method managed to achieve 87%, 89% and 75% in accuracy, specificity and sensitivity, respectively. This study represents a novel attempt to improve the reliability and robustness of DL algorithms in the domain of seizure detection. This study underscores the value of integrating uncertainty quantification into ML algorithms for seizure detection, offering clinicians a practical tool to gauge the applicability of ML models for individual patients.
Collapse
Affiliation(s)
- Sheng Wong
- Applied Artificial Intelligence Institute, Deakin University, Burwood, VIC 3125, Australia
| | - Anj Simmons
- Applied Artificial Intelligence Institute, Deakin University, Burwood, VIC 3125, Australia
| | | | - Scott Barnett
- Applied Artificial Intelligence Institute, Deakin University, Burwood, VIC 3125, Australia
| |
Collapse
|
19
|
Agten A, Blázquez-Moreno A, Crabbe M, Tuefferd M, Goehlmann H, Geys H, Peng CY, Claes J, Neyens T, Faes C. Measures of spatial heterogeneity in the liver tissue micro-environment as predictive factors for fibrosis score. Comput Biol Med 2023; 165:107382. [PMID: 37634463 DOI: 10.1016/j.compbiomed.2023.107382] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Revised: 08/02/2023] [Accepted: 08/14/2023] [Indexed: 08/29/2023]
Abstract
The organization and interaction between hepatocytes and other hepatic non-parenchymal cells plays a pivotal role in maintaining normal liver function and structure. Although spatial heterogeneity within the tumor micro-environment has been proven to be a fundamental feature in cancer progression, the role of liver tissue topology and micro-environmental factors in the context of liver damage in chronic infection has not been widely studied yet. We obtained images from 110 core needle biopsies from a cohort of chronic hepatitis B patients with different fibrosis stages according to METAVIR score. The tissue sections were immunofluorescently stained and imaged to determine the locations of CD45 positive immune cells and HBsAg-negative and HBsAg-positive hepatocytes within the tissue. We applied several descriptive techniques adopted from ecology, including Getis-Ord, the Shannon Index and the Morisita-Horn Index, to quantify the extent to which immune cells and different types of liver cells co-localize in the tissue biopsies. Additionally, we modeled the spatial distribution of the different cell types using a joint log-Gaussian Cox process and proposed several features to quantify spatial heterogeneity. We then related these measures to the patient fibrosis stage by using a linear discriminant analysis approach. Our analysis revealed that the co-localization of HBsAg-negative hepatocytes with immune cells and the co-localization of HBsAg-positive hepatocytes with immune cells are equally important factors for explaining the METAVIR score in chronic hepatitis B patients. Moreover, we found that if we allow for an error of 1 on the METAVIR score, we are able to reach an accuracy of around 80%. With this study we demonstrate how methods adopted from ecology and applied to the liver tissue micro-environment can be used to quantify heterogeneity and how these approaches can be valuable in biomarker analyses for liver topology.
Collapse
Affiliation(s)
- Annelies Agten
- Data Science Institute, UHasselt - Hasselt University, Agoralaan 1, BE 3590 Diepenbeek, Belgium.
| | - Alfonso Blázquez-Moreno
- Discovery Statistics, Global Development, Janssen Research and Development, Turnhoutseweg 30, 2340 Beerse, Belgium
| | - Marjolein Crabbe
- Discovery Statistics, Global Development, Janssen Research and Development, Turnhoutseweg 30, 2340 Beerse, Belgium
| | - Marianne Tuefferd
- Translational Biomarkers, Infectious Diseases, Janssen Research and Development, Turnhoutseweg 30, 2340 Beerse, Belgium
| | - Hinrich Goehlmann
- Translational Biomarkers, Infectious Diseases, Janssen Research and Development, Turnhoutseweg 30, 2340 Beerse, Belgium
| | - Helena Geys
- Discovery Statistics, Global Development, Janssen Research and Development, Turnhoutseweg 30, 2340 Beerse, Belgium
| | | | - Jari Claes
- Data Science Institute, UHasselt - Hasselt University, Agoralaan 1, BE 3590 Diepenbeek, Belgium
| | - Thomas Neyens
- Data Science Institute, UHasselt - Hasselt University, Agoralaan 1, BE 3590 Diepenbeek, Belgium; L-BioStat, KU Leuven, Kapucijnenvoer 35, 3000 Leuven, Belgium
| | - Christel Faes
- Data Science Institute, UHasselt - Hasselt University, Agoralaan 1, BE 3590 Diepenbeek, Belgium
| |
Collapse
|