1
|
Qin C, Ma H, Hu M, Xu X, Ji C. Performance of artificial intelligence in predicting the prognossis of severe COVID-19: a systematic review and meta-analysis. Front Public Health 2024; 12:1371852. [PMID: 39145161 PMCID: PMC11322443 DOI: 10.3389/fpubh.2024.1371852] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Accepted: 07/18/2024] [Indexed: 08/16/2024] Open
Abstract
Background COVID-19-induced pneumonia has become a persistent health concern, with severe cases posing a significant threat to patient lives. However, the potential of artificial intelligence (AI) in assisting physicians in predicting the prognosis of severe COVID-19 patients remains unclear. Methods To obtain relevant studies, two researchers conducted a comprehensive search of the PubMed, Web of Science, and Embase databases, including all studies published up to October 31, 2023, that utilized AI to predict mortality rates in severe COVID-19 patients. The PROBAST 2019 tool was employed to assess the potential bias in the included studies, and Stata 16 was used for meta-analysis, publication bias assessment, and sensitivity analysis. Results A total of 19 studies, comprising 26 models, were included in the analysis. Among them, the models that incorporated both clinical and radiological data demonstrated the highest performance. These models achieved an overall sensitivity of 0.81 (0.64-0.91), specificity of 0.77 (0.71-0.82), and an overall area under the curve (AUC) of 0.88 (0.85-0.90). Subgroup analysis revealed notable findings. Studies conducted in developed countries exhibited significantly higher predictive specificity for both radiological and combined models (p < 0.05). Additionally, investigations involving non-intensive care unit patients demonstrated significantly greater predictive specificity (p < 0.001). Conclusion The current evidence suggests that artificial intelligence prediction models show promising performance in predicting the prognosis of severe COVID-19 patients. However, due to variations in the suitability of different models for specific populations, it is not yet certain whether they can be fully applied in clinical practice. There is still room for improvement in their predictive capabilities, and future research and development efforts are needed. Systematic review registration https://www.crd.york.ac.uk/prospero/ with the Unique Identifier CRD42023431537.
Collapse
Affiliation(s)
- Chu Qin
- School of Public Health, Zhejiang Chinese Medical University, Hangzhou, China
| | - Huan Ma
- School of Public Health, Zhejiang Chinese Medical University, Hangzhou, China
| | - Mahong Hu
- Department of Critical Care Medicine, Tongde Hospital of Zhejiang Province, Hangzhou, China
| | - Xiujuan Xu
- Department of Critical Care Medicine, Tongde Hospital of Zhejiang Province, Hangzhou, China
| | - Conghua Ji
- School of Public Health, Zhejiang Chinese Medical University, Hangzhou, China
| |
Collapse
|
2
|
Dos Santos L, Silva LL, Pelloso FC, Maia V, Pujals C, Borghesan DH, Carvalho MD, Pedroso RB, Pelloso SM. Use of machine learning to identify protective factors for death from COVID-19 in the ICU: a retrospective study. PeerJ 2024; 12:e17428. [PMID: 38881861 PMCID: PMC11179634 DOI: 10.7717/peerj.17428] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Accepted: 04/29/2024] [Indexed: 06/18/2024] Open
Abstract
Background Patients in serious condition due to COVID-19 often require special care in intensive care units (ICUs). This disease has affected over 758 million people and resulted in 6.8 million deaths worldwide. Additionally, the progression of the disease may vary from individual to individual, that is, it is essential to identify the clinical parameters that indicate a good prognosis for the patient. Machine learning (ML) algorithms have been used for analyzing complex medical data and identifying prognostic indicators. However, there is still an urgent need for a model to elucidate the predictors related to patient outcomes. Therefore, this research aimed to verify, through ML, the variables involved in the discharge of patients admitted to the ICU due to COVID-19. Methods In this study, 126 variables were collected with information on demography, hospital length stay and outcome, chronic diseases and tumors, comorbidities and risk factors, complications and adverse events, health care, and vital indicators of patients admitted to an ICU in southern Brazil. These variables were filtered and then selected by a ML algorithm known as decision trees to identify the optimal set of variables for predicting patient discharge using logistic regression. Finally, a confusion matrix was performed to evaluate the model's performance for the selected variables. Results Of the 532 patients evaluated, 180 were discharged: female (16.92%), with a central venous catheter (23.68%), with a bladder catheter (26.13%), and with an average of 8.46- and 23.65-days using bladder catheter and submitted to mechanical ventilation, respectively. In addition, the chances of discharge increase by 14% for each additional day in the hospital, by 136% for female patients, 716% when there is no bladder catheter, and 737% when no central venous catheter is used. However, the chances of discharge decrease by 3% for each additional year of age and by 9% for each other day of mechanical ventilation. The performance of the training data presented a balanced accuracy of 0.81, sensitivity of 0.74, specificity of 0.88, and the kappa value was 0.64. The test performance had a balanced accuracy of 0.85, sensitivity 0.75, specificity 0.95, and kappa value of 0.73. The McNemar test found that there were no significant differences in the error rates in the training and test data, suggesting good classification. This work showed that female, the absence of a central venous catheter and bladder catheter, shorter mechanical ventilation, and bladder catheter duration were associated with a greater chance of hospital discharge. These results may help develop measures that lead to a good prognosis for the patient.
Collapse
Affiliation(s)
- Lander Dos Santos
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| | - Lincoln Luis Silva
- Department of Emergency Medicine, Duke University School of Medicine, Durham, NC, United States of America
| | | | | | - Constanza Pujals
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| | | | - Maria Dalva Carvalho
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| | - Raíssa Bocchi Pedroso
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| | - Sandra Marisa Pelloso
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| |
Collapse
|
3
|
Hamar Á, Mohammed D, Váradi A, Herczeg R, Balázsfalvi N, Fülesdi B, László I, Gömöri L, Gergely PA, Kovacs GL, Jáksó K, Gombos K. COVID-19 mortality prediction in Hungarian ICU settings implementing random forest algorithm. Sci Rep 2024; 14:11941. [PMID: 38789490 PMCID: PMC11126653 DOI: 10.1038/s41598-024-62791-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Accepted: 05/19/2024] [Indexed: 05/26/2024] Open
Abstract
The emergence of newer SARS-CoV-2 variants of concern (VOCs) profoundly changed the ICU demography; this shift in the virus's genotype and its correlation to lethality in the ICUs is still not fully investigated. We aimed to survey ICU patients' clinical and laboratory parameters in correlation with SARS-CoV-2 variant genotypes to lethality. 503 COVID-19 ICU patients were included in our study beginning in January 2021 through November 2022 in Hungary. Furthermore, we implemented random forest (RF) as a potential predictor regarding SARS-CoV-2 lethality among 649 ICU patients in two ICU centers. Survival analysis and comparison of hypertension (HT), diabetes mellitus (DM), and vaccination effects were conducted. Logistic regression identified DM as a significant mortality risk factor (OR: 1.55, 95% CI 1.06-2.29, p = 0.025), while HT showed marginal significance. Additionally, vaccination demonstrated protection against mortality (p = 0.028). RF detected lethality with 81.42% accuracy (95% CI 73.01-88.11%, [AUC]: 91.6%), key predictors being PaO2/FiO2 ratio, lymphocyte count, and chest Computed Tomography Severity Score (CTSS). Although a smaller number of patients require ICU treatment among Omicron cases, the likelihood of survival has not proportionately increased for those who are admitted to the ICU. In conclusion, our RF model supports more effective clinical decision-making among ICU COVID-19 patients.
Collapse
Affiliation(s)
- Ágoston Hamar
- Department of Laboratory Medicine, Medical School, University of Pécs, Pécs, Hungary
- Molecular Medicine Research Group, Szentágothai Research Centre, University of Pécs, Pécs, Hungary
| | - Daryan Mohammed
- Molecular Medicine Research Group, Szentágothai Research Centre, University of Pécs, Pécs, Hungary
| | - Alex Váradi
- Molecular Medicine Research Group, Szentágothai Research Centre, University of Pécs, Pécs, Hungary
- Institute of Metagenomics, University of Debrecen, Debrecen, Hungary
| | - Róbert Herczeg
- Molecular Medicine Research Group, Szentágothai Research Centre, University of Pécs, Pécs, Hungary
| | - Norbert Balázsfalvi
- Department of Anaesthesiology and Intensive Care, University of Debrecen, Debrecen, Hungary
| | - Béla Fülesdi
- Department of Anaesthesiology and Intensive Care, University of Debrecen, Debrecen, Hungary
| | - István László
- Department of Anaesthesiology and Intensive Care, University of Debrecen, Debrecen, Hungary
| | - Lídia Gömöri
- Doctoral School of Neuroscience, University of Debrecen, Debrecen, Hungary
| | | | - Gabor Laszlo Kovacs
- Department of Laboratory Medicine, Medical School, University of Pécs, Pécs, Hungary
- Molecular Medicine Research Group, Szentágothai Research Centre, University of Pécs, Pécs, Hungary
| | - Krisztián Jáksó
- Department of Anaesthesiology and Intensive Care, Clinical Centre, University of Pécs, Pécs, Hungary
| | - Katalin Gombos
- Department of Laboratory Medicine, Medical School, University of Pécs, Pécs, Hungary.
- Molecular Medicine Research Group, Szentágothai Research Centre, University of Pécs, Pécs, Hungary.
| |
Collapse
|
4
|
Yan JK. A methodological showcase: utilizing minimal clinical parameters for early-stage mortality risk assessment in COVID-19-positive patients. PeerJ Comput Sci 2024; 10:e2017. [PMID: 38855224 PMCID: PMC11157615 DOI: 10.7717/peerj-cs.2017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Accepted: 04/03/2024] [Indexed: 06/11/2024]
Abstract
The scarcity of data is likely to have a negative effect on machine learning (ML). Yet, in the health sciences, data is diverse and can be costly to acquire. Therefore, it is critical to develop methods that can reach similar accuracy with minimal clinical features. This study explores a methodology that aims to build a model using minimal clinical parameters to reach comparable performance to a model trained with a more extensive list of parameters. To develop this methodology, a dataset of over 1,000 COVID-19-positive patients was used. A machine learning model was built with over 90% accuracy when combining 24 clinical parameters using Random Forest (RF) and logistic regression. Furthermore, to obtain minimal clinical parameters to predict the mortality of COVID-19 patients, the features were weighted using both Shapley values and RF feature importance to get the most important factors. The six most highly weighted features that could produce the highest performance metrics were combined for the final model. The accuracy of the final model, which used a combination of six features, is 90% with the random forest classifier and 91% with the logistic regression model. This performance is close to that of a model using 24 combined features (92%), suggesting that highly weighted minimal clinical parameters can be used to reach similar performance. The six clinical parameters identified here are acute kidney injury, glucose level, age, troponin, oxygen level, and acute hepatic injury. Among those parameters, acute kidney injury was the highest-weighted feature. Together, a methodology was developed using significantly minimal clinical parameters to reach performance metrics similar to a model trained with a large dataset, highlighting a novel approach to address the problems of clinical data collection for machine learning.
Collapse
|
5
|
Gao J, Zhu Y, Wang W, Wang Z, Dong G, Tang W, Wang H, Wang Y, Harrison EM, Ma L. A comprehensive benchmark for COVID-19 predictive modeling using electronic health records in intensive care. PATTERNS (NEW YORK, N.Y.) 2024; 5:100951. [PMID: 38645764 PMCID: PMC11026964 DOI: 10.1016/j.patter.2024.100951] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Revised: 02/12/2024] [Accepted: 02/13/2024] [Indexed: 04/23/2024]
Abstract
The COVID-19 pandemic highlighted the need for predictive deep-learning models in health care. However, practical prediction task design, fair comparison, and model selection for clinical applications remain a challenge. To address this, we introduce and evaluate two new prediction tasks-outcome-specific length-of-stay and early-mortality prediction for COVID-19 patients in intensive care-which better reflect clinical realities. We developed evaluation metrics, model adaptation designs, and open-source data preprocessing pipelines for these tasks while also evaluating 18 predictive models, including clinical scoring methods and traditional machine-learning, basic deep-learning, and advanced deep-learning models, tailored for electronic health record (EHR) data. Benchmarking results from two real-world COVID-19 EHR datasets are provided, and all results and trained models have been released on an online platform for use by clinicians and researchers. Our efforts contribute to the advancement of deep-learning and machine-learning research in pandemic predictive modeling.
Collapse
Affiliation(s)
- Junyi Gao
- Centre for Medical Informatics, University of Edinburgh, EH16 4UX Edinburgh, UK
- Health Data Research UK, NW1 2BE London, UK
| | | | | | | | - Guiying Dong
- Peking University People’s Hospital, Beijing 100044, China
| | - Wen Tang
- Peking University Third Hospital, Beijing 100191, China
| | - Hao Wang
- Peking University, Beijing 100871, China
| | - Yasha Wang
- Peking University, Beijing 100871, China
| | - Ewen M. Harrison
- Centre for Medical Informatics, University of Edinburgh, EH16 4UX Edinburgh, UK
| | - Liantao Ma
- Peking University, Beijing 100871, China
| |
Collapse
|
6
|
Datta D, George Dalmida S, Martinez L, Newman D, Hashemi J, Khoshgoftaar TM, Shorten C, Sareli C, Eckardt P. Using machine learning to identify patient characteristics to predict mortality of in-patients with COVID-19 in south Florida. Front Digit Health 2023; 5:1193467. [PMID: 37588022 PMCID: PMC10426497 DOI: 10.3389/fdgth.2023.1193467] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Accepted: 07/12/2023] [Indexed: 08/18/2023] Open
Abstract
Introduction The SARS-CoV-2 (COVID-19) pandemic has created substantial health and economic burdens in the US and worldwide. As new variants continuously emerge, predicting critical clinical events in the context of relevant individual risks is a promising option for reducing the overall burden of COVID-19. This study aims to train an AI-driven decision support system that helps build a model to understand the most important features that predict the "mortality" of patients hospitalized with COVID-19. Methods We conducted a retrospective analysis of "5,371" patients hospitalized for COVID-19-related symptoms from the South Florida Memorial Health Care System between March 14th, 2020, and January 16th, 2021. A data set comprising patients' sociodemographic characteristics, pre-existing health information, and medication was analyzed. We trained Random Forest classifier to predict "mortality" for patients hospitalized with COVID-19. Results Based on the interpretability of the model, age emerged as the primary predictor of "mortality", followed by diarrhea, diabetes, hypertension, BMI, early stages of kidney disease, smoking status, sex, pneumonia, and race in descending order of importance. Notably, individuals aged over 65 years (referred to as "older adults"), males, Whites, Hispanics, and current smokers were identified as being at higher risk of death. Additionally, BMI, specifically in the overweight and obese categories, significantly predicted "mortality". These findings indicated that the model effectively learned from various categories, such as patients' sociodemographic characteristics, pre-hospital comorbidities, and medications, with a predominant focus on characterizing pre-hospital comorbidities. Consequently, the model demonstrated the ability to predict "mortality" with transparency and reliability. Conclusion AI can potentially provide healthcare workers with the ability to stratify patients and streamline optimal care solutions when time is of the essence and resources are limited. This work sets the platform for future work that forecasts patient responses to treatments at various levels of disease severity and assesses health disparities and patient conditions that promote improved health care in a broader context. This study contributed to one of the first predictive analyses applying AI/ML techniques to COVID-19 data using a vast sample from South Florida.
Collapse
Affiliation(s)
- Debarshi Datta
- Christine E. Lynn College of Nursing, Florida Atlantic University, Boca Raton, FL, United States
| | - Safiya George Dalmida
- Christine E. Lynn College of Nursing, Florida Atlantic University, Boca Raton, FL, United States
| | - Laurie Martinez
- Christine E. Lynn College of Nursing, Florida Atlantic University, Boca Raton, FL, United States
| | - David Newman
- Christine E. Lynn College of Nursing, Florida Atlantic University, Boca Raton, FL, United States
| | - Javad Hashemi
- College of Engineering & Computer Science, Florida Atlantic University, Boca Raton, FL, United States
| | - Taghi M. Khoshgoftaar
- College of Engineering & Computer Science, Florida Atlantic University, Boca Raton, FL, United States
| | - Connor Shorten
- College of Engineering & Computer Science, Florida Atlantic University, Boca Raton, FL, United States
| | - Candice Sareli
- Memorial Healthcare System, Hollywood, FL, United States
| | - Paula Eckardt
- Memorial Healthcare System, Hollywood, FL, United States
| |
Collapse
|
7
|
Aghakhani A, Shoshtarian Malak J, Karimi Z, Vosoughi F, Zeraati H, Yekaninejad MS. Predicting the COVID-19 mortality among Iranian patients using tree-based models: A cross-sectional study. Health Sci Rep 2023; 6:e1279. [PMID: 37223657 PMCID: PMC10200963 DOI: 10.1002/hsr2.1279] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 05/04/2023] [Accepted: 05/05/2023] [Indexed: 05/25/2023] Open
Abstract
Background and Aims To explore the use of different machine learning models in prediction of COVID-19 mortality in hospitalized patients. Materials and Methods A total of 44,112 patients from six academic hospitals who were admitted for COVID-19 between March 2020 and August 2021 were included in this study. Variables were obtained from their electronic medical records. Random forest-recursive feature elimination was used to select key features. Decision tree, random forest, LightGBM, and XGBoost model were developed. Sensitivity, specificity, accuracy, F-1 score, and receiver operating characteristic (ROC)-AUC were used to compare the prediction performance of different models. Results Random forest-recursive feature elimination selected following features to include in the prediction model: Age, sex, hypertension, malignancy, pneumonia, cardiac problem, cough, dyspnea, and respiratory system disease. XGBoost and LightGBM showed the best performance with an ROC-AUC of 0.83 [0.822-0.842] and 0.83 [0.816-0.837] and sensitivity of 0.77. Conclusion XGBoost, LightGBM, and random forest have a relatively high predictive performance in prediction of mortality in COVID-19 patients and can be applied in hospital settings, however, future research are needed to externally confirm the validation of these models.
Collapse
Affiliation(s)
- Amirhossein Aghakhani
- Department of Epidemiology and Biostatistics, School of Public HealthTehran University of Medical SciencesTehranIran
| | - Jaleh Shoshtarian Malak
- Department of Digital Health, School of MedicineTehran University of Medical SciencesTehranIran
| | - Zahra Karimi
- Department of Epidemiology and Biostatistics, School of Public HealthTehran University of Medical SciencesTehranIran
| | - Fardis Vosoughi
- Department of Orthopedics and Trauma Surgery, Shariati Hospital and School of MedicineTehran University of Medical SciencesTehranIran
| | - Hojjat Zeraati
- Department of Epidemiology and Biostatistics, School of Public HealthTehran University of Medical SciencesTehranIran
| | - Mir Saeed Yekaninejad
- Department of Epidemiology and Biostatistics, School of Public HealthTehran University of Medical SciencesTehranIran
| |
Collapse
|
8
|
Barough SS, Safavi-Naini SAA, Siavoshi F, Tamimi A, Ilkhani S, Akbari S, Ezzati S, Hatamabadi H, Pourhoseingholi MA. Generalizable machine learning approach for COVID-19 mortality risk prediction using on-admission clinical and laboratory features. Sci Rep 2023; 13:2399. [PMID: 36765157 PMCID: PMC9911952 DOI: 10.1038/s41598-023-28943-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Accepted: 01/27/2023] [Indexed: 02/12/2023] Open
Abstract
We aimed to propose a mortality risk prediction model using on-admission clinical and laboratory predictors. We used a dataset of confirmed COVID-19 patients admitted to three general hospitals in Tehran. Clinical and laboratory values were gathered on admission. Six different machine learning models and two feature selection methods were used to assess the risk of in-hospital mortality. The proposed model was selected using the area under the receiver operator curve (AUC). Furthermore, a dataset from an additional hospital was used for external validation. 5320 hospitalized COVID-19 patients were enrolled in the study, with a mortality rate of 17.24% (N = 917). Among 82 features, ten laboratories and 27 clinical features were selected by LASSO. All methods showed acceptable performance (AUC > 80%), except for K-nearest neighbor. Our proposed deep neural network on features selected by LASSO showed AUC scores of 83.4% and 82.8% in internal and external validation, respectively. Furthermore, our imputer worked efficiently when two out of ten laboratory parameters were missing (AUC = 81.8%). We worked intimately with healthcare professionals to provide a tool that can solve real-world needs. Our model confirmed the potential of machine learning methods for use in clinical practice as a decision-support system.
Collapse
Affiliation(s)
- Siavash Shirzadeh Barough
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Seyed Amir Ahmad Safavi-Naini
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Fatemeh Siavoshi
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Atena Tamimi
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Saba Ilkhani
- Department of Surgery, Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School and Harvard T.H Chan School of Public Health, Boston, MA, USA
| | - Setareh Akbari
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Sadaf Ezzati
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Hamidreza Hatamabadi
- Department of Emergency Medicine, School of Medicine, Safety Promotion and Injury Prevention Research Center, Imam Hossein Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Mohamad Amin Pourhoseingholi
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| |
Collapse
|
9
|
Cardiovascular and Renal Comorbidities Included into Neural Networks Predict the Outcome in COVID-19 Patients Admitted to an Intensive Care Unit: Three-Center, Cross-Validation, Age- and Sex-Matched Study. J Cardiovasc Dev Dis 2023; 10:jcdd10020039. [PMID: 36826535 PMCID: PMC9967447 DOI: 10.3390/jcdd10020039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 01/16/2023] [Accepted: 01/19/2023] [Indexed: 01/25/2023] Open
Abstract
Here, we performed a multicenter, age- and sex-matched study to compare the efficiency of various machine learning algorithms in the prediction of COVID-19 fatal outcomes and to develop sensitive, specific, and robust artificial intelligence tools for the prompt triage of patients with severe COVID-19 in the intensive care unit setting. In a challenge against other established machine learning algorithms (decision trees, random forests, extra trees, neural networks, k-nearest neighbors, and gradient boosting: XGBoost, LightGBM, and CatBoost) and multivariate logistic regression as a reference, neural networks demonstrated the highest sensitivity, sufficient specificity, and excellent robustness. Further, neural networks based on coronary artery disease/chronic heart failure, stage 3-5 chronic kidney disease, blood urea nitrogen, and C-reactive protein as the predictors exceeded 90% sensitivity and 80% specificity, reaching AUROC of 0.866 at primary cross-validation and 0.849 at secondary cross-validation on virtual samples generated by the bootstrapping procedure. These results underscore the impact of cardiovascular and renal comorbidities in the context of thrombotic complications characteristic of severe COVID-19. As aforementioned predictors can be obtained from the case histories or are inexpensive to be measured at admission to the intensive care unit, we suggest this predictor composition is useful for the triage of critically ill COVID-19 patients.
Collapse
|
10
|
Deng Y, Liu S, Wang Z, Wang Y, Jiang Y, Liu B. Explainable time-series deep learning models for the prediction of mortality, prolonged length of stay and 30-day readmission in intensive care patients. Front Med (Lausanne) 2022; 9:933037. [PMID: 36250092 PMCID: PMC9554013 DOI: 10.3389/fmed.2022.933037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2022] [Accepted: 09/01/2022] [Indexed: 11/14/2022] Open
Abstract
Background In-hospital mortality, prolonged length of stay (LOS), and 30-day readmission are common outcomes in the intensive care unit (ICU). Traditional scoring systems and machine learning models for predicting these outcomes usually ignore the characteristics of ICU data, which are time-series forms. We aimed to use time-series deep learning models with the selective combination of three widely used scoring systems to predict these outcomes. Materials and methods A retrospective cohort study was conducted on 40,083 patients in ICU from the Medical Information Mart for Intensive Care-IV (MIMIC-IV) database. Three deep learning models, namely, recurrent neural network (RNN), gated recurrent unit (GRU), and long short-term memory (LSTM) with attention mechanisms, were trained for the prediction of in-hospital mortality, prolonged LOS, and 30-day readmission with variables collected during the initial 24 h after ICU admission or the last 24 h before discharge. The inclusion of variables was based on three widely used scoring systems, namely, APACHE II, SOFA, and SAPS II, and the predictors consisted of time-series vital signs, laboratory tests, medication, and procedures. The patients were randomly divided into a training set (80%) and a test set (20%), which were used for model development and model evaluation, respectively. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and Brier scores were used to evaluate model performance. Variable significance was identified through attention mechanisms. Results A total of 33 variables for 40,083 patients were enrolled for mortality and prolonged LOS prediction and 36,180 for readmission prediction. The rates of occurrence of the three outcomes were 9.74%, 27.54%, and 11.79%, respectively. In each of the three outcomes, the performance of RNN, GRU, and LSTM did not differ greatly. Mortality prediction models, prolonged LOS prediction models, and readmission prediction models achieved AUCs of 0.870 ± 0.001, 0.765 ± 0.003, and 0.635 ± 0.018, respectively. The top significant variables co-selected by the three deep learning models were Glasgow Coma Scale (GCS), age, blood urea nitrogen, and norepinephrine for mortality; GCS, invasive ventilation, and blood urea nitrogen for prolonged LOS; and blood urea nitrogen, GCS, and ethnicity for readmission. Conclusion The prognostic prediction models established in our study achieved good performance in predicting common outcomes of patients in ICU, especially in mortality prediction. In addition, GCS and blood urea nitrogen were identified as the most important factors strongly associated with adverse ICU events.
Collapse
Affiliation(s)
- Yuhan Deng
- School of Public Health, Peking University, Beijing, China
| | - Shuang Liu
- School of Public Health, Peking University, Beijing, China
| | - Ziyao Wang
- School of Public Health, Peking University, Beijing, China
| | - Yuxin Wang
- School of Public Health, Peking University, Beijing, China
| | - Yong Jiang
- Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
- China National Clinical Research Center for Neurological Diseases, Beijing, China
- Yong Jiang,
| | - Baohua Liu
- School of Public Health, Peking University, Beijing, China
- *Correspondence: Baohua Liu,
| |
Collapse
|
11
|
Gao J, Yang C, Heintz J, Barrows S, Albers E, Stapel M, Warfield S, Cross A, Sun J. MedML: Fusing medical knowledge and machine learning models for early pediatric COVID-19 hospitalization and severity prediction. iScience 2022; 25:104970. [PMID: 35992304 PMCID: PMC9384332 DOI: 10.1016/j.isci.2022.104970] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2022] [Revised: 08/10/2022] [Accepted: 08/12/2022] [Indexed: 11/21/2022] Open
Abstract
The COVID-19 pandemic has caused devastating economic and social disruption. This has led to a nationwide call for models to predict hospitalization and severe illness in patients with COVID-19 to inform the distribution of limited healthcare resources. To address this challenge, we propose a machine learning model, MedML, to conduct the hospitalization and severity prediction for the pediatric population using electronic health records. MedML extracts the most predictive features based on medical knowledge and propensity scores from over 6 million medical concepts and incorporates the inter-feature relationships in medical knowledge graphs via graph neural networks. We evaluate MedML on the National Cohort Collaborative (N3C) dataset. MedML achieves up to a 7% higher AUROC and 14% higher AUPRC compared to the best baseline machine learning models. MedML is a new machine learnig framework to incorporate clinical domain knowledge and is more predictive and explainable than current data-driven methods.
Collapse
Affiliation(s)
- Junyi Gao
- University of Illinois Urbana Champaign, Champaign, IL, USA
| | - Chaoqi Yang
- University of Illinois Urbana Champaign, Champaign, IL, USA
| | - Joerg Heintz
- University of Illinois Urbana Champaign, Champaign, IL, USA
| | | | | | | | - Sara Warfield
- University of Illinois, College of Medicine Peoria, Department of Research Services, Peoria, IL, USA
- Center of Excellence for Suicide Prevention, Department of Veterans Affairs, Department of Veterans Affairs, Canandaigua, NY, USA
- University of Illinois, College of Medicine Peoria, Department of Pediatrics, Peoria, IL, USA
| | - Adam Cross
- University of Illinois, College of Medicine Peoria, Department of Research Services, Peoria, IL, USA
| | - Jimeng Sun
- University of Illinois Urbana Champaign, Champaign, IL, USA
| | - N3C consortium
- University of Illinois Urbana Champaign, Champaign, IL, USA
- University of Illinois, College of Medicine Peoria, Department of Research Services, Peoria, IL, USA
- OSF HealthCare, Peoria, IL, USA
- Center of Excellence for Suicide Prevention, Department of Veterans Affairs, Department of Veterans Affairs, Canandaigua, NY, USA
- University of Illinois, College of Medicine Peoria, Department of Pediatrics, Peoria, IL, USA
| |
Collapse
|
12
|
Mortality Prediction Analysis among COVID-19 Inpatients Using Clinical Variables and Deep Learning Chest Radiography Imaging Features. Tomography 2022; 8:1791-1803. [PMID: 35894016 PMCID: PMC9326627 DOI: 10.3390/tomography8040151] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Revised: 07/04/2022] [Accepted: 07/07/2022] [Indexed: 11/17/2022] Open
Abstract
The emergence of the COVID-19 pandemic over a relatively brief interval illustrates the need for rapid data-driven approaches to facilitate clinical decision making. We examined a machine learning process to predict inpatient mortality among COVID-19 patients using clinical and chest radiographic data. Modeling was performed with a de-identified dataset of encounters prior to widespread vaccine availability. Non-imaging predictors included demographics, pre-admission clinical history, and past medical history variables. Imaging features were extracted from chest radiographs by applying a deep convolutional neural network with transfer learning. A multi-layer perceptron combining 64 deep learning features from chest radiographs with 98 patient clinical features was trained to predict mortality. The Local Interpretable Model-Agnostic Explanations (LIME) method was used to explain model predictions. Non-imaging data alone predicted mortality with an ROC-AUC of 0.87 ± 0.03 (mean ± SD), while the addition of imaging data improved prediction slightly (ROC-AUC: 0.91 ± 0.02). The application of LIME to the combined imaging and clinical model found HbA1c values to contribute the most to model prediction (17.1 ± 1.7%), while imaging contributed 8.8 ± 2.8%. Age, gender, and BMI contributed 8.7%, 8.2%, and 7.1%, respectively. Our findings demonstrate a viable explainable AI approach to quantify the contributions of imaging and clinical data to COVID mortality predictions.
Collapse
|
13
|
A Machine Learning Approach to Predict the Rehabilitation Outcome in Convalescent COVID-19 Patients. J Pers Med 2022; 12:jpm12030328. [PMID: 35330328 PMCID: PMC8953386 DOI: 10.3390/jpm12030328] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Revised: 02/18/2022] [Accepted: 02/18/2022] [Indexed: 11/18/2022] Open
Abstract
Background: After the acute disease, convalescent coronavirus disease 2019 (COVID-19) patients may experience several persistent manifestations that require multidisciplinary pulmonary rehabilitation (PR). By using a machine learning (ML) approach, we aimed to evaluate the clinical characteristics predicting the effectiveness of PR, expressed by an improved performance at the 6-min walking test (6MWT). Methods: Convalescent COVID-19 patients referring to a Pulmonary Rehabilitation Unit were consecutively screened. The 6MWT performance was partitioned into three classes, corresponding to different degrees of improvement (low, medium, and high) following PR. A multiclass supervised classification learning was performed with random forest (RF), adaptive boosting (ADA-B), and gradient boosting (GB), as well as tree-based and k-nearest neighbors (KNN) as instance-based algorithms. Results: To train and validate our model, we included 189 convalescent COVID-19 patients (74.1% males, mean age 59.7 years). RF obtained the best results in terms of accuracy (83.7%), sensitivity (84.0%), and area under the ROC curve (94.5%), while ADA-B reached the highest specificity (92.7%). Conclusions: Our model enables a good performance in predicting the rehabilitation outcome in convalescent COVID-19 patients.
Collapse
|