1
|
Jin Q, Amal S, Rabb JB, Mazhude F, Shivandi V, Kramer RS, Sawyer DB, Winslow RL. Development and Validation of Machine Learning Models for Adverse Events after Cardiac Surgery. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2025:2025.02.24.25322811. [PMID: 40061347 PMCID: PMC11888533 DOI: 10.1101/2025.02.24.25322811] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 03/18/2025]
Abstract
Importance Early recognition of adverse events after cardiac surgery is vital for treatment. However, the widely used Society of Thoracic Surgery (STS) risk model has modest performance in predicting adverse events and only applies <80% of cardiac surgeries. Objective To develop and validate machine learning (ML) models for predicting outcomes after cardiac surgery. Design setting and participants ML models, referred as Roux-MMC model, were developed and validated using a retrospective cohort extracted from the STS Adult Cardiac Surgery Database (ACSD) at Maine Medical Center (MMC) between January 2012 to December 2021. It was further validated on a prospective cohort of MMC between January 2022 to February 2024. The performance of Roux-MMC model is compared with the STS model.Exposure cardiac surgery. Main outcomes and measures Postoperative outcomes: mortality, stroke, renal failure, reoperation, prolonged ventilation, major morbidity or mortality, prolonged length of stay (PLOS) and short length of stay (SLOS). Primary measure: area under the receiver-operating curve (AUROC). Results A retrospective cohort of 9,841 patients (median [IQR] age, 67 [59-74] years; 7,127 [72%] males) and a prospective cohort of 2,305 patients (median [IQR] age, 67 [59-73] years; 1,707 [74%] males) were included. In the prospective cohort, the Roux-MMC model achieves performance for prolonged ventilation (AUROC 0.911 [95% CI, 0.887-0.935]), PLOS (AUROC 0.875 [95% CI, 0.848-0.898]), renal failure (AUROC 0.878 [95% CI, 0.829-0.921]), mortality (AUROC 0.882 [95% CI, 0.837-0.920]), reoperation (AUROC 0.824 [95% CI, 0.787-0.860]), SLOS (AUROC 0.818 [95% CI, 0.801-0.835]) and major morbidity or mortality (AUROC 0.859 [95% CI, 0.832-0.884]). The Roux-MMC model outperforms the STS model for all 8 outcomes, achieving 0.020-0.167 greater AUROC. The Roux-MMC model covers all cardiac surgery patients, while the STS model applies to only 65% in the retrospective and 77% in the prospective cohorts. Conclusion and relevance We developed ML models to predict 8 postoperative outcomes on all cardiac surgery patients using preoperative and intraoperative variables. The Roux-MMC model outperforms the STS model in the prospective cohort. The Roux-MMC model is built on STS ACSD, a data system used in ~1000 US hospitals, thus, it has the potential to easily applied in other hospitals.
Collapse
Affiliation(s)
- Qingchu Jin
- Roux Institute at Northeastern University, Portland ME, USA
- Those authors contribute equally
| | - Saeed Amal
- Roux Institute at Northeastern University, Portland ME, USA
- Department of Bioengineering, Northeastern University, Boston MA, USA
- Those authors contribute equally
| | | | | | - Venkatesh Shivandi
- Roux Institute at Northeastern University, Portland ME, USA
- Koury College of Computer Science, Northeastern University, Boston MA, USA
| | | | - Douglas B Sawyer
- Maine Medical Center, Portland, ME, USA
- MaineHealth Institute for Research, Portland ME, USA
| | - Raimond L Winslow
- Roux Institute at Northeastern University, Portland ME, USA
- Koury College of Computer Science, Northeastern University, Boston MA, USA
- Department of Bioengineering, Northeastern University, Boston MA, USA
- School of Clinical and Rehabilitation Sciences, Northeastern University, Boston MA, USA
- Corresponding author
| |
Collapse
|
2
|
Ni P, Zhang S, Hu W, Diao M. Application of multi-feature-based machine learning models to predict neurological outcomes of cardiac arrest. Resusc Plus 2024; 20:100829. [PMID: 39639943 PMCID: PMC11617783 DOI: 10.1016/j.resplu.2024.100829] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2024] [Revised: 11/01/2024] [Accepted: 11/12/2024] [Indexed: 12/07/2024] Open
Abstract
Cardiac arrest (CA) is a major disease burden worldwide and has a poor prognosis. Early prediction of CA outcomes helps optimize the therapeutic regimen and improve patients' neurological function. As the current guidelines recommend, many factors can be used to evaluate the neurological outcomes of CA patients. Machine learning (ML) has strong analytical abilities and fast computing speed; thus, it plays an irreplaceable role in prediction model development. An increasing number of researchers are using ML algorithms to incorporate demographics, arrest characteristics, clinical variables, biomarkers, physical examination findings, electroencephalograms, imaging, and other factors with predictive value to construct multi-feature prediction models for neurological outcomes of CA survivors. In this review, we explore the current application of ML models using multiple features to predict the neurological outcomes of CA patients. Although the outcome prediction model is still in development, it has strong potential to become a powerful tool in clinical practice.
Collapse
Affiliation(s)
- Peifeng Ni
- Department of Critical Care Medicine, Zhejiang University School of Medicine, Hangzhou, Zhejiang 310000, China
- Department of Critical Care Medicine, Hangzhou First People’s Hospital, Westlake University School of Medicine, Hangzhou, Zhejiang 310000, China
| | - Sheng Zhang
- Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200000, China
| | - Wei Hu
- Department of Critical Care Medicine, Hangzhou First People’s Hospital, Westlake University School of Medicine, Hangzhou, Zhejiang 310000, China
| | - Mengyuan Diao
- Department of Critical Care Medicine, Zhejiang University School of Medicine, Hangzhou, Zhejiang 310000, China
- Department of Critical Care Medicine, Hangzhou First People’s Hospital, Westlake University School of Medicine, Hangzhou, Zhejiang 310000, China
| |
Collapse
|
3
|
Mathur R, Meyfroidt G, Robba C, Stevens RD. Neuromonitoring in the ICU - what, how and why? Curr Opin Crit Care 2024; 30:99-105. [PMID: 38441121 DOI: 10.1097/mcc.0000000000001138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
PURPOSE OF REVIEW We selectively review emerging noninvasive neuromonitoring techniques and the evidence that supports their use in the ICU setting. The focus is on neuromonitoring research in patients with acute brain injury. RECENT FINDINGS Noninvasive intracranial pressure evaluation with optic nerve sheath diameter measurements, transcranial Doppler waveform analysis, or skull mechanical extensometer waveform recordings have potential safety and resource-intensity advantages when compared to standard invasive monitors, however each of these techniques has limitations. Quantitative electroencephalography can be applied for detection of cerebral ischemia and states of covert consciousness. Near-infrared spectroscopy may be leveraged for cerebral oxygenation and autoregulation computation. Automated quantitative pupillometry and heart rate variability analysis have been shown to have diagnostic and/or prognostic significance in selected subtypes of acute brain injury. Finally, artificial intelligence is likely to transform interpretation and deployment of neuromonitoring paradigms individually and when integrated in multimodal paradigms. SUMMARY The ability to detect brain dysfunction and injury in critically ill patients is being enriched thanks to remarkable advances in neuromonitoring data acquisition and analysis. Studies are needed to validate the accuracy and reliability of these new approaches, and their feasibility and implementation within existing intensive care workflows.
Collapse
Affiliation(s)
- Rohan Mathur
- Department of Anesthesiology & Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, MD
- Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Geert Meyfroidt
- Department of Intensive Care Medicine, University Hospitals Leuven, Belgium and Laboratory of Intensive Care Medicine, Department of Cellular and Molecular Medicine, KU Leuven, Belgium
| | - Chiara Robba
- IRCCS Ospedale Policlinico San Martino, Genova, Italy
- Dipartimento di Scienze Chirurgiche e Diagnostiche Integrate, Università degli Studi di Genova, Genova, Italy
| | - Robert D Stevens
- Department of Anesthesiology & Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, MD
- Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD, USA
| |
Collapse
|
4
|
Boshen Y, Yuankang Z, Xinjie Z, Taixi L, Kaifan N, Zhixiang W, Juan S, Junli D, Suiji L, Xia L, Chengxing S. Triglyceride-glucose index is associated with the occurrence and prognosis of cardiac arrest: a multicenter retrospective observational study. Cardiovasc Diabetol 2023; 22:190. [PMID: 37501144 PMCID: PMC10375765 DOI: 10.1186/s12933-023-01918-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Accepted: 07/06/2023] [Indexed: 07/29/2023] Open
Abstract
BACKGROUND Triglyceride-glucose (TyG) index is an efficient indicator of insulin resistance and is proven to be a valuable marker in several cardiovascular diseases. However, the relationship between TyG index and cardiac arrest (CA) remains unclear. The present study aimed to investigate the association of the TyG index with the occurrence and clinical outcomes of CA. METHODS In this retrospective, multicenter, observational study, critically ill patients, including patients post-CA, were identified from the eICU Collaborative Research Database and evaluated. The TyG index for each patient was calculated using values of triglycerides and glucose recorded within 24 h of intensive care unit (ICU) admission. In-hospital mortality and ICU mortality were the primary clinical outcomes. Logistic regression, restricted cubic spline (RCS), and correlation analyses were performed to explore the relationship between the TyG index and clinical outcomes. Propensity score matching (PSM), overlap weighting (OW), and inverse probability of treatment weighting (IPTW) were adopted to balance the baseline characteristics of patients and minimize selection bias to confirm the robustness of the results. Subgroup analysis based on different modifiers was also performed. RESULTS Overall, 24,689 critically ill patients, including 1021 patients post-CA, were enrolled. The TyG index was significantly higher in patients post-CA than in those without CA (9.20 (8.72-9.69) vs. 8.89 (8.45-9.41)), and the TyG index had a moderate discrimination ability to identify patients with CA from the overall population (area under the curve = 0.625). Multivariate logistic regression indicated that the TyG index was an independent risk factor for in-hospital mortality (OR = 1.28, 95% CI: 1.03-1.58) and ICU mortality (OR = 1.27, 95% CI: 1.02-1.58) in patients post-CA. RCS curves revealed that an increased TyG index was linearly related to higher risks of in-hospital and ICU mortality (P for nonlinear: 0.225 and 0.271, respectively). Even after adjusting by PSM, IPTW, and OW, the TyG index remained a risk factor for in-hospital mortality and ICU mortality in patients experiencing CA, which was independent of age, BMI, sex, etc. Correlation analyses revealed that TyG index was negatively correlated with the neurological status of patients post-CA. CONCLUSION Elevated TyG index is significantly associated with the occurrence of CA and higher mortality risk in patients post-CA. Our findings extend the landscape of TyG index in cardiovascular diseases, which requires further prospective cohort study.
Collapse
Affiliation(s)
- Yang Boshen
- Department of Cardiology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Zhu Yuankang
- Institute for Developmental and Regenerative Cardiovascular Medicine, Xinhua Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
- Department of Gerontology, Xinhua Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Zheng Xinjie
- Department of Respiratory Medicine, The Fourth Affiliated Hospital, College of Medicine, Zhejiang University, Yiwu, China
| | - Li Taixi
- Department of Cardiology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Niu Kaifan
- Department of Cardiology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Wang Zhixiang
- Department of Cardiology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Song Juan
- Xiamen Cardiovascular Hospital, Xiamen University, Xiamen, China
| | - Duan Junli
- Institute for Developmental and Regenerative Cardiovascular Medicine, Xinhua Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
- Department of Gerontology, Xinhua Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Li Suiji
- Xiamen Cardiovascular Hospital, Xiamen University, Xiamen, China.
| | - Lu Xia
- Department of Cardiology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| | - Shen Chengxing
- Department of Cardiology, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| |
Collapse
|
5
|
Rajendran S, Xu Z, Pan W, Ghosh A, Wang F. Data heterogeneity in federated learning with Electronic Health Records: Case studies of risk prediction for acute kidney injury and sepsis diseases in critical care. PLOS DIGITAL HEALTH 2023; 2:e0000117. [PMID: 36920974 PMCID: PMC10016691 DOI: 10.1371/journal.pdig.0000117] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Accepted: 02/10/2023] [Indexed: 03/16/2023]
Abstract
With the wider availability of healthcare data such as Electronic Health Records (EHR), more and more data-driven based approaches have been proposed to improve the quality-of-care delivery. Predictive modeling, which aims at building computational models for predicting clinical risk, is a popular research topic in healthcare analytics. However, concerns about privacy of healthcare data may hinder the development of effective predictive models that are generalizable because this often requires rich diverse data from multiple clinical institutions. Recently, federated learning (FL) has demonstrated promise in addressing this concern. However, data heterogeneity from different local participating sites may affect prediction performance of federated models. Due to acute kidney injury (AKI) and sepsis' high prevalence among patients admitted to intensive care units (ICU), the early prediction of these conditions based on AI is an important topic in critical care medicine. In this study, we take AKI and sepsis onset risk prediction in ICU as two examples to explore the impact of data heterogeneity in the FL framework as well as compare performances across frameworks. We built predictive models based on local, pooled, and FL frameworks using EHR data across multiple hospitals. The local framework only used data from each site itself. The pooled framework combined data from all sites. In the FL framework, each local site did not have access to other sites' data. A model was updated locally, and its parameters were shared to a central aggregator, which was used to update the federated model's parameters and then subsequently, shared with each site. We found models built within a FL framework outperformed local counterparts. Then, we analyzed variable importance discrepancies across sites and frameworks. Finally, we explored potential sources of the heterogeneity within the EHR data. The different distributions of demographic profiles, medication use, and site information contributed to data heterogeneity.
Collapse
Affiliation(s)
- Suraj Rajendran
- Tri-Institutional Computational Biology & Medicine Program, Cornell University, New York, New York, United States of America
| | - Zhenxing Xu
- Division of Health Informatics, Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, United States of America
| | - Weishen Pan
- Division of Health Informatics, Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, United States of America
| | - Arnab Ghosh
- Departments of Medicine, Weill Cornell Medical College, Cornell University, New York, New York, United States of America
| | - Fei Wang
- Division of Health Informatics, Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, United States of America
| |
Collapse
|
6
|
Nguyen HT, Vasconcellos HD, Keck K, Reis JP, Lewis CE, Sidney S, Lloyd-Jones DM, Schreiner PJ, Guallar E, Wu CO, Lima JA, Ambale-Venkatesh B. Multivariate longitudinal data for survival analysis of cardiovascular event prediction in young adults: insights from a comparative explainable study. BMC Med Res Methodol 2023; 23:23. [PMID: 36698064 PMCID: PMC9878947 DOI: 10.1186/s12874-023-01845-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 01/18/2023] [Indexed: 01/26/2023] Open
Abstract
BACKGROUND Multivariate longitudinal data are under-utilized for survival analysis compared to cross-sectional data (CS - data collected once across cohort). Particularly in cardiovascular risk prediction, despite available methods of longitudinal data analysis, the value of longitudinal information has not been established in terms of improved predictive accuracy and clinical applicability. METHODS We investigated the value of longitudinal data over and above the use of cross-sectional data via 6 distinct modeling strategies from statistics, machine learning, and deep learning that incorporate repeated measures for survival analysis of the time-to-cardiovascular event in the Coronary Artery Risk Development in Young Adults (CARDIA) cohort. We then examined and compared the use of model-specific interpretability methods (Random Survival Forest Variable Importance) and model-agnostic methods (SHapley Additive exPlanation (SHAP) and Temporal Importance Model Explanation (TIME)) in cardiovascular risk prediction using the top-performing models. RESULTS In a cohort of 3539 participants, longitudinal information from 35 variables that were repeatedly collected in 6 exam visits over 15 years improved subsequent long-term (17 years after) risk prediction by up to 8.3% in C-index compared to using baseline data (0.78 vs. 0.72), and up to approximately 4% compared to using the last observed CS data (0.75). Time-varying AUC was also higher in models using longitudinal data (0.86-0.87 at 5 years, 0.79-0.81 at 10 years) than using baseline or last observed CS data (0.80-0.86 at 5 years, 0.73-0.77 at 10 years). Comparative model interpretability analysis revealed the impact of longitudinal variables on model prediction on both the individual and global scales among different modeling strategies, as well as identifying the best time windows and best timing within that window for event prediction. The best strategy to incorporate longitudinal data for accuracy was time series massive feature extraction, and the easiest interpretable strategy was trajectory clustering. CONCLUSION Our analysis demonstrates the added value of longitudinal data in predictive accuracy and epidemiological utility in cardiovascular risk survival analysis in young adults via a unified, scalable framework that compares model performance and explainability. The framework can be extended to a larger number of variables and other longitudinal modeling methods. TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT00005130, Registration Date: 26/05/2000.
Collapse
Affiliation(s)
- Hieu T. Nguyen
- grid.21107.350000 0001 2171 9311Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD USA
| | - Henrique D. Vasconcellos
- grid.21107.350000 0001 2171 9311Department of Cardiology, Johns Hopkins University, Baltimore, MD USA
| | - Kimberley Keck
- grid.21107.350000 0001 2171 9311Department of Cardiology, Johns Hopkins University, Baltimore, MD USA
| | - Jared P. Reis
- grid.279885.90000 0001 2293 4638National Heart, Lung, and Blood Institute, Bethesda, MD USA
| | - Cora E. Lewis
- grid.265892.20000000106344187Department of Epidemiology, School of Public Health, University of Alabama at Birmingham, Birmingham, AL USA
| | - Steven Sidney
- grid.280062.e0000 0000 9957 7758Division of Research, Kaiser Permanente, Oakland, CA USA
| | - Donald M. Lloyd-Jones
- grid.16753.360000 0001 2299 3507Department of Preventive Medicine, Northwestern University, Chicago, IL USA
| | - Pamela J. Schreiner
- grid.17635.360000000419368657School of Public Health, University of Minnesota, Minneapolis, MN USA
| | - Eliseo Guallar
- grid.21107.350000 0001 2171 9311Department of Epidemiology, Johns Hopkins University School of Public Health, Baltimore, MD USA
| | - Colin O. Wu
- grid.279885.90000 0001 2293 4638National Heart, Lung, and Blood Institute, Bethesda, MD USA
| | - João A.C. Lima
- grid.21107.350000 0001 2171 9311Department of Cardiology, Johns Hopkins University, Baltimore, MD USA ,grid.21107.350000 0001 2171 9311Department of Radiology, Johns Hopkins University, Baltimore, MD USA
| | - Bharath Ambale-Venkatesh
- grid.21107.350000 0001 2171 9311Department of Radiology, Johns Hopkins University, Baltimore, MD USA
| |
Collapse
|