Machine learning-based in-hospital mortality risk prediction tool for intensive care unit patients with heart failure

Development, Validation, and Deployment of a Time-Dependent Machine Learning Model for Predicting One-Year Mortality Risk in Critically Ill Patients with Heart Failure

Background: Heart failure (HF) ranks among the foremost causes of mortality globally, exhibiting particularly high prevalence and significant impact within intensive care units (ICUs). This study sought to develop, validate, and deploy a time-dependent machine learning model aimed at predicting the one-year all-cause mortality risk in ICU patients diagnosed with HF, thereby facilitating precise prognostic evaluation and risk stratification. Methods: This study encompassed a cohort of 8960 ICU patients with HF sourced from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database (version 3.1). This latest version of the database added data from 2020 to 2022 on the basis of version 2.2 (covering data from 2008 to 2019); therefore, data spanning 2008 to 2019 (n = 5748) were designated for the training set, while data from 2020 to 2022 (n = 3212) were reserved for the test set. The primary endpoint of interest was one-year all-cause mortality. Least Absolute Shrinkage and Selection Operator (LASSO) regression was employed to select predictive features from an initial pool of 64 candidate variables (including demographic characteristics, vital signs, comorbidities and complications, therapeutic interventions, routine laboratory data, and disease severity scores). Four predictive models were developed and compared: Cox proportional hazards, random survival forest (RSF), Cox proportional hazards deep neural network (DeepSurv), and eXtreme Gradient Boosting (XGBoost). Model performance was assessed using the concordance index (C-index) and Brier score, with model interpretability addressed through SHapley Additive exPlanations (SHAP) and time-dependent Survival SHapley Additive exPlanations (SurvSHAP(t)). Results: This study revealed a one-year mortality rate of 46.1% within the population under investigation. In the training set, LASSO effectively identified 24 features in the model. In the test set, the XGBoost model exhibited superior predictive performance, as evidenced by a C-index of 0.772 and a Brier score of 0.161, outperforming the Cox model (C-index: 0.740, Brier score: 0.175), the RSF model (C-index: 0.747, Brier score: 0.178), and the DeepSur model (C-index: 0.723, Brier score: 0.183). Decision curve analysis validated the clinical utility of the XGBoost model across a broad spectrum of risk thresholds. Feature importance analysis identified the red cell distribution width-to-albumin ratio (RAR), Charlson Comorbidity Index, Simplified Acute Physiology Score II (SAPS II), Acute Physiology Score III (APS III), and the age-bilirubin-INR-creatinine (ABIC) score as the top five predictive factors. Consequently, an online risk prediction tool based on this model has been developed and is publicly accessible. Conclusions: The time-dependent XGBoost model demonstrated robust predictive capability in evaluating the one-year all-cause mortality risk in critically ill HF patients. This model offered a useful tool for early risk identification and supported timely interventions.

Independent prognostic importance of endothelial activation and stress index (EASIX) in critically ill patients with heart failure: modulating role of inflammation

Background

The connection between endothelial activation and stress index (EASIX) and risk of mortality in critically ill patients with heart failure (HF) remains unclear. This research sought to explore this relationship.

Methods

MIMIC-IV database (version 3.1) was utilized to provide clinical data. Due to the non-normal distribution, EASIX was logarithmic. An optimal cut-off value for log2(EASIX) was determined to serve as an indicator of mortality risk under the maximally selected rank statistics. Kaplan-Meier survival analysis and Cox regression models were used to assess the link between log2(EASIX) and mortality within 1 year. Subgroup analyses were performed to ascertain the prognostic impact of log2(EASIX) in various patient groups. Mediation analysis was employed to uncover and elucidate causal pathways connecting log2(EASIX) to mortality.

Results

It encompassed 7,901 patients. According to the Kaplan-Meier curves, increased log2(EASIX) levels correlated with a higher likelihood of all-cause mortality (p < 0.001). Cox models and subgroup analyses further revealed that groups with high log2(EASIX) levels exhibited a greater mortality risk than those with lower levels (hazard ratio (HR): 1.62, 95% CI: 1.47-1.78), a trend that persisted across most subgroups, with the exception of varying levels of APS III, body mass index, white blood cell counts, or albumin (p for interaction < 0.05 for all). Subsequent mediation analysis suggested that blood urea nitrogen and red cell distribution width partially mediated the relationship between log2(EASIX) and mortality with 17.3% and 36.5% of the mediating effect.

Conclusion

It found an independent association between elevated log2(EASIX) levels and a higher risk of 1 year all-cause mortality in ICU patients suffering from HF, with a stronger effect observed in patients with low levels of APS III or white blood cell counts, or high levels of body mass index or albumin. This association may be partially mediated by blood urea nitrogen and red cell distribution width.

Evaluation of machine learning methods for prediction of heart failure mortality and readmission: meta-analysis

BACKGROUND

Heart failure (HF) impacts nearly 6 million individuals in the U.S., with a projected 46% increase by 2030, is creating significant healthcare burdens. Predictive models, particularly machine learning (ML)-based models, offer promising solutions to identify patients at greater risk of adverse outcomes, such as mortality and hospital readmission. This review aims to assess the effectiveness of ML models in predicting HF-related outcomes, with a focus on their potential to improve patient care and clinical decision-making. We aim to assess how effectively machine learning models predict mortality and readmission in heart failure patients to improve clinical outcomes.

METHOD

The study followed PRISMA 2020 guidelines and was registered in the PROSPERO database (CRD42023481167). We conducted a systematic search in PubMed, Scopus, and Web of Science databases using specific keywords related to heart failure, machine learning, mortality and readmission. Extracted data focused on study characteristics, machine learning details, and outcomes, with AUC or c-index used as the primary outcomes for pooling analysis. The PROBAST tool was used to assess bias risk, evaluating models based on participants, predictors, outcomes, and statistical analysis. The meta-analysis pooled AUCs for different machine learning models predicting mortality and readmission. Prediction accuracy data was categorized by timeframes, with high heterogeneity determined by an I² value above 50%, leading to a random-effects model when applicable. Publication bias was assessed using Egger's and Begg's tests, with a p-value below 0.05 considered significant RESULT: A total of 4,505 studies were identified, and after screening, 64 were included in the final analysis, covering 943,941 patients. Of these, 40 studies focused on mortality, 17 on readmission, and 7 on both outcomes. In total, 346 machine learning models were evaluated, with the most common algorithms being random forest, logistic regression, and gradient boosting. The neural network model achieved the highest overall AUC for mortality prediction (0.808), while the support vector machine performed best for readmission prediction (AUC 0.733). The analysis revealed a significant risk of bias, primarily due to reliance on retrospective data and inadequate sample size justification.

CONCLUSION

In conclusion, this review emphasizes the strong potential of ML models in predicting HF readmission and mortality. ML algorithms show promise in improving prognostic accuracy and enabling personalized patient care. However, challenges like model interpretability, generalizability, and clinical integration persist. Overcoming these requires refined ML techniques and a robust regulatory framework to enhance HF outcomes.

Association between red cell distribution width-albumin ratio and all-cause mortality in intensive care unit patients with heart failure

Aim

The association between red cell distribution width-albumin ratio (RAR) and the risk of all-cause mortality in intensive care unit (ICU) patients with heart failure remains uncertain. This study aimed to investigate this association.

Methods

Clinical data from MIMIC-Ⅳ (version 2.2) database was utilized for the analysis of ICU patients with heart failure. Patients were categorized into quartiles (Q1-Q4) based on RAR levels. Kaplan-Meier survival analysis and multivariate adjusted Cox regression models were employed to assess the association between RAR levels and mortality outcomes within 1 year. Subgroup analysis was used to evaluate the prognostic impact of RAR across diverse populations. Restricted cubic spline curves and threshold effect analysis were utilized to quantify the dose-response relationship between RAR levels and mortality. The time-concordance index curve was carried out to explore the additional prognostic value of RAR on mortality over the existing scoring systems, Serial Organ Failure Assessment (SOFA) and Acute Physiology and Chronic Health Evaluation Ⅱ (APACHE Ⅱ).

Results

The analysis encompassed a cohort of 4,506 patients, with Kaplan-Meier curves indicating that individuals with higher RAR levels exhibited an elevated risk of all-cause mortality (p < 0.001). Multivariate adjusted Cox regression and subgroup analysis demonstrated that individuals in Q2 [hazard ratio (HR) 1.15, 95%CI 0.98-1.34], Q3 (HR 1.65, 95%CI 1.39-1.96) and Q4 (HR 2.16, 95%CI 1.74-2.68) had an increased risk of mortality compared to individuals in Q1 (p for trend < 0.001), and this relationship was consistently observed across most subgroups, except for different ages. Subsequent analysis revealed that the inclusion of RAR significantly improved the prognostic value on the basis of SOFA and APACHE Ⅱ, and the concordance index increased from 0.636 to 0.658 for SOFA, from 0.682 to 0.695 for APACHE Ⅱ (p < 0.001 for both).

Conclusion

The study found that high level of RAR was independently associated with an increased risk of 1-year all-cause mortality in ICU patients with heart failure, with a stronger effect in young and middle-aged patients and a threshold effect, which could potentially serve as an early warning indicator for high-risk populations.

Prioritizing Patient Selection in Clinical Trials: A Machine Learning Algorithm for Dynamic Prediction of In-Hospital Mortality for ICU Admitted Patients Using Repeated Measurement Data

Background: A machine learning prognostic mortality scoring system was developed to address challenges in patient selection for clinical trials within the Intensive Care Unit (ICU) environment. The algorithm incorporates Red blood cell Distribution Width (RDW) data and other demographic characteristics to predict ICU mortality alongside existing ICU mortality scoring systems like Simplified Acute Physiology Score (SAPS). Methods: The developed algorithm, defined as a Mixed-effects logistic Random Forest for binary data (MixRFb), integrates a Random Forest (RF) classification with a mixed-effects model for binary outcomes, accounting for repeated measurement data. Performance comparisons were conducted with RF and the proposed MixRFb algorithms based solely on SAPS scoring, with additional evaluation using a descriptive receiver operating characteristic curve incorporating RDW's predictive mortality ability. Results: MixRFb, incorporating RDW and other covariates, outperforms the SAPS-based variant, achieving an area under the curve of 0.882 compared to 0.814. Age and RDW were identified as the most significant predictors of ICU mortality, as reported by the variable importance plot analysis. Conclusions: The MixRFb algorithm demonstrates superior efficacy in predicting in-hospital mortality and identifies age and RDW as primary predictors. Implementation of this algorithm could facilitate patient selection for clinical trials, thereby improving trial outcomes and strengthening ethical standards. Future research should focus on enriching algorithm robustness, expanding its applicability across diverse clinical settings and patient demographics, and integrating additional predictive markers to improve patient selection capabilities.

Predictive Value of Machine Learning for the Risk of In-Hospital Death in Patients With Heart Failure: A Systematic Review and Meta-Analysis

BACKGROUND

The efficiency of machine learning (ML) based predictive models in predicting in-hospital mortality for heart failure (HF) patients is a topic of debate. In this context, this study's objective is to conduct a meta-analysis to compare and assess existing prognostic models designed for predicting in-hospital mortality in HF patients.

METHODS

A systematic search of databases was conducted, including PubMed, Embase, Web of Science, and Cochrane Library up to January 2023. To ensure comprehensiveness, we performed an additional search in June 2023. The Prediction Model Risk of Bias Assessment Tool was employed to assess the validity and reliability of ML models.

RESULTS

Our analysis incorporated 28 studies involving a total of 106 predictive models based on 14 different ML techniques. In the training data set, these models showed a combined C-index of 0.781, sensitivity of 0.56, and specificity of 0.94. In the validation data set, the models exhibited a combined C-index of 0.758, sensitivity of 0.57, and specificity of 0.84. Logistic regression (LR) was the most frequently used ML algorithm. LR models in the training set had a combined C-index of 0.795, sensitivity of 0.63, and specificity of 0.85, and these measures for LR models in the validation set were 0.751, 0.66, and 0.79, respectively.

CONCLUSIONS

Our study indicates that although ML is increasingly being leveraged to predict in-hospital mortality for HF patients, the predictive performance remains suboptimal. Although these models have relatively high C-index and specificity, their ability to predict positive events is limited, as indicated by their low sensitivity.

Predicting in-hospital mortality in patients with heart failure combined with atrial fibrillation using stacking ensemble model: an analysis of the medical information mart for intensive care IV (MIMIC-IV)

BACKGROUND

Heart failure (HF) and atrial fibrillation (AF) usually coexist and are associated with a poorer prognosis. This study aimed to develop a model to predict in-hospital mortality in patients with HF combined with AF.

METHODS

Patients with HF and AF were obtained from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database from 2008 to 2019. Feature selection was based on the Mann-Whitney U test and the least absolute shrinkage and selection operator (LASSO) regression model. Random Forest, eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LGBM), K-Nearest Neighbor (KNN) models, and their stacked model (the stacking ensemble model) were established. The area under of the curve (AUC) with 95% confidence interval (CI), sensitivity, specificity, as well as accuracy were applied to assess the performance of the predictive models.

RESULTS

A total of 5,998 patients with HF combined with AF were included, of which 4,198 patients were assigned to the training set and 1,800 to the testing set (7:3). Among these 4,198 patients, 624 (14.86%) died in-hospital and 3,574 (85.14%) survived. Twenty-two features were used to construct the predictive model. Among these four single models, the AUC was 0.747 (95%CI: 0.717-0.777) for the Random Forest model, 0.755 (95%CI: 0.725-0.785) for the XGBoost model, 0.754 (95%CI: 0.724-0.784) for the LGBM model, and 0.746 (95%CI: 0.716-0.776) for the KNN model in the testing set. The stacking ensemble model had the highest AUC compared to the four single models, with AUCs of 0.837 (95%CI: 0.821-0.852) and 0.768 (95%CI: 0.740-0.796) for the training set and testing set, respectively.

CONCLUSION

The stacking ensemble model showed a good predictive effect in predicting in-hospital mortality in patients with HF combined with AF and may provide clinicians with a reference tool for early identification of mortality risk.

Independent prognostic importance of the albumin-corrected anion gap in critically ill patients with congestive heart failure: a retrospective study from MIMIC-IV database

BACKGROUND

The associations between the albumin-corrected anion gap (ACAG) and all-cause mortality in patients with congestive heart failure in the intensive care unit remain uncertain. This study aimed to investigate this unknown.

METHODS

The MIMIC-IV (version 3.0) database was used to analyze critically ill patients with congestive heart failure. Patients were grouped into tertiles (T1-T3) on the basis of the ACAG. The association between ACAG levels and 1-year all-cause mortality was assessed using Kaplan-Meier survival analyses, multivariate adjusted Cox regression models, and restricted cubic spline curves. An analysis of subgroups was performed to evaluate ACAG's prognostic impact across diverse populations. Mediation analysis was conducted to identify and elucidate potential causal pathways linking ACAG to all-cause mortality.

RESULTS

A cohort of 7787 patients was analyzed. On the basis of Kaplan-Meier curves, Cox regression, restricted cubic spline curves and subgroup analysis, T2 (hazard ratio 1.09, 95% confidence interval 1.02 ~ 1.16) and T3 (hazard ratio 1.25, 95% confidence interval 1.17 ~ 1.33) individuals presented a greater mortality risk compared to T1 individuals (p for linear trend < 0.001), and most subgroups consistently observed this relationship, except for those with different levels of left ventricular ejection fraction. Mediation analysis indicated that the red cell distribution width, stage of acute kidney injury, chloride and acute physiology score III partially mediated the relationship between ACAG and mortality, accounting for 12.4%, 7.0%, 12.9%, and 31.2% of the mediating effect, respectively.

CONCLUSIONS

The ACAG was associated with higher 1-year all-cause mortality in critically ill patients with congestive heart failure, with stronger impact in those with lower left ventricular ejection fractions. The ACAG may serve as an indicator in high-risk groups.

CLINICAL TRIAL NUMBER

Not applicable.

A machine learning-based predictive model for the in-hospital mortality of critically ill patients with atrial fibrillation

BACKGROUND

Atrial fibrillation (AF) is common among intensive care unit (ICU) patients and significantly raises the in-hospital mortality rate. Existing scoring systems or models have limited predictive capabilities for AF patients in ICU. Our study developed and validated machine learning models to predict the risk of in-hospital mortality in ICU patients with AF.

METHODS AND RESULTS

Medical Information Mart for Intensive Care (MIMIC)-IV dataset and eICU Collaborative Research Database (eICU-CRD) were analyzed. Among ten classifiers compared, adaptive boosting (AdaBoost) showed better performance in predicting all-cause mortality in AF patients. A compact model with 15 features was developed and validated. Both the all variable and compact models exhibited excellent performance with area under the receiver operating characteristic curves (AUCs) of 1(95%confidence interval [CI]: 1.0-1.0) in the training set. In the MIMIC-IV testing set, the AUCs of the all variable and compact models were 0.978 (95% CI: 0.973-0.982) and 0.977 (95% CI: 0.972-0.982), respectively. In the external validation set, the AUCs of all variable and compact models were 0.825 (95% CI: 0.815-0.834) and 0.807 (95% CI: 0.796-0.817), respectively.

CONCLUSION

An AdaBoost-based predictive model was subjected to internal and external validation, highlighting its strong predictive capacity for assessing the risk of in-hospital mortality in ICU patients with AF.

Predictive Analytics in Heart Failure Risk, Readmission, and Mortality Prediction: A Review

Heart failure is a leading cause of death among people worldwide. The cost of treatment can be prohibitive, and early prediction of heart failure would reduce treatment costs to patients and hospitals. Improved readmission prediction would also greatly help hospitals, allowing them to manage their treatment programs and budgets better. This literature review aims to summarize recent studies of predictive analytics models that have been constructed to predict heart failure risk, readmission, and mortality. Random forest, logistic regression, neural networks, and XGBoost were among the most common modeling techniques applied. Most selected studies leveraged structured electronic health record data, including demographics, clinical values, lifestyle, and comorbidities, with some incorporating unstructured clinical notes. Preprocessing through imputation and feature selection were frequently employed in building the predictive analytics models. The reviewed studies exhibit demonstrated promise for predictive analytics in improving early heart failure diagnosis, readmission risk stratification, and mortality prediction. This review study highlights rising research activities and the potential of predictive analytics, especially the implementation of machine learning, in advancing heart failure outcomes. Further rigorous, comprehensive syntheses and head-to-head benchmarking of predictive models are needed to derive robust evidence for clinical adoption.

Prediction of 28-Day All-Cause Mortality in Heart Failure Patients with Clostridioides difficile Infection Using Machine Learning Models: Evidence from the MIMIC-IV Database

INTRODUCTION

Heart failure (HF) may induce bowel hypoperfusion, leading to hypoxia of the villa of the bowel wall and the occurrence of Clostridioides difficile infection (CDI). However, the risk factors for the development of CDI in HF patients have yet to be fully illustrated, especially because of a lack of evidence from real-world data.

METHODS

Clinical data and survival situations of HF patients with CDI admitted to ICU were extracted from the Medical Information Mart for Intensive Care (MIMIC)-IV database. For developing a model that can predict 28-day all-cause mortality in HF patients with CDI, the Recursive Feature Elimination with Cross-Validation (RFE-CV) method was used for feature selection. And nine machine learning (ML) algorithms, including logistic regression (LR), decision tree, Bayesian, adaptive boosting, random forest (RF), gradient boosting decision tree, XGBoost, light gradient boosting machine, and categorical boosting, were applied for model construction. After training and hyperparameter optimization of the models through grid search 5-fold cross-validation, the performance of models was evaluated by the area under curve (AUC), accuracy, sensitivity, specificity, precision, negative predictive value, and F1 score. Furthermore, the SHapley Additive exPlanations (SHAP) method was used to interpret the optimal model.

RESULTS

A total of 526 HF patients with CDI were included in the study, of whom 99 cases (18.8%) experienced death within 28 days. Eighteen of the 57 variables were selected for the model construction algorithm for model construction. Among the ML models considered, the RF model emerged as the optimal model achieving the accuracy, F1-score, and AUC values of 0.821, 0.596, and 0.864, respectively. The net benefit of the model surpassed other models at 16%-22% threshold probabilities based on decision curve analysis. According to the importance of features in the RF model, red blood cell distribution width, blood urea nitrogen, Simplified Acute Physiology Score II, Sequential Organ Failure Assessment, and white blood cell count were highlighted as the five most influential variables.

CONCLUSIONS

We developed ML models to predict 28-day all-cause mortality in HF patients associated with CDI in the ICU, which are more effective than the conventional LR model. The RF model has the best performance among all the ML models employed. It may be useful to help clinicians identify high-risk HF patients with CDI.

[Prediction of risk of in-hospital death in patients with chronic heart failure complicated by lung infections using interpretable machine learning]

OBJECTIVE

To predict the risk of in-hospital death in patients with chronic heart failure (CHF) complicated by lung infections using interpretable machine learning.

METHODS

The clinical data of 1415 patients diagnosed with CHF complicated by lung infections were obtained from the MIMIC-IV database. According to the pathogen type, the patients were categorized into bacterial pneumonia and non-bacterial pneumonia groups, and their risks of in-hospital death were compared using Kaplan-Meier survival curves. Univariate analysis and LASSO regression were used to select the features for constructing LR, AdaBoost, XGBoost, and LightGBM models, and their performance was compared in terms of accuracy, precision, F1 value, and AUC. External validation of the models was performed using the data from eICU-CRD database. SHAP algorithm was applied for interpretive analysis of XGBoost model.

RESULTS

Among the 4 constructed models, the XGBoost model showed the highest accuracy and F1 value for predicting the risk of in-hospital death in CHF patients with lung infections in the training set. In the external test set, the XGBoost model had an AUC of 0.691 (95% CI: 0.654-0.720) in bacterial pneumonia group and an AUC of 0.725 (95% CI: 0.577-0.782) in non-bacterial pneumonia group, and showed better predictive ability and stability than the other models.

CONCLUSION

The overall performance of the XGBoost model is superior to the other 3 models for predicting the risk of in-hospital death in CHF patients with lung infections. The SHAP algorithm provides a clear interpretation of the model to facilitate decision-making in clinical settings.

Machine learning in heart failure diagnosis, prediction, and prognosis: review

Globally, cardiovascular diseases take the lives of over 17 million people each year, mostly through myocardial infarction, or MI, and heart failure (HF). This comprehensive literature review examines various aspects related to the diagnosis, prediction, and prognosis of HF in the context of machine learning (ML). The review covers an array of topics, including the diagnosis of HF with preserved ejection fraction (HFpEF) and the identification of high-risk patients with HF with reduced ejection fraction (HFrEF). The prediction of mortality in different HF populations using different ML approaches is explored, encompassing patients in the ICU, and HFpEF patients using biomarkers and gene expression. The review also delves into the prediction of mortality and hospitalization rates in HF patients with mid-range ejection fraction (HFmrEF) using ML methods. The findings highlight the significance of a multidimensional approach that encompasses clinical evaluation, laboratory assessments, and comprehensive research to improve our understanding and management of HF. Promising predictive models incorporating biomarkers, gene expression, and consideration of epigenetics demonstrate potential in estimating mortality and identifying high-risk HFpEF patients. This literature review serves as a valuable resource for researchers, clinicians, and healthcare professionals seeking a comprehensive and updated understanding of the role of ML diagnosis, prediction, and prognosis of HF across different subtypes and patient populations.

Patient Selection and End Point Definitions for Decongestion Studies in Acute Decompensated Heart Failure: Part 1

Despite recent advances in the treatment of patients with chronic heart failure, acute decompensated heart failure remains associated with significant mortality and morbidity because many novel therapies have failed to demonstrate meaningful benefit. Persistent congestion in the setting of escalating diuretic therapy has been repeatedly shown to be a marker of poor prognosis and is currently being targeted by various emerging device-based therapies. Because these therapies inherently carry procedural risk, patient selection is key in the future trial design. However, it remains unclear which patients are at a higher risk of residual congestion or adverse outcomes despite maximally tolerated decongestive therapy. In the first part of this 2-part review, we aimed to outline patient risk factors and summarize current evidence for early recognition of high-risk profile for residual congestion and adverse outcomes. These factors are classified as relating to the following: (1) previous clinical course, (2) severity of congestion, (3) diuretic response, and (4) degree of renal impairment. We also aimed to provide an overview of key inclusion criteria in recent acute decompensated heart failure trials and investigational device studies and propose potential criteria for selection of high-risk patients in future trials.