Comparison of Machine Learning Algorithms for Predicting Hospital Readmissions and Worsening Heart Failure Events in Patients With Heart Failure With Reduced Ejection Fraction: Modeling Study

Evaluation of machine learning methods for prediction of heart failure mortality and readmission: meta-analysis

BACKGROUND

Heart failure (HF) impacts nearly 6 million individuals in the U.S., with a projected 46% increase by 2030, is creating significant healthcare burdens. Predictive models, particularly machine learning (ML)-based models, offer promising solutions to identify patients at greater risk of adverse outcomes, such as mortality and hospital readmission. This review aims to assess the effectiveness of ML models in predicting HF-related outcomes, with a focus on their potential to improve patient care and clinical decision-making. We aim to assess how effectively machine learning models predict mortality and readmission in heart failure patients to improve clinical outcomes.

METHOD

The study followed PRISMA 2020 guidelines and was registered in the PROSPERO database (CRD42023481167). We conducted a systematic search in PubMed, Scopus, and Web of Science databases using specific keywords related to heart failure, machine learning, mortality and readmission. Extracted data focused on study characteristics, machine learning details, and outcomes, with AUC or c-index used as the primary outcomes for pooling analysis. The PROBAST tool was used to assess bias risk, evaluating models based on participants, predictors, outcomes, and statistical analysis. The meta-analysis pooled AUCs for different machine learning models predicting mortality and readmission. Prediction accuracy data was categorized by timeframes, with high heterogeneity determined by an I² value above 50%, leading to a random-effects model when applicable. Publication bias was assessed using Egger's and Begg's tests, with a p-value below 0.05 considered significant RESULT: A total of 4,505 studies were identified, and after screening, 64 were included in the final analysis, covering 943,941 patients. Of these, 40 studies focused on mortality, 17 on readmission, and 7 on both outcomes. In total, 346 machine learning models were evaluated, with the most common algorithms being random forest, logistic regression, and gradient boosting. The neural network model achieved the highest overall AUC for mortality prediction (0.808), while the support vector machine performed best for readmission prediction (AUC 0.733). The analysis revealed a significant risk of bias, primarily due to reliance on retrospective data and inadequate sample size justification.

CONCLUSION

In conclusion, this review emphasizes the strong potential of ML models in predicting HF readmission and mortality. ML algorithms show promise in improving prognostic accuracy and enabling personalized patient care. However, challenges like model interpretability, generalizability, and clinical integration persist. Overcoming these requires refined ML techniques and a robust regulatory framework to enhance HF outcomes.

Development and validation of a machine learning model to predict the risk of readmission within one year in HFpEF patients: Short title: Prediction of HFpEF readmission

BACKGROUND

Heart failure with preserved ejection fraction (HFpEF) is associated with elevated rates of readmission and mortality. Accurate prediction of readmission risk is essential for optimizing healthcare resources and enhancing patient outcomes.

METHODS

We conducted a retrospective cohort study utilizing HFpEF patient data from two institutions: the First Affiliated Hospital Zhejiang University School of Medicine for model development and internal validation, and the Affiliated Hospital of Xuzhou Medical University for external validation. A machine learning (ML) model was developed and validated using 53 variables to predict the risk of readmission within one year. The model's performance was assessed using several metrics, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, F1 score, model training time, model prediction time and brier score. SHAP (SHapley Additive exPlanations) analysis was employed to enhance model interpretability, and a dynamic nomogram was constructed to visualize the predictive model.

RESULTS

Among the 766 HFpEF patients included in the study, 203 (26.5%) were readmitted within one year. The LightGBM model exhibited the highest predictive performance, with an AUC of 0.88 (95% confidence interval (CI):0.84-0.91), an accuracy of 0.79, a sensitivity of 0.81, and a specificity of 0.78. Key predictors included the E/e' ratio, NYHA classification, LVEF, age, BNP levels, MLR, history of atrial fibrillation (AF), use of ACEI/ARB/ARNI, and history of myocardial infarction (MI). External validation also demonstrated strong predictive performance, with an AUC of 0.87 (95%CI:0.83-0.91).

CONCLUSIONS

The LightGBM model exhibited robust performance in predicting one-year readmission risk among HFpEF patients, providing a valuable tool for clinicians to identify high-risk individuals and implement timely interventions.

Machine learning-based model for worsening heart failure risk in Chinese chronic heart failure patients

AIMS

This study aims to develop and validate an optimal model for predicting worsening heart failure (WHF). Multiple machine learning (ML) algorithms were compared, and the results were interpreted using SHapley Additive exPlanations (SHAP). A clinical risk calculation tool was subsequently developed based on these findings.

METHODS AND RESULTS

This nested case-control study included 200 patients with chronic heart failure (CHF) from the China-Japan Friendship Hospital (September 2019 to December 2022). Sixty-five variables were collected, including basic information, physical and chemical examinations, and quality of life assessments. WHF occurrence within a 3-month follow-up was the outcome event. Variables were screened using LASSO regression, univariate analysis, and comparison of key variables in multiple ML models. Eighty per cent of the data was used for training and 20% for testing. The best models were identified by integrating nine ML algorithms and interpreted using SHAP, and to develop a final risk calculation tool. Among participants, 68 (34.0%) were female, with a mean age (standard deviation, SD) of 68.57 (12.80) years. During the follow-up, 60 participants (30%) developed WHF. N-terminal pro-brain natriuretic peptide (NT-proBNP), creatinine (Cr), uric acid (UA), haemoglobin (Hb), and emotional area score on the Minnesota Heart Failure Quality of Life Questionnaire were critical predictors of WHF occurrence. The random forest (RF) model was the best model to predict WHF with an area under the curve (AUC) (95% confidence interval, CI) of 0.842 (0.675-1.000), accuracy of 0.775, sensitivity of 0.900, specificity of 0.833, negative predictive value of 0.800, and positive predictive value of 0.600 for the test set. SHAP analysis highlighted NT-proBNP, UA, and Cr as significant predictors. An online risk predictor based on the RF model was developed for personalized WHF risk assessment.

CONCLUSIONS

This study identifies NT-proBNP, Cr, UA, Hb, and emotional area scores as crucial predictors of WHF in CHF patients. Among the nine ML algorithms assessed, the RF model showed the highest predictive accuracy. SHAP analysis further emphasized NT-proBNP, UA, and Cr as the most significant predictors. An online risk prediction tool based on the RF model was subsequently developed to enhance early and personalized WHF risk assessment in clinical settings.

An explainable model for predicting Worsening Heart Failure based on genetic programming

Heart Failure (HF) poses a challenge for our health systems, and early detection of Worsening HF (WHF), defined as a deterioration in symptoms and clinical and instrumental signs of HF, is vital to improving prognosis. Predicting WHF in a phase that is currently undiagnosable by physicians would enable prompt treatment of such events in patients at a higher risk of WHF. Although the role of Artificial Intelligence in cardiovascular diseases is becoming part of clinical practice, especially for diagnostic and prognostic purposes, its usage is often considered not completely reliable due to the incapacity of these models to provide a valid explanation about their output results. Physicians are often reluctant to make decisions based on unjustified results and see these models as black boxes. This study aims to develop a novel diagnostic model capable of predicting WHF while also providing an easy interpretation of the outcomes. We propose a threshold-based binary classifier built on a mathematical model derived from the Genetic Programming approach. This model clearly indicates that WHF is closely linked to creatinine, sPAP, and CAD, even though the relationship of these variables and WHF is almost complex. However, the proposed mathematical model allows for providing a 3D graphical representation, which medical staff can use to better understand the clinical situation of patients. Experiments conducted using retrospectively collected data from 519 patients treated at the HF Clinic of the University Hospital of Salerno have demonstrated the effectiveness of our model, surpassing the most commonly used machine learning algorithms. Indeed, the proposed GP-based classifier achieved a 96% average score for all considered evaluation metrics and fully supported the controls of medical staff. Our solution has the potential to impact clinical practice for HF by identifying patients at high risk of WHF and facilitating more rapid diagnosis, targeted treatment, and a reduction in hospitalizations.

Machine learning-based 30-day readmission prediction models for patients with heart failure: a systematic review

AIMS

Heart failure (HF) is one of the most frequent diagnoses for 30-day readmission after hospital discharge. Nurses have a role in reducing unplanned readmission and providing quality of care during HF trajectories. This systematic review assessed the quality and significant factors of machine learning (ML)-based 30-day HF readmission prediction models.

METHODS AND RESULTS

Eight academic and electronic databases were searched to identify all relevant articles published between 2013 and 2023. Thirteen studies met our inclusion criteria. The sample sizes of the selected studies ranged from 1778 to 272 778 patients, and the patients' average age ranged from 70 to 81 years. Quality appraisal was performed.

CONCLUSION

The most commonly used ML approaches were random forest and extreme gradient boosting. The 30-day HF readmission rates ranged from 1.2 to 39.4%. The area under the receiver operating characteristic curve for models predicting 30-day HF readmission was between 0.51 and 0.93. Significant predictors included 60 variables with 9 categories (socio-demographics, vital signs, medical history, therapy, echocardiographic findings, prescribed medications, laboratory results, comorbidities, and hospital performance index). Future studies using ML algorithms should evaluate the predictive quality of the factors associated with 30-day HF readmission presented in this review, considering different healthcare systems and types of HF. More prospective cohort studies by combining structured and unstructured data are required to improve the quality of ML-based prediction model, which may help nurses and other healthcare professionals assess early and accurate 30-day HF readmission predictions and plan individualized care after hospital discharge.

REGISTRATION

PROSPERO: CRD 42023455584.

Comparing Machine Learning and Advanced Methods with Traditional Methods to Generate Weights in Inverse Probability of Treatment Weighting: The INFORM Study

Purpose

Observational research provides valuable insights into treatments used in patient populations in real-world settings. However, confounding is likely to occur if there are differences in patient characteristics associated with both the exposure and outcome between the groups being evaluated. One approach to reduce confounding and facilitate unbiased comparisons is inverse probability of treatment weighting (IPTW) using propensity scores. Machine learning (ML) and entropy balancing can potentially be used in generating propensity scores for IPTW, but there is limited literature on this application. We aimed to assess the feasibility of applying these methods for reducing confounding in observational studies. These methods were assessed in a study comparing cardiovascular outcomes in adults with type 2 diabetes and established atherosclerotic cardiovascular disease taking once-weekly glucagon-like peptide-1 receptor agonists or dipeptidyl peptidase-4 inhibitors.

Methods

We applied advanced methods to generate the propensity scores compared to the original logistic regression method in terms of covariate balance. After calculating weights, a weighted Cox proportional hazards model was used to calculate the sample average treatment effect. Support Vector Classification, Support Vector Regression, XGBoost, and LightGBM were the ML models used. Entropy balancing was also performed on features identified in the original cardiovascular outcomes study.

Results

Accuracy (range: 0.71 to 0.73), area under the curve (0.77 to 0.79), precision (0.53 to 0.60), recall (0.66 to 0.68), and F1 score (0.60 to 0.64) were similar between all of the advanced propensity score methods and traditional logistic regression. Among ML models, only XGBoost achieved balance in all measured baseline characteristics between the two treatment groups, closely approximating the performance of the original logistic regression. Entropy balancing weights provided the best performance among all models in balancing baseline characteristics, achieving near perfect balancing.

Conclusion

Among the advanced methods examined, entropy balancing weights performed the best for optimizing balancing and can produce similar results compared to traditional logistic regression.

Artificial intelligence based real-time prediction of imminent heart failure hospitalisation in patients undergoing non-invasive telemedicine

Background

Remote patient management may improve prognosis in heart failure. Daily review of transmitted data for early recognition of patients at risk requires substantial resources that represent a major barrier to wide implementation. An automated analysis of incoming data for detection of risk for imminent events would allow focusing on patients requiring prompt medical intervention.

Methods

We analysed data of the Telemedical Interventional Management in Heart Failure II (TIM-HF2) randomized trial that were collected during quarterly in-patient visits and daily transmissions from non-invasive monitoring devices. By application of machine learning, we developed and internally validated a risk score for heart failure hospitalisation within seven days following data transmission as estimate of short-term patient risk for adverse heart failure events. Score performance was assessed by the area under the receiver-operating characteristic (ROCAUC) and compared with a conventional algorithm, a heuristic rule set originally applied in the randomized trial.

Results

The machine learning model significantly outperformed the conventional algorithm (ROCAUC 0.855 vs. 0.727, p < 0.001). On average, the machine learning risk score increased continuously in the three weeks preceding heart failure hospitalisations, indicating potential for early detection of risk. In a simulated one-year scenario, daily review of only the one third of patients with the highest machine learning risk score would have led to detection of 95% of HF hospitalisations occurring within the following seven days.

Conclusions

A machine learning model allowed automated analysis of incoming remote monitoring data and reliable identification of patients at risk of heart failure hospitalisation requiring immediate medical intervention. This approach may significantly reduce the need for manual data review.

Investigating the impact of extreme weather events and related indicators on cardiometabolic multimorbidity

BACKGROUND

The impact of weather on human health has been proven, but the impact of extreme weather events on cardiometabolic multimorbidity (CMM) needs to be urgently explored.

OBJECTIVES

Investigating the impact of extreme temperature, relative humidity (RH), and laboratory testing parameters at admission on adverse events in CMM hospitalizations.

DESIGNS

Time-stratified case-crossover design.

METHODS

A distributional lag nonlinear model with a time-stratified case-crossover design was used to explore the nonlinear lagged association between environmental factors and CMM. Subsequently, unbalanced data were processed by 1:2 propensity score matching (PSM) and conditional logistic regression was employed to analyze the association between laboratory indicators and unplanned readmissions for CMM. Finally, the previously identified environmental factors and relevant laboratory indicators were incorporated into different machine learning models to predict the risk of unplanned readmission for CMM.

RESULTS

There are nonlinear associations and hysteresis effects between temperature, RH and hospital admissions for a variety of CMM. In addition, the risk of admission is higher under low temperature and high RH conditions with the addition of particulate matter (PM, PM2.5 and PM10) and O3_8h. The risk is greater for females and adults aged 65 and older. Compared with first quartile (Q1), the fourth quartile (Q4) had a higher association between serum calcium (HR = 1.3632, 95% CI: 1.0732 ~ 1.7334), serum creatinine (HR = 1.7987, 95% CI: 1.3528 ~ 2.3958), fasting plasma glucose (HR = 1.2579, 95% CI: 1.0839 ~ 1.4770), aspartate aminotransferase/ alanine aminotransferase ratio (HR = 2.3131, 95% CI: 1.9844 ~ 2.6418), alanine aminotransferase (HR = 1.7687, 95% CI: 1.2388 ~ 2.2986), and gamma-glutamyltransferase (HR = 1.4951, 95% CI: 1.2551 ~ 1.7351) were independently and positively associated with unplanned readmission for CMM. However, serum total bilirubin and High-Density Lipoprotein (HDL) showed negative correlations. After incorporating environmental factors and their lagged terms, eXtreme Gradient Boosting (XGBoost) demonstrated a more prominent predictive performance for unplanned readmission of CMM patients, with an average area under the receiver operating characteristic curve (AUC) of 0.767 (95% CI:0.7486 ~ 0.7854).

CONCLUSIONS

Extreme cold or wet weather is linked to worsened adverse health effects in female patients with CMM and in individuals aged 65 years and older. Moreover, meteorologic factors and environmental pollutants may elevate the likelihood of unplanned readmissions for CMM.

Medical-informed machine learning: integrating prior knowledge into medical decision systems

BACKGROUND

Clinical medicine offers a promising arena for applying Machine Learning (ML) models. However, despite numerous studies employing ML in medical data analysis, only a fraction have impacted clinical care. This article underscores the importance of utilising ML in medical data analysis, recognising that ML alone may not adequately capture the full complexity of clinical data, thereby advocating for the integration of medical domain knowledge in ML.

METHODS

The study conducts a comprehensive review of prior efforts in integrating medical knowledge into ML and maps these integration strategies onto the phases of the ML pipeline, encompassing data pre-processing, feature engineering, model training, and output evaluation. The study further explores the significance and impact of such integration through a case study on diabetes prediction. Here, clinical knowledge, encompassing rules, causal networks, intervals, and formulas, is integrated at each stage of the ML pipeline, resulting in a spectrum of integrated models.

RESULTS

The findings highlight the benefits of integration in terms of accuracy, interpretability, data efficiency, and adherence to clinical guidelines. In several cases, integrated models outperformed purely data-driven approaches, underscoring the potential for domain knowledge to enhance ML models through improved generalisation. In other cases, the integration was instrumental in enhancing model interpretability and ensuring conformity with established clinical guidelines. Notably, knowledge integration also proved effective in maintaining performance under limited data scenarios.

CONCLUSIONS

By illustrating various integration strategies through a clinical case study, this work provides guidance to inspire and facilitate future integration efforts. Furthermore, the study identifies the need to refine domain knowledge representation and fine-tune its contribution to the ML model as the two main challenges to integration and aims to stimulate further research in this direction.

The Efficacy of Machine Learning Models for Predicting the Prognosis of Heart Failure: A Systematic Review and Meta-Analysis

INTRODUCTION

Heart failure (HF) is a major global public health concern. The application of machine learning (ML) to identify individuals at high risk and enable early intervention is a promising approach for improving HF prognosis. We aim to systematically evaluate the performance and value of ML models for predicting HF prognosis.

METHODS

PubMed, Web of Science, Scopus, and Embase online databases were searched up to April 30, 2023, to identify studies on the use of ML models to predict HF prognosis. HF prognosis primarily encompasses readmission and mortality. The meta-analysis was conducted by MedCalc software. Subgroup analyses include grouping based on types of ML models, time intervals, sample sizes, the number of predictive variables, validation methods, whether to conduct hyperparameter optimization and calibration, data set partitioning methods.

RESULTS

A total of 31 studies were included. The most common ML models were random forest, boosting, support vector machine, neural network. The area under the receiver operating characteristic curve (AUC) for predicting HF readmission was 0.675 (95% CI: 0.651-0.699, p < 0.001), and the AUC for predicting HF mortality was 0.790 (95% CI: 0.765-0.816, p < 0.001). Subgroup analyses revealed that models with the prediction time interval of 1 year, sample sizes ≥10,000, the number of predictive variables ≥100, external validation, hyperparameter tuning, calibration adjustment, and data set partitioning using 10-fold cross-validation exhibited favorable performance within their respective subgroups.

CONCLUSION

The performance of ML models in predicting HF readmission is relatively poor, while its performance in predicting HF mortality is moderate. The quality of the relevant studies is generally low, it is essential to enhance the predictive capabilities of ML models through targeted improvements in practical applications.

A machine learning model to predict heart failure readmission: toward optimal feature set

Background

Hospital readmissions for heart failure patients remain high despite efforts to reduce them. Predictive modeling using big data provides opportunities to identify high-risk patients and inform care management. However, large datasets can constrain performance.

Objective

This study aimed to develop a machine learning based prediction model leveraging a nationwide hospitalization database to predict 30-day heart failure readmissions. Another objective of this study is to find the optimal feature set that leads to the highest AUC value in the prediction model.

Material and methods

Heart failure patient data was extracted from the 2020 Nationwide Readmissions Database. A heuristic feature selection process incrementally incorporated predictors into logistic regression and random forest models, which yields a maximum increase in the AUC metric. Discrimination was evaluated through accuracy, sensitivity, specificity and AUC.

Results

A total of 566,019 discharges with heart failure diagnosis were recognized. Readmission rate was 8.9% for same-cause and 20.6% for all-cause diagnoses. Random forest outperformed logistic regression, achieving AUCs of 0.607 and 0.576 for same-cause and all-cause readmissions respectively. Heuristic feature selection resulted in the identification of optimal feature sets including 20 and 22 variables from a pool of 30 and 31 features for the same-cause and all-cause datasets. Key predictors included age, payment method, chronic kidney disease, disposition status, number of ICD-10-CM diagnoses, and post-care encounters.

Conclusion

The proposed model attained discrimination comparable to prior analyses that used smaller datasets. However, reducing the sample enhanced performance, indicating big data complexity. Improved techniques like heuristic feature selection enabled effective leveraging of the nationwide data. This study provides meaningful insights into predictive modeling methodologies and influential features for forecasting heart failure readmissions.

Clinical Predictive Modeling of Heart Failure: Domain Description, Models' Characteristics and Literature Review

This study attempts to identify and briefly describe the current directions in applied and theoretical clinical prediction research. Context-rich chronic heart failure syndrome (CHFS) telemedicine provides the medical foundation for this effort. In the chronic stage of heart failure, there are sudden exacerbations of syndromes with subsequent hospitalizations, which are called acute decompensation of heart failure (ADHF). These decompensations are the subject of diagnostic and prognostic predictions. The primary purpose of ADHF predictions is to clarify the current and future health status of patients and subsequently optimize therapeutic responses. We proposed a simplified discrete-state disease model as an attempt at a typical summarization of a medical subject before starting predictive modeling. The study tries also to structure the essential common characteristics of quantitative models in order to understand the issue in an application context. The last part provides an overview of prediction works in the field of CHFS. These three parts provide the reader with a comprehensive view of quantitative clinical predictive modeling in heart failure telemedicine with an emphasis on several key general aspects. The target community is medical researchers seeking to align their clinical studies with prognostic or diagnostic predictive modeling, as well as other predictive researchers. The study was written by a non-medical expert.

Predicting unplanned readmission due to cardiovascular disease in hospitalized patients with cancer: a machine learning approach

Cardiovascular disease (CVD) in cancer patients can affect the risk of unplanned readmissions, which have been reported to be costly and associated with worse mortality and prognosis. We aimed to demonstrate the feasibility of using machine learning techniques in predicting the risk of unplanned 180-day readmission attributable to CVD among hospitalized cancer patients using the 2017-2018 Nationwide Readmissions Database. We included hospitalized cancer patients, and the outcome was unplanned hospital readmission due to any CVD within 180 days after discharge. CVD included atrial fibrillation, coronary artery disease, heart failure, stroke, peripheral artery disease, cardiomegaly, and cardiomyopathy. Decision tree (DT), random forest, extreme gradient boost (XGBoost), and AdaBoost were implemented. Accuracy, precision, recall, F2 score, and receiver operating characteristic curve (AUC) were used to assess the model's performance. Among 358,629 hospitalized patients with cancer, 5.86% (n = 21,021) experienced unplanned readmission due to any CVD. The three ensemble algorithms outperformed the DT, with the XGBoost displaying the best performance. We found length of stay, age, and cancer surgery were important predictors of CVD-related unplanned hospitalization in cancer patients. Machine learning models can predict the risk of unplanned readmission due to CVD among hospitalized cancer patients.