Derivation and validation of a machine learning-based risk prediction model in patients with acute heart failure

Usefulness of hypochloremia at the time of discharge to predict prognosis in patients with chronic heart failure after hospitalization

BACKGROUND

Hypochloremia has been suggested as a strong marker of mortality in hospitalized patients with heart failure (HF). This study aimed to clarify whether incorporating hypochloremia into pre-existing prognostic models improved the performance of the models.

METHODS

We tested the prognostic value of hypochloremia (<97 mEq/L) measured at discharge in hospitalized patients with HF registered in the REALITY-AHF and NARA-HF studies. The primary outcome was 1-year mortality after discharge.

RESULTS

Among 2496 patients with HF, 316 (12.6 %) had hypochloremia at the time of discharge, and 387 (15.5 %) deaths were observed within 1 year of discharge. The presence of hypochloremia was strongly associated with higher 1-year mortality compared to those without hypochloremia (log-rank: p < 0.001), and this association remained even after adjustment for the Get With the Guideline-HF risk model (GWTG-HF), anemia, New York Heart Association (NYHA) classification, and log-brain natriuretic peptide (BNP) [hazard ratio (HR) 1.64; p < 0.001]. Furthermore, adding hypochloremia to the prediction model composed of GWTG-HF + anemia + NYHA class + log-BNP yielded a numerically larger area under the curve (0.740 vs 0.749; p = 0.059) and significant improvement in net reclassification (0.159, p = 0.010).

CONCLUSIONS

Incorporating the presence of hypochloremia at discharge into pre-existing risk prediction models provides incremental prognostic information for hospitalized patients with HF.

Deep learning model for identifying acute heart failure patients using electrocardiography in the emergency room

AIMS

Acute heart failure (AHF) poses significant diagnostic challenges in the emergency room (ER) because of its varied clinical presentation and limitations of traditional diagnostic methods. This study aimed to develop and evaluate a deep learning model using electrocardiogram (ECG) data to enhance AHF identification in the ER.

METHODS AND RESULTS

In this retrospective cohort study, we analysed the ECG data of 19 285 patients who visited ERs of three hospitals between 2016 and 2020; 9119 with available left ventricular ejection fraction and N-terminal prohormone of brain natriuretic peptide level data and who were diagnosed with AHF were included in the study. We extracted morphological and clinical parameters from ECG data to train and validate four machine learning models: baseline linear regression and more advanced models including XGBoost, Light GBM, and CatBoost. The CatBoost algorithm outperformed other models, showing superior area under the receiver operating characteristic and area under the precision-recall curve diagnostic accuracy across both internal (0.89 ± 0.01 and 0.89 ± 0.01) and external (0.90 and 0.89) validation data sets, respectively. The model demonstrated high accuracy, precision, recall, and f1 score, indicating robust performance in AHF identification.

CONCLUSION

The developed machine learning model significantly enhanced AHF detection in the ER using conventional 12-lead ECGs combined with clinical data. These findings suggest that ECGs, a common tool in the ER, can effectively help screen for AHF.

Predictive Value of Machine Learning for the Risk of In-Hospital Death in Patients With Heart Failure: A Systematic Review and Meta-Analysis

BACKGROUND

The efficiency of machine learning (ML) based predictive models in predicting in-hospital mortality for heart failure (HF) patients is a topic of debate. In this context, this study's objective is to conduct a meta-analysis to compare and assess existing prognostic models designed for predicting in-hospital mortality in HF patients.

METHODS

A systematic search of databases was conducted, including PubMed, Embase, Web of Science, and Cochrane Library up to January 2023. To ensure comprehensiveness, we performed an additional search in June 2023. The Prediction Model Risk of Bias Assessment Tool was employed to assess the validity and reliability of ML models.

RESULTS

Our analysis incorporated 28 studies involving a total of 106 predictive models based on 14 different ML techniques. In the training data set, these models showed a combined C-index of 0.781, sensitivity of 0.56, and specificity of 0.94. In the validation data set, the models exhibited a combined C-index of 0.758, sensitivity of 0.57, and specificity of 0.84. Logistic regression (LR) was the most frequently used ML algorithm. LR models in the training set had a combined C-index of 0.795, sensitivity of 0.63, and specificity of 0.85, and these measures for LR models in the validation set were 0.751, 0.66, and 0.79, respectively.

CONCLUSIONS

Our study indicates that although ML is increasingly being leveraged to predict in-hospital mortality for HF patients, the predictive performance remains suboptimal. Although these models have relatively high C-index and specificity, their ability to predict positive events is limited, as indicated by their low sensitivity.

Development of interpretable machine learning models to predict in-hospital prognosis of acute heart failure patients

AIMS

In recent years, there has been remarkable development in machine learning (ML) models, showing a trend towards high prediction performance. ML models with high prediction performance often become structurally complex and are frequently perceived as black boxes, hindering intuitive interpretation of the prediction results. We aimed to develop ML models with high prediction performance, interpretability, and superior risk stratification to predict in-hospital mortality and worsening heart failure (WHF) in patients with acute heart failure (AHF).

METHODS AND RESULTS

Based on the Kyoto Congestive Heart Failure registry, which enrolled 4056 patients with AHF, we developed prediction models for in-hospital mortality and WHF using information obtained on the first day of admission (demographics, physical examination, blood test results, etc.). After excluding 16 patients who died on the first or second day of admission, the original dataset (n = 4040) was split 4:1 into training (n = 3232) and test datasets (n = 808). Based on the training dataset, we developed three types of prediction models: (i) the classification and regression trees (CART) model; (ii) the random forest (RF) model; and (iii) the extreme gradient boosting (XGBoost) model. The performance of each model was evaluated using the test dataset, based on metrics including sensitivity, specificity, area under the receiver operating characteristic curve (AUC), Brier score, and calibration slope. For the complex structure of the XGBoost model, we performed SHapley Additive exPlanations (SHAP) analysis, classifying patients into interpretable clusters. In the original dataset, the proportion of females was 44.8% (1809/4040), and the average age was 77.9 ± 12.0. The in-hospital mortality rate was 6.3% (255/4040) and the WHF rate was 22.3% (900/4040) in the total study population. In the in-hospital mortality prediction, the AUC for the XGBoost model was 0.816 [95% confidence interval (CI): 0.815-0.818], surpassing the AUC values for the CART model (0.683, 95% CI: 0.680-0.685) and the RF model (0.755, 95% CI: 0.753-0.757). Similarly, in the WHF prediction, the AUC for the XGBoost model was 0.766 (95% CI: 0.765-0.768), outperforming the AUC values for the CART model (0.688, 95% CI: 0.686-0.689) and the RF model (0.713, 95% CI: 0.711-0.714). In the XGBoost model, interpretable clusters were formed, and the rates of in-hospital mortality and WHF were similar among each cluster in both the training and test datasets.

CONCLUSIONS

The XGBoost models with SHAP analysis provide high prediction performance, interpretability, and reproducible risk stratification for in-hospital mortality and WHF for patients with AHF.

Positive association between sodium-to-chloride ratio and in-hospital mortality of acute heart failure

Previous studies have suggested that levels of sodium and chloride in the blood may be indicative of the prognosis of different medical conditions. Nevertheless, the assessment of the prognostic significance of the sodium-to-chloride (Na/Cl) ratio in relation to in-hospital mortality among individuals suffering from acute heart failure (AHF) remains unexplored. In this study, the participants were selected from the Medical Information Mart for Intensive Care IV database and divided into three groups based on the Na/Cl ratio level upon admission. The primary results were the mortality rate within the hospital. Cox regression, Kaplan-Meier curves, receiver operator characteristic (ROC) curve analysis and subgroup analyses were utilized to investigate the correlation between the admission Na/Cl ratio and outcomes in critically ill patients with AHF. A total of 7844 patients who met the selection criteria were included in this study. After adjusting for confounders, the multivariable Cox regression analysis revealed that the baseline Na/Cl ratio significantly elevated the risk of in-hospital mortality among critically ill patients with AHF (HR = 1.34, 95% CI 1.21-1.49). Furthermore, when the Na/Cl ratio was converted into a categorical factor and the initial tertile was taken as a point of comparison, the hazard ratios (HRs) and 95% confidence intervals (CIs) for the second and third tertiles were 1.27 (1.05-1.54) and 1.53 (1.27-1.84), respectively. Additionally, a P value indicating a significant trend of < 0.001 was observed. ROC curve analysis showed that Na/Cl ratio had a more sensitive prognostic value in predicting in-hospital mortality of AHF than the sodium or chloride level alone (0.564 vs. 0.505, 0.544). Subgroup examinations indicated that the association between the Na/Cl ratio upon admission and the mortality rate of critically ill patients with AHF remained consistent in the subgroups of hyponatremia and hypochlorhydria (P for interaction > 0.05). The linear relationship between the Na/Cl ratio and in-hospital mortality in AHF patients indicates a positive association.