Performance of current risk stratification models for predicting mortality in patients with heart failure: a systematic review and meta-analysis

Risk Factors for All-Cause Mortality in Patients Diagnosed with Advanced Heart Failure: A Scoping Review

Introduction: Identifying the evolving needs of patients with advanced heart failure (AdHF) and triaging those at high risk of death can facilitate timely referrals to palliative care and advance patient-centered individualized care. There are limited models specific for patients with end-stage HF. We aim to identify risk factors associated with up to three-year all-cause mortality (ACM) and describe prognostic models developed or validated in AdHF populations. Methods: Frameworks proposed by Arksey, O'Malley, and Levac were adopted for this scoping review. We searched the Medline, EMBASE, PubMed, CINAHL, Cochrane library, Web of Science and gray literature databases for articles published between January 2010 and September 2020. Primary studies that included adults aged ≥ 18 years, diagnosed with AdHF defined as New York Heart Association class III/IV, American Heart Association/American College of Cardiology Stage D, end-stage HF, and assessed for risk factors associated with up to three-year ACM using multivariate analysis were included. Studies were appraised using the Quality of Prognostic Studies tool. Data were analyzed using a narrative synthesis approach. Results: We reviewed 167 risk factors that were associated with up to three-year ACM and prognostic models specific to AdHF patients across 65 articles with low-to-moderate bias. Studies were mostly based in Western and/or European cohorts (n = 60), in the acute care setting (n = 56), and derived from clinical trials (n = 40). Risk factors were grouped into six domains. Variables related to cardiovascular and overall health were frequently assessed. Ten prognostic models developed/validated on AdHF patients displayed acceptable model performance [area under the curve (AUC) range: 0.71-0.81]. Among the ten models, the model for end-stage-liver disease (MELD-XI) and acute decompensated HF with N-terminal pro b-type natriuretic peptide (ADHF/proBNP) model attained the highest discriminatory performance against short-term ACM (AUC: 0.81). Conclusions: To enable timely referrals to palliative care interventions, further research is required to develop or validate prognostic models that consider the evolving landscape of AdHF management.

Risk classification for long-term mortality among patients with acute heart failure: China PEACE 4YMortality

AIMS

There are limited tools to predict long-term mortality among patients hospitalized with acute heart failure (AHF) in China. This study aimed to develop and validate a model to predict long-term mortality risk among patients who were hospitalized with AHF and discharged alive.

METHODS

We used data from China Patient-Centred Evaluative Assessment of Cardiac Events Prospective Heart Failure Study. Multivariate Cox proportional hazard model was used to develop and internal validate a model to predict 4 year mortality risk.

RESULTS

The study included 4875 patients hospitalized for AHF, of whom 2066 (42.38%) died within 4 years following admission, with a median survival time of 3.91 (interquartile range: 1.67, 4.00) years. We selected 13 predictors to establish the model, including age, medical history of hypertension, chronic obstructive pulmonary disease and HF, systolic blood pressure, blood urea nitrogen, albumin, high-sensitivity troponin T, N-terminal pro-brain natriuretic peptide, serum creatine, Kansas City Cardiomyopathy Questionnaire-12 score and left ventricular ejection fraction. The model showed a reasonable performance with the discrimination [C-index was 0.726 (95% confidence interval, CI: 0.714, 0.739) in the development cohort and 0.727 (95% CI: 0.708, 0.747) in the validation cohort]. We then built a point-based risk score algorithm and the patients were stratified to low-risk (0-14), intermediate-risk (15-19) and high-risk (≥20) groups.

CONCLUSIONS

By using readily accessible predictors, we developed and validated a risk prediction model to predict 4 year mortality risk among patients who were hospitalized with AHF and discharged alive. This model proved beneficial for individual risk stratification and facilitating ongoing enhancements in patient outcomes.

Machine learning-based 30-day readmission prediction models for patients with heart failure: a systematic review

AIMS

Heart failure (HF) is one of the most frequent diagnoses for 30-day readmission after hospital discharge. Nurses have a role in reducing unplanned readmission and providing quality of care during HF trajectories. This systematic review assessed the quality and significant factors of machine learning (ML)-based 30-day HF readmission prediction models.

METHODS AND RESULTS

Eight academic and electronic databases were searched to identify all relevant articles published between 2013 and 2023. Thirteen studies met our inclusion criteria. The sample sizes of the selected studies ranged from 1778 to 272 778 patients, and the patients' average age ranged from 70 to 81 years. Quality appraisal was performed.

CONCLUSION

The most commonly used ML approaches were random forest and extreme gradient boosting. The 30-day HF readmission rates ranged from 1.2 to 39.4%. The area under the receiver operating characteristic curve for models predicting 30-day HF readmission was between 0.51 and 0.93. Significant predictors included 60 variables with 9 categories (socio-demographics, vital signs, medical history, therapy, echocardiographic findings, prescribed medications, laboratory results, comorbidities, and hospital performance index). Future studies using ML algorithms should evaluate the predictive quality of the factors associated with 30-day HF readmission presented in this review, considering different healthcare systems and types of HF. More prospective cohort studies by combining structured and unstructured data are required to improve the quality of ML-based prediction model, which may help nurses and other healthcare professionals assess early and accurate 30-day HF readmission predictions and plan individualized care after hospital discharge.

REGISTRATION

PROSPERO: CRD 42023455584.

Development of interpretable machine learning models to predict in-hospital prognosis of acute heart failure patients

AIMS

In recent years, there has been remarkable development in machine learning (ML) models, showing a trend towards high prediction performance. ML models with high prediction performance often become structurally complex and are frequently perceived as black boxes, hindering intuitive interpretation of the prediction results. We aimed to develop ML models with high prediction performance, interpretability, and superior risk stratification to predict in-hospital mortality and worsening heart failure (WHF) in patients with acute heart failure (AHF).

METHODS AND RESULTS

Based on the Kyoto Congestive Heart Failure registry, which enrolled 4056 patients with AHF, we developed prediction models for in-hospital mortality and WHF using information obtained on the first day of admission (demographics, physical examination, blood test results, etc.). After excluding 16 patients who died on the first or second day of admission, the original dataset (n = 4040) was split 4:1 into training (n = 3232) and test datasets (n = 808). Based on the training dataset, we developed three types of prediction models: (i) the classification and regression trees (CART) model; (ii) the random forest (RF) model; and (iii) the extreme gradient boosting (XGBoost) model. The performance of each model was evaluated using the test dataset, based on metrics including sensitivity, specificity, area under the receiver operating characteristic curve (AUC), Brier score, and calibration slope. For the complex structure of the XGBoost model, we performed SHapley Additive exPlanations (SHAP) analysis, classifying patients into interpretable clusters. In the original dataset, the proportion of females was 44.8% (1809/4040), and the average age was 77.9 ± 12.0. The in-hospital mortality rate was 6.3% (255/4040) and the WHF rate was 22.3% (900/4040) in the total study population. In the in-hospital mortality prediction, the AUC for the XGBoost model was 0.816 [95% confidence interval (CI): 0.815-0.818], surpassing the AUC values for the CART model (0.683, 95% CI: 0.680-0.685) and the RF model (0.755, 95% CI: 0.753-0.757). Similarly, in the WHF prediction, the AUC for the XGBoost model was 0.766 (95% CI: 0.765-0.768), outperforming the AUC values for the CART model (0.688, 95% CI: 0.686-0.689) and the RF model (0.713, 95% CI: 0.711-0.714). In the XGBoost model, interpretable clusters were formed, and the rates of in-hospital mortality and WHF were similar among each cluster in both the training and test datasets.

CONCLUSIONS

The XGBoost models with SHAP analysis provide high prediction performance, interpretability, and reproducible risk stratification for in-hospital mortality and WHF for patients with AHF.

Predicting the risk of 1-year mortality among patients hospitalized for acute heart failure in China

BACKGROUND

We aimed to develop and validate a model to predict 1-year mortality risk among patients hospitalized for acute heart failure (AHF), build a risk score and interpret its application in clinical decision making.

METHODS

By using data from China Patient-Centred Evaluative Assessment of Cardiac Events Prospective Heart Failure Study, which prospectively enrolled patients hospitalized for AHF in 52 hospitals across 20 provinces, we used multivariate Cox proportional hazard model to develop and validate a model to predict 1-year mortality.

RESULTS

There were 4,875 patients included in the study, 857 (17.58%) of them died within 1-year following discharge of index hospitalization. A total of 13 predictors were selected to establish the prediction model, including age, medical history of chronic obstructive pulmonary disease and hypertension, systolic blood pressure, Kansas City Cardiomyopathy Questionnaire-12 score, angiotensin converting enzyme inhibitor or angiotensin receptor blocker at discharge, discharge symptom, N-terminal pro-brain natriuretic peptide, high-sensitivity troponin T, serum creatine, albumin, blood urea nitrogen, and highly sensitive C-reactive protein. The model showed a high performance on discrimination (C-index was 0.759 [95% confidence interval: 0.739, 0.778] in development cohort and 0.761 [95% confidence interval: 0.731, 0.791] in validation cohort), accuracy, calibration, and outperformed than several existed risk scores. A point-based risk score was built to stratify low- (0-12), intermediate- (13-16), and high-risk group (≥17) among patients.

CONCLUSIONS

A prediction model using readily available predictors was developed and internal validated to predict 1-year mortality risk among patients hospitalized for AHF. It may serve as a useful tool for individual risk stratification and informing decision making to improve clinical care.

Stroke risk prediction models: A systematic review and meta-analysis

BACKGROUND

Prediction algorithms/models are viable methods for identifying individuals at high risk of stroke across diverse populations for timely intervention. However, evidence summarizing the performance of these models is limited. This study examined the performance and weaknesses of existing stroke risk-score-prediction models (SRSMs) and whether performance varied by population and region.

METHODS

PubMed, EMBASE, and Web of Science were searched for articles on SRSMs from the earliest records until February 2022. The Prediction Model Risk of Bias Assessment Tool was used to assess the quality of eligible articles. The performance of the SRSMs was assessed by meta-analyzing C-statistics (0 and 1) estimates from identified studies to determine the overall pooled C-statistics by fitting a linear restricted maximum likelihood in a random effect model.

RESULTS

Overall, 17 articles (cohort study = 15, nested case-control study = 2) comprising 739,134 stroke cases from 6,396,594 participants from diverse populations/regions (Asia; n = 8, United States; n = 3, and Europe and the United Kingdom; n = 6) were eligible for inclusion. The overall pooled c-statistics of SRSMs was 0.78 (95%CI: 0.75, 0.80; I2 = 99.9%), with most SRSMs developed using cohort studies; 0.78 (95%CI: 0.75, 0.80; I2 = 99.9%). The subgroup analyses by geographical region: Asia [0.81 (95%CI: 0.79, 0.83; I2 = 99.8%)], Europe and the United Kingdom [0.76 (95%CI: 0.69, 0.83; I2 = 99.9%)] and the United States only [0.75 (95%CI: 0.72, 0.78; I2 = 73.5%)] revealed relatively indifferent performances of SRSMs.

CONCLUSION

SRSM performance varied widely, and the pooled c-statistics of SRSMs suggested a fair predictive performance, with very few SRSMs validated in independent population group(s) from diverse world regions.

The cardiopulmonary exercise test in the prognostic evaluation of patients with heart failure and cardiomyopathies: the long history of making a one-size-fits-all suit

Cardiopulmonary exercise test (CPET) has become pivotal in the functional evaluation of patients with chronic heart failure (HF), supplying a holistic evaluation both in terms of exercise impairment degree and possible underlying mechanisms. Conversely, there is growing interest in investigating possible multiparametric approaches in order to improve the overall HF risk stratification. In such a context, in 2013, a group of 13 Italian centres skilled in HF management and CPET analysis built the Metabolic Exercise test data combined with Cardiac and Kidney Indexes (MECKI) score, based on the dynamic assessment of HF patients and on some other instrumental and laboratory parameters. Subsequently, the MECKI score, initially developed on a cohort of 2716 HF patients, has been extensively validated as well as challenged with the other multiparametric scores, achieving optimal results. Meanwhile, the MECKI score research group has grown over time, involving up to now a total of 27 centres with an available database accounting for nearly 8000 HF patients. This exciting joint effort from multiple HF Italian centres allowed to investigate different HF research field in order to deepen the mechanisms underlying HF, to improve the ability to identify patients at the highest risk as well as to analyse particular HF categories. Most recently, some of the participants of the MECKI score group started to join the forces in investigating a possible additive role of CPET assessment in the cardiomyopathy setting too. The present study tells the ten-year history of the MECKI score presenting the most important results achieved as well as those projects in the pipeline, this exciting journey being far to be concluded.

Cardiopulmonary exercise testing and heart failure: a tale born from oxygen uptake

Since 50 years, cardiopulmonary exercise testing (CPET) plays a central role in heart failure (HF) assessment. Oxygen uptake (VO2) is one of the main HF prognostic indicators, then paralleled by ventilation to carbon dioxide (VE/VCO2) relationship slope. Also anaerobic threshold retains a strong prognostic power in severe HF, especially if expressed as a percent of maximal VO2 predicted value. Moving beyond its absolute value, a modern approach is to consider the percentage of predicted value for peak VO2 and VE/VCO2 slope, thus allowing a better comparison between genders, ages, and races. Several VO2 equations have been adopted to predict peak VO2, built considering different populations. A step forward was made possible by the introduction of reliable non-invasive methods able to calculate cardiac output during exercise: the inert gas rebreathing method and the thoracic electrical bioimpedance. These techniques made possible to calculate the artero-venous oxygen content differences (ΔC(a-v)O2), a value related to haemoglobin concentration, pO2, muscle perfusion, and oxygen extraction. The role of haemoglobin, frequently neglected, is however essential being anaemia a frequent HF comorbidity. Finally, peak VO2 is traditionally obtained in a laboratory setting while performing a standardized physical effort. Recently, different wearable ergo-spirometers have been developed to allow an accurate metabolic data collection during different activities that better reproduce HF patients' everyday life. The evaluation of exercise performance is now part of the holistic approach to the HF syndrome, with the inclusion of CPET data into multiparametric prognostic scores, such as the MECKI score.