Prediction of Neurological Outcomes in Out-of-hospital Cardiac Arrest Survivors Immediately after Return of Spontaneous Circulation: Ensemble Technique with Four Machine Learning Models

Application of Machine Learning for Patients With Cardiac Arrest: Systematic Review and Meta-Analysis

BACKGROUND

Currently, there is a lack of effective early assessment tools for predicting the onset and development of cardiac arrest (CA). With the increasing attention of clinical researchers on machine learning (ML), some researchers have developed ML models for predicting the occurrence and prognosis of CA, with certain models appearing to outperform traditional scoring tools. However, these models still lack systematic evidence to substantiate their efficacy.

OBJECTIVE

This systematic review and meta-analysis was conducted to evaluate the prediction value of ML in CA for occurrence, good neurological prognosis, mortality, and the return of spontaneous circulation (ROSC), thereby providing evidence-based support for the development and refinement of applicable clinical tools.

METHODS

PubMed, Embase, the Cochrane Library, and Web of Science were systematically searched from their establishment until May 17, 2024. The risk of bias in all prediction models was assessed using the Prediction Model Risk of Bias Assessment Tool.

RESULTS

In total, 93 studies were selected, encompassing 5,729,721 in-hospital and out-of-hospital patients. The meta-analysis revealed that, for predicting CA, the pooled C-index, sensitivity, and specificity derived from the imbalanced validation dataset were 0.90 (95% CI 0.87-0.93), 0.83 (95% CI 0.79-0.87), and 0.93 (95% CI 0.88-0.96), respectively. On the basis of the balanced validation dataset, the pooled C-index, sensitivity, and specificity were 0.88 (95% CI 0.86-0.90), 0.72 (95% CI 0.49-0.95), and 0.79 (95% CI 0.68-0.91), respectively. For predicting the good cerebral performance category score 1 to 2, the pooled C-index, sensitivity, and specificity based on the validation dataset were 0.86 (95% CI 0.85-0.87), 0.72 (95% CI 0.61-0.81), and 0.79 (95% CI 0.66-0.88), respectively. For predicting CA mortality, the pooled C-index, sensitivity, and specificity based on the validation dataset were 0.85 (95% CI 0.82-0.87), 0.83 (95% CI 0.79-0.87), and 0.79 (95% CI 0.74-0.83), respectively. For predicting ROSC, the pooled C-index, sensitivity, and specificity based on the validation dataset were 0.77 (95% CI 0.74-0.80), 0.53 (95% CI 0.31-0.74), and 0.88 (95% CI 0.71-0.96), respectively. In predicting CA, the most significant modeling variables were respiratory rate, blood pressure, age, and temperature. In predicting a good cerebral performance category score 1 to 2, the most significant modeling variables in the in-hospital CA group were rhythm (shockable or nonshockable), age, medication use, and gender; the most significant modeling variables in the out-of-hospital CA group were age, rhythm (shockable or nonshockable), medication use, and ROSC.

CONCLUSIONS

ML represents a currently promising approach for predicting the occurrence and outcomes of CA. Therefore, in future research on CA, we may attempt to systematically update traditional scoring tools based on the superior performance of ML in specific outcomes, achieving artificial intelligence-driven enhancements.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42024518949; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=518949.

Post-Cardiac arrest outcome prediction using machine learning: A systematic review and meta-analysis

BACKGROUND

Early and reliable prognostication in post-cardiac arrest patients remains challenging, with various factors linked to return of spontaneous circulation (ROSC), survival, and neurological results. Machine learning and deep learning models show promise in improving these predictions. This systematic review and meta-analysis evaluates how effective these approaches are in predicting clinical outcomes at different time points using structured data.

METHODS

This study followed PRISMA guidelines, involving a comprehensive search across PubMed, Scopus, and Web of Science databases until March 2024. Studies aimed at predicting ROSC, survival (or mortality), and neurological outcomes after cardiac arrest through the application of machine learning or deep learning techniques with structured data were included. Data extraction followed the guidelines of the CHARMS checklist, and the bias risk was evaluated using PROBAST tool. Models reporting the AUC metric with 95 % confidence intervals were incorporated into the quantitative synthesis and meta-analysis.

RESULTS

After extracting 2,753 initial records, 41 studies met the inclusion criteria, yielding 97 machine learning and 16 deep learning models. The pooled AUC for predicting favorable neurological outcomes (CPC 1 or 2) at hospital discharge was 0.871 (95 % CI: 0.813 - 0.928) for machine learning models and 0.877 (95 % CI: 0.831-0.924) across deep learning algorithms. For survival prediction, this value was found to be 0.837 (95 % CI: 0.757-0.916). Considerable heterogeneity and high risk of bias were observed, mainly attributable to inadequate management of missing data and the absence of calibration plots. Most studies focused on pre-hospital factors, with age, sex, and initial arrest rhythm being the most frequent features.

CONCLUSION

Predictive models utilizing AI-based approaches, including machine and deep learning models exhibit enhanced effectiveness compared to previous regression algorithms, but significant heterogeneity and high risk of bias limit their dependability. Evaluating state-of-the-art deep learning models tailored for tabular data and their clinical generalizability can enhance outcome prediction after cardiac arrest.

Prediction of neurologic outcome after out-of-hospital cardiac arrest: An interpretable approach with machine learning

Out-of-hospital cardiac arrest (OHCA) is a critical condition with low survival rates. In patients with a return of spontaneous circulation, brain injury is a leading cause of death. In this study, we propose an interpretable machine learning approach for predicting neurologic outcome after OHCA, using information available at the time of hospital admission.

METHODS

The study population were 55 615 OHCA cases registered in the Swedish Cardiopulmonary Resuscitation Registry between 2010 and 2020. The dataset was split to training and validation sets (for model development) and test set (for evaluation of the final model). We used an XGBoost algorithm with stratified, repeated 10-fold cross-validation along with Optuna framework for hyperparameters tuning. The final model was trained on 10 features selected based on the importance scores and evaluated on the test set in terms of discrimination, calibration and bias-variance tradeoff. We used SHapley Additive exPlanations to address the 'black-box' model and align with eXplainable artificial intelligence.

RESULTS

The final model achieved: area under the receiver operating characteristic value 0.964 (95% confidence interval (CI) [0.960-0.968]), sensitivity 0.606 (95% CI [0.573-0.634]), specificity 0.975 (95% CI [0.972-0.978]), positive predictive value (PPV) 0.664 (95% CI [0.625-0.696]), negative predictive value (NPV) 0.969 (95% CI [0.966-0.972]), macro F1 0.803 (95% CI [0.788-0.816]), and showed a very good calibration. SHAP features with the highest impact on the model's output were:'ROSC on arrival to hospital', 'Initial rhythm asystole' and 'Conscious on arrival to hospital'.

CONCLUSIONS

The XGBoost machine learning model with 10 features available at the time of hospital admission showed good performance for predicting neurologic outcome after OHCA, with no apparent signs of overfitting.

Development of a Predictive Model for Survival Over Time in Patients With Out-of-Hospital Cardiac Arrest Using Ensemble-Based Machine Learning

As of now, a model for predicting the survival of patients with out-of-hospital cardiac arrest has not been established. This study aimed to develop a model for identifying predictors of survival over time in patients with out-of-hospital cardiac arrest during their stay in the emergency department, using ensemble-based machine learning. A total of 26 013 patients from the Korean nationwide out-of-hospital cardiac arrest registry were enrolled between January 1 and December 31, 2019. Our model, comprising 38 variables, was developed using the Survival Quilts model to improve predictive performance. We found that changes in important variables of patients with out-of-hospital cardiac arrest were observed 10 minutes after arrival at the emergency department. The important score of the predictors showed that the influence of patient age decreased, moving from the highest rank to the fifth. In contrast, the significance of reperfusion attempts increased, moving from the fourth to the highest rank. Our research suggests that the ensemble-based machine learning model, particularly the Survival Quilts, offers a promising approach for predicting survival in patients with out-of-hospital cardiac arrest. The Survival Quilts model may potentially assist emergency department staff in making informed decisions quickly, reducing preventable deaths.

Prediction model of in-hospital mortality in intensive care unit patients with cardiac arrest: a retrospective analysis of MIMIC -IV database based on machine learning

BACKGROUND

Both in-hospital cardiac arrest (IHCA) and out-of-hospital cardiac arrest (OHCA) have higher incidence and lower survival rates. Predictors of in-hospital mortality for intensive care unit (ICU) admitted cardiac arrest (CA) patients remain unclear.

METHODS

The Medical Information Mart for Intensive Care IV (MIMIC-IV) database was used to perform a retrospective study. Patients meeting the inclusion criteria were identified from the MIMIC-IV database and randomly divided into training set (n = 1206, 70%) and validation set (n = 516, 30%). Candidate predictors consisted of the demographics, comorbidity, vital signs, laboratory test results, scoring systems, and treatment information on the first day of ICU admission. Independent risk factors for in-hospital mortality were screened using the least absolute shrinkage and selection operator (LASSO) regression model and the extreme gradient boosting (XGBoost) in the training set. Multivariate logistic regression analysis was used to build prediction models in training set, and then validated in validation set. Discrimination, calibration and clinical utility of these models were compared using the area under the curve (AUC) of the receiver operating characteristic (ROC) curves, calibration curves and decision curve analysis (DCA). After pairwise comparison, the best performing model was chosen to build a nomogram.

RESULTS

Among the 1722 patients, in-hospital mortality was 53.95%. In both sets, the LASSO, XGBoost,the logistic regression(LR) model and the National Early Warning Score 2 (NEWS 2) models showed acceptable discrimination. In pairwise comparison, the prediction effectiveness was higher with the LASSO,XGBoost and LR models than the NEWS 2 model (p < 0.001). The LASSO,XGBoost and LR models also showed good calibration. The LASSO model was chosen as our final model for its higher net benefit and wider threshold range. And the LASSO model was presented as the nomogram.

CONCLUSIONS

The LASSO model enabled good prediction of in-hospital mortality in ICU admission CA patients, which may be widely used in clinical decision-making.