Prediction of 30-day mortality in heart failure patients with hypoxic hepatitis: Development and external validation of an interpretable machine learning model

Evaluation of machine learning methods for prediction of heart failure mortality and readmission: meta-analysis

BACKGROUND

Heart failure (HF) impacts nearly 6 million individuals in the U.S., with a projected 46% increase by 2030, is creating significant healthcare burdens. Predictive models, particularly machine learning (ML)-based models, offer promising solutions to identify patients at greater risk of adverse outcomes, such as mortality and hospital readmission. This review aims to assess the effectiveness of ML models in predicting HF-related outcomes, with a focus on their potential to improve patient care and clinical decision-making. We aim to assess how effectively machine learning models predict mortality and readmission in heart failure patients to improve clinical outcomes.

METHOD

The study followed PRISMA 2020 guidelines and was registered in the PROSPERO database (CRD42023481167). We conducted a systematic search in PubMed, Scopus, and Web of Science databases using specific keywords related to heart failure, machine learning, mortality and readmission. Extracted data focused on study characteristics, machine learning details, and outcomes, with AUC or c-index used as the primary outcomes for pooling analysis. The PROBAST tool was used to assess bias risk, evaluating models based on participants, predictors, outcomes, and statistical analysis. The meta-analysis pooled AUCs for different machine learning models predicting mortality and readmission. Prediction accuracy data was categorized by timeframes, with high heterogeneity determined by an I² value above 50%, leading to a random-effects model when applicable. Publication bias was assessed using Egger's and Begg's tests, with a p-value below 0.05 considered significant RESULT: A total of 4,505 studies were identified, and after screening, 64 were included in the final analysis, covering 943,941 patients. Of these, 40 studies focused on mortality, 17 on readmission, and 7 on both outcomes. In total, 346 machine learning models were evaluated, with the most common algorithms being random forest, logistic regression, and gradient boosting. The neural network model achieved the highest overall AUC for mortality prediction (0.808), while the support vector machine performed best for readmission prediction (AUC 0.733). The analysis revealed a significant risk of bias, primarily due to reliance on retrospective data and inadequate sample size justification.

CONCLUSION

In conclusion, this review emphasizes the strong potential of ML models in predicting HF readmission and mortality. ML algorithms show promise in improving prognostic accuracy and enabling personalized patient care. However, challenges like model interpretability, generalizability, and clinical integration persist. Overcoming these requires refined ML techniques and a robust regulatory framework to enhance HF outcomes.

Predicting the risk of heart failure after acute myocardial infarction using an interpretable machine learning model

Background

Early prediction of heart failure (HF) after acute myocardial infarction (AMI) is essential for personalized treatment. We aimed to use interpretable machine learning (ML) methods to develop a risk prediction model for HF in AMI patients.

Methods

We retrospectively included patients initially with AMI who received percutaneous coronary intervention (PCI) in our hospital from November 2016 to February 2020. The primary endpoint was the occurrence of HF within 3 years after operation. For developing a predictive model for HF risk in AMI patients, the least absolute shrinkage and selection operator (LASSO) Regression was used to feature selection, and four ML algorithms including Random Forest (RF), Extreme Gradient Boost (XGBoost), Support Vector Machine (SVM), and Logistic Regression (LR) were employed to develop the model on the training set. The performance evaluation of the prediction model was carried out on the training set and the testing set, utilizing metrics including AUC (Area under the receiver operating characteristic curve), calibration plot, and decision curve analysis (DCA). In addition, we used the Shapley Additive Explanations (SHAP) value to determine the importance of the selected features and interpret the optimal model.

Results

A total of 1220 AMI patients were included and 244 (20%) patients developed HF during follow-up. Among the four evaluated ML models, the XGBoost model exhibited exceptional accuracy, with an AUC value of 0.922. The SHAP method showed that left ventricular ejection fraction (LVEF), left ventricular end-systolic diameter (LVDs) and lactate dehydrogenase (LDH) were identified as the three most important characteristics to predict HF risk in AMI patients. Individual risk assessment was performed using SHAP plots and waterfall plot analysis.

Conclusions

Our research demonstrates the potential of ML methods in the early prediction of HF risk in AMI patients. Furthermore, it enhances the interpretability of the XGBoost model through SHAP analysis to guide clinical decision-making.

Prediction of preterm birth using machine learning: a comprehensive analysis based on large-scale preschool children survey data in Shenzhen of China

BACKGROUND

Preterm birth (PTB) is a significant cause of neonatal mortality and long-term health issues. Accurate prediction and timely prevention of PTB are essential for reducing associated child mortality and morbidity. Traditional predictive methods face challenges due to heterogeneous risk factors and their interaction effects. This study aims to develop and evaluate six machine learning (ML) models to predict PTB using large-scale children survey data from Shenzhen, China, and to identify key predictors through Shapley Additive Explanations (SHAP) analysis.

METHODS

Data from 84,050 mother-child pairs, collected in 2021 and 2022, were processed and divided into training, validation, and test sets. Six ML models were tested: L1-Regularised Logistic Regression, Light Gradient Boosting Machine (LightGBM), Naive Bayes, Random Forests, Support Vector Machine, and Extreme Gradient Boosting (XGBoost). Model performance was evaluated based on discrimination, calibration and clinical utility. SHAP analysis was used to interpret the importance and impact of individual features on PTB prediction.

RESULTS

The XGBoost model demonstrated the best overall performance, with the area under the receiver operating characteristic curve (AUC) scores of 0.752 and 0.757 in the validation and test sets, respectively, along with favorable calibration and clinical utility. Key predictors identified were multiple pregnancies, threatened abortion, and maternal age of conception. SHAP analysis highlighted the positive impacts of multiple pregnancies and threatened abortion, as well as the negative impact of micronutrient supplementation on PTB.

CONCLUSION

Our study found that ML models, particularly XGBoost, show promise in accurately predicting PTB and identifying key risk factors. These findings provide the potential of ML for enhancing clinical interventions, personalizing prenatal care, and informing public health initiatives.

Unveiling the Power of Gut Microbiome in Predicting Neoadjuvant Immunochemotherapy Responses in Esophageal Squamous Cell Carcinoma

The role of the gut microbiome in enhancing the efficacy of anticancer treatments like chemotherapy and radiotherapy is well acknowledged. However, there is limited empirical evidence on its predictive capabilities for neoadjuvant immunochemotherapy (NICT) responses in esophageal squamous cell carcinoma (ESCC). Our study fills this gap by comprehensively analyzing the gut microbiome's influence on NICT outcomes. We analyzed 16S rRNA gene sequences from 136 fecal samples from 68 ESCC patients before and after NICT, along with 19 samples from healthy controls. After NICT, marked microbiome composition changes were noted, including a decrease in ESCC-associated pathogens and an increase in beneficial microbes such as Limosilactobacillus, Lacticaseibacillus, and Staphylococcus. Baseline microbiota profiles effectively differentiated responders from nonresponders, with responders showing higher levels of short-chain fatty acid (SCFA)-producing bacteria such as Faecalibacterium, Eubacterium_eligens_group, Anaerostipes, and Odoribacter, and nonresponders showing increases in Veillonella, Campylobacter, Atopobium, and Trichococcus. We then divided our patient cohort into training and test sets at a 4:1 ratio and utilized the XGBoost-RFE algorithm to identify 7 key microbial biomarkers-Faecalibacterium, Subdoligranulum, Veillonella, Hungatella, Odoribacter, Butyricicoccus, and HT002. A predictive model was developed using LightGBM, which achieved an area under the receiver operating characteristic curve (AUC) of 86.8% [95% confidence interval (CI), 73.8% to 99.4%] in the training set, 76.8% (95% CI, 41.2% to 99.7%) in the validation set, and 76.5% (95% CI, 50.4% to 100%) in the testing set. Our findings underscore the gut microbiome as a novel source of biomarkers for predicting NICT responses in ESCC, highlighting its potential to enhance personalized treatment strategies and advance the integration of microbiome profiling into clinical practice for modulating cancer treatment responses.

A review of evaluation approaches for explainable AI with applications in cardiology

Explainable artificial intelligence (XAI) elucidates the decision-making process of complex AI models and is important in building trust in model predictions. XAI explanations themselves require evaluation as to accuracy and reasonableness and in the context of use of the underlying AI model. This review details the evaluation of XAI in cardiac AI applications and has found that, of the studies examined, 37% evaluated XAI quality using literature results, 11% used clinicians as domain-experts, 11% used proxies or statistical analysis, with the remaining 43% not assessing the XAI used at all. We aim to inspire additional studies within healthcare, urging researchers not only to apply XAI methods but to systematically assess the resulting explanations, as a step towards developing trustworthy and safe models.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10462-024-10852-w.

Development and Validation of an Interpretable Machine Learning Model for Early Prognosis Prediction in ICU Patients with Malignant Tumors and Hyperkalemia

This study aims to develop and validate a machine learning (ML) predictive model for assessing mortality in patients with malignant tumors and hyperkalemia (MTH). We extracted data on patients with MTH from the Medical Information Mart for Intensive Care-IV, version 2.2 (MIMIC-IV v2.2) database. The dataset was split into a training set (75%) and a validation set (25%). We used the Least Absolute Shrinkage and Selection Operator (LASSO) regression to identify potential predictors, which included clinical laboratory indicators and vital signs. Pearson correlation analysis tested the correlation between predictors. In-hospital death was the prediction target. The Area Under the Curve (AUC) and accuracy of the training and validation sets of 7 ML algorithms were compared, and the optimal 1 was selected to develop the model. The calibration curve was used to evaluate the prediction accuracy of the model further. SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) enhanced model interpretability. 496 patients with MTH in the Intensive Care Unit (ICU) were included. After screening, 17 clinical features were included in the construction of the ML model, and the Pearson correlation coefficient was <0.8, indicating that the correlation between the clinical features was small. eXtreme Gradient Boosting (XGBoost) outperformed other algorithms, achieving perfect scores in the training set (accuracy: 1.000, AUC: 1.000) and high scores in the validation set (accuracy: 0.734, AUC: 0.733). The calibration curves indicated good predictive calibration of the model. SHAP analysis identified the top 8 predictive factors: urine output, mean heart rate, maximum urea nitrogen, minimum oxygen saturation, minimum mean blood pressure, maximum total bilirubin, mean respiratory rate, and minimum pH. In addition, SHAP and LIME performed in-depth individual case analyses. This study demonstrates the effectiveness of ML methods in predicting mortality risk in ICU patients with MTH. It highlights the importance of predictors like urine output and mean heart rate. SHAP and LIME significantly enhanced the model's interpretability.

Development and Validation of an ICU-Venous Thromboembolism Prediction Model Using Machine Learning Approaches: A Multicenter Study

Purpose

The purpose of this study was to establish and validate machine learning-based models for predicting the risk of venous thromboembolism (VTE) in intensive care unit (ICU) patients.

Patients and Methods

The clinical data of 1494 ICU patients who underwent Doppler ultrasonography or venography between December 2020 and March 2023 were extracted from three tertiary hospitals. The Boruta algorithm was used to screen the essential variables associated with VTE. Five machine learning algorithms were employed: Random Forest (RF), eXtreme Gradient Boosting (XGBoost), Support Vector Machine (SVM), Gradient Boosting Decision Tree (GBDT), and Logistic Regression (LR). Hyperparameter optimization was conducted on the predictive model of the training dataset. The performance in the validation dataset was measured using indicators, including the area under curve (AUC) of the receiver operating characteristic (ROC) curve, specificity, and F1 score. Finally, the optimal model was interpreted using the SHapley Additive exPlanation (SHAP) package.

Results

The incidence of VTE among the ICU patients in this study was 26.04%. We screened 19 crucial features for the risk prediction model development. Among the five models, the RF model performed best, with an AUC of 0.788 (95% CI: 0.738-0.838), an accuracy of 0.759 (95% CI: 0.709-0.809), a sensitivity of 0.633, and a Brier score of 0.166.

Conclusion

A machine learning-based model for prediction of VTE in ICU patients were successfully developed, which could assist clinical medical staff in identifying high-risk populations for VTE in the early stages so that prevention measures can be implemented to reduce the burden on the ICU patients.

Predicting the risk of mortality and rehospitalization in heart failure patients: A retrospective cohort study by machine learning approach

BACKGROUND

Heart failure (HF) is a global problem, affecting more than 26 million people worldwide. This study evaluated the performance of 10 machine learning (ML) algorithms and chose the best algorithm to predict mortality and readmission of HF patients by using The Fasa Registry on Systolic HF (FaRSH) database.

HYPOTHESIS

ML algorithms may better identify patients at increased risk of HF readmission or death with demographic and clinical data.

METHODS

Through comprehensive evaluation, the best-performing model was used for prediction. Finally, all the trained models were applied to the test data, which included 20% of the total data. For the final evaluation and comparison of the models, five metrics were used: accuracy, F1-score, sensitivity, specificity and Area Under Curve (AUC).

RESULTS

Ten ML algorithms were evaluated. The CatBoost (CAT) algorithm uses a series of decision tree models to create a nonlinear model, and this CAT algorithm performed the best of the 10 models studied. According to the three final outcomes from this study, which involved 2488 participants, 366 (14.7%) of the patients were readmitted to the hospital, 97 (3.9%) of the patients died within 1 month of the follow-up, and 342 (13.7%) of the patients died within 1 year of the follow-up. The most significant variables to predict the events were length of stay in the hospital, hemoglobin level, and family history of MI.

CONCLUSIONS

The ML-based risk stratification tool was able to assess the risk of 5-year all-cause mortality and readmission in patients with HF. ML could provide an explicit explanation of individualized risk prediction and give physicians an intuitive understanding of the influence of critical features in the model.

The application of explainable artificial intelligence (XAI) in electronic health record research: A scoping review

Machine Learning (ML) and Deep Learning (DL) models show potential in surpassing traditional methods including generalised linear models for healthcare predictions, particularly with large, complex datasets. However, low interpretability hinders practical implementation. To address this, Explainable Artificial Intelligence (XAI) methods are proposed, but a comprehensive evaluation of their effectiveness is currently limited. The aim of this scoping review is to critically appraise the application of XAI methods in ML/DL models using Electronic Health Record (EHR) data. In accordance with PRISMA scoping review guidelines, the study searched PUBMED and OVID/MEDLINE (including EMBASE) for publications related to tabular EHR data that employed ML/DL models with XAI. Out of 3220 identified publications, 76 were included. The selected publications published between February 2017 and June 2023, demonstrated an exponential increase over time. Extreme Gradient Boosting and Random Forest models were the most frequently used ML/DL methods, with 51 and 50 publications, respectively. Among XAI methods, Shapley Additive Explanations (SHAP) was predominant in 63 out of 76 publications, followed by partial dependence plots (PDPs) in 11 publications, and Locally Interpretable Model-Agnostic Explanations (LIME) in 8 publications. Despite the growing adoption of XAI methods, their applications varied widely and lacked critical evaluation. This review identifies the increasing use of XAI in tabular EHR research and highlights a deficiency in the reporting of methods and a lack of critical appraisal of validity and robustness. The study emphasises the need for further evaluation of XAI methods and underscores the importance of cautious implementation and interpretation in healthcare settings.

Predicting the 5-Year Risk of Nonalcoholic Fatty Liver Disease Using Machine Learning Models: Prospective Cohort Study

BACKGROUND

Nonalcoholic fatty liver disease (NAFLD) has emerged as a worldwide public health issue. Identifying and targeting populations at a heightened risk of developing NAFLD over a 5-year period can help reduce and delay adverse hepatic prognostic events.

OBJECTIVE

This study aimed to investigate the 5-year incidence of NAFLD in the Chinese population. It also aimed to establish and validate a machine learning model for predicting the 5-year NAFLD risk.

METHODS

The study population was derived from a 5-year prospective cohort study. A total of 6196 individuals without NAFLD who underwent health checkups in 2010 at Zhenhai Lianhua Hospital in Ningbo, China, were enrolled in this study. Extreme gradient boosting (XGBoost)-recursive feature elimination, combined with the least absolute shrinkage and selection operator (LASSO), was used to screen for characteristic predictors. A total of 6 machine learning models, namely logistic regression, decision tree, support vector machine, random forest, categorical boosting, and XGBoost, were utilized in the construction of a 5-year risk model for NAFLD. Hyperparameter optimization of the predictive model was performed in the training set, and a further evaluation of the model performance was carried out in the internal and external validation sets.

RESULTS

The 5-year incidence of NAFLD was 18.64% (n=1155) in the study population. We screened 11 predictors for risk prediction model construction. After the hyperparameter optimization, CatBoost demonstrated the best prediction performance in the training set, with an area under the receiver operating characteristic (AUROC) curve of 0.810 (95% CI 0.768-0.852). Logistic regression showed the best prediction performance in the internal and external validation sets, with AUROC curves of 0.778 (95% CI 0.759-0.794) and 0.806 (95% CI 0.788-0.821), respectively. The development of web-based calculators has enhanced the clinical feasibility of the risk prediction model.

CONCLUSIONS

Developing and validating machine learning models can aid in predicting which populations are at the highest risk of developing NAFLD over a 5-year period, thereby helping delay and reduce the occurrence of adverse liver prognostic events.

Artificial intelligence and heart failure: A state-of-the-art review

Heart failure (HF) is a heterogeneous syndrome affecting more than 60 million individuals globally. Despite recent advancements in understanding of the pathophysiology of HF, many issues remain including residual risk despite therapy, understanding the pathophysiology and phenotypes of patients with HF and preserved ejection fraction, and the challenges related to integrating a large amount of disparate information available for risk stratification and management of these patients. Risk prediction algorithms based on artificial intelligence (AI) may have superior predictive ability compared to traditional methods in certain instances. AI algorithms can play a pivotal role in the evolution of HF care by facilitating clinical decision making to overcome various challenges such as allocation of treatment to patients who are at highest risk or are more likely to benefit from therapies, prediction of adverse outcomes, and early identification of patients with subclinical disease or worsening HF. With the ability to integrate and synthesize large amounts of data with multidimensional interactions, AI algorithms can supply information with which physicians can improve their ability to make timely and better decisions. In this review, we provide an overview of the AI algorithms that have been developed for establishing early diagnosis of HF, phenotyping HF with preserved ejection fraction, and stratifying HF disease severity. This review also discusses the challenges in clinical deployment of AI algorithms in HF, and the potential path forward for developing future novel learning-based algorithms to improve HF care.