Interpretable Machine Learning Model for Early Prediction of Mortality in ICU Patients with Rhabdomyolysis

Early sepsis mortality prediction model based on interpretable machine learning approach: development and validation study

Sepsis triggers a harmful immune response due to infection, causing high mortality. Predicting sepsis outcomes early is vital. Despite machine learning's (ML) use in medical research, local validation within the Medical Information Mart for Intensive Care IV (MIMIC-IV) database is lacking. We aimed to devise a prognostic model, leveraging MIMIC-IV data, to predict sepsis mortality and validate it in a Chinese teaching hospital. MIMIC-IV provided patient data, split into training and internal validation sets. Four ML models logistic regression (LR), support vector machine (SVM), deep neural networks (DNN), and extreme gradient boosting (XGBoost) were employed. Shapley additive interpretation offered early and interpretable mortality predictions. Area under the ROC curve (AUROC) gaged predictive performance. Results were cross verified in a Chinese teaching hospital. The study included 27,134 sepsis patients from MIMIC-IV and 487 from China. After comparing, 52 clinical indicators were selected for ML model development. All models exhibited excellent discriminative ability. XGBoost surpassed others, with AUROC of 0.873 internally and 0.844 externally. XGBoost outperformed other ML models (LR: 0.829; SVM: 0.830; DNN: 0.837) and clinical scores (Simplified Acute Physiology Score II: 0.728; Sequential Organ Failure Assessment: 0.728; Oxford Acute Severity of Illness Score: 0.738; Glasgow Coma Scale: 0.691). XGBoost's hospital mortality prediction achieved AUROC 0.873, sensitivity 0.818, accuracy 0.777, specificity 0.768, and F1 score 0.551. We crafted an interpretable model for sepsis death risk prediction. ML algorithms surpassed traditional scores for sepsis mortality forecast. Validation in a Chinese teaching hospital echoed these findings.

COL4A2 -Related Disorder Presenting in Adulthood With Rhabdomyolysis

The alpha 1 and 2 chains of type IV collagen, encoded by the COL4A1 (MIM 120130) and COL4A2 (MIM 120090) respectively, play essential roles in the vascular basement membranes. Pathogenic variants in COL4A1/ COL4A2 are associated with autosomal dominant cerebral angiopathies. The clinical manifestations of COL4A1/COL4A2-related disorders include: aneurysms, intracerebral hemorrhage, polymicrogyria, porencephaly, heterotopia, periventricular leukomalacia, epilepsy, and neurodevelopmental disorders. COL4A1 pathogenic variants that are in exons 24 and 25 have been associated with hereditary angiopathy, nephropathy, aneurysms, and cramps. The multisystemic phenotypes of COL4A1/COL4A2-related disorders are increasingly being studied. Animal models have suggested that COL4A2-related disorders may also manifest with a variable combination of multisystemic abnormalities affecting the eyes, muscles, and kidneys. Okano and colleagues recently reported a case of recurrent episodes of rhabdomyolysis in a 2-year-old with COL4A1-related disorder raising fundamental questions on mechanisms of COL4A1/COL4A2 variants in muscle homeostasis. To date, rhabdomyolysis has not been associated with COL4A2-related disorder in humans. Rhabdomyolysis is a medical emergency, where there is elevated creatine kinase (CK) level in the blood and increased excretion of myoglobin in urine, due to skeletal muscle damage and release of intracytoplasmic proteins into systemic circulation. Rhabdomyolysis is a serious medical condition. It require intensive care management due to an increased risk of some life-threatening complications [including disseminated intravascular coagulation, renal failure, and severe hyperkalemia]. Herein, we report a case of rhabdomyolysis in an adult with COL4A2-related structural brain malformations (including polymicrogyria and heterotopia).

Advances in rhabdomyolysis: A review of pathogenesis, diagnosis, and treatment

Rhabdomyolysis (RM) is a multifactorial clinical syndrome characterized by the disintegration and necrosis of muscle tissue, leading to the release of cellular contents into the circulation. One of the most severe complications of RM is acute kidney injury, with a mortality rate of 20%-50%. Early and timely diagnosis is the key to improving the prognosis of patients with RM. The etiology of RM is complex and associated with various traumas, drugs, medications, and hereditary diseases, and the clinical symptoms are nonspecific. Therefore, its diagnosis highly relies on the doctor's experience and the level of medical equipment. However, RM often occurs in situations with limited medical resources, such as natural disasters, battlefields, and large-scale traffic accidents. In these scenarios, the varying levels of expertise among rescue personnel can lead to delays in diagnosis and treatment, thereby increasing the risk of mortality. This article provides a comprehensive review of the etiology, pathogenesis, complications, diagnostic, and treatment methods of RM. It also aims to offer new perspectives on the diagnosis and prognosis of RM by integrating machine learning and artificial intelligence. It is believed that this article can help pre-hospital rescuers and in-hospital doctors have a comprehensive understanding of RM to improve the patients' outcomes and overcome the challenges.

Predictive model for assessing the prognosis of rhabdomyolysis patients in the intensive care unit

Background

Rhabdomyolysis (RM) frequently gives rise to diverse complications, ultimately leading to an unfavorable prognosis for patients. Consequently, there is a pressing need for early prediction of survival rates among RM patients, yet reliable and effective predictive models are currently scarce.

Methods

All data utilized in this study were sourced from the MIMIC-IV database. A multivariable Cox regression analysis was conducted on the data, and the performance of the new model was evaluated based on the Harrell's concordance index (C-index) and the area under the receiver operating characteristic curve (AUC). Furthermore, the clinical utility of the predictive model was assessed through decision curve analysis (DCA).

Results

A total of 725 RM patients admitted to the intensive care unit (ICU) were included in the analysis, comprising 507 patients in the training cohort and 218 patients in the testing cohort. For the development of the predictive model, 37 variables were carefully selected. Multivariable Cox regression revealed that age, phosphate max, RR mean, and SOFA score were independent predictors of survival outcomes in RM patients. In the training cohort, the AUCs of the new model for predicting 28-day, 60-day, and 90-day survival rates were 0.818 (95% CI: 0.766-0.871), 0.810 (95% CI: 0.761-0.855), and 0.819 (95% CI: 0.773-0.864), respectively. In the validation cohort, the AUCs of the new model for predicting 28-day, 60-day, and 90-day survival rates were 0.840 (95% CI: 0.772-0.900), 0.842 (95% CI: 0.780-0.899), and 0.842 (95% CI: 0.779-0.897), respectively.

Conclusion

This study identified crucial demographic factors, vital signs, and laboratory parameters associated with RM patient prognosis and utilized them to develop a more accurate and convenient prognostic prediction model for assessing 28-day, 60-day, and 90-day survival rates.

Implications for clinical practice

This study specifically targets patients with RM admitted to ICU and presents a novel clinical prediction model that surpasses the conventional SOFA score. By integrating specific prognostic indicators tailored to RM, the model significantly enhances prediction accuracy, thereby enabling a more targeted and effective approach to managing RM patients.

Interpretable machine learning model for early prediction of 28-day mortality in ICU patients with sepsis-induced coagulopathy: development and validation

OBJECTIVE

Sepsis-induced coagulopathy (SIC) is extremely common in individuals with sepsis, significantly associated with poor outcomes. This study attempted to develop an interpretable and generalizable machine learning (ML) model for early predicting the risk of 28-day death in patients with SIC.

METHODS

In this retrospective cohort study, we extracted SIC patients from the Medical Information Mart for Intensive Care III (MIMIC-III), MIMIC-IV, and eICU-CRD database according to Toshiaki Iba's scale. And the overlapping in the MIMIC-IV was excluded for this study. Afterward, only the MIMIC-III cohort was randomly divided into the training set, and the internal validation set according to the ratio of 7:3, while the MIMIC-IV and eICU-CRD databases were considered the external validation sets. The predictive factors for 28-day mortality of SIC patients were determined using recursive feature elimination combined with tenfold cross-validation (RFECV). Then, we constructed models using ML algorithms. Multiple metrics were used for evaluation of performance of the models, including the area under the receiver operating characteristic curve (AUROC), area under the precision recall curve (AUPRC), accuracy, sensitivity, specificity, negative predictive value, positive predictive value, recall, and F1 score. Finally, Shapley Additive Explanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME) were employed to provide a reasonable interpretation for the prediction results.

RESULTS

A total of 3280, 2798, and 1668 SIC patients were screened from MIMIC-III, MIMIC-IV, and eICU-CRD databases, respectively. Seventeen features were selected to construct ML prediction models. XGBoost had the best performance in predicting the 28-day mortality of SIC patients, with AUC of 0.828, 0.913 and 0.923, the AUPRC of 0.807, 0.796 and 0.921, the accuracy of 0.785, 0.885 and 0.891, the F1 scores were 0.63, 0.69 and 0.70 in MIMIC-III (internal validation set), MIMIC-IV, and eICU-CRD databases. The importance ranking and SHAP analyses showed that initial SOFA score, red blood cell distribution width (RDW), and age were the top three critical features in the XGBoost model.

CONCLUSIONS

We developed an optimal and explainable ML model to predict the risk of 28-day death of SIC patients 28-day death risk. Compared with conventional scoring systems, the XGBoost model performed better. The model established will have the potential to improve the level of clinical practice for SIC patients.

MACHINE LEARNING FOR PREDICTING HEMODYNAMIC DETERIORATION OF PATIENTS WITH INTERMEDIATE-RISK PULMONARY EMBOLISM IN INTENSIVE CARE UNIT

ABSTRACT

Background: Intermediate-risk pulmonary embolism (PE) patients in the intensive care unit (ICU) are at a higher risk of hemodynamic deterioration than those in the general ward. This study aimed to construct a machine learning (ML) model to accurately identify the tendency for hemodynamic deterioration in the ICU patients with intermediate-risk PE. Method: A total of 704 intermediate-risk PE patients from the MIMIC-IV database were retrospectively collected. The primary outcome was defined as hemodynamic deterioration occurring within 30 days after admission to ICU. Four ML algorithms were used to construct models on the basis of all variables from MIMIC IV database with missing values less than 20%. The extreme gradient boosting (XGBoost) model was further simplified for clinical application. The performance of the ML models was evaluated by using the receiver operating characteristic curve, calibration plots, and decision curve analysis. Predictive performance of simplified XGBoost was compared with the simplified Pulmonary Embolism Severity Index score. SHapley Additive explanation (SHAP) was performed on a simplified XGBoost model to calculate the contribution and impact of each feature on the predicted outcome and presents it visually. Results: Among the 704 intermediate-risk PE patients included in this study, 120 patients experienced hemodynamic deterioration within 30 days after admission to the ICU. Simplified XGBoost model demonstrated the best predictive performance with an area under the curve of 0.866 (95% confidence interval, 0.800-0.925), and after recalibrated by isotonic regression, the area under the curve improved to 0.885 (95% confidence interval, 0.822-0.935). Based on the simplified XGBoost model, a web app was developed to identify the tendency for hemodynamic deterioration in ICU patients with intermediate-risk PE. Conclusion: A simplified XGBoost model can accurately predict the occurrence of hemodynamic deterioration for intermediate-risk PE patients in the ICU, assisting clinical workers in providing more personalized management for PE patients in the ICU.

A Simple Nomogram for Predicting Hospital Mortality of Patients Over 80 Years in ICU: An International Multicenter Retrospective Study

OBJECTIVES

This study aimed to develop and validate an easy-to-use intensive care unit (ICU) illness scoring system to evaluate the in-hospital mortality for very old patients (VOPs, over 80 years old).

METHODS

We performed a multicenter retrospective study based on the electronic ICU (eICU) Collaborative Research Database (eICU-CRD), Medical Information Mart for Intensive Care Database (MIMIC-III CareVue and MIMIC-IV), and the Amsterdam University Medical Centers Database (AmsterdamUMCdb). Least Absolute Shrinkage and Selection Operator regression was applied to variables selection. The logistic regression algorithm was used to develop the risk score and a nomogram was further generated to explain the score.

RESULTS

We analyzed 23 704 VOPs, including 3 726 deaths (10 183 [13.5% mortality] from eICU-CRD [development set], 12 703 [17.2%] from the MIMIC, and 818 [20.8%] from the AmsterdamUMC [external validation sets]). Thirty-four variables were extracted on the first day of ICU admission, and 10 variables were finally chosen including Glasgow Coma Scale, shock index, respiratory rate, partial pressure of carbon dioxide, lactate, mechanical ventilation (yes vs no), oxygen saturation, Charlson Comorbidity Index, blood urea nitrogen, and urine output. The nomogram was developed based on the 10 variables (area under the receiver operating characteristic curve: training of 0.792, testing of 0.788, MIMIC of 0.764, and AmsterdamUMC of 0.808 [external validating]), which consistently outperformed the Sequential Organ Failure Assessment, acute physiology score III, and simplified acute physiology score II.

CONCLUSIONS

We developed and externally validated a nomogram for predicting mortality in VOPs based on 10 commonly measured variables on the first day of ICU admission. It could be a useful tool for clinicians to identify potentially high risks of VOPs.

Development and validation of a machine learning-based predictive model for secondary post-tonsillectomy hemorrhage

Background

The main obstacle to a patient's recovery following a tonsillectomy is complications, and bleeding is the most frequent culprit. Predicting post-tonsillectomy hemorrhage (PTH) allows for accurate identification of high-risk populations and the implementation of protective measures. Our study aimed to investigate how well machine learning models predict the risk of PTH.

Methods

Data were obtained from 520 patients who underwent a tonsillectomy at The 940th Hospital of Joint Logistics Support Force of Chinese People's Liberation Army. The age range of the patients was 2-57 years, and 364 (70%) were male. The prediction models were developed using five machine learning models: decision tree, support vector machine (SVM), extreme gradient boosting (XGBoost), random forest, and logistic regression. The performance of the models was evaluated using the area under the receiver operating characteristic curve (AUC). Shapley additive explanation (SHAP) was used to interpret the results of the best-performing model.

Results

The frequency of PTH was 11.54% among the 520 patients, with 10.71% in the training group and 13.46% in the validation set. Age, BMI, season, smoking, blood type, INR, combined secretory otitis media, combined adenoidectomy, surgical wound, and use of glucocorticoids were selected by mutual information (MI) method. The XGBoost model had best AUC (0.812) and Brier score (0.152). Decision curve analysis (DCA) showed that the model had a high clinical utility. The SHAP method revealed the top 10 variables of MI according to the importance ranking, and the average of the age was recognized as the most important predictor variable.

Conclusion

This study built a PTH risk prediction model using machine learning. The XGBoost model is a tool with potential to facilitate population management strategies for PTH.

Neural networks based on attention architecture are robust to data missingness for early predicting hospital mortality in intensive care unit patients

Background

Although the machine learning model developed on electronic health records has become a promising method for early predicting hospital mortality, few studies focus on the approaches for handling missing data in electronic health records and evaluate model robustness to data missingness. This study proposes an attention architecture that shows excellent predictive performance and is robust to data missingness.

Methods

Two public intensive care unit databases were used for model training and external validation, respectively. Three neural networks (masked attention model, attention model with imputation, attention model with missing indicator) based on the attention architecture were developed, using masked attention mechanism, multiple imputation, and missing indicator to handle missing data, respectively. Model interpretability was analyzed by attention allocations. Extreme gradient boosting, logistic regression with multiple imputation and missing indicator (logistic regression with imputation, logistic regression with missing indicator) were used as baseline models. Model discrimination and calibration were evaluated by area under the receiver operating characteristic curve, area under precision-recall curve, and calibration curve. In addition, model robustness to data missingness in both model training and validation was evaluated by three analyses.

Results

In total, 65,623 and 150,753 intensive care unit stays were respectively included in the training set and the test set, with mortality of 10.1% and 8.5%, and overall missing rate of 10.3% and 19.7%. attention model with missing indicator had the highest area under the receiver operating characteristic curve (0.869; 95% CI: 0.865 to 0.873) in external validation; attention model with imputation had the highest area under precision-recall curve (0.497; 95% CI: 0.480-0.513). Masked attention model and attention model with imputation showed better calibration than other models. The three neural networks showed different patterns of attention allocation. In terms of robustness to data missingness, masked attention model and attention model with missing indicator are more robust to missing data in model training; while attention model with imputation is more robust to missing data in model validation.

Conclusions

The attention architecture has the potential to become an excellent model architecture for clinical prediction task with data missingness.

Development and Validation of Machine Learning Models for Real-Time Mortality Prediction in Critically Ill Patients With Sepsis-Associated Acute Kidney Injury

Background

Sepsis-associated acute kidney injury (SA-AKI) is common in critically ill patients, which is associated with significantly increased mortality. Existing mortality prediction tools showed insufficient predictive power or failed to reflect patients' dynamic clinical evolution. Therefore, the study aimed to develop and validate machine learning-based models for real-time mortality prediction in critically ill patients with SA-AKI.

Methods

The multi-center retrospective study included patients from two distinct databases. A total of 12,132 SA-AKI patients from the Medical Information Mart for Intensive Care IV (MIMIC-IV) were randomly allocated to the training, validation, and internal test sets. An additional 3,741 patients from the eICU Collaborative Research Database (eICU-CRD) served as an external test set. For every 12 h during the ICU stays, the state-of-the-art eXtreme Gradient Boosting (XGBoost) algorithm was used to predict the risk of in-hospital death in the following 48, 72, and 120 h and in the first 28 days after ICU admission. Area under the receiver operating characteristic curves (AUCs) were calculated to evaluate the models' performance.

Results

The XGBoost models, based on routine clinical variables updated every 12 h, showed better performance in mortality prediction than the SOFA score and SAPS-II. The AUCs of the XGBoost models for mortality over different time periods ranged from 0.848 to 0.804 in the internal test set and from 0.818 to 0.748 in the external test set. The shapley additive explanation method provided interpretability for the XGBoost models, which improved the understanding of the association between the predictor variables and future mortality.

Conclusions

The interpretable machine learning XGBoost models showed promising performance in real-time mortality prediction in critically ill patients with SA-AKI, which are useful tools for early identification of high-risk patients and timely clinical interventions.

Evolution of Modeled Cortisol Is Prognostic of Death in Hospitalized Patients With COVID-19 Syndrome

Introduction

Patients hospitalized with SARS-CoV-2 have an elevated risk of mortality related to a severe inflammatory response. We hypothesized that biological modeling with a complete blood count (CBC) would be predictive of mortality.

Method

In 2020, 81 patients were randomly selected from La Rochelle Hospital, France for a simple blinded retrospective study. Demographic, vital signs, CBC and CRP were obtained on admission, at days 2-3 and 3-5. From a CBC, two biological modeling indexes were resulted: the neutrophil-to-lymphocyte ratio (NLR) and cortisol index adjusted (CA).

Results

By ANOVA, in survivors vs. non-survivors there was statistical different at p < 0.01 for age (66.2 vs. 80), CRP (92 vs. 179 mg/dL, normal < 10), cortisol index adjusted (323 vs. 698, normal 3-7) and genito-thyroid indexes (7.5 vs. 18.2, normal 1.5–2.5), and at p = 0.02 creatinine (1.03 vs. 1.48, normal 0.73–1.8 mg/dL). By mixed model analysis, CA and NLR improved in those who survived across all three time points, but worsened again after 3–5 days in non-survivors. CRP continued to improve over time in survivors and non-survivors. Positive vs. Negative predictive value were: CRP (91.1%, 30.4%), NLR (94.5%, 22.7%), CA (100%, 0%).

Discussion

Cortisol modeling and the neutrophil-to-lymphocyte ratio were more accurate in describing the course of non-survivors than CRP.

Conclusion

In patients admitted for SARS CoV-2 infection, biological modeling with a CBC predicted risk of death better than CRP. This approach is inexpensive and easily repeated.

Construction and Interpretation of Prediction Model of Teicoplanin Trough Concentration via Machine Learning

Objective

To establish an optimal model to predict the teicoplanin trough concentrations by machine learning, and explain the feature importance in the prediction model using the SHapley Additive exPlanation (SHAP) method.

Methods

A retrospective study was performed on 279 therapeutic drug monitoring (TDM) measurements obtained from 192 patients who were treated with teicoplanin intravenously at the First Affiliated Hospital of Army Medical University from November 2017 to July 2021. This study included 27 variables, and the teicoplanin trough concentrations were considered as the target variable. The whole dataset was divided into a training group and testing group at the ratio of 8:2, and predictive performance was compared among six different algorithms. Algorithms with higher model performance (top 3) were selected to establish the ensemble prediction model and SHAP was employed to interpret the model.

Results

Three algorithms (SVR, GBRT, and RF) with high R² scores (0.676, 0.670, and 0.656, respectively) were selected to construct the ensemble model at the ratio of 6:3:1. The model with R² = 0.720, MAE = 3.628, MSE = 22.571, absolute accuracy of 83.93%, and relative accuracy of 60.71% was obtained, which performed better in model fitting and had better prediction accuracy than any single algorithm. The feature importance and direction of each variable were visually demonstrated by SHAP values, in which teicoplanin administration and renal function were the most important factors.

Conclusion

We firstly adopted a machine learning approach to predict the teicoplanin trough concentration, and interpreted the prediction model by the SHAP method, which is of great significance and value for the clinical medication guidance.