Demystifying the Black Box: The Importance of Interpretability of Predictive Models in Neurocritical Care

Machine Learning Identifies Variation in Timing of Palliative Care Consultations Among Traumatic Brain Injury Patients

Background and Objective Timely palliative care involvement offers demonstrable benefits for traumatic brain injury (TBI) patients; however, palliative care consultations (PCCs) are used inconsistently during TBI management. This study aimed to employ advanced machine learning techniques to elucidate the primary drivers of PCC timing variability for TBI patients. Methods Data on admission, hospital course, and outcomes were collected for a cohort of 232 TBI patients who received both PCCs and neurosurgical consultations during the same hospitalization. Principal Component Analysis (PCA) and K-means clustering were used to identify patient phenotypes, which were then compared using Kaplan-Meier analysis. An extreme gradient boosting model (XGBoost) was employed to determine drivers of PCC timing, with model interpretation performed using SHapley Additive exPlanations (SHAP). Results Cluster A (n = 86) consisted mainly of older (median [IQR] = 87 [78, 94] years), White females with mild TBIs and demonstrated the shortest time-to-PCC (2.5 [1.0, 7.0] days). Cluster B (n = 108) also sustained mild TBIs but comprised moderately younger (81 [75, 86] years) married White males with later PCC (5.0 [3.0, 10.8] days). Cluster C (n = 38) represented much younger (46.5 [29.5, 59.8] years), more severely injured, non-White patients with the latest PCC initiation (9.0 [4.2, 17.0] days). The clusters did not differ by discharge disposition (p = 0.4) or frequency inpatient mortality (p > 0.9); however, Kaplan-Meier analysis revealed a significant difference in the time from admission to PCC (p < 0.001), despite no differences in time from admission to mortality (p = 0.18). SHAP analysis of the XGBoost model identified age, sex, and race as the most influential drivers of PCC timing. Conclusions This study highlights crucial disparities in PCC timing for TBI patients and underscores the need for targeted strategies to ensure timely and equitable palliative care integration for this vulnerable population.

Machine learning vs. traditional regression analysis for fluid overload prediction in the ICU

Fluid overload, while common in the ICU and associated with serious sequelae, is hard to predict and may be influenced by ICU medication use. Machine learning (ML) approaches may offer advantages over traditional regression techniques to predict it. We compared the ability of traditional regression techniques and different ML-based modeling approaches to identify clinically meaningful fluid overload predictors. This was a retrospective, observational cohort study of adult patients admitted to an ICU ≥ 72 h between 10/1/2015 and 10/31/2020 with available fluid balance data. Models to predict fluid overload (a positive fluid balance ≥ 10% of the admission body weight) in the 48-72 h after ICU admission were created. Potential patient and medication fluid overload predictor variables (n = 28) were collected at either baseline or 24 h after ICU admission. The optimal traditional logistic regression model was created using backward selection. Supervised, classification-based ML models were trained and optimized, including a meta-modeling approach. Area under the receiver operating characteristic (AUROC), positive predictive value (PPV), and negative predictive value (NPV) were compared between the traditional and ML fluid prediction models. A total of 49 of the 391 (12.5%) patients developed fluid overload. Among the ML models, the XGBoost model had the highest performance (AUROC 0.78, PPV 0.27, NPV 0.94) for fluid overload prediction. The XGBoost model performed similarly to the final traditional logistic regression model (AUROC 0.70; PPV 0.20, NPV 0.94). Feature importance analysis revealed severity of illness scores and medication-related data were the most important predictors of fluid overload. In the context of our study, ML and traditional models appear to perform similarly to predict fluid overload in the ICU. Baseline severity of illness and ICU medication regimen complexity are important predictors of fluid overload.

Evaluation of nutritional status and clinical depression classification using an explainable machine learning method

Introduction

Depression is a prevalent disorder worldwide, with potentially severe implications. It contributes significantly to an increased risk of diseases associated with multiple risk factors. Early accurate diagnosis of depressive symptoms is a critical first step toward management, intervention, and prevention. Various nutritional and dietary compounds have been suggested to be involved in the onset, maintenance, and severity of depressive disorders. Despite the challenges to better understanding the association between nutritional risk factors and the occurrence of depression, assessing the interplay of these markers through supervised machine learning remains to be fully explored.

Methods

This study aimed to determine the ability of machine learning-based decision support methods to identify the presence of depression using publicly available health data from the Korean National Health and Nutrition Examination Survey. Two exploration techniques, namely, uniform manifold approximation and projection and Pearson correlation, were performed for explanatory analysis among datasets. A grid search optimization with cross-validation was performed to fine-tune the models for classifying depression with the highest accuracy. Several performance measures, including accuracy, precision, recall, F1 score, confusion matrix, areas under the precision-recall and receiver operating characteristic curves, and calibration plot, were used to compare classifier performances. We further investigated the importance of the features provided: visualized interpretation using ELI5, partial dependence plots, and local interpretable using model-agnostic explanations and Shapley additive explanation for the prediction at both the population and individual levels.

Results

The best model achieved an accuracy of 86.18% for XGBoost and an area under the curve of 84.96% for the random forest model in original dataset and the XGBoost algorithm with an accuracy of 86.02% and an area under the curve of 85.34% in the quantile-based dataset. The explainable results revealed a complementary observation of the relative changes in feature values, and, thus, the importance of emergent depression risks could be identified.

Discussion

The strength of our approach is the large sample size used for training with a fine-tuned model. The machine learning-based analysis showed that the hyper-tuned model has empirically higher accuracy in classifying patients with depressive disorder, as evidenced by the set of interpretable experiments, and can be an effective solution for disease control.