Demystifying the Black Box: The Importance of Interpretability of Predictive Models in Neurocritical Care

Prediction of the functional outcome of intensive inpatient rehabilitation after stroke using machine learning methods

An accurate and reliable functional prognosis is vital to stroke patients addressing rehabilitation, to their families, and healthcare providers. This study aimed at developing and validating externally patient-wise prognostic models of the global functional outcome at discharge from intensive inpatient post-acute rehabilitation after stroke, based on a standardized comprehensive multidimensional assessment performed at admission to rehabilitation. Patients addressing intensive inpatient rehabilitation pathways within 30 days from stroke were prospectively enrolled in two consecutive multisite studies. Demographics, description of the event, clinical/functional, and psycho-social data were collected. The outcome of interest was disability in basic daily living activities at discharge, measured by the modified Barthel Index (mBI). Machine learning-based prognostic models were developed, internally cross-validated, and externally validated. Interpretability techniques were applied for the analysis of predictors. 385 patients were considered, 220 (165) for training (external test) sets. A 50.9% (55.8%) of women, 79.5% (80.0%) of ischemic, and a median [interquartile range- IQR] age of 80.0[15.0] (79.0[17.0]) were registered. The Support Vector Machine obtained the best validation performances and a median absolute error [IQR] on discharge mBI estimation of 11.5[15.0] and 9.2[13.0] points on the internal and external testing, respectively. The baseline variables providing the main contributions to the predictions were mBI, motor upper-limb score, age, and cognitive screening score. We achieved a solution to support the formulation of a functional prognosis at intensive rehabilitation admission. The interpretability analysis confirms the relevance of easily collected motor and cognitive dataat admission and of the patient's age.Trial registration: Prospectively registered on ClinicalTrials.gov (registration numbers RIPS NCT03866057, STRATEGY NCT05389878).

Machine Learning Reveals Demographic Disparities in Palliative Care Timing Among Patients With Traumatic Brain Injury Receiving Neurosurgical Consultation

BACKGROUND

Timely palliative care (PC) consultations offer demonstrable benefits for patients with traumatic brain injury (TBI), yet their implementation remains inconsistent. This study employs machine learning methods to identify distinct patient phenotypes and elucidate the primary drivers of PC consultation timing variability in TBI management, aiming to uncover disparities and inform more equitable care strategies.

METHODS

Data on admission, hospital course, and outcomes were collected for a cohort of 232 patients with TBI who received both PC consultations and neurosurgical consultations during the same hospitalization. Patient phenotypes were uncovered using principal component analysis and K-means clustering; time-to-PC consultation for each phenotype was subsequently compared by Kaplan-Meier analysis. An extreme gradient boosting model with Shapley Additive Explanations identified key factors influencing PC consultation timing.

RESULTS

Three distinct patient clusters emerged: cluster A (n = 86), comprising older adult White women (median 87 years) with mild TBI, received the earliest PC consultations (median 2.5 days); cluster B (n = 108), older adult White men (median 81 years) with mild TBI, experienced delayed PC consultations (median 5.0 days); and cluster C (n = 38), middle-aged (median: 46.5 years), severely injured, non-White patients, had the latest PC consultations (median 9.0 days). The clusters did not differ by discharge disposition (p = 0.4) or inpatient mortality (p > 0.9); however, Kaplan-Meier analysis revealed a significant difference in time-to-PC consultation (p < 0.001), despite no differences in time-to-mortality (p = 0.18). Shapley Additive Explanations analysis of the extreme gradient boosting model identified age, sex, and race as the most influential drivers of PC consultation timing.

CONCLUSIONS

This study unveils crucial disparities in PC consultation timing for patients with TBI, primarily driven by demographic factors rather than clinical presentation or injury characteristics. The identification of distinct patient phenotypes and quantification of factors influencing PC consultation timing provide a foundation for developing for standardized protocols and decision support tools to ensure timely and equitable palliative care access for patients with TBI.

Multiple imputation integrated to machine learning: predicting post-stroke recovery of ambulation after intensive inpatient rehabilitation

Good data quality is vital for personalising plans in rehabilitation. Machine learning (ML) improves prognostics but integrating it with Multiple Imputation (MImp) for dealing missingness is an unexplored field. This work aims to provide post-stroke ambulation prognosis, integrating MImp with ML, and identify the prognostic influential factors. Stroke survivors in intensive rehabilitation were enrolled. Data on demographics, events, clinical, physiotherapy, and psycho-social assessment were collected. An independent ambulation at discharge, using the Functional Ambulation Category scale, was the outcome. After handling missingness using MImp, ML models were optimised, cross-validated, and tested. Interpretability techniques analysed predictor contributions. Pre-MImp, the dataset included 54.1% women, 79.2% ischaemic patients, median age 80.0 (interquartile range: 15.0). Post-MImp, 368 non-ambulatory patients on 10 imputed datasets were used for training, 80 for testing. The random forest (the validation best-performing algorithm) obtained 75.5% aggregated balanced accuracy on the test set. The main predictors included modified Barthel index, Fugl-Meyer assessment/motricity index, short physical performance battery, age, Charlson comorbidity index/cumulative illness rating scale, and trunk control test. This is among the first studies applying ML, together with MImp, to predict ambulation recovery in post-stroke rehabilitation. This pipeline reliably exploits the potential of incomplete datasets for healthcare prognosis, identifying relevant predictors.

International e-Delphi survey to define best practice in the reporting of intracranial pressure monitoring recording data

Introduction

Intracranial pressure (ICP) monitoring is a very commonly performed neurosurgical procedure but there is a wide variation in how it is reported, hindering analysis of it. The current study sought to generate consensus on the reporting of ICP monitoring recording data.

Research question

"What should be included in an ICP monitoring report?"

Material and methods

The exercise was completed via a modified eDelphi survey. An expert panel discussion was held from which themes were identified and used to produce a code to annotate the transcript of the discussion. Statements were generated for a further two rounds of electronic questionnaires distributed via the REDcap platform. A Likert scale was used to grade agreement with each statement in the survey. A statement was accepted if more than 70% agreement was achieved between respondents. Data was collated using Microsoft Excel and analysed using R.

Results

149 relevant statements were identified from the transcript and categorised into recording parameters, waveform characteristics or reporting. A total of 22 statements were generated for the first round of the survey which was answered by 39 respondents. Following the electronic round of surveys consensus was achieved for all but one statement regarding the acceptability of automating ICP reporting. This was put forward to a second round after which 79% agreement was reached.

Discussion and conclusion

The themes and statements from this eDelphi can be used as a framework to allow the standardisation of the reporting of intracranial pressure monitoring data.

From bed to bench and back again: Challenges facing deployment of intracranial pressure data analysis in clinical environments

Introduction

Numerous complex physiological models derived from intracranial pressure (ICP) monitoring data have been developed. More recently, techniques such as machine learning are being used to develop increasingly sophisticated models to aid in clinical decision-making tasks such as diagnosis and prediction. Whilst their potential clinical impact may be significant, few models based on ICP data are routinely available at a patient's bedside. Further, the ability to refine models using ongoing patient data collection is rare. In this paper we identify and discuss the challenges faced when converting insight from ICP data analysis into deployable tools at the patient bedside.

Research question

To provide an overview of challenges facing implementation of sophisticated ICP models and analyses at the patient bedside.

Material and methods

A narrative review of the barriers facing implementation of sophisticated ICP models and analyses at the patient bedside in a neurocritical care unit combined with a descriptive case study (the CHART-ADAPT project) on the topic.

Results

Key barriers found were technical, analytical, and integrity related. Examples included: lack of interoperability of medical devices for data collection and/or model deployment; inadequate infrastructure, hindering analysis of large volumes of high frequency patient data; a lack of clinical confidence in a model; and ethical, trust, security and patient confidentiality considerations governing the secondary use of patient data.

Discussion and conclusion

To realise the benefits of ICP data analysis, the results need to be promptly delivered and meaningfully communicated. Multiple barriers to implementation remain and solutions which address real-world challenges are required.

Machine Learning Identifies Variation in Timing of Palliative Care Consultations Among Traumatic Brain Injury Patients

Background and Objective Timely palliative care involvement offers demonstrable benefits for traumatic brain injury (TBI) patients; however, palliative care consultations (PCCs) are used inconsistently during TBI management. This study aimed to employ advanced machine learning techniques to elucidate the primary drivers of PCC timing variability for TBI patients. Methods Data on admission, hospital course, and outcomes were collected for a cohort of 232 TBI patients who received both PCCs and neurosurgical consultations during the same hospitalization. Principal Component Analysis (PCA) and K-means clustering were used to identify patient phenotypes, which were then compared using Kaplan-Meier analysis. An extreme gradient boosting model (XGBoost) was employed to determine drivers of PCC timing, with model interpretation performed using SHapley Additive exPlanations (SHAP). Results Cluster A (n = 86) consisted mainly of older (median [IQR] = 87 [78, 94] years), White females with mild TBIs and demonstrated the shortest time-to-PCC (2.5 [1.0, 7.0] days). Cluster B (n = 108) also sustained mild TBIs but comprised moderately younger (81 [75, 86] years) married White males with later PCC (5.0 [3.0, 10.8] days). Cluster C (n = 38) represented much younger (46.5 [29.5, 59.8] years), more severely injured, non-White patients with the latest PCC initiation (9.0 [4.2, 17.0] days). The clusters did not differ by discharge disposition (p = 0.4) or frequency inpatient mortality (p > 0.9); however, Kaplan-Meier analysis revealed a significant difference in the time from admission to PCC (p < 0.001), despite no differences in time from admission to mortality (p = 0.18). SHAP analysis of the XGBoost model identified age, sex, and race as the most influential drivers of PCC timing. Conclusions This study highlights crucial disparities in PCC timing for TBI patients and underscores the need for targeted strategies to ensure timely and equitable palliative care integration for this vulnerable population.

Accelerated and Interpretable Oblique Random Survival Forests

The oblique random survival forest (RSF) is an ensemble supervised learning method for right-censored outcomes. Trees in the oblique RSF are grown using linear combinations of predictors, whereas in the standard RSF, a single predictor is used. Oblique RSF ensembles have high prediction accuracy, but assessing many linear combinations of predictors induces high computational overhead. In addition, few methods have been developed for estimation of variable importance (VI) with oblique RSFs. We introduce a method to increase computational efficiency of the oblique RSF and a method to estimate VI with the oblique RSF. Our computational approach uses Newton-Raphson scoring in each non-leaf node, We estimate VI by negating each coefficient used for a given predictor in linear combinations, and then computing the reduction in out-of-bag accuracy. In benchmarking experiments, we find our implementation of the oblique RSF is hundreds of times faster, with equivalent prediction accuracy, compared to existing software for oblique RSFs. We find in simulation studies that "negation VI" discriminates between relevant and irrelevant numeric predictors more accurately than permutation VI, Shapley VI, and a technique to measure VI using analysis of variance. All oblique RSF methods in the current study are available in the aorsf R package, and additional supplemental materials are available online.

Machine learning vs. traditional regression analysis for fluid overload prediction in the ICU

Fluid overload, while common in the ICU and associated with serious sequelae, is hard to predict and may be influenced by ICU medication use. Machine learning (ML) approaches may offer advantages over traditional regression techniques to predict it. We compared the ability of traditional regression techniques and different ML-based modeling approaches to identify clinically meaningful fluid overload predictors. This was a retrospective, observational cohort study of adult patients admitted to an ICU ≥ 72 h between 10/1/2015 and 10/31/2020 with available fluid balance data. Models to predict fluid overload (a positive fluid balance ≥ 10% of the admission body weight) in the 48-72 h after ICU admission were created. Potential patient and medication fluid overload predictor variables (n = 28) were collected at either baseline or 24 h after ICU admission. The optimal traditional logistic regression model was created using backward selection. Supervised, classification-based ML models were trained and optimized, including a meta-modeling approach. Area under the receiver operating characteristic (AUROC), positive predictive value (PPV), and negative predictive value (NPV) were compared between the traditional and ML fluid prediction models. A total of 49 of the 391 (12.5%) patients developed fluid overload. Among the ML models, the XGBoost model had the highest performance (AUROC 0.78, PPV 0.27, NPV 0.94) for fluid overload prediction. The XGBoost model performed similarly to the final traditional logistic regression model (AUROC 0.70; PPV 0.20, NPV 0.94). Feature importance analysis revealed severity of illness scores and medication-related data were the most important predictors of fluid overload. In the context of our study, ML and traditional models appear to perform similarly to predict fluid overload in the ICU. Baseline severity of illness and ICU medication regimen complexity are important predictors of fluid overload.

Evaluation of nutritional status and clinical depression classification using an explainable machine learning method

Introduction

Depression is a prevalent disorder worldwide, with potentially severe implications. It contributes significantly to an increased risk of diseases associated with multiple risk factors. Early accurate diagnosis of depressive symptoms is a critical first step toward management, intervention, and prevention. Various nutritional and dietary compounds have been suggested to be involved in the onset, maintenance, and severity of depressive disorders. Despite the challenges to better understanding the association between nutritional risk factors and the occurrence of depression, assessing the interplay of these markers through supervised machine learning remains to be fully explored.

Methods

This study aimed to determine the ability of machine learning-based decision support methods to identify the presence of depression using publicly available health data from the Korean National Health and Nutrition Examination Survey. Two exploration techniques, namely, uniform manifold approximation and projection and Pearson correlation, were performed for explanatory analysis among datasets. A grid search optimization with cross-validation was performed to fine-tune the models for classifying depression with the highest accuracy. Several performance measures, including accuracy, precision, recall, F1 score, confusion matrix, areas under the precision-recall and receiver operating characteristic curves, and calibration plot, were used to compare classifier performances. We further investigated the importance of the features provided: visualized interpretation using ELI5, partial dependence plots, and local interpretable using model-agnostic explanations and Shapley additive explanation for the prediction at both the population and individual levels.

Results

The best model achieved an accuracy of 86.18% for XGBoost and an area under the curve of 84.96% for the random forest model in original dataset and the XGBoost algorithm with an accuracy of 86.02% and an area under the curve of 85.34% in the quantile-based dataset. The explainable results revealed a complementary observation of the relative changes in feature values, and, thus, the importance of emergent depression risks could be identified.

Discussion

The strength of our approach is the large sample size used for training with a fine-tuned model. The machine learning-based analysis showed that the hyper-tuned model has empirically higher accuracy in classifying patients with depressive disorder, as evidenced by the set of interpretable experiments, and can be an effective solution for disease control.