Predicting the Individual Treatment Effect of Neurosurgery for Patients with Traumatic Brain Injury in the Low-Resource Setting: A Machine Learning Approach in Uganda

Machine learning-based models to predict the need for neurosurgical intervention after moderate traumatic brain injury

Background and Aims

Traumatic brain injury (TBI) is a widespread global health issue with significant economic consequences. However, no existing model exists to predict the need for neurosurgical intervention in moderate TBI patients with positive initial computed tomography scans. This study determines the efficacy of machine learning (ML)-based models in predicting the need for neurosurgical intervention.

Methods

This is a retrospective study of patients admitted to the neuro-intensive care unit of Emtiaz Hospital, Shiraz, Iran, between January 2018 and December 2020. The most clinically important variables from patients that met our inclusion and exclusion criteria were collected and used as predictors. We developed models using multilayer perceptron, random forest, support vector machines (SVM), and logistic regression. To evaluate the models, their F1-score, sensitivity, specificity, and accuracy were assessed using a fourfold cross-validation method.

Results

Based on predictive models, SVM showed the highest performance in predicting the need for neurosurgical intervention, with an F1-score of 0.83, an area under curve of 0.93, sensitivity of 0.82, specificity of 0.84, a positive predictive value of 0.83, and a negative predictive value of 0.83.

Conclusion

The use of ML-based models as decision-making tools can be effective in predicting with high accuracy whether neurosurgery will be necessary after moderate TBIs. These models may ultimately be used as decision-support tools to evaluate early intervention in TBI patients.

Machine Learning for Predicting Discharge Disposition After Traumatic Brain Injury

BACKGROUND

Current traumatic brain injury (TBI) prognostic calculators are commonly used to predict the mortality and Glasgow Outcome Scale, but these outcomes are most relevant for severe TBI. Because mild and moderate TBI rarely reaches severe outcomes, there is a need for novel prognostic endpoints.

OBJECTIVE

To generate machine learning (ML) models with a strong predictive capacity for trichotomized discharge disposition, an outcome not previously used in TBI prognostic models. The outcome can serve as a proxy for patients' functional status, even in mild and moderate patients with TBI.

METHODS

Using a large data set (n = 5292) of patients with TBI from a quaternary care center and 84 predictors, including vitals, demographics, mechanism of injury, initial Glasgow Coma Scale, and comorbidities, we trained 6 different ML algorithms using a nested-stratified-cross-validation protocol. After optimizing hyperparameters and performing model selection, isotonic regression was applied to calibrate models.

RESULTS

When maximizing the microaveraged area under the receiver operating characteristic curve during hyperparameter optimization, a random forest model exhibited top performance. A random forest model was also selected when maximizing the microaveraged area under the precision-recall curve. For both models, the weighted average area under the receiver operating characteristic curves was 0.84 (95% CI 0.81-0.87) and the weighted average area under the precision-recall curves was 0.85 (95% CI 0.82-0.88).

CONCLUSION

Our group presents high-performing ML models to predict trichotomized discharge disposition. These models can assist in optimization of patient triage and treatment, especially in cases of mild and moderate TBI.

Machine Learning for Predicting In-Hospital Mortality After Traumatic Brain Injury in Both High-Income and Low- and Middle-Income Countries

BACKGROUND

Machine learning (ML) holds promise as a tool to guide clinical decision making by predicting in-hospital mortality for patients with traumatic brain injury (TBI). Previous models such as the international mission for prognosis and clinical trials in TBI (IMPACT) and the corticosteroid randomization after significant head injury (CRASH) prognosis calculators can potentially be improved with expanded clinical features and newer ML approaches.

OBJECTIVE

To develop ML models to predict in-hospital mortality for both the high-income country (HIC) and the low- and middle-income country (LMIC) settings.

METHODS

We used the Duke University Medical Center National Trauma Data Bank and Mulago National Referral Hospital (MNRH) registry to predict in-hospital mortality for the HIC and LMIC settings, respectively. Six ML models were built on each data set, and the best model was chosen through nested cross-validation. The CRASH and IMPACT models were externally validated on the MNRH database.

RESULTS

ML models built on National Trauma Data Bank (n = 5393, 84 predictors) demonstrated an area under the receiver operating curve (AUROC) of 0.91 (95% CI: 0.85-0.97) while models constructed on MNRH (n = 877, 31 predictors) demonstrated an AUROC of 0.89 (95% CI: 0.81-0.97). Direct comparison with CRASH and IMPACT models showed significant improvement of the proposed LMIC models regarding AUROC (P = .038).

CONCLUSION

We developed high-performing well-calibrated ML models for predicting in-hospital mortality for both the HIC and LMIC settings that have the potential to influence clinical management and traumatic brain injury patient trajectories.

Machine Learning to Predict Successful Opioid Dose Reduction or Stabilization After Spinal Cord Stimulation

BACKGROUND

Spinal cord stimulation (SCS) effectively reduces opioid usage in some patients, but preoperatively, there is no objective measure to predict who will most benefit.

OBJECTIVE

To predict successful reduction or stabilization of opioid usage after SCS using machine learning models we developed and to assess if deep learning provides a significant benefit over logistic regression (LR).

METHODS

We used the IBM MarketScan national databases to identify patients undergoing SCS from 2010 to 2015. Our models predict surgical success as defined by opioid dose stability or reduction 1 year after SCS. We incorporated 30 predictors, primarily regarding medication patterns and comorbidities. Two machine learning algorithms were applied: LR with recursive feature elimination and deep neural networks (DNNs). To compare model performances, we used nested 5-fold cross-validation to calculate area under the receiver operating characteristic curve (AUROC).

RESULTS

The final cohort included 7022 patients, of whom 66.9% had successful surgery. Our 5-variable LR performed comparably with the full 30-variable version (AUROC difference <0.01). The DNN and 5-variable LR models demonstrated similar AUROCs of 0.740 (95% CI, 0.727-0.753) and 0.737 (95% CI, 0.728-0.746) (P = .25), respectively. The simplified model can be accessed at SurgicalML.com.

CONCLUSION

We present the first machine learning-based models for predicting reduction or stabilization of opioid usage after SCS. The DNN and 5-variable LR models demonstrated comparable performances, with the latter revealing significant associations with patients' pre-SCS pharmacologic patterns. This simplified, interpretable LR model may augment patient and surgeon decision making regarding SCS.

Deep Learning to Predict Traumatic Brain Injury Outcomes in the Low-Resource Setting

OBJECTIVE

Traumatic brain injury (TBI) disproportionately affects low- and middle-income countries (LMICs). In these settings, accurate patient prognostication is both difficult and essential for high-quality patient care. With the ultimate goal of enhancing TBI triage in LMICs, we aim to develop the first deep learning model to predict outcomes after TBI and compare its performance to that of less complex algorithms.

METHODS

TBI patients' data were prospectively collected in Kampala, Uganda, from 2016 to 2020. To predict good versus poor outcome at hospital discharge, we created deep neural network (DNN), shallow neural network (SNN), and elastic-net regularized logistic regression (LRnet) models. Predictors included 13 easily acquirable clinical variables. We assessed model performance with 5-fold cross-validation to calculate areas under both the receiver operating characteristic curve (AUROC) and precision-recall curve (AUPRC), in addition to standardized partial AUPRC (spAUPRC) to focus on comparisons at clinically-relevant operating points.

RESULTS

We included 2164 patients for model training, of which 12% had poor outcomes. The DNN performed best as measured by AUROC (0.941) and spAUPRC in region maximizing recall (0.291), whereas the SNN was best by AUPRC (0.770). In several other comparisons, the LRnet was non-inferior to the neural networks.

CONCLUSION

We present the first use of deep learning for TBI prognostication, with an emphasis on LMICs, where there is great need for decision support to allocate limited resources. Optimal algorithm selection depends on the specific clinical setting; deep learning is not a panacea, though it may have a role in these efforts.

Comparison of intracranial injury predictability between machine learning algorithms and the nomogram in pediatric traumatic brain injury

OBJECTIVE

The overuse of head CT examinations has been much discussed, especially those for minor traumatic brain injury (TBI). In the disruptive era, machine learning (ML) is one of the prediction tools that has been used and applied in various fields of neurosurgery. The objective of this study was to compare the predictive performance between ML and a nomogram, which is the other prediction tool for intracranial injury following cranial CT in children with TBI.

METHODS

Data from 964 pediatric patients with TBI were randomly divided into a training data set (75%) for hyperparameter tuning and supervised learning from 14 clinical parameters, while the remaining data (25%) were used for validation purposes. Moreover, a nomogram was developed from the training data set with similar parameters. Therefore, models from various ML algorithms and the nomogram were built and deployed via web-based application.

RESULTS

A random forest classifier (RFC) algorithm established the best performance for predicting intracranial injury following cranial CT of the brain. The area under the receiver operating characteristic curve for the performance of RFC algorithms was 0.80, with 0.34 sensitivity, 0.95 specificity, 0.73 positive predictive value, 0.80 negative predictive value, and 0.79 accuracy.

CONCLUSIONS

The ML algorithms, particularly the RFC, indicated relatively excellent predictive performance that would have the ability to support physicians in balancing the overuse of head CT scans and reducing the treatment costs of pediatric TBI in general practice.

Artificial intelligence-enhanced intraoperative neurosurgical workflow: state of the art and future perspectives

BACKGROUND

Artificial Intelligence (AI) and Machine Learning (ML) augment decision-making processes and productivity by supporting surgeons over a range of clinical activities: from diagnosis and preoperative planning to intraoperative surgical assistance. We reviewed the literature to identify current AI platforms applied to neurosurgical perioperative and intraoperative settings and describe their role in multiple subspecialties.

METHODS

A systematic review of the literature was conducted following the PRISMA guidelines. PubMed, EMBASE, and Scopus databases were searched from inception to December 31, 2020. Original articles were included if they: presented AI platforms implemented in perioperative, intraoperative settings and reported ML models' performance metrics. Due to the heterogeneity in neurosurgical applications, a qualitative synthesis was deemed appropriate. The risk of bias and applicability of predicted outcomes were assessed using the PROBAST tool.

RESULTS

41 articles were included. All studies evaluated a supervised learning algorithm. A total of 10 ML models were described; the most frequent were neural networks (n = 15) and tree-based models (n = 13). Overall, the risk of bias was medium-high, but applicability was considered positive for all studies. Articles were grouped into 4 categories according to the subspecialty of interest: neuro-oncology, spine, functional and other. For each category, different prediction tasks were identified.

CONCLUSIONS

In this review, we summarize the state-of-art applications of AI for the intraoperative augmentation of neurosurgical workflows across multiple subspecialties. ML models may boost surgical team performances by reducing human errors and providing patient-tailored surgical plans, but further and higher-quality studies need to be conducted.

Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study

BACKGROUND

Chest x-rays are widely used in clinical practice; however, interpretation can be hindered by human error and a lack of experienced thoracic radiologists. Deep learning has the potential to improve the accuracy of chest x-ray interpretation. We therefore aimed to assess the accuracy of radiologists with and without the assistance of a deep-learning model.

METHODS

In this retrospective study, a deep-learning model was trained on 821 681 images (284 649 patients) from five data sets from Australia, Europe, and the USA. 2568 enriched chest x-ray cases from adult patients (≥16 years) who had at least one frontal chest x-ray were included in the test dataset; cases were representative of inpatient, outpatient, and emergency settings. 20 radiologists reviewed cases with and without the assistance of the deep-learning model with a 3-month washout period. We assessed the change in accuracy of chest x-ray interpretation across 127 clinical findings when the deep-learning model was used as a decision support by calculating area under the receiver operating characteristic curve (AUC) for each radiologist with and without the deep-learning model. We also compared AUCs for the model alone with those of unassisted radiologists. If the lower bound of the adjusted 95% CI of the difference in AUC between the model and the unassisted radiologists was more than -0·05, the model was considered to be non-inferior for that finding. If the lower bound exceeded 0, the model was considered to be superior.

FINDINGS

Unassisted radiologists had a macroaveraged AUC of 0·713 (95% CI 0·645-0·785) across the 127 clinical findings, compared with 0·808 (0·763-0·839) when assisted by the model. The deep-learning model statistically significantly improved the classification accuracy of radiologists for 102 (80%) of 127 clinical findings, was statistically non-inferior for 19 (15%) findings, and no findings showed a decrease in accuracy when radiologists used the deep-learning model. Unassisted radiologists had a macroaveraged mean AUC of 0·713 (0·645-0·785) across all findings, compared with 0·957 (0·954-0·959) for the model alone. Model classification alone was significantly more accurate than unassisted radiologists for 117 (94%) of 124 clinical findings predicted by the model and was non-inferior to unassisted radiologists for all other clinical findings.

INTERPRETATION

This study shows the potential of a comprehensive deep-learning model to improve chest x-ray interpretation across a large breadth of clinical practice.

FUNDING

Annalise.ai.

Corticosteroid Randomization after Significant Head Injury and International Mission for Prognosis and Clinical Trialsin Traumatic Brain Injury Models Compared with a Machine Learning-Based Predictive Model from Tanzania

Hospitals in low- and middle-income countries (LMICs) could benefit from decision support technologies to reduce time to triage, diagnosis, and surgery for patients with traumatic brain injury (TBI). Corticosteroid Randomization after Significant Head Injury (CRASH) and International Mission for Prognosis and Clinical Trials in Traumatic Brain Injury (IMPACT) are robust examples of TBI prognostic models, although they have yet to be validated in Sub-Saharan Africa (SSA). Moreover, machine learning and improved data quality in LMICs provide an opportunity to develop context-specific, and potentially more accurate, prognostic models. We aim to externally validate CRASH and IMPACT on our TBI registry and compare their performances to that of the locally derived model (from the Kilimanjaro Christian Medical Center [KCMC]). We developed a machine learning-based prognostic model from a TBI registry collected at a regional referral hospital in Moshi, Tanzania. We also used the core CRASH and IMPACT online risk calculators to generate risk scores for each patient. We compared the discrimination (area under the curve [AUC]) and calibration before and after Platt scaling (Brier, Hosmer-Lemeshow Test, and calibration plots) for CRASH, IMPACT, and the KCMC model. The outcome of interest was unfavorable in-hospital outcome defined as a Glasgow Outcome Scale score of 1-3. There were 2972 patients included in the TBI registry, of whom 11% had an unfavorable outcome. The AUCs for the KCMC model, CRASH, and IMPACT were 0.919, 0.876, and 0.821, respectively. Prior to Platt scaling, CRASH was the best calibrated model (χ² = 68.1) followed by IMPACT (χ² = 380.9) and KCMC (χ² = 1025.6). We provide the first SSA validation of the core CRASH and IMPACT models. The KCMC model had better discrimination than either of these. CRASH had the best calibration, although all model predictions could be successfully calibrated. The top performing models, KCMC and CRASH, were both developed using LMIC data, suggesting that locally derived models may outperform imported ones from different contexts of care. Further work is needed to externally validate the KCMC model.