Machine learning model prediction of 6-month functional outcome in elderly patients with intracerebral hemorrhage

Diagnostic test accuracy of machine learning algorithms for the detection intracranial hemorrhage: a systematic review and meta-analysis study

BACKGROUND

This systematic review and meta-analysis were conducted to objectively evaluate the evidence of machine learning (ML) in the patient diagnosis of Intracranial Hemorrhage (ICH) on computed tomography (CT) scans.

METHODS

Until May 2023, systematic searches were conducted in ISI Web of Science, PubMed, Scopus, Cochrane Library, IEEE Xplore Digital Library, CINAHL, Science Direct, PROSPERO, and EMBASE for studies that evaluated the diagnostic precision of ML model-assisted ICH detection. Patients with and without ICH as the target condition who were receiving CT-Scan were eligible for the research, which used ML algorithms based on radiologists' reports as the gold reference standard. For meta-analysis, pooled sensitivities, specificities, and a summary receiver operating characteristics curve (SROC) were used.

RESULTS

At last, after screening the title, abstract, and full paper, twenty-six retrospective and three prospective, and two retrospective/prospective studies were included. The overall (Diagnostic Test Accuracy) DTA of retrospective studies with a pooled sensitivity was 0.917 (95% CI 0.88-0.943, I2 = 99%). The pooled specificity was 0.945 (95% CI 0.918-0.964, I2 = 100%). The pooled diagnostic odds ratio (DOR) was 219.47 (95% CI 104.78-459.66, I2 = 100%). These results were significant for the specificity of the different network architecture models (p-value = 0.0289). However, the results for sensitivity (p-value = 0.6417) and DOR (p-value = 0.2187) were not significant. The ResNet algorithm has higher pooled specificity than other algorithms with 0.935 (95% CI 0.854-0.973, I2 = 93%).

CONCLUSION

This meta-analysis on DTA of ML algorithms for detecting ICH by assessing non-contrast CT-Scans shows the ML has an acceptable performance in diagnosing ICH. Using ResNet in ICH detection remains promising prediction was improved via training in an Architecture Learning Network (ALN).

Endoscopic surgery versus craniotomy for spontaneous intracerebral hemorrhage in the late elderly patients

OBJECTIVES

To compare the prognosis of late elderly patients with spontaneous intracerebral hemorrhage (ICH) treated by endoscopic evacuation and craniotomy MATERIALS AND METHODS: Using the Diagnosis Procedure Combination database, we identified patients aged ≥ 75 years who underwent surgery for spontaneous ICH within 48 hours after admission between April 2014 and March 2018. Eligible patients were classified into two groups according to the type of surgery (endoscopic surgery and craniotomy). Propensity-score matching weight analysis was conducted to compare the good neurological outcome modified Rankin Scale (mRS) score (0-4) at discharge as the primary endpoint between the two groups. Secondary endpoints were postoperative meningitis, tracheostomy, reoperation within 3 days and total hospitalization costs.

RESULTS

Among the 5,396 eligible patients, endoscopic surgery and craniotomy were performed in 895 and 4,501 patients, respectively. In the propensity-score matching weight analysis, all covariates were well balanced. The proportions of patients with a good prognosis (mRS score at discharge: 0-4) did not significantly differ between the surgical procedures (42.1% vs. 42.8%, p = 0.828). The proportions of meningitis, tracheostomy and reoperation were not significantly different between the two groups. Hospitalization costs were significantly higher in the craniotomy group than in the endoscopic surgery group (25,536 vs. 29,603 US dollars, p = 0.012).

CONCLUSIONS

Inhospital outcomes did not differ between endoscopic and open surgeries for spontaneous ICH in the late-stage elderly patients aged ≥75 years. Hospitalization costs were significantly higher in the craniotomy group, suggesting that endoscopic surgery may be more acceptable.

A comparison of random survival forest and Cox regression for prediction of mortality in patients with hemorrhagic stroke

OBJECTIVE

To evaluate RSF and Cox models for mortality prediction of hemorrhagic stroke (HS) patients in intensive care unit (ICU).

METHODS

In the training set, the optimal models were selected using five-fold cross-validation and grid search method. In the test set, the bootstrap method was used to validate. The area under the curve(AUC) was used for discrimination, Brier Score (BS) was used for calibration, positive predictive value(PPV), negative predictive value(NPV), and F1 score were combined to compare.

RESULTS

A total of 2,990 HS patients were included. For predicting the 7-day mortality, the mean AUCs for RSF and Cox regression were 0.875 and 0.761, while the mean BS were 0.083 and 0.108. For predicting the 28-day mortality, the mean AUCs for RSF and Cox regression were 0.794 and 0.649, while the mean BS were 0.129 and 0.174. The mean AUCs of RSF and Cox versus conventional scores for predicting patients' 7-day mortality were 0.875 (RSF), 0.761 (COX), 0.736 (SAPS II), 0.723 (OASIS), 0.632 (SIRS), and 0.596 (SOFA), respectively.

CONCLUSIONS

RSF provided a better clinical reference than Cox. Creatine, temperature, anion gap and sodium were important variables in both models.

Prediction of Mortality in Geriatric Traumatic Brain Injury Patients Using Machine Learning Algorithms

Background: The number of geriatric traumatic brain injury (TBI) patients is increasing every year due to the population’s aging in most of the developed countries. Unfortunately, there is no widely recognized tool for specifically evaluating the prognosis of geriatric TBI patients. We designed this study to compare the prognostic value of different machine learning algorithm-based predictive models for geriatric TBI. Methods: TBI patients aged ≥65 from the Medical Information Mart for Intensive Care-III (MIMIC-III) database were eligible for this study. To develop and validate machine learning algorithm-based prognostic models, included patients were divided into a training set and a testing set, with a ratio of 7:3. The predictive value of different machine learning based models was evaluated by calculating the area under the receiver operating characteristic curve, sensitivity, specificity, accuracy and F score. Results: A total of 1123 geriatric TBI patients were included, with a mortality of 24.8%. Non-survivors had higher age (82.2 vs. 80.7, p = 0.010) and lower Glasgow Coma Scale (14 vs. 7, p < 0.001) than survivors. The rate of mechanical ventilation was significantly higher (67.6% vs. 25.9%, p < 0.001) in non-survivors while the rate of neurosurgical operation did not differ between survivors and non-survivors (24.3% vs. 23.0%, p = 0.735). Among different machine learning algorithms, Adaboost (AUC: 0.799) and Random Forest (AUC: 0.795) performed slightly better than the logistic regression (AUC: 0.792) on predicting mortality in geriatric TBI patients in the testing set. Conclusion: Adaboost, Random Forest and logistic regression all performed well in predicting mortality of geriatric TBI patients. Prognostication tools utilizing these algorithms are helpful for physicians to evaluate the risk of poor outcomes in geriatric TBI patients and adopt personalized therapeutic options for them.

Joint modeling strategy for using electronic medical records data to build machine learning models: an example of intracerebral hemorrhage

Background

Outliers and class imbalance in medical data could affect the accuracy of machine learning models. For physicians who want to apply predictive models, how to use the data at hand to build a model and what model to choose are very thorny problems. Therefore, it is necessary to consider outliers, imbalanced data, model selection, and parameter tuning when modeling.

Methods

This study used a joint modeling strategy consisting of: outlier detection and removal, data balancing, model fitting and prediction, performance evaluation. We collected medical record data for all ICH patients with admissions in 2017–2019 from Sichuan Province. Clinical and radiological variables were used to construct models to predict mortality outcomes 90 days after discharge. We used stacking ensemble learning to combine logistic regression (LR), random forest (RF), artificial neural network (ANN), support vector machine (SVM), and k-nearest neighbors (KNN) models. Accuracy, sensitivity, specificity, AUC, precision, and F1 score were used to evaluate model performance. Finally, we compared all 84 combinations of the joint modeling strategy, including training set with and without cross-validated committees filter (CVCF), five resampling techniques (random under-sampling (RUS), random over-sampling (ROS), adaptive synthetic sampling (ADASYN), Borderline synthetic minority oversampling technique (Borderline SMOTE), synthetic minority oversampling technique and edited nearest neighbor (SMOTEENN)) and no resampling, seven models (LR, RF, ANN, SVM, KNN, Stacking, AdaBoost).

Results

Among 4207 patients with ICH, 2909 (69.15%) survived 90 days after discharge, and 1298 (30.85%) died within 90 days after discharge. The performance of all models improved with removing outliers by CVCF except sensitivity. For data balancing processing, the performance of training set without resampling was better than that of training set with resampling in terms of accuracy, specificity, and precision. And the AUC of ROS was the best. For seven models, the average accuracy, specificity, AUC, and precision of RF were the highest. Stacking performed best in F1 score. Among all 84 combinations of joint modeling strategy, eight combinations performed best in terms of accuracy (0.816). For sensitivity, the best performance was SMOTEENN + Stacking (0.662). For specificity, the best performance was CVCF + KNN (0.987). Stacking and AdaBoost had the best performances in AUC (0.756) and F1 score (0.602), respectively. For precision, the best performance was CVCF + SVM (0.938).

Conclusion

This study proposed a joint modeling strategy including outlier detection and removal, data balancing, model fitting and prediction, performance evaluation, in order to provide a reference for physicians and researchers who want to build their own models. This study illustrated the importance of outlier detection and removal for machine learning and showed that ensemble learning might be a good modeling strategy. Due to the low imbalanced ratio (IR, the ratio of majority class and minority class) in this study, we did not find any improvement in models with resampling in terms of accuracy, specificity, and precision, while ROS performed best on AUC.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-02018-x.