Early Detection of Bacteraemia Using Ten Clinical Variables with an Artificial Neural Network Approach

Early prediction of sepsis-induced respiratory tract infection using a biomarker-based machine-learning algorithm

Early and differential diagnosis of sepsis is essential to avoid unnecessary antibiotic use and further reduce patient morbidity and mortality. Here, we aimed to identify predictors of sepsis and advance a machine-learning strategy to predict sepsis-induced respiratory tract infection (RTI). Patients with sepsis and RTI were selected via retrospective analysis, and essential population characteristics and laboratory parameters were recorded. To improve the performance of the primary model and avoid over-fitting, a recursive feature elimination with cross-validation (RFECV) strategy was used to screen the optimal subset of biomarkers and construct nine machine-learning models based on this subset; the average accuracy, precision, recall, and F1-score were used for evaluation of the models. We identified 430 patients with sepsis and 686 patients with RTI. A total of 39 features were collected, with 23 features identified for initial model construction. Using the RFECV algorithm, we found that the XGBoost classifier, which only needed to include seven biomarkers, demonstrated the best performance among all prediction models, with an average accuracy of 89.24 ± 2.28, while the Ridge classifier, which included 11 biomarkers, had an average accuracy of only 83.87 ± 4.69. The remaining models had prediction accuracies greater than 88%. We developed nine models for predicting sepsis using a strategy that combined RFECV with machine learning. Among these models, the XGBoost classifier, which included seven biomarkers, showed the best performance and highest accuracy for predicting sepsis and may be a promising tool for the timely identification of sepsis.

A Machine Learning Predictive Model of Bloodstream Infection in Hospitalized Patients

The aim of the study was to build a machine learning-based predictive model to discriminate between hospitalized patients at low risk and high risk of bloodstream infection (BSI). A Data Mart including all patients hospitalized between January 2016 and December 2019 with suspected BSI was built. Multivariate logistic regression was applied to develop a clinically interpretable machine learning predictive model. The model was trained on 2016-2018 data and tested on 2019 data. A feature selection based on a univariate logistic regression first selected candidate predictors of BSI. A multivariate logistic regression with stepwise feature selection in five-fold cross-validation was applied to express the risk of BSI. A total of 5660 hospitalizations (4026 and 1634 in the training and the validation subsets, respectively) were included. Eleven predictors of BSI were identified. The performance of the model in terms of AUROC was 0.74. Based on the interquartile predicted risk score, 508 (31.1%) patients were defined as being at low risk, 776 (47.5%) at medium risk, and 350 (21.4%) at high risk of BSI. Of them, 14.2% (72/508), 30.8% (239/776), and 64% (224/350) had a BSI, respectively. The performance of the predictive model of BSI is promising. Computational infrastructure and machine learning models can help clinicians identify people at low risk for BSI, ultimately supporting an antibiotic stewardship approach.

Development of an artificial intelligence bacteremia prediction model and evaluation of its impact on physician predictions focusing on uncertainty

Prediction of bacteremia is a clinically important but challenging task. An artificial intelligence (AI) model has the potential to facilitate early bacteremia prediction, aiding emergency department (ED) physicians in making timely decisions and reducing unnecessary medical costs. In this study, we developed and externally validated a Bayesian neural network-based AI bacteremia prediction model (AI-BPM). We also evaluated its impact on physician predictive performance considering both AI and physician uncertainties using historical patient data. A retrospective cohort of 15,362 adult patients with blood cultures performed in the ED was used to develop the AI-BPM. The AI-BPM used structured and unstructured text data acquired during the early stage of ED visit, and provided both the point estimate and 95% confidence interval (CI) of its predictions. High AI-BPM uncertainty was defined as when the predetermined bacteremia risk threshold (5%) was included in the 95% CI of the AI-BPM prediction, and low AI-BPM uncertainty was when it was not included. In the temporal validation dataset (N = 8,188), the AI-BPM achieved area under the receiver operating characteristic curve (AUC) of 0.754 (95% CI 0.737-0.771), sensitivity of 0.917 (95% CI 0.897-0.934), and specificity of 0.340 (95% CI 0.330-0.351). In the external validation dataset (N = 7,029), the AI-BPM's AUC was 0.738 (95% CI 0.722-0.755), sensitivity was 0.927 (95% CI 0.909-0.942), and specificity was 0.319 (95% CI 0.307-0.330). The AUC of the post-AI physicians predictions (0.703, 95% CI 0.654-0.753) was significantly improved compared with that of the pre-AI predictions (0.639, 95% CI 0.585-0.693; p-value < 0.001) in the sampled dataset (N = 1,000). The AI-BPM especially improved the predictive performance of physicians in cases with high physician uncertainty (low subjective confidence) and low AI-BPM uncertainty. Our results suggest that the uncertainty of both the AI model and physicians should be considered for successful AI model implementation.

Developing a machine learning prediction algorithm for early differentiation of urosepsis from urinary tract infection

OBJECTIVES

Early recognition and timely intervention for urosepsis are key to reducing morbidity and mortality. Blood culture has low sensitivity, and a long turnaround time makes meeting the needs of clinical diagnosis difficult. This study aimed to use biomarkers to build a machine learning model for early prediction of urosepsis.

METHODS

Through retrospective analysis, we screened 157 patients with urosepsis and 417 patients with urinary tract infection. Laboratory data of the study participants were collected, including data on biomarkers, such as procalcitonin, D-dimer, and C-reactive protein. We split the data into training (80%) and validation datasets (20%) and determined the average model prediction accuracy through cross-validation.

RESULTS

In total, 26 variables were initially screened and 18 were statistically significant. The influence of the 18 variables was sorted using three ranking methods to further determine the best combination of variables. The Gini importance ranking method was found to be suitable for variable filtering. The accuracy rates of the six machine learning models in predicting urosepsis were all higher than 80%, and the performance of the artificial neural network (ANN) was the best among all. When the ANN included the eight biomarkers with the highest influence ranking, its model had the best prediction performance, with an accuracy rate of 92.9% and an area under the receiver operating characteristic curve of 0.946.

CONCLUSIONS

Urosepsis can be predicted using only the top eight biomarkers determined by the ranking method. This data-driven predictive model will enable clinicians to make quick and accurate diagnoses.

Early predicting 30-day mortality in sepsis in MIMIC-III by an artificial neural networks model

OBJECTIVE

Early identifying sepsis patients who had higher risk of poor prognosis was extremely important. The aim of this study was to develop an artificial neural networks (ANN) model for early predicting clinical outcomes in sepsis.

METHODS

This study was a retrospective design. Sepsis patients from the Medical Information Mart for Intensive Care-III (MIMIC-III) database were enrolled. A predictive model for predicting 30-day morality in sepsis was performed based on the ANN approach.

RESULTS

A total of 2874 patients with sepsis were included and 30-day mortality was 29.8%. The study population was categorized into the training set (n = 1698) and validation set (n = 1176) based on the ratio of 6:4. 11 variables which showed significant differences between survivor group and nonsurvivor group in training set were selected for constructing the ANN model. In training set, the predictive performance based on the area under the receiver-operating characteristic curve (AUC) were 0.873 for ANN model, 0.720 for logistic regression, 0.629 for APACHEII score and 0.619 for SOFA score. In validation set, the AUCs of ANN, logistic regression, APAHCEII score, and SOFA score were 0.811, 0.752, 0.607, and 0.628, respectively.

CONCLUSION

An ANN model for predicting 30-day mortality in sepsis was performed. Our predictive model can be beneficial for early detection of patients with higher risk of poor prognosis.

A Data-Driven Framework for Identifying Intensive Care Unit Admissions Colonized With Multidrug-Resistant Organisms

Background

The rising prevalence of multi-drug resistant organisms (MDROs), such as Methicillin-resistant Staphylococcus aureus (MRSA), Vancomycin-resistant Enterococci (VRE), and Carbapenem-resistant Enterobacteriaceae (CRE), is an increasing concern in healthcare settings.

Materials and Methods

Leveraging data from electronic healthcare records and a unique MDRO universal screening program, we developed a data-driven modeling framework to predict MRSA, VRE, and CRE colonization upon intensive care unit (ICU) admission, and identified the associated socio-demographic and clinical factors using logistic regression (LR), random forest (RF), and XGBoost algorithms. We performed threshold optimization for converting predicted probabilities into binary predictions and identified the cut-off maximizing the sum of sensitivity and specificity.

Results

Four thousand six hundred seventy ICU admissions (3,958 patients) were examined. MDRO colonization rate was 17.59% (13.03% VRE, 1.45% CRE, and 7.47% MRSA). Our study achieved the following sensitivity and specificity values with the best performing models, respectively: 80% and 66% for VRE with LR, 73% and 77% for CRE with XGBoost, 76% and 59% for MRSA with RF, and 82% and 83% for MDRO (i.e., VRE or CRE or MRSA) with RF. Further, we identified several predictors of MDRO colonization, including long-term care facility stay, current diagnosis of skin/subcutaneous tissue or infectious/parasitic disease, and recent isolation precaution procedures before ICU admission.

Conclusion

Our data-driven modeling framework can be used as a clinical decision support tool for timely predictions, characterization and identification of high-risk patients, and selective and timely use of infection control measures in ICUs.

Prediction of Bacteremia Based on 12-Year Medical Data Using a Machine Learning Approach: Effect of Medical Data by Extraction Time

Early detection of bacteremia is important to prevent antibiotic abuse. Therefore, we aimed to develop a clinically applicable bacteremia prediction model using machine learning technology. Data from two tertiary medical centers’ electronic medical records during a 12-year-period were extracted. Multi-layer perceptron (MLP), random forest, and gradient boosting algorithms were applied for machine learning analysis. Clinical data within 12 and 24 hours of blood culture were analyzed and compared. Out of 622,771 blood cultures, 38,752 episodes of bacteremia were identified. In MLP with 128 hidden layer nodes, the area under the receiver operating characteristic curve (AUROC) of the prediction performance in 12- and 24-h data models was 0.762 (95% confidence interval (CI); 0.7617–0.7623) and 0.753 (95% CI; 0.7520–0.7529), respectively. AUROC of causative-pathogen subgroup analysis predictive value for Acinetobacter baumannii bacteremia was the highest at 0.839 (95% CI; 0.8388–0.8394). Compared to primary bacteremia, AUROC of sepsis caused by pneumonia was highest. Predictive performance of bacteremia was superior in younger age groups. Bacteremia prediction using machine learning technology appeared possible for acute infectious diseases. This model was more suitable especially to pneumonia caused by Acinetobacter baumannii. From the 24-h blood culture data, bacteremia was predictable by substituting only the continuously variable values.

Prediction of bacteremia at the emergency department during triage and disposition stages using machine learning models

INTRODUCTION

Bacteremia is a common but critical condition with high mortality that requires timely and optimal treatment in the emergency department (ED). The prediction of bacteremia at the ED during triage and disposition stages could support the clinical decisions of ED physicians regarding the appropriate treatment course and safe ED disposition. This study developed and validated machine learning models to predict bacteremia in the emergency department during triage and disposition stages.

METHODS

This study enrolled adult patients who visited a single tertiary hospital from 2016 to 2018 and had at least two sets of blood cultures during their ED stay. Demographic information, chief complaint, triage level, vital signs, and laboratory data were used as model predictors. We developed and validated prediction models using 10 variables at the time of ED triage and 42 variables at the time of disposition. The extreme gradient boosting (XGB) model was compared with the random forest and multivariable logistic regression models. We compared model performance by assessing the area under the receiver operating characteristic curve (AUC), test characteristics, and decision curve analysis.

RESULTS

A total of 24,768 patients were included: 16,197 cases were assigned to development, and 8571 cases were assigned to validation. The proportion of bacteremia was 10.9% and 10.4% in the development and validation datasets, respectively. The Triage XGB model (AUC, 0.718; 95% confidence interval (CI), 0.701-0.735) showed acceptable discrimination performance with a sensitivity over 97%. The Disposition XGB model (AUC, 0.853; 95% CI, 0.840-0.866) showed excellent performance and provided the greatest net benefit throughout the range of thresholds probabilities.

CONCLUSIONS

The Triage XGB model could be used to identify patients with a low risk of bacteremia immediately after initial ED triage. The Disposition XGB model showed excellent discriminative performance.

Applying artificial neural network for early detection of sepsis with intentionally preserved highly missing real-world data for simulating clinical situation

Purpose

Some predictive systems using machine learning models have been developed to predict sepsis; however, they were mostly built with a low percent of missing values, which does not correspond with the actual clinical situation. In this study, we developed a machine learning model with a high rate of missing and erroneous data to enable prediction under missing, noisy, and erroneous inputs, as in the actual clinical situation.

Materials and methods

The proposed artificial neural network model was implemented using the MATLAB ANN toolbox, based on stochastic gradient descent. The dataset was collected over the past decade with approval from the appropriate institutional review boards, and the sepsis status was identified and labeled using Sepsis-3 clinical criteria. The imputation method was built by last observation carried forward and mean value, aimed to simulate clinical situation.

Results

The mean area under the receiver operating characteristic (ROC) curve (AUC) of classifying sepsis and nonsepsis patients was 0.82 and 0.786 at 0 h and 40 h prior to onset, respectively. The highest model performance was found for one-hourly data, demonstrating that our ANN model can perform adequately with limited hourly data provided.

Conclusions

Our model has the moderate ability to predict sepsis up to 40 h in advance under simulated clinical situation with real-world data.

Diagnosing hospital bacteraemia in the framework of predictive, preventive and personalised medicine using electronic health records and machine learning classifiers

Background

The bacteraemia prediction is relevant because sepsis is one of the most important causes of morbidity and mortality. Bacteraemia prognosis primarily depends on a rapid diagnosis. The bacteraemia prediction would shorten up to 6 days the diagnosis, and, in conjunction with individual patient variables, should be considered to start the early administration of personalised antibiotic treatment and medical services, the election of specific diagnostic techniques and the determination of additional treatments, such as surgery, that would prevent subsequent complications. Machine learning techniques could help physicians make these informed decisions by predicting bacteraemia using the data already available in electronic hospital records.

Objective

This study presents the application of machine learning techniques to these records to predict the blood culture's outcome, which would reduce the lag in starting a personalised antibiotic treatment and the medical costs associated with erroneous treatments due to conservative assumptions about blood culture outcomes.

Methods

Six supervised classifiers were created using three machine learning techniques, Support Vector Machine, Random Forest and K-Nearest Neighbours, on the electronic health records of hospital patients. The best approach to handle missing data was chosen and, for each machine learning technique, two classification models were created: the first uses the features known at the time of blood extraction, whereas the second uses four extra features revealed during the blood culture.

Results

The six classifiers were trained and tested using a dataset of 4357 patients with 117 features per patient. The models obtain predictions that, for the best case, are up to a state-of-the-art accuracy of 85.9%, a sensitivity of 87.4% and an AUC of 0.93.

Conclusions

Our results provide cutting-edge metrics of interest in predictive medical models with values that exceed the medical practice threshold and previous results in the literature using classical modelling techniques in specific types of bacteraemia. Additionally, the consistency of results is reasserted because the three classifiers' importance ranking shows similar features that coincide with those that physicians use in their manual heuristics. Therefore, the efficacy of these machine learning techniques confirms their viability to assist in the aims of predictive and personalised medicine once the disease presents bacteraemia-compatible symptoms and to assist in improving the healthcare economy.

An Artificial Intelligence Approach to Bloodstream Infections Prediction

This study aimed to develop an early prediction model for identifying patients with bloodstream infections. The data resource was taken from 2015 to 2019 at Taichung Veterans General Hospital, and a total of 1647 bloodstream infection episodes and 3552 non-bloodstream infection episodes in the intensive care unit (ICU) were included in the model development and evaluation. During the data analysis, 30 clinical variables were selected, including patients’ basic characteristics, vital signs, laboratory data, and clinical information. Five machine learning algorithms were applied to examine the prediction model performance. The findings indicated that the area under the receiver operating characteristic curve (AUROC) of the prediction performance of the XGBoost model was 0.825 for the validation dataset and 0.821 for the testing dataset. The random forest model also presented higher values for the AUROC on the validation dataset and testing dataset, which were 0.855 and 0.851, respectively. The tree-based ensemble learning model enabled high detection ability for patients with bloodstream infections in the ICU. Additionally, the analysis of importance of features revealed that alkaline phosphatase (ALKP) and the period of the central venous catheter are the most important predictors for bloodstream infections. We further explored the relationship between features and the risk of bloodstream infection by using the Shapley Additive exPlanations (SHAP) visualized method. The results showed that a higher prothrombin time is more prominent in a bloodstream infection. Additionally, the impact of a lower platelet count and albumin was more prominent in a bloodstream infection. Our results provide additional clinical information for cut-off laboratory values to assist clinical decision-making in bloodstream infection diagnostics.

An Artificial Neural Networks Model for Early Predicting In-Hospital Mortality in Acute Pancreatitis in MIMIC-III

Background

Early and accurate evaluation of severity and prognosis in acute pancreatitis (AP), especially at the time of admission is very significant. This study was aimed to develop an artificial neural networks (ANN) model for early prediction of in-hospital mortality in AP.

Methods

Patients with AP were identified from the Medical Information Mart for Intensive Care-III (MIMIC-III) database. Clinical and laboratory data were utilized to perform a predictive model by back propagation ANN approach.

Results

A total of 337 patients with AP were analyzed in the study, and the in-hospital mortality rate was 11.2%. A total of 12 variables that differed between patients in survivor group and nonsurvivor group were applied to construct ANN model. Three independent variables were identified as risk factors associated with in-hospital mortality by multivariate logistic regression analysis. The predictive performance based on the area under the receiver operating characteristic curve (AUC) was 0.769 for ANN model, 0.607 for logistic regression, 0.652 for Ranson score, and 0.401 for SOFA score.

Conclusion

An ANN predictive model for in-hospital mortality in patients with AP in MIMIC-III database was first performed. The patients with high risk of fatal outcome can be screened out easily in the early stage of AP by our model.

Machine learning techniques for mortality prediction in critical traumatic patients: anatomic and physiologic variables from the RETRAUCI study

Background

Interest in models for calculating the risk of death in traumatic patients admitted to ICUs remains high. These models use variables derived from the deviation of physiological parameters and/or the severity of anatomical lesions with respect to the affected body areas. Our objective is to create different predictive models of the mortality of critically traumatic patients using machine learning techniques.

Methods

We used 9625 records from the RETRAUCI database (National Trauma Registry of 52 Spanish ICUs in the period of 2015–2019). Hospital mortality was 12.6%. Data on demographic variables, affected anatomical areas and physiological repercussions were used. The Weka Platform was used, along with a ten-fold cross-validation for the construction of nine supervised algorithms: logistic regression binary (LR), neural network (NN), sequential minimal optimization (SMO), classification rules (JRip), classification trees (CT), Bayesian networks (BN), adaptive boosting (ADABOOST), bootstrap aggregating (BAGGING) and random forest (RFOREST). The performance of the models was evaluated by accuracy, specificity, precision, recall, F-measure, and AUC.

Results

In all algorithms, the most important factors are those associated with traumatic brain injury (TBI) and organic failures. The LR finds thorax and limb injuries as independent protective factors of mortality. The CT generates 24 decision rules and uses those related to TBI as the first variables (range 2.0–81.6%). The JRip detects the eight rules with the highest risk of mortality (65.0–94.1%). The NN model uses a hidden layer of ten nodes, which requires 200 weights for its interpretation. The BN find the relationships between the different factors that identify different patient profiles. Models with the ensemble methodology (ADABOOST, BAGGING and RandomForest) do not have greater performance. All models obtain high values ​​in accuracy, specificity, and AUC, but obtain lower values ​​in recall. The greatest precision is achieved by the SMO model, and the BN obtains the best recall, F-measure, and AUC.

Conclusion

Machine learning techniques are useful for creating mortality classification models in critically traumatic patients. With clinical interpretation, the algorithms establish different patient profiles according to the relationship between the variables used, determine groups of patients with different evolutions, and alert clinicians to the presence of rules that indicate the greatest severity.

Detection of Bacteremia in Surgical In-Patients Using Recurrent Neural Network Based on Time Series Records: Development and Validation Study

Background

Detecting bacteremia among surgical in-patients is more obscure than other patients due to the inflammatory condition caused by the surgery. The previous criteria such as systemic inflammatory response syndrome or Sepsis-3 are not available for use in general wards, and thus, many clinicians usually rely on practical senses to diagnose postoperative infection.

Objective

This study aims to evaluate the performance of continuous monitoring with a deep learning model for early detection of bacteremia for surgical in-patients in the general ward and the intensive care unit (ICU).

Methods

In this retrospective cohort study, we included 36,023 consecutive patients who underwent general surgery between October and December 2017 at a tertiary referral hospital in South Korea. The primary outcome was the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) for detecting bacteremia by the deep learning model, and the secondary outcome was the feature explainability of the model by occlusion analysis.

Results

Out of the 36,023 patients in the data set, 720 cases of bacteremia were included. Our deep learning–based model showed an AUROC of 0.97 (95% CI 0.974-0.981) and an AUPRC of 0.17 (95% CI 0.147-0.203) for detecting bacteremia in surgical in-patients. For predicting bacteremia within the previous 24-hour period, the AUROC and AUPRC values were 0.93 and 0.15, respectively. Occlusion analysis showed that vital signs and laboratory measurements (eg, kidney function test and white blood cell group) were the most important variables for detecting bacteremia.

Conclusions

A deep learning model based on time series electronic health records data had a high detective ability for bacteremia for surgical in-patients in the general ward and the ICU. The model may be able to assist clinicians in evaluating infection among in-patients, ordering blood cultures, and prescribing antibiotics with real-time monitoring.