Emergency Severity Index as a predictor of in-hospital mortality in suspected sepsis patients in the emergency department

Early prediction of sepsis in emergency department patients using various methods and scoring systems

BACKGROUND

Early recognition of sepsis, a common life-threatening condition in intensive care units (ICUs), is beneficial for improving patient outcomes. However, most sepsis prediction models were trained and assessed in the ICU, which might not apply to emergency department (ED) settings.

AIM

To establish an early predictive model based on basic but essential information collected upon ED presentation for the follow-up diagnosis of sepsis observed in the ICU.

STUDY DESIGN

This study developed and validated a reliable model of sepsis prediction among ED patients by comparing 10 different methods based on retrospective electronic health record data from the MIMIC-IV database. In-ICU sepsis was identified as the primary outcome. The potential predictors encompassed baseline demographics, vital signs, pain scale, chief complaints and Emergency Severity Index (ESI). 80% and 20% of the total of 425 737 ED visit records were randomly selected for the train set and the test set for model development and validation, respectively.

RESULTS

Among the methods evaluated, XGBoost demonstrated an optimal predictive performance with an area under the curve (AUC) of 0.90 (95% CI: 0.90-0.91). Logistic regression exhibited a comparable predictive ability to XGBoost, with an AUC of 0.89 (95% CI: 0.89-0.90), along with a sensitivity and specificity of 85% (95% CI: 0.83-0.86) and 78% (95% CI: 0.77-0.80), respectively. Neither of the five commonly used severity scoring systems demonstrated satisfactory performance for sepsis prediction. The predictive ability of using ESI as the sole predictor (AUC: 0.79, 95% CI: 0.78-0.80) was also inferior to the model integrating ESI and other basic information.

CONCLUSIONS

The use of ESI combined with basic clinical information upon ED presentation accurately predicted sepsis among ED patients, strengthening its application in ED.

RELEVANCE TO CLINICAL PRACTICE

The proposed model may assist nurses in risk stratification management and prioritize interventions for potential sepsis patients, even in low-resource settings.

Prospective and External Validation of Machine Learning Models for Short- and Long-Term Mortality in Acutely Admitted Patients Using Blood Tests

Background: Predicting mortality in emergency departments (EDs) using machine learning models presents challenges, particularly in balancing simplicity with performance. This study aims to develop models that are both simple and effective for predicting short- and long-term mortality in ED patients. Our approach uses a minimal set of variables derived from one single blood sample obtained at admission. Methods: Data from three cohorts at two large Danish university hospitals were analyzed, including one retrospective and two prospective cohorts where prognostic models were applied to predict individual mortality risk, spanning the years 2013-2022. Routine biochemistry analyzed in blood samples collected at admission was the primary data source for the prediction models. The outcomes were mortality at 10, 30, 90, and 365 days after admission to the ED. The models were developed using Light Gradient Boosting Machines. The evaluation of mortality predictions involved metrics such as Area Under the Receiver Operating Characteristic Curve (AUC), sensitivity, specificity, negative predictive values, positive predictive values, and Matthews correlation coefficient (MCC). Results: A total of 43,648 unique patients with 65,484 admissions were analyzed. The models showed high accuracy, with very good to excellent AUC values between 0.87 and 0.93 across different time intervals. Conclusions: This study demonstrates that a single assessment of routine clinical biochemistry upon admission can serve as a powerful predictor for both short-term and long-term mortality in ED admissions.

Interpretable Machine Learning Models for Predicting Critical Outcomes in Patients with Suspected Urinary Tract Infection with Positive Urine Culture

(1) Background: Urinary tract infection (UTI) is a leading cause of emergency department visits and hospital admissions. Despite many studies identifying UTI-related risk factors for bacteremia or sepsis, a significant gap remains in developing predictive models for in-hospital mortality or the necessity for emergent intensive care unit admission in the emergency department. This study aimed to construct interpretable machine learning models capable of identifying patients at high risk for critical outcomes. (2) Methods: This was a retrospective study of adult patients with urinary tract infection (UTI), extracted from the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) database. The critical outcome is defined as either in-hospital mortality or transfer to an intensive care unit within 12 h. ED visits were randomly partitioned into a 70%/30% split for training and validation. The extreme gradient boosting (XGBoost), random forest (RF), and support vector machine (SVM) algorithms were constructed using variables selected from the stepwise logistic regression model. The XGBoost model was then compared to the traditional model and clinical decision rules (CDRs) on the validation data using the area under the curve (AUC). (3) Results: There were 3622 visits among 3235 unique patients diagnosed with UTI. Of the 2535 patients in the training group, 836 (33%) experienced critical outcomes, and of the 1087 patients in the validation group, 358 (32.9%) did. The AUCs for different machine learning models were as follows: XGBoost, 0.833; RF, 0.814; and SVM, 0.799. The XGBoost model performed better than others. (4) Conclusions: Machine learning models outperformed existing traditional CDRs for predicting critical outcomes of ED patients with UTI. Future research should prospectively evaluate the effectiveness of this approach and integrate it into clinical practice.

An AI-based multiphase framework for improving the mechanical ventilation availability in emergency departments during respiratory disease seasons: a case study

BACKGROUND

Shortages of mechanical ventilation have become a constant problem in Emergency Departments (EDs), thereby affecting the timely deployment of medical interventions that counteract the severe health complications experienced during respiratory disease seasons. It is then necessary to count on agile and robust methodological approaches predicting the expected demand loads to EDs while supporting the timely allocation of ventilators. In this paper, we propose an integration of Artificial Intelligence (AI) and Discrete-event Simulation (DES) to design effective interventions ensuring the high availability of ventilators for patients needing these devices.

METHODS

First, we applied Random Forest (RF) to estimate the mechanical ventilation probability of respiratory-affected patients entering the emergency wards. Second, we introduced the RF predictions into a DES model to diagnose the response of EDs in terms of mechanical ventilator availability. Lately, we pretested two different interventions suggested by decision-makers to address the scarcity of this resource. A case study in a European hospital group was used to validate the proposed methodology.

RESULTS

The number of patients in the training cohort was 734, while the test group comprised 315. The sensitivity of the AI model was 93.08% (95% confidence interval, [88.46 - 96.26%]), whilst the specificity was 85.45% [77.45 - 91.45%]. On the other hand, the positive and negative predictive values were 91.62% (86.75 - 95.13%) and 87.85% (80.12 - 93.36%). Also, the Receiver Operator Characteristic (ROC) curve plot was 95.00% (89.25 - 100%). Finally, the median waiting time for mechanical ventilation was decreased by 17.48% after implementing a new resource capacity strategy.

CONCLUSIONS

Combining AI and DES helps healthcare decision-makers to elucidate interventions shortening the waiting times for mechanical ventilators in EDs during respiratory disease epidemics and pandemics.

Development and validation of prognostic machine learning models for short- and long-term mortality among acutely admitted patients based on blood tests

Several scores predicting mortality at the emergency department have been developed. However, all with shortcomings either simple and applicable in a clinical setting, with poor performance, or advanced, with high performance, but clinically difficult to implement. This study aimed to explore if machine learning algorithms could predict all-cause short- and long-term mortality based on the routine blood test collected at admission.

METHODS

We analyzed data from a retrospective cohort study, including patients > 18 years admitted to the Emergency Department (ED) of Copenhagen University Hospital Hvidovre, Denmark between November 2013 and March 2017. The primary outcomes were 3-, 10-, 30-, and 365-day mortality after admission. PyCaret, an automated machine learning library, was used to evaluate the predictive performance of fifteen machine learning algorithms using the area under the receiver operating characteristic curve (AUC).

RESULTS

Data from 48,841 admissions were analyzed, of these 34,190 (70%) were randomly divided into training data, and 14,651 (30%) were in test data. Eight machine learning algorithms achieved very good to excellent results of AUC on test data in a of range 0.85-0.93. In prediction of short-term mortality, lactate dehydrogenase (LDH), leukocyte counts and differentials, Blood urea nitrogen (BUN) and mean corpuscular hemoglobin concentration (MCHC) were the best predictors, whereas prediction of long-term mortality was favored by age, LDH, soluble urokinase plasminogen activator receptor (suPAR), albumin, and blood urea nitrogen (BUN).

CONCLUSION

The findings suggest that measures of biomarkers taken from one blood sample during admission to the ED can identify patients at high risk of short-and long-term mortality following emergency admissions.

SIRS, SOFA, qSOFA, and NEWS in the diagnosis of sepsis and prediction of adverse outcomes: a systematic review and meta-analysis

BACKGROUND

We compared Systemic Inflammatory Response Syndrome (SIRS), Sequential Organ Failure Assessment (SOFA), Quick Sepsis-related Organ Failure Assessment (qSOFA), and National Early Warning Score (NEWS) for sepsis diagnosis and adverse outcomes prediction.

METHODS

Clinical studies that used SIRS, SOFA, qSOFA, and NEWS for sepsis diagnosis and prognosis assessment were included. Data were extracted, and meta-analysis was performed for outcome measures, including sepsis diagnosis, in-hospital mortality, 7/10/14-day mortality, 28/30-day mortality, and ICU admission.

RESULTS

Fifty-seven included studies showed good overall quality. Regarding sepsis prediction, SIRS demonstrated high sensitivity (0.85) but low specificity (0.41), qSOFA showed low sensitivity (0.42) but high specificity (0.98), and NEWS exhibited high sensitivity (0.71) and specificity (0.85). For predicting in-hospital mortality, SOFA demonstrated the highest sensitivity (0.89) and specificity (0.69). In terms of predicting 7/10/14-day mortality, SIRS exhibited high sensitivity (0.87), while qSOFA had high specificity (0.75). For predicting 28/30-day mortality, SOFA showed high sensitivity (0.97) but low specificity (0.14), whereas qSOFA displayed low sensitivity (0.41) but high specificity (0.88).

CONCLUSIONS

NEWS independently demonstrates good diagnostic capability for sepsis, especially in high-income countries. SOFA emerges as the optimal choice for predicting in-hospital mortality and can be employed as a screening tool for 28/30-day mortality in low-income countries.

Audit of a computerized version of the Manchester triage system and a SIRS-based system for the detection of sepsis at triage in the emergency department

BACKGROUND AND IMPORTANCE

Different triage systems can be used to screen for sepsis and are often incorporated into local electronic health records. Often the design and interface of these digitalizations are not audited, possibly leading to deleterious effects on screening test performance.

OBJECTIVE

To audit a digital version of the MTS for detection of sepsis during triage in the ED.

DESIGN

A single-center retrospective study SETTINGS AND PARTICIPANTS: Patients (n=29766) presenting to an ED of a tertiary-care center who received formal triage were included.

OUTCOME MEASURES AND ANALYSIS

Calculated performance measures included sensitivity, specificity, likelihood ratios, and AUC for the detection of sepsis. Errors in the application of the specific sepsis discriminator of the MTS were recorded.

MAIN RESULTS

A total of 189 (0.7%) subjects met the Sepsis-3 criteria, with 47 cases meeting the criteria for septic shock. The MTS had a low sensitivity of 47.6% (95% CI 40.3 to 55.0) for allocating sepsis patients to the correct triage category. However, specificity was high at 99.4% (95% CI 99.3 to 99.5).

The prognostic value of sepsis scores and dichotomized triage score in patients presenting to the emergency department with fever: A prospective, observational study

BACKGROUND

The performance of the Quick Sequential Organ Failure Assessment (qSOFA) score needs to be explored further in the emergency triage room. This study aims to explore the performance of triage (tqSOFA) versus the dichotomized triage score (DTS) in patients admitted to the emergency room triage with fever.

METHODS

This research was designed as a prospective, observational study within a six-month period, including patients who presented to the emergency room triage with infrared fever ≥ 37.5 °C.

RESULTS

771 patients were analyzed.The highest sensitivity for predicting overall hospitalization and intensive care admission was seen for DTS (95.4 %, 100 %; p < 0.0001, p < 0.0001, respectively) (AUC:0.697, 95 % CI 0.663 to 0.730; AUC:0.684, 95 % CI 0.650 to 0.717, respectively). The highest sensitivity for predicting 1st week and 1st month mortality was found for DTS (100 %, 96.3 %; p < 0.0001, p < 0.0001, respectively). However, the highest specificity for predicting 1st week and 1st month mortality was observed in tqSOFA (94.1 %, 95.16; p = 0.0845, p < 0.0001, respectively) (AUC:0.658, 95 % CI 0.623 to 0.691; AUC:0.698, 95 % CI 0.664 to 0.730, respectively).

CONCLUSION

We found DTS to be as effective as tqSOFA and SIRS in determining all hospitalization times and mortality.

A comparison of qSOFA, SIRS and NEWS in predicting the accuracy of mortality in patients with suspected sepsis: A meta-analysis

Objective

To identify and compare prognostic accuracy of quick Sequential Organ Failure Assessment (qSOFA) score, Systemic Inflammatory Response Syndrome (SIRS) criteria, and National Early Warning Score (NEWS) to predict mortality in patients with suspected sepsis.

Methods

This meta-analysis followed accordance with the recommendations of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. We searched PubMed, EMBASE, Web of Science, and the Cochrane Library databases from establishment of the database to November 29, 2021. The pooled sensitivity and specificity with 95% CIs were calculated using a bivariate random-effects model (BRM). Hierarchical summary receiver operating characteristic (HSROC) curves were generated to assess the overall prognostic accuracy.

Results

Data of 62338 patients from 26 studies were included in this meta-analysis. qSOFA had the highest specificity and the lowest sensitivity with a specificity of 0.82 (95% CI: 0.76–0.86) and a sensitivity of 0.46 (95% CI: 0.39–0.53). SIRS had the highest sensitivity and the lowest specificity with a sensitivity of 0.82 (95% CI: 0.78–0.85) and a specificity 0.24 (95% CI: 0.19–0.29). NEWS had both an intermediate sensitivity and specificity with a sensitivity of 0.73 (95% CI: 0.63–0.81) and a specificity 0.52 (95% CI: 0.39–0.65). qSOFA showed higher overall prognostic accuracy than SIRS and NEWS by comparing HSROC curves.

Conclusions

Among qSOFA, SIRS and NEWS, qSOFA showed higher overall prognostic accuracy than SIRS and NEWS. However, no scoring system has both high sensitivity and specificity for predicting the accuracy of mortality in patients with suspected sepsis.

A systematic review of the discrimination and absolute mortality predicted by the National Early Warning Scores according to different cut-off values and prediction windows

BACKGROUND

Although early warning scores were intended to simply identify patients in need of life-saving interventions, prediction has become their commonest metric. This review examined variation in the ability of the National Early Warning Scores (NEWS) in adult patients to predict absolute mortality at different times and cut-offs values.

METHOD

Following PRISMA guidelines, all studies reporting NEWS and NEWS2 providing enough information to fulfil the review's aims were included.

RESULTS

From 121 papers identified, the average area under the Receiver Operating Characteristic curve (AUC) for mortality declined from 0.90 at 24-hours to 0.76 at 30-days. Studies with a low overall mortality had a higher AUC for 24-hour mortality, as did general ward patients compared to patients seen earlier in their treatment. 24-hour mortality increased from 1.8% for a NEWS ≥3 to 7.8% for NEWS ≥7. Although 24-hour mortality for NEWS <3 was only 0.07% these deaths accounted for 9% of all deaths within 24-hours; for NEWS <7 24-hour mortality was 0.23%, which accounted for 44% of all 24-hour deaths. Within 30-days of a NEWS recording 22% of all deaths occurred in patients with a NEWS <3, 52% in patients with a NEWS <5, and 75% in patient with a NEWS <7.

CONCLUSION

NEWS reliably identifies patients most and least likely to die within 24-hours, which is what it was designed to do. However, many patients identified to have a low risk of imminent death die within 30-days. NEWS mortality predictions beyond 24-hours are unreliable.

Comparing the prehospital NEWS with in-hospital ESI in predicting 30-day severe outcomes in emergency patients

BACKGROUND

In Iran, the emergency departments (EDs) have largely adopted the emergency severity index (ESI) to prioritize the emergency patients, however emergency medical services (EMS) mainly triage the patients based on the paramedics' gestalt. The National Early Warning Score (NEWS) is a recommended prehospital triage in the UK. We aimed to compare prehospital NEWS and ED ESI for predicting severe outcomes in emergency patients.

METHODS

An observational study was conducted in a university-affiliated ED between January and April 2021. Adult patients who arrived in the ED by EMS were included. EMS providers calculated the patients' NEWS upon arriving on the scene using an Android NEWS application. In the ED, triage nurses utilized the ESI algorithm to prioritize patients with higher clinical risk. Then, Research nurses recorded patients' 30-day severe outcomes (death or ICU admission). Finally, The prognostic properties of ESI and NEWS were evaluated.

RESULTS

One thousand forty-eight cases were included in the final analysis, of which 29 (2.7%) patients experienced severe outcomes. The difference between the prehospital NEWS and ED ESI in predicting severe outcomes was not statistically significant (AUC = 0.825, 95% CI: 0.74-0.91 and 0.897, 95% CI, 0.83-0.95, for prehospital NEWS and ESI, respectively).

CONCLUSION

Our findings indicated that prehospital NEWS compares favorably with ED ESI in predicting 30-day severe outcomes in emergency patients.

Performance of the Simple Clinical Score (SCS) and the Rapid Emergency Medicine Score (REMS) to predict severity level and mortality rate among patients with sepsis in the emergency department

Nurses play a key role as the first line of service for patients with medical conditions and injuries in the emergency department (ED), which includes assessing patients for sepsis. The researchers evaluated tools to examine the performance of the Simple Clinical Score (SCS) and the Rapid Emergency Medicine Score (REMS) to predict sepsis severity and mortality among sepsis patients in the ED. A retrospective survey was performed, selecting participants by using a purposive sampling method, and including the medical records of all patients diagnosed with sepsis admitted to the ED at Singburi Hospital, Thailand. Data were analysed using the ROC curve and the Area Under Curve (AUC) to calculate the accuracy of each patient's mortality prediction. A total of 225 patients diagnosed with sepsis was identified, with a mortality rate of 59.11% after admission to the medical service and intensive care unit. The AUC analysis showed that the accuracy of the model generated from the REMS (88.6%) was higher than that of the SCS (76.7%). The authors also recommend that key variables identified in this research should be used to develop screening and assessment tools for sepsis in the context of the ED.

National Early Warning Score Does Not Accurately Predict Mortality for Patients With Infection Outside the Intensive Care Unit: A Systematic Review and Meta-Analysis

Background: The prognostic value of the national early warning score (NEWS) in patients with infections remains controversial. We aimed to evaluate the prognostic accuracy of NEWS for prediction of in-hospital mortality in patients with infections outside the intensive care unit (ICU). Methods: We searched PubMed, Embase, and Scopus for related articles from January 2012 to April 2021. Sensitivity, specificity, and likelihood ratios were pooled by using the bivariate random-effects model. Overall prognostic performance was summarized by using the area under the curve (AUC). We performed subgroup analyses to assess the prognostic accuracy of NEWS in selected populations. Results: A total of 21 studies with 107,008 participants were included. The pooled sensitivity and specificity of NEWS were 0.71 and 0.60. The pooled AUC of NEWS was 0.70, which was similar to quick sequential organ failure assessment (qSOFA, AUC: 0.70) and better than systemic inflammatory response syndrome (SIRS, AUC: 0.60). However, the sensitivity (0.55) and AUC (0.63) of NEWS were poor in elder patients. The NEWS of 5 was more sensitive, which was a better threshold for activating urgent assessment and treatment. Conclusions: The NEWS had good diagnostic accuracy for early prediction of mortality in patients with infections outside the ICU, and the sensitivity and specificity were more moderate when compared with qSOFA and SIRS. Insufficient sensitivity and poor performance in the elder population may have limitations as an early warning score for adverse outcomes. NEWS should be used for continuous monitoring rather than a single time point predictive tool.

Clinical performance of early warning scoring systems for identifying sepsis among anti-hypertensive agent users

BACKGROUND

Little is known about the accuracy of the quick Sequential Organ Failure Assessment (qSOFA) and the National Early Warning Score (NEWS) in identifying sepsis patients with a history of hypertension on anti-hypertensive agents, which affect vital signs as components of the scoring systems. We aimed to examine the ability of qSOFA and NEWS to predict sepsis among anti-hypertensive agent users by comparing them with non-users.

METHODS

We retrospectively identified adult patients (aged ≥18years) with suspected infection who presented to an emergency department (ED) of a large tertiary medical center in Japan between April 2018 and March 2020. Suspected infection was defined based on the chief complaint of fever, high body temperature, or the clinical context on arrival at the ED. We excluded patients who had trauma or cardiac arrest, those who were transported to other hospitals after arrival at the ED, and those whose vital signs data were mostly missing. The predictive performances of qSOFA and NEWS based on initial vital signs were examined separately for sepsis, ICU admission, and in-hospital mortality and compared between anti-hypertensive agent users and non-users.

RESULTS

Among 2900 patients with suspected infection presenting to the ED, 291 (10%) had sepsis, 1023 (35%) were admitted to the ICU, and 188 (6.5%) died. The prediction performances of qSOFA and NEWS for each outcome among anti-hypertensive agent users were lower than that among non-users (e.g., c-statistics of qSOFA for sepsis, 0.66 vs. 0.71, p = 0.07; and for ICU admission, 0.70 vs. 0.75, p = 0.01). For identifying sepsis, the sensitivity and specificity of qSOFA ≥2 were 0.43 and 0.77 in anti-hypertensive agent users and 0.51 and 0.82 in non-users. Similar associations were observed for identifying ICU admission and in-hospital mortality. Regardless of the use of anti-hypertensive agents, NEWS had better prediction abilities for each outcome than qSOFA.

CONCLUSION

The clinical performance of qSOFA and NEWS for identifying sepsis among anti-hypertensive agent users was likely lower than that among non-users.