A pre-hospital risk score predicts critical illness in non-trauma patients transported by ambulance to a Dutch tertiary referral hospital

Development and internal validation of an algorithm for estimating mortality in patients encountered by physician-staffed helicopter emergency medical services

BACKGROUND

Severity of illness scoring systems are used in intensive care units to enable the calculation of adjusted outcomes for audit and benchmarking purposes. Similar tools are lacking for pre-hospital emergency medicine. Therefore, using a national helicopter emergency medical services database, we developed and internally validated a mortality prediction algorithm.

METHODS

We conducted a multicentre retrospective observational register-based cohort study based on the patients treated by five physician-staffed Finnish helicopter emergency medical service units between 2012 and 2019. Only patients aged 16 and over treated by physician-staffed units were included. We analysed the relationship between 30-day mortality and physiological, patient-related and circumstantial variables. The data were imputed using multiple imputations employing chained equations. We used multivariate logistic regression to estimate the variable effects and performed derivation of multiple multivariable models with different combinations of variables. The models were combined into an algorithm to allow a risk estimation tool that accounts for missing variables. Internal validation was assessed by calculating the optimism of each performance estimate using the von Hippel method with four imputed sets.

RESULTS

After exclusions, 30 186 patients were included in the analysis. 8611 (29%) patients died within the first 30 days after the incident. Eleven predictor variables (systolic blood pressure, heart rate, oxygen saturation, Glasgow Coma Scale, sex, age, emergency medical services vehicle type [helicopter vs ground unit], whether the mission was located in a medical facility or nursing home, cardiac rhythm [asystole, pulseless electrical activity, ventricular fibrillation, ventricular tachycardia vs others], time from emergency call to physician arrival and patient category) were included. Adjusted for optimism after internal validation, the algorithm had an area under the receiver operating characteristic curve of 0.921 (95% CI 0.918 to 0.924), Brier score of 0.097, calibration intercept of 0.000 (95% CI -0.040 to 0.040) and slope of 1.000 (95% CI 0.977 to 1.023).

CONCLUSIONS

Based on 11 demographic, mission-specific, and physiologic variables, we developed and internally validated a novel severity of illness algorithm for use with patients encountered by physician-staffed helicopter emergency medical services, which may help in future quality improvement.

Spanish vs USA cohort comparison of prehospital trauma scores to predict short-term mortality

BACKGROUND

This study aimed to evaluate three prehospital early warning scores (EWSs): RTS, MGAP and MREMS, to predict short-term mortality in acute life-threatening trauma and injury/illness by comparing United States (US) and Spanish cohorts.

METHODS

A total of 8,854 patients, 8,598/256 survivors/nonsurvivors, comprised the unified cohort. Datasets were randomly divided into training and test sets. Training sets were used to analyse the discriminative power of the scores in terms of the area under the curve (AUC), and the score performance was assessed in the test set in terms of sensitivity (SE), specificity (SP), accuracy (ACC) and balanced accuracy (BAC).

RESULTS

The three scores showed great discriminative power with AUCs>0.90, and no significant differences between cohorts were found. In the test set, RTS/MREMS/MGAP showed SE/SP/ACC/BAC values of 86.0/89.9/89.6/87.1%, 91.0/86.9/87.5/88.5%, and 87.7/82.9/83.4/85.2%, respectively.

CONCLUSIONS

All EWSs showed excellent ability to predict the risk of short-term mortality, independent of the country.

Recognition of Critically Ill Patients by Acute Health Care Providers: A Multicenter Observational Study

OBJECTIVES

Although the Modified Early Warning Score (MEWS) is increasingly being used in the acute care chain to recognize disease severity, its superiority compared with clinical gestalt remains unproven. Therefore, the aim of this study was to compare the accuracy of medical caregivers and MEWS in predicting the development of critical illness.

DESIGN

This was a multicenter observational prospective study.

SETTING

It was performed in a level-1 trauma center with two different sites and emergency departments (EDs) with a combined capacity of about 50.000 patients annually.

PATIENTS

It included all adult patients presented to the ED by Emergency Medical Services (EMS).

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

For all patients, the acute caregivers were asked several standardized questions regarding clinical predicted outcome (clinical gestalt), and the MEWS was calculated. The primary outcome was the occurrence of critical illness, defined as ICU admission, serious adverse events, and mortality within 72 hours. The sensitivity, specificity, and discriminative power of both clinical gestalt and MEWS for the occurrence of critical illness were calculated as the area under the receiver operating characteristic curve (AUROC). Among the total of 800 included patients, 113 patients (14.1%) suffered from critical illness. The specificity for predicting three-day critical illness for all caregivers (for EMS nurses, ED nurses, and physicians) was 93.2%; 97.3%, and 96.8%, respectively, and was significantly (p < 0.01) better than an MEWS score of 3 or higher (70.4%). The sensitivity was significantly lower for EMS and ED nurses, but not significantly different for physicians compared with MEWS. The AUROCs for prediction of 3-day critical illness by both the ED nurses (AUROC = 0.809) and the physicians (AUROC = 0.848) were significantly higher (p = 0.032 and p = 0.010, respectively) compared with MEWS (AUROC = 0.731).

CONCLUSIONS

For patients admitted to the ED by EMS, medical professionals can predict the development of critical illness within 3 days significantly better than the MEWS. Although MEWS is able to correctly predict those patients that become critically ill, its use leads to overestimation due to a substantial number of false positives.

Performance of early warning and risk stratification scores versus clinical judgement in the acute setting: a systematic review

OBJECTIVE

Risk stratification is increasingly based on Early Warning Score (EWS)-based models, instead of clinical judgement. However, it is unknown how risk-stratification models and EWS perform as compared with the clinical judgement of treating acute healthcare providers. Therefore, we performed a systematic review of all available literature evaluating clinical judgement of healthcare providers to the use of risk-stratification models in predicting patients' clinical outcome.

METHODS

Studies comparing clinical judgement and risk-stratification models in predicting outcomes in adult patients presenting at the ED were eligible for inclusion. Outcomes included the need for intensive care unit (ICU) admission; severe adverse events; clinical deterioration and mortality. Risk of bias among the included studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool.

RESULTS

Six studies (6419 participants) were included of which 4 studies were judged to be at high risk of bias. Only descriptive analysis was performed as a meta-analysis was not possible due to few included studies and high clinical heterogeneity. The performance of clinical judgement and risk-stratification models were both moderate in predicting mortality, deterioration and need for ICU admission with area under the curves between 0.70 and 0.89. The performance of clinical judgement did not significantly differ from risk-stratification models in predicting mortality (n=2 studies) or deterioration (n=1 study). However, clinical judgement of healthcare providers was significantly better in predicting the need for ICU admission (n=2) and severe adverse events (n=1 study) as compared with risk-stratification models.

CONCLUSION

Based on limited existing data, clinical judgement has greater accuracy in predicting the need for ICU admission and the occurrence of severe adverse events compared with risk-stratification models in ED patients. However, performance is similar in predicting mortality and deterioration.

PROSPERO REGISTRATION NUMBER

CRD42020218893.

Effect of Designating Emergency Medical Centers for Critical Care on Emergency Medical Service Systems during the COVID-19 Pandemic: A Retrospective Observational Study

During the coronavirus disease 2019 (COVID-19) pandemic, prehospital times were delayed for patients who needed to arrive at the hospital in a timely manner to receive treatment. To address this, in March 2020, the Korean government designated emergency medical centers for critical care (EMC-CC). This study retrospectively analyzed whether this intervention effectively reduced ambulance diversion (AD) and shortened prehospital times using emergency medical service records from 219,763 patients from the Gyeonggi Province, collected between 1 January and 31 December 2020. We included non-traumatic patients aged 18 years or older. We used interrupted time series analysis to investigate the intervention effects on the daily AD rate and compared prehospital times before and after the intervention. Following the intervention, the proportion of patients transported 30–35 km and 50 km or more was 13.8% and 5.7%, respectively, indicating an increased distance compared to before the intervention. Although the change in the AD rate was insignificant, the daily AD rate significantly decreased after the intervention. Prehospital times significantly increased after the intervention in all patients (p < 0.001) and by disease group; all prehospital times except for the scene time of cardiac arrest patients increased. In order to achieve optimal treatment times for critically ill patients in a situation that pushes the limits of the medical system, such as the COVID-19 pandemic, even regional distribution of EMC-CC may be necessary, and priority should be given to the allocation of care for patients with mild symptoms.