Introduction of an electronic physiological early warning system: effects on mortality and length of stay

Effect of Digital Early Warning Scores on Hospital Vital Sign Observation Protocol Adherence: Stepped-Wedge Evaluation

BACKGROUND

Early warning scores (EWS) are routinely used in hospitals to assess a patient's risk of deterioration. EWS are traditionally recorded on paper observation charts but are increasingly recorded digitally. In either case, evidence for the clinical effectiveness of such scores is mixed, and previous studies have not considered whether EWS leads to changes in how deteriorating patients are managed.

OBJECTIVE

This study aims to examine whether the introduction of a digital EWS system was associated with more frequent observation of patients with abnormal vital signs, a precursor to earlier clinical intervention.

METHODS

We conducted a 2-armed stepped-wedge study from February 2015 to December 2016, over 4 hospitals in 1 UK hospital trust. In the control arm, vital signs were recorded using paper observation charts. In the intervention arm, a digital EWS system was used. The primary outcome measure was time to next observation (TTNO), defined as the time between a patient's first elevated EWS (EWS ≥3) and subsequent observations set. Secondary outcomes were time to death in the hospital, length of stay, and time to unplanned intensive care unit admission. Differences between the 2 arms were analyzed using a mixed-effects Cox model. The usability of the system was assessed using the system usability score survey.

RESULTS

We included 12,802 admissions, 1084 in the paper (control) arm and 11,718 in the digital EWS (intervention) arm. The system usability score was 77.6, indicating good usability. The median TTNO in the control and intervention arms were 128 (IQR 73-218) minutes and 131 (IQR 73-223) minutes, respectively. The corresponding hazard ratio for TTNO was 0.99 (95% CI 0.91-1.07; P=.73).

CONCLUSIONS

We demonstrated strong clinical engagement with the system. We found no difference in any of the predefined patient outcomes, suggesting that the introduction of a highly usable electronic system can be achieved without impacting clinical care. Our findings contrast with previous claims that digital EWS systems are associated with improvement in clinical outcomes. Future research should investigate how digital EWS systems can be integrated with new clinical pathways adjusting staff behaviors to improve patient outcomes.

Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review

OBJECTIVE

Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes.

MATERIALS AND METHODS

The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus.

RESULTS

Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome.

CONCLUSIONS

Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.

Implementation of an automated early warning scoring system in a surgical ward: Practical use and effects on patient outcomes

Introduction

Early warning scores (EWS) are being increasingly embedded in hospitals over the world due to their promise to reduce adverse events and improve the outcomes of clinical patients.

The aim of this study was to evaluate the clinical use of an automated modified EWS (MEWS) for patients after surgery.

Methods

This study conducted retrospective before-and-after comparative analysis of non-automated and automated MEWS for patients admitted to the surgical high-dependency unit in a tertiary hospital. Operational outcomes included number of recorded assessments of the individual MEWS elements, number of complete MEWS assessments, as well as adherence rate to related protocols. Clinical outcomes included hospital length of stay, in-hospital and 28-day mortality, and ICU readmission rate.

Results

Recordings in the electronic medical record from the control period contained 7929 assessments of MEWS elements and were performed in 320 patients. Recordings from the intervention period contained 8781 assessments of MEWS elements in 273 patients, of which 3418 were performed with the automated EWS system. During the control period, 199 (2.5%) complete MEWS were recorded versus 3991 (45.5%) during intervention period. With the automated MEWS systems, the percentage of missing assessments and the time until the next assessment for patients with a MEWS of ≥2 decreased significantly. The protocol adherence improved from 1.1% during the control period to 25.4% when the automated MEWS system was involved. There were no significant differences in clinical outcomes.

Conclusion

Implementation of an automated EWS system on a surgical high dependency unit improves the number of complete MEWS assessments, registered vital signs, and adherence to the EWS hospital protocol. However, this positive effect did not translate into a significant decrease in mortality, hospital length of stay, or ICU readmissions. Future research and development on automated EWS systems should focus on data management and technology interoperability.

Impact of Electronic Versus Paper Vital Sign Observations on Length of Stay in Trauma Patients: Stepped-Wedge, Cluster Randomized Controlled Trial

Background

Electronic recording of vital sign observations (e-Obs) has become increasingly prevalent in hospital care. The evidence of clinical impact for these systems is mixed.

Objective

The objective of our study was to assess the effect of e-Obs versus paper documentation (paper) on length of stay (time between trauma unit admission and “fit to discharge”) for trauma patients.

Methods

A single-center, randomized stepped-wedge study of e-Obs against paper was conducted in two 26-bed trauma wards at a medium-sized UK teaching hospital. Randomization of the phased intervention order to 12 study areas was computer generated. The primary outcome was length of stay.

Results

A total of 1232 patient episodes were randomized (paper: 628, e-Obs: 604). There were 37 deaths in hospital: 21 in the paper arm and 16 in the e-Obs arm. For discharged patients, the median length of stay was 5.4 (range: 0.2-79.0) days on the paper arm and 5.6 (range: 0.1-236.7) days on the e-Obs arm. Competing risks regression analysis for time to discharge showed no difference between the treatment arms (subhazard ratio: 1.05; 95% CI 0.82-1.35; P=.68). A greater proportion of patient episodes contained an Early Warning Score (EWS) ≥3 using the e-Obs system than using paper (subhazard ratio: 1.63; 95% CI 1.28-2.09; P<.001). However, there was no difference in the time to the subsequent observation, “escalation time” (hazard ratio 1.05; 95% CI 0.80-1.38; P=.70).

Conclusions

The phased introduction of an e-Obs documentation system was not associated with a change in length of stay. A greater proportion of patient episodes contained an EWS≥3 using the e-Obs system, but this was not associated with a change in “escalation time.”

Trial Registration

ISRCTN Registry ISRCTN91040762; http://www.isrctn.com/ISRCTN91040762 (Archived by WebCite at http://www.webcitation.org/72prakGTU)

A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: A multi-centre database study

Aims

To compare the ability of the National Early Warning Score (NEWS) and the National Early Warning Score 2 (NEWS2) to identify patients at risk of in-hospital mortality and other adverse outcomes.

Methods

We undertook a multi-centre retrospective observational study at five acute hospitals from two UK NHS Trusts. Data were obtained from completed adult admissions who were not fit enough to be discharged alive on the day of admission. Diagnostic coding and oxygen prescriptions were used to identify patients with type II respiratory failure (T2RF). The primary outcome was in-hospital mortality within 24 h of a vital signs observation. Secondary outcomes included unanticipated intensive care unit admission or cardiac arrest within 24 h of a vital signs observation. Discrimination was assessed using the c-statistic.

Results

Among 251,266 adult admissions, 48,898 were identified to be at risk of T2RF by diagnostic coding. In this group, NEWS2 showed statistically significant lower discrimination (c-statistic, 95% CI) for identifying in-hospital mortality within 24 h (0.860, 0.857–0.864) than NEWS (0.881, 0.878-0.884). For 1394 admissions with documented T2RF, discrimination was similar for both systems: NEWS2 (0.841, 0.827-0.855), NEWS (0.862, 0.848–0.875). For all secondary endpoints, NEWS2 showed no improvements in discrimination.

Conclusions

NEWS2 modifications to NEWS do not improve discrimination of adverse outcomes in patients with documented T2RF and decrease discrimination in patients at risk of T2RF. Further evaluation of the relationship between SpO₂ values, oxygen therapy and risk should be investigated further before wide-scale adoption of NEWS2.

The Predictive Value of Integrated Pulmonary Index after Off-Pump Coronary Artery Bypass Grafting: A Prospective Observational Study

Background

The early warning scores may increase the safety of perioperative period. The objective of this study was to assess the diagnostic and predictive role of Integrated Pulmonary Index (IPI) after off-pump coronary artery bypass grafting (OPCAB).

Materials and Methods

Forty adult patients undergoing elective OPCAB were enrolled into a single-center prospective observational study. We assessed respiratory function using IPI that includes oxygen saturation, end-tidal CO₂, respiratory rate, and pulse rate. In addition, we evaluated blood gas analyses and hemodynamics, including ECG, invasive arterial pressure, and cardiac index. The measurements were performed after transfer to the intensive care unit, after spontaneous breathing trial and at 2, 6, 12, and 18 h after extubation.

Results and Discussion

The value of IPI registered during respiratory support correlated weakly with cardiac index (rho = 0.4; p = 0.04) and ScvO₂ (rho = 0.4, p = 0.02). After extubation, IPI values decreased significantly, achieving a minimum by 18 h. The IPI value ≤9 at 6 h after extubation was a predictor of complicated early postoperative period (AUC = 0.71; p = 0.04) observed in 13 patients.

Conclusion

In off-pump coronary surgery, the IPI decreases significantly after tracheal extubation and may predict postoperative complications.

Early warning score: An indicator of adverse outcomes in postoperative patients on a gynecologic oncology service

OBJECTIVE

In 2014, our hospital implemented an early warning score (EWS) to identify inpatients at risk for clinical deterioration. EWS≥8 is associated with ≥10% mortality in medical admissions. Since postoperative hemodynamic changes may alter EWS, we evaluated EWS in post-laparotomy patients.

METHODS

Gynecologic oncology patients admitted for laparotomy from 9/1/2014 to 7/31/2015 were categorized by highest EWS during admission: <5, 5-7, and ≥8. The primary outcome was a composite including death, ICU transfer, rapid response team activation, pulmonary embolus, sepsis, and reoperation. For patients with the composite, highest EWS prior to that outcome was evaluated. Secondary outcomes were length of stay (LOS), readmission, and transfusion. Groups were compared using chi-square test for trend, analysis of variance, and Kruskal-Wallis tests. A receiver operating characteristic (ROC) curve estimated the association between EWS and the composite outcome.

RESULTS

411 patients were included: 217 (52.8%) with EWS<5, 151 (36.7%) with EWS 5-7, and 43 (10.5%) with EWS≥8. The composite occurred in 32.6% of patients with EWS≥8, 7.3% with EWS 5-7, and 0% with EWS<5 (p<0.01). EWS≥8 was associated with longer LOS, higher readmission rate, and more transfusions. For the composite, the area under the ROC curve was 0.89 (95% CI 0.84-0.94). EWS≥5 had 100% sensitivity and 56.2% specificity for the primary outcome; EWS≥8 had 56.0% sensitivity and 92.5% specificity for the primary outcome.

CONCLUSIONS

EWS≥5 after laparotomy is associated with adverse outcomes. Future studies should evaluate the ability of EWS to predict and prevent these outcomes.

Evaluation of the effects of implementing an electronic early warning score system: protocol for a stepped wedge study

Background

An Early Warning Score is a clinical risk score based upon vital signs intended to aid recognition of patients in need of urgent medical attention. The use of an escalation of care policy based upon an Early Warning Score is mandated as the standard of practice in British hospitals. Electronic systems for recording vital sign observations and Early Warning Score calculation offer theoretical benefits over paper-based systems. However, the evidence for their clinical benefit is limited. Previous studies have shown inconsistent results. The majority have employed a “before and after” study design, which may be strongly confounded by simultaneously occurring events. This study aims to examine how the implementation of an electronic early warning score system, System for Notification and Documentation (SEND), affects the recognition of clinical deterioration occurring in hospitalised adult patients.

Methods

This study is a non-randomised stepped wedge evaluation carried out across the four hospitals of the Oxford University Hospitals NHS Trust, comparing charting on paper and charting using SEND. We assume that more frequent monitoring of acutely ill patients is associated with better recognition of patient deterioration.

The primary outcome measure is the time between a patient’s first observations set with an Early Warning Score above the alerting threshold and their subsequent set of observations. Secondary outcome measures are in-hospital mortality, cardiac arrest and Intensive Care admission rates, hospital length of stay and system usability measured using the System Usability Scale. We will also measure Intensive Care length of stay, Intensive Care mortality, Acute Physiology and Chronic Health Evaluation (APACHE) II acute physiology score on admission, to examine whether the introduction of SEND has any effect on Intensive Care-related outcomes.

Discussion

The development of this protocol has been informed by guidance from the Agency for Healthcare Research and Quality (AHRQ) Health Information Technology Evaluation Toolkit and Delone and McLeans’s Model of Information System Success. Our chosen trial design, a stepped wedge study, is well suited to the study of a phased roll out. The choice of primary endpoint is challenging. We have selected the time from the first triggering observation set to the subsequent observation set. This has the benefit of being easy to measure on both paper and electronic charting and having a straightforward interpretation. We have collected qualitative measures of system quality via a user questionnaire and organisational descriptors to help readers understand the context in which SEND has been implemented.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-016-0257-8) contains supplementary material, which is available to authorized users.

SEND: a system for electronic notification and documentation of vital sign observations

BACKGROUND

Recognising the limitations of a paper-based approach to documenting vital sign observations and responding to national clinical guidelines, we have explored the use of an electronic solution that could improve the quality and safety of patient care. We have developed a system for recording vital sign observations at the bedside, automatically calculating an Early Warning Score, and saving data such that it is accessible to all relevant clinicians within a hospital trust. We have studied current clinical practice of using paper observation charts, and attempted to streamline the process. We describe our user-focussed design process, and present the key design decisions prior to describing the system in greater detail.

RESULTS

The system has been deployed in three pilot clinical areas over a period of 9 months. During this time, vital sign observations were recorded electronically using our system. Analysis of the number of observations recorded (21,316 observations) and the number of active users (111 users) confirmed that the system is being used for routine clinical observations. Feedback from clinical end-users was collected to assess user acceptance of the system. This resulted in a System Usability Scale score of 77.8, indicating high user acceptability.

CONCLUSIONS

Our system has been successfully piloted, and is in the process of full implementation throughout adult inpatient clinical areas in the Oxford University Hospitals. Whilst our results demonstrate qualitative acceptance of the system, its quantitative effect on clinical care is yet to be evaluated.