Sepsis Alerts in Emergency Departments: A Systematic Review of Accuracy and Quality Measure Impact

External validation of the Epic sepsis predictive model in 2 county emergency departments

Objective

To describe the diagnostic characteristics of the proprietary Epic sepsis predictive model best practice advisory (BPA) alert for physicians in the emergency department (ED).

Materials and Methods

The Epic Sepsis Predictive Model v1.0 (ESPMv1), a proprietary algorithm, is intended to improve provider alerting of patients with a likelihood of developing sepsis. This retrospective cohort study conducted at 2 county EDs from January 1, 2023 to December 31, 2023 evaluated the predictive characteristics of the ESPMv1 for 145 885 encounters. Sepsis was defined according to the Sepsis-3 definition with the onset of sepsis defined as an increase in 2 points on the Sequential Organ Function Assessment (SOFA) score in patients with the ordering of at least one blood culture and antibiotic. Alerting occurred at an Epic recommended model threshold of 6.

Results

The ESPMv1 BPA alert was present in 7183 (4.9%) encounters of which 2253 had sepsis, and not present in 138 702 encounters of which 3180 had sepsis. Within a 6-hour time window for sepsis, the ESPMv1 had a sensitivity of 14.7%, specificity of 95.3%, positive predictive value of 7.6%, and negative predictive value of 97.7%. Providers were alerted with a median lead time of 0 minutes (80% CI, -6 hours and 42 minutes to 12 hours and 0 minutes).

Discussion

In our population, the ESPMv1 alerted providers with a median lead time of 0 minutes (80% CI, -6 hours and 42 minutes to 12 hours and 0 minutes) and only alerted providers in half of the cases prior to sepsis occurrence. This suggests that the ESPMv1 alert is adding little assistance to physicians identifying sepsis. With clinicians treating sepsis 50% of the time without an alert, pop-ups can only marginally assist in disease identification.

Conclusions

The ESPMv1 provides suboptimal diagnostic characteristics for undifferentiated patients in a county ED.

Views and Uses of Sepsis Digital Alerts in National Health Service Trusts in England: Qualitative Study With Health Care Professionals

BACKGROUND

Sepsis is a common cause of serious illness and death. Sepsis management remains challenging and suboptimal. To support rapid sepsis diagnosis and treatment, screening tools have been embedded into hospital digital systems to appear as digital alerts. The implementation of digital alerts to improve the management of sepsis and deterioration is a complex intervention that has to fit with team workflow and the views and practices of hospital staff. Despite the importance of human decision-making and behavior in optimal implementation, there are limited qualitative studies that explore the views and experiences of health care professionals regarding digital alerts as sepsis or deterioration computerized clinician decision support systems (CCDSSs).

OBJECTIVE

This study aims to explore the views and experiences of health care professionals on the use of sepsis or deterioration CCDSSs and to identify barriers and facilitators to their implementation and use in National Health Service (NHS) hospitals.

METHODS

We conducted a qualitative, multisite study with unstructured observations and semistructured interviews with health care professionals from emergency departments, outreach teams, and intensive or acute units in 3 NHS hospital trusts in England. Data from both interviews and observations were analyzed together inductively using thematic analysis.

RESULTS

A total of 22 health care professionals were interviewed, and 12 observation sessions were undertaken. A total of four themes regarding digital alerts were identified: (1) support decision-making as nested in electronic health records, but never substitute professionals' knowledge and experience; (2) remind to take action according to the context, such as the hospital unit and the job role; (3) improve the alerts and their introduction, by making them more accessible, easy to use, not intrusive, more accurate, as well as integrated across the whole health care system; and (4) contextual factors affecting views and use of alerts in the NHS trusts. Digital alerts are more optimally used in general hospital units with a lower senior decision maker:patient ratio and by health care professionals with experience of a similar technology. Better use of the alerts was associated with quality improvement initiatives and continuous sepsis training. The trusts' features, such as the presence of a 24/7 emergency outreach team, good technological resources, and staffing and teamwork, favored a more optimal use.

CONCLUSIONS

Trust implementation of sepsis or deterioration CCDSSs requires support on multiple levels and at all phases of the intervention, starting from a prego-live analysis addressing organizational needs and readiness. Advancements toward minimally disruptive and smart digital alerts as sepsis or deterioration CCDSSs, which are more accurate and specific but at the same time scalable and accessible, require policy changes and investments in multidisciplinary research.

Improving Sepsis Outcomes in the Era of Pay-for-Performance and Electronic Quality Measures: A Joint IDSA/ACEP/PIDS/SHEA/SHM/SIDP Position Paper

The Centers for Medicare & Medicaid Services (CMS) introduced the Severe Sepsis/Septic Shock Management Bundle (SEP-1) as a pay-for-reporting measure in 2015 and is now planning to make it a pay-for-performance measure by incorporating it into the Hospital Value-Based Purchasing Program. This joint IDSA/ACEP/PIDS/SHEA/SHM/SIPD position paper highlights concerns with this change. Multiple studies indicate that SEP-1 implementation was associated with increased broad-spectrum antibiotic use, lactate measurements, and aggressive fluid resuscitation for patients with suspected sepsis but not with decreased mortality rates. Increased focus on SEP-1 risks further diverting attention and resources from more effective measures and comprehensive sepsis care. We recommend retiring SEP-1 rather than using it in a payment model and shifting instead to new sepsis metrics that focus on patient outcomes. CMS is developing a community-onset sepsis 30-day mortality electronic clinical quality measure (eCQM) that is an important step in this direction. The eCQM preliminarily identifies sepsis using systemic inflammatory response syndrome (SIRS) criteria, antibiotic administrations or diagnosis codes for infection or sepsis, and clinical indicators of acute organ dysfunction. We support the eCQM but recommend removing SIRS criteria and diagnosis codes to streamline implementation, decrease variability between hospitals, maintain vigilance for patients with sepsis but without SIRS, and avoid promoting antibiotic use in uninfected patients with SIRS. We further advocate for CMS to harmonize the eCQM with the Centers for Disease Control and Prevention's (CDC) Adult Sepsis Event surveillance metric to promote unity in federal measures, decrease reporting burden for hospitals, and facilitate shared prevention initiatives. These steps will result in a more robust measure that will encourage hospitals to pay more attention to the full breadth of sepsis care, stimulate new innovations in diagnosis and treatment, and ultimately bring us closer to our shared goal of improving outcomes for patients.

Evaluation of evolving sepsis screening criteria in discriminating suspected sepsis and mortality among adult patients admitted to the intensive care unit

BACKGROUND

Institutions struggle with successful use of sepsis alerts within electronic health records.

OBJECTIVE

Test the association of sepsis screening measurement criteria in discrimination of mortality and detection of sepsis in a large dataset.

DESIGN

Retrospective, cohort study using a large United States (U.S.) intensive care database. The Institutional Review Board exempt status was obtained from Kansas University Medical Center Human Research Protection Program (10-1-2015).

SETTING

334 U.S. hospitals participating in the eICU Research Institute.

PARTICIPANTS

Nine hundred twelve thousand five hundred and nine adult intensive care admissions from 183 hospitals.

METHODS

Exposures included: systemic inflammatory response syndrome criteria ≥ 2 (Sepsis-1); systemic inflammatory response syndrome criteria with organ failure criteria ≥ 3.5 points (Sepsis-2); and sepsis-related organ failure assessment score ≥ 2 and quick score ≥ 2 (Sepsis-3). Discrimination of outcomes was determined with/without (adjusted/unadjusted) baseline risk exposure to a model. The receiver operating characteristic curve (AUROC) and odds ratios (ORs) for each decile of baseline risk of sepsis or death were assessed.

RESULTS

Within the eligible cohort of 912,509, a total of 86,219 (9.4 %) patients did not survive their hospital stay and 186,870 (20.5 %) met the definition of suspected sepsis. For suspected sepsis discrimination, Sepsis-2 (unadjusted AUROC 0.67, 99 % CI: 0.66-0.67 and adjusted AUROC 0.77, 99 % CI: 0.77-0.77) outperformed Sepsis-3 (SOFA unadjusted AUROC 0.61, 99 % CI: 0.61-0.61 and adjusted AUROC 0.74, 99 % CI: 0.74-0.74) (qSOFA unadjusted AUROC 0.59, 99 % CI: 0.59-0.60 and adjusted AUROC 0.73, 99 % CI: 0.73-0.73). Sepsis-2 also outperformed Sepsis-1 (unadjusted AUROC 0.58, 99 % CI: 0.58-0.58 and adjusted AUROC 0.73, 99 % CI: 0.73-0.73). In between differences of AUROCs were statistically significantly different. Sepsis-2 ORs were higher for the outcome of suspected sepsis when considering deciles of risk than the other measurement systems.

CONCLUSIONS AND RELEVANCE

Sepsis-2 outperformed other systems in suspected sepsis detection and was comparable to SOFA in prognostic accuracy of mortality in adult intensive care patients.

Prevalence of electronic screening for sepsis in National Health Service acute hospitals in England

Sepsis is a worldwide public health problem. Rapid identification is associated with improved patient outcomes-if followed by timely appropriate treatment.

OBJECTIVES

Describe digital sepsis alerts (DSAs) in use in English National Health Service (NHS) acute hospitals.

METHODS

A Freedom of Information request surveyed acute NHS Trusts on their adoption of electronic patient records (EPRs) and DSAs.

RESULTS

Of the 99 Trusts that responded, 84 had an EPR. Over 20 different EPR system providers were identified as operational in England. The most common providers were Cerner (21%). System C, Dedalus and Allscripts Sunrise were also relatively common (13%, 10% and 7%, respectively). 70% of NHS Trusts with an EPR responded that they had a DSA; most of these use the National Early Warning Score (NEWS2). There was evidence that the EPR provider was related to the DSA algorithm. We found no evidence that Trusts were using EPRs to introduce data driven algorithms or DSAs able to include, for example, pre-existing conditions that may be known to increase risk.Not all Trusts were willing or able to provide details of their EPR or the underlying algorithm.

DISCUSSION

The majority of NHS Trusts use an EPR of some kind; many use a NEWS2-based DSA in keeping with national guidelines.

CONCLUSION

Many English NHS Trusts use DSAs; even those using similar triggers vary and many recreate paper systems. Despite the proliferation of machine learning algorithms being developed to support early detection of sepsis, there is little evidence that these are being used to improve personalised sepsis detection.

Introduction of an emergency medicine pharmacist-led sepsis alert response system in the emergency department: A cohort study

OBJECTIVE

To determine effects of implementing a sepsis alert response system in the ED that included early intervention by emergency medicine (EM) pharmacists.

METHODS

A prospective cohort (8 February 2016 to 28 February 2018) of patients after implementation of a sepsis alert response system in an Australian ED was compared to a retrospective cohort (3 January 2015 to 7 February 2016) of patients with sepsis who presented during EM pharmacist working hours and were admitted to the ICU.

RESULTS

There were 184 patients, including 80 patients pre- and 104 patients post-implementation. The post-intervention cohort was triaged at a higher acuity, had higher quick Sepsis-related Organ Failure Assessment (qSOFA) scores and higher initial lactate measurements. After the intervention, antimicrobial agents were administered to patients within 60 min of presentation more often (21 [26.3%] to 85 [81.7%], P < 0.001). After adjusting for presenting triage category, admission lactate and presenting qSOFA scores, this association remained significant (adjusted odds ratio 9.99; 95% confidence interval 4.7-21.3). Significant improvements were observed for proportion of patients who had intravenous fluids initiated within 60 min (47.5% vs 72.1%); proportion of patients who had serum lactate measured within 60 min (50.0% vs 77.9%) and proportion of patients who had blood cultures performed within 60 min (52.5% vs 85.6%).

CONCLUSION

Implementation of a sepsis alert response that included early involvement of the EM pharmacist was associated with improvement in time to antimicrobials and other components of the sepsis bundle. An upfront, multidisciplinary approach to patients presenting to the ED with suspected sepsis should be considered more broadly.

Audit of a computerized version of the Manchester triage system and a SIRS-based system for the detection of sepsis at triage in the emergency department

BACKGROUND AND IMPORTANCE

Different triage systems can be used to screen for sepsis and are often incorporated into local electronic health records. Often the design and interface of these digitalizations are not audited, possibly leading to deleterious effects on screening test performance.

OBJECTIVE

To audit a digital version of the MTS for detection of sepsis during triage in the ED.

DESIGN

A single-center retrospective study SETTINGS AND PARTICIPANTS: Patients (n=29766) presenting to an ED of a tertiary-care center who received formal triage were included.

OUTCOME MEASURES AND ANALYSIS

Calculated performance measures included sensitivity, specificity, likelihood ratios, and AUC for the detection of sepsis. Errors in the application of the specific sepsis discriminator of the MTS were recorded.

MAIN RESULTS

A total of 189 (0.7%) subjects met the Sepsis-3 criteria, with 47 cases meeting the criteria for septic shock. The MTS had a low sensitivity of 47.6% (95% CI 40.3 to 55.0) for allocating sepsis patients to the correct triage category. However, specificity was high at 99.4% (95% CI 99.3 to 99.5).

Impact of an electronic alert system for pediatric sepsis screening a tertiary hospital experience

This study aimed to assess the potential impact of implementing an electronic alert system (EAS) for systemic inflammatory syndrome (SIRS) and sepsis in pediatric patients mortality. This retrospective study had a pre and post design. We enrolled patients aged ≤ 14 years who were diagnosed with sepsis/severe sepsis upon admission to the pediatric intensive care unit (PICU) of our tertiary hospital from January 2014 to December 2018. We implemented an EAS for the patients with SIRS/sepsis. The patients who met the inclusion criteria pre-EAS implementation comprised the control group, and the group post-EAS implementation was the experimental group. Mortality was the primary outcome, while length of stay (LOS) and mechanical ventilation in the first hour were the secondary outcomes. Of the 308 enrolled patients, 147 were in the pre-EAS group and 161 in the post-EAS group. In terms of mortality, 44 patients in the pre-EAS group and 28 in the post-EAS group died (p 0.011). The average LOS in the PICU was 7.9 days for the pre-EAS group and 6.8 days for the post-EAS group (p 0.442). Considering the EAS initiation time as the "zero time", early recognition of SIRS and sepsis via the EAS led to faster treatment interventions in post-EAS group, which included fluid boluses with median (25th, 75th percentile) time of 107 (37, 218) min vs. 30 (11,112) min, p < 0.001) and time to initiate antimicrobial therapy median (25th, 75th percentile) of 170.5 (66,320) min vs. 131 (53,279) min, p 0.042). The difference in mechanical ventilation in the first hour of admission was not significant between the groups (25.17% vs. 24.22%, p 0.895). The implementation of the EAS resulted in a statistically significant reduction in the mortality rate among the patients admitted to the PICU in our study. An EAS can play an important role in saving lives and subsequent reduction in healthcare costs. Further enhancement of systematic screening is therefore highly recommended to improve the prognosis of pediatric SIRS and sepsis. The implementation of the EAS, warrants further validation in multicenter or national studies.

Effectiveness of automated alerting system compared to usual care for the management of sepsis

There is a large body of evidence showing that delayed initiation of sepsis bundle is associated with adverse clinical outcomes in patients with sepsis. However, it is controversial whether electronic automated alerts can help improve clinical outcomes of sepsis. Electronic databases are searched from inception to December 2021 for comparative effectiveness studies comparing automated alerts versus usual care for the management of sepsis. A total of 36 studies are eligible for analysis, including 6 randomized controlled trials and 30 non-randomized studies. There is significant heterogeneity in these studies concerning the study setting, design, and alerting methods. The Bayesian meta-analysis by using pooled effects of non-randomized studies as priors shows a beneficial effect of the alerting system (relative risk [RR]: 0.71; 95% credible interval: 0.62 to 0.81) in reducing mortality. The automated alerting system shows less beneficial effects in the intensive care unit (RR: 0.90; 95% CI: 0.73-1.11) than that in the emergency department (RR: 0.68; 95% CI: 0.51-0.90) and ward (RR: 0.71; 95% CI: 0.61-0.82). Furthermore, machine learning-based prediction methods can reduce mortality by a larger magnitude (RR: 0.56; 95% CI: 0.39-0.80) than rule-based methods (RR: 0.73; 95% CI: 0.63-0.85). The study shows a statistically significant beneficial effect of using the automated alerting system in the management of sepsis. Interestingly, machine learning monitoring systems coupled with better early interventions show promise, especially for patients outside of the intensive care unit.

Big data analyses and individual health profiling in the arena of rheumatic and musculoskeletal diseases (RMDs)

Health care processes are under constant development and will need to embrace advances in technology and health science aiming to provide optimal care. Considering the perspective of increasing treatment options for people with rheumatic and musculoskeletal diseases, but in many cases not reaching all treatment targets that matter to patients, care systems bare potential to improve on a holistic level. This review provides an overview of systems and technologies under evaluation over the past years that show potential to impact diagnosis and treatment of rheumatic diseases in about 10 years from now. We summarize initiatives and studies from the field of electronic health records, biobanking, remote monitoring, and artificial intelligence. The combination and implementation of these opportunities in daily clinical care will be key for a new era in care of our patients. This aims to inform rheumatologists and healthcare providers concerned with chronic inflammatory musculoskeletal conditions about current important and promising developments in science that might substantially impact the management processes of rheumatic diseases in the 2030s.

Coprescribing of opioids and high-risk medications in the USA: a cross-sectional study with data from national ambulatory and emergency department settings

OBJECTIVE

Describe trends in opioid plus high-risk medication coprescribing in the USA.

DESIGN

Analyses of serial, cross-sectional, nationally representative data of the National Ambulatory Medical Care Survey (NAMCS) over 2007-2016 and the National Hospital Ambulatory Medical Care Survey (NHAMCS) over 2007-2018.

SETTING

US ambulatory (NAMCS) and emergency department (ED, NHAMCS) settings.

PARTICIPANTS

Patient visits in which the patient was 18 years and older with an opioid prescription in the NAMCS or NHAMCS databases.

PRIMARY AND SECONDARY OUTCOME MEASURES

Frequency of opioid plus high-risk medication coprescribing.

RESULTS

From a combined sample of 700 499 visits over 2007-2018, there were 105 720 visits (15.1%) where opioids were prescribed. n=31 825 were from NAMCS and n=73 895 were from NHAMCS. The mean prevalence of coprescription of opioids and high-risk medications for the combined NAMCS and NHAMCS sample was 18.4% in 2007, peaked at 33.2% in 2014 and declined to 23.8% in 2016. Compared with adults receiving opioid prescriptions alone, those coprescribed opioids and high-risk medications were older, more likely female, white and using private or Medicare insurance (p<0.0001).

CONCLUSIONS

Coprescribing is more common in ambulatory than ED settings and has been declining, yet one in four patient visits where opioids were prescribed resulted in coprescribed, high-risk medications in 2016. Efforts and research to help lower the rates of high-risk prescribing are needed.

Recent Advances in Bedside Device-Based Early Detection of Sepsis

Early detection of sepsis is challenging to achieve with current diagnostic methods, leading to expenditures of $27 billion annually in the United States with significant associated mortality. Various scoring systems have been proposed such as the sequential organ failure assessment (SOFA) and systemic inflammatory response syndrome (SIRS) criteria for identification of sepsis, but their sensitivities range from 60% to 70% when used in the emergency department triage. Other methods for the recognition of sepsis may rely on laboratory work, in addition to vitals monitoring, and are often outpaced by the development of sepsis. Automated alerts have not shown any reduction in mortality thus far. New technology may fill a critical gap in the early detection of sepsis. The ideal bedside screening device for would demonstrate rapid time to result, high portability, and high sensitivity to not miss cases, but also reasonable specificity to prevent provider fatigue from excessive false alerts. Non-invasive end-organ perfusion devices analyzing lactate and capillary refill time (CRT) tend to perform well in speed and portability, but may be less sensitive. Biomarker devices demonstrate a wider array of performance metrics. Those analyzing a single biomarker tend to be more sensitive but are less specific to the diagnosis of sepsis than technologies that assess multiple biomarkers, which in turn have lower sensitivity. Additionally, biomarker devices are generally invasive requiring blood samples, which may or may not be feasible in all patients especially when serial draws are needed. Sepsis is a complex disease process and most likely will require a combination of improved technology in addition to vital signs and high-risk patient history for better recognition. This review examines recent advances in the device-based early detection of sepsis between 2017 and 2020 with emphasis on bedside diagnostics, divided into markers of perfusion and biomarkers commonly implicated in sepsis.

Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation

Accurate stratification of sepsis can effectively guide the triage of patient care and shared decision making in the emergency department (ED). However, previous research on sepsis identification models focused mainly on ICU patients, and discrepancies in model performance between the development and external validation datasets are rarely evaluated. The aim of our study was to develop and externally validate a machine learning model to stratify sepsis patients in the ED. We retrospectively collected clinical data from two geographically separate institutes that provided a different level of care at different time periods. The Sepsis-3 criteria were used as the reference standard in both datasets for identifying true sepsis cases. An eXtreme Gradient Boosting (XGBoost) algorithm was developed to stratify sepsis patients and the performance of the model was compared with traditional clinical sepsis tools; quick Sequential Organ Failure Assessment (qSOFA) and Systemic Inflammatory Response Syndrome (SIRS). There were 8296 patients (1752 (21%) being septic) in the development and 1744 patients (506 (29%) being septic) in the external validation datasets. The mortality of septic patients in the development and validation datasets was 13.5% and 17%, respectively. In the internal validation, XGBoost achieved an area under the receiver operating characteristic curve (AUROC) of 0.86, exceeding SIRS (0.68) and qSOFA (0.56). The performance of XGBoost deteriorated in the external validation (the AUROC of XGBoost, SIRS and qSOFA was 0.75, 0.57 and 0.66, respectively). Heterogeneity in patient characteristics, such as sepsis prevalence, severity, age, comorbidity and infection focus, could reduce model performance. Our model showed good discriminative capabilities for the identification of sepsis patients and outperformed the existing sepsis identification tools. Implementation of the ML model in the ED can facilitate timely sepsis identification and treatment. However, dataset discrepancies should be carefully evaluated before implementing the ML approach in clinical practice. This finding reinforces the necessity for future studies to perform external validation to ensure the generalisability of any developed ML approaches.

Protective Effect of Arbidol Against Pulmonary Fibrosis and Sepsis in Mice

From the perspective of epidemiology, viral immunology and current clinical research, pulmonary fibrosis may become one of the complications of patients with Coronavirus Disease 2019 (COVID-19). Cytokine storm is a major cause of new coronavirus death. The purpose of this study was to explore the effects of antiviral drug arbidol on cytokine storm and pulmonary fibrosis. Here, we use a mouse model of bleomycin-induced pulmonary fibrosis and a mouse model of fecal dilution-induced sepsis to evaluate the effects of arbidol on pulmonary fibrosis and cytokine storm. The results showed that arbidol significantly reduced the area of pulmonary fibrosis and improved lung function (reduced inspiratory resistance, lung dynamic compliance and forced vital capacity increased). Treatment with arbidol promoted reduced sepsis severity 48 h after sepsis induction, based on weight, murine sepsis score and survival rate. Arbidol observably alleviates inflammatory infiltrates and injury in the lungs and liver. Finally, we also found that arbidol reduced serum levels of pro-inflammatory factors such as TNF-α and IL-6 induced by fecal dilution. In conclusion, our results indicate that arbidol can alleviate the severity of pulmonary fibrosis and sepsis, and provide some reference for the treatment of cytokine storm and sequelae of pulmonary fibrosis in patients with COVID-19.