To catch a killer: electronic sepsis alert tools reaching a fever pitch?

Views and Uses of Sepsis Digital Alerts in National Health Service Trusts in England: Qualitative Study With Health Care Professionals

BACKGROUND

Sepsis is a common cause of serious illness and death. Sepsis management remains challenging and suboptimal. To support rapid sepsis diagnosis and treatment, screening tools have been embedded into hospital digital systems to appear as digital alerts. The implementation of digital alerts to improve the management of sepsis and deterioration is a complex intervention that has to fit with team workflow and the views and practices of hospital staff. Despite the importance of human decision-making and behavior in optimal implementation, there are limited qualitative studies that explore the views and experiences of health care professionals regarding digital alerts as sepsis or deterioration computerized clinician decision support systems (CCDSSs).

OBJECTIVE

This study aims to explore the views and experiences of health care professionals on the use of sepsis or deterioration CCDSSs and to identify barriers and facilitators to their implementation and use in National Health Service (NHS) hospitals.

METHODS

We conducted a qualitative, multisite study with unstructured observations and semistructured interviews with health care professionals from emergency departments, outreach teams, and intensive or acute units in 3 NHS hospital trusts in England. Data from both interviews and observations were analyzed together inductively using thematic analysis.

RESULTS

A total of 22 health care professionals were interviewed, and 12 observation sessions were undertaken. A total of four themes regarding digital alerts were identified: (1) support decision-making as nested in electronic health records, but never substitute professionals' knowledge and experience; (2) remind to take action according to the context, such as the hospital unit and the job role; (3) improve the alerts and their introduction, by making them more accessible, easy to use, not intrusive, more accurate, as well as integrated across the whole health care system; and (4) contextual factors affecting views and use of alerts in the NHS trusts. Digital alerts are more optimally used in general hospital units with a lower senior decision maker:patient ratio and by health care professionals with experience of a similar technology. Better use of the alerts was associated with quality improvement initiatives and continuous sepsis training. The trusts' features, such as the presence of a 24/7 emergency outreach team, good technological resources, and staffing and teamwork, favored a more optimal use.

CONCLUSIONS

Trust implementation of sepsis or deterioration CCDSSs requires support on multiple levels and at all phases of the intervention, starting from a prego-live analysis addressing organizational needs and readiness. Advancements toward minimally disruptive and smart digital alerts as sepsis or deterioration CCDSSs, which are more accurate and specific but at the same time scalable and accessible, require policy changes and investments in multidisciplinary research.

The Role of Genome Sequencing in Neonatal Intensive Care Units

Genetic diseases disrupt the functionality of an infant's genome during fetal-neonatal adaptation and represent a leading cause of neonatal and infant mortality in the United States. Due to disease acuity, gene locus and allelic heterogeneity, and overlapping and diverse clinical phenotypes, diagnostic genome sequencing in neonatal intensive care units has required the development of methods to shorten turnaround times and improve genomic interpretation. From 2012 to 2021, 31 clinical studies documented the diagnostic and clinical utility of first-tier rapid or ultrarapid whole-genome sequencing through cost-effective identification of pathogenic genomic variants that change medical management, suggest new therapeutic strategies, and refine prognoses. Genomic diagnosis also permits prediction of reproductive recurrence risk for parents and surviving probands. Using implementation science and quality improvement, deployment of a genomic learning healthcare system will contribute to a reduction of neonatal and infant mortality through the integration of genome sequencing into best-practice neonatal intensive care.

Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system

While a growing number of machine learning (ML) systems have been deployed in clinical settings with the promise of improving patient care, many have struggled to gain adoption and realize this promise. Based on a qualitative analysis of coded interviews with clinicians who use an ML-based system for sepsis, we found that, rather than viewing the system as a surrogate for their clinical judgment, clinicians perceived themselves as partnering with the technology. Our findings suggest that, even without a deep understanding of machine learning, clinicians can build trust with an ML system through experience, expert endorsement and validation, and systems designed to accommodate clinicians’ autonomy and support them across their entire workflow.

Electronic nudge tool technology used in the critical care and peri-anaesthetic setting: a scoping review protocol

INTRODUCTION

Electronic clinical decision support (eCDS) tools are used to assist clinical decision making. Using computer-generated algorithms with evidence-based rule sets, they alert clinicians to events that require attention. eCDS tools generating alerts using nudge principles present clinicians with evidence-based clinical treatment options to guide clinician behaviour without restricting freedom of choice. Although eCDS tools have shown beneficial outcomes, challenges exist with regard to their acceptability most likely related to implementation. Furthermore, the pace of progress in this field has allowed little time to effectively evaluate the experience of the intended user. This scoping review aims to examine the development and implementation strategies, and the impact on the end user of eCDS tools that generate alerts using nudge principles, specifically in the critical care and peri-anaesthetic setting.

METHODS AND ANALYSIS

This review will follow the Arksey and O'Malley framework. A search will be conducted of literature published in the last 15 years in MEDLINE, EMBASE, CINAHL, CENTRAL, Web of Science and SAGE databases. Citation screening and data extraction will be performed by two independent reviewers. Extracted data will include context, e-nudge tool type and design features, development, implementation strategies and associated impact on end users.

ETHICS AND DISSEMINATION

This scoping review will synthesise published literature therefore ethical approval is not required. Review findings will be published in topic relevant peer-reviewed journals and associated conferences.

Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing

Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.

Postimplementation Evaluation of a Machine Learning-Based Deterioration Risk Alert to Enhance Sepsis Outcome Improvements

Machine learning-based early warning systems (EWSs) can detect clinical deterioration more accurately than point-score tools. In patients with sepsis, however, the timing and scope of sepsis interventions relative to an advanced EWS alert are not well understood. The objectives of this study were to evaluate the timing and frequency of fluid bolus therapy, new antibiotics, and Do Not Resuscitate (DNR) status relative to the time of an advanced EWS alert. We conducted 2 rounds of chart reviews of patients with an EWS alert admitted to community hospitals of a large integrated health system in Northern California (round 1: n = 21; round 2: n = 47). We abstracted patient characteristics and process measures of sepsis intervention and performed summary statistics. Sepsis decedents were older and sicker at admission and alert time. Most EWS alerts occurred near admission, and most sepsis interventions occurred before the first alert. Of 14 decedents, 12 (86%) had a DNR order before death. Fluid bolus therapy and new intravenous antibiotics frequently occurred before the alert, suggesting a potential overlap between sepsis care in the emergency department and the first alert following admission. Two tactics to minimize alerts that may not motivate new sepsis interventions are (1) locking out the alert during the immediate time after hospital admission; and (2) triaging and reviewing patients with alerts outside of the unit before activating a bedside response. Some decedents may have been on a palliative/end-of-life trajectory, because DNR orders were very common among decedents. Nurse leaders sponsoring or leading machine learning projects should consider tactics to reduce false-positive and clinically meaningless alerts dispatched to clinical staff.

Design and Implementation of a Real-time Monitoring Platform for Optimal Sepsis Care in an Emergency Department: Observational Cohort Study

Background

Sepsis is the leading cause of death in US hospitals. Compliance with bundled care, specifically serial lactates, blood cultures, and antibiotics, improves outcomes but is often delayed or missed altogether in a busy practice environment.

Objective

This study aims to design, implement, and validate a novel monitoring and alerting platform that provides real-time feedback to frontline emergency department (ED) providers regarding adherence to bundled care.

Methods

This single-center, prospective, observational study was conducted in three phases: the design and technical development phase to build an initial version of the platform; the pilot phase to test and refine the platform in the clinical setting; and the postpilot rollout phase to fully implement the study intervention.

Results

During the design and technical development, study team members and stakeholders identified the criteria for patient inclusion, selected bundle measures from the Center for Medicare and Medicaid Sepsis Core Measure for alerting, and defined alert thresholds, message content, delivery mechanisms, and recipients. Additional refinements were made based on 70 provider survey results during the pilot phase, including removing alerts for vasopressor initiation and modifying text in the pages to facilitate patient identification. During the 48 days of the postpilot rollout phase, 15,770 ED encounters were tracked and 711 patient encounters were included in the active monitoring cohort. In total, 634 pages were sent at a rate of 0.98 per attending physician shift. Overall, 38.3% (272/711) patients had at least one page. The missing bundle elements that triggered alerts included: antibiotics 41.6% (136/327), repeat lactate 32.4% (106/327), blood cultures 20.8% (68/327), and initial lactate 5.2% (17/327). Of the missing Sepsis Core Measures elements for which a page was sent, 38.2% (125/327) were successfully completed on time.

Conclusions

A real-time sepsis care monitoring and alerting platform was created for the ED environment. The high proportion of patients with at least one alert suggested the significant potential for such a platform to improve care, whereas the overall number of alerts per clinician suggested a low risk of alarm fatigue. The study intervention warrants a more rigorous evaluation to ensure that the added alerts lead to better outcomes for patients with sepsis.

Implementation of an Automated Sepsis Screening Tool in a Community Hospital Setting

BACKGROUND

Early identification of sepsis remains the greatest barrier to compliance with recommended evidence-based bundles.

PURPOSE

The purpose was to improve the early identification and treatment of sepsis by developing an automated screening tool.

METHODS

Six variables associated with sepsis were identified. Logistic regression was used to weigh the variables, and a predictive model was developed to help identify patients at risk. A retrospective review of 10 792 records of hospitalizations was conducted including 339 cases of sepsis to retrieve data for the model.

RESULTS

The final model resulted an area under the curve of 0.857 (95% CI, 0.850-0.863), suggesting that the screening tool may assist in the early identification of patients developing sepsis.

CONCLUSION

By using artificial intelligence capabilities, we were able to screen 100% of our inpatient population and deliver results directly to the caregiver without any manual intervention by nursing staff.

Evaluating a digital sepsis alert in a London multisite hospital network: a natural experiment using electronic health record data

Objective

The study sought to determine the impact of a digital sepsis alert on patient outcomes in a UK multisite hospital network.

Materials and Methods

A natural experiment utilizing the phased introduction (without randomization) of a digital sepsis alert into a multisite hospital network. Sepsis alerts were either visible to clinicians (patients in the intervention group) or running silently and not visible (the control group). Inverse probability of treatment-weighted multivariable logistic regression was used to estimate the effect of the intervention on individual patient outcomes.

Outcomes

In-hospital 30-day mortality (all inpatients), prolonged hospital stay (≥7 days) and timely antibiotics (≤60 minutes of the alert) for patients who alerted in the emergency department.

Results

The introduction of the alert was associated with lower odds of death (odds ratio, 0.76; 95% confidence interval [CI], 0.70-0.84; n = 21 183), lower odds of prolonged hospital stay ≥7 days (OR, 0.93; 95% CI, 0.88-0.99; n = 9988), and in patients who required antibiotics, an increased odds of receiving timely antibiotics (OR, 1.71; 95% CI, 1.57-1.87; n = 4622).

Discussion

Current evidence that digital sepsis alerts are effective is mixed. In this large UK study, a digital sepsis alert has been shown to be associated with improved outcomes, including timely antibiotics. It is not known whether the presence of alerting is responsible for improved outcomes or whether the alert acted as a useful driver for quality improvement initiatives.

Conclusions

These findings strongly suggest that the introduction of a network-wide digital sepsis alert is associated with improvements in patient outcomes, demonstrating that digital based interventions can be successfully introduced and readily evaluated.

Early Detection of Sepsis With Machine Learning Techniques: A Brief Clinical Perspective

Sepsis is a major cause of death worldwide. Over the past years, prediction of clinically relevant events through machine learning models has gained particular attention. In the present perspective, we provide a brief, clinician-oriented vision on the following relevant aspects concerning the use of machine learning predictive models for the early detection of sepsis in the daily practice: (i) the controversy of sepsis definition and its influence on the development of prediction models; (ii) the choice and availability of input features; (iii) the measure of the model performance, the output, and their usefulness in the clinical practice. The increasing involvement of artificial intelligence and machine learning in health care cannot be disregarded, despite important pitfalls that should be always carefully taken into consideration. In the long run, a rigorous multidisciplinary approach to enrich our understanding in the application of machine learning techniques for the early recognition of sepsis may show potential to augment medical decision-making when facing this heterogeneous and complex syndrome.

Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials

OBJECTIVE

To report the improvements achieved with clinical decision support systems and examine the heterogeneity from pooling effects across diverse clinical settings and intervention targets.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Medline up to August 2019.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES AND METHODS

Randomised or quasi-randomised controlled trials reporting absolute improvements in the percentage of patients receiving care recommended by clinical decision support systems. Multilevel meta-analysis accounted for within study clustering. Meta-regression was used to assess the degree to which the features of clinical decision support systems and study characteristics reduced heterogeneity in effect sizes. Where reported, clinical endpoints were also captured.

RESULTS

In 108 studies (94 randomised, 14 quasi-randomised), reporting 122 trials that provided analysable data from 1 203 053 patients and 10 790 providers, clinical decision support systems increased the proportion of patients receiving desired care by 5.8% (95% confidence interval 4.0% to 7.6%). This pooled effect exhibited substantial heterogeneity (I2=76%), with the top quartile of reported improvements ranging from 10% to 62%. In 30 trials reporting clinical endpoints, clinical decision support systems increased the proportion of patients achieving guideline based targets (eg, blood pressure or lipid control) by a median of 0.3% (interquartile range -0.7% to 1.9%). Two study characteristics (low baseline adherence and paediatric settings) were associated with significantly larger effects. Inclusion of these covariates in the multivariable meta-regression, however, did not reduce heterogeneity.

CONCLUSIONS

Most interventions with clinical decision support systems appear to achieve small to moderate improvements in targeted processes of care, a finding confirmed by the small changes in clinical endpoints found in studies that reported them. A minority of studies achieved substantial increases in the delivery of recommended care, but predictors of these more meaningful improvements remain undefined.

Comparison of Automated Sepsis Identification Methods and Electronic Health Record-based Sepsis Phenotyping: Improving Case Identification Accuracy by Accounting for Confounding Comorbid Conditions

Supplemental Digital Content is available in the text.

To develop and evaluate a novel strategy that automates the retrospective identification of sepsis using electronic health record data.