Adjusting Early Warning Score by clinical assessment: a study protocol for a Danish cluster-randomised, multicentre study of an Individual Early Warning Score (I-EWS)

Real-Time Machine Learning Alerts to Prevent Escalation of Care: A Nonrandomized Clustered Pragmatic Clinical Trial

OBJECTIVES

Machine learning algorithms can outperform older methods in predicting clinical deterioration, but rigorous prospective data on their real-world efficacy are limited. We hypothesized that real-time machine learning generated alerts sent directly to front-line providers would reduce escalations.

DESIGN

Single-center prospective pragmatic nonrandomized clustered clinical trial.

SETTING

Academic tertiary care medical center.

PATIENTS

Adult patients admitted to four medical-surgical units. Assignment to intervention or control arms was determined by initial unit admission.

INTERVENTIONS

Real-time alerts stratified according to predicted likelihood of deterioration sent either to the primary team or directly to the rapid response team (RRT). Clinical care and interventions were at the providers' discretion. For the control units, alerts were generated but not sent, and standard RRT activation criteria were used.

MEASUREMENTS AND MAIN RESULTS

The primary outcome was the rate of escalation per 1000 patient bed days. Secondary outcomes included the frequency of orders for fluids, medications, and diagnostic tests, and combined in-hospital and 30-day mortality. Propensity score modeling with stabilized inverse probability of treatment weight (IPTW) was used to account for differences between groups. Data from 2740 patients enrolled between July 2019 and March 2020 were analyzed (1488 intervention, 1252 control). Average age was 66.3 years and 1428 participants (52%) were female. The rate of escalation was 12.3 vs. 11.3 per 1000 patient bed days (difference, 1.0; 95% CI, -2.8 to 4.7) and IPTW adjusted incidence rate ratio 1.43 (95% CI, 1.16-1.78; p < 0.001). Patients in the intervention group were more likely to receive cardiovascular medication orders (16.1% vs. 11.3%; 4.7%; 95% CI, 2.1-7.4%) and IPTW adjusted relative risk (RR) (1.74; 95% CI, 1.39-2.18; p < 0.001). Combined in-hospital and 30-day-mortality was lower in the intervention group (7% vs. 9.3%; -2.4%; 95% CI, -4.5% to -0.2%) and IPTW adjusted RR (0.76; 95% CI, 0.58-0.99; p = 0.045).

CONCLUSIONS

Real-time machine learning alerts do not reduce the rate of escalation but may reduce mortality.

Translating Data Analytics Into Improved Spine Surgery Outcomes: A Roadmap for Biomedical Informatics Research in 2021

STUDY DESIGN

Narrative review.

OBJECTIVES

There is growing interest in the use of biomedical informatics and data analytics tools in spine surgery. Yet despite the rapid growth in research on these topics, few analytic tools have been implemented in routine spine practice. The purpose of this review is to provide a health information technology (HIT) roadmap to help translate data assets and analytics tools into measurable advances in spine surgical care.

METHODS

We conducted a narrative review of PubMed and Google Scholar to identify publications discussing data assets, analytical approaches, and implementation strategies relevant to spine surgery practice.

RESULTS

A variety of data assets are available for spine research, ranging from commonly used datasets, such as administrative billing data, to emerging resources, such as mobile health and biobanks. Both regression and machine learning techniques are valuable for analyzing these assets, and researchers should recognize the particular strengths and weaknesses of each approach. Few studies have focused on the implementation of HIT, and a variety of methods exist to help translate analytic tools into clinically useful interventions. Finally, a number of HIT-related challenges must be recognized and addressed, including stakeholder acceptance, regulatory oversight, and ethical considerations.

CONCLUSIONS

Biomedical informatics has the potential to support the development of new HIT that can improve spine surgery quality and outcomes. By understanding the development life-cycle that includes identifying an appropriate data asset, selecting an analytic approach, and leveraging an effective implementation strategy, spine researchers can translate this potential into measurable advances in patient care.

Nurses' Experiences and Perceptions of two Early Warning Score systems to Identify Patient Deterioration-A Focus Group Study

AIMS

To explore Registered Nurses' experiences and perceptions with National Early Warning Score and Individual Early Warning Score to identify patient deterioration.

DESIGN

A qualitative exploratory design.

METHODS

Six focus groups were conducted at six Danish hospitals from February to June 2019. Registered Nurses from both medical, surgical and emergency departments participated. The focus groups were analysed using content analysis.

RESULTS

One theme and four categories were identified. Theme: Meaningful in identifying patient deterioration but causing frustration due to lack of flexibility. Categories: (a) Inter-professional collaboration strengthened through the use of Early Warning Score systems, (b) Enhanced professional development and communication among nurses when using Early Warning Score systems, (c) Detecting patient deterioration by integrating nurses' clinical gaze with Early Warning Score systems and (d) Modification and fear of making mistakes when using Early Warning Score systems.

Validation of National Early Warning Score for predicting 30-day mortality after rapid response system activation in Japan

Aim

Although rapid response systems (RRS) are used to prevent adverse events, Japan reportedly has low activation rates and high mortality rates. The National Early Warning Score (NEWS) could provide a solution, but it has not been validated in Japan. We aimed to validate NEWS for Japanese patients.

Methods

This retrospective observational study included data of 2,255 adult patients from 33 facilities registered in the In‐Hospital Emergency Registry in Japan between January 2014 and March 2018. The primary evaluated outcome was mortality rate 30 days after RRS activation. Accuracy of NEWS was analyzed with the correlation coefficient and area under the receiver operating characteristic curve. Prediction weights of NEWS parameters were then analyzed using multiple logistic regression and a machine learning method, classification and regression trees.

Results

The correlation coefficient of NEWS for 30‐day mortality rate was 0.95 (95% confidence interval [CI], 0.88–0.98) and the area under the receiver operating characteristic curve was 0.668 (95% CI, 0.642–0.693). Sensitivity and specificity values with a cut‐off score of 7 were 89.8% and 45.1%, respectively. Regarding prediction values of each parameter, oxygen saturation showed the highest odds ratio of 1.36 (95% CI, 1.25–1.48), followed by altered mental status 1.23 (95% CI, 1.14–1.32), heart rate 1.21 (95% CI, 1.09–1.34), systolic blood pressure 1.12 (95% CI, 1.04–1.22), and respiratory rate 1.03 (95% CI, 1.05–1.26). Body temperature and oxygen supplementation were not significantly associated. Classification and regression trees showed oxygen saturation as the most heavily weighted parameter, followed by altered mental status and respiratory rate.

Conclusions

National Early Warning Score could stratify 30‐day mortality risk following RRS activation in Japanese patients.

Comparison of the Performance of 24 Early Warning Scores with the Updated National Early Warning Score (NEWS2) for Predicting Unplanned Intensive Care Unit (ICU) Admission in Postoperative Brain Tumor Patients: A Retrospective Study at a Single Center

Background

There have been few studies to evaluate early warning score (EWS) systems, or track and trigger systems (TTS), to identify early clinical deterioration in patients following brain tumor surgery who are admitted to the Intensive Care Unit (ICU). The National Early Warning Score (NEWS2) is an established method used in the U.K. National Health Service to improve care for in-hospital patients. This retrospective study from a single center aimed to compare the performance of NEWS2 with 24 other types of EWS to evaluate unplanned ICU admissions within 72 h after brain tumor surgery.

Material/Methods

A total of 326 patients with brain tumors were included in the study. Patients who experienced unplanned ICU transfer after surgery (69 cases) were diagnostically matched with patients who did not require intensive care (257 controls). We collected the physiological variables to calculate the area under the receiver operator characteristic curve (AUROC), sensitivity, specificity, Youden index values, cutoff values, positive predictive values, and negative predictive values.

Results

The NEWS2 identified postoperative brain tumor patients with AUROC (0.860, p=0.000). The Patient-At-Risk (PAR) score was higher than NEWS2 in terms of AUROC value (0.870, P=0.000), Youden index (0.589 vs 0.542).

Conclusions

The findings showed that although the NEWS 2 performed well when used to evaluate unplanned ICU admissions within 72 h of postoperative brain tumor patients, the PAR score was also an accurate EWS.