Beyond Mortality: Improving Outcomes for Children Who Deteriorate in Inpatient Settings

The Deterioration Risk Index: Developing and Piloting a Machine Learning Algorithm to Reduce Pediatric Inpatient Deterioration

OBJECTIVES

Develop and deploy a disease cohort-based machine learning algorithm for timely identification of hospitalized pediatric patients at risk for clinical deterioration that outperforms our existing situational awareness program.

DESIGN

Retrospective cohort study.

SETTING

Nationwide Children's Hospital, a freestanding, quaternary-care, academic children's hospital in Columbus, OH.

PATIENTS

All patients admitted to inpatient units participating in the preexisting situational awareness program from October 20, 2015, to December 31, 2019, excluding patients over 18 years old at admission and those with a neonatal ICU stay during their hospitalization.

INTERVENTIONS

We developed separate algorithms for cardiac, malignancy, and general cohorts via lasso-regularized logistic regression. Candidate model predictors included vital signs, supplemental oxygen, nursing assessments, early warning scores, diagnoses, lab results, and situational awareness criteria. Model performance was characterized in clinical terms and compared with our previous situational awareness program based on a novel retrospective validation approach. Simulations with frontline staff, prior to clinical implementation, informed user experience and refined interdisciplinary workflows. Model implementation was piloted on cardiology and hospital medicine units in early 2021.

MEASUREMENTS AND MAIN RESULTS

The Deterioration Risk Index (DRI) was 2.4 times as sensitive as our existing situational awareness program (sensitivities of 53% and 22%, respectively; p < 0.001) and required 2.3 times fewer alarms per detected event (121 DRI alarms per detected event vs 276 for existing program). Notable improvements were a four-fold sensitivity gain for the cardiac diagnostic cohort (73% vs 18%; p < 0.001) and a three-fold gain (81% vs 27%; p < 0.001) for the malignancy diagnostic cohort. Postimplementation pilot results over 18 months revealed a 77% reduction in deterioration events (three events observed vs 13.1 expected, p = 0.001).

CONCLUSIONS

The etiology of pediatric inpatient deterioration requires acknowledgement of the unique pathophysiology among cardiology and oncology patients. Selection and weighting of diverse candidate risk factors via machine learning can produce a more sensitive early warning system for clinical deterioration. Leveraging preexisting situational awareness platforms and accounting for operational impacts of model implementation are key aspects to successful bedside translation.

Observational cohort study with internal and external validation of a predictive tool for identification of children in need of hospital admission from the emergency department: the Paediatric Admission Guidance in the Emergency Department (PAGE) score

OBJECTIVES

To devise an assessment tool to aid discharge and admission decision-making in relation to children and young people in hospital urgent and emergency care facilities, and thereby improve the quality of care that patients receive, using a clinical prediction modelling approach.

DESIGN

Observational cohort study with internal and external validation of a predictive tool.

SETTING

Two general emergency departments (EDs) and an urgent care centre in the North of England.

PARTICIPANTS

The eligibility criteria were children and young people 0-16 years of age who attended one of the three hospital sites within one National Health Service (NHS) organisation. Children were excluded if they opted out of the study, were brought to the ED following their death in the community or arrived in cardiac arrest when the heart rate and respiratory rate would be unmeasurable.

MAIN OUTCOME MEASURES

Admission or discharge. A participant was defined as being admitted to hospital if they left the ED to enter the hospital for further assessment, (including being admitted to an observation and assessment unit or hospital ward), either on first presentation or with the same complaint within 7 days. Those who were not admitted were defined as having been discharged.

RESULTS

The study collected data on 36 365 participants. 15 328 participants were included in the final analysis cohort (21 045 observations) and 17 710 participants were included in the validation cohort (23 262 observations). There were 14 variables entered into the regression analysis. Of the 13 that remained in the final model, 10 were present in all 500 bootstraps. The resulting Paediatric Admission Guidance in the Emergency Department (PAGE) score demonstrated good internal validity. The C-index (area under the ROC) was 0.779 (95% CI 0.772 to 0.786).

CONCLUSIONS

For units without the immediate availability of paediatricians the PAGE score can assist staff to determine risk of admission. Cut-off values will need to be adjusted to local circumstance.

STUDY PROTOCOL

The study protocol has been published in an open access journal: Riaz et al Refining and testing the diagnostic accuracy of an assessment tool (Pennine Acute Hospitals NHS Trust-Paediatric Observation Priority Score) to predict admission and discharge of children and young people who attend an ED: protocol for an observational study. BMC Pediatr 18, 303 (2018). https://doi.org/10.1186/s12887-018-1268-7.

TRIAL REGISTRATION NUMBER

The protocol has been published and the study registered (NIHR RfPB Grant: PB-PG-0815-20034; ClinicalTrials.gov:213469).