Delirium screening in an acute care setting with a machine learning classifier based on routinely collected nursing data: A model development study

Delirium detection tools show varying completion rates and positive score rates when used at scale in routine practice in general hospital settings: A systematic review

BACKGROUND

Multiple short delirium detection tools have been validated in research studies and implemented in routine care, but there has been little study of these tools in real-world conditions. This systematic review synthesized literature reporting completion rates and/or delirium positive score rates of detection tools in large clinical populations in general hospital settings.

METHODS

PROSPERO (CRD42022385166). Medline, Embase, PsycINFO, CINAHL, and gray literature were searched from 1980 to December 31, 2022. Included studies or audit reports used a validated delirium detection tool performed directly with the patient as part of routine care in large clinical populations (n ≥ 1000) within a general acute hospital setting. Narrative synthesis was performed.

RESULTS

Twenty-two research studies and four audit reports were included. Tools used alone or in combination were the Confusion Assessment Method (CAM), 4 'A's Test (4AT), Delirium Observation Screening Scale (DOSS), Brief CAM (bCAM), Nursing Delirium Screening Scale (NuDESC), and Intensive Care Delirium Screening Checklist (ICDSC). Populations and settings varied and tools were used at different stages and frequencies in the patient journey, including on admission only; inpatient, daily or more frequently; on admission and as inpatient; inpatient post-operatively. Tool completion rates ranged from 19% to 100%. Admission positive score rates ranged from: CAM 8%-51%; 4AT 13%-20%. Inpatient positive score rates ranged from: CAM 2%-20%, DOSS 6%-42%, and NuDESC 5-13%. Postoperative positive score rates were 21% and 28% (4AT). All but two studies had moderate-high risk of bias.

CONCLUSIONS

This systematic review of delirium detection tool implementation in large acute patient populations found clinically important variability in tool completion rates, and in delirium positive score rates relative to expected delirium prevalence. This study highlights a need for greater reporting and analysis of relevant healthcare systems data. This is vital to advance understanding of effective delirium detection in routine care.

Evaluating the performance of machine learning methods for risk estimation of delirium in patients hospitalized from the emergency department

INTRODUCTION

Delirium is a cerebral dysfunction seen commonly in the acute care setting. It is associated with increased mortality and morbidity and is frequently missed in the emergency department (ED) and inpatient care by clinical gestalt alone. Identifying those at risk of delirium may help prioritize screening and interventions in the hospital setting.

OBJECTIVE

Our objective was to leverage electronic health records to identify a clinically valuable risk estimation model for prevalent delirium in patients being transferred from the ED to inpatient units.

METHODS

This was a retrospective cohort study to develop and validate a risk model to detect delirium using patient data available from prior visits and ED encounter. Electronic health records were extracted for patients hospitalized from the ED between January 1, 2014, and December 31, 2020. Eligible patients were aged 65 or older, admitted to an inpatient unit from the emergency department, and had at least one DOSS assessment or CAM-ICU recorded within 72 h of hospitalization. Six machine learning models were developed to estimate the risk of delirium using clinical variables including demographic features, physiological measurements, medications administered, lab results, and diagnoses.

RESULTS

A total of 28,531 patients met the inclusion criteria with 8057 (28.4%) having a positive delirium screening within the outcome observation period. Machine learning models were compared using the area under the receiver operating curve (AUC). The gradient boosted machine achieved the best performance with an AUC of 0.839 (95% CI, 0.837-0.841). At a 90% sensitivity threshold, this model achieved a specificity of 53.5% (95% CI 53.0%-54.0%) a positive predictive value of 43.5% (95% CI 43.2%-43.9%), and a negative predictive value of 93.1% (95% CI 93.1%-93.2%). A random forest model and L1-penalized logistic regression also demonstrated notable performance with AUCs of 0.837 (95% CI, 0.835-0.838) and 0.831 (95% CI, 0.830-0.833) respectively.

CONCLUSION

This study demonstrated the use of machine learning algorithms to identify a combination of variables that enables an estimation of risk of positive delirium screens early in hospitalization to develop prevention or management protocols.