Validation of an Automatic Tagging System for Identifying Respiratory and Hemodynamic Deterioration Events in the Intensive Care Unit

Application of Machine Learning Models to Biomedical and Information System Signals From Critically Ill Adults

BACKGROUND

Machine learning (ML)-derived notifications for impending episodes of hemodynamic instability and respiratory failure events are interesting because they can alert physicians in time to intervene before these complications occur.

RESEARCH QUESTION

Do ML alerts, telemedicine system (TS)-generated alerts, or biomedical monitors (BMs) have superior performance for predicting episodes of intubation or administration of vasopressors?

STUDY DESIGN AND METHODS

An ML algorithm was trained to predict intubation and vasopressor initiation events among critically ill adults. Its performance was compared with BM alarms and TS alerts.

RESULTS

ML notifications were substantially more accurate and precise, with 50-fold lower alarm burden than TS alerts for predicting vasopressor initiation and intubation events. ML notifications of internal validation cohorts demonstrated similar performance for independent academic medical center external validation and COVID-19 cohorts. Characteristics were also measured for a control group of recent patients that validated event detection methods and compared TS alert and BM alarm performance. The TS test characteristics were substantially better, with 10-fold less alarm burden than BM alarms. The accuracy of ML alerts (0.87-0.94) was in the range of other clinically actionable tests; the accuracy of TS (0.28-0.53) and BM (0.019-0.028) alerts were not. Overall test performance (F scores) for ML notifications were more than fivefold higher than for TS alerts, which were higher than those of BM alarms.

INTERPRETATION

ML-derived notifications for clinically actioned hemodynamic instability and respiratory failure events represent an advance because the magnitude of the differences of accuracy, precision, misclassification rate, and pre-event lead time is large enough to allow more proactive care and has markedly lower frequency and interruption of bedside physician work flows.

Diagnostic accuracy of clinical outcome prediction using nursing data in intensive care patients: A systematic review

BACKGROUND

Nursing data consist of observations of patients' conditions and information on nurses' clinical judgment based on critically ill patients' behavior and physiological signs. Nursing data in electronic health records were recently emphasized as important predictors of patients' deterioration but have not been systematically reviewed.

OBJECTIVE

We conducted a systematic review of prediction models using nursing data for clinical outcomes, such as prolonged hospital stay, readmission, and mortality in intensive care patients, compared to physiological data only. In addition, the type of nursing data used in prediction model developments was investigated.

DESIGN

A systematic review.

METHODS

PubMed, CINAHL, Cochrane CENTRAL, EMBASE, IEEE Xplore Digital Library, Web of Science, and Scopus were searched. Clinical outcome prediction models using nursing data for intensive care patients were included. Clinical outcomes were prolonged hospital stay, readmission, and mortality. Data were extracted from selected studies such as study design, data source, outcome definition, sample size, predictors, reference test, model development, model performance, and evaluation. The risk of bias and applicability was assessed using the Prediction model Risk of Bias Assessment Tool checklist. Descriptive summaries were produced based on paired forest plots and summary receiver operating characteristic curves.

RESULTS

Sixteen studies were included in the systematic review. The data types of predictors used in prediction models were categorized as physiological data, nursing data, and clinical notes. The types of nursing data consisted of nursing notes, assessments, documentation frequency, and flowsheet comments. The studies using physiological data as a reference test showed higher predictive performance in combined data or nursing data than in physiological data. The overall risk of bias indicated that most of the included studies have a high risk.

CONCLUSIONS

This study was conducted to identify and review the diagnostic accuracy of clinical outcome prediction using nursing data in intensive care patients. Most of the included studies developed models using nursing notes, and other studies used nursing assessments, documentation frequency, and flowsheet comments. Although the findings need careful interpretation due to the high risk of bias, the area under the curve scores of nursing data and combined data were higher than physiological data alone. It is necessary to establish a strategy in prediction modeling to utilize nursing data, clinical notes, and physiological data as predictors, considering the clinical context rather than physiological data alone.

REGISTRATION

The protocol for this study is registered with PROSPERO (registration number: CRD42021273319).

Enabling Timely Medical Intervention by Exploring Health-Related Multivariate Time Series with a Hybrid Attentive Model

Modern healthcare practice, especially in intensive care units, produces a vast amount of multivariate time series of health-related data, e.g., multi-lead electrocardiogram (ECG), pulse waveform, blood pressure waveform and so on. As a result, timely and accurate prediction of medical intervention (e.g., intravenous injection) becomes possible, by exploring such semantic-rich time series. Existing works mainly focused on onset prediction at the granularity of hours that was not suitable for medication intervention in emergency medicine. This research proposes a Multi-Variable Hybrid Attentive Model (MVHA) to predict the impending need of medical intervention, by jointly mining multiple time series. Specifically, a two-level attention mechanism is designed to capture the pattern of fluctuations and trends of different time series. This work applied MVHA to the prediction of the impending intravenous injection need of critical patients at the intensive care units. Experiments on the MIMIC Waveform Database demonstrated that the proposed model achieves a prediction accuracy of 0.8475 and an ROC-AUC of 0.8318, which significantly outperforms baseline models.