Errors, Omissions, and Outliers in Hourly Vital Signs Measurements in Intensive Care

Let's get in sync: current standing and future of AI-based detection of patient-ventilator asynchrony

BACKGROUND

Patient-ventilator asynchrony (PVA) is a mismatch between the patient's respiratory drive/effort and the ventilator breath delivery. It occurs frequently in mechanically ventilated patients and has been associated with adverse events and increased duration of ventilation. Identifying PVA through visual inspection of ventilator waveforms is highly challenging and time-consuming. Automated PVA detection using Artificial Intelligence (AI) has been increasingly studied, potentially offering real-time monitoring at the bedside. In this review, we discuss advances in automatic detection of PVA, focusing on developments of the last 15 years.

RESULTS

Nineteen studies were identified. Multiple forms of AI have been used for the automated detection of PVA, including rule-based algorithms, machine learning and deep learning. Three licensed algorithms are currently reported. Results of algorithms are generally promising (average reported sensitivity, specificity and accuracy of 0.80, 0.93 and 0.92, respectively), but most algorithms are only available offline, can detect a small subset of PVAs (focusing mostly on ineffective effort and double trigger asynchronies), or remain in the development or validation stage (84% (16/19 of the reviewed studies)). Moreover, only in 58% (11/19) of the studies a reference method for monitoring patient's breathing effort was available. To move from bench to bedside implementation, data quality should be improved and algorithms that can detect multiple PVAs should be externally validated, incorporating measures for breathing effort as ground truth. Last, prospective integration and model testing/finetuning in different ICU settings is key.

CONCLUSIONS

AI-based techniques for automated PVA detection are increasingly studied and show potential. For widespread implementation to succeed, several steps, including external validation and (near) real-time employment, should be considered. Then, automated PVA detection could aid in monitoring and mitigating PVAs, to eventually optimize personalized mechanical ventilation, improve clinical outcomes and reduce clinician's workload.

Statistical process monitoring creates a hemodynamic trajectory map after pediatric cardiac surgery: A case study of the arterial switch operation

Postoperative critical care management of congenital heart disease patients requires prompt intervention when the patient deviates significantly from clinician-determined vital sign and hemodynamic goals. Current monitoring systems only allow for static thresholds to be set on individual variables, despite the expectations that these signals change as the patient recovers and that variables interact. To address this incongruency, we have employed statistical process monitoring (SPM) techniques originally developed to monitor batch industrial processes to monitor high-frequency vital sign and hemodynamic data to establish multivariate trajectory maps for patients with d-transposition of the great arteries following the arterial switch operation. In addition to providing multivariate trajectory maps, the multivariate control charts produced by the SPM framework allow for assessment of adherence to the desired trajectory at each time point as the data is collected. Control charts based on slow feature analysis were compared with those based on principal component analysis. Alarms generated by the multivariate control charts are discussed in the context of the available clinical documentation.

Agreement between Vital Signs Measured Using Mat-Type Noncontact Sensors and Those from Conventional Clinical Assessment

Vital signs are crucial for assessing the condition of a patient and detecting early symptom deterioration. Noncontact sensor technology has been developed to take vital measurements with minimal burden. This study evaluated the accuracy of a mat-type noncontact sensor in measuring respiratory and pulse rates in patients with cardiovascular diseases compared to conventional methods. Forty-eight hospitalized patients were included; a mat-type sensor was used to measure their respiratory and pulse rates during bed rest. Differences between mat-type sensors and conventional methods were assessed using the Bland-Altman analysis. The mean difference in respiratory rate was 1.9 breaths/min (limits of agreement (LOA): -4.5 to 8.3 breaths/min), and proportional bias existed with significance (r = 0.63, p < 0.05). For pulse rate, the mean difference was -2.0 beats/min (LOA: -23.0 to 19.0 beats/min) when compared to blood pressure devices and 0.01 beats/min (LOA: -11.4 to 11.4 beats/min) when compared to 24-h Holter electrocardiography. The proportional bias was significant for both comparisons (r = 0.49, p < 0.05; r = 0.52, p < 0.05). These were considered clinically acceptable because there was no tendency to misjudge abnormal values as normal. The mat-type noncontact sensor demonstrated sufficient accuracy to serve as an alternative to conventional assessments, providing long-term monitoring of vital signs in clinical settings.

Improving awareness of kidney function through electronic urine output monitoring: a comparative study

BACKGROUND

The current classification for acute kidney injury (AKI) according to the Kidney Disease: Improving Global Outcomes (KDIGO) criteria integrates both serum creatinine (SCr) and urine output (UO). Most reports on AKI claim to use KDIGO guidelines but fail to include the UO criterion. It has been shown that patients who had intensive UO monitoring, with or without AKI, had significantly less cumulative fluid volume and fluid overload, reduced vasopressor use, and improved 30-day mortality. We examined whether real-time monitoring of this simple, sensitive, and easy-to-use biomarker in the ICU led to more appropriate intervention by healthcare providers and better outcomes.  METHODS: RenalSense Clarity RMS Consoles were installed in the General ICU at the Hadassah Medical Center, Israel, from December 2019 to November 2020. The Clarity RMS system continuously and electronically monitors UO in real-time. 100 patients were randomly selected from this period as the study group (UO_elec) and compared to a matched control group (UO_manual) from the same period two years earlier. To test whether there was an association between oliguric hours and fluid treatment in each group, the correlation was calculated and analyzed for each of the different UO monitoring methods.

RESULTS

Therapeutic intervention: The correlation of the sum of all oliguric hours on Day 1 and 2 with the sum of any therapeutic intervention (fluid bolus or furosemide) showed a significant correlation for the study group UO_elec (P = 0.017). The matched control group UO_manual showed no such correlation (P = 0.932). Length of Stay (LOS): Median LOS [IQR] in the ICU of UO_elec versus UO_manual was 69.46 [44.7, 125.9] hours and 116.5 [62.46, 281.3] hours, respectively (P = 0.0002).

CONCLUSIONS

The results of our study strongly suggest that ICU patients had more meaningful and better medical intervention, and improved outcomes, with electronic UO monitoring than with manual monitoring.

Acute vital signs changes are underrepresented by a conventional electronic health record when compared with automatically acquired data in a single-center tertiary pediatric cardiac intensive care unit

OBJECTIVE

We sought to evaluate the fidelity with which the patient's clinical state is represented by the electronic health record (EHR) flow sheet vital signs data compared to a commercially available automated data aggregation platform in a pediatric cardiac intensive care unit (CICU).

METHODS

This is a retrospective observational study of heart rate (HR), systolic blood pressure (SBP), respiratory rate (RR), and pulse oximetry (SpO2) data archived in a conventional EHR and an automated data platform for 857 pediatric patients admitted postoperatively to a tertiary pediatric CICU. Automated data captured for 72 h after admission were analyzed for significant HR, SBP, RR, and SpO2 deviations from baseline (events). Missed events were identified when the EHR failed to reflect the events reflected in the automated platform.

RESULTS

Analysis of 132 054 622 data entries, including 264 966 (0.2%) EHR entries and 131 789 656 (99.8%) automated entries, identified 15 839 HR events, 5851 SBP events, 9648 RR events, and 2768 SpO2 events lasting 3-60 min; these events were missing in the EHR 48%, 58%, 50%, and 54% of the time, respectively. Subanalysis identified 329 physiologically implausible events (eg, likely operator or device error), of which 104 (32%) were nonetheless documented in the EHR.

CONCLUSION

In this single-center retrospective study of CICU patients, EHR vital sign documentation was incomplete compared to an automated data aggregation platform. Significant events were underrepresented by the conventional EHR, regardless of event duration. Enrichment of the EHR with automated data aggregation capabilities may improve representation of patient condition.

Guiding Efficient, Effective, and Patient-Oriented Electrolyte Replacement in Critical Care: An Artificial Intelligence Reinforcement Learning Approach

Both provider- and protocol-driven electrolyte replacement have been linked to the over-prescription of ubiquitous electrolytes. Here, we describe the development and retrospective validation of a data-driven clinical decision support tool that uses reinforcement learning (RL) algorithms to recommend patient-tailored electrolyte replacement policies for ICU patients. We used electronic health records (EHR) data that originated from two institutions (UPHS; MIMIC-IV). The tool uses a set of patient characteristics, such as their physiological and pharmacological state, a pre-defined set of possible repletion actions, and a set of clinical goals to present clinicians with a recommendation for the route and dose of an electrolyte. RL-driven electrolyte repletion substantially reduces the frequency of magnesium and potassium replacements (up to 60%), adjusts the timing of interventions in all three electrolytes considered (potassium, magnesium, and phosphate), and shifts them towards orally administered repletion over intravenous replacement. This shift in recommended treatment limits risk of the potentially harmful effects of over-repletion and implies monetary savings. Overall, the RL-driven electrolyte repletion recommendations reduce excess electrolyte replacements and improve the safety, precision, efficacy, and cost of each electrolyte repletion event, while showing robust performance across patient cohorts and hospital systems.

Developing, implementing and governing artificial intelligence in medicine: a step-by-step approach to prevent an artificial intelligence winter

Objective

Although the role of artificial intelligence (AI) in medicine is increasingly studied, most patients do not benefit because the majority of AI models remain in the testing and prototyping environment. The development and implementation trajectory of clinical AI models are complex and a structured overview is missing. We therefore propose a step-by-step overview to enhance clinicians’ understanding and to promote quality of medical AI research.

Methods

We summarised key elements (such as current guidelines, challenges, regulatory documents and good practices) that are needed to develop and safely implement AI in medicine.

Conclusion

This overview complements other frameworks in a way that it is accessible to stakeholders without prior AI knowledge and as such provides a step-by-step approach incorporating all the key elements and current guidelines that are essential for implementation, and can thereby help to move AI from bytes to bedside.

Moving from bytes to bedside: a systematic review on the use of artificial intelligence in the intensive care unit

PURPOSE

Due to the increasing demand for intensive care unit (ICU) treatment, and to improve quality and efficiency of care, there is a need for adequate and efficient clinical decision-making. The advancement of artificial intelligence (AI) technologies has resulted in the development of prediction models, which might aid clinical decision-making. This systematic review seeks to give a contemporary overview of the current maturity of AI in the ICU, the research methods behind these studies, and the risk of bias in these studies.

METHODS

A systematic search was conducted in Embase, Medline, Web of Science Core Collection and Cochrane Central Register of Controlled Trials databases to identify eligible studies. Studies using AI to analyze ICU data were considered eligible. Specifically, the study design, study aim, dataset size, level of validation, level of readiness, and the outcomes of clinical trials were extracted. Risk of bias in individual studies was evaluated by the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

RESULTS

Out of 6455 studies identified through literature search, 494 were included. The most common study design was retrospective [476 studies (96.4% of all studies)] followed by prospective observational [8 (1.6%)] and clinical [10 (2%)] trials. 378 (80.9%) retrospective studies were classified as high risk of bias. No studies were identified that reported on the outcome evaluation of an AI model integrated in routine clinical practice.

CONCLUSION

The vast majority of developed ICU-AI models remain within the testing and prototyping environment; only a handful were actually evaluated in clinical practice. A uniform and structured approach can support the development, safe delivery, and implementation of AI to determine clinical benefit in the ICU.

A systematic review of rapid response applications based on early warning score for early detection of inpatient deterioration

AIM

The aim of this study was to investigate the effectiveness of current rapid response applications available in acute care settings for escalation of patient deterioration. Current challenges and barriers, as well as key recommendations, were also discussed.

METHODS

We adopted PRISMA review methodology and screened a total of 559 articles. After considering the eligibility and selection criteria, we selected 13 articles published between 2015 and 2019. The selection criteria were based on the inclusion of studies that report on the advancement made to the current practice for providing rapid response to the patient deterioration in acute care settings.

RESULTS

We found that current rapid response applications are complicated and time-consuming for detecting inpatient deterioration. Existing applications are either siloed or challenging to use, where clinicians are required to move between two or three different applications to complete an end-to-end patient escalation workflow - from vital signs collection to escalation of deteriorating patients. We found significant differences in escalation and responses when using an electronic tool compared to the manual approach. Moreover, encouraging results were reported in extensive documentation of vital signs and timely alerts for patient deterioration.

CONCLUSION

The electronic vital signs monitoring applications are proved to be efficient and clinically suitable if they are user-friendly and interoperable. As an outcome, several key recommendations and features were identified that would be crucial to the successful implementation of any rapid response system in all clinical settings.

Characterizing the Patients, Hospitals, and Data Quality of the eICU Collaborative Research Database

OBJECTIVES

The eICU Collaborative Research Database is a publicly available repository of granular data from more than 200,000 ICU admissions. The quantity and variety of its entries hold promise for observational critical care research. We sought to understand better the data available within this resource to guide its future use.

DESIGN

We conducted a descriptive analysis of the eICU Collaborative Research Database, including patient, practitioner, and hospital characteristics. We investigated the completeness of demographic and hospital data, as well as those values required to calculate an Acute Physiology and Chronic Health Evaluation score. We also assessed the rates of ventilation, intubation, and dialysis, and looked for potential errors in the vital sign data.

SETTING

American ICUs that participated in the Philips Healthcare eICU program between 2014 and 2015.

PATIENTS

A total of 139,367 individuals who were admitted to one of the 335 participating ICUs between 2014 and 2015.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Most encounters were from small- and medium-sized hospitals, and managed by nonintensivists. The median ICU length of stay was 1.57 days (interquartile range, 0.82-2.97 d). The median Acute Physiology and Chronic Health Evaluation IV-predicted ICU mortality was 2.2%, with an observed mortality of 5.4%. Rates of ventilation (20-33%), intubation (15-24%), and dialysis (3-5%) varied according to the query method used. Most vital sign readings fell into realistic ranges, with manually curated data less likely to contain implausible results than automatically entered data.

CONCLUSIONS

Data in the eICU Collaborative Research Database are for the most part complete and plausible. Some ambiguity exists in determining which encounters are associated with various interventions, most notably mechanical ventilation. Caution is warranted in extrapolating findings from the eICU Collaborative Research Database to larger ICUs with higher acuity.

Discrete Heart Rate Values or Continuous Streams? Representation, Variability, and Meaningful Use of Vital Sign Data

Documentation and review of patient heart rate are a fundamental process across a myriad of clinical settings. While historically recorded manually, bedside monitors now provide for the automated collection of such data. Despite the availability of continuous streaming data, patients' charts continue to reflect only a subset of this information as snapshots recorded throughout a hospitalization. Over the past decade, prominent works have explored the implications of such practices and established fundamental differences in the alignment of discrete charted vitals and steaming data captured by monitoring systems. Limited work has examined the temporal properties of these differences, how they manifest, and their relation to clinical applications. The work presented in this article addresses this disparity, providing evidence that differences between charting techniques extend to measures of variability. Our results demonstrate how variability manifests with respect to temporal elements of charting timing and how it can facilitate personalized care by contextualizing deviations in magnitude. This work also highlights the utility of variability metrics with relation to clinical measures including associations to severity scores and a case study utilizing complex variability metrics derived from the complete set of monitor data.

Adding Continuous Vital Sign Information to Static Clinical Data Improves the Prediction of Length of Stay After Intubation: A Data-Driven Machine Learning Approach

BACKGROUND

Bedside monitors in the ICU routinely measure and collect patients' physiologic data in real time to continuously assess the health status of patients who are critically ill. With the advent of increased computational power and the ability to store and rapidly process big data sets in recent years, these physiologic data show promise in identifying specific outcomes and/or events during patients' ICU hospitalization.

METHODS

We introduced a methodology designed to automatically extract information from continuous-in-time vital sign data collected from bedside monitors to predict if a patient will experience a prolonged stay (length of stay) on mechanical ventilation, defined as >4 d, in a pediatric ICU.

RESULTS

Continuous-in-time vital signs information and clinical history data were retrospectively collected for 284 ICU subjects from their first 24 h on mechanical ventilation from a medical-surgical pediatric ICU at Boston Children's Hospital. Multiple machine learning models were trained on multiple subsets of these subjects to predict the likelihood that each of these subjects would experience a long stay. We evaluated the predictive power of our models strictly on unseen hold-out validation sets of subjects. Our methodology achieved model performance of >83% (area under the curve) by using only vital sign information as input, and performances of 90% (area under the curve) by combining vital sign information with subjects' static clinical data readily available in electronic health records. We implemented this approach on 300 independently trained experiments with different choices of training and hold-out validation sets to ensure the consistency and robustness of our results in our study sample. The predictive power of our approach outperformed recent efforts that used deep learning to predict a similar task.

CONCLUSIONS

Our proposed workflow may prove useful in the design of scalable approaches for real-time predictive systems in ICU environments, exploiting real-time vital sign information from bedside monitors. (ClinicalTrials.gov registration NCT02184208.).

A Tool to Predict Readmission to the Intensive Care Unit in Surgical Critical Care Patients-The RISC Score

BACKGROUND

Readmission to the Intensive Care Unit (ICU) is associated with a high risk of in-hospital mortality and higher health care costs. Previously published tools to predict ICU readmission in surgical ICU patients have important limitations that restrict their clinical implementation. We sought to develop a clinically intuitive score that can be implemented to predict readmission to the ICU after surgery or trauma. We designed the score to emphasize modifiable predictors.

METHODS

In this retrospective cohort study, we included surgical patients requiring critical care between June 2015 and January 2019 at Beth Israel Deaconess Medical Center, Harvard Medical School, MA, USA. We used logistic regression to fit a prognostic model for ICU readmission from a priori defined, widely available candidate predictors. The score performance was compared with existing prediction instruments.

RESULTS

Of 7,126 patients, 168 (2.4%) were readmitted to the ICU during the same hospitalization. The final score included 8 variables addressing demographical factors, surgical factors, physiological parameters, ICU treatment and the acuity of illness. The maximum score achievable was 13 points. Potentially modifiable predictors included the inability to ambulate at ICU discharge, substantial positive fluid balance (>5 liters), severe anemia (hemoglobin <7 mg/dl), hyperglycemia (>180 mg/dl), and long ICU length of stay (>5 days). The score yielded an area under the receiver operating characteristic curve of 0.78 (95% CI 0.74-0.82) and significantly outperformed previously published scores. The performance of the underlying model was confirmed by leave-one-out cross-validation.

CONCLUSION

The RISC-score is a clinically intuitive prediction instrument that helps identify surgical ICU patients at high risk for ICU readmission. The simplicity of the score facilitates its clinical implementation across surgical divisions.

Artifact Processing Methods Influence on Intraoperative Hypotension Quantification and Outcome Effect Estimates

BACKGROUND

Physiologic data that is automatically collected during anesthesia is widely used for medical record keeping and clinical research. These data contain artifacts, which are not relevant in clinical care, but may influence research results. The aim of this study was to explore the effect of different methods of filtering and processing artifacts in anesthesiology data on study findings in order to demonstrate the importance of proper artifact filtering.

METHODS

The authors performed a systematic literature search to identify artifact filtering methods. Subsequently, these methods were applied to the data of anesthesia procedures with invasive blood pressure monitoring. Different hypotension measures were calculated (i.e., presence, duration, maximum deviation below threshold, and area under threshold) across different definitions (i.e., thresholds for mean arterial pressure of 50, 60, 65, 70 mmHg). These were then used to estimate the association with postoperative myocardial injury.

RESULTS

After screening 3,585 papers, the authors included 38 papers that reported artifact filtering methods. The authors applied eight of these methods to the data of 2,988 anesthesia procedures. The occurrence of hypotension (defined with a threshold of 50 mmHg) varied from 24% with a median filter of seven measurements to 55% without an artifact filtering method, and between 76 and 90% with a threshold of 65 mmHg. Standardized odds ratios for presence of hypotension ranged from 1.16 (95% CI, 1.07 to 1.26) to 1.24 (1.14 to 1.34) when hypotension was defined with a threshold of 50 mmHg. Similar variations in standardized odds ratios were found when applying methods to other hypotension measures and definitions.

CONCLUSIONS

The method of artifact filtering can have substantial effects on estimates of hypotension prevalence. The effect on the association between intraoperative hypotension and postoperative myocardial injury was relatively small. Nevertheless, the authors recommend that researchers carefully consider artifacts handling and report the methodology used.

Low brain tissue oxygenation contributes to the development of delirium in critically ill patients: A prospective observational study

PURPOSE

To test the hypothesis that poor brain tissue oxygenation (BtO₂) during the first 24h of critical illness correlates with the proportion of time spent delirious. We also sought to define the physiological determinants of BtO₂.

MATERIALS AND METHODS

Adult patients admitted to the ICU within the previous 24h were considered eligible for enrollment if they required mechanical ventilation, and/or vasopressor support. BtO₂ was measured using near-infrared spectroscopy, for 24h after enrollment. Hourly vital signs and clinically ordered arterial and central venous blood gases were collected throughout BtO₂ monitoring. Patients were screened daily for delirium with the confusion assessment method for the intensive care unit (CAM-ICU).

RESULTS

BtO₂ and the proportion of time spent delirious did not result in a significant correlation (p=0.168). However, critically ill patients who spent the majority of their ICU stay delirious had significantly lower mean BtO₂ compared to non-delirious patients, (p=0.017). BtO₂ correlated positively with central venous pO₂ (p=0.00003) and hemoglobin concentration (p=0.001). Logistic regression indicated that lower BtO₂, higher narcotic doses and a history of alcohol abuse were independent risk factors for delirium.

CONCLUSIONS

Poor cerebral oxygenation during the first 24 hours of critical illness contributes to the development of delirium.

TRIAL REGISTRATION

This trial is registered on clinicaltrials.gov (Identifier: NCT02344043), retrospectively registered January 8, 2015.

A path to precision in the ICU

Precision medicine is increasingly touted as a groundbreaking new paradigm in biomedicine. In the ICU, the complexity and ambiguity of critical illness syndromes have been identified as fundamental justifications for the adoption of a precision approach to research and practice. Inherently protean diseases states such as sepsis and acute respiratory distress syndrome have manifestations that are physiologically and anatomically diffuse, and that fluctuate over short periods of time. This leads to considerable heterogeneity among patients, and conditions in which a “one size fits all” approach to therapy can lead to widely divergent results. Current ICU therapy can thus be seen as imprecise, with the potential to realize substantial gains from the adoption of precision medicine approaches. A number of challenges still face the development and adoption of precision critical care, a transition that may occur incrementally rather than wholesale. This article describes a few concrete approaches to addressing these challenges.

First, novel clinical trial designs, including registry randomized controlled trials and platform trials, suggest ways in which conventional trials can be adapted to better accommodate the physiologic heterogeneity of critical illness. Second, beyond the “omics” technologies already synonymous with precision medicine, the data-rich environment of the ICU can generate complex physiologic signatures that could fuel precision-minded research and practice. Third, the role of computing infrastructure and modern informatics methods will be central to the pursuit of precision medicine in the ICU, necessitating close collaboration with data scientists. As work toward precision critical care continues, small proof-of-concept studies may prove useful in highlighting the potential of this approach.