Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study

Dynamic Nomogram for Subsyndromal Delirium in Adult Intensive Care Unit: A Prospective Cohort Study

Purpose

To develop a dynamic nomogram of subsyndromal delirium (SSD) in intensive care unit (ICU) patients and internally validate its efficacy in predicting SSD.

Patients and Methods

Patients who met the inclusion and exclusion criteria in the ICU of a tertiary hospital in Zhejiang from September 2021 to June 2022 were selected as the research objects. The patient data were randomly divided into the training set and validation set according to the ratio of 7:3. The least absolute shrinkage and selection operator (LASSO) and multivariate logistic regression were used to screen the predictors of SSD, and R software was used to construct a dynamic nomogram. Receiver operating characteristic (ROC) curve, calibration band and decision curve were used to evaluate the discrimination, calibration and clinical effectiveness of the model.

Results

A total of 1000 eligible patients were included, including 700 in the training set and 300 in the validation set. Age, drinking history, C reactive protein level, APACHE II, indwelling urinary catheter, mechanical ventilation, cerebrovascular disease, respiratory failure, constraint, dexmedetomidine, and propofol were predictors of SSD in ICU patients. The ROC curve values of the training set was 0.902 (95% confidence interval: 0.879-0.925), the best cutoff value was 0.264, the specificity was 78.4%, and the sensitivity was 88.0%. The ROC curve values of the validation set was 0.888 (95% confidence interval: 0.850-0.930), the best cutoff value was 0.543, the specificity was 94.9%, and the sensitivity was 70.9%. The calibration band showed good calibration in the training and validation set. Decision curve analysis showed that the net benefit in the model was significantly high.

Conclusion

The dynamic nomogram has good predictive performance, so it is a precise and effective tool for medical staff to predict and manage SSD in the early stage.

Inclusion of frailty improved performance of delirium prediction for elderly patients in the cardiac intensive care unit (D-FRAIL): A prospective derivation and external validation study

BACKGROUND

The elderly patients admitted to cardiac intensive care unit (CICU) are at relatively high risk for developing delirium. A simple and reliable predictive model can benefit them from early recognition of delirium followed by timely and appropriate preventive strategies.

OBJECTIVE

To explore the role of frailty in delirium prediction and develop and validate a delirium predictive model including frailty for elderly patients in CICU.

DESIGN

A prospective, observational cohort study.

SETTINGS

CICU at China-Japan Friendship Hospital from March 1, 2022 to August 25, 2022 (derivation cohort); CICU at Beijing Anzhen Hospital affiliated to Capital Medical University from March 14, 2023 to May 8, 2023 (external validation cohort).

PARTICIPANTS

A total of 236 and 90 participants were enrolled in the derivation and external validation cohorts, respectively. Participants in the derivation cohort were assigned into either the delirium (n = 70) or non-delirium group (n = 166) based on the occurrence of delirium.

METHODS

The simplified Chinese version of the Confusion Assessment Method for the Diagnosis of Delirium in the Intensive Care Unit was used to assess delirium twice a day at 8:00-10:00 and 18:00-20:00 until the onset of delirium or discharge from the CICU. Frailty was assessed using the FRAIL scale during the first 24 h in the CICU. Other possible risk factors were collected prospectively through patient interviews and medical records review. After processing missing data via multiple imputations, univariate analysis and bootstrapped forward stepwise logistic regression were performed to select optimal predictors and develop the models. The models were internally validated using bootstrapping and evaluated comprehensively via discrimination, calibration, and clinical utility in both the derivation and external validation cohorts.

RESULTS

The study developed D-FRAIL predictive model using FRAIL score, hearing impairment, Acute Physiology and Chronic Health Evaluation-II score, and fibrinogen. The area under the receiver operating characteristic curve (AUC) was 0.937 (95% confidence interval [CI]: 0.907-0.967) and 0.889 (95%CI: 0.840-0.938) even after bootstrapping in the derivation cohort. Inclusion of frailty was demonstrated to improve the model performance greatly with the AUC increased from 0.851 to 0.937 (p < 0.001). In the external validation cohort, the AUC of D-FRAIL model was 0.866 (95%CI: 0.782-0.907). Calibration plots and decision curve analysis suggested good calibration and clinical utility of the D-FRAIL model in both the derivation and external validation cohorts.

CONCLUSIONS

For elderly patients in the CICU, FRAIL score is an independent delirium predictor and the D-FRAIL model demonstrates superior performance in predicting delirium.

Fixing the Leaky Pipe: How to Improve the Uptake of Patient-Reported Outcomes-Based Prognostic and Predictive Models in Cancer Clinical Practice

PURPOSE

This discussion paper outlines challenges and proposes solutions for successfully implementing prediction models that incorporate patient-reported outcomes (PROs) in cancer practice.

METHODS

We organized a full-day multidisciplinary meeting of people with expertise in cancer care delivery, PRO collection, PRO use in prediction modeling, computing, implementation, and decision science. The discussions presented here focused on identifying challenges to the development, implementation and use of prediction models incorporating PROs, and suggesting possible solutions.

RESULTS

Specific challenges and solutions were identified across three broad areas. (1) Understanding decision making and implementation: necessitating multidisciplinary collaboration in the early stages and throughout; early stakeholder engagement to define the decision problem and ensure acceptability of PROs in prediction; understanding patient/clinician interpretation of PRO predictions and uncertainty to optimize prediction impact; striving for model integration into existing electronic health records; and early regulatory alignment. (2) Recognizing the limitations to PRO collection and their impact on prediction: incorporating validated, clinically important PROs to maximize model generalizability and clinical engagement; and minimizing missing PRO data (resulting from both structural digital exclusion and time-varying factors) to avoid exacerbating existing inequalities. (3) Statistical and modeling challenges: incorporating statistical methods to address missing data; ensuring predictive modeling recognizes complex causal relationships; and considering temporal and geographic recalibration so that model predictions reflect the relevant population.

CONCLUSION

Developing and implementing PRO-based prediction models in cancer care requires extensive multidisciplinary working from the earliest stages, recognition of implementation challenges because of PRO collection and model presentation, and robust statistical methods to manage missing data, causality, and calibration. Prediction models incorporating PROs should be viewed as complex interventions, with their development and impact assessment carried out to reflect this.

Calibration of the PREdiction of DELIRium in ICu Patients (PRE-DELIRIC) Score in a Cohort of Critically Ill Patients: A Retrospective Cohort Study

BACKGROUND

To predict delirium in intensive care unit (ICU) patients, the Prediction of Delirium in ICU Patients (PRE-DELIRIC) score may be used. This model may help nurses to predict delirium in high-risk ICU patients.

OBJECTIVES

The aims of this study were to externally validate the PRE-DELIRIC model and to identify predictive factors and outcomes for ICU delirium.

METHOD

All patients underwent delirium risk assessment by the PRE-DELIRIC model at admission. We used the Intensive Care Delirium Screening Check List to identify patients with delirium. The receiver operating characteristic curve measured discrimination capacity among patients with or without ICU delirium. Calibration ability was determined by slope and intercept.

RESULTS

The prevalence of ICU delirium was 55.8%. Discrimination capacity (Intensive Care Delirium Screening Check List score ≥4) expressed by the area under the receiver operating characteristic curve was 0.81 (95% confidence interval, 0.75-0.88), whereas sensitivity was 91.3% and specificity was 64.4%. The best cut-off was 27%, obtained by the max Youden index. Calibration of the model was adequate, with a slope of 1.03 and intercept of 8.14. The onset of ICU delirium was associated with an increase in ICU length of stay (P < .0001), higher ICU mortality (P = .008), increased duration of mechanical ventilation (P < .0001), and more prolonged respiratory weaning (P < .0001) compared with patients without delirium.

DISCUSSION

The PRE-DELIRIC score is a sensitive measure that may be useful in early detection of patients at high risk for developing delirium. The baseline PRE-DELIRIC score could be useful to trigger use of standardized protocols, including nonpharmacologic interventions.

Early prediction of delirium upon intensive care unit admission: Model development, validation, and deployment

STUDY OBJECTIVE

To develop, validate, and deploy models for predicting delirium in critically ill adult patients as early as upon intensive care unit (ICU) admission.

DESIGN

Retrospective cohort study.

SETTING

Single university teaching hospital in Taipei, Taiwan.

PATIENTS

6238 critically ill patients from August 2020 to August 2021.

MEASUREMENTS

Data were extracted, pre-processed, and split into training and testing datasets based on the time period. Eligible variables included demographic characteristics, Glasgow Coma Scale, vital signs parameters, treatments, and laboratory data. The predicted outcome was delirium, defined as any positive result (a score ≥ 4) of the Intensive Care Delirium Screening Checklist that was assessed by primary care nurses in each 8-h shift within 48 h after ICU admission. We trained models to predict delirium upon ICU admission (ADM) and at 24 h (24H) after ICU admission by using logistic regression (LR), gradient boosted trees (GBT), and deep learning (DL) algorithms and compared the models' performance.

MAIN RESULTS

Eight features were extracted from the eligible features to train the ADM models, including age, body mass index, medical history of dementia, postoperative intensive monitoring, elective surgery, pre-ICU hospital stays, and GCS score and initial respiratory rate upon ICU admission. In the ADM testing dataset, the incidence of ICU delirium occurred within 24 h and 48 h was 32.9% and 36.2%, respectively. The area under the receiver operating characteristic curve (AUROC) (0.858, 95% CI 0.835-0.879) and area under the precision-recall curve (AUPRC) (0.814, 95% CI 0.780-0.844) for the ADM GBT model were the highest. The Brier scores of the ADM LR, GBT, and DL models were 0.149, 0.140, and 0.145, respectively. The AUROC (0.931, 95% CI 0.911-0.949) was the highest for the 24H DL model and the AUPRC (0.842, 95% CI 0.792-0.886) was the highest for the 24H LR model.

CONCLUSION

Our early prediction models based on data obtained upon ICU admission could achieve good performance in predicting delirium occurred within 48 h after ICU admission. Our 24-h models can improve delirium prediction for patients discharged >1 day after ICU admission.

Cortical reactivity to transcranial magnetic stimulation predicts risk of post-stroke delirium

OBJECTIVE

Post-stroke delirium (PSD) is a frequent and with regard to outcome unfavorable complication in acute stroke. The neurobiological mechanisms predisposing to PSD remain poorly understood, and biomarkers predicting its risk have not been established. We tested the hypothesis that hypoexcitable or disconnected brain networks predispose to PSD by measuring brain reactivity to transcranial magnetic stimulation with electroencephalography (TMS-EEG).

METHODS

We conducted a cross-sectional study in 33 acute stroke patients within 48 hours of stroke onset. Brain reactivity to single-pulse TMS of dorsolateral prefrontal cortex, primary motor cortex and superior parietal lobule of the right hemisphere was quantified by response intensity, effective connectivity, perturbational complexity index (PCI^ST), and natural frequency of the TMS-EEG response. PSD development was clinically tracked every 8 hours before and for 7 days following TMS-EEG.

RESULTS

Fourteen patients developed PSD while 19 patients did not. The PSD group showed lower excitability, effective connectivity, PCI^ST and natural frequency compared to the non-PSD group. The maximum PCI^ST over all three TMS sites demonstrated largest classification accuracy with a ROC-AUC of 0.943. This effect was independent of lesion size, affected hemisphere and stroke severity. Maximum PCI^ST and maximum natural frequency correlated inversely with delirium duration.

CONCLUSIONS

Brain reactivity to TMS-EEG can unravel brain network states of reduced excitability, effective connectivity, perturbational complexity and natural frequency that identify acute stroke patients at high risk for development of delirium.

SIGNIFICANCE

Findings provide novel insight into the pathophysiology of pre-delirium brain states and may promote effective delirium prevention strategies in those patients at high risk.

Deep Learning-Based Recurrent Delirium Prediction in Critically Ill Patients

OBJECTIVES

To predict impending delirium in ICU patients using recurrent deep learning.

DESIGN

Retrospective cohort study.

SETTING

Fifteen medical-surgical ICUs across Alberta, Canada, between January 1, 2014, and January 24, 2020.

PATIENTS

Forty-three thousand five hundred ten ICU admissions from 38,426 patients.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We used ICU and administrative health data to train deep learning models to predict delirium episodes in the next two 12-hour periods (0-12 and 12-24 hr), starting at 24 hours after ICU admission, and to generate new predictions every 12 hours. We used a comprehensive set of 3,643 features, capturing patient history, early ICU admission information (first 24 hr), and the temporal dynamics of various clinical variables throughout the ICU admission. Our deep learning architecture consisted of a feature embedding, a recurrent, and a prediction module. Our best model based on gated recurrent units yielded a sensitivity of 0.810, a specificity of 0.848, a precision (positive predictive value) of 0.704, and an area under the receiver operating characteristic curve (AUROC) of 0.909 in the hold-out test set for the 0-12-hour prediction horizon. For the 12-24-hour prediction horizon, the same model achieved a sensitivity of 0.791, a specificity of 0.807, a precision of 0.637, and an AUROC of 0.895 in the test set.

CONCLUSIONS

Our delirium prediction model achieved strong performance by applying deep learning to a dataset that is at least one order of magnitude larger than those used in previous studies. Another novel aspect of our study is the temporal nature of our features and predictions. Our model enables accurate prediction of impending delirium in the ICU, which can potentially lead to early intervention, more efficient allocation of ICU resources, and improved patient outcomes.

Survey of Guideline Compliance and Attitude Toward Symptom Management in Japanese Intensive Care Units

Introduction

The Clinical Practice Guideline for the Management of Pain, Agitation, and Delirium in Adult Patients in the Intensive Care Unit (ICU) was revised in 2018 to include sleep disruption and immobility. Inadequate management of these symptoms can lead to negative consequences. A 2019 survey in Japan found that the guideline was recognized but needed to be consistently implemented.

Objective

This study aimed to examine compliance with the guideline for symptom management of pain, agitation, delirium, and sleep in Japanese ICUs.

Methods

This study included all ICUs in Japan and asked one representative from each unit to respond to the web survey from January 2022 to February 2022.

Results

Of a potential 643 units, 125 respondents from the ICU were included in the analysis (19.4% response rate). Compared to the guideline's recommendations, (a) pain assessment was performed in 86.3% of patients who could self-report, and in 72.0% of those who could not self-report; (b) agitation and sedation assessment was performed in 99% of patients; (c) only 66.1% of nurses reported assessing sleep quality on the units, and 9.1% performed the subjective sleep quality assessment; (d) the use of the recommended risk factor of the delirium assessment tool was low (9.6%). Additionally, according to the survey respondents, contrary to the guideline, many units administered medications to prevent and treat delirium, and approximately 30% used multiple non-drug interventions. The data are expressed as numbers and percentages. Some datasets were incomplete due to missing values.

Conclusion

Most units used drugs for delirium prevention and treatment, and only a few used non-drug interventions. There is a need to popularize the assessment of sleep and delirium risk factors and use non-drug interventions to promote patient-centered care in the future.

Occupational therapist-guided cognitive interventions in critically ill patients: a feasibility randomized controlled trial

PURPOSE

Intensive care unit (ICU) delirium is a common complication of critical illness requiring a multimodal approach to management. We assessed the feasibility of a novel occupational therapist (OT)-guided cognitive intervention protocol, titrated according to sedation level, in critically ill patients.

METHODS

Patients aged ≥ 18 yr admitted to a medical/surgical ICU were randomized to the standard delirium prevention protocol or to the OT-guided cognitive intervention protocol in addition to standard of care. The target enrolment number was N = 112. Due to the COVID-19 pandemic, the study enrolment period was truncated. The primary outcome was feasibility of the intervention as measured by the proportion of eligible cognitive interventions delivered by the OT. Secondary outcomes included feasibility of goal session length (20 min), participant clinical outcomes (delirium prevalence and duration, cognitive status, functional status, quality of life, and ICU length of stay), and a description of methodological challenges and solutions for future research.

RESULTS

Seventy patients were enrolled and 69 patients were included in the final analysis. The majority of OT-guided sessions (110/137; 80%) were completed. The mean (standard deviation [SD]) number of sessions per patient was 4.1 (3.8). The goal session length was achieved (mean [SD], 19.8 [3.1] min), with few sessions (8/110; 7%) terminated early per patient request.

CONCLUSION

This novel OT-guided cognitive intervention protocol is feasible in medical/surgical ICU patients. A larger randomized controlled trial is required to determine the impact of such a protocol on delirium prevalence or duration.

STUDY REGISTRATION

www.

CLINICALTRIALS

gov (NCT03604809); registered 18 June 2018.

Comparison of Prognostic Accuracy of 3 Delirium Prediction Models

BACKGROUND

Delirium is a severe complication in critical care patients. Accurate prediction could facilitate determination of which patients are at risk. In the past decade, several delirium prediction models have been developed.

OBJECTIVES

To compare the prognostic accuracy of the PRE-DELIRIC, E-PRE-DELIRIC, and Lanzhou models, and to investigate the difference in prognostic accuracy of the PRE-DELIRIC model between patients receiving and patients not receiving mechanical ventilation.

METHODS

This retrospective study involved adult patients admitted to the intensive care unit during a 2-year period. Delirium was assessed by using the Confusion Assessment Method for the Intensive Care Unit or any administered dose of haloperidol or quetiapine. Model discrimination was assessed by calculating the area under the receiver operating characteristic curve (AUC); values were compared using the DeLong test.

RESULTS

The study enrolled 1353 patients. The AUC values were calculated as 0.716 (95% CI, 0.688-0.745), 0.681 (95% CI, 0.650-0.712), and 0.660 (95% CI, 0.629-0.691) for the PRE-DELIRIC, E-PRE-DELIRIC, and Lanzhou models, respectively. The difference in model discrimination was statistically significant for comparison of the PRE-DELIRIC with the E-PRE-DELIRIC (AUC difference, 0.035; P = .02) and Lanzhou models (AUC difference, 0.056; P < .001). In the PRE-DELIRIC model, the AUC was 0.711 (95% CI, 0.680-0.743) for patients receiving mechanical ventilation and 0.664 (95% CI, 0.586-0.742) for those not receiving it (difference, 0.047; P = .27).

CONCLUSION

Statistically significant differences in prognostic accuracy were found between delirium prediction models. The PRE-DELIRIC model was the best-performing model and can be used in patients receiving or not receiving mechanical ventilation.

Two-stage prediction model for postoperative delirium in patients in the intensive care unit after cardiac surgery

OBJECTIVES

Postoperative delirium is a common severe complication in patients in the intensive care unit after cardiac surgery. We developed a two-stage prediction model and quantified the risk of developing postoperative delirium to assist in early prevention before and after surgery.

METHODS

We conducted a prospective cohort study and consecutively recruited adult patients after cardiac surgery. The Confusion Assessment Method for patients in the intensive care unit was used to diagnose delirium 5 days postoperatively. The stage I model was constructed using patient demographics, health conditions and laboratory results obtained preoperatively, whereas the stage II model was built on both pre- and postoperative predictors. The model was validated internally using the bootstrap method and externally using data from an external cohort.

RESULTS

The two-stage model was developed with 654 patients and was externally validated with 214 patients undergoing cardiac surgery. The stage I model contained 6 predictors, whereas the stage II model included 10 predictors. The stage I model had an area under the receiver operating characteristic curve of 0.76 (95% confidence interval: 0.68-0.81), and the stage II model's area under the receiver operating characteristic curve increased to 0.85 [95% confidence interval (CI): 0.81-0.89]. The external validation resulted in an area under the curve of 0.76 (95% CI: 0.67-0.86) for the stage I model and 0.78 (95% CI: 0.69-0.86) for the stage II model.

CONCLUSIONS

The two-stage model assisted medical staff in identifying patients at high risk for postoperative delirium before and 24 h after cardiac surgery. This model showed good discriminative power and predictive accuracy and can be easily accessed in clinical settings.

TRIAL REGISTRATION

The study was registered with the US National Institutes of Health ClinicalTrials.gov (NCT03704324; registered 11 October 2018).

Prediction models for postoperative delirium after cardiac surgery: Systematic review and critical appraisal

BACKGROUND

Many studies have developed or validated prediction models to estimate the risk of delirium after cardiac surgery, but the quality of the model development and model applicability remain unknown.

OBJECTIVES

To systematically review and critically evaluate currently available prediction models for delirium after cardiac surgery.

DATA SOURCES

PubMed, EMBASE, and MEDLINE were systematically searched. This systematic review was registered in PROSPERO (Registration ID: CRD42021251226).

STUDY SELECTION

Prospective or retrospective cohort studies were considered eligible if they developed or validated prediction models or scoring systems for delirium in the ICU. We included studies involving adults (age ≥18 years) undergoing cardiac surgery and excluded studies that did not validate a prediction model.

DATA EXTRACTION

Data extraction was independently performed by two authors using a standardized data extraction form based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies checklist. Quality of the models was assessed with the Prediction Model Risk of Bias Assessment Tool (PROBAST).

DATA SYNTHESIS

Of 5469 screened studies, 13 studies described 10 prediction models. The postoperative delirium incidence varied from 11.3 % to 51.6 %. The most frequently used predictors were age and cognitive impairment. The reported areas under the curve or C-statistics were between of 0.74 and 0.91 in the derivation set. The reported AUCs in the external validation set were between 0.54 and 0.90. All the studies had a high risk of bias, mainly owing to poor reporting of the outcome domain and analysis domain; 10 studies were of high concern regarding applicability.

CONCLUSIONS

The current models for predicting postoperative delirium in the ICU after cardiac surgery had a high risk of bias according to the PROBAST. Future studies should focus on improving current prediction models or developing new models with rigorous methodology.

Development and validation of a nomogram to predict postoperative delirium in type B aortic dissection patients underwent thoracic endovascular aortic repair

Objective

Postoperative delirium (POD) is a common postoperative complication after cardiovascular surgery with adverse outcomes. No prediction tools are currently available for assessing POD in the type B aortic dissection (TBAD) population. The purposes of this study were to develop and validate a nomogram for predicting POD among TBAD patients who underwent thoracic endovascular aortic repair (TEVAR).

Methods

The retrospective cohort included 631 eligible TBAD patients who underwent TEVAR from January 2019 to July 2021. 434 patients included before 2021 were in the develop set; 197 others were in the independent validation set. Least absolute shrinkage and selection operator (LASSO) and logistic regression were applied to identify the most useful predictive variables for constructing the nomogram. Discrimination and the agreement of the model was assessed with the area under the receiver operating characteristic curve (AUC), Brier score and the Hosmer-Lemeshow goodness-of-fit test. The results were validated using a bootstrap resampling and the validation set.

Results

The incidence rate of POD observed in the development and validation cohort were 15.0% and 14.2%, respectively. Seven independent risk factors, including age ≥60 years, syncope or coma, postoperative blood transfusion, atelectasis, estimated glomerular filtration rate (eGFR) &lt;80 ml/min/1.73 m2, albumin &lt;30 g/L, and neutrophil to lymphocyte ratio, were included in the nomogram. The model showed a good discrimination with an AUC of 0.819 (95% CI, 0.762–0.876) in the developed set, and adjusted to 0.797 (95% CI, 0.735–0.849) and 0.791 (95% CI, 0.700–0.881) in the internal validation set and the external validation, respectively. Favorable calibration of the nomogram was confirmed in both the development and validation cohorts.

Conclusion

The nomogram based on seven readily available predictors has sufficient validity to identify POD risk in this population. This tool may facilitate targeted initiation of POD preventive intervention for healthcare providers.

Validation of a delirium predictive model in patients admitted to surgical intensive care units: a multicentre prospective observational cohort study

OBJECTIVE

To internally and externally validate a delirium predictive model for adult patients admitted to intensive care units (ICUs) following surgery.

DESIGN

A prospective, observational, multicentre study.

SETTING

Three university-affiliated teaching hospitals in Thailand.

PARTICIPANTS

Adults aged over 18 years were enrolled if they were admitted to a surgical ICU (SICU) and had the surgery within 7 days before SICU admission.

MAIN OUTCOME MEASURES

Postoperative delirium was assessed using the Thai version of the Confusion Assessment Method for the ICU. The assessments commenced on the first day after the patient's operation and continued for 7 days, or until either discharge from the ICU or the death of the patient. Validation was performed of the previously developed delirium predictive model: age+(5×SOFA)+(15×benzodiazepine use)+(20×DM)+(20×mechanical ventilation)+(20×modified IQCODE>3.42).

RESULTS

In all, 380 SICU patients were recruited. Internal validation on 150 patients with the mean age of 75±7.5 years resulted in an area under a receiver operating characteristic curve (AUROC) of 0.76 (0.683 to 0.837). External validation on 230 patients with the mean age of 57±17.3 years resulted in an AUROC of 0.85 (0.789 to 0.906). The AUROC of all validation cohorts was 0.83 (0.785 to 0.872). The optimum cut-off value to discriminate between a high and low probability of postoperative delirium in SICU patients was 115. This cut-off offered the highest value for Youden's index (0.50), the best AUROC, and the optimum values for sensitivity (78.9%) and specificity (70.9%).

CONCLUSIONS

The model developed by the previous study was able to predict the occurrence of postoperative delirium in critically ill surgical patients admitted to SICUs.

TRIAL REGISTRATION NUMBER

Thai Clinical Trail Registry (TCTR20180105001).

Risk assessment of perioperative neurocognitive disorders, where are we now?

PURPOSE OF REVIEW

Perioperative neurocognitive disorders (PNDs) are among the most frequent complications after surgery and are associated with considerable morbidity and mortality. We analysed the recent literature regarding risk assessment of PND.

RECENT FINDINGS

Certain genetic variants of the cholinergic receptor muscarinic 2 and 4, as well as a marked degree of frailty but not the kind of anaesthesia (general or spinal) are associated with the risk to develop postoperative delirium (POD). Models predict POD with a discriminative power, for example, area under the receiver operating characteristics curve between 0.52 and 0.94.

SUMMARY

Advanced age as well as preexisting cognitive, functional and sensory deficits remain to be the main risk factors for the development of PND. Therefore, aged patients should be routinely examined for both preexisting and new developing deficits, as recommended in international guidelines. Appropriate tests should have a high discrimination rate, be feasible to be administered by staff that do not require excessive training, and only take a short time to be practical for a busy outpatient clinic. Models to predict PND, should be validated appropriately (and externally if possible) and should not contain a too large number of predictors to prevent overfitting of models.

Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare

Machine learning (ML) and artificial intelligence (AI) algorithms have the potential to derive insights from clinical data and improve patient outcomes. However, these highly complex systems are sensitive to changes in the environment and liable to performance decay. Even after their successful integration into clinical practice, ML/AI algorithms should be continuously monitored and updated to ensure their long-term safety and effectiveness. To bring AI into maturity in clinical care, we advocate for the creation of hospital units responsible for quality assurance and improvement of these algorithms, which we refer to as “AI-QI” units. We discuss how tools that have long been used in hospital quality assurance and quality improvement can be adapted to monitor static ML algorithms. On the other hand, procedures for continual model updating are still nascent. We highlight key considerations when choosing between existing methods and opportunities for methodological innovation.

2021 KSCCM clinical practice guidelines for pain, agitation, delirium, immobility, and sleep disturbance in the intensive care unit

We revised and expanded the “2010 Guideline for the Use of Sedatives and Analgesics in the Adult Intensive Care Unit (ICU).” We revised the 2010 Guideline based mainly on the 2018 “Clinical Practice Guidelines for the Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption (PADIS) in Adult Patients in the ICU,” which was an updated 2013 pain, agitation, and delirium guideline with the inclusion of two additional topics (rehabilitation/mobility and sleep). Since it was not possible to hold face-to-face meetings of panels due to the coronavirus disease 2019 (COVID-19) pandemic, all discussions took place via virtual conference platforms and e-mail with the participation of all panelists. All authors drafted the recommendations, and all panelists discussed and revised the recommendations several times. The quality of evidence for each recommendation was classified as high (level A), moderate (level B), or low/very low (level C), and all panelists voted on the quality level of each recommendation. The participating panelists had no conflicts of interest on related topics. The development of this guideline was independent of any industry funding. The Pain, Agitation/Sedation, Delirium, Immobility (rehabilitation/mobilization), and Sleep Disturbance panels issued 42 recommendations (level A, 6; level B, 18; and level C, 18). The 2021 clinical practice guideline provides up-to-date information on how to prevent and manage pain, agitation/sedation, delirium, immobility, and sleep disturbance in adult ICU patients. We believe that these guidelines can provide an integrated method for clinicians to manage PADIS in adult ICU patients.

Predicting brain function status changes in critically ill patients via Machine learning

OBJECTIVE

In intensive care units (ICUs), a patient's brain function status can shift from a state of acute brain dysfunction (ABD) to one that is ABD-free and vice versa, which is challenging to forecast and, in turn, hampers the allocation of hospital resources. We aim to develop a machine learning model to predict next-day brain function status changes.

MATERIALS AND METHODS

Using multicenter prospective adult cohorts involving medical and surgical ICU patients from 2 civilian and 3 Veteran Affairs hospitals, we trained and externally validated a light gradient boosting machine to predict brain function status changes. We compared the performances of the boosting model against state-of-the-art models-an ABD predictive model and its variants. We applied Shapley additive explanations to identify influential factors to develop a compact model.

RESULTS

There were 1026 critically ill patients without evidence of prior major dementia, or structural brain diseases, from whom 12 295 daily transitions (ABD: 5847 days; ABD-free: 6448 days) were observed. The boosting model achieved an area under the receiver-operating characteristic curve (AUROC) of 0.824 (95% confidence interval [CI], 0.821-0.827), compared with the state-of-the-art models of 0.697 (95% CI, 0.693-0.701) with P < .001. Using 13 identified top influential factors, the compact model achieved 99.4% of the boosting model on AUROC. The boosting and the compact models demonstrated high generalizability in external validation by achieving an AUROC of 0.812 (95% CI, 0.812-0.813).

CONCLUSION

The inputs of the compact model are based on several simple questions that clinicians can quickly answer in practice, which demonstrates the model has direct prospective deployment potential into clinical practice, aiding in critical hospital resource allocation.

Development of an early prediction model for postoperative delirium in neurosurgical patients admitted to the ICU after elective craniotomy (E-PREPOD-NS): A secondary analysis of a prospective cohort study

Postoperative delirium (POD) is a significant clinical problem in neurosurgical patients after intracranial surgery. Identification of high-risk patients may optimize perioperative management, but an adequate risk model for use at early phase after operation has not been developed. In the secondary analysis of a prospective cohort study, 800 adult patients admitted to the ICU after elective intracranial surgeries were included. The POD was diagnosed as Confusion Assessment Method for the ICU positive on postoperative day 1 to 3. Multivariate logistic regression analysis was used to develop early prediction model (E-PREPOD-NS) and the final model was validated with 200 bootstrap samples. The incidence of POD in this cohort was19.6%. We identified nine variables independently associated with POD in the final model: advanced age (OR 3.336, CI 1.765-6.305, 1 point), low education level (OR 2.528, 1.446-4.419, 1), smoking history (OR 2.582, 1.611-4.140, 1), diabetes (OR 2.541, 1.201-5.377, 1), supra-tentorial lesions (OR 3.424, 2.021-5.802, 1), anesthesia duration > 360 min (OR 1.686, 1.062-2.674, 0.5), GCS < 9 at ICU admission (OR 6.059, 3.789-9.690, 1.5), metabolic acidosis (OR 13.903, 6.248-30.938, 2.5), and neurosurgical drainage tube (OR 1.924, 1.132-3.269, 0.5). The area under the receiver operator curve (AUROC) of the risk score for prediction of POD was 0.865 (95% CI 0.835-0.895). The AUROC was 0.851 after internal validation (95% CI 0.791-0.912). The model showed good calibration. The E-PREPOD-NS model can predict POD in patients admitted to the ICU after elective intracranial surgery with good accuracy. External validation is needed in the future.

Development and Validation of a Postoperative Delirium Prediction Model for Elderly Orthopedic Patients in the Intensive Care Unit

We developed a prediction model for delirium in elderly patients in the intensive care unit who underwent orthopedic surgery and then temporally validated its predictive power in the same hospital. In the development stage, we designed a prospective cohort study, and 319 consecutive patients aged over 65 years from January 2018 to December 2019 were screened. Demographic characteristics and clinical variables were evaluated, and a final prediction model was developed using the multivariate logistic regression analysis. In the validation stage, 108 patients were included for temporal validation between January 2020 and June 2020. The effectiveness of the model was evaluated through discrimination and calibration. As a result, the prediction model contains seven risk factors (age, anesthesia method, score of mini-mental state examination, hypoxia, major hemorrhage, level of interleukin-6, and company of family members), which had an area under the receiver operating characteristics curve of 0.82 (95% confidence interval 0.76-0.88) and was stable after bootstrapping. The temporal validation resulted in an area under the curve of 0.80 (95% confidence interval 0.67-0.93). Our prediction model had excellent discrimination power in predicting postoperative delirium in elderly patients and could assist intensive care physicians with early prevention.

Heterogeneity in design and analysis of ICU delirium randomized trials: a systematic review

BACKGROUND

There is a growing number of randomized controlled trials (RCTs) evaluating interventions to prevent or treat delirium in the intensive care unit (ICU). Efforts to improve the conduct of delirium RCTs are underway, but none address issues related to statistical analysis. The purpose of this review is to evaluate heterogeneity in the design and analysis of delirium outcomes and advance methodological recommendations for delirium RCTs in the ICU.

METHODS

Relevant databases, including PubMed and Embase, were searched with no restrictions on language or publication date; the search was conducted on July 8, 2019. RCTs conducted on adult ICU patients with delirium as the primary outcome were included where trial results were available. Data on frequency and duration of delirium assessments, delirium outcome definitions, and statistical methods were independently extracted in duplicate. The review was registered with PROSPERO (CRD42020141204).

RESULTS

Among 65 eligible RCTs, 44 (68%) targeted the prevention of delirium. The duration of follow-up varied, with 31 (48%) RCTs having ≤7 days of follow-up, and only 24 (37%) conducting delirium assessments after ICU discharge. The incidence of delirium was the most common outcome (50 RCTs, 77%) for which 8 unique statistical methods were applied. The most common method, applied to 51 of 56 (91%) delirium incidence outcomes, was the two-sample test comparing the proportion of patients who ever experienced delirium. In the presence of censoring of patients at ICU discharge or death, this test may be misleading. The impact of censoring was also not considered in most analyses of the duration of delirium, as evaluated in 24 RCTs, with 21 (88%) delirium duration outcomes analyzed using a non-parametric test or two-sample t test. Composite outcomes (e.g., rank-based delirium- and coma-free days), used in 11 (17%) RCTs, seldom explicitly defined how ICU discharge, and death were incorporated into the definition and were analyzed using non-parametric tests (11 of 13 (85%) composite outcomes).

CONCLUSIONS

To improve delirium RCTs, outcomes should be explicitly defined. To account for censoring due to ICU discharge or death, survival analysis methods should be considered for delirium incidence and duration outcomes; non-parametric tests are recommended for rank-based delirium composite outcomes.

TRIAL REGISTRATION

PROSPERO CRD42020141204 . Registration date: 7/3/2019.

Delirium and Associated Length of Stay and Costs in Critically Ill Patients

Purpose

Delirium frequently affects critically ill patients in the intensive care unit (ICU). The purpose of this study is to evaluate the impact of delirium on ICU and hospital length of stay (LOS) and perform a cost analysis.

Materials and Methods

Prospective studies and randomized controlled trials of patients in the ICU with delirium published between January 1, 2015, and December 31, 2020, were evaluated. Outcome variables including ICU and hospital LOS were obtained, and ICU and hospital costs were derived from the respective LOS.

Results

Forty-one studies met inclusion criteria. The mean difference of ICU LOS between patients with and without delirium was significant at 4.77 days (p < 0.001); for hospital LOS, this was significant at 6.67 days (p < 0.001). Cost data were extractable for 27 studies in which both ICU and hospital LOS were available. The mean difference of ICU costs between patients with and without delirium was significant at $3,921 (p < 0.001); for hospital costs, the mean difference was $5,936 (p < 0.001).

Conclusion

ICU and hospital LOS and associated costs were significantly higher for patients with delirium, compared to those without delirium. Further research is necessary to elucidate other determinants of increased costs and cost-reducing strategies for critically ill patients with delirium. This can provide insight into the required resources for the prevention of delirium, which may contribute to decreasing healthcare expenditure while optimizing the quality of care.

Nursing intervention to prevent delirium in critically ill adults

OBJECTIVE

To determine the effectiveness of a nursing intervention for delirium prevention in critically ill patients.

METHOD

A quasi-experimental study was conducted with a non-equivalent control group and with evaluation before and after the intervention. 157 Patients were part of the intervention group and 134 of the control group. Patients were followed-up until they were discharged from the ICU or died. The incidence of delirium in both groups was compared. Additionally, the effect measures were adjusted for the propensity score.

RESULTS

The incidence and incidence rate of delirium in the control group were 20.1% and 33.1 per 1000 person-days (CI 95% 22.7 to 48.3) and in the intervention group was 0.6% and 0.64 per 1000 person-days (CI 95% 0.22 to 11.09), respectively. The crude Hazard Ratio was 0.06 (CI 95% 0,008 to 0,45) and adjusted 0.07 (CI 95% 0,009 to 0,60). The number needed to be treated was six.

CONCLUSION

Low incidence of delirium in critically ill patients intervened demonstrated the effectiveness of interventions. The average intervention time was 4 days with a 15-minutes dedication for each patient.

Development and External Validation of a Delirium Prediction Model for Hospitalized Patients With Coronavirus Disease 2019

Background

The coronavirus disease 2019 pandemic has placed unprecedented stress on health systems and has been associated with elevated risk for delirium. The convergence of pandemic resource limitation and clinical demand associated with delirium requires careful risk stratification for targeted prevention efforts.

Objectives

To develop an incident delirium predictive model among coronavirus disease 2019 patients.

Methods

We applied supervised machine learning to electronic health record data for inpatients with coronavirus disease 2019 at three hospitals to build an incident delirium diagnosis prediction model. We validated this model in three different hospitals. Both hospital cohorts included academic and community settings.

Results

Among 2907 patients across 6 hospitals, 488 (16.8%) developed delirium. Applying the predictive model in the external validation cohort of 755 patients, the c-index was 0.75 (0.71–0.79) and the lift in the top quintile was 2.1. At a sensitivity of 80%, the specificity was 56%, negative predictive value 92%, and positive predictive value 30%. Equivalent model performance was observed in subsamples stratified by age, sex, race, need for critical care and care at community vs. academic hospitals.

Conclusion

Machine learning applied to electronic health records available at the time of inpatient admission can be used to risk-stratify patients with coronavirus disease 2019 for incident delirium. Delirium is common among patients with coronavirus disease 2019, and resource constraints during a pandemic demand careful attention to the optimal application of predictive models.

Derivation and Validation of an Ensemble Model for the Prediction of Agitation in Mechanically Ventilated Patients Maintained Under Light Sedation

OBJECTIVES

Light sedation is recommended over deep sedation for invasive mechanical ventilation to improve clinical outcome but may increase the risk of agitation. This study aimed to develop and prospectively validate an ensemble machine learning model for the prediction of agitation on a daily basis.

DESIGN

Variables collected in the early morning were used to develop an ensemble model by aggregating four machine learning algorithms including support vector machines, C5.0, adaptive boosting with classification trees, and extreme gradient boosting with classification trees, to predict the occurrence of agitation in the subsequent 24 hours.

SETTING

The training dataset was prospectively collected in 95 ICUs from 80 Chinese hospitals on May 11, 2016, and the validation dataset was collected in 20 out of these 95 ICUs on December 16, 2019.

PATIENTS

Invasive mechanical ventilation patients who were maintained under light sedation for 24 hours prior to the study day and who were to be maintained at the same sedation level for the next 24 hours.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

A total of 578 invasive mechanical ventilation patients from 95 ICUs in 80 Chinese hospitals, including 459 in the training dataset and 119 in the validation dataset, were enrolled. Agitation was observed in 36% (270/578) of the invasive mechanical ventilation patients. The stepwise regression model showed that higher body temperature (odds ratio for 1°C increase: 5.29; 95% CI, 3.70-7.84; p < 0.001), greater minute ventilation (odds ratio for 1 L/min increase: 1.15; 95% CI, 1.02-1.30; p = 0.019), higher Richmond Agitation-Sedation Scale (odds ratio for 1-point increase: 2.43; 95% CI, 1.92-3.16; p < 0.001), and days on invasive mechanical ventilation (odds ratio for 1-d increase: 0.95; 95% CI, 0.93-0.98; p = 0.001) were independently associated with agitation in the subsequent 24 hours. In the validation dataset, the ensemble model showed good discrimination (area under the receiver operating characteristic curve, 0.918; 95% CI, 0.866-0.969) and calibration (Hosmer-Lemeshow test p = 0.459) in predicting the occurrence of agitation within 24 hours.

CONCLUSIONS

This study developed an ensemble model for the prediction of agitation in invasive mechanical ventilation patients under light sedation. The model showed good calibration and discrimination in an independent dataset.

Delirium care knowledge in critical care nurses: A multiple-choice question-based quiz

BACKGROUND

Lack of evidence regarding whether a useful examination instrument such as an multiple choice question (MCQ) quiz is reliable for assessing delirium care knowledge.

AIM

To develop and psychometrically test a MCQ-based quiz for assessing the delirium care knowledge in critical care nurses.

DESIGN

Instrument development and psychometric evaluation study.

METHODS

The development and validation process consisted of two phases. The first Phase focused on the quiz development, which was achieved through the following steps: (a) generation of an initial 20-item pool; (b) assessment of content validity; (c) assessment of face validity; (d) conduction of a pilot test, involving the collection of data from 217 critical care nurses through an online survey; and (e) item analysis and item elimination according to item difficulty and discrimination indices. The MCQ quiz was finalized through the development process. The second phase emphasized quiz validation through estimation of the internal consistency, split-half and test-retest reliability, and construct validity using parallel analysis with exploratory factor analysis (EFA).

RESULTS

A final 16-item MCQ quiz was emerged from the item analysis. The Kuder-Richardson formula 20 coefficient for the overall quiz indicated good internal consistency (0.85), and the intraclass correlation coefficient with a 30-day interval also indicated that the questionnaire had satisfactory stability (0.97). EFA confirmed that the quiz had appropriate construct validity, and four factors could explain 60.87% of the total variance.

CONCLUSION

In this study, the MCQ, and single best answer quiz for assessing delirium care knowledge was developed, and its reliability and validity for this purpose were demonstrated.

RELEVANCE TO CLINICAL PRACTICE

This study introduced an evidence-based quiz designed for future use in delirium care research and education that has significant implications for MCQ-based knowledge assessment in clinical practice.

Tools Are Needed to Promote Sedation Practices for Mechanically Ventilated Patients

Suboptimal sedation practices continue to be frequent, although the updated guidelines for management of pain, agitation, and delirium in mechanically ventilated (MV) patients have been published for several years. Causes of low adherence to the recommended minimal sedation protocol are multifactorial. However, the barriers to translation of these protocols into standard care for MV patients have yet to be analyzed. In our view, it is necessary to develop fresh insights into the interaction between the patients' responses to nociceptive stimuli and individualized regulation of patients' tolerance when using analgesics and sedatives. By better understanding this interaction, development of novel tools to assess patient pain tolerance and to define and predict oversedation or delirium may promote better sedation practices in the future.

ICU Delirium-Prediction Models: A Systematic Review

Supplemental Digital Content is available in the text.

Objective:

Summarize performance and development of ICU delirium-prediction models published within the past 5 years.

Data Sources:

Systematic electronic searches were conducted in April 2019 using PubMed, Embase, Cochrane Central, Web of Science, and Cumulative Index to Nursing and Allied Health Literature to identify peer-reviewed studies.

Study Selection:

Eligible studies were published in English during the past 5 years that specifically addressed the development, validation, or recalibration of delirium-prediction models in adult ICU populations.

Data Extraction:

Screened citations were extracted independently by three investigators with a 42% overlap to verify consistency using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies.

Data Synthesis:

Eighteen studies featuring 23 distinct prediction models were included. Model performance varied greatly, as assessed by area under the receiver operating characteristic curve (0.62–0.94), specificity (0.50–0.97), and sensitivity (0.45–0.96). Most models used data collected from a single time point or window to predict the occurrence of delirium at any point during hospital or ICU admission, and lacked mechanisms for providing pragmatic, actionable predictions to clinicians.

Conclusions:

Although most ICU delirium-prediction models have relatively good performance, they have limited applicability to clinical practice. Most models were static, making predictions based on data collected at a single time-point, failing to account for fluctuating conditions during ICU admission. Further research is needed to create clinically relevant dynamic delirium-prediction models that can adapt to changes in individual patient physiology over time and deliver actionable predictions to clinicians.

Utility of a prediction model for delirium in intensive care unit patients (PRE-DELIRIC) in mechanically ventilated patients with sepsis

Aim

Delirium frequently develops in patients with sepsis during their intensive care unit (ICU) stay, which is associated with increased morbidity and mortality. A prediction model for delirium in patients in ICU, PRE‐DELIRIC, has been utilized in overall ICU patients, but its utility is uncertain among patients with sepsis. This study aims to examine the utility of PRE‐DELIRIC to predict delirium in mechanically ventilated patients with sepsis.

Methods

This is a post hoc analysis of a randomized clinical trial in eight Japanese ICUs, which aimed to evaluate the sedative strategy with/without dexmedetomidine in adult mechanically ventilated patients with sepsis. The Confusion Assessment Method for the ICU was used every day to assess for delirium throughout their ICU stay. We excluded patients who were delirious on the first day of ICU, those who were under sustained coma throughout their ICU stay, and those who stayed in the ICU less than 24 h. The discriminative ability of PRE‐DELIRIC was evaluated by measuring the area under the receiver operating characteristic curve (AUROC).

Results

Of the 201 patients enrolled in the trial, we analyzed 158 patients. The mean age was 69.4 ± 14.0 years, and 99 patients (63%) were men. Delirium occurred at least once during the ICU stay of 63 patients (40%). The AUROC of PRE‐DELIRIC was 0.60 (95% confidence interval, 0.50–0.69). Subgroup analyses indicated that PRE‐DELIRIC was useful in those with Sequential Organ Failure Assessment score >8 with AUROC of 0.65 (95% confidence interval, 0.51–0.77).

Conclusions

The PRE‐DELIRIC model could not predict delirium in mechanically ventilated patients with sepsis.

Single assessment of delirium severity during postacute intensive care of chronically critically ill patients and its associated factors: post hoc analysis of a prospective cohort study in Germany

OBJECTIVES

To assess the delirium severity (DS), its risk factors and association with adverse patient outcomes in chronically critically ill (CCI) patients.

DESIGN

A prospective cohort study.

SETTING

A tertiary care hospital with postacute intensive care units (ICUs) in Germany.

PARTICIPANTS

N=267 CCI patients with critical illness polyneuropathy and/or critical illness myopathy, aged 18-75 years, who had undergone elective tracheotomy for weaning failure.

INTERVENTIONS

None.

MEASURES

Primary outcomes: DS was assessed using the Confusion Assessment Method for the Intensive Care Unit-7 delirium severity score, within 4 weeks (t1) after the transfer to a tertiary care hospital. In post hoc analyses, univariate linear regressions were employed, examining the relationship of DS with clinical, sociodemographic and psychological variables. Secondary outcomes: additionally, correlations of DS with fatigue (using the Multidimensional Fatigue Inventory-20), quality of life (using the Euro-Quality of Life) and institutionalisation/mortality at 3 (t2) and 6 (t3) months follow-up were computed.

RESULTS

Of the N=267 patients analysed, 9.4% showed severe or most severe delirium symptoms. 4.1% had a full-syndromal delirium. DS was significantly associated with the severity of illness (p=0.016, 95% CI -0.1 to -0.3), number of medical comorbidities (p<0.001, 95% CI .1 to .3) and sepsis (p<0.001, 95% CI .3 to 1.0). Patients with a higher DS at postacute ICU (t1), showed a higher mental fatigue at t2 (p=0.008, 95% CI .13 to .37) and an increased risk for institutionalisation/mortality (p=0.043, 95% CI 1.1 to 28.9/p=0.015, 95% CI 1.5 to 43.2).

CONCLUSIONS

Illness severity is positively associated with DS during postacute care in CCI patients. An adequate management of delirium is essential in order to mitigate functional and cognitive long-term sequelae following ICU.

TRIAL REGISTRATION NUMBER

DRKS00003386.

Validation of PREdiction of DELIRium in ICu patients (PRE-DELIRIC) among patients in intensive care units: A retrospective cohort study

BACKGROUND

An intensive care unit (ICU) delirium prediction tool, PREdiction of DELIRium in ICu patients (PRE-DELIRIC), has been developed and calibrated in a multinational project. However, there is a lack of evidence regarding the predictive ability of the PRE-DELIRIC among Chinese ICU patients.

AIM

To evaluate the predictive validity (discrimination and calibration) of PRE-DELIRIC.

DESIGN

This is a retrospective cohort study.

METHODS

A retrospective cohort study was conducted. Consecutive participants (a) admitted to the ICU for ≥24 hours, (b) aged ≥18 years, and (c) admitted to the ICU for the first time were included. Ten predictors (age, APACHE-II, urgent and admission category, urea level, metabolic acidosis, infection, coma, sedation, and morphine use) assessed within 24 hours upon ICU admission were assessed. Delirium was assessed using the Confusion Assessment Method for ICU. Outcomes included ICU length of stay and mortality. Discrimination and calibration were determined by the areas under the receiver operating characteristic curve (AUROC), box plot, and calibration plot.

RESULTS

A total of 375 ICU patients were included, with 44.0% of patients being delirious. Delirium was significantly associated with age, PRE-DELIRIC score, ICU length of stay, and mortality. The AUROC was 0.81 (95% confidence interval, 0.77-0.86). The optimal cut-off point identified by max Youden index was 49%. The calibration plot of pooled data demonstrated a calibration slope of 0.894 and an intercept of -0.178.

CONCLUSIONS

The PRE-DELIRIC has high predictive value and is suggested to be adopted in ICUs for early initiation of preventive interventions against delirium among high-risk patients.

RELEVANCE TO CLINICAL PRACTICE

Clinicians can adopt the PRE-DELIRIC among ICU patients to screen patients at high risk of developing delirium. Early initiative interventions could be implemented to reduce the negative impacts of ICU delirium.

Delirium risk prediction models for intensive care unit patients: A systematic review

OBJECTIVE

To systematically review the delirium risk prediction models for intensive care unit (ICU) patients.

METHODS

A systematic review was conducted. The Cochrane Library, PubMed, Ovid and Web of Science were searched to collect studies on delirium risk prediction models for ICU patients from database establishment to 31 March 2019. Two reviewers independently screened the literature according to the pre-determined inclusion and exclusion criteria, extracted the data and evaluated the risk of bias of the included studies using the CHARMS (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies) checklist. A descriptive analysis was used to describe and summarise the data.

RESULTS

A total of six models were included. All studies reported the area under the receiver operating characteristic curve (AUROC) of the prediction models in the derivation and (or) validation datasets as over 0.7 (from 0.75 to 0.9). Five models reported calibration metrics. Decreased cognitive reserve and the Acute Physiology and Chronic Health Evaluation II (APACHE-II) score were the most commonly reported predisposing and precipitating factors, respectively, of ICU delirium among all models. The small sample size, lack of external validation and the absence of or unreported blinding method increased the risk of bias.

CONCLUSION

According to the discrimination and calibration statistics reported in the original studies, six prediction models may have moderate power in predicting ICU delirium. However, this finding should be interpreted with caution due to the risk of bias in the included studies. More clinical studies should be carried out to validate whether these tools have satisfactory predictive performance in delirium risk prediction for ICU patients.

Delirium prediction in the intensive care unit: a temporal approach

The incidence of delirium in intensive care units is high and associated with poor outcomes; therefore, its prediction is desirable to establish preventive treatments. This retrospective study proposes a novel approach for delirium prediction. We analyzed static and temporal data from 10,475 patients admitted to one of 15 intensive care units (ICUs) in Alberta, Canada between January 1, 2014 and June 30, 2016. We tested 168 different combinations of study design parameters and five different predictive models (logistic regression, support vector machines, random forests, adaptive boosting and neural networks). The area under the receiver operating characteristic curve (AUROC) ranged from 0.754 (CI 95% ± 0.018) to 0.852 (± 0.033), with sensitivity and specificity respectively ranging from 0.739 (CI 95% ± 0.047) to 0.840 (CI 95% ± 0.064), and 0.770 (CI 95% ± 0.030) to 0.865 (CI 95% ± 0.038). These results are similar to previous studies; however, our approach allows for continuous updates and short-term prediction horizons which might provide major advantages.

External Validation of Two Models to Predict Delirium in Critically Ill Adults Using Either the Confusion Assessment Method-ICU or the Intensive Care Delirium Screening Checklist for Delirium Assessment

OBJECTIVES

To externally validate two delirium prediction models (early prediction model for ICU delirium and recalibrated prediction model for ICU delirium) using either the Confusion Assessment Method-ICU or the Intensive Care Delirium Screening Checklist for delirium assessment.

DESIGN

Prospective, multinational cohort study.

SETTING

Eleven ICUs from seven countries in three continents.

PATIENTS

Consecutive, delirium-free adults admitted to the ICU for greater than or equal to 6 hours in whom delirium could be reliably assessed.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The predictors included in each model were collected at the time of ICU admission (early prediction model for ICU delirium) or within 24 hours of ICU admission (recalibrated prediction model for ICU delirium). Delirium was assessed using the Confusion Assessment Method-ICU or the Intensive Care Delirium Screening Checklist. Discrimination was determined using the area under the receiver operating characteristic curve. The predictive performance was determined for the Confusion Assessment Method-ICU and Intensive Care Delirium Screening Checklist cohort, and compared with both prediction models' original reported performance. A total of 1,286 Confusion Assessment Method-ICU-assessed patients and 892 Intensive Care Delirium Screening Checklist-assessed patients were included. Compared with the area under the receiver operating characteristic curve of 0.75 (95% CI, 0.71-0.79) in the original study, the area under the receiver operating characteristic curve of the early prediction model for ICU delirium was 0.67 (95% CI, 0.64-0.71) for delirium as assessed using the Confusion Assessment Method-ICU and 0.70 (95% CI, 0.66-0.74) using the Intensive Care Delirium Screening Checklist. Compared with the original area under the receiver operating characteristic curve of 0.77 (95% CI, 0.74-0.79), the area under the receiver operating characteristic curve of the recalibrated prediction model for ICU delirium was 0.75 (95% CI, 0.72-0.78) for assessing delirium using the Confusion Assessment Method-ICU and 0.71 (95% CI, 0.67-0.75) using the Intensive Care Delirium Screening Checklist.

CONCLUSIONS

Both the early prediction model for ICU delirium and recalibrated prediction model for ICU delirium are externally validated using either the Confusion Assessment Method-ICU or the Intensive Care Delirium Screening Checklist for delirium assessment. Per delirium prediction model, both assessment tools showed a similar moderate-to-good statistical performance. These results support the use of either the early prediction model for ICU delirium or recalibrated prediction model for ICU delirium in ICUs around the world regardless of whether delirium is evaluated with the Confusion Assessment Method-ICU or Intensive Care Delirium Screening Checklist.

Association between delirium prediction scores and days spent with delirium

PURPOSE

To determine the correlation and discriminative value of the E-PRE-DELIRIC and PRE-DELIRIC scores with delirium exposure to evaluate the prognostic value of both models.

METHODS

A secondary analysis of a randomized clinical trial enrolling 1506 delirium-free, critically ill adults with an anticipated ICU stay of ≥2 days. Days spent with delirium (≥1 positive CAM-ICU) or coma (≥1 RASS ≤-4) in the 28-days after ICU admission were calculated. Patients were categorized into four groups: no delirium, short-exposure (1 delirium day), moderate-exposure (2-5 delirium days), and long- exposure (≥6 delirium days) to determine the correlation and discriminative value of the E-PRE-DELIRIC and the PRE-DELIRIC with days spent with delirium.

RESULTS

The correlation between the overall E-PRE-DELIRIC and PRE-DELIRIC scores and days spent with delirium were: R = 0.08 (P = .005) and R = 0.26 (P < .001), respectively. The correlation between both prediction scores and days spent with coma or delirium were R = 0.21 (P < .0001) and R = 0.46 (P < .0001), respectively. The highest Area Under the Receiver Operating Characteristic for both E-PRE-DELIRIC [0.57 (95% CI:0.51-0.62)] and PRE-DELIRIC [0.58 (95% CI:0.53-0.62)] was found in the long delirium exposure group.

CONCLUSION

The E-PRE-DELIRIC and PRE-DELIRIC model each poorly correlate and discriminate with days spent with delirium in the 28 days after ICU admission.

Development and validation of a postoperative delirium prediction model for patients admitted to an intensive care unit in China: a prospective study

OBJECTIVES

We aimed to develop and validate a postoperative delirium (POD) prediction model for patients admitted to the intensive care unit (ICU).

DESIGN

A prospective study was conducted.

SETTING

The study was conducted in the surgical, cardiovascular surgical and trauma surgical ICUs of an affiliated hospital of a medical university in Heilongjiang Province, China.

PARTICIPANTS

This study included 400 patients (≥18 years old) admitted to the ICU after surgery.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome measure was POD assessment during ICU stay.

RESULTS

The model was developed using 300 consecutive ICU patients and was validated using 100 patients from the same ICUs. The model was based on five risk factors: Physiological and Operative Severity Score for the enumeration of Mortality and morbidity; acid-base disturbance and history of coma, diabetes or hypertension. The model had an area under the receiver operating characteristics curve of 0.852 (95% CI 0.802 to 0.902), Youden index of 0.5789, sensitivity of 70.73% and specificity of 87.16%. The Hosmer-Lemeshow goodness of fit was 5.203 (p=0.736). At a cutoff value of 24.5%, the sensitivity and specificity were 71% and 69%, respectively.

CONCLUSIONS

The model, which used readily available data, exhibited high predictive value regarding risk of ICU-POD at admission. Use of this model may facilitate better implementation of preventive treatments and nursing measures.

Clinical Practice Guidelines for the Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption in Adult Patients in the ICU

OBJECTIVE

To update and expand the 2013 Clinical Practice Guidelines for the Management of Pain, Agitation, and Delirium in Adult Patients in the ICU.

DESIGN

Thirty-two international experts, four methodologists, and four critical illness survivors met virtually at least monthly. All section groups gathered face-to-face at annual Society of Critical Care Medicine congresses; virtual connections included those unable to attend. A formal conflict of interest policy was developed a priori and enforced throughout the process. Teleconferences and electronic discussions among subgroups and whole panel were part of the guidelines' development. A general content review was completed face-to-face by all panel members in January 2017.

METHODS

Content experts, methodologists, and ICU survivors were represented in each of the five sections of the guidelines: Pain, Agitation/sedation, Delirium, Immobility (mobilization/rehabilitation), and Sleep (disruption). Each section created Population, Intervention, Comparison, and Outcome, and nonactionable, descriptive questions based on perceived clinical relevance. The guideline group then voted their ranking, and patients prioritized their importance. For each Population, Intervention, Comparison, and Outcome question, sections searched the best available evidence, determined its quality, and formulated recommendations as "strong," "conditional," or "good" practice statements based on Grading of Recommendations Assessment, Development and Evaluation principles. In addition, evidence gaps and clinical caveats were explicitly identified.

RESULTS

The Pain, Agitation/Sedation, Delirium, Immobility (mobilization/rehabilitation), and Sleep (disruption) panel issued 37 recommendations (three strong and 34 conditional), two good practice statements, and 32 ungraded, nonactionable statements. Three questions from the patient-centered prioritized question list remained without recommendation.

CONCLUSIONS

We found substantial agreement among a large, interdisciplinary cohort of international experts regarding evidence supporting recommendations, and the remaining literature gaps in the assessment, prevention, and treatment of Pain, Agitation/sedation, Delirium, Immobility (mobilization/rehabilitation), and Sleep (disruption) in critically ill adults. Highlighting this evidence and the research needs will improve Pain, Agitation/sedation, Delirium, Immobility (mobilization/rehabilitation), and Sleep (disruption) management and provide the foundation for improved outcomes and science in this vulnerable population.

Incidence and risk factors of PostopeRativE delirium in intensive care unit patients: A study protocol for the PREDICt study

AIM

The aims of this study are: (a) to determine the incidence of postoperative delirium (POD) among surgical intensive care unit (ICU) patients in China and identify risk factors, especially, which are modifiable and have value for developing a prediction model; (b) to develop and validate a prediction model of delirium to recognize high-risk patients in surgical ICUs; (c) to investigate the short- and long-term outcomes of delirious patients and identify the predictors of patient outcomes.

DESIGN

A single-centre prospective cohort study.

METHODS

Patients will be enrolled from three surgical ICUs in a tertiary teaching hospital. Delirium assessment and perioperative data will be collected throughout the hospitalization. Delirious patients will be followed up for 2 years. The study was approved by the ethics committee in May 2018 and was funded by the clinical research grant from Zhongshan hospital, Fudan University, Shanghai.

DISCUSSION

Developing POD can be a burden to patients both for the short- and long-term period. Due to the lack of effective treatments for POD, prevention remains the best strategy. This study will provide an effective tool for early screening of high-risk patients of POD and provide a better understanding of the aetiology and outcome of delirium.

IMPACT

In clinical practice, a prediction model will offer an effective tool for ICU nurses to assess high-risk patients, which can support them to implement preventive strategies at the early stages to targeted patients. The follow-up results will help us better understand the impact of delirium on patients' long-term outcome.

Prevention and management of delirium in critically ill adult patients in the intensive care unit: a review based on the 2018 PADIS guidelines

Delirium is an acute, confusional state characterized by altered consciousness and a reduced ability to focus, sustain, or shift attention. It is associated with a number of complex underlying medical conditions and can be difficult to recognize. Many critically ill patients (e.g., up to 80% of patients in the intensive care unit [ICU]) experience delirium due to underlying medical or surgical health problems, recent surgical or other invasive procedures, medications, or various noxious stimuli (e.g., underlying psychological stressors, mechanical ventilation, noise, light, patient care interactions, and drug-induced sleep disruption or deprivation). Delirium is associated with a longer duration of mechanical ventilation and ICU admittance as well as an increased risk of death, disability, and long-term cognitive dysfunction. Therefore, the early recognition of delirium is important and ICU medical staff should devote careful attention to both watching for the occurrence of delirium and its prevention and management. This review presents a brief overview of delirium and an update of the literature with reference to the 2018 Society of Critical Care Medicine Clinical Practice Guidelines for the Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption in Adult Patients in the ICU.

Prediction of ICU Delirium: Validation of Current Delirium Predictive Models in Routine Clinical Practice

OBJECTIVES

To investigate the ability of available delirium risk assessment tools to identify patients at risk of delirium in an Australian tertiary ICU.

DESIGN

Prospective observational study.

SETTING

An Australian tertiary ICU.

PATIENTS

All patients admitted to the study ICU between May 8, 2017, and December 31, 2017, were assessed bid for delirium throughout their ICU stay using the Confusion Assessment Method for ICU. Patients were included in this study if they remained in ICU for over 24 hours and were excluded if they were delirious on ICU admission, or if they were unable to be assessed using the Confusion Assessment Method for ICU during their ICU stay. Delirium risk was calculated for each patient using the prediction of delirium in ICU patients, early prediction of delirium in ICU patients, and Lanzhou models. Data required for delirium predictor models were obtained retrospectively from patients medical records.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

There were 803 ICU admissions during the study period, of which 455 met inclusion criteria. 35.2% (n = 160) were Confusion Assessment Method for ICU positive during their ICU admission. Delirious patients had significantly higher Acute Physiology and Chronic Health Evaluation III scores (median, 72 vs 54; p < 0.001), longer ICU (median, 4.8 vs 1.8 d; p < 0.001) and hospital stay (16.0 vs 8.16 d; p < 0.001), greater requirement of invasive mechanical ventilation (70% vs 21.4%; p < 0.001), and increased ICU mortality (6.3% vs 2.4%; p = 0.037). All models included in this study displayed moderate to good discriminative ability. Area under the receiver operating curve for the prediction of delirium in ICU patients was 0.79 (95% CI, 0.75-0.83); recalibrated prediction of delirium in ICU patients was 0.79 (95% CI, 0.75-0.83); early prediction of delirium in ICU patients was 0.72 (95% CI, 0.67-0.77); and the Lanzhou model was 0.77 (95% CI, 0.72-0.81).

CONCLUSIONS

The predictive models evaluated in this study demonstrated moderate to good discriminative ability to predict ICU patients' risk of developing delirium. Models calculated at 24-hours post-ICU admission appear to be more accurate but may have limited utility in practice.

Diffusing an Innovation: Clinician Perceptions of Continuous Predictive Analytics Monitoring in Intensive Care

BACKGROUND

The purpose of this article is to describe neonatal intensive care unit clinician perceptions of a continuous predictive analytics technology and how those perceptions influenced clinician adoption. Adopting and integrating new technology into care is notoriously slow and difficult; realizing expected gains remain a challenge.

METHODS

Semistructured interviews from a cross-section of neonatal physicians (n = 14) and nurses (n = 8) from a single U.S. medical center were collected 18 months following the conclusion of the predictive monitoring technology randomized control trial. Following qualitative descriptive analysis, innovation attributes from Diffusion of Innovation Theory-guided thematic development.

RESULTS

Results suggest that the combination of physical location as well as lack of integration into work flow or methods of using data in care decisionmaking may have delayed clinicians from routinely paying attention to the data. Once data were routinely collected, documented, and reported during patient rounds and patient handoffs, clinicians came to view data as another vital sign. Through clinicians' observation of senior physicians and nurses, and ongoing dialogue about data trends and patient status, clinicians learned how to integrate these data in care decision making (e.g., differential diagnosis) and came to value the technology as beneficial to care delivery.

DISCUSSION

The use of newly created predictive technologies that provide early warning of illness may require implementation strategies that acknowledge the risk-benefit of treatment clinicians must balance and take advantage of existing clinician training methods.

Development and validation of a dynamic delirium prediction rule in patients admitted to the Intensive Care Units (DYNAMIC-ICU): A prospective cohort study

BACKGROUND

Delirium is one of the most common cognitive complications among patients admitted to the intensive care units (ICU).

OBJECTIVE

To develop and validate a DYNAmic deliriuM predICtion rule for ICU patients (DYNAMIC-ICU) and to stratify patients into different risk levels among patients in various types of ICUs.

DESIGN

Prospective cohort study.

SETTING AND PARTICIPANTS

A total of 560 (median age of 66 years, 62.5% male) consecutively enrolled patients from four ICUs were included in the study. The patients were randomly assigned into either the derivation (n = 336, 60%) or the validation (n = 224, 40%) cohort by stratified randomization based on delirium/non-delirium and types of ICU.

METHODS

The simplified Chinese version of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) was used to assess delirium until patients were discharged from the ICUs. Potential predisposing, disease-related, and iatrogenic and environmental risk factors as well as data on patients' outcomes were collected prospectively.

RESULTS

Of the enrolled patients, 20.2% and 20.5% developed delirium in the derivation and validation cohorts, respectively. Predisposing factors (history of chronic diseases, hearing deficits), disease-related factors (infection, higher APACHE II scores at admission), and iatrogenic and environmental factors (the use of sedatives and analgesics, indwelling catheter, and sleep disturbance) were identified as independent predictors of delirium. Points were assigned to each predictor according to their odds ratio to create a prediction rule which was internally validated based on total scores and by bootstrapping (AUCs of 0.907 [95% CI 0. 871 -0.944], 0.888 [95% CI 0.845-0.932], and 0.874 [95% CI 0.828-0.920]), respectively. The total score of the DYNAMIC-ICU ranged from 0 to 33 and patients were divided into low risk (0-9), moderate risk (10-17), high risk (18-33) groups in developing delirium according to their total score with incidence of delirium at 2.8%, 16.8% and 75.9% in the derivation group, respectively. The DYNAMIC-ICU and its performance of risk level stratification were further validated in the validation cohort (AUC = 0.900 [95% CI 0.858-0.941]). The all-cause mortality was increased and the length of hospital stay was prolonged dramatically with the increase of delirium risk levels in both derivation (p = 0.034, p < 0.001) and validation cohorts (p < 0.001, p < 0.001).

CONCLUSIONS

Seven predictors for ICU delirium were identified to create DYNAMIC-ICU, which could well stratify ICU patients into three different delirium risk levels, tailor risk level changes, and predict in-hospital outcomes by a dynamic assessment approach.

Assessment of delirium using the PRE-DELIRIC model in an intensive care unit in Argentina

OBJECTIVE

To describe the incidence of and risk factors for delirium in the intensive care unit of a tertiary care teaching hospital in Argentina and to conduct the first non-European study exploring the performance of the PREdiction of DELIRium in ICu patients (PRE-DELIRIC) model.

METHODS

Prospective observational study in a 20-bed intensive care unit of a tertiary care teaching hospital in Buenos Aires, Argentina. The PRE-DELIRIC model was applied to 178 consecutive patients within 24 hours of admission to the intensive care unit; delirium was assessed with the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU).

RESULTS

The mean age was 64.3 ± 17.9 years. The median time of stay in the intensive care unit was 6 (range, 2 - 56) days. Of the total number of patients, 49/178 (27.5%) developed delirium, defined as a positive CAM-ICU assessment, during their stay in the intensive care unit. Patients in the delirium group were significantly older and had a significantly higher Acute Physiological and Chronic Health Evaluation II (APACHE II) score. The mortality rate in the intensive care unit was 14.6%; no significant difference was observed between the two groups. Predictive factors for the development of delirium were increased age, prolonged intensive care unit stay, and opioid use. The area under the curve for the PRE-DELIRIC model was 0.83 (95%CI; 0.77 - 0.90).

CONCLUSIONS

The observed incidence of delirium highlights the importance of this problem in the intensive care unit setting. In this first study conducted outside Europe, PRE-DELIRIC accurately predicted the development of delirium.