Head-to-head comparison of 14 prediction models for postoperative delirium in elderly non-ICU patients: an external validation study

Introducing a machine learning algorithm for delirium prediction-the Supporting SURgery with GEriatric Co-Management and AI project (SURGE-Ahead)

INTRODUCTION

Post-operative delirium (POD) is a common complication in older patients, with an incidence of 14-56%. To implement preventative procedures, it is necessary to identify patients at risk for POD. In the present study, we aimed to develop a machine learning (ML) model for POD prediction in older patients, in close cooperation with the PAWEL (patient safety, cost-effectiveness and quality of life in elective surgery) project.

METHODS

The model was trained on the PAWEL study's dataset of 878 patients (no intervention, age ≥ 70, 209 with POD). Presence of POD was determined by the Confusion Assessment Method and a chart review. We selected 15 features based on domain knowledge, ethical considerations and a recursive feature elimination. A logistic regression and a linear support vector machine (SVM) were trained, and evaluated using receiver operator characteristics (ROC).

RESULTS

The selected features were American Society of Anesthesiologists score, multimorbidity, cut-to-suture time, estimated glomerular filtration rate, polypharmacy, use of cardio-pulmonary bypass, the Montreal cognitive assessment subscores 'memory', 'orientation' and 'verbal fluency', pre-existing dementia, clinical frailty scale, age, recent falls, post-operative isolation and pre-operative benzodiazepines. The linear SVM performed best, with an ROC area under the curve of 0.82 [95% CI 0.78-0.85] in the training set, 0.81 [95% CI 0.71-0.88] in the test set and 0.76 [95% CI 0.71-0.79] in a cross-centre validation.

CONCLUSION

We present a clinically useful and explainable ML model for POD prediction. The model will be deployed in the Supporting SURgery with GEriatric Co-Management and AI project.

Risk of bias in prognostic models of hospital-induced delirium for medical-surgical units: A systematic review

PURPOSE

The purpose of this systematic review was to assess risk of bias in existing prognostic models of hospital-induced delirium for medical-surgical units.

METHODS

APA PsycInfo, CINAHL, MEDLINE, and Web of Science Core Collection were searched on July 8, 2022, to identify original studies which developed and validated prognostic models of hospital-induced delirium for adult patients who were hospitalized in medical-surgical units. The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies was used for data extraction. The Prediction Model Risk of Bias Assessment Tool was used to assess risk of bias. Risk of bias was assessed across four domains: participants, predictors, outcome, and analysis.

RESULTS

Thirteen studies were included in the qualitative synthesis, including ten model development and validation studies and three model validation only studies. The methods in all of the studies were rated to be at high overall risk of bias. The methods of statistical analysis were the greatest source of bias. External validity of models in the included studies was tested at low levels of transportability.

CONCLUSIONS

Our findings highlight the ongoing scientific challenge of developing a valid prognostic model of hospital-induced delirium for medical-surgical units to tailor preventive interventions to patients who are at high risk of this iatrogenic condition. With limited knowledge about generalizable prognosis of hospital-induced delirium in medical-surgical units, existing prognostic models should be used with caution when creating clinical practice policies. Future research protocols must include robust study designs which take into account the perspectives of clinicians to identify and validate risk factors of hospital-induced delirium for accurate and generalizable prognosis in medical-surgical units.

Development and validation of an international preoperative risk assessment model for postoperative delirium

BACKGROUND

Postoperative delirium (POD) is a frequent complication in older adults, characterised by disturbances in attention, awareness and cognition, and associated with prolonged hospitalisation, poor functional recovery, cognitive decline, long-term dementia and increased mortality. Early identification of patients at risk of POD can considerably aid prevention.

METHODS

We have developed a preoperative POD risk prediction algorithm using data from eight studies identified during a systematic review and providing individual-level data. Ten-fold cross-validation was used for predictor selection and internal validation of the final penalised logistic regression model. The external validation used data from university hospitals in Switzerland and Germany.

RESULTS

Development included 2,250 surgical (excluding cardiac and intracranial) patients 60 years of age or older, 444 of whom developed POD. The final model included age, body mass index, American Society of Anaesthesiologists (ASA) score, history of delirium, cognitive impairment, medications, optional C-reactive protein (CRP), surgical risk and whether the operation is a laparotomy/thoracotomy. At internal validation, the algorithm had an AUC of 0.80 (95% CI: 0.77-0.82) with CRP and 0.79 (95% CI: 0.77-0.82) without CRP. The external validation consisted of 359 patients, 87 of whom developed POD. The external validation yielded an AUC of 0.74 (95% CI: 0.68-0.80).

CONCLUSIONS

The algorithm is named PIPRA (Pre-Interventional Preventive Risk Assessment), has European conformity (ce) certification, is available at http://pipra.ch/ and is accepted for clinical use. It can be used to optimise patient care and prioritise interventions for vulnerable patients and presents an effective way to implement POD prevention strategies in clinical practice.

Independent External Validation of a Preoperative Prediction Model for Delirium After Cardiac Surgery: A Prospective Observational Cohort Study

OBJECTIVE

This investigation provided independent external validation of an existing preoperative risk prediction model.

DESIGN

A prospective observational cohort study of patients undergoing cardiac surgery covering the period between April 16, 2018 and January 18, 2022.

SETTING

Two academic hospitals in Switzerland.

PARTICIPANTS

Adult patients (≥60 years of age) who underwent elective cardiac surgery, including coronary artery bypass graft, mitral or aortic valve replacement or repair, and combined procedures.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The primary outcome measure was the incidence of postoperative delirium (POD) in the intensive or intermediate care unit, diagnosed using the Intensive Care Delirium Screening Checklist. The prediction model contained 4 preoperative risk factors to which the following points were assigned: Mini-Mental State Examination (MMSE) score ≤23 received 2 points; MMSE 24-27, Geriatric Depression Scale (GDS) >4, prior stroke and/or transient ischemic attack (TIA), and abnormal serum albumin (≤3.5 or ≥4.5 g/dL) received 1 point each. The missing data were handled using multiple imputation. In total, 348 patients were included in the study. Sixty patients (17.4%) developed POD. For point levels in the prediction model of 0, 1, 2, and ≥3, the cumulative incidence of POD was 12.6%, 22.8%, 25.8%, and 35%, respectively. The validation resulted in a pooled area under the receiver operating characteristics curve of 0.60 (median CI, 0.525-0.679).

CONCLUSIONS

The evaluated predictive model for delirium after cardiac surgery in this patient cohort showed only poor discriminative capacity but fair calibration.