A Tool to Predict Readmission to the Intensive Care Unit in Surgical Critical Care Patients-The RISC Score

Multicenter validation of a machine learning model to predict intensive care unit readmission within 48 hours after discharge

Background

Intensive care unit (ICU) readmission is a crucial indicator of patient safety. However, discharge decisions often rely on subjective assessment due to a lack of standardized guidelines. We aimed to develop a machine-learning model to predict ICU readmission within 48 h and compare its performance to traditional scoring systems.

Methods

We developed an ensemble model, iREAD, that generates a probability score at ICU discharge, representing the likelihood of the patient being readmitted to the ICU within 48 h, using data from Seoul National University Hospital (SNUH) and validated it using the MIMIC-III and eICU-CRD datasets. From September 2007 to August 2021, a total of 70,842 patients were included from SNUH. The MIMIC-III datasets comprised 43,237 patients admitted to ICUs between 2001 and 2012 at Beth Israel Deaconess Medical Center, and the eICU-CRD datasets included 90,271 ICU admissions across 208 hospitals between 2014 and 2015. Patients younger than 18, those who died in ICUs, or who refused life-sustaining treatment were excluded from the final analysis. The model's performance was evaluated using the area under the receiver operating characteristic curve (AUROC) and compared to the traditional scores and conventional machine learning models. Kaplan-Meier analysis was performed to compare the outcome between the high-risk and low-risk groups.

Findings

We developed the iREAD, that utilized 30 input features, encompassing demographics, length of stay, vital signs, GCS, and laboratory values. iREAD demonstrated superior performance compared with other models across all cohorts (all P < 0.001). In the internal validation, iREAD achieved AUROCs of 0.771 (95% CI 0.743-0.798), 0.834 (0.821-0.846), and 0.820 (0.808-0.832) for early (≤48 h), late (>48 h), and overall ICU readmissions, respectively. External validations with MIMIC-III and eICU-CRD also showed modest performance with AUROCs of 0.768 (0.748-0.787) and 0.725 (0.712-0.739) for overall readmission in MIMIC-III and eICU-CRD respectively, demonstrating superior performance compared to other models (All P < 0.001; higher than other models). Kaplan-Meier analysis revealed that over 40% of high-risk patients predicted by iREAD were readmitted within 48 h, representing a more than four-fold increase in predictive performance compared to the traditional scores.

Interpretation

iREAD demonstrates superior performance in predicting ICU readmission within 48 h after discharge compared to traditional scoring systems or conventional machine learning models in both internal and external validations. While the performance degradation observed in the external validations suggests the need for further prospective validation on diverse patient populations, the robust performance and ability to identify high-risk patients have the potential to guide clinical decision-making.

Funding

This work was supported by the Korea Health Technology Research & Development Project through the Korea Health Industry Development Institute, funded by the Ministry of Health and Welfare, Republic of Korea (grant number RS-2021-KH114109).

Prediction model of ICU readmission in Chinese patients with acute type A aortic dissection: a retrospective study

BACKGROUND

Readmission to the intensive care unit (ICU) remains a severe challenge, leading to higher rates of death and a greater financial burden. This study aimed to develop a nomogram-based prediction model for individuals with acute type A aortic dissection (ATAAD).

METHODS

A total of 846 ATAAD patients were retrospectively enrolled between May 2014 and October 2021. Logistic regression was employed to identify the independent risk factors. The prediction model was evaluated using the Hosmer-Lemeshow (H-L) test, the calibration curve, and the area under the receiver operating characteristic curve (AUC). Decision curve analysis (DCA) was used to assess the clinical utility.

RESULTS

57 (6.7%) ATAAD patients were readmitted to ICU following their release from the ICU. ICU readmission was predicted with age ≥ 65 years old, body mass index (BMI) ≥ 28 kg/m2, tracheotomy, continuous renal replacement therapy (CRRT), and the length of initial ICU stay were predictors of ICU readmission. The AUC was 0.837 (95%CI: 0.789-0.884) and the model fit the data well (H-L test, P = 0.519). DCA also demonstrated good clinical practicability.

CONCLUSIONS

This prediction model may be helpful for clinicians to assess the risk of ICU readmission, and facilitate the early identification of ATAAD patients at high risk.

Intensive Care Unit Readmissions in a Level I Trauma Center

INTRODUCTION

Intensive care unit (ICU) readmissions are associated with increased morbidity and mortality rates, longer hospitalization, and increased health-care expenditures. This study sought to present a large cohort of trauma patients readmitted to the ICU, characterizing risk factors and providing quality improvement strategies to limit ICU readmission.

METHODS

A retrospective cohort analysis was conducted on adult trauma patients admitted to the ICU at a single level I trauma center from 2014 to 2021. Patients were split into readmission and no readmission groups. Patients experiencing readmission were compared to a similar group that was not readmitted using descriptive statistics and logistic regression.

RESULTS

In this study, 3632 patients were included and 278 (7.7%) were readmitted to the ICU. Significant differences were found in age, Elixhauser Comorbidity score, number of days on a ventilator, and number of patients requiring ventilator support. Furthermore, logistic regression showed that increasing age and the Elixhauser Comorbidity Score were associated with an increased likelihood of ICU readmission. Over the study period, the ICU readmission rate increased while the ICU length decreased.

CONCLUSIONS

Age, Elixhauser Comorbidity score, and ventilator use were all significant risk factors for ICU readmission. During our study period, a concerning trend of increasing ICU readmissions and decreased ICU length of stay was found. By identifying this trend, our institution was able to employ mitigation strategies that have successfully reversed the trend in ICU readmissions, decreasing the rate below the national average.

Predicting Unplanned Readmissions to the Intensive Care Unit in the Trauma Population

Background: Unplanned readmission to intensive care units (UR-ICU) in trauma is associated with increased hospital length of stay and significant morbidity and mortality. We identify independent predictors of UR-ICU and construct a nomogram to estimate readmission probability. Materials and Methods: We performed an IRB-approved retrospective case-control study at a Level I trauma center between January 2019 and December 2021. Patients with UR-ICU (n = 175) were matched with patients who were not readmitted (NR-ICU) (n = 175). Univariate and multivariable binary linear regressionanalyses were performed (SPSS Version 28, IBM Corp), and a nomogram was created (Stata 18.0, StataCorp LLC). Results: Demographics, comorbidities, and injury- and hospital course-related factors were examined as potential prognostic indicators of UR-ICU. The mortality rate of UR-ICU was 22.29% vs 6.29% for NR-ICU (P < .001). Binary linear regression identified seven independent predictors that contributed to UR-ICU: shock (P < .001) or intracranial surgery (P = .015) during ICU admission, low hematocrit (P = .001) or sedation administration in the 24 hours before ICU discharge (P < .001), active infection treatment (P = .192) or leukocytosis on ICU discharge (P = .01), and chronic obstructive pulmonary disease (COPD) (P = .002). A nomogram was generated to estimate the probability of UR-ICU and guide decisions on ICU discharge appropriateness. Discussion: In trauma, UR-ICU is often accompanied by poor outcomes and death. Shock, intracranial surgery, anemia, sedative administration, ongoing infection treatment, leukocytosis, and COPD are significant risk factors for UR-ICU. A predictive nomogram may help better assess readiness for ICU discharge.

The association of the Sequential Organ Failure Assessment score at intensive care unit discharge with intensive care unit readmission in the cardiac intensive care unit

AIMS

Unplanned intensive care unit (ICU) readmissions contribute to increased morbidity, mortality, and healthcare costs. The severity of patient illness at ICU discharge may predict early ICU readmission. Thus, in this study, we investigated the association of cardiac ICU (CICU) discharge Sequential Organ Failure Assessment (SOFA) score with unplanned CICU readmission in patients admitted to the CICU.

METHODS AND RESULTS

We retrospectively reviewed the hospital medical records of 4659 patients who were admitted to the CICU from 2012 to 18. Sequential Organ Failure Assessment scores at CICU admission and discharge were obtained. The predictive performance of organ failure scoring was evaluated by using area under the receiver operating characteristic (AUROC) curves. The primary outcome was unplanned CICU readmission. Of the 3949 patients successfully discharged from the CICU, 184 (4.7%) had an unplanned CICU readmission or they experienced a deteriorated condition but died without being readmitted to the CICU (readmission group). The readmission group had significantly higher rates of organ failure in all organ systems at both CICU admission and discharge than the non-readmission group. The AUROC of the discharge SOFA score for CICU readmission was 0.731, showing good predictive performance. The AUROC of the discharge SOFA score was significantly greater than that of either the initial SOFA score (P = 0.020) or the Acute Physiology and Chronic Health Evaluation II score (P < 0.001). In the multivariable regression analysis, SOFA score, overweight or obese status, history of heart failure, and acute heart failure as reasons for ICU admission were independent predictors of unplanned ICU readmission during the same hospital stay.

CONCLUSION

The discharge SOFA score may identify patients at a higher risk of unplanned CICU readmission, enabling targeted interventions to reduce readmission rates and improve patient outcomes.

Prognostic factors for critically ill surgical patients with unplanned intensive care unit readmission: Developing a novel predictive scoring model for predicting readmission

BACKGROUND

Unplanned readmission to the surgical intensive care unit has been demonstrated to worsen patient outcomes. Our objective was to identify risk factors and outcomes associated with unplanned surgical intensive care unit readmission and to develop a predictive scoring model to identify patients at high risk of readmission.

METHODS

We retrospectively analyzed patients admitted to the surgical intensive care unit (2020-2021) and categorized them as either with or without unplanned readmission.

RESULTS

Of 1,112 patients in the derivation cohort, 76 (6.8%) experienced unplanned surgical intensive care unit readmission, with sepsis being the leading cause of readmission (35.5%). Patients who were readmitted had significantly higher in-hospital mortality rates than those who were not. Multivariate analysis identified congestive heart failure, high Sequential Organ Failure Assessment-Hepatic score, use of carbapenem during surgical intensive care unit stay, as well as factors before surgical intensive care unit discharge such as inadequate glycemic control, positive fluid balance, low partial pressure of oxygen in arterial blood/fraction of inspired oxygen ratio, and receipt of total parenteral nutrition as independent predictors for unplanned readmission. The scoring model developed using these predictors exhibited good discrimination between readmitted and non-readmitted patients, with an area under the curve of 0.74. The observed rates of unplanned readmission for scores of <4 points and ≥4 points were 4% and 20.2% (P < .001), respectively. The model also demonstrated good performance in the validation cohort, with an area under the curve of 0.74 and 19% observed unplanned readmission rate for scores ≥4 points.

CONCLUSION

Besides congestive heart failure, clinicians should meticulously re-evaluate critical variables such as the Sequential Organ Failure Assessment-Hepatic score, partial pressure of oxygen in arterial blood/fraction of inspired oxygen ratio, glycemic control, and fluid status before releasing the patient from the surgical intensive care unit. It is crucial to determine the reasons for using carbapenems during surgical intensive care unit stay and the causes for the inability to discontinue total parenteral nutrition before discharging the patient from the surgical intensive care unit.

Perioperative risk factors associated with unplanned neurological intensive care unit readmission following elective supratentorial brain tumor resection

OBJECTIVE

The aim of this study was to describe the clinical and procedural risk factors associated with the unplanned neurosurgical intensive care unit (NICU) readmission of patients after elective supratentorial brain tumor resection and serves as an exploratory analysis toward the development of a risk stratification tool that may be prospectively applied to this patient population.

METHODS

This was a retrospective observational cohort study. The electronic medical records of patients admitted to an institutional NICU between September 2018 and November 2021 after elective supratentorial brain tumor resection were reviewed. Demographic and perioperative clinical factors were recorded. A prognostic model was derived from the data of 4892 patients recruited between September 2018 and May 2021 (development cohort). A nomogram was created to display these predictor variables and their corresponding points and risks of readmission. External validation was evaluated using a series of 1118 patients recruited between June 2021 and November 2021 (validation cohort). Finally, a decision curve analysis was performed to determine the clinical usefulness of the prognostic model.

RESULTS

Of the 4892 patients in the development cohort, 220 (4.5%) had an unplanned NICU readmission. Older age, lesion type, Karnofsky Performance Status (KPS) < 70 at admission, longer duration of surgery, retention of endotracheal intubation on NICU entry, and longer NICU length of stay (LOS) after surgery were independently associated with an unplanned NICU readmission. A total of 1118 patients recruited between June 2021 and November 2021 were included for external validation, and the model's discrimination remained acceptable (C-statistic = 0.744, 95% CI 0.675-0.814). The decision curve analysis for the prognostic model in the development and validation cohorts showed that at a threshold probability between 0.05 and 0.8, the prognostic model showed a positive net benefit.

CONCLUSIONS

A predictive model that included age, lesion type, KPS < 70 at admission, duration of surgery, retention of endotracheal intubation on NICU entry, and NICU LOS after surgery had an acceptable ability to identify elective supratentorial brain tumor resection patients at high risk for an unplanned NICU readmission. These risk factors and this prediction model may facilitate better resource allocation in the NICU and improve patient outcomes.

Intensive care unit readmission and unexpected death after emergency general surgery

Background

Intensive care unit (ICU) readmission and unexpected death are closely associated with increased length of hospitalization and total mortality. However, data about readmission or unexpected death after discharge from ICU in patients who have undergone emergency general surgery (EGS) is very limited.

Methods

In total, 1133 patients who underwent EGS were identified in the Multiparameter Intelligent Monitoring in Intensive Care IV (MIMIC-IV) database. Of these 1133 patients, 124 underwent readmission into the ICU or death unexpectedly after their initial discharge. The clinical characteristics of the patients were investigated. A logistic regression model was implemented for the analysis of the independent risk factors associated with ICU readmission or unexpected death. A nomogram model was established to predict the risk of ICU readmission or unexpected death within 72 h after EGS.

Results

Peripheral vascular disease and atrial fibrillation, vasopressor requirement, a higher respiratory rate or heart rate, a lower pulse oxygen saturation or a platelet count of <150 K/μL and a relatively low Glasgow coma scale score in the last 24 h before ICU discharge were independent risk factors for ICU readmission or death within 72 h. The nomogram had moderate accuracy with an area under the curve of 0.852, which had a stronger prediction power than the Stability and Workload Index for Transfer (SWIFT) score, a classic prediction model for ICU readmission risk.

Conclusions

In critically ill patients who undergo EGS, ICU readmission or unexpected death within 72 h can be predicted using a nomogram model based on eight parameters including physiological and laboratory test values in the last 24 h before discharge and comorbidities. ICU physicians should prudently assess patients to make effective discharge decisions.

The risk assessment tool for intensive care unit readmission: A systematic review and meta-analysis

OBJECTIVE

To review and evaluate existing risk assessment tools for intensive care unitreadmission.

METHODS

Nine electronic databases (Medline, CINAHL, Web of Science, Cochrane Library, Embase, Sino Med, CNKI, VIP, and Wan fang) were systematically searched from their inception to September 2022. Two authors independently extracted data from the literature included. Meta-analysis was performed under the bivariate modeling and summary receiver operating characteristic curve method.

RESULTS

A total of 29 studies were included in this review, among which 11 were quantitatively Meta-analyzed. The results showed Stability and Workload Index for Transfer: Sensitivity = 0.55, Specificity = 0.65, Area under curve = 0.63. And Early warning score: Sensitivity = 0.78, Specificity = 0.83, Area under curve = 0.88. The remaining tools included scores, nomograms, machine learning models, and deep learning models. These studies, with varying reports on thresholds, case selection, data preprocessing, and model performance, have a high risk of bias.

CONCLUSION

We cannot identify a tool that can be used directly in intensive care unit readmission risk assessment. Scores based on early warning score are moderately accurate in predicting readmission, but there is heterogeneity and publication bias that requires model adjustment for local factors such as resources, demographics, and case mix. Machine learning models present a promising modeling technique but have a high methodological bias and require further validation.

IMPLICATIONS FOR CLINICAL PRACTICE

Using reliable risk assessment tools is essential for the early identification of unplanned intensive care unit readmission risk in critically ill patients.  A reliable risk assessment tool must be developed, which is the focus of further research.

Predictive Modeling for Readmission to Intensive Care: A Systematic Review

To evaluate the methodologic rigor and predictive performance of models predicting ICU readmission; to understand the characteristics of ideal prediction models; and to elucidate relationships between appropriate triage decisions and patient outcomes.

DATA SOURCES

PubMed, Web of Science, Cochrane, and Embase.

STUDY SELECTION

Primary literature that reported the development or validation of ICU readmission prediction models within from 2010 to 2021.

DATA EXTRACTION

Relevant study information was extracted independently by two authors using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist. Bias was evaluated using the Prediction model Risk Of Bias ASsessment Tool. Data sources, modeling methodology, definition of outcomes, performance, and risk of bias were critically evaluated to elucidate relevant relationships.

DATA SYNTHESIS

Thirty-three articles describing models were included. Six studies had a high overall risk of bias due to improper inclusion criteria or omission of critical analysis details. Four other studies had an unclear overall risk of bias due to lack of detail describing the analysis. Overall, the most common (50% of studies) source of bias was the filtering of candidate predictors via univariate analysis. The poorest performing models used existing clinical risk or acuity scores such as Acute Physiologic Assessment and Chronic Health Evaluation II, Sequential Organ Failure Assessment, or Stability and Workload Index for Transfer as the sole predictor. The higher-performing ICU readmission prediction models used homogenous patient populations, specifically defined outcomes, and routinely collected predictors that were analyzed over time.

CONCLUSIONS

Models predicting ICU readmission can achieve performance advantages by using longitudinal time series modeling, homogenous patient populations, and predictor variables tailored to those populations.

Development of the PIP score: A metric for predicting Intensive Care Unit admission among patients undergoing emergency laparotomy

Background

Emergency laparotomy cases account for a significant proportion of the surgical caseload requiring postoperative intensive care. However, access to Intensive Care Unit (ICU) services has been limited by the scarcity of resources, lack of guidelines, and paucity of triaging tools.

Objective

This study aimed at developing a feasible Post-emergency laparotomy ICU admission Predictive (PIP) scoring tool.

Methodology

A case-control study utilizing the records of 108 patients who underwent emergency laparotomy was conducted. The primary outcome was the postoperative disposition status. Cases were defined as emergency laparotomy patients admitted to the ICU. The control group constituted patients admitted to the general ward. Logistic regression analysis was performed to identify the perioperative predictors of outcome. The PIP score was developed as a composite of each statistically significant variable remaining in the final logistic regression model.

Results

The significant positive predictors of ICU admission included a worsening American Society of Anesthesiologists - Physical Status, decreasing preoperative baseline axillary temperature, increasing preoperative baseline pulse rate, and intraoperative blood-product transfusion. The scoring system incorporating the identified predictors was presented as a numeric scale ranging from zero to four. Two levels of prediction were defined with reference to the optimum cut-off value; a score of <3 (low-intermediate prediction) and a score of ≥3 (high prediction [OR = 37.00, 95% CI = 11.22-122.02, p <0.001]). The score demonstrated an excellent predictive ability on the Receiver Operator Characteristic Curve (Area Under the Curve = 0.91, 95% CI = 0.851-0.973, p <0.001).

Conclusion

The PIP score proves useful as a feasible postoperative triaging adjunct for emergency laparotomy cases. Nonetheless, further validation studies are required.

Week-One Anaemia was Associated with Increased One-Year Mortality in Critically Ill Surgical Patients

BACKGROUND

Anaemia has a deleterious effect on surgical patients, but the long-term impact of anaemia in critically ill surgical patients remains unclear.

METHODS

We enrolled consecutive patients who were admitted to surgical intensive care units (ICUs) at a tertiary referral centre in central Taiwan between 2015 and 2020. We used both Cox proportional hazards analysis and propensity score-based analyses, including propensity score matching (PSM), inverse probability of treatment weighting (IPTW), and covariate balancing propensity score (CBPS) to determine hazard ratios (HRs) and 95% confidence intervals (CIs) for one-year mortality.

RESULTS

A total of 7,623 critically ill surgical patients were enrolled, and 29.9% (2,280/7,623) of them had week-one anaemia (haemoglobin <10 g/dL). We found that anaemia was independently associated with an increased risk of one-year mortality after adjustment for relevant covariates (aHR, 1.170; 95% CI, 1.045-1.310). We further identified a consistent strength of association between anaemia and one-year mortality in propensity score-based analyses, with the adjusted HRs in the PSM, IPTW, and CBPS were 1.164 (95% CI 1.025-1.322), 1.179 (95% CI 1.030-1.348), and 1.181 (1.034-1.349), respectively.

CONCLUSIONS

We identified the impact on one-year mortality of anaemia in critically ill surgical patients, and more studies are needed to validate our findings.

The Promise for Reducing Healthcare Cost with Predictive Model: An Analysis with Quantized Evaluation Metric on Readmission

Quality of care data has gained transparency captured through various measurements and reporting. Readmission measure is especially related to unfavorable patient outcomes that directly bends the curve of healthcare cost. Under the Hospital Readmission Reduction Program, payments to hospitals were reduced for those with excessive 30-day rehospitalization rates. These penalties have intensified efforts from hospital stakeholders to implement strategies to reduce readmission rates. One of the key strategies is the deployment of predictive analytics stratified by patient population. The recent research in readmission model is focused on making its prediction more accurate. As cost-saving improvements through artificial intelligent-based health solutions are expected, the broad economic impact of such digital tool remains unknown. Meanwhile, reducing readmission rate is associated with increased operating expenses due to targeted interventions. The increase in operating margin can surpass native readmission cost. In this paper, we propose a quantized evaluation metric to provide a methodological mean in assessing whether a predictive model represents cost-effective way of delivering healthcare. Herein, we evaluate the impact machine learning has had on transitional care and readmission with proposed metric. The final model was estimated to produce net healthcare savings at over $1 million given a 50% rate of successfully preventing a readmission.

Current Trends in Readmission Prediction: An Overview of Approaches

Hospital readmission shortly after discharge threatens the quality of patient care and leads to increased medical care costs. In the United States, hospitals with high readmission rates are subject to federal financial penalties. This concern calls for incentives for healthcare facilities to reduce their readmission rates by predicting patients who are at high risk of readmission. Conventional practices involve the use of rule-based assessment scores and traditional statistical methods, such as logistic regression, in developing risk prediction models. The recent advancements in machine learning driven by improved computing power and sophisticated algorithms have the potential to produce highly accurate predictions. However, the value of such models could be overrated. Meanwhile, the use of other flexible models that leverage simple algorithms offer great transparency in terms of feature interpretation, which is beneficial in clinical settings. This work presents an overview of the current trends in risk prediction models developed in the field of readmission. The various techniques adopted by researchers in recent years are described, and the topic of whether complex models outperform simple ones in readmission risk stratification is investigated.