Cross-validation of a modified score to predict mortality in cancer patients admitted to the intensive care unit

Evaluating High-Dimensional Machine Learning Models to Predict Hospital Mortality Among Older Patients With Cancer

PURPOSE

Older hospitalized cancer patients face high risks of hospital mortality. Improved risk stratification could help identify high-risk patients who may benefit from future interventions, although we lack validated tools to predict in-hospital mortality for patients with cancer. We evaluated the ability of a high-dimensional machine learning prediction model to predict inpatient mortality and compared the performance of this model to existing prediction indices.

METHODS

We identified patients with cancer older than 75 years from the National Emergency Department Sample between 2016 and 2018. We constructed a high-dimensional predictive model called Cancer Frailty Assessment Tool (cFAST), which used an extreme gradient boosting algorithm to predict in-hospital mortality. cFAST model inputs included patient demographic, hospital variables, and diagnosis codes. Model performance was assessed with an area under the curve (AUC) from receiver operating characteristic curves, with an AUC of 1.0 indicating perfect prediction. We compared model performance to existing indices including the Modified 5-Item Frailty Index, Charlson comorbidity index, and Hospital Frailty Risk Score.

RESULTS

We identified 2,723,330 weighted emergency department visits among older patients with cancer, of whom 144,653 (5.3%) died in the hospital. Our cFAST model included 240 features and demonstrated an AUC of 0.92. Comparator models including the Modified 5-Item Frailty Index, Charlson comorbidity index, and Hospital Frailty Risk Score achieved AUCs of 0.58, 0.62, and 0.71, respectively. Predictive features of the cFAST model included acute conditions (respiratory failure and shock), chronic conditions (lipidemia and hypertension), patient demographics (age and sex), and cancer and treatment characteristics (metastasis and palliative care).

CONCLUSION

High-dimensional machine learning models enabled accurate prediction of in-hospital mortality among older patients with cancer, outperforming existing prediction indices. These models show promise in identifying patients at risk of severe adverse outcomes, although additional validation and research studying clinical implementation of these tools is needed.

Effect of hydrocortisone versus methylprednisolone on clinical outcomes in oncology patients with septic shock

BACKGROUND

Corticosteroids are used as adjunctive treatment of critical illness-related corticosteroid insufficiency in patients with septic shock. This study aims to compare the impact of hydrocortisone versus methylprednisolone on duration of septic shock in critically ill oncology patients.

METHODS

Single-center, retrospective cohort study of adult patients receiving hydrocortisone ≥200 mg/day or methylprednisolone ≥40 mg/day with septic shock. The primary outcome was time to shock reversal defined as time to systolic blood pressure ≥90 mmHg without vasopressors for ≥24 h.

RESULTS

Eighty-eight patients were included, 49 patients received hydrocortisone and 39 patients received methylprednisolone. Solid tumor malignancy was more common in the hydrocortisone group, while hematological malignancy was more common in the methylprednisolone group (p = 0.009). Time to shock reversal was similar between hydrocortisone and methylprednisolone groups (72.4 versus 70.4 h; p = 0.825). Intensive care unit mortality occurred in 51.02% versus 53.85% of patients in hydrocortisone versus methylprednisolone, respectively (p = 0.792). Patients who received methylprednisolone had higher rates of mechanical ventilation (89.74% versus 55.1%, p < 0.001) and longer intensive care unit and hospital lengths of stay (4.2 versus 11.4 days and 14.3 versus 25.7 days; p < 0.001) compared to hydrocortisone. No differences were seen in incidence of steroid-related adverse effects between groups.

CONCLUSIONS

In oncology patients with septic shock, the use of hydrocortisone versus methylprednisolone does not appear to affect time to shock reversal.

Deep Natural Language Processing Identifies Variation in Care Preference Documentation

CONTEXT

Documentation of care preferences within 48 hours of admission to an intensive care unit (ICU) is a National Quality Forum-endorsed quality metric for older adults. Care preferences are poorly captured by administrative data.

OBJECTIVES

Using deep natural language processing, our aim was to determine the rate of care preference documentation in free-text notes and to assess associated patient factors.

METHODS

Retrospective review of notes by clinicians using a deep natural language processing to identify care preference documentation, including goals-of-care and treatment limitations, within 48 hours of ICU admission within five ICUs (medical, cardiac, surgery, trauma surgery, and cardiac surgery) for adults 75 years and older. Covariates included demographics, ICU type, sequential organ failure assessment score, and need for mechanical ventilation.

RESULTS

Deep natural language processing reviewed 11,575 clinician notes for 1350 ICU admissions. Median patient age was 84.0 years (interquartile range 78.0-88.4). Overall, 64.7% had documentation of care preferences. Patients with documentation were older (85 vs. 83 years; P < 0.001) and more often female (53.8% vs. 43.4%; P < 0.001). In adjusted analysis, rates of care preference documentation were higher for older patients, females, nonelective admissions, and admissions to the medical vs. the cardiac or surgical ICUs (all P ≤ 0.01).

CONCLUSION

Care preference documentation within 48 hours was absent in more than one-third of ICU admissions among patients aged 75 years and older and was more likely to occur in medical vs. cardiac or surgical ICUs.

Comparison of a modified Sequential Organ Failure Assessment Score using RASS and FOUR

OBJECTIVE

ICU severity scores such as the Sequential Organ Failure Assessment (SOFA) determine neurologic dysfunction based on the Glasgow Coma Scale, a tool that may be limited in a critically ill population. It remains unknown whether alternative methods to assess for neurologic dysfunction, such as FOUR and RASS, are superior. This study aimed to determine the predictive performance of a modified SOFA tool in a large Brazilian ICU cohort.

DESIGN

Prospective cohort single center study.

SETTING

Mixed surgical and medical ICU in Salvador, Bahia, Brazil between August 2015 and December 2018.

PATIENTS

All acutely ill ICU admissions, other than postoperative patients or those with insufficient data, were eligible for study inclusion.

MEASUREMENTS AND MAIN RESULTS

2147 patients were admitted to the ICU, of which 999 meeting inclusion criteria were included in the final analysis with a median age of 72 years (IQR 58-83) and a female predominance 545 (54%). The SOFA score using GCS, RASS and FOUR for the neurologic component performed marginally in the ability to predict general ICU mortality (SOFAGCS AUC 0.74 vs SOFARASS AUC 0.71 and SOFAFOUR AUC 0.67), with SOFAFOUR performing significantly lower compared to either SOFARASS and SOFAGCS (p<0.04, p<0.004 respectively). All three scores demonstrated decreased discriminate function in the mechanically ventilated population (SOFAGCS AUC 0.70 vs SOFARASS AUC 0.70 and SOFAFOUR AUC 0.55), though SOFAFOUR remained significantly worse when compared to SOFAGCS or SOFARASS (p = 0.034, p = 0.014, respectively).. Furthermore, performance was poor in a subset of patients with sepsis (n = 145) at time of admission (SOFAGCS AUC 0.66 vs SOFARASS AUC 0.55 and SOFAFOUR AUC 0.56).

CONCLUSION

Modification of the neurologic component in the SOFA score does not appear to improve mortality prediction in the ICU.

The prognostic value of a trend in modified SOFA score for patients with hematological malignancies in the intensive care unit

BACKGROUND

Patients with hematological malignancies admitted to an intensive care unit (ICU) have a poor prognosis. The Sequential Organ Failure Assessment (SOFA) score is used to monitor patients on the ICU. Little is known about the value of this score in hematology patients. Therefore, the prognostic value of the SOFA score and a modified hematological SOFA score (SOFAhem) was studied.

METHODS

Patients with hematological malignancies admitted to the ICU between 1999 and 2009 were analyzed in a retrospective cohort study. The SOFAhem score was defined as the original SOFA score omitting the coagulation and neurological parameters.

RESULTS

In 149 admissions, ICU mortality was 52%. Mortality was significantly associated with higher SOFA and SOFAhem scores on admission, and trend in SOFAhem scores. An unchanged and increased SOFAhem score compared to decreasing SOFAhem scores was associated with a higher mortality rate (53% resp 67% resp 25%).

CONCLUSIONS

Trends in SOFA or SOFAhem score are both suitable as prognostic parameter. The trend in SOFAhem score seems to be independently related to mortality in hematological patients admitted to the ICU, and because of the higher odds ratios and lower P-values compared to the SOFA score, it is probably stronger related to mortality than the classical score, but its prognostic value should be tested in a larger cohort.

Two Decades of ICU Utilization and Hospital Outcomes in a Comprehensive Cancer Center

OBJECTIVE

To investigate ICU utilization and hospital outcomes of oncological patients admitted to a comprehensive cancer center.

DESIGN

Observational cohort study.

SETTING

The University of Texas MD Anderson Cancer Center.

PATIENTS

Consecutive adults with cancer discharged over a 20-year period.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The Cochran-Armitage test for trend was used to evaluate ICU utilization and hospital mortality rates by primary service over time. A negative binomial log linear regression model was fitted to the data to investigate length of stay over time. Among 387,306 adult hospitalized patients, the ICU utilization rate was 12.9%. The overall hospital mortality rate was 3.6%: 16.2% among patients with an ICU stay and 1.8% among non-ICU patients. Among those admitted to the ICU, the mean (SD) admission Sequential Organ Failure Assessment score was 6.1 (3.8) for all ICU patients: 7.3 (4.4) for medical ICU patients and 4.9 (2.8) for surgical ICU patients. Hematologic disorders were associated with the highest hospital mortality rate in ICU patients (42.8%); metastatic disease had the highest mortality rate in non-ICU patients (4.2%); sepsis, pneumonia, and other infections had the highest mortality rate for all inpatients (8.5%).

CONCLUSIONS

This study provides a longitudinal view of ICU utilization rates, hospital and ICU length of stay, and severity-adjusted mortality rates. Although the data arise from a single institution, it encompasses a large number of hospital admissions over two decades and can serve as a point of comparison for future oncological studies at similar institutions. More studies of this nature are needed to determine whether consolidation of cancer care into specialized large-volume facilities may improve outcomes, while simultaneously sustaining appropriate resource utilization and reducing unnecessary healthcare costs.

Validity of a Modified Sequential Organ Failure Assessment Score Using the Richmond Agitation-Sedation Scale

OBJECTIVES

The Sequential Organ Failure Assessment and other severity of illness scales rely on the Glasgow Coma Scale to measure acute neurologic dysfunction, but the Glasgow Coma Scale is unavailable or inconsistently applied in some institutions. The objective of this study was to assess the validity of a modified Sequential Organ Failure Assessment that uses the Richmond Agitation-Sedation Scale instead of Glasgow Coma Scale.

DESIGN

Prospective cohort study.

SETTING

Medical and surgical ICUs within a large, tertiary care hospital.

PATIENTS

Critically ill medical/surgical ICU patients.

INTERVENTIONS

We calculated daily Sequential Organ Failure Assessment scores by using electronic medical record-derived data. By using bedside nurse-recorded Glasgow Coma Scale and Richmond Agitation-Sedation Scale measures, we calculated neurologic Sequential Organ Failure Assessment scores using the original Glasgow Coma Scale-based approach and a novel Richmond Agitation-Sedation Scale-based approach, converting the 10-point Richmond Agitation-Sedation Scale to a 4-point neurologic Sequential Organ Failure Assessment score. We assessed construct validity of Richmond Agitation-Sedation Scale-based Sequential Organ Failure Assessment by analyzing correlations with established severity of illness constructs (Acute Physiology and Chronic Health Evaluation II and Glasgow Coma Scale-based Sequential Organ Failure Assessment) and predictive validity by using logistic regression to determine whether Richmond Agitation-Sedation Scale-based Sequential Organ Failure Assessment predicts ICU, hospital, and 1-year mortality. We assessed discriminative performance with c-statistics.

MEASUREMENTS AND MAIN RESULTS

Among 513 patients (5,199 patient-days), Richmond Agitation-Sedation Scale-based Sequential Organ Failure Assessment was strongly correlated with Acute Physiology and Chronic Health Evaluation II acute physiology score at enrollment (r = 0.583; 95% CI, 0.518-0.642) and daily Glasgow Coma Scale-based Sequential Organ Failure Assessment scores (r = 0.963; 95% CI, 0.956-0.968). Mean Richmond Agitation-Sedation Scale-based Sequential Organ Failure Assessment scores predicted ICU mortality (areas under the curve = 0.814)-as did mean Glasgow Coma Scale-based Sequential Organ Failure Assessment (0.799)-as well as hospital and 1-year mortality. Admission Sequential Organ Failure Assessment scores, whether using Richmond Agitation-Sedation Scale or Glasgow Coma Scale, were less accurate predictors of mortality; areas under the curves for ICU mortality for Richmond Agitation-Sedation Scale-based and Glasgow Coma Scale-based Sequential Organ Failure Assessment, for example, were 0.622 and 0.608, respectively.

CONCLUSION

A modified Sequential Organ Failure Assessment score that uses bedside Richmond Agitation-Sedation Scale when Glasgow Coma Scale data are not available is a valid means of assessing daily severity of illness in the ICU and may be valuable for risk-adjustment and benchmarking purposes.

Validation of computerized automatic calculation of the sequential organ failure assessment score

Purpose. To validate the use of a computer program for the automatic calculation of the sequential organ failure assessment (SOFA) score, as compared to the gold standard of manual chart review. Materials and Methods. Adult admissions (age > 18 years) to the medical ICU with a length of stay greater than 24 hours were studied in the setting of an academic tertiary referral center. A retrospective cross-sectional analysis was performed using a derivation cohort to compare automatic calculation of the SOFA score to the gold standard of manual chart review. After critical appraisal of sources of disagreement, another analysis was performed using an independent validation cohort. Then, a prospective observational analysis was performed using an implementation of this computer program in AWARE Dashboard, which is an existing real-time patient EMR system for use in the ICU. Results. Good agreement between the manual and automatic SOFA calculations was observed for both the derivation (N=94) and validation (N=268) cohorts: 0.02 ± 2.33 and 0.29 ± 1.75 points, respectively. These results were validated in AWARE (N=60). Conclusion. This EMR-based automatic tool accurately calculates SOFA scores and can facilitate ICU decisions without the need for manual data collection. This tool can also be employed in a real-time electronic environment.

Cross-validation of a Sequential Organ Failure Assessment score-based model to predict mortality in patients with cancer admitted to the intensive care unit

PURPOSE

This study aims to validate the performance of the Sequential Organ Failure Assessment (SOFA) score to predict death of critically ill patients with cancer.

MATERIAL AND METHODS

We conducted a retrospective observational study including adults admitted to the intensive care unit (ICU) between January 1, 2006, and December 31, 2008. We randomly selected training and validation samples in medical and surgical admissions to predict ICU and in-hospital mortality. By using logistic regression, we calculated the probabilities of death in the training samples and applied them to the validation samples to test the goodness-of-fit of the models, construct receiver operator characteristics curves, and calculate the areas under the curve (AUCs).

RESULTS

In predicting mortality at discharge from the unit, the AUC from the validation group of medical admissions was 0.7851 (95% confidence interval [CI], 0.7437-0.8264), and the AUC from the surgical admissions was 0.7847 (95% CI, 0.6319-0.937). The AUCs of the SOFA score to predict mortality in the hospital after ICU admission were 0.7789 (95% CI, 0.74-0.8177) and 0.7572 (95% CI, 0.6719-0.8424) for the medical and surgical validations groups, respectively.

CONCLUSIONS

The SOFA score had good discrimination to predict ICU and hospital mortality. However, the observed underestimation of ICU deaths and unsatisfactory goodness-of-fit test of the model in surgical patients to indicate calibration of the score to predict ICU mortality is advised in this group.