Risk-Adjusted Cumulative Sum for Early Detection of Hospitals With Excess Perioperative Mortality

Risk Adjusted Continuous Monitoring of Postoperative Mortality After Cardiac Surgery

OBJECTIVE

To compare continuous monitoring with a risk-adjusted cumulative sum (CUSUM) to standard episodic risk-adjusted evaluation for the detection of hospitals with higher-than-expected postoperative mortality after cardiac surgery.

STUDY SETTING AND DESIGN

In this national, observational, hospital-level study, the number of hospitals identified with higher-than-expected quarterly, risk-adjusted 30-day mortality and time to identification were compared using standard episodic evaluation (i.e., observed-to-expected [O-E] ratios) and the risk-adjusted CUSUM.

DATA SOURCES AND ANALYTIC SAMPLE

VA Surgical Quality Improvement Program (VASQIP) data (2016-2020) for patients 18 years and older who underwent a cardiac operation at a Veterans Affairs (VA) hospital.

PRINCIPAL FINDINGS

The cohort included 20,927 patients treated at 41 hospitals across 20 quarters of data. Overall, 1.8% of hospital quarters were identified using O-E compared to 3.8% with CUSUM. Hospitals concurrently identified using both CUSUM and O-E were identified a median of 17 days earlier with CUSUM (interquartile range [IQR] 7-51 days before quarter end). This translated to a median of 12 (IQR 8-37) surgical cases and 71 (IQR 34-331) postoperative inpatient days occurring after a CUSUM signal but before the quarter ended. At hospitals identified by CUSUM but not O-E, a median of 2 deaths (IQR 2-2) during a median of 22 days (IQR 12-38) triggered detection.

CONCLUSIONS

CUSUM identifies hospitals with higher-than-expected mortality rates earlier than episodic analysis. Considering the time lag between data collection and report generation by national quality improvement (QI) programs, CUSUM represents a potentially useful tool that could facilitate more real-time recognition of performance concerns and encourage earlier implementation of interventions that can help avoid potentially preventable patient harm. Balancing sensitivity with the risk of false signaling will be essential for ensuring its effective application in national QI efforts.

Machine learning adjusted sequential CUSUM-analyses are superior to cross-sectional analysis of excess mortality after surgery

BACKGROUND

The assessment of clinical outcome quality, particularly in surgery, is crucial for healthcare improvement. Traditional cross-sectional analyses often fall short in timely and systematic identification of clinical quality issues. This study explores the efficacy of machine learning adjusted sequential CUSUM (Cumulative Sum) analyses in monitoring post-surgical mortality.

MATERIAL AND METHODS

Utilizing the Global Open Source Severity of Illness Score (GOSSIS) dataset involving 91,714 patient records from 147 hospitals, this study involved the development of a machine learning model for mortality using a modified LightGBM algorithm. With this, sequential and cross sectional quality monitoring was simulated and compared.

RESULTS

The modified LightGBM model demonstrated superior predictive accuracy (ROC AUC of 0.88). Simulations revealed that the AI risk-adjusted CUSUM required fewer patient outcome alterations to detect atypical trends compared to standard methods.

CONCLUSION

The AI risk-adjusted CUSUM analysis represents a significant advancement in monitoring clinical outcome quality in healthcare, especially in surgery. Its ability to detect minor discrepancies in mortality rates with greater sensitivity and specificity positions it as a valuable tool for healthcare providers. This approach could lead to earlier interventions and improved patient care.

Within-hospital Temporal Clustering of Postoperative Complications and Implications for Safety Monitoring and Benchmarking Using ACS-NSQIP Data

Objective

To determine the extent to which within-hospital temporal clustering of postoperative complications is observed in the American College of Surgeons, National Surgical Quality Improvement Program (ACS-NSQIP).

Background

ACS-NSQIP relies on periodic and on-demand reports for quality benchmarking. However, if rapid increases in postoperative complication rates (clusters) are common, other reporting methods might be valuable additions to the program. This article focuses on estimating the incidence of within-hospital temporal clusters.

Methods

ACS-NSQIP data from 1,547,440 patients, in 425 hospitals, over a 2-year period was examined. Hospital-specific Cox proportional hazards regression was used to estimate the incidence of mortality, morbidity, and surgical site infection (SSI) over a 30-day postoperative period, with risk adjustment for patient and procedure and with additional adjustments for linear trend, day-of-week, and season. Clusters were identified using scan statistics, and cluster counts were compared, using unpaired and paired t tests, for different levels of adjustment and when randomization of cases across time eliminated all temporal influences.

Results

Temporal clusters were rarely observed. When clustering was adjusted only for patient and procedure risk, an annual average of 0.31, 0.85, and 0.51 clusters were observed per hospital for mortality, morbidity, and SSI, respectively. The number of clusters dropped after adjustment for linear trend, day-of-week, and season (0.31-0.24; P = 0.012; 0.85-0.80; P = 0.034; and 0.51-0.36; P < 0.001; using paired t tests) for mortality, morbidity, and SSI, respectively. There was 1 significant difference in the number of clusters when comparing data with all adjustments and after data were randomized (0.24 and 0.25 for mortality; P = 0.853; 0.80 and 0.82 for morbidity; P = 0.529; and 0.36 and 0.46 [randomized data had more clusters] for SSI; P = 0.001; using paired t tests) for mortality, morbidity, and SSI, respectively.

Conclusions

Temporal clusters of postoperative complications were rarely observed in ACS-NSQIP data. The described methodology may be useful in assessing clustering in other surgical arenas.

A 10-year review on older patients undergoing colorectal cancer surgery: surgeon-led geriatric service and good quality surgery are drivers of good outcomes

BACKGROUND

Older age and frailty are associated with worse postoperative outcomes and prolonged length of stay (LOS). In this study, we aimed to analyze the long-term outcomes after the implementation of our geriatric surgical service (GSS).

METHODS

This was a single-center retrospective study from July 2010 to December 2021 on patients aged ≥75 years or patients aged ≥65 years with frailty. Our GSS includes multidisciplinary assessment and optimization by specialized nurses, physiotherapists, anesthetists, dietitians, and geriatricians. Cumulative sum (CUSUM) analysis was used to assess the performance of our GSS. Our primary outcome was defined as the presence of 30-day mortality, prolonged LOS ≥ 14 days, and/or >10% decrease in the modified Barthel Index at 6 weeks, which depicts the failure of GSS. A downsloping CUSUM curve implies consecutive cases of success.

RESULTS

There were 233 patients with a mean age of 79.0 ± 4.9 years; of these, 73 patients (31.3%) were frail. The overall 30-day mortality (1.7%), Clavien-Dindo ≥ grade IIIA complications (12.0%), and LOS (median, 7.0 days) were low. The CUSUM analysis showed 3 phases with overall sustained improvement in outcomes. Transient inconsistency in the second phase (during midimplementation of GSS) may be due to the early adoption of laparoscopic surgery (44.6% vs 24.1%; adjusted P =.031) and expansion of service to include patients with higher perioperative risks (weighted Charlson Comorbidity Index score ≥4: 64.9% vs 38.0%; adjusted P =.002) in the second period compared with the first period. The outcomes subsequently improved in the third phase after overcoming the learning curve.

CONCLUSION

Our GSS showed sustained performance over the past decade. Good quality surgery and surgeon-led geriatric service are paramount for good postoperative outcomes.