Quality Assurance and Benchmarking for Radical Cystectomy: Monitoring Early Complications and Mortality Using Cumulative Summation Charts

Prospective monitoring of imaging guideline adherence by physicians in a surgical collaborative: comparison of statistical process control methods for detecting outlying performance

BACKGROUND

Systematic, automated methods for monitoring physician performance are necessary if outlying behavior is to be detected promptly and acted on. In the Michigan Urological Surgery Improvement Collaborative (MUSIC), we evaluated several statistical process control (SPC) methods to determine the sensitivity and ease of interpretation for assessing adherence to imaging guidelines for patients with newly diagnosed prostate cancer.

METHODS

Following dissemination of imaging guidelines within the Michigan Urological Surgery Improvement Collaborative (MUSIC) for men with newly diagnosed prostate cancer, MUSIC set a target of imaging < 10% of patients for which bone scan is not indicated. We compared four SPC methods using Monte Carlo simulation: p-chart, weighted binomial CUSUM, Bernoulli cumulative sum (CUSUM), and exponentially weighted moving average (EWMA). We simulated non-indicated bone scan rates ranging from 5.9% (within target) to 11.4% (above target) for a representative MUSIC practice. Sensitivity was determined using the average run length (ARL), the time taken to signal a change. We then plotted actual non-indicated bone scan rates for a representative MUSIC practice using each SPC method to qualitatively assess graphical interpretation.

RESULTS

EWMA had the lowest ARL and was able to detect changes significantly earlier than the other SPC methodologies (p < 0.001). The p-chart had the highest ARL and thus detected changes slowest (p < 0.001). EWMA and p-charts were easier to interpret graphically than CUSUM methods due to their ability to display historical imaging rates.

CONCLUSIONS

SPC methods can be used to provide informative and timely feedback regarding adherence to healthcare performance target rates in quality improvement collaboratives. We found the EWMA method most suited for detecting changes in imaging utilization.

Systematic methods for measuring outcomes: How they may be used to improve outcomes after Radical cystectomy

In the era of managed healthcare, the measuring and reporting of surgical outcomes is a universal mandate. The outcomes should be monitored and reported in a timely manner. Methods for measuring surgical outcomes should be continuous, free of bias and accommodate variations in patient factors. The traditional methods of surgical audits are periodic, resource-intensive and have a potential for bias. These audits are typically annual and therefore there is a long time lag before any effective remedial action could be taken. To reduce this delay the manufacturing industry has long used statistical control-chart monitoring systems, as they offer continuous monitoring and are better suited to monitoring outcomes systematically and promptly. The healthcare industry is now embracing such systematic methods. Radical cystectomy (RC) is one of the most complex surgical procedures. Systematic methods for measuring outcomes after RC can identify areas of improvements on an ongoing basis, which can be used to initiate timely corrective measures. We review the available methods to improve the outcomes. Cumulative summation charts have the potential to be a robust method which can prompt early warnings and thus initiate an analysis of root causes. This early-warning system might help to resolve the issue promptly with no need to wait for the report of annual audits. This system can also be helpful for monitoring learning curves for individuals, both in training or when learning a new technology.

Is postoperative epidural analgesia better than patient-controlled analgesia for radical cystectomy?

Background:

The purpose of this study was to evaluate postoperative epidural analgesia (EPA) and intravenous patient-controlled analgesia (PCA) in terms of morbidity and mortality in patients undergoing radical cystectomy for bladder cancer.

Methods:

A retrospective study on patients undergoing radical cystectomy for clinical Tis-4N0M0 urothelial carcinoma of the bladder was performed. Patients were separated into two groups: primary EPA or PCA for postoperative analgesia. The surgical complication severity was determined according to the Clavien system. Mann-Whitney U tests, χ2 with Yates’ correction, or Fisher’s exact test were used. Predictive risk factors were explored using univariable and multivariable Cox regression models.

Results:

Of the 274 patients studied, 209 (76%) received EPA and 65 (24%) had PCA. Baseline balance was observed. Similar complication rates were observed between the EPA (36%) and PCA (34%) ( p=0.382). Patients greater than 70 years of age had more complications (35% vs 21%, p=0.002). PCA patients had higher rates of high-grade complications compared with EPA patients [40% vs 20% ( p=0.0007)]. Only age at time of surgery ( p=0.032) was associated with complications. Patients with pulmonary disease had a higher risk of complications ( p=0.001). EPA or PCA were not predictors for overall survival.

Conclusions:

There does not appear to be a significant difference in terms of morbidity or mortality between EPA and PCA following radical cystectomy (RC). Rare, catastrophic complications specific to EPA may occur. Standardized reporting of surgical complications is essential to compare studies and appropriately counsel patients.

Cumulative summation (CUSUM) charts in the monitoring of hypospadias outcomes: a tool for quality improvement initiative

OBJECTIVE

Cumulative summation (CUSUM) charting is a statistical tool that allows an individual surgeon or surgical department to monitor any binary outcome and rapidly detect when complications are outside the acceptable limits. We applied CUSUM statistical analysis to hypospadias repair to utilize the results in our own quality Improvement process.

MATERIALS AND METHODS

An institutional review board-approved retrospective review of all patients who underwent hypospadias repair by a single fellowship trained pediatric urologist at a single institution between September 2004 to July 2009 was performed. To graphically represent the complication rates and to assess for unacceptable rates, the use of CUSUM control charting was employed.

RESULTS

In our retrospective review, there were a total of 184 patients who underwent a total of 203 surgeries. Using CUSUM analysis, our incidence of major complications was within acceptable limits until approximately the first 150 operations had been performed, at which time the complication rate fell below the lower limit, indicating performance exceeded expectations.

CONCLUSION

CUSUM statistical charting was successfully applied to the retrospective monitoring of hypospadias outcomes at our institution. This is the first known publication in which CUSUM charts were used to evaluate complications of hypospadias repair.

Quality and performance indicators in an academic department of head and neck surgery

OBJECTIVE

to create a method for assessing physician performance and care outcomes that are adjusted for procedure acuity and patient comorbidity.

DESIGN

between 2004 and 2008 surgical procedures performed by 10 surgeons were stratified into high-acuity procedures (HAPs) and low-acuity procedures (LAPs). Risk adjustment was made for comorbid conditions examined singly or in groups of 2 or more.

SETTING

a tertiary care medical center.

PATIENTS

a total of 2618 surgical patients.

MAIN OUTCOME MEASURES

performance measures included length of stay; return to operating room within 7 days of surgery; and the occurrence of mortality, hospital readmission, transfusion, and wound infection within 30 days of surgery.

RESULTS

the transfusion rate was 2.7% and 40.6% for LAPs and HAPs, respectively. Wound infection rates were 1.4% for LAPs vs 14.1% for HAPs, while 30-day mortality rate was 0.3% and 1.6% for LAPs and HAPs, respectively. The mean (SD) hospital stay for LAPs was 2.1 (3.6) vs 10.5 (7.0) days for HAPs. Negative performance factors were significantly higher for patients who underwent HAPs and had comorbid conditions. Differences among surgeons significantly affect the incidence of negative performance indicators. Factors affecting performance measures were procedure acuity, the surgeon, and comorbidity, in order of decreasing significance. Surgeons were ranked low, middle, and high based on negative performance indicators.

CONCLUSIONS

performance measures following oncologic procedures were significantly affected by comorbid conditions and by procedure acuity. Although the latter most strongly affects quality and performance indicators, both should weigh heavily in physician comparisons. The incidence of negative performance indicators was also influenced by the individual surgeon. These data may serve as a tool to evaluate and improve physician performance and outcomes and to develop risk-adjusted benchmarks. Ultimately, reimbursement may be tied to quantifiable measures of physician and institutional performance.

Cumulative summation graphs are a useful tool for monitoring positive surgical margin rates in robot-assisted radical prostatectomy

OBJECTIVE

• To explore the usefulness of cumulative summation (CUSUM) graphs for monitoring positive surgical margin (PSM) rates during a surgeon's transition from open to robot-assisted radical prostatectomy (RARP).

PATIENTS AND METHODS

• Data were prospectively collected from patients undergoing RARP by a single surgeon. • Preoperatively all patients were either low or moderate risk under the D'Amico classification system. • A CUSUM graph was charted retrospectively to analyse the PSM rate in patients undergoing RARP for pathological stage T2 (pT2) disease. • Acceptable and unacceptable PSM rates were set at 10% and 15% respectively.

RESULTS

• From a cohort of 226 patients, 158 patients with pT2 disease were selected. The mean (range) age of these patients was 59.2 (39-73) years, the median (range) Gleason score was 6 (4-9), the mean (range) PSA was 6.43 (0.52-17.5) ng/mL and the mean (range) prostate volume was 44 (18-120) cm(3). In all, 21 patients had PSMs (13%). • CUSUM graphs were produced and clearly demonstrated the change in PSM rate over time.

CONCLUSION

• CUSUM graphs are a novel and useful visual representation of the learning curve for surgeons. • PSM rates in patients with pT2 disease are a good outcome to monitor using CUSUM graphs as they are binary and lack the confounding factors associated with other outcomes such as continence and erectile dysfunction. • We advocate the use of CUSUM graphs as a method of quality assurance with the introduction of a robotics programme.