Assessing the learning curve in off-pump coronary artery surgery via CUSUM failure analysis

Defining the cardiac surgical learning curve: a longitudinal cumulative analysis of a surgeon's experience and performance monitoring in the first decade of practice

BACKGROUND

Individual surgeons' learning curves are a crucial factor impacting patient outcomes. While many studies investigate procedure-specific learning curves, very few carried out a longitudinal analysis of individual cardiac surgeons over the course of their career. Given the evolving landscape of cardiac surgery with the introduction of transcatheter and robotic procedures, a contemporary evaluation of the cardiac surgical learning curve is justified and a method of personal performance monitoring is proposed in this study.

METHODS

A retrospective study of 1578 consecutive patients of a cardiac surgeon over ten years was undertaken. Risk adjustment was based on Euroscore. Cumulative risk adjusted morbidity (CRAM) charts of operative mortality, return to theatre and length of stay were constructed. Secondary endpoints included postoperative stroke and deep sternal wound infection. Change-point detection was applied to investigate temporal trends and identify when a significant change in outcome occurred. Multivariate analysis was performed to assess the influence of patient and system factors on operative mortality.

RESULTS

Patient average risk profile was highest in the later years of practice. Cardiopulmonary bypass time remained stable from 86.5 to 92 min across the decade. The frequency of redo operations increased from 4.07% in the first two years of practice to 9.29% in the last two years. The proportion of aortic surgery increased from 6.98 to 10.58% of total cases. There was a significantly reduced operative mortality signalled at case 1220 with the change point identified around case 970.

CONCLUSION

This prompts training colleges to consider application of sequential performance monitoring in surgical training programs, to confirm the progress of trainees and identify early evolving patterns that suggest support is required or milestones are being achieved.

Expertise-based design in surgical trials: a narrative review

Randomized controlled trials (RCTs) are the most robust study design for evaluating the safety and efficacy of a therapeutic intervention. However, their internal validity are at risk when evaluating surgical interventions. This review summarizes existing expertise- based trials in surgery and related methodological concepts to guide surgeons performing this work. We provide caseloads required to reach the learning curve for various surgical interventions and report criteria for expertise from published and unpublished expertise-based trials. In addition, we review design and implementation concepts of expertise-based trials, including recruitment of surgeons, crossover, ethics, generalizability, sample size and definitions for learning curve. Several RCTs have used an expertise-based design. We found that the majority of definitions used for expertise were vague, heterogeneous, and inconsistent across trials evaluating the same surgical intervention. Statistical methods exist to adjust for the learning curve; however, there is limited guidance. We developed the following criteria for surgical expertise for future trials: 1) decide on the proxy to be used for the learning curve, and 2) assess eligible surgeons by comparing their performance to the previously defined expertise criteria.

Statistical Process Control Charts for Monitoring Next-Generation Sequencing and Bioinformatics Turnaround in Precision Medicine Initiatives

Purpose

Precision oncology, such as next generation sequencing (NGS) molecular analysis and bioinformatics are used to guide targeted therapies. The laboratory turnaround time (TAT) is a key performance indicator of laboratory performance. This study aims to formally apply statistical process control (SPC) methods such as CUSUM and EWMA to a precision medicine programme to analyze the learning curves of NGS and bioinformatics processes.

Patients and Methods

Trends in NGS and bioinformatics TAT were analyzed using simple regression models with TAT as the dependent variable and chronologically-ordered case number as the independent variable. The M-estimator "robust" regression and negative binomial regression were chosen to serve as sensitivity analyses to each other. Next, two popular statistical process control (SPC) approaches which are CUSUM and EWMA were utilized and the CUSUM log-likelihood ratio (LLR) charts were also generated. All statistical analyses were done in Stata version 16.0 (StataCorp), and nominal P < 0.05 was considered to be statistically significant.

Results

A total of 365 patients underwent successful molecular profiling. Both the robust linear model and negative binomial model showed statistically significant reductions in TAT with accumulating experience. The EWMA and CUSUM charts of overall TAT largely corresponded except that the EWMA chart consistently decreased while the CUSUM analyses indicated improvement only after a nadir at the 82^nd case. CUSUM analysis found that the bioinformatics team took a lower number of cases (54 cases) to overcome the learning curve compared to the NGS team (85 cases).

Conclusion

As NGS and bioinformatics lead precision oncology into the forefront of cancer management, characterizing the TAT of NGS and bioinformatics processes improves the timeliness of data output by potentially spotlighting problems early for rectification, thereby improving care delivery.

Safe implementation of robotic-assisted minimally invasive direct coronary artery bypass: application of learning curves and cumulative sum analysis

Background

Learning curves are inevitably encountered when first implementing an innovative and complex surgical technique. Nevertheless, a cluster of failures or complications should be detected early, but not deter learning, to ensure safe implementation. Here, we aimed to examine the presence and impact of learning curves on outcome after robotic-assisted minimally invasive direct coronary artery bypass (RA-MIDCAB).

Methods

A retrospective analysis of the first 300 RA-MIDCAB surgeries between July 2015 and December 2020 was performed. Learning curves were obtained via logarithmic regression for surgical time. Cumulative sum (CUSUM) analysis was performed for (I) major complications including MI, stroke, repeat revascularization, and mortality, and (II) other complications, including prolonged ventilation, pneumonia, pleura puncture, lung herniation, pericarditis, pleuritis, arrhythmia, wound complications, and delirium. Expected and unacceptable rates were set at 12% and 20%, respectively, for major complications, and at 40% and 60% for other complications, based on historical data in conventional coronary artery bypass grafting (CABG).

Results

Demographic characteristics did not differ between terciles, except for more smokers in the first tercile, and less hypercholesterolemia and more complex procedures in the third tercile. The mean surgical time for all operations was 258±81 minutes, ranging from 127 to 821 minutes. A learning curve was only observed in the first tercile. Subgroup analysis revealed that this learning curve was only observed for procedures consisting of single internal mammary artery (SIMA) with 1 or 2 distal anastomoses but not with bilateral internal mammary arteries (BIMA) or more than 2 distal anastomoses. CUSUM analysis showed that the cumulative rate of major and other complications never crossed the lines for unacceptable rates. Rather, the lower 95% confidence boundary was crossed after 50 cases, indicating improvement in safety.

Conclusions

These results suggest that integration of RA-MIDCAB in the surgical landscape can be safely achieved and complication rates can quickly be reduced below those expected in traditional CABG. Collective experience plays a key role in overcoming the learning curve when more complex procedures and cases are introduced.

Learning curve of mitral valve repair: cumulative sum failure analysis from single high-volume center

Background

Mitral valve (MV) repair has become the gold standard for treating degenerative mitral regurgitation (MR), yet the success rate of MV repair is still low in clinical practice. While studies focused on the learning process of MV repair are scarce, fully understanding the learning curve could provide valuable information for education and the quality control of MV repair, thus benefiting patients. This observational study aimed to evaluate the learning process and performances of individual surgeon for MV repair for degenerative mitral disease using data from a single high-volume center.

Methods

Profiles of patients who underwent MV repair for degenerative MR at our institution from January 2003 to December 2016 were analyzed retrospectively. Overall and individual learning curves for the repair rate and major adverse events were calculated using sequential probability cumulative sum failure analysis. Average learning curves for major adverse events and operative time were also analyzed, by calculating the average incidence of adverse events and operative time of all operations stratified by accumulated operation numbers of individual surgeon. Altogether, we evaluated 2,482 operations performed by 14 surgeons.

Results

There was an obvious learning curve for the repair rate at the institution and individual surgeon levels. Altogether, 50 to 200 operations were needed to overcome the repair rate learning curve, yet wide variation was observed among individual surgeons. The learning process for individual surgeons became faster after the turning point in the institutional learning curve appeared. No obvious learning curve was observed at the institution or individual level for major adverse events and in-hospital mortality.

Conclusions

The number of cases required to overcome the learning curve for repair rate is substantial, although there is marked variation among surgeons. Individuals’ learning curves accelerate as the institution accumulates experience. MV repair is safe in experienced high-volume center. Close monitoring is necessary when surgeons begin to practice new techniques.

Quantifying the learning curve of emergent total arch replacement in acute type A aortic dissection

Background

Acute type A aortic dissection with arch involvement is a life-threatening condition, which requires immediate surgical attention. Emergent total arch replacement and root reconstruction is a technically demanding operation with varying outcomes based on surgeon experience. The human factors in total arch replacement in the emergent setting have never been systematically investigated. The ability of surgeons with low volumes to achieve acceptable results in their start-up period is not known.

Methods

From January 2013 to December 2016, patients with acute type A aortic dissection who underwent emergent total arch replacement with three surgeons were enrolled. Basic characteristics, procedural and postoperative outcomes were collected. The time of critical surgical steps and operative mortality were calculated using descriptive statistics and cumulative SUM (CUSUM) analysis.

Results

A total of 300 patients (age 53.8±11.5 years, female 63, 21.0%) with acute type A aortic dissection underwent emergent total arch replacement. A total of 219 patients (73.0%) had root reinforcement, 295 patients (98.3%) underwent frozen elephant trunk repair. Mean circulatory arrest and cross-clamp times were 29.8±9.8 and 112.3±32.1 min, respectively. The operative mortality was 6.7%, the stroke rate was 4.0%. The mean length of postoperative ICU and hospital stays were 8.4±10.6 and 18.0±12.2 days, respectively. By CUSUM depictions, surgeons appeared to have different learning curves with regards to operative time. By CUSUM failure analysis on operative mortality, two newly appointed surgeons in their start-up period stayed in an acceptable range, while one senior surgeon with higher volumes experienced superior outcomes and better performance.

Conclusions

Although emergent total arch replacement for acute type A dissection is a complex scenario, surgeons well-trained in adult cardiac surgery are able to achieve acceptable results in their start-up period.

Six-year single-centre experience in minimally invasive mitral valve repair - impact of the team learning curve on in-hospital clinical outcome

Introduction

Minimally invasive mitral repair is less traumatic and more acceptable for the patient than traditional surgery. However, it is a challenging procedure that requires effort from all the personnel involved.

Aim

To investigate the results of the minimally invasive mitral valve repair learning curve at the institution.

Material and methods

The indication for the surgery was severe mitral regurgitation. Patients with other valvular insufficiency, body mass index (BMI) > 30 kg/m², ejection fraction (EF) < 45%, aortic dilatation, reoperation, pleural adhesions, coronary artery disease requiring invasive treatment, and pregnant women were disqualified. The patients were assigned to one of three groups regarding their surgery date – group 1 (2012–2013), group 2 (2014–2015) and group 3 (2016–2017). The primary endpoints were death, myocardial infarction, stroke, an reoperation for mitral dysfunction. The investigation was performed to determine preoperative parameters (EuroSCORE, age, sex, BMI, arrhythmias, EF), intraoperative parameters (procedure, cross-clamp, extracorporeal circulation), and postoperative parameters (chest revision, transfusion, drainage, ventilation time, pleurocentesis, hospitalization time).

Results

There were 173 patients in total. One patient from group 1 (0.6% overall) died. No myocardial infarction or stroke was observed in any of the three groups. Chest revision count (5 vs. 1 vs. 1; p = 0.0004), total drainage (797.20 vs. 517.92 vs. 449.69; p = 0.0018) and hospitalization time (7.89 vs. 7.18 vs. 6.73; p = 0.0005) were significantly different among the groups. The ventilation time, transfusion number and pleurocentesis count did not differ significantly.

Conclusions

The procedure is safe and ensures optimal perioperative results. The number of complications is low and acceptable.

Learning curves and surgical outcomes for proctored adoption of laparoscopic ventral mesh rectopexy: cumulative sum curve analysis

BACKGROUND

Laparoscopic ventral mesh rectopexy (VMR) is an effective and well-recognised treatment for symptoms of obstructive defecation in the context of rectal prolapse and recto-rectal intussusception. However, due to the technical complexity of VMR, a significant learning curve has been previously described. This paper examines the effect of proctored adoption of VMR on learning curves, operative times, and outcomes.

METHODS

A retrospective database analysis of two district general hospitals was conducted, with inclusion of all cases performed by two surgeons since first adoption of the procedure in 2007-2015. Operative time, length of stay, and in-hospital complications were evaluated, with learning curves assessed using cumulative sum curves.

RESULTS

Three hundred and eleven patients underwent VMR during the study period and were included for analysis. Patients were near-equally distributed between surgeons (surgeon A: n = 151, surgeon B, n = 160) with no significant differences between gender, age, or ASA grade. In-hospital morbidity was 3.2 %, with 0 % mortality. Cumulative sum curve analysis suggested a change point of between 25 and 30 cases based on operative times and length of stay and was similar between both surgeons. No significant change point was seen for morbidity or mortality.

CONCLUSION

VMR is an effective and safe treatment for rectal prolapse. Surgeons in this study were proctored during the adoption process by another surgeon experienced in VMR; this may contribute to increased safety and abbreviated learning curve. In the context of proctored adoption, this study estimates a learning curve of 25-30 cases, without detrimental impact on patient outcomes.

Evaluation of the learning curve for transcatheter aortic valve implantation via the transfemoral approach

BACKGROUND

The aim of this study was to evaluate the learning curve in performing transfemoral TAVI (TF-TAVI).

METHODS

Between October 2006 and October 2013, 312 consecutive TF-TAVI cases performed by 6 interventional cardiologists, using the Edwards Sapien valve and 104 using the CoreValve, were included in the present analysis. Cumulative sum (CUSUM) failure analysis of combined 30-day safety endpoint was used to evaluate learning curves.

RESULTS

The CUSUM analysis revealed a learning curve regarding the occurrence of 30-day adverse events with an improvement after the initial 86 cases using the Edwards valve and 40 cases using the CoreValve. We divided the Edwards valve cases into two groups (early experience: Cases 1 to 86; late experience: Cases 87 to 312). The rate of 30-day mortality and 1-year mortality significantly decreased in the late experience group (17% to 7%, p=0.019; 34% to 21%, p=0.035, respectively). We divided the CoreValve cases into two groups (early experience: Cases 1 to 40; late experience: Cases 41 to 104). The rate of 30-day mortality and 1-year mortality significantly decreased in the late experience group (20% to 6%, p=0.033; 38% to 15%, p=0.040, respectively). The groups including both valves were also analyzed after propensity-matching (early [n=52] vs late [n=52]). This model also showed that 30-day and 1-year mortality rates were significantly lower in the late experience group (13% to 1%, p=0.028; 34% to 20%, p=0.042, respectively).

CONCLUSIONS

An appropriate level of experience is needed to reduce the complication rate and mortality in TF-TAVI.

Calibrated breast density methods for full field digital mammography: a system for serial quality control and inter-system generalization

PURPOSE

The authors are developing a system for calibrated breast density measurements using full field digital mammography (FFDM). Breast tissue equivalent (BTE) phantom images are used to establish baseline (BL) calibration curves at time zero. For a given FFDM unit, the full BL dataset is comprised of approximately 160 phantom images, acquired prior to calibrating prospective patient mammograms. BL curves are monitored serially to ensure they produce accurate calibration and require updating when calibration accuracy degrades beyond an acceptable tolerance, rather than acquiring full BL datasets repeatedly. BL updating is a special case of generalizing calibration datasets across FFDM units, referred to as cross-calibration. Serial monitoring, BL updating, and cross-calibration techniques were developed and evaluated.

METHODS

BL curves were established for three Hologic Selenia FFDM units at time zero. In addition, one set of serial phantom images, comprised of equal proportions of adipose and fibroglandular BTE materials (50/50 compositions) of a fixed height, was acquired biweekly and monitored with the cumulative sum (Cusum) technique. These 50/50 composition images were used to update the BL curves when the calibration accuracy degraded beyond a preset tolerance of ±4 standardized units. A second set of serial images, comprised of a wide-range of BTE compositions, was acquired biweekly to evaluate serial monitoring, BL updating, and cross-calibration techniques.

RESULTS

Calibration accuracy can degrade serially and is a function of acquisition technique and phantom height. The authors demonstrated that all heights could be monitored simultaneously while acquiring images of a 50/50 phantom with a fixed height for each acquisition technique biweekly, translating into approximately 16 image acquisitions biweekly per FFDM unit. The same serial images are sufficient for serial monitoring, BL updating, and cross-calibration. Serial calibration accuracy was maintained within ±4 standardized unit variation from the ideal when applying BL updating. BL updating is a special case of cross-calibration; the BL dataset of unit 1 can be converted to the BL dataset for another similar unit (i.e., unit 2) at any given time point using the 16 serial monitoring 50/50 phantom images of unit 2 (or vice versa) acquired near this time point while maintaining the ±4 standardized unit tolerance.

CONCLUSIONS

A methodology for monitoring and maintaining serial calibration accuracy for breast density measurements was evaluated. Calibration datasets for a given unit can be translated forward in time with minimal phantom imaging effort. Similarly, cross-calibration is a method for generalizing calibration datasets across similar units without additional phantom imaging. This methodology will require further evaluation with mammograms for complete validation.

Learning curves for cardiothoracic and vascular surgical procedures--a systematic review

OBJECTIVES

The aim of this systematic review is to evaluate the learning curve (LC) literature and identify the LC of cardiothoracic and vascular surgical procedures.

SUMMARY AND BACKGROUND

The LC describes an observation that a learner's performance improves over time during acquisition of new motor skills. Measuring the LC of surgical procedures has important implications for surgical innovation, education, and patient safety. Numerous studies have investigated LCs of isolated operations in cardiothoracic and vascular surgeries, but a lack of uniformity in the methods and variables used to measure LCs has led to a lack of systematic reviews.

METHODS

The MEDLINE®, EMBASE™, and PsycINFO® databases were systematically searched until July 2013. Articles describing LCs for cardiothoracic and vascular procedures were included. The type of procedure, statistical analysis, number of participants, procedure setting, level of participants, outcomes, and LCs were reviewed.

RESULTS

A total of 48 studies investigated LCs in cardiothoracic and vascular surgeries. Based on operating time, the LC for coronary artery bypass surgery ranged between 15 and 100 cases; for endoscopic vessel harvesting and other cardiac vessel surgery between 7 and 35 cases; for valvular surgery, which included repair and replacement, between 20 and 135 cases; for video-assisted thoracoscopic surgery, between 15 and 35 cases; for vascular neurosurgical procedures between 100 and 500 cases, based on complications; for endovascular vessel repairs between 5 and 40 cases; and for ablation procedures between 25 and 60 cases. However there was a distinct lack of standardization in the variables/outcome measures used, case selection, prior experience, and supervision of participating surgeons and a range of statistical analyses to compute LCs was noted.

CONCLUSION

LCs in cardiothoracic and vascular procedures are hugely variable depending on the procedure type, outcome measures, level of prior experience, and methods/statistics used. Uniformity in methods, variables, and statistical analysis is needed to derive meaningful comparisons of LCs. Acknowledgment and application of learning processes other than those reliant on volume-outcomes relationship will benefit LC research and training of surgeons.

A Learning Curve in Bentall and De Bono Procedure with the Use of the Risk-Adjusted Cumulative Sum Analysis Based on the Japan SCORE

<p><b>Objective:</b> The risk-adjusted cumulative sum analysis of an individual surgical learning curve for the Bentall and De Bono procedure on aortic root pathologies was used to monitor and enhance quality control.</p><p><b>Methods:</b> From April 2004 to March 2009, 20 consecutive patients were operated upon by a single surgeon for aortic root pathologies by means of the Bentall and De Bono procedure. Operative variables, mortality, and major morbidities were analyzed. The learning curve was calculated using descriptive statistics and cumulative sum failure observed minus expected failure rate derived from the Japan SCORE calculator.</p><p><b>Results:</b> In this series, the Japan SCORE calculator expected 3.91% � 5.14% of 30-day operative mortality rate and 16.74 � 13.47% of 30-day operative mortality and morbidity. In reality, there was no operative and in-hospital death. Major postoperative morbidity rate was 15.0% (3 patients). The risk-adjusted cumulative sum analysis revealed that the learning curve of mortality was downward below the lower 95% confidence interval throughout all the patients and that of morbidity was upward till the seventh case and remained downward thereafter. The polynomial approximation coefficient analysis revealed a significant negative correlation between aortic cross-clamp time, cardiopulmonary time, and operation time and case load.</p><p><b>Conclusions:</b> The Bentall and De Bono procedure for aortic root pathologies was performed at the professionally permissive level from the beginning, even in a low-volume environment. It showed one good example of a learning curve. The risk-adjusted cumulative sum analysis based on the Japan SCORE was a very useful tool for monitoring the performance.</p>

Neonatal endosurgical congenital diaphragmatic hernia repair: a systematic review and meta-analysis

OBJECTIVE

To compare outcomes of open and endosurgical neonatal congenital diaphragmatic hernia (CDH) repairs.

BACKGROUND

Historically a surgical emergency, neonatal CDH repair is now deferred pending stabilization of characteristically labile cardiopulmonary physiology. Usually accomplished via laparotomy, surgical repair may acutely worsen lung function; conversely, by reducing the visceral hernia, surgery might improve it. Theoretically, endosurgical repair could minimize deleterious effects of surgery while garnering benefits from decompressing the CDH lung. As endosurgical repair gains popularity, it is important to investigate whether or not minimally-invasive neonatal CDH repair has benefits.

METHODS

We searched Medline, Embase, and Cochrane Trials databases for studies comparing open with endosurgical CDH repair. Non-neonatal series and reports without comparison groups were excluded. References from papers and conference proceedings were also hand searched. Meta-analysis used a fixed effects model and was reported in accordance with PRISMA.

RESULTS

We included 3 studies (1 unpublished; none randomized); all compared thoracoscopic and open CDH repair and together described 143 patients. All studies had limitations, including use of historical controls. Demographics, CDH sidedness, APGAR and associated anomaly prevalence were similar between groups. For endosurgical repair, recurrence was higher (RR: 3.2 [1.1, 9.3], P = 0.03) and operative time longer (WMD 50 minutes [32, 69], P < 0.00001). Survival and patch usage were not different between open and endosurgical groups.

CONCLUSIONS

Neonatal thoracoscopic CDH repair has greater recurrence rates and operative times but similar survival and patch usage compared with open surgery. A prospective registry for all such cases would guide development of trials (Stage 2b; IDEAL recommendations).

No surgical innovation without evaluation: the IDEAL recommendations

Surgery and other invasive therapies are complex interventions, the assessment of which is challenged by factors that depend on operator, team, and setting, such as learning curves, quality variations, and perception of equipoise. We propose recommendations for the assessment of surgery based on a five-stage description of the surgical development process. We also encourage the widespread use of prospective databases and registries. Reports of new techniques should be registered as a professional duty, anonymously if necessary when outcomes are adverse. Case series studies should be replaced by prospective development studies for early technical modifications and by prospective research databases for later pre-trial evaluation. Protocols for these studies should be registered publicly. Statistical process control techniques can be useful in both early and late assessment. Randomised trials should be used whenever possible to investigate efficacy, but adequate pre-trial data are essential to allow power calculations, clarify the definition and indications of the intervention, and develop quality measures. Difficulties in doing randomised clinical trials should be addressed by measures to evaluate learning curves and alleviate equipoise problems. Alternative prospective designs, such as interrupted time series studies, should be used when randomised trials are not feasible. Established procedures should be monitored with prospective databases to analyse outcome variations and to identify late and rare events. Achievement of improved design, conduct, and reporting of surgical research will need concerted action by editors, funders of health care and research, regulatory bodies, and professional societies.

Cumulative sum failure analysis for eight surgeons performing minimally invasive direct coronary artery bypass

OBJECTIVE

Analysis of average and individual surgical performance for minimally invasive direct coronary artery bypass was used to enhance quality control for that operation.

METHODS

A total of 1441 standard minimally invasive direct coronary artery bypass procedures performed from August 1996 to January 2006 were analyzed for mortality and 10 other major perioperative complications. Learning curves and assessment of perioperative outcome were calculated using descriptive statistics and cumulative sum observed minus expected failure analysis for 8 involved surgeons with a personal experience ranging from 27 to 443 procedures.

RESULTS

The incidence of in-hospital mortality was 0.9% and compared favorably with the predicted mortality calculated by the logistic EuroSCORE (3.6%, P < .01). Cumulative sum analysis revealed that 2 surgeons crossed the 95% reassurance boundary after 50 operations and that 2 surgeons crossed the 95% reassurance boundary after 100 operations. There were significant differences between surgeons with regard to the learning curves and perioperative complications (3.6%-29.6%, P < .01). Two surgeons crossed the 95% alarm-line indicating unacceptably high failure rates.

CONCLUSIONS

Minimally invasive direct coronary artery bypass has become a procedure with low mortality and low complication rates, but results are case-load and surgeon dependent. Cumulative sum analysis is a valuable method allowing for a breakdown of complication rates over time displaying individual surgeons' strengths.

CUMULATIVE SUM TECHNIQUES FOR SURGEONS: A BRIEF REVIEW

There has been increasing awareness of the need for monitoring the quality of health care, particularly in the area of surgery. The Cumulative Summation (Cusum) techniques have emerged as a popular tool for performance monitoring in surgery. They allow one to judge whether a given variation in performance is probably due to chance or greater than could be expected from random variation and thus a cause for concern. The Cusum techniques are simple to carry out and can be applied to any surgical process with a binary outcome. Four parameters need to be set in advance: acceptable outcome rate, unacceptable outcome rate, Type I and Type II error rates. In this article, we review the history, statistical methods and potential applications for the Cusum techniques in the field of surgery and illustrate the two common forms of charting (cumulative failure and Cusum charting) by using unadjusted outcome data from the Geelong Hospital and St Vincent's Hospital cardiac surgery databases.

Quality control of surgical and interventional procedures: a review of the CUSUM

BACKGROUND

The report of the CUSUM across surgical and interventional procedures has spawned a fair confusion in the literature.

AIM

To assess the use of the CUSUM and to clarify its utilisation in the perspective of future studies. Nature of the study: Retrospective review.

METHODS

A systematic literature search of Medline was carried out. From each article, data regarding the design of the study, the specialty, the performance criterion, the unit under control, the methodology and the model of the CUSUM used, the use of a graph, the use of a test and the type of test applied were retrieved.

RESULTS

31 studies were found relevant. The design was mainly retrospective for the analysis of the learning curve. The main performance criteria under control were morbidity, mortality and success of the procedure. A graph was plotted in all studies as a CUSUM plot or as cumulative sums of non-negative values. A test was used in 17 studies. Mislabelling of the plot and the test, and misuse of control limits were the most commonly reported mistakes.

CONCLUSION

The CUSUM tool is not yet properly reported in the surgical literature. Therefore, reporting of the CUSUM should be clarified and standardised before its use widens.

Acquiring proficiency in off-pump surgery: traversing the learning curve, reproducibility, and quality control

As the risk profile of patients considered for surgical revascularization worsens, the cumulative benefit of off-pump coronary artery bypass (OPCAB) over conventional coronary artery bypass grafting, in terms of lower morbidity and reduced healthcare costs, may increase. There is still resistance to the introduction of OPCAB surgery however, its practice is variable and surgical residents are rarely trained in these techniques. This article considers how the learning curve in OPCAB may be negotiated and prospectively monitored to ensure quality control. The evidence suggests that situations in which suitable senior expertise exists, OPCAB surgery can be introduced into surgical practice and safely taught to trainees without detriment to patients. This is achieved by a progressive increase in the complexity of the case mix and careful early supervision. The introduction of OPCAB has coincided with the increasing use of control charts as quality control tools. Performance monitoring provides reassurance that patients are not being put at risk during the introduction of OPCAB; control chart methods can be used prospectively for real time performance monitoring by consultant surgeons and residents alike. These techniques may ultimately be used to determine proficiency and accreditation. Increasing use of parallel training techniques, the development of structured training programs that encompass OPCAB and other new technologies in cardiac surgery, coupled with objective performance monitoring are warranted to meet the needs of a changing patient population.

Learning curve of coronary surgery by a cardiac surgeon in Japan with the use of cumulative sum analysis

OBJECTIVE

Case-volume has been believed to be of paramount importance in becoming a good cardiac surgeon. However, for a training cardiac surgeon there is no evidence regarding the specific number of necessary cases to attain the medico-legally acceptable level. We attempted to observe the learning curve in performing coronary surgery with the use of the cumulative sum (CUSUM) method to provide some evidence.

METHODS

From April 2000 to March 2004, a cardiac surgeon, being board-certified but not being on the attending staff, performed 50 coronary artery bypass grafting (CABG) under supervision of three different chief attending surgeons at three different hospitals. His learning curve was analyzed with the use of CUSUM. Mortality and morbidity and the average time of crossclamp and operation at each hospital were examined. Also, the learning curve of 100 left internal mammary artery (LIMA) harvestings were analyzed via CUSUM in terms of harvesting time and injury rate.

RESULTS

The CUSUM curve tended to come closer to the alert line (0.80 confidence) until the 23rd case, but never reached the alert line thereafter until the 50th case. The CUSUM curve never transgressed the alarm line (0.95 confidence) throughout 50 cases. The CUSUM curve of LIMA harvesting approached the alert line in the 38th case and the 59th case, but thereafter never reached the alert line and remained below the reassurance line from the 73rd case.

CONCLUSION

Tentatively, it is inferred that approximately 23 cases of CABG may be sufficient to allow for independent practice and that 73 cases of LIMA harvesting are sufficient to allow independent practice.

Monitoring the performance of residents during training in off-pump coronary surgery

OBJECTIVE

Control charts (eg, cumulative sum charts) plot changes in performance with time and can alert a surgeon to suboptimal performance. They were used to compare performance of off-pump coronary artery bypass surgery between a consultant and four resident surgeons and to compare performance of off-pump coronary artery bypass surgery and conventional coronary artery bypass grafting within surgeons.

METHODS

Data were analyzed for consecutive patients undergoing coronary artery bypass grafting who were operated on by one consultant or one of four residents. Conversions were analyzed by intention to treat. Perioperative death or one or more of 10 adverse events constituted failure. Predicted risks of failure for individual patients were derived from the study population. Variable life-adjusted displays and risk-adjusted sequential probability ratio test charts were plotted.

RESULTS

Data for 1372 patients were analyzed; 769 of the procedures were off-pump coronary artery bypass operations (56.0%). The consultant operated on 382 patients (293 off-pump, 76.7%), and the residents operated on 990 (474 off-pump, 47.9%). Patients operated on by residents tended to be older, more obese, more likely to require an urgent operation, and more likely to need a circumflex artery graft but less likely to have triple-vessel disease. There were 7 conversions (consultant 5, residents 2). The overall failure rate was 8.5% (9.2% for consultant's operations and 8.2% for residents' operations), including 10 deaths (0.7%). Predicted and observed risks of failure were similar for all five surgeons. After 100 off-pump coronary artery bypass operations, performance was the same or better for the residents as for the consultant. For all surgeons, performance was the same or better for off-pump as for conventional coronary artery bypass grafting.

CONCLUSIONS

Off-pump coronary artery bypass surgery can be safely taught to cardiothoracic residents. Implementation of continuous performance monitoring for residents is practicable.

Cumulative sum failure analysis of the learning curve with endovascular abdominal aortic aneurysm repair

PURPOSE

The purpose of this study was to evaluate the importance of experience and the learning curve with endovascular abdominal aortic aneurysm (AAA) repair.

METHODS

A retrospective analysis was performed of all elective endovascular AAA repairs attempted by an individual surgeon and radiologist over a 4-year period. The primary outcome variable was achievement and 30-day maintenance of initial clinical success as defined by the Society for Vascular Surgery/American Association of Vascular Surgery reporting standards. Following standard statistical analysis, the cumulative sum (CUSUM) method was used to analyze the learning curve, with a predetermined acceptable failure rate of 10% and calculated 80% alert and 95% alarm lines.

RESULTS

Ninety-six elective endovascular AAA repairs were attempted by this team between 1998 and 2002 (mean age 74 +/- 0.8 years; mean aneurysm diameter 5.98 +/- 0.8 cm). Initial clinical success was achieved and maintained in 85 of 96 patients (88.5%). Although results were acceptable throughout the study period, improved results with respect to the target failure rate (10%) were not achieved until 60 patients were treated. The learning or CUSUM curves did not differ for different device manufacturers, with improved results being achieved following 20 implantations of each device. The results did differ when comparing aortouniiliac grafts (n = 27) and bifurcated grafts (n = 64). Results with bifurcated grafts remained consistent throughout the study period, whereas with aortouniiliac grafts, results improved after only a few procedures in comparison with the target failure rate.

CONCLUSION

Success rates with endovascular aneurysm repair will improve with an individual's experience. The CUSUM method is a valuable tool in the evaluation of this learning curve, which has credentialing and training implications. Although acceptable results were obtained throughout the study period, this analysis indicates that 60 endovascular aneurysm repairs, or 20 with an individual device, are necessary before optimal rates of initial clinical success can be achieved. These results can be achieved more readily with aortouniiliac grafts than with bifurcated grafts.

Meta-analysis of short-term and mid-term outcomes following off-pump coronary artery bypass grafting

BACKGROUND

Uncertainty continues to surround the relative benefits and harms of conventional coronary artery bypass grafting (CABG) and off-pump coronary artery bypass grafting (OPCABG). Possible reasons are that high-quality studies have not comprehensively examined relevant patient outcomes and have enrolled a limited range of patients. Some studies may have been too small to detect clinically important differences in patient outcomes. The present study addresses these issues using meta-analysis.

METHODS

We comprehensively retrieved randomized and nonrandomized controlled studies according to predetermined criteria. We performed meta-analyses for each outcome and empirically determined whether potential biases that might result from differences in study design or patient characteristics actually biased a study's results. We also conducted sensitivity analyses and tested for publication bias.

RESULTS

Rates of perioperative myocardial infarction, stroke, reoperation for bleeding, renal failure, and mortality were lower after OPCABG than after CABG. Reductions in length of hospital stay, atrial fibrillation, and wound infection were also associated with OPCABG, but statistically significant differences among study results for these outcomes could not be explained by available information. Midterm (3 to 25 months) angina recurrence did not appear to differ between treatments; a trend was noticed toward lower reintervention rates with CABG, and a trend toward lower overall mortality with OPCABG, at least when performed at experienced centers. These midterm outcome results require confirmation.

CONCLUSIONS

Off-pump coronary artery bypass grafting appears to reduce length of hospital stay, operative morbidity, and operative mortality relative to on-pump CABG. More studies are required before firm conclusions can be drawn concerning the effect of OPCABG on midterm mortality, angina recurrence, and repeat intervention.

Effect of off-pump coronary artery bypass grafting on risk-adjusted and cumulative sum failure outcomes after coronary artery surgery

BACKGROUND AND AIM

We have shown that cumulative sum (CUSUM) failure analysis may be more sensitive than standard statistical methods in detecting a cluster of adverse patient outcomes after cardiac surgical procedures. We therefore applied CUSUM, as well as standard statistical techniques, to analyze a surgeon's experience with off-pump coronary artery bypass grafting (OPCAB) and on-pump procedures to determine whether the two techniques have similar or different outcomes.

METHODS

In 320 patients undergoing nonemergent, first time coronary artery bypass grafting, preoperative patient characteristics, rates of mortality and major complications, and ICU and hospital lengths of stay were compared between the on-pump and OPCAB cohorts using Fisher's exact tests and Wilcoxon two sample tests. Predicted mortality and length of stay were determined using previously validated models of the Cardiac Care Network of Ontario. Observed versus expected ratios of both variables were calculated for the two types of procedures. Furthermore, CUSUM curves were constructed for the on-pump and OPCAB cohorts. A multivariable analysis of the predictors of hospital length of stay was also performed to determine whether the type of coronary artery bypass procedure had an independent impact on this variable.

RESULTS

The predicted mortality risk and predicted hospital length of stay were almost identical in the 208 on-pump patients (2.2 +/- 3.9%; 8.2 +/- 2.5 days) and the 112 OPCAB patients (2.0 +/- 2.2%; 7.8 +/- 2.1 days). The incidence of hospital mortality and postoperative stroke were 2.9% and 2.4% in on-pump patients versus zero in OPCAB patients (p = 0.09 and 0.17, respectively). Mechanical ventilation for greater than 48 hours was significantly less common in OPCAB (1.8%) than in on-pump patients (7.7%, p = 0.04). The rate of 10 major complications was 14.9% in on-pump versus 8.0% in OPCAB patients (p = 0.08). OPCAB patients experienced a hospital length of stay that was a median of 1.0 day shorter than on-pump patients (p = 0.01). The observed versus expected ratio for length of stay was 0.78 in OPCAB patients versus 0.95 in on-pump patients. On CUSUM analysis, the failure curve in OPCAB patients was negative and was flatter than that of on-pump patients throughout the duration of the study. Furthermore, OPCAB was an independent predictor of a reduced hospital length of stay on multivariable analysis.

CONCLUSIONS

OPCAB was associated with better outcomes than on-pump coronary artery bypass despite a similar predicted risk. This robust finding was documented on sensitive CUSUM analysis, using standard statistical techniques and on a multivariable analysis of the independent predictors of hospital length of stay.