The CUSUM chart method as a tool for continuous monitoring of clinical outcomes using routinely collected data

Treatment burden of robotic gastrectomy for locally advanced gastric cancer (LAGC): a single western experience

Background

This study compares standard of care (SOC) open and robotic D2-gastrectomy for locally advanced gastric cancer (LAGC) in the Western context of low disease prevalence, reduced surgical volume, and neoadjuvant chemotherapy (NAC). We hypothesized that robotic gastrectomy (RG) after NAC reduces treatment burden for LAGC across multiple outcome domains vs. SOC.

Methods

Single institution, interrupted time series comparing SOC (2008–2013) for LAGC (T2–4Nany/TanyN+) vs. NAC + RG (2013–2018). Treatment burden was a composite metric of narcotic consumption, oncologic efficacy, cumulative morbidity, and 90-day resource utilization. Predictors were evaluated via multivariate modeling. Learning curve analysis was done using CUSUM.

Results

After exclusions, 87 subjects with equivalent baseline characteristics, aside from male sex, were treated via SOC (n=55) or NAC + RG (n=32). All four domains of treatment burden were significantly reduced in the NAC + RG cohort compared to SOC (P=0.003). The odds ratio for excess treatment burden in the NAC/RG was 0.23 (95% CI: 0.07–0.72, P=0.0117) vs. SOC upon multivariable modeling, whereas the extent of resection (total/subtotal), tumor size, T-stage, sex, and early learning curve had no effect. Differences in treatment burden persisted in subgroup analysis for NAC (n=51).

Conclusions

NAC + RG was associated with decreased treatment burden relative to SOC for LAGC. Frequencies of unfavorable hospitalization, adverse oncological outcomes, major morbidity, and narcotic consumption all decreased in this interrupted time series.

Cumulative summation analysis of learning curve for robotic-assisted hiatal hernia repairs

BACKGROUND

Robotic-assisted laparoscopic surgery (RALS) is evolving as an important surgical approach in the field of general surgery. We aimed to evaluate the learning curve for RALS procedures involving repair of hiatal hernias.

METHODS

A series of robotic-assisted hiatal hernia (HH) repairs were performed between 2013 and 2017 by a surgeon at a single institution. Data were entered into a retrospective database. Patient demographics and intraoperative parameters including console time (CT), surgery time (ST), and total operative time (OT) were examined and abstracted for learning curve analysis using the cumulative sum (CUSUM) method. Assessment of perioperative and post-operative outcomes were calculated using descriptive statistics.

RESULTS

The average age of the patients was 57.4 years, average BMI was 29.9 kg/m², median American Society of Anesthesiologists (ASA) classification was 2, and average Charlson Comorbidity Index (CCI) score was 2.8. The series had a mean CT of 132.6 min, mean ST of 145.1 min, and mean OT of 197.4 min. The CUSUM learning curve for CT was best approximated as a third-order polynomial consisting of three unique phases: the initial training phase (case 1-40), the improvement phase (case 41-85), and the mastery phase (case 86 onwards). There was no significant difference in perioperative complications between the phases. Short-term clinical outcomes were comparable with national standards and did not correlate significantly with operative experience.

CONCLUSIONS

The three phases identified with CUSUM analysis represented characteristic stages of the learning curve for robotic hiatal hernia procedures. Our data suggest the training phase is achieved after 40 cases and a high level of mastery is achieved after approximately 85 cases. Thus, the CUSUM method serves as a useful tool for objectively evaluating practical skills for surgeons and can ultimately help establish milestones that assess surgical competency during robotic surgery training.

Constructing Learning Curves to Benchmark Operative Performance of General Surgery Residents Against a National Cohort of Peers

OBJECTIVE

No method or data exist to allow surgical trainees or their programs to contextualize their technical progress. The objective of this study was to create peer benchmarks for Cumulative Sum (CUSUM) charts based upon operative evaluations from a national cohort of general surgery residents.

DESIGN, SETTING, PARTICIPANTS

In 2016-2018, faculty from 26 general surgery residency programs nationwide rated 328 residents' operative performance on a case-by-case basis using a validated 5-point Likert scale. An individual case was considered a "misstep" if scoring below the national median score for that procedure in that postgraduate year (PGY). We constructed 2-sided observed-expected CUSUM charts to capture each resident's cumulative performance over time relative to the national medians. Upper (failure) and lower (positive outlier) benchmarks were established based on the PGY-specific 75th percentile and median misstep rates; consistent/repeated missteps are reflected by crossing of the upper boundary. Procedures with ≤10 observations and residents who were evaluated <10 times for each PGY were excluded.

RESULTS

Around 8,161 evaluations on 76 procedure types were analyzed. The individual misstep rate was lowest among PGY-3s at 13.3% and highest among PGY-4s at 28.6%. No interns had curves that crossed the failure boundary. 8.7% of PGY-2s and 8.9% of PGY-3s finished the year past the failure boundary. PGY-2s had the most positive outliers, with 28.3% of them demonstrating an outlying success performance beyond the lower boundary for at least once. PGY-5s most frequently failed, with 16.7% ever crossing the upper boundary and 11.1% remaining above it at graduation.

CONCLUSIONS

CUSUM is a valid statistical approach for benchmarking individual residents' operative performance against national peers as they progress through the year in real-time. With further validation, CUSUM could be used to set progression and/or graduation standards and objectively identify residents who might benefit from remediation.

The Learning Curve for Transurethral Resection of Bladder Tumour: How Many is Enough to be Independent, Safe and Effective Surgeon?

OBJECTIVE

Transurethral resection of the bladder tumour (TURBT) is one the most common urological procedures. It is also one the fundamental surgeries performed by residents. The learning curve (LC) for TUR has never been analysed. The aim of the study was to analyse the learning curve of TURBT in a residency setting.

DESIGN, SETTING AND PARTICIPANTS

This retrospective multicentre analysis of prospectively maintained databases enrolled 993 consecutive TURBTs performed by 10 urology residents in 3 academic institutions. Study end-points were as follows: the absence of muscularis propria in a specimen, any intra- or postoperative surgical complication and 3-month recurrence-free survival.

RESULTS

With increasing experience, residents operated more complex cases defined by higher rate of large, multifocal or high-risk tumours. In the same time, surgery time, postoperative catheterization time and hospital stay became shorter. An improvement has been noticed regarding the muscularis propria sampling and 3-month recurrence-free survival, but not regarding the risk of surgical complications. Evident improvement in study end-points was noticed after 101 operations; surgeons achieved the best clinical outcomes after performing 170 procedures, whereas the poorest results for the first 45 operations.

CONCLUSIONS

TURBT has a flat LC with 100 cases being the absolute minimum for a resident in training to achieve acceptable oncological and surgical outcomes.

Including non-concurrent control patients in the analysis of platform trials: is it worth it?

Background

Platform trials allow adding new experimental treatments to an on-going trial. This feature is attractive to practitioners due to improved efficiency. Nevertheless, the operating characteristics of a trial that adds arms have not been well-studied. One controversy is whether just the concurrent control data (i.e. of patients who are recruited after a new arm is added) should be used in the analysis of the newly added treatment(s), or all control data (i.e. non-concurrent and concurrent).

Methods

We investigate the benefits and drawbacks of using non-concurrent control data within a two-stage setting. We perform simulation studies to explore the impact of a linear and a step trend on the inference of the trial. We compare several analysis approaches when one includes all the control data or only concurrent control data in the analysis of the newly added treatment.

Results

When there is a positive trend and all the control data are used, the marginal power of rejecting the corresponding hypothesis and the type one error rate can be higher than the nominal value. A model-based approach adjusting for a stage effect is equivalent to using concurrent control data; an adjustment with a linear term may not guarantee valid inference when there is a non-linear trend.

Conclusions

If strict error rate control is required then non-concurrent control data should not be used; otherwise it may be beneficial if the trend is sufficiently small. On the other hand, the root mean squared error of the estimated treatment effect can be improved through using non-concurrent control data.

Detecting change in comparison to peers in NHS prescribing data: a novel application of cumulative sum methodology

Background

The widely used OpenPrescribing.net service provides standard measures which compare prescribing of Clinical Commissioning Groups (CCGs) and English General Practices against that of their peers. Detecting changes in prescribing behaviour compared with peers can help identify missed opportunities for medicines optimisation. Automating the process of detecting these changes is necessary due to the volume of data, but challenging due to variation in prescribing volume for different measures and locations. We set out to develop and implement a method of detecting change on all individual prescribing measures, in order to notify CCGs and practices of such changes in a timely manner.

Methods

We used the statistical process control method CUSUM to detect prescribing behaviour changes in relation to population trends for the individual standard measures on OpenPrescribing. Increases and decreases in percentile were detected separately, using a multiple of standard deviation as the threshold for detecting change. The algorithm was modified to continue re-triggering when trajectory persists. It was deployed, user-tested, and summary statistics generated on the number of alerts by CCG and practice.

Results

The algorithm detected changes in prescribing for 32 prespecified measures, across a wide range of CCG and practice sizes. Across the 209 English CCGs, a mean of 2.5 increase and 2.4 decrease alerts were triggered per CCG, per month. For the 7578 practices, a mean of 1.3 increase and 1.4 decrease alerts were triggered per practice, per month.

Conclusions

The CUSUM method appears to effectively discriminate between random noise and sustained change in prescribing behaviour. This method aims to allow practices and CCGs to be informed of important changes quickly, with a view to improve their prescribing behaviour. The number of alerts triggered for CCGs and practices appears to be appropriate. Prescribing behaviour after users are alerted to changes will be monitored in order to assess the impact of these alerts.

Electronic supplementary material

The online version of this article (10.1186/s12911-018-0642-6) contains supplementary material, which is available to authorized users.

Epidemiology of pertussis in Alberta, Canada 2004-2015

BACKGROUND

We describe the epidemiology of pertussis in Alberta, Canada by person, place, and time between 2004 and 2015, identify outbreak years, and examine vaccination coverage and vaccination timeliness.

METHODS

We used health data from Alberta's Communicable Disease Registry System for the period of January 1, 2004 through August 31, 2015 to identify unique cases of pertussis. Unique cases were deterministically linked to data in Alberta's immunization repository and health care insurance plan registry. Population estimates and vaccination coverage were extracted from Alberta's online Interactive Health Data Application. We estimated pertussis incidence rates per 100,000 persons by year, age group, gender, and health zone. Outbreak years were identified using a one-sided cumulative sum (CUSUM) analysis by comparing annual incidence rates to baseline rates.

RESULTS

Over the period, 3510 cases of pertussis were confirmed by laboratory testing or epidemiological linkage. Incidence rates per 100,000 persons were highest in 2004 (20.5), 2005 (13.6), and 2015 (10.4) for all age groups. Incidence rates were highest among the youngest age groups and decreased as age groups increased. Based on CUSUM analysis, 2008 and 2012 met the criteria for outbreak years. Vaccination coverage was over 90% among the general population, however only 61% of cases received at least one dose. About 60% of cases were diagnosed 5+ years after receiving the vaccine. Approximately 87-91% of vaccinated cases did not receive the first three vaccine doses in a timely manner.

CONCLUSION

Pertussis incidence rates fluctuated over the period across all age groups. The majority of cases had no record of vaccination or were delayed in receiving vaccines. CUSUM analysis was an effective method for identifying outbreaks.

Variation in and risk factors for paediatric inpatient all-cause mortality in a low income setting: data from an emerging clinical information network

BACKGROUND

Hospital mortality data can inform planning for health interventions and may help optimize resource allocation if they are reliable and appropriately interpreted. However such data are often not available in low income countries including Kenya.

METHODS

Data from the Clinical Information Network covering 12 county hospitals' paediatric admissions aged 2-59 months for the periods September 2013 to March 2015 were used to describe mortality across differing contexts and to explore whether simple clinical characteristics used to classify severity of illness in common treatment guidelines are consistently associated with inpatient mortality. Regression models accounting for hospital identity and malaria prevalence (low or high) were used. Multiple imputation for missing data was based on a missing at random assumption with sensitivity analyses based on pattern mixture missing not at random assumptions.

RESULTS

The overall cluster adjusted crude mortality rate across hospitals was 6 · 2% with an almost 5 fold variation across sites (95% CI 4 · 9 to 7 · 8; range 2 · 1% - 11 · 0%). Hospital identity was significantly associated with mortality. Clinical features included in guidelines for common diseases to assess severity of illness were consistently associated with mortality in multivariable analyses (AROC =0 · 86).

CONCLUSION

All-cause mortality is highly variable across hospitals and associated with clinical risk factors identified in disease specific guidelines. A panel of these clinical features may provide a basic common data framework as part of improved health information systems to support evaluations of quality and outcomes of care at scale and inform health system strengthening efforts.

Learning curve for EUS in gastric cancer T staging by using cumulative sum analysis

BACKGROUND

EUS is an operator-dependent procedure and requires more technical and cognitive skills than a routine endoscopic procedure. The learning curve for the staging of gastric cancer, however, has not been evaluated.

OBJECTIVE

To evaluate the threshold number of EUS examinations for gastric cancer T staging.

DESIGN

Retrospective study.

SETTING

University-affiliated tertiary care hospital in the Republic of Korea.

PATIENTS

Four trainees with no previous EUS experience.

INTERVENTION

Analyzing performance of EUS trainees in gastric cancer T staging by using cumulative sum (CUSUM) analysis.

MAIN OUTCOME MEASUREMENTS

CUSUM plot and a minimal number of procedures for reaching a plateau.

RESULTS

A total of 553 initial EUS examinations for treatment-naïve gastric cancers, performed by trainees, were enrolled in the study. The final T stage was determined by experts by using EUS in 332 gastric cancer cases, whereas the T stage of the other 221 lesions was determined by trainees by using EUS. The accuracy of EUS examinations performed by trainees and experts was 72.6% and 84.3%, respectively. The number of EUS examinations required to reach the first plateau in each trainee was 20, 41, 60, and 65.

LIMITATIONS

Retrospective study with a relatively small number of trainees.

CONCLUSION

The CUSUM scores of all of 4 trainees in the study reached a plateau by the 65th examination.

An analysis of the learning curve to achieve competency at colonoscopy using the JETS database

OBJECTIVE

The number of colonoscopies required to reach competency is not well established. The primary aim of this study was to determine the number of colonoscopies trainees need to perform to attain competency, defined by a caecal intubation rate (CIR) ≥90%. As competency depends on completion, we also investigated trainee factors that were associated with colonoscopy completion.

DESIGN

The Joint Advisory Group on GI Endoscopy in the UK has developed a trainee e-portfolio from which colonoscopy data were retrieved. Inclusion criteria were all trainees who had performed a total of ≥20 colonoscopies and had performed ≤50 colonoscopies prior to submission of data to the e-portfolio. The primary outcome measure was colonoscopy completion. The number of colonoscopies required to achieve CIR ≥90% was calculated by the moving average method and learning curve cumulative summation (LC-Cusum) analysis. To determine factors which determine colonoscopy completion, a mixed effect logistic regression model was developed which allowed for nesting of patients within trainees and nesting of patients within hospitals, with various patient, trainee and training factors entered as fixed effects.

RESULTS

297 trainees undertook 36 730 colonoscopies. By moving average analysis, the cohort of trainees reached a CIR of 90% at 233 procedures. By LC-Cusum analysis, 41% of trainees were competent after 200 procedures. Of the trainee factors, the number of colonoscopies, intensity of training and previous flexible sigmoidoscopy experience were significant factors associated with colonoscopy completion.

CONCLUSIONS

This is the largest study to date investigating the number of procedures required to achieve competency in colonoscopy. The current training certification benchmark in the UK of 200 procedures does not appear to be an inappropriate minimum requirement. The LC-Cusum chart provides real time feedback on individual learning curves for trainees. The association of training intensity and flexible sigmoidoscopy experience with colonoscopy completion could be exploited in training programmes.

A new risk-adjusted Bernoulli cumulative sum chart for monitoring binary health data

To monitor a health event in patients with a specific risk of developing the event, a risk-adjusted cumulative sum chart is needed. The risk-adjusted cumulative sum chart proposed in the literature has some limitations. Setting appropriate control limits is not straightforward, there is no simple formula for constructing them, and they remain sensitive to changes in the underlying risk distribution and the baseline incidence rate. To overcome these limits, we propose a new risk-adjusted Bernoulli cumulative sum chart as a simple and efficient solution. Analyses of simulated and real data sets illustrate the performance and usefulness of the proposed procedure.

Learning curves of macintosh laryngoscope in nurse anesthetist trainees using cumulative sum method

Background. Tracheal intubation is a potentially life-saving procedure. This skill is taught to many anesthetic healthcare professionals, including nurse anesthetists. Our goal was to evaluate the learning ability of nurse anesthetist trainees in their performance of orotracheal intubation with the Macintosh laryngoscope. Methods. Eleven nurse anesthetist trainees were enrolled in the study during the first three months of their training. All trainees attended formal lectures and practice sessions with manikins at least one time on performing successful tracheal intubation under supervision of anesthesiology staff. Learning curves for each nurse anesthetist trainee were constructed with the standard cumulative summation (cusum) methods. Results. Tracheal intubation was attempted on 388 patients. Three hundred and six patients (78.9%) were successfully intubated on the trainees' first attempt and 17 patients (4.4%) on the second attempt. The mean ± SD number of orotracheal intubations per trainee was 35.5 ± 5.1 (range 30-47). Ten (90.9%) of 11 trainees crossed the 20% acceptable failure rate line. A median of 22 procedures was required to achieve an 80% orotracheal intubations success rate. Conclusion. At least 22 procedures were required to reach an 80% success rate for orotracheal intubation using Macintosh laryngoscope in nonexperienced nurse anesthetist trainees.

Graphical model-based O/E control chart for monitoring multiple outcomes from a multi-stage healthcare procedure

Most statistical process control programmes in healthcare focus on surveillance of outcomes at the final stage of a procedure, such as mortality or failure rates. Such an approach ignores the multi-stage nature of these procedures, in which a patient progresses through several stages prior to the final stage. In this paper, we introduce a novel approach to statistical process control programmes in healthcare. Our proposed approach is based on the regression adjustment and multi-stage control charts that have been in use in industrial applications for decades. Three advantages of the approach are: better understanding of how outcomes at different stages relate to each other, explicit monitoring of upstream stage outcomes may help curtail trends that lead to poorer end-stage outcomes and understanding the impact of each stage can help determine the most effective allocation of quality improvement resources. A test statistic for the control charts is proposed. Simulations are performed to test the control charts, and the results are summarised using an empirical probability of true detection. An illustrative example using data from a maternity unit is included. A main result from the simulation study is that taking a multi-stage approach makes it easer to explicitly identify shifts in upstream stage outcomes that might otherwise be signalled in final stage outcomes if dependence between stages is ignored.

Demographic transition in India: an evolutionary interpretation of population and health trends using 'change-point analysis'

BACKGROUND AND RATIONALE

Lack of a robust analytical tool for trend analysis of population and health indicators is the basic rationale of this study. In an effort to fill this gap, this study advances 'Change-Point analyzer' as a new analytical tool for assessment of the progress and its pattern in population and health indicators.

METHODOLOGY/PRINCIPAL FINDINGS

The defining feature of 'change-point analyzer' is that, it detects subtle changes that are often missed in simple trend line plots and also quantified the volume of change that is not possible in simple trend line plots. A long-term assessment of 'change-point analyses' of trends in population and health indicators such as IMR, Population size, TFR, and LEB in India show multiple points of critical changes. Measured change points of demographic and health trends helps in understanding the demographic transitional shifts connecting it to contextual policy shifts. Critical change-points in population and health indicators in India are associated with the evolution of structural changes in population and health policy framework.

CONCLUSIONS

This study, therefore, adds significantly to the evolutionary interpretation of critical change-points in long-term trajectories of population and health indicators vis-a-vis population and health policy shifts in India. The results have not only helped in reassessing the historical past and the current demographic transition trajectory but also advanced a new method of assessing the population and health trends which are necessary for robust monitoring of the progress in population and health policies.

Statistical methods to monitor risk factors in a clinical database: example of a national cardiac surgery registry

BACKGROUND

Comparison of outcomes requires adequate risk adjustment for differences in patient risk and the type of intervention performed. Both unintentional and intentional misclassification (also called gaming) of risk factors might lead to incorrect benchmark results. Therefore, misclassification of risk factors should be detected. We investigated the use of statistical process control techniques to monitor the frequency of risk factors in a clinical database.

METHODS AND RESULTS

A national population-based study was performed using simulation and statistical process control. All patients who underwent cardiac surgery between January 1, 2007, and December 31, 2009, in all 16 cardiothoracic surgery centers in the Netherlands were included. Data on 46 883 consecutive cardiac surgery interventions were extracted. The expected risk factor frequencies were based on 2007 and 2008 data. Monthly frequency rates of 18 risk factors in 2009 were monitored using a Shewhart control chart, exponentially weighted moving average chart, and cumulative sum chart. Upcoding (ie, gaming) in random patients was simulated and detected in 100% of the simulations. Subtle forms of gaming, involving specifically high-risk patients, were more difficult to identify (detection rate of 44%). However, the accompanying rise in mean logistic European system for cardiac operative risk evaluation (EuroSCORE) was detected in all simulations.

CONCLUSIONS

Statistical process control in the form of a Shewhart control chart, exponentially weighted moving average, and cumulative sum charts provide a means to monitor changes in risk factor frequencies in a clinical database. Surveillance of the overall expected risk in addition to the separate risk factors ensures a high sensitivity to detect gaming. The use of statistical process control for risk factor surveillance is recommended.

Real time monitoring of risk-adjusted paediatric cardiac surgery outcomes using variable life-adjusted display: implementation in three UK centres

Objective

To implement routine in-house monitoring of risk-adjusted 30-day mortality following paediatric cardiac surgery.

Design

Collaborative monitoring software development and implementation in three specialist centres.

Patients and methods

Analyses incorporated 2 years of data routinely audited by the National Institute of Cardiac Outcomes Research (NICOR). Exclusion criteria were patients over 16 or undergoing non-cardiac or only catheter procedures. We applied the partial risk adjustment in surgery (PRAiS) risk model for death within 30 days following surgery and generated variable life-adjusted display (VLAD) charts for each centre. These were shared with each clinical team and feedback was sought.

Results

Participating centres were Great Ormond Street Hospital, Evelina Children's Hospital and The Royal Hospital for Sick Children in Glasgow. Data captured all procedures performed between 1 January 2010 and 31 December 2011. This incorporated 2490 30-day episodes of care, 66 of which were associated with a death within 30 days.The VLAD charts generated for each centre displayed trends in outcomes benchmarked to recent national outcomes. All centres ended the 2-year period within four deaths from what would be expected. The VLAD charts were shared in multidisciplinary meetings and clinical teams reported that they were a useful addition to existing quality assurance initiatives. Each centre is continuing to use the prototype software to monitor their in-house surgical outcomes.

Conclusions

Timely and routine monitoring of risk-adjusted mortality following paediatric cardiac surgery is feasible. Close liaison with hospital data managers as well as clinicians was crucial to the success of the project.

The use of statistical process control (risk-adjusted CUSUM, risk-adjusted RSPRT and CRAM with prediction limits) for monitoring the outcomes of out-of-hospital cardiac arrest patients rescued by the EMS system

OBJECTIVE

Based on previous experience from surgical surveillance, risk-adjusted cumulative sum (CUSUM)-type charts were applied to monitor out-of-hospital cardiac arrest (OHCA) patient mortality.

MATERIALS AND METHODS

Data from 2356 OHCA patients were collected by the Taipei County Fire Bureau from June 2006 to November 2007. Logistic regression analysis was applied to create a risk-adjusted model. Next, a risk-adjusted CUSUM chart, a risk-adjusted resetting sequential probability ratio test chart and a cumulative risk-adjusted mortality with prediction limits chart were used to detect excess deaths of the OHCA patients rescued by the emergency medical service (EMS) system.

RESULTS

The overall mortality rate, defined as having no return of spontaneous circulation, was 79.3%. These three charts signalled an increase in the death rate at similar sites, and also suggested a small process shift.

CONCLUSION

A visual approach to EMS systems monitoring that combines the risk-adjusted cumulative sum, Risk-adjusted resetting sequential probability ratio test and cumulative risk-adjusted mortality with prediction limits charts was established. It was found that this approach can be effectively used by the EMS community to monitor OHCA outcomes in real time.

Control chart methods for monitoring surgical performance: a case study from gastro-oesophageal surgery

Graphical methods are becoming increasingly used to monitor adverse outcomes from surgical interventions. However, uptake of such methods has largely been in the area of cardiothoracic surgery or in transplants with relatively little impact made in surgical oncology. A number of the more commonly used graphical methods including the Cumulative Mortality plot, Variable Life-Adjusted Display, Cumulative Sum (CUSUM) and funnel plots will be described. Accounting for heterogeneity in case-mix will be discussed and how ignoring case-mix can have considerable consequences. All methods will be illustrated using data from the Scottish Audit of Gastro-Oesophageal Cancer services (SAGOCS) data set.

Statistical process control as a tool for controlling operating room performance: retrospective analysis and benchmarking

BACKGROUND

There is much research using statistical process control (SPC) to monitor surgical performance, including comparisons among groups to detect small process shifts, but few of these studies have included a stabilization process. This study aimed to analyse the performance of surgeons in operating room (OR) and set a benchmark by SPC after stabilized process.

METHODS

The OR profile of 499 patients who underwent laparoscopic cholecystectomy performed by 16 surgeons at a tertiary hospital in Taiwan during 2005 and 2006 were recorded. SPC was applied to analyse operative and non-operative times using the following five steps: first, the times were divided into two segments; second, they were normalized; third, they were evaluated as individual processes; fourth, the ARL(0) was calculated;, and fifth, the different groups (surgeons) were compared. Outliers were excluded to ensure stability for each group and to facilitate inter-group comparison.

RESULTS

The results showed that in the stabilized process, only one surgeon exhibited a significantly shorter total process time (including operative time and non-operative time).

CONCLUSION

In this study, we use five steps to demonstrate how to control surgical and non-surgical time in phase I. There are some measures that can be taken to prevent skew and instability in the process. Also, using SPC, one surgeon can be shown to be a real benchmark.

Monitoring healthcare quality in an obstetrics and gynaecology department using a CUSUM chart

OBJECTIVE

To use cumulative sum (CUSUM) charts for the early detection of variations in quality of care in a maternity department.

DESIGN

Retrospective analysis of prospectively collected data.

SETTING

Maternity department of a teaching hospital in Paris (France).

POPULATION

Data from 20 519 women and 21 448 infants were collected between January 2000 and December 2007.

METHODS

CUSUM charts were used to monitor the rate of 19 pre-selected quality indicators over 3 years (2005-2007), against standards developed by department obstetrician gynaecologists. Periods with adverse event rates that did not meet the standards were identified.

MAIN OUTCOME MEASURES

Quality indicator rates.

RESULTS

Indicators fell into three groups based on the number of periods with unacceptable rates: less than one per year [e.g. the rate of intensive care unit (ICU) admission of mothers and rate of third- or fourth-degree perineal tears]; one every 2-12 months on average (e.g. blood transfusion and sulprostone use in the overall population of women); and at least one per month (insufficient availability of epidural analgesia).

CONCLUSION

CUSUM charts for a broad range of quality indicators can be used to monitor the quality of care in an obstetrics department. A prospective study investigating the ability of CUSUM-based monitoring to improve maternal and neonatal outcomes would be of interest.

Continuous independent quality control for fetal ultrasound biometry provided by the cumulative summation technique

OBJECTIVE

To apply the cumulative summation (CUSUM) technique for an evaluation of the learning process of sonographic fetal weight estimation at term in combination with the z-scores of biometry determinants and to assess the time of appearance and sources of errors.

METHODS

Learning curve (LC-CUSUM) and double CUSUM charts for systematic error detection based on absolute and signed mean percentage error were generated to retrospectively estimate the longitudinal accuracy of sonographic fetal weight estimation conducted by three trainees and one experienced examiner. For LC-CUSUM analysis an examination was considered to be a failure when there was an absolute error in birth weight estimation >/= 15%. Fetal biometry measurements (head circumference, abdominal circumference (AC) and femur length (FL)) from 227 routine ultrasound scans of one examiner were separately transformed into z-scores and double CUSUM charts were generated to assess the systematic errors for each determinant.

RESULTS

The LC-CUSUM charts revealed that different numbers of scans are required for different examiners to achieve competence in estimating birth weight. AC and FL deviated most significantly from expected values (P < 0.05). The double CUSUM charts revealed exact periods of systematic errors in the measurement of biometry determinants, clearly reflecting errors of fetal weight estimation.

CONCLUSIONS

The use of CUSUM techniques in the analysis of sonographic data allows observation of the development of an examiner's skill and maintenance of competence. The CUSUM technique not only allows the reasons for impaired fetal weight estimation to be revealed but also allows determination of the exact time when inaccurate measurements start to occur. We suggest that CUSUM charts should be implemented in routine clinical practice as a measure of objective quality evaluation of sonographic fetal biometry.

Comparing early outbreak detection algorithms based on their optimized parameter values

BACKGROUND

Many researchers have evaluated the performance of outbreak detection algorithms with recommended parameter values. However, the influence of parameter values on algorithm performance is often ignored.

METHODS

Based on reported case counts of bacillary dysentery from 2005 to 2007 in Beijing, semi-synthetic datasets containing outbreak signals were simulated to evaluate the performance of five outbreak detection algorithms. Parameters' values were optimized prior to the evaluation.

RESULTS

Differences in performances were observed as parameter values changed. Of the five algorithms, space-time permutation scan statistics had a specificity of 99.9% and a detection time of less than half a day. The exponential weighted moving average exhibited the shortest detection time of 0.1 day, while the modified C1, C2 and C3 exhibited a detection time of close to one day.

CONCLUSION

The performance of these algorithms has a correlation to their parameter values, which may affect the performance evaluation.

Prospective evaluation of a continuous monitoring and quality-improvement system for reducing adverse neonatal outcomes

OBJECTIVE

Our objective was to evaluate a prospective monitoring and quality improvement system for studying trends in the rates of an adverse neonatal outcome, the low Apgar scores (Apgar score <7).

STUDY DESIGN

A cumulative sum (CUSUM) chart-based system was used to monitor the rate of low Apgar scores over 2 years. Root cause analysis (RCA) was used to investigate for causes of periods of increased low Apgar score rates.

RESULTS

A period of deteriorated outcome (increased rates of low Apgar) occurred in August 2006. RCA identified deficiencies in cardiotocograph education, which were addressed by targeted training and mentoring. Prompt resolution followed, with the rates returning to baseline and staying within acceptable limits through to the end of evaluation in December 2007.

CONCLUSION

Prospective and continuous monitoring of clinical outcomes using the CUSUM chart method is feasible and may be beneficial. Early detection of an adverse trend allows for timely corrective action, and may lead to overall improvements in performance.

Sentinel lymph node biopsy for breast cancer using methylene blue dye manifests a short learning curve among experienced surgeons: a prospective tabular cumulative sum (CUSUM) analysis

Background

The benefits of sentinel lymph node biopsy (SLNB) for breast cancer patients with histologically negative axillary nodes, in whom axillary lymph node dissection (ALND) is thereby avoided, are now established. Low false negative rate, certainly with blue dye technique, mostly reflects the established high inherent accuracy of SLNB and low axillary nodal metastatic load (subject to patient selection). SLN identification rate is influenced by volume, injection site and choice of mapping agent, axillary nodal metastatic load, SLN location and skill at axillary dissection. Being more subject to technical failure, SLN identification seems to be a more reasonable variable for learning curve assessment than false negative rate.

Methylene blue is as good an SLN mapping agent as Isosulfan blue and is much cheaper. Addition of radio-colloid mapping to blue dye does not achieve a sufficiently higher identification rate to justify the cost. Methylene blue is therefore the agent of choice for SLN mapping in developing countries.

The American Society of Breast Surgeons recommends that, for competence, surgeons should perform 20 SLNB but admits that the learning curve with a standardized technique may be "much shorter". One appropriate remedy for this dilemma is to plot individual learning curves.

Methods

Using methylene blue dye, experienced breast surgeons performed SLNB in selected patients with breast cancer (primary tumor < 5 cm and clinically negative ipsilateral axilla). Intraoperative assessment and completion ALND were performed for standardization on the first 13 of 24 cases. SLN identification was plotted for each surgeon on a tabular cumulative sum (CUSUM) chart with sequential probability ratio test (SPRT) limits based on a target identification rate of 85%.

Results

The CUSUM plot crossed the SPRT limit line after 8 consecutive, positively identified SLN, signaling achievement of an acceptable level of competence.

Conclusion

Tabular CUSUM charting, based on a justified choice of parameters, indicates that the learning curve for SLNB using methylene blue dye is completed after 8 consecutive, positively identified SLN. CUSUM charting may be used to plot individual learning curves for trainee surgeons by applying a proxy parameter for failure in the presence of a mentor (such as failed SLN identification within 15 minutes).

Managing an emergency department by analysing HIS medical data: a focus on elderly patient clinical pathways

INTRODUCTION

The objective of this paper is to present complementary views of the activity of the emergency department for a specific group of patients. Once validated, these views will be used as decision support tools for better managing the department and providing better care delivery for this population. The views are produced from the data stored in Healthcare Information Systems that correspond potentially to a vast source of information for supporting decisions on management or public health issues.

METHOD

The study focuses on two groups of patients: the elderly population (over 75-years-old) and the under 75-year-old patients, at the Rennes hospital. The validation of the views is performed by comparing results for the two distinct groups. Relevant data were extracted from the Emergency Department database. Several analysis (like cusum chart) and representation tools (Graphviz) were used to study the patients' pathways, the dynamics of arrivals and the patients' characteristics.

RESULTS

The representations provided a synthetic, global and comprehensive view of the department activities, to the satisfaction of the clinicians. The study showed that ICD-10 coding, assigned at the patient's departure from the emergency department hence from all available known clinical data, is not appropriate for the elderly population as these patients are mainly diagnosed by "symptoms" and several solutions are proposed. Finally, it is stressed out that a proper delivery of care to elderly patients should require some level of scheduling in the emergency department that is by essence characterized by its non scheduled activity.