A national quality improvement programme to improve survival after emergency abdominal surgery: the EPOCH stepped-wedge cluster RCT

Quality and safety interventions in surgery

Medical errors resulting in treatment-related harm have been a challenge for many years, with particularly severe consequences in surgery. Efforts to improve safety should focus on system-based changes to response and rescue pathways, and will require further research and adequate engagement by clinical staff.

Associations Between Care Bundles and Postoperative Outcomes After Major Emergency Abdominal Surgery: A Systematic Review and Meta-Analysis

INTRODUCTION

Care bundles were found to improve postoperative outcomes in elective surgery. However, in major emergency general surgery studies show a divergent impact on mortality and length of stay. This meta-analysis aimed to evaluate associations between care bundles and mortality, complications, and length of stay when applied in major emergency general surgery.

METHODS

A systematic literature search in PubMed and Embase was performed on the May 1, 2021. Only comparative studies on care bundles in major emergency general surgery were included. Meta-analysis and trial sequential analysis were performed on 30-d mortality. We undertook a narrative approach of long-term mortality, complications, and length of stay.

RESULTS

Meta-analysis of 13 studies with 35,771 patients demonstrated that care bundles in emergency surgery were not associated with a significant reduction in odds of 30-d mortality (odds ratio = 0.8, 95% confidence interval 0.62-1.03). Trial sequential analysis confirmed that the meta-analysis was underpowered with a minimum of 78,901 patients required for firm conclusions. Seven studies reported complication rates whereof six reported lower complication rates using care bundles.

CONCLUSIONS

Care bundles were reported to decrease postoperative complications in five out of seven studies and seven out of 11 studies reported a shortening in length of stay.

Implementing Bundle Care in Major Abdominal Emergency Surgery: Long-Term Mortality and Comprehensive Complication Index

BACKGROUND

Major abdominal emergency surgery (MAES) has a high risk of postoperative mortality and a high complication rate. The aim of this study was to evaluate whether the implementation of a perioperative care bundle reduced long-term mortality and the Comprehensive Complication Index (CCI) after MAES.

METHODS

This study was a single-centre retrospective cohort study. Data in the intervention group were collected prospectively and compared with a historical cohort from the same centre. It includes adult patients undergoing MAES. We implemented a care bundle under the name Abdominal Surgery Acute Protocol (ASAP). We initiated fast-track initiatives and standardised optimised care in before, during and after surgery. Data were analysed using survival analysis and multiple regression.

RESULTS

We included 120 patients in the intervention cohort and 258 in the historical cohort. The one-year mortality rate was 21.7% in the intervention cohort compared to 28.3% in the standard care cohort. Adjusted odds ratio of one-year mortality 0.81 (CI95% 0.41-1.56). The 30-day mortality was lowered from 19.0 to 6.7% (p = 0.003). The CCI in the intervention cohort was 8.7 (IQR 0-34) compared to 21 (IQR 0-36) in the control cohort (p = 0.932) The length of stay increased by two days (p = 0.021). Most cases had 71-80% protocol compliance.

CONCLUSION

Implementing bundle care in major abdominal emergency surgery lowered the 30-day postoperative mortality. The difference in mortality was preserved over time although not significant after one year. The changes in the Comprehensive Complication Index were not statistically significant.

External validation of a surgical mortality risk prediction model for inpatient noncardiac surgery in an Australian private health insurance dataset

BACKGROUND

We previously conducted a systematic review to identify surgical mortality risk prediction tools suitable for adapting in the Australian context and identified the Surgical Outcome Risk Tool (SORT) as an ideal model. The primary aim was to investigate the external validity of SORT for predicting in-hospital mortality in a large Australian private health insurance dataset.

METHODS

A cohort study using a prospectively collected Australian private health insurance dataset containing over 2 million deidentified records. External validation was conducted by applying the predictive equation for SORT to the complete case analysis dataset. Model re-estimation (recalibration) was performed by logistic regression.

RESULTS

The complete case analysis dataset contained 161 277 records. In-hospital mortality was 0.2% (308/161277). The mean estimated risk given by SORT was 0.2% and the median (IQR) was 0.01% (0.003%-0.08%). Discrimination was high (c-statistic 0.96) and calibration was accurate over the range 0%-10%, beyond which mortality was over-predicted but confidence intervals included or closely approached the perfect prediction line. Re-estimation of the equation did not improve over-prediction. Model diagnostics suggested the presence of outliers or highly influential values.

CONCLUSION

The low perioperative mortality rate suggests the dataset was not representative of the overall Australian surgical population, primarily due to selection bias and classification bias. Our results suggest SORT may significantly under-predict 30-day mortality in this dataset. Given potential differences in perioperative mortality, private health insurance status and hospital setting should be considered as covariables when a locally validated national surgical mortality risk prediction model is developed.

Statistical analysis of publicly funded cluster randomised controlled trials: a review of the National Institute for Health Research Journals Library

BACKGROUND

In cluster randomised controlled trials (cRCTs), groups of individuals (rather than individuals) are randomised to minimise the risk of contamination and/or efficiently use limited resources or solve logistic and administrative problems. A major concern in the primary analysis of cRCT is the use of appropriate statistical methods to account for correlation among outcomes from a particular group/cluster. This review aimed to investigate the statistical methods used in practice for analysing the primary outcomes in publicly funded cluster randomised controlled trials, adherence to the CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines for cRCTs and the recruitment abilities of the cluster trials design.

METHODS

We manually searched the United Kingdom's National Institute for Health Research (NIHR) online Journals Library, from 1 January 1997 to 15 July 2021 chronologically for reports of cRCTs. Information on the statistical methods used in the primary analyses was extracted. One reviewer conducted the search and extraction while the two other independent reviewers supervised and validated 25% of the total trials reviewed.

RESULTS

A total of 1942 reports, published online in the NIHR Journals Library were screened for eligibility, 118 reports of cRCTs met the initial inclusion criteria, of these 79 reports containing the results of 86 trials with 100 primary outcomes analysed were finally included. Two primary outcomes were analysed at the cluster-level using a generalized linear model. At the individual-level, the generalized linear mixed model was the most used statistical method (80%, 80/100), followed by regression with robust standard errors (7%) then generalized estimating equations (6%). Ninety-five percent (95/100) of the primary outcomes in the trials were analysed with appropriate statistical methods that accounted for clustering while 5% were not. The mean observed intracluster correlation coefficient (ICC) was 0.06 (SD, 0.12; range, - 0.02 to 0.63), and the median value was 0.02 (IQR, 0.001-0.060), although 42% of the observed ICCs for the analysed primary outcomes were not reported.

CONCLUSIONS

In practice, most of the publicly funded cluster trials adjusted for clustering using appropriate statistical method(s), with most of the primary analyses done at the individual level using generalized linear mixed models. However, the inadequate analysis and poor reporting of cluster trials published in the UK is still happening in recent times, despite the availability of the CONSORT reporting guidelines for cluster trials published over a decade ago.

Health Economic Evaluation Alongside Stepped Wedge Trials: A Methodological Systematic Review

BACKGROUND

Recently, there has been an increase in use of the stepped wedge trial (SWT) design in the context of health services research, due to its pragmatic and methodological advantages over the parallel group design.

OBJECTIVE

Our objective was to summarise the statistical methods used when conducting economic evaluations alongside SWTs.

METHODS

A systematic literature search extending to February 2020 was conducted in the PubMed, Scopus, Cochrane and National Health Service Economic Evaluation Database (NHS-EED) databases to find and evaluate studies where there was an intention to conduct an economic evaluation alongside an SWT. Studies were assessed for their eligibility, findings, reporting of statistical methods and quality of reporting.

RESULTS

Of the 586 studies retrieved from the literature search, 69 studies were identified and included in this systematic review. A total of 54 studies were published protocols, with eight economic evaluations and seven studies reporting full trial results. Included studies varied in terms of their reporting of statistical methods, in both detail and methodology. There were 34 studies that did not report any statistical methods for the economic evaluation, and only 16 studies reported appropriate methods, mainly using some form of mixed/multilevel model, and two used seemingly unrelated regression. Twelve studies reported the use of generic bootstrap methods and other modelling techniques, whilst the remaining studies failed to appropriately account for clustering, correlation or adjustment for time.

CONCLUSIONS

The use of appropriate statistical methods that account for time, clustering and correlation between costs and outcomes is an important part of SWT health economics analysis, one that will benefit from an effort to communicate the methods available and their performance.

How and under what circumstances do quality improvement collaboratives lead to better outcomes? A systematic review

BACKGROUND

Quality improvement collaboratives are widely used to improve health care in both high-income and low and middle-income settings. Teams from multiple health facilities share learning on a given topic and apply a structured cycle of change testing. Previous systematic reviews reported positive effects on target outcomes, but the role of context and mechanism of change is underexplored. This realist-inspired systematic review aims to analyse contextual factors influencing intended outcomes and to identify how quality improvement collaboratives may result in improved adherence to evidence-based practices.

METHODS

We built an initial conceptual framework to drive our enquiry, focusing on three context domains: health facility setting; project-specific factors; wider organisational and external factors; and two further domains pertaining to mechanisms: intra-organisational and inter-organisational changes. We systematically searched five databases and grey literature for publications relating to quality improvement collaboratives in a healthcare setting and containing data on context or mechanisms. We analysed and reported findings thematically and refined the programme theory.

RESULTS

We screened 962 abstracts of which 88 met the inclusion criteria, and we retained 32 for analysis. Adequacy and appropriateness of external support, functionality of quality improvement teams, leadership characteristics and alignment with national systems and priorities may influence outcomes of quality improvement collaboratives, but the strength and quality of the evidence is weak. Participation in quality improvement collaborative activities may improve health professionals' knowledge, problem-solving skills and attitude; teamwork; shared leadership and habits for improvement. Interaction across quality improvement teams may generate normative pressure and opportunities for capacity building and peer recognition.

CONCLUSION

Our review offers a novel programme theory to unpack the complexity of quality improvement collaboratives by exploring the relationship between context, mechanisms and outcomes. There remains a need for greater use of behaviour change and organisational psychology theory to improve design, adaptation and evaluation of the collaborative quality improvement approach and to test its effectiveness. Further research is needed to determine whether certain contextual factors related to capacity should be a precondition to the quality improvement collaborative approach and to test the emerging programme theory using rigorous research designs.

Improving care at scale: process evaluation of a multi-component quality improvement intervention to reduce mortality after emergency abdominal surgery (EPOCH trial)

BACKGROUND

Improving the quality and safety of perioperative care is a global priority. The Enhanced Peri-Operative Care for High-risk patients (EPOCH) trial was a stepped-wedge cluster randomised trial of a quality improvement (QI) programme to improve 90-day survival for patients undergoing emergency abdominal surgery in 93 hospitals in the UK National Health Service.

METHODS

The aim of this process evaluation is to describe how the EPOCH intervention was planned, delivered and received, at both cluster and local hospital levels. The QI programme comprised of two interventions: a care pathway and a QI intervention to aid pathway implementation, focussed on stakeholder engagement, QI teamwork, data analysis and feedback and applying the model for improvement. Face-to-face training and online resources were provided to support senior clinicians in each hospital (QI leads) to lead improvement. For this evaluation, we collated programme activity data, administered an exit questionnaire to QI leads and collected ethnographic data in six hospitals. Qualitative data were analysed with thematic or comparative analysis; quantitative data were analysed using descriptive statistics.

RESULTS

The EPOCH trial did not demonstrate any improvement in survival or length of hospital stay. Whilst the QI programme was delivered as planned at the cluster level, self-assessed intervention fidelity at the hospital level was variable. Seventy-seven of 93 hospitals responded to the exit questionnaire (60 from a single QI lead response on behalf of the team); 33 respondents described following the QI intervention closely (35%) and there were only 11 of 37 care pathway processes that > 50% of respondents reported attempting to improve. Analysis of qualitative data suggests QI leads were often attempting to deliver the intervention in challenging contexts: the social aspects of change such as engaging colleagues were identified as important but often difficult and clinicians frequently attempted to lead change with limited time or organisational resources.

CONCLUSIONS

Significant organisational challenges faced by QI leads shaped their choice of pathway components to focus on and implementation approaches taken. Adaptation causing loss of intervention fidelity was therefore due to rational choices made by those implementing change within constrained contexts. Future large-scale QI programmes will need to focus on dedicating local time and resources to improvement as well as on training to develop QI capabilities.

EPOCH TRIAL REGISTRATION

ISRCTN80682973 https://doi.org/10.1186/ISRCTN80682973 Registered 27 February 2014 and Lancet protocol 13PRT/7655.