Challenges to quality assurance of surgical interventions in clinical oncology trials: A systematic review

A proposed framework of strategies to overcome challenges to surgical quality assurance in oncology trials (SQA-Onc.)

INTRODUCTION

Randomised controlled trials (RCTs) with surgical interventions frequently lack a framework to ensure surgical quality. We aimed to investigate surgical quality assurance (SQA) in oesophagogastric oncology trials and to develop a translatable framework of strategies to overcome challenges in the design and implementation of SQA.

METHODS

Seventy-one peer-nominated, international, expert trial stakeholders included surgeons; oncologists; trial managers and trial methodologists. Semi-structured interviews were conducted with expert stakeholders examining challenges to SQA in oncology trials followed by a Delphi process to gain consensus on mitigating strategies. Relevant expert consensus strategies were selected for inclusion within a separate written survey and Delphi process in the active ADDICT RCT.

RESULTS

Expert consensus was reached for 59 strategies to overcome challenges to SQA in oncology trials. 19 of these strategies were selected for inclusion within the ADDICT survey and Delphi process, of which 14 (74 %) gained consensus amongst ADDICT trial stakeholders across two Delphi rounds, indicating their relevance within an active surgical oncology RCT. Prominent mitigating strategies included operative monitoring using photographs and/or videos with a structured objective assessment tool. Summarising the expert Delphi consensus allowed formulation of a framework of strategies to overcome challenges to SQA in oncology trials (SQA-Onc.) CONCLUSION: In this first international expert consensus within this area, agreement was reached for 59 strategies to overcome challenges to implementation of SQA. The proposed SQA-Onc. tool is intended to support SQA measures within future trials. Validating this framework within the next generation of RCTs should be the focus of future research.

Methods of surgical quality assurance in cervical and endometrial cancer trials: a systematic review and meta-analysis

Surgery quality in gynecology oncology trials varies, potentially biasing results. This systematic review examines methods for assuring surgical quality in multi-center gynecologic oncology trials and the impact on patient outcomes. A systematic search (2000-2023) was conducted in Medline, Embase, and Web of Science. Multi-center randomized controlled trials reporting on surgical endometrial and cervical cancer trials and lymph node harvest, short-term mortality, or conversion rate were included. Studies were assessed using a 10-point checklist to determine surgical quality. This was used to assess the association with variation in lymph node harvest, post-operative mortality, and conversion rate. Overall, 5963 titles and abstracts were screened for their eligibility and 10 studies reporting on 22 surgical-only arms were included for further analysis. The total number of included patients was 7434 from 366 centers. Analysis showed that standardization of surgical approach (β = -6.6, 95%, p = .043), standardization of the extent of lymphadenectomy (β = -2.432, p = .004), video assessment pre-trial (β = -3.492, p = .04) and monitoring of data including clinical outcome measures (β = -4.018, p = .009) were significantly associated with reducing variation in lymph node harvest. It also showed that standardization of the extent of lymphadenectomy (β = -0.718, p < .001) and pre-trial case/procedure volume assessment (β = -0.531, p = .049) were significantly associated with reducing short-term mortality. The regression model showed standardization of the extent of lymphadenectomy (β = -3.123, p = .034) was significantly associated with reducing conversion rate. In conclusion, the heterogeneity of surgical quality measures showed that there is no clear consensus on the approach to delivering surgical quality assurance in gynecology oncology trials. The analysis in this evidence synthesis has shown a potential association between different aspects of surgical quality assurance and clinical outcomes. Further research is required to develop a framework ensuring surgical quality deliverance in gynecology oncology trials.

Quality assurance in surgical trials of arteriovenous grafts for haemodialysis: A systematic review, a narrative exploration and expert recommendations

BACKGROUND

Introducing new procedures and challenging established paradigms requires well-designed randomised controlled trials (RCT). However, RCT in surgery present unique challenges with much of treatment tailored to the individual patient circumstances, refined by experience and limited by organisational factors. There has been considerable debate over the outcomes of arteriovenous grafts (AVG) compared to AVF, but any differences may reflect differing practice and potential variability. It is essential, therefore, when considering an RCT of a novel surgical procedure or device that quality assurance (QA) is defined for both the new approach and the comparator. The aim of this systematic review was to evaluate the QA standards performed in RCT of AVG using a multi-national, multi-disciplinary approach and propose an approach for future RCT.

METHOD

The methods of this have been previously registered (PROSPERO: CRD420234284280) and published. In summary, a four-stage review was performed: identification of RCT of AVG, initial review, multidisciplinary appraisal of QA methods and reconciliation. QA measures were sought in four areas - generic, credentialing, standardisation and monitoring, with data abstracted by a multi-national, multi-speciality review body.

RESULTS

QA in RCT involving AVG in all four domains is highly variable, often sub-optimally described and has not improved over the past three decades. Few RCT established or defined a pre-RCT level of experience, none documented a pre-trial education programme, or had minimal standards of peri-operative management, no study had a defined pre-trial monitoring programme, and none assessed technical performance.

CONCLUSION

QA in RCT is a relatively new area that is expanding to ensure evidence is reliable and reproducible. This review demonstrates that QA has not previously been detailed, but can be measured in surgical RCT of vascular access, and that a four-domain approach can easily be implemented into future RCT.

The ELECTRA Trial: Approach to Contemporary Challenges in the Development and Implementation of Double-Blinded, Randomised, Controlled Clinical Trials in Low-Volume High-Complexity Surgical Oncology

Background: Achieving evidence-based practice change in surgery has always been challenging, with many aspects of common clinical practice evolving through lower-level studies that are susceptible to bias and confounding rather than high-quality evidence. This challenge is even more pronounced in the setting of low-volume, high-complexity surgical oncology. Additionally, when the costs of interventions or technologies are high, designing and developing such studies within financially constrained national healthcare systems becomes even more complicated, potentially widening perceived healthcare inequalities between private and publicly funded systems. However, this is precisely the area where a lack of evidence can either hinder the development of significant new clinical advances or lead to the adoption of expensive and ineffective treatments. Here, we describe the novel approaches adopted in the design, development, and implementation of the ELECTRA trial, a randomised, controlled, double-blinded feasibility study with a planned extension to a late-phase trial. Methods: The Cancer Research UK ELECTRA (NCT05877352) trial is a three-armed randomised, controlled clinical trial designed to evaluate the incremental benefit of adding intraoperative electron beam radiotherapy (IOERT) to pelvic exenteration surgery for locally advanced and locally recurrent rectal cancer. ELECTRA is double-blinded, with patients, surgeons, and oncologists unaware of whether IOERT is administered or not. The primary feasibility outcome focuses on the ability to successfully recruit and randomise participants, while the subsequent primary outcome assesses IOERT field local control. Results: We describe the collaborative process involved in developing the trial, including national and international consultations to determine the best study design and the most optimal outcome measures to evaluate. We outline the extensive patient participation and input into the study design. Given the complexity and evolving nature of the field, with no clear international standardisations, we outline the processes used to address internationally agreed definitions, radiological standardisation, surgical learning curves, quality assurance, and pathological standardisation, as well as the broader impact and benefits of these activities. Finally, we describe the novel design utilised to facilitate the involvement of national and international units with varying levels of equipoise regarding IOERT. Conclusions: Historically, randomised clinical trials have not been the standard approach for evaluating surgical interventions due to their practical and methodological challenges, particularly in high-complexity, low-volume settings. Despite these difficulties, they remain the gold standard for evidence-based practice. The ELECTRA trial exemplifies a complex, innovative trial design that addresses an unmet need in a specialised area of high-complexity surgery. Using ELECTRA as an example, we highlight the genuine challenges in designing such complex trials and provide recommendations to facilitate the conduct of future well-designed surgical studies.

Exploring the Potential of Electronic Patient-Reported Outcome Measures to Inform and Assess Care in Sarcoma Centers: A Longitudinal Multicenter Pilot Study

BACKGROUND

Electronic patient-reported outcome measures (ePROMs) are useful tools to assess care needs of patients diagnosed with cancer and to monitor their symptoms along the illness trajectory. Studies regarding the application of ePROMs by advanced practice nurses (APNs) specialized in sarcoma care and the use of such electronic measures for care planning and assessing quality of care are lacking.

OBJECTIVE

To explore the potential of ePROMs in clinical practice for assessing the patient's quality of life, physical functionality, needs, and fear of progression, as well as distress and the quality of care in sarcoma centers.

METHODS

A multicenter longitudinal pilot study design was chosen. Three sarcoma centers with and without APN service located in Switzerland were included. The instruments EQ-5D-5L, Pearman Mayo Survey of Needs, the National Comprehensive Cancer Network Distress Thermometer, PA-F12, and Toronto Extremity Salvage Score were used as ePROMs. Data were analyzed descriptively.

RESULTS

Overall, 55 patients participated in the pilot study; 33 (60%) received an intervention by an APN, and 22 (40%) did not. Patients in sarcoma centers with APN service reported overall higher scores in quality of life and functional outcome. The number of needs and distress level were lower in sarcoma centers with APN service. No differences were found with respect to patients' fear of progression.

CONCLUSIONS

Most of the ePROMs proved to be reasonable in clinical practice. PA-F12 has shown low clinical relevance.

IMPLICATIONS FOR PRACTICE

Using ePROMs appears to be reasonable to obtain clinically relevant patient information and to evaluate the quality of care in sarcoma centers.

Development of a reliable surgical quality assurance tool for gastrectomy in oncological trials

BACKGROUND

Despite its recognized importance, there is currently no reliable tool for surgical quality assurance (SQA) of gastrectomy in surgical oncology. The aim of this study was to develop an SQA tool for gastrectomy and to apply this tool within the ADDICT Trial in order to assess the extent and completeness of lymphadenectomy.

METHODS

The operative steps for D1+ and D2 gastrectomy have been previously described in the literature and ADDICT trial manual. Two researchers also performed fieldwork in the UK and Japan to document key operative steps through photographs and semi-structured interviews with expert surgeons. This provided the steps that were used as the framework for the SQA tool. Sixty-two photographic cases from the ADDICT Trial were rated by three independent surgeons. Generalizability (G) theory determined inter-rater reliability. D-studies examined the effect of varying the number of assessors and photographic series they rated. Chi-square assessed intra-rater reliability, comparing how the individual assessor's responses corresponded to their global rating for extent of lymphadenectomy.

RESULTS

The tool comprised 20 items, including 19 anatomical landmarks and a global rating score. Overall reliability had G-coefficient of 0.557. Internal consistency was measured with a Cronbach's alpha score of 0.869 and Chi-square confirmed intra-rater reliability for each assessor as < 0.05.

CONCLUSIONS

A photographic surgical quality assurance tool is presented for gastrectomy. Using this tool, the assessor can reliably determine not only the quality but also the extent of the lymphadenectomy performed based on remaining anatomy rather than the excised specimen.

Discordance in Total Mesorectal Excision Specimen Grading in a Prospective Phase 2 Multicenter Rectal Cancer Trial: Are We Overestimating the Quality of Our Resections?

OBJECTIVES

To report the results of a rigorous quality control (QC) process in the grading of total mesorectal excision (TME) specimens during a multicenter prospective phase 2 trial of transanal TME.

BACKGROUND

Grading of TME specimens is based on the macroscopic assessment of the mesorectum and standardized through synoptic pathology reporting. TME grade is a strong predictor of outcomes with incomplete (IC) TME associated with increased rates of local recurrence relative to complete or near complete (NC) TME. Although TME grade serves as an endpoint in most rectal cancer trials, in protocols incorporating centralized review of TME specimens for quality assurance, discordance in grading and the management thereof has not been previously described.

METHODS

A phase 2 prospective transanal TME trial was conducted from 2017 to 2022 across 11 North American centers with TME quality as the primary study endpoint. QC measures included (1) training of site pathologists in TME protocols, (2) blinded grading of de-identified TME specimen photographs by central pathologists, and (3) reconciliation of major discordance before trial reporting. Cohen Kappa statistic was used to assess agreement in grading.

RESULTS

Overall agreement in grading of 100 TME specimens between site and central reviewer was rated as fair, (κ = 0.35; 95% CI: 0.10-0.61; P < 0.0001). Concordance was noted in 54%, with minor and major discordance in 32% and 14% of cases, respectively. Upon reconciliation, 13/14 (93%) major discordances were resolved. Pre versus postreconciliation rates of complete or NC and IC TME are 77%/16% and 7% versus 69%/21% and 10%. Reconciliation resulted in a major upgrade (IC-NC; N = 1) or major downgrade (NC/C-IC, N = 4) in 5 cases overall (5%).

CONCLUSIONS

A 14% rate of major discordance was observed in TME grading between the site and central reviewers. The resolution resulted in a major change in final TME grade in 5% of cases, which suggests that reported rates or TME completeness are likely overestimated in trials. QC through a central review of TME photographs and reconciliation of major discordances is strongly recommended.

Quality assurance in surgical trials of arteriovenous grafts for haemodialysis: protocol for a systematic review

INTRODUCTION

Decisions regarding the optimal vascular access for haemodialysis patients are becoming increasingly complex, and the provision of vascular access is open to variations in systems of care as well as surgical experience and practice. Two main surgical options are recognised: arteriovenous fistula and arteriovenous graft (AVG). All recommendations regarding AVG are based on a limited number of randomised controlled trials (RCTs). It is essential that when considering an RCT of a surgical procedure, an appropriate definition of quality assurance (QA) is made for both the new approach and the comparator, otherwise replication of results or implementation into clinical practice may differ from published results. The aim of this systematic review will be to assess the methodological quality of RCT involving AVG, and the QA measures implemented in delivering interventions in these trials.

METHODS AND ANALYSIS

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines will be followed. A systematic search will be performed of the MEDLINE, Embase and Cochrane databases to identify relevant literature. Studies will be selected by title and abstract review, followed by a full-text review using inclusion and exclusion criteria. Data collected will pertain to generic measures of QA, credentialing of investigators, procedural standardisation and performance monitoring. Trial methodology will be compared against a standardised template developed by a multinational, multispecialty review body with experience in vascular access. A narrative approach will be taken to synthesise and report data.

ETHICS AND DISSEMINATION

Ethical approval is not required as it is a protocol for a systematic review. Findings will be disseminated through peer-reviewed publications and conference presentations, with the ultimate aim of providing recommendations for future RCT of AVG design.

Measurement challenges and causes of incomplete results reporting of biomedical animal studies: Results from an interview study

Background

Existing evidence indicates that a significant amount of biomedical research involving animals remains unpublished. At the same time, we lack standards for measuring the extent of results reporting in animal research. Publication rates may vary significantly depending on the level of measurement such as an entire animal study, individual experiments within a study, or the number of animals used.

Methods

Drawing on semi-structured interviews with 18 experts and qualitative content analysis, we investigated challenges and opportunities for the measurement of incomplete reporting of biomedical animal research with specific reference to the German situation. We further investigate causes of incomplete reporting.

Results

The in-depth expert interviews revealed several reasons for why incomplete reporting in animal research is difficult to measure at all levels under the current circumstances. While precise quantification based on regulatory approval documentation is feasible at the level of entire studies, measuring incomplete reporting at the more individual experiment and animal levels presents formidable challenges. Expert-interviews further identified six drivers of incomplete reporting of results in animal research. Four of these are well documented in other fields of research: a lack of incentives to report non-positive results, pressures to ‘deliver’ positive results, perceptions that some data do not add value, and commercial pressures. The fifth driver, reputational concerns, appears to be far more salient in animal research than in human clinical trials. The final driver, socio-political pressures, may be unique to the field.

Discussion

Stakeholders in animal research should collaborate to develop a clear conceptualisation of complete reporting in animal research, facilitate valid measurements of the phenomenon, and develop incentives and rewards to overcome the causes for incomplete reporting.

Textbook outcome and survival of robotic versus laparoscopic total gastrectomy for gastric cancer: a propensity score matched cohort study

Textbook outcome is a composite quality measurement of short-term outcomes for evaluating complex surgical procedures. We compared textbook outcome and survival of robotic total gastrectomy (RTG) with those of laparoscopic total gastrectomy (LTG). We retrospectively reviewed 395 patients (RTG, n = 74; LTG, n = 321) who underwent curative total gastrectomy for gastric cancer via minimally invasive approaches from 2009 to 2018. We performed propensity score matched analysis to adjust for potential selection bias. Textbook outcome included a negative resection margin, no intraoperative complication, retrieved lymph nodes > 15, no severe complication, no reintervention, no unplanned intensive care unit admission, hospitalization ≤ 21 days, no readmission after discharge, and no postoperative mortality. Survival outcomes included 3-year overall and relapse-free survival rates. After matching, 74 patients in each group were selected. Textbook outcome was similar in the RTG and LTG groups (70.3% and 75.7%, respectively), although RTG required a longer operative time. The quality metric least often achieved was the presence of severe complications in both groups (77.0% in both groups). There were no differences in the 3-year overall survival rate (98.6% and 89.7%, respectively; log-rank P = 0.144) and relapse-free survival rate between the RTG and LTG groups (97.3% and 87.0%, respectively; log-rank P = 0.167). Textbook outcome and survival outcome of RTG were similar to those of LTG for gastric cancer.