A meta-epidemiological study of bias in randomized clinical trials of open and laparoscopic surgery

Empirical evidence of observer bias in randomized clinical trials: updated and expanded analysis of trials with both blinded and non-blinded outcome assessors

OBJECTIVES

To study the impact of lack of blinding of outcome assessors on estimated treatment effects of randomized clinical trials.

STUDY DESIGN AND SETTING

Meta-epidemiological study. We included randomized trials with binary or measurement scale outcomes that (1) allocated patients to subtrials with and without blinded outcome assessment, or (2) had both blinded and non-blinded assessments of the same outcome. We identified trials from previous meta-epidemiological studies and searched six databases from 2013 to 2024. We calculated a ratio of odds ratios (ROR) for each trial. A ROR < 1 indicated a more favorable effect estimate by the non-blinded assessor. We pooled RORs using random effects meta-analysis and conducted meta-regression, subgroup, and sensitivity analyses.

RESULTS

We included 66 trials (9451 patients) across 18 clinical specialties. The pooled RORs in 43 trials (7055 patients) was 0.71 (0.55-0.92). Thirty of the 43 trials assessed highly subjective outcomes. Meta-regression showed no statistically significant association between ROR and scores for outcome subjectivity (P = .53), vulnerability (P = .91), and involvement (P = .99). Heterogeneity was partly explained by a larger impact in non-drug trials, ROR 0.62 (0.46-0.84), and industry-funded trials, ROR 0.57 (0.37-0.88). Sensitivity analyses, including imputed data for 23 trials (2396 patients randomized), did not modify the observed impact importantly.

CONCLUSION

We provide empirical evidence of considerable bias in effect estimates of randomized trials with non-blinded assessors of subjective binary and measurement scale outcomes. Non-blinded assessors exaggerated effect estimates, expressed as odds ratios, by 29% (8%-45%) on average. This strongly supports blinding outcome assessors of subjective outcomes.

PLAIN LANGUAGE SUMMARY

In a randomized clinical trial, the person evaluating the results (assessor) may be either unaware of the intervention received by participants (blinded assessor) or aware of it (non-blinded assessor). Knowing which treatment a patient received can influence the assessor's evaluation of the effect; for example, if an assessor has high expectations for a new experimental intervention, they may rate a patient's improvement more favorably in the group that received the intervention compared to the group that did not. We call this observer bias. In this study, we compared the results obtained from blinded assessors to those from non-blinded assessors within the same trials to estimate the impact of observer bias in randomized trials. We found that non-blinded assessors exaggerated the experimental intervention effect by approximately 29%, on average, compared to blinded assessors. Our results indicate that when an evaluation of a patient in a randomized trial requires judgment, there is potential for substantial bias if assessors are not blinded. To ensure more reliable results, randomized clinical trials should blind assessors whenever possible.

Introducing the LATITUDES network: a library of assessment tools and training to improve transparency, utility and dissemination in evidence synthesis

An assessment of the validity of studies is an essential component of most evidence syntheses (systematic reviews) to understand the risk of bias (ROB) and applicability of the evidence. A formal validity assessment requires a structured and comprehensive approach, which can be implemented using an assessment tool, specifically developed for this purpose. Many different tools are available, marking it difficult for researchers to choose the best tool for their evidence synthesis. We have established the LATITUDES Network to assist researchers in identifying the most appropriate tool to use in their evidence synthesis and to support researchers using these tools. The LATITUDES website (www.latitudes-network.org) includes a searchable library of validity assessment tools designed for use in evidence syntheses, bringing tools together in one place and providing researchers with clear information on suitable tools, categorized by study design. The website also provides links to training on the process of validity assessment and a list of tools currently under development. To be included in the LATITUDES library, tools must meet the following criteria: be designed for use in evidence syntheses; assess multidimensional aspects of validity of individual studies or reviews; and be developed for use by the wider research community rather than for a single research group. We highlight 'key' tools, those that are considered to be the most robust and reliable tools based on prespecified criteria agreed in conjunction with our advisory board, an international group of experts in the area of evidence synthesis and ROB tools.

Compelling evidence from meta-epidemiological studies demonstrates overestimation of effects in randomized trials that fail to optimize randomization and blind patients and outcome assessors

OBJECTIVES

To investigate the impact of potential risk of bias elements on effect estimates in randomized trials.

STUDY DESIGN AND SETTING

We conducted a systematic survey of meta-epidemiological studies examining the influence of potential risk of bias elements on effect estimates in randomized trials. We included only meta-epidemiological studies that either preserved the clustering of trials within meta-analyses (compared effect estimates between trials with and without the potential risk of bias element within each meta-analysis, then combined across meta-analyses; between-trial comparisons), or preserved the clustering of substudies within trials (compared effect estimates between substudies with and without the element, then combined across trials; within-trial comparisons). Separately for studies based on between- and within-trial comparisons, we extracted ratios of odds ratios (RORs) from each study and combined them using a random-effects model. We made overall inferences and assessed certainty of evidence based on Grading of Recommendations, Assessment, development, and Evaluation and Instrument to assess the Credibility of Effect Modification Analyses.

RESULTS

Forty-one meta-epidemiological studies (34 of between-, 7 of within-trial comparisons) proved eligible. Inadequate random sequence generation (ROR 0.94, 95% confidence interval [CI] 0.90-0.97) and allocation concealment (ROR 0.92, 95% CI 0.88-0.97) probably lead to effect overestimation (moderate certainty). Lack of patients blinding probably overestimates effects for patient-reported outcomes (ROR 0.36, 95% CI 0.28-0.48; moderate certainty). Lack of blinding of outcome assessors results in effect overestimation for subjective outcomes (ROR 0.69, 95% CI 0.51-0.93; high certainty). The impact of patients or outcome assessors blinding on other outcomes, and the impact of blinding of health-care providers, data collectors, or data analysts, remain uncertain. Trials stopped early for benefit probably overestimate effects (moderate certainty). Trials with imbalanced cointerventions may overestimate effects, while trials with missing outcome data may underestimate effects (low certainty). Influence of baseline imbalance, compliance, selective reporting, and intention-to-treat analysis remain uncertain.

CONCLUSION

Failure to ensure random sequence generation or adequate allocation concealment probably results in modest overestimates of effects. Lack of patients blinding probably leads to substantial overestimates of effects for patient-reported outcomes. Lack of blinding of outcome assessors results in substantial effect overestimation for subjective outcomes. For other elements, though evidence for consistent systematic overestimate of effect remains limited, failure to implement these safeguards may still introduce important bias.

Usefulness of cranio-dorsal approach for laparoscopic left lateral sectionectomy

Most surgeons perform laparoscopic left lateral sectionectomy (Lap LLS) using the caudo-peripheral approach (C-P approach). However, recently, a cranio-dorsal approach (C-D approach) has been applied to various types of hepatectomy owing to its advantage of preventing split injury. No studies yet have compared the perioperative outcomes of Lap LLS using each approach. Therefore, this study aimed to determine whether the C-D approach is useful for Lap LLS by comparing its perioperative outcomes with the C-P approach. Data of patients who underwent Lap LLS in our institution between 2010 and 2022 for liver tumors were retrospectively collected. We compared the perioperative outcomes of Lap LLS using a conventional C-P approach, which transects hepatic parenchyma in the caudo-peripheral direction and a C-D approach, which transects hepatic parenchyma in the cranio-caudal direction. All surgeries were performed only by board-certified expert surgeons to minimize technical bias. Furthermore, the perioperative procedures employed at our institution remained unchanged throughout the study period. A total of 36 patients were included in the study (C-P approach, n = 25; C-D approach, n = 11). The C-D approach showed a significantly shorter operation time than the C-P approach (median, 225 min vs. 262 min, p = 0.04). In addition, the C-D approach showed significantly lower blood loss than the C-P approach (median, 20 mL vs. 100 mL, p < 0.01). Other parameters, such as morbidity and hospital stay, were comparable between groups. The C-D approach could offer better surgical outcomes than the conventional C-P approach.

Association of Preoperative High-Intensity Interval Training With Cardiorespiratory Fitness and Postoperative Outcomes Among Adults Undergoing Major Surgery: A Systematic Review and Meta-Analysis

Importance

Preoperative high-intensity interval training (HIIT) is associated with improved cardiorespiratory fitness (CRF) and may improve surgical outcomes.

Objective

To summarize data from studies comparing the association of preoperative HIIT vs standard hospital care with preoperative CRF and postoperative outcomes.

Data Sources

Data sources included Medline, Embase, Cochrane Central Register of Controlled Trials Library, and Scopus databases with no language constraints, including abstracts and articles published before May 2023.

Study Selection

The databases were searched for randomized clinical trials and prospective cohort studies with HIIT protocols in adult patients undergoing major surgery. Thirty-four of 589 screened studies met initial selection criteria.

Data Extraction and Synthesis

A meta-analysis was performed according to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines. Data were extracted by multiple independent observers and pooled in a random-effects model.

Main Outcomes and Measures

The primary outcome was change in CRF, as measured by either peak oxygen consumption (V̇o2 peak) or 6-Minute Walk Test (6MWT) distance. Secondary outcomes included postoperative complications; hospital length of stay (LOS); and changes in quality of life, anaerobic threshold, and peak power output.

Results

Twelve eligible studies including 832 patients were identified. Pooled results indicated several positive associations for HIIT when compared with standard care either on CRF (V̇o2 peak, 6MWT, anaerobic threshold, or peak power output) or postoperative outcomes (complications, LOS, quality of life), although there was significant heterogeneity in study results. In 8 studies including 627 patients, there was moderate-quality evidence of significant improvement in V̇o2 peak (cumulative mean difference, 2.59 mL/kg/min; 95% CI, 1.52-3.65 mL/kg/min; P < .001). In 8 studies including 770 patients, there was moderate-quality evidence of a significant reduction in complications (odds ratio, 0.44; 95% CI, 0.32-0.60; P < .001). There was no evidence that HIIT differed from standard care in hospital LOS (cumulative mean difference, -3.06 days; 95% CI, -6.41 to 0.29 days; P = .07). The analysis showed a high degree of heterogeneity in study outcomes and an overall low risk of bias.

Conclusions and Relevance

The results of this meta-analysis suggest that preoperative HIIT may be beneficial for surgical populations through the improvement of exercise capacity and reduced postoperative complications. These findings support including HIIT in prehabilitation programs before major surgery. The high degree of heterogeneity in both exercise protocols and study results supports the need for further prospective, well-designed studies.

Has network meta-analysis resolved the controversies related to bowel preparation in elective colorectal surgery?

AIM

There are discrepancies in the guidelines on preparation for colorectal surgery. While intravenous antibiotics (IV) are usually administered, the use of mechanical bowel preparation (MBP) and/or oral antibiotics (OA) is controversial. A recent network meta-analysis (NMA) demonstrated that the addition of OA reduced incisional surgical site infections (iSSIs) by more than 50%. We aimed to perform a NMA including only the highest quality randomized clinical trials (RCTs) in order to determine the ranking of different treatment strategies and assess these RCTs for methodological problems that may affect the conclusions of the NMAs.

METHOD

A NMA was performed according to PRISMA guidelines. RCTs of adult patients undergoing elective colorectal surgery with appropriate antibiotic cover and with at least 250 participants recruited, clear definition of endpoints and duration of follow-up extending beyond discharge from hospital were included. The search included Medline, Embase, Cochrane and SCOPUS databases. Primary outcomes were iSSI and anastomotic leak (AL). Statistical analysis was performed in Stata v.15.1 using frequentist routines.

RESULTS

Ten RCTs including 5107 patients were identified. Treatments compared IV (2218 patients), IV + OA (460 patients), MBP + IV (1405 patients), MBP + IV + OA (538 patients) and OA (486 patients). The likelihood of iSSI was significantly lower for IV + OA (rank 1) and MBP + IVA + OA (rank 2), reducing iSSIs by more than 50%. There were no differences between treatments for AL. Methodological issues included differences in definition, assessment and frequency of primary endpoint infections and the limited number of participants included in some treatment options.

CONCLUSION

While this NMA supports the addition of OA to IV to reduce iSSI it also highlights unanswered questions and the need for well-designed pragmatic RCTs.

Does vaginal progesterone prevent recurrent preterm birth in women with a singleton gestation and a history of spontaneous preterm birth? Evidence from a systematic review and meta-analysis

OBJECTIVE

To assess the efficacy and safety of vaginal progesterone to prevent recurrent preterm birth and adverse perinatal outcomes in singleton gestations with a history of spontaneous preterm birth.

DATA SOURCES

MEDLINE, Embase, LILACS, and CINAHL (from their inception to February 28, 2022), Cochrane databases, Google Scholar, bibliographies, and conference proceedings.

STUDY ELIGIBILITY CRITERIA

Randomized controlled trials that compared vaginal progesterone to placebo or no treatment in asymptomatic women with a singleton gestation and a history of spontaneous preterm birth.

METHODS

The primary outcomes were preterm birth <37 and <34 weeks of gestation. The secondary outcomes included adverse maternal and perinatal outcomes. Pooled relative risks with 95% confidence intervals were calculated. We assessed the risk of bias in the included studies, heterogeneity (I2 test), small-study effects, publication bias, and quality of evidence; performed subgroup and sensitivity analyses; and calculated 95% prediction intervals and adjusted relative risks.

RESULTS

Ten studies (2958 women) met the inclusion criteria: 7 with a sample size <150 (small studies) and 3 with a sample size >600 (large studies). Among the 7 small studies, 4 were at high risk of bias, 2 were at some concerns of bias, and only 1 was at low risk of bias. All the large studies were at low risk of bias. Vaginal progesterone significantly decreased the risk of preterm birth <37 weeks (relative risk, 0.64; 95% confidence interval, 0.50-0.81; I2=75%; 95% prediction interval, 0.31-1.32; very low-quality evidence) and <34 weeks (relative risk, 0.62; 95% confidence interval, 0.42-0.92; I2=66%; 95% prediction interval, 0.23-1.68; very low-quality evidence), and the risk of admission to the neonatal intensive care unit (relative risk, 0.53; 95% confidence interval, 0.33-0.85; I2=67%; 95% prediction interval, 0.16-1.79; low-quality evidence). There were no significant differences between the vaginal progesterone and the placebo or no treatment groups in other adverse perinatal and maternal outcomes. Subgroup analyses revealed that vaginal progesterone decreased the risk of preterm birth <37 weeks (relative risk, 0.43; 95% confidence interval, 0.33-0.55; I2=0%) and <34 weeks (relative risk, 0.27; 95% confidence interval, 0.15-0.49; I2=0%) in the small but not in the large studies (relative risk, 0.98; 95% confidence interval, 0.88-1.09; I2=0% for preterm birth <37 weeks; and relative risk, 0.94; 95% confidence interval, 0.78-1.13; I2=0% for preterm birth <34 weeks). Sensitivity analyses restricted to studies at low risk of bias indicated that vaginal progesterone did not reduce the risk of preterm birth <37 weeks (relative risk, 0.96; 95% confidence interval, 0.84-1.09) and <34 weeks (relative risk, 0.90; 95% confidence interval, 0.71-1.15). There was clear evidence of substantial small-study effects in the meta-analyses of preterm birth <37 and <34 weeks of gestation because of funnel plot asymmetry and the marked differences in the pooled relative risks obtained from fixed-effect and random-effects models. The adjustment for small-study effects resulted in a markedly reduced and nonsignificant effect of vaginal progesterone on preterm birth <37 weeks (relative risk, 0.86; 95% confidence interval, 0.68-1.10) and <34 weeks (relative risk, 0.92; 95% confidence interval, 0.60-1.42).

CONCLUSION

There is no convincing evidence supporting the use of vaginal progesterone to prevent recurrent preterm birth or to improve perinatal outcomes in singleton gestations with a history of spontaneous preterm birth.