ANALYSIS OF SPIN IN VASCULAR SURGERY RANDOMIZED CONTROLLED TRIALS WITH NONSIGNIFICANT OUTCOMES

Primary prevention of venous thromboembolism for cancer patients in randomized controlled trials: a bibliographical analysis of funding and trial characteristics

Background

The majority of randomized controlled trials (RCTs) investigating venous thromboembolism (VTE) prophylaxis in patients with cancer involve commercial sponsorship. Commercial sponsorship overcomes feasibility limitations inherent in RCTs, such as recruitment and funding, but has attracted scrutiny for its potential for bias.

Objectives

In RCTs of VTE prophylaxis in patients with cancer, how do trial characteristics compare between commercially sponsored RCTs and noncommercially sponsored RCTs?

Methods

Medline, Embase, and Cochrane Central Register of Controlled Trials were searched for RCTs that investigated at least 1 pharmacologic intervention for VTE prophylaxis in adult patients with cancer. Screening and data extraction were conducted by independent reviewers. Outcomes included trial characteristics, reporting of favorable outcomes, protocol-manuscript discrepancies, and appraisal of spin. Outcomes were compared using the independent t-test, Mann-Whitney U-test, Pearson chi-squared test, and Fisher's exact test. Logistic regression was performed to identify factors associated with possible bias.

Results

Of the 54 trials analyzed, 34 (63%) reported commercial sponsorship. Commercial sponsorship was not associated with the reporting of favorable outcomes, presence of spin, retrospective registration, or protocol-manuscript discrepancy. Spin was most prevalent in the abstract conclusions (9 out of 17 [53.3%]) and manuscript conclusions (8 out of 17 [46.7%]).Commercially sponsored trials had a higher rate of intention-to-treat analysis. Noncommercially sponsored trials were more likely to report retrospective registration of trial protocol and the use of composite primary outcomes.

Conclusion

There were few significant differences between trial characteristics, suggesting that the evidence from commercially sponsored trials investigating VTE prophylaxis in patients with cancer is unlikely to be subject to bias attributable to commercial sponsorship.

Assessing the robustness of positive vascular surgery randomized controlled trials using their fragility index

OBJECTIVE

The fragility index (FI) measures the robustness of statistically significant findings in randomized controlled trials (RCTs) by quantifying the minimum number of event conversions required to reverse a dichotomous outcome's statistical significance. In vascular surgery, many clinical guidelines and critical decision-making points are informed by a handful of key RCTs, especially regarding open surgical versus endovascular treatment. The objective of this study is to evaluate the FI of RCTs with statistically significant primary outcomes that compared open vs endovascular surgery in vascular surgery.

METHODS

In this meta-epidemiological study and systematic review, MEDLINE, Embase, and CENTRAL were searched for RCTs comparing open versus endovascular treatments for abdominal aortic aneurysms, carotid artery stenosis, and peripheral arterial disease to December 2022. RCTs with statistically significant primary outcomes were included. Data screening and extraction were conducted in duplicate. The FI was calculated by adding an event to the group with the smaller number of events while subtracting a nonevent to the same group until Fisher's exact test produced a nonstatistically significant result. The primary outcome was the FI and proportion of outcomes where the loss to follow-up was greater than the FI. The secondary outcomes assessed the relationship of the FI to disease state, presence of commercial funding, and study design.

RESULTS

Overall, 5133 articles were captured in the initial search with 21 RCTs reporting 23 different primary outcomes being included in the final analysis. The median FI (first quartile, third quartile) was 3 (3, 20) with 16 outcomes (70%) reporting a loss to follow-up greater than its FI. Mann-Whitney U test revealed that commercially funded RCTs and composite outcomes had greater FIs (median, 20.0 [5.5, 24.5] vs median, 3.0 [2.0, 5.5], P = .035; median, 21 [8, 38] vs median, 3.0 [2.0, 8.5], P = .01, respectively). The FI did not vary between disease states (P = .285) or between index and follow-up trials (P = .147). There were significant correlations between the FI and P values (Pearson r = 0.90; 95% confidence interval, 0.77-0.96), and the number of events (r = 0.82; 95% confidence interval, 0.48-0.97).

CONCLUSIONS

A small number of event conversions (median, 3) are needed to alter the statistical significance of primary outcomes in vascular surgery RCTs evaluating open surgical and endovascular treatments. Most studies had loss to follow-up greater than its FI, which can call into question trial results, and commercially funded studies had a greater FI. The FI and these findings should be considered in future trial design in vascular surgery.

Assessing the robustness of negative vascular surgery randomized controlled trials using their reverse fragility index

OBJECTIVE

The reverse fragility index (RFI) describes the number of event conversions needed to convert a statistically nonsignificant dichotomous outcome to a significant one. The objective of the present study was to assess the RFI of vascular surgery randomized controlled trials (RCTs) comparing endovascular vs open surgery for the treatment of abdominal aortic aneurysms (AAAs), carotid artery stenosis (CAS), and peripheral artery disease (PAD).

METHODS

MEDLINE and Embase were searched for RCTs that had investigated AAAs, CAS, or PAD with statistically nonsignificant binary primary outcomes. The primary outcome for the present study was the median RFI. Calculation of the RFI was performed by creating two-by-two contingency tables and subtracting events from the group with fewer events and adding nonevents to the same group until a two-tailed Fisher exact test had produced a statistically significant result (P ≤ .05).

RESULTS

Of 4187 reports, 49 studies reporting 103 different primary end points were included. The overall median RFI was 7 (interquartile range [IQR], 5-13). The specific RFIs for AAA, CAS, and PAD were 10 (IQR, 6-15.5), 6 (IQR, 5-9.5), and 7 (IQR, 5.5-10), respectively. Of the 103 end points, 42 (47%) had had a loss to follow-up greater than the RFI, of which 10 were AAA trials (24%), 23 were CAS trials (55%), and 9 were PAD trials (21%). The Pearson correlation demonstrated a significant positive relationship between a study's RFI and the impact factor of its publishing journal (r = 0.38; 95% confidence interval [CI], 0.20-0.54; P < .01), length of follow-up (r = 0.43; 95% CI, 0.26-0.58; P < .01), and sample size (r = 0.28; 95% CI, 0.09-0.45; P < .01).

CONCLUSIONS

A small number of events (median, 7) was required to change the outcome of negative RCTs from statistically nonsignificant to significant, with 47% of the studies having missing data that could have reversed the finding of its primary outcome. Reporting of the RFI relative to the loss to follow-up could be of benefit in future trials and provide confidence regarding the robustness of the P value.