A practical guide to pre-trial simulations for Bayesian adaptive trials using SAS and BUGS

Designing a Bayesian adaptive clinical trial to evaluate novel mechanical ventilation strategies in acute respiratory failure using integrated nested Laplace approximations

BACKGROUND

Adaptive trials usually require simulations to determine values for design parameters, demonstrate error rates, and establish the sample size. We designed a Bayesian adaptive trial comparing ventilation strategies for patients with acute hypoxemic respiratory failure using simulations. The complexity of the analysis would usually require computationally expensive Markov Chain Monte Carlo methods but this barrier to simulation was overcome using the Integrated Nested Laplace Approximations (INLA) algorithm to provide fast, approximate Bayesian inference.

METHODS

We simulated two-arm Bayesian adaptive trials with equal randomization that stratified participants into two disease severity states. The analysis used a proportional odds model, fit using INLA. Trials were stopped based on pre-specified posterior probability thresholds for superiority or futility, separately for each state. We calculated the type I error and power across 64 scenarios that varied the probability thresholds and the initial minimum sample size before commencing adaptive analyses. Two designs that maintained a type I error below 5%, a power above 80%, and a feasible mean sample size were evaluated further to determine the optimal design.

RESULTS

Power generally increased as the initial sample size and the futility threshold increased. The chosen design had an initial recruitment of 500 and a superiority threshold of 0.9925, and futility threshold of 0.95. It maintained high power and was likely to reach a conclusion before exceeding a feasible sample size.

CONCLUSIONS

We designed a Bayesian adaptive trial to evaluate novel strategies for ventilation using the INLA algorithm to efficiently evaluate a wide range of designs through simulation.

The Impact of Sample Size Misestimations on the Interpretation of ARDS Trials: Systematic Review and Meta-analysis

BACKGROUND

Indeterminate randomized controlled trials (RCTs) in ARDS may arise from sample size misspecification, leading to abandonment of efficacious therapies.

RESEARCH QUESTIONS

If evidence exists for sample size misspecification in ARDS RCTs, has this led to rejection of potentially beneficial therapies? Does evidence exist for prognostic enrichment in RCTs using mortality as a primary outcome?

STUDY DESIGN AND METHODS

We identified 150 ARDS RCTs commencing recruitment after the 1994 American European Consensus Conference ARDS definition and published before October 31, 2020. We examined predicted-observed sample size, predicted-observed control event rate (CER), predicted-observed average treatment effect (ATE), and the relationship between observed CER and observed ATE for RCTs with mortality and nonmortality primary outcome measures. To quantify the strength of evidence, we used Bayesian-averaged meta-analysis, trial sequential analysis, and Bayes factors.

RESULTS

Only 84 of 150 RCTs (56.0%) reported sample size estimations. In RCTs with mortality as the primary outcome, CER was overestimated in 16 of 28 RCTs (57.1%). To achieve predicted ATE, interventions needed to prevent 40.8% of all deaths, compared with the original prediction of 29.3%. Absolute reduction in mortality ≥ 10% was observed in 5 of 28 RCTs (17.9%), but predicted in 21 of 28 RCTs (75%). For RCTs with mortality as the primary outcome, no association was found between observed CER and observed ATE (pooled OR: β = -0.04; 95% credible interval, -0.18 to 0.09). We identified three interventions that are not currently standard of care with a Bayesian-averaged effect size of > 0.20 and moderate strength of existing evidence: corticosteroids, airway pressure release ventilation, and noninvasive ventilation.

INTERPRETATION

Reporting of sample size estimations was inconsistent in ARDS RCTs, and misspecification of CER and ATE was common. Prognostic enrichment strategies in ARDS RCTs based on all-cause mortality are unlikely to be successful. Bayesian methods can be used to prioritize interventions for future effectiveness RCTs.

Systematic review and network meta-analysis comparing Chinese herbal injections with chemotherapy for treating patients with esophageal cancer

Objective

Methods

Results

Conclusions

A practical guide to pre-trial simulations for Bayesian adaptive trials using SAS and BUGS

It is often unclear what specific adaptive trial design features lead to an efficient design which is also feasible to implement. Before deciding on a particular design, it is generally advisable to carry out a simulation study to characterise the properties of candidate designs under a range of plausible assumptions. The implementation of such pre‐trial simulation studies presents many challenges and requires considerable statistical programming effort and time. Despite the scale and complexity, there is little existing literature to guide the implementation of such projects using commonly available software. This Teacher's Corner article provides a practical step‐by‐step guide to implementing such simulation studies including how to specify and fit a Bayesian model in WinBUGS or OpenBUGS using SAS, and how results from the Bayesian analysis may be pulled back into SAS and used for adaptation of allocation probabilities before simulating subsequent stages of the trial. The interface between the two software platforms is described in detail along with useful tips and tricks. A key strength of our approach is that the entire exercise can be defined and controlled from within a single SAS program.