The Stroke Hyperglycemia Insulin Network Effort (SHINE) trial: an adaptive trial design case study

Application of sample size re-estimation in clinical trials: A systematic review

Background

Sample size re-estimation (SSR) is a method used to recalculate sample size during clinical trial conduct to address a lack of adequate information and can have a significant impact on study size, duration, resources, and cost. Few studies to date have summarized the conditions and circumstances under which SSR is applied. We therefore performed a systematic review of the literature related to SSR to better understand its application in clinical trial settings.

Methods

PubMed was used as the primary search source, supplemented with information from ClinicalTrials.gov where necessary details were lacking from PubMed. A systematic review was performed according to a pre-specified search strategy to identify clinical trials using SSR. Features of SSR, such as study phase and study start year, were summarized.

Results

In total, 253 publications met the pre-specified search criteria and 27 clinical trials were subsequently determined as relevant in SSR usage. Among trials where the study phase was provided, 2 (7.4%) trials were Phase I, 5 (18.5%) trials were Phase II, 11 (40.7%) trials were Phase III, and 2 (7.4%) trials were Phase IV.

Conclusion

Our results showed that SSR is also used in Phase I and II, which involve earlier decision making. We expect that SSR will continue to be used in early-phase trials where sufficient prior information may not be available. Furthermore, no major trends were observed in relation to therapy area or type of SSR, meaning that SSR may become a feasible and widely applied method in the future.

Comparison of Bayesian vs Frequentist Adaptive Trial Design in the Stroke Hyperglycemia Insulin Network Effort Trial

IMPORTANCE

Bayesian adaptive trial design has the potential to create more efficient clinical trials. However, a barrier to the uptake of bayesian adaptive designs for confirmatory trials is limited experience with how they may perform compared with a frequentist design.

OBJECTIVE

To compare the performance of a bayesian and a frequentist adaptive clinical trial design.

DESIGN, SETTING, AND PARTICIPANTS

This prospective cohort study compared 2 trial designs for a completed multicenter acute stroke trial conducted within a National Institutes of Health neurologic emergencies clinical trials network, with individual patient-level data, including the timing and order of enrollments and outcome ascertainment, from 1151 patients with acute stroke and hyperglycemia randomized to receive intensive or standard insulin therapy. The implemented frequentist design had group sequential boundaries for efficacy and futility interim analyses at 90 days after randomization for 500, 700, 900, and 1100 patients. The bayesian alternative used predictive probability of trial success to govern early termination for efficacy and futility with a first interim analysis at 500 randomized patients and subsequent interims after every 100 randomizations.

MAIN OUTCOMES AND MEASURES

The main outcome was the sample size at end of study, which was defined as the sample size at which each of the studies stopped accrual of patients.

RESULTS

Data were collected from 1151 patients. As conducted, the frequentist design passed the futility boundary after 936 participants were randomized. Using the same sequence and timing of randomization and outcome data, the bayesian alternative crossed the futility boundary approximately 3 months earlier after 800 participants were randomized.

CONCLUSIONS AND RELEVANCE

Both trial designs stopped for futility before reaching the planned maximum sample size. In both cases, the clinical community and patients would benefit from learning the answer to the trial's primary question earlier. The common feature across the 2 designs was frequent interim analyses to stop early for efficacy or for futility. Differences between how these analyses were implemented between the 2 trials resulted in the differences in early stopping.

The Bayesian Design of Adaptive Clinical Trials

This paper presents a brief overview of the recent literature on adaptive design of clinical trials from a Bayesian perspective for statistically not so sophisticated readers. Adaptive designs are attracting a keen interest in several disciplines, from a theoretical viewpoint and also—potentially—from a practical one, and Bayesian adaptive designs, in particular, have raised high expectations in clinical trials. The main conceptual tools are highlighted here, with a mention of several trial designs proposed in the literature that use these methods, including some of the registered Bayesian adaptive trials to this date. This review aims at complementing the existing ones on this topic, pointing at further interesting reading material.

Determining a Bayesian predictive power stopping rule for futility in a non-inferiority trial with binary outcomes

Background/Aims

Non-inferiority trials investigate whether a novel intervention, which typically has other benefits (i.e., cheaper or safer), has similar clinical effectiveness to currently available treatments. In situations where interim evidence in a non-inferiority trial suggests that the novel treatment is truly inferior, ethical concerns with continuing randomisation to the “inferior” intervention are raised. Thus, if interim data indicate that concluding non-inferiority at the end of the trial is unlikely, stopping for futility should be considered. To date, limited examples are available to guide the development of stopping rules for non-inferiority trials.

Methods

We used a Bayesian predictive power approach to develop a stopping rule for futility for a trial collecting binary outcomes. We evaluated the frequentist operating characteristics of the stopping rule to ensure control of the Type I and Type II error. Our case study is the Intranasal Ketamine for Procedural Sedation trial (INK trial), a non-inferiority trial designed to assess the sedative properties of ketamine administered using two alternative routes.

Results

We considered implementing our stopping rule after the INK trial enrols 140 patients out of 560. The trial would be stopped if 12 more patients experience a failure on the novel treatment compared to standard care. This trial has a type I error rate of 2.2% and a power of 80%.

Conclusions

Stopping for futility in non-inferiority trials reduces exposure to ineffective treatments and preserves resources for alternative research questions. Futility stopping rules based on Bayesian predictive power are easy to implement and align with trial aims.

Trial registration

ClinicalTrials.gov NCT02828566 July 11, 2016.

•

It is important to consider stopping for futility in non-inferiority trials.

•

We develop a rule to stop a non-inferiority trial using Bayesian predictive power.

•

We provide code and an online application to implement this method.

•

We reduce the complexity of developing stopping rules in non-inferiority trials.

Using Bayesian adaptive designs to improve phase III trials: a respiratory care example

BACKGROUND

Bayesian adaptive designs can improve the efficiency of trials, and lead to trials that can produce high quality evidence more quickly, with fewer patients and lower costs than traditional methods. The aim of this work was to determine how Bayesian adaptive designs can be constructed for phase III clinical trials in critical care, and to assess the influence that Bayesian designs would have on trial efficiency and study results.

METHODS

We re-designed the High Frequency OSCillation in Acute Respiratory distress syndrome (OSCAR) trial using Bayesian adaptive design methods, to allow for the possibility of early stopping for success or futility. We constructed several alternative designs and studied their operating characteristics via simulation. We then performed virtual re-executions by applying the Bayesian adaptive designs using the OSCAR data to demonstrate the practical applicability of the designs.

RESULTS

We constructed five alternative Bayesian adaptive designs and identified a preferred design based on the simulated operating characteristics, which had similar power to the original design but recruited fewer patients on average. The virtual re-executions showed the Bayesian sequential approach and original OSCAR trial yielded similar trial conclusions. However, using a Bayesian sequential design could have led to a reduced sample size and earlier completion of the trial.

CONCLUSIONS

Using the OSCAR trial as an example, this case study found that Bayesian adaptive designs can be constructed for phase III critical care trials. If the OSCAR trial had been run using one of the proposed Bayesian adaptive designs, it would have terminated at a smaller sample size with fewer deaths in the trial, whilst reaching the same conclusions. We recommend the wider use of Bayesian adaptive approaches in phase III clinical trials.

TRIAL REGISTRATION

OSCAR Trial registration ISRCTN, ISRCTN10416500 . Retrospectively registered 13 June 2007.

Sliding scale insulin for non-critically ill hospitalised adults with diabetes mellitus

BACKGROUND

Diabetes mellitus is a metabolic disorder resulting from a defect in insulin secretion, function, or both. Hyperglycaemia in non-critically ill hospitalised people is associated with poor clinical outcomes (infections, prolonged hospital stay, poor wound healing, higher morbidity and mortality). In the hospital setting people diagnosed with diabetes receive insulin therapy as part of their treatment in order to achieve metabolic control. However, insulin therapy can be provided by different strategies (sliding scale insulin (SSI), basal-bolus insulin, and other modalities). Sliding scale insulin is currently the most commonly used method, however there is uncertainty about which strategy provides the best patient outcomes.

OBJECTIVES

To assess the effects of SSI for non-critically ill hospitalised adults with diabetes mellitus.

SEARCH METHODS

We identified eligible trials by searching MEDLINE, Embase, LILACS, and the Cochrane Library. We searched the World Health Organization International Clinical Trials Registry Platform (WHO ICTRP) and ClinicalTrials.gov trial registers. The date of the last search for all databases was December 2017. We also examined reference lists of identified randomised controlled trials (RCTs) and systematic reviews, and contacted trial authors.

SELECTION CRITERIA

We included RCTs comparing SSI with other strategies for glycaemic control in non-critically ill hospitalised adult participants of any sex with diabetes mellitus.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data, assessed trials for risk of bias, and evaluated the overall certainty of evidence utilising the GRADE instrument. We synthesised data using a random-effects model meta-analysis with 95% prediction intervals, if possible, or descriptive analysis, as appropriate.

MAIN RESULTS

Of 720 records screened, we included eight trials that randomised 1048 participants with type 2 diabetes (387 SSI participants and 615 participants in comparator groups were available for final analysis). We included non-critically ill medical and surgical adults with the diagnosis of diabetes mellitus. The mean follow-up time was measured by the mean length of hospital stay and ranged between five and 24 days. The mean age of participants was 44.5 years to 71 years.Overall, we judged the risk of bias on the trial level as unclear for selection bias, high for outcome-related performance and detection bias with regard to hypoglycaemic episodes, other adverse events, and mean glucose levels, and low for all-cause mortality and length of hospital stay. Attrition bias was low for all outcome measures.Six trials compared SSI with a basal-bolus insulin scheme, three of which investigating 64% of all participants in this category also applying an SSI approach in the bolus comparator part. One trial had a basal insulin-only comparator arm, and the remaining trial used continuous insulin infusion as the comparator. For our main comparison of SSI versus basal-bolus insulin, the results were as follows. Four trials reported mortality data. One out of 268 participants in the SSI group (0.3%) compared with two out of 334 participants in the basal-bolus group (0.6%) died (low-certainty evidence). Severe hypoglycaemic episodes, defined as blood glucose levels below 40 mg/dL (2.2 mmol/L), showed a risk ratio (RR) of 0.22, 95% confidence interval (CI) 0.05 to 1.00; P = 0.05; 5 trials; 667 participants; very low-certainty evidence. The 95% prediction interval ranged between 0.02 and 2.57. All nine severe hypoglycaemic episodes were observed among the 369 participants on basal-bolus insulin (2.4%). The mean length of hospital stay was 0.5 days longer for the SSI group, 95% CI -0.5 to 1.4; P = 0.32; 6 trials; 717 participants; very low-certainty evidence. The 95% prediction interval ranged between -1.7 days and 2.7 days. Adverse events other than hypoglycaemic episodes, such as postoperative infections, showed a RR of 1.16, 95% CI 0.25 to 5.37; P = 0.85; 3 trials; 481 participants; very low-certainty evidence. The mean blood glucose levels ranged across basal-bolus groups from 156 mg/dL (8.7 mmol/L) to 221 mg/dL (12.3 mmol/L). The mean blood glucose level in the SSI groups was 14.8 mg/dL (0.8 mmol/L) higher (95% CI 7.8 (0.4) to 21.8 (1.2); P < 0.001; 6 trials; 717 participants; low-certainty evidence). The 95% prediction interval ranged between -3.6 mg/dL (-0.2 mmol/L) and 33.2 mg/dL (1.8 mmol/L). No trial reported on diabetes-related mortality or socioeconomic effects.

AUTHORS' CONCLUSIONS

We are uncertain which insulin strategy (SSI or basal-bolus insulin) is best for non-critically hospitalised adults with diabetes mellitus. A basal-bolus insulin strategy in these patients might result in better short-term glycaemic control but could increase the risk for severe hypoglycaemic episodes. The certainty of the body of evidence comparing SSI with basal-bolus insulin was low to very low and needs to be improved by adequately performed, well-powered RCTs in different hospital environments with well-educated medical staff using identical short-acting insulins in both intervention and comparator arms to compare the rigid SSI approach with flexible insulin application strategies.

The life cycles of six multi-center adaptive clinical trials focused on neurological emergencies developed for the Advancing Regulatory Science initiative of the National Institutes of Health and US Food and Drug Administration: Case studies from the Adaptive Designs Accelerating Promising Treatments Into Trials Project

OBJECTIVES

Clinical trials are complicated, expensive, time-consuming, and frequently do not lead to discoveries that improve the health of patients with disease. Adaptive clinical trials have emerged as a methodology to provide more flexibility in design elements to better answer scientific questions regarding whether new treatments are efficacious. Limited observational data exist that describe the complex process of designing adaptive clinical trials. To address these issues, the Adaptive Designs Accelerating Promising Treatments Into Trials project developed six, tailored, flexible, adaptive, phase-III clinical trials for neurological emergencies, and investigators prospectively monitored and observed the processes. The objective of this work is to describe the adaptive design development process, the final design, and the current status of the adaptive trial designs that were developed.

METHODS

To observe and reflect upon the trial development process, we employed a rich, mixed methods evaluation that combined quantitative data from visual analog scale to assess attitudes about adaptive trials, along with in-depth qualitative data about the development process gathered from observations.

RESULTS

The Adaptive Designs Accelerating Promising Treatments Into Trials team developed six adaptive clinical trial designs. Across the six designs, 53 attitude surveys were completed at baseline and after the trial planning process completed. Compared to baseline, the participants believed significantly more strongly that the adaptive designs would be accepted by National Institutes of Health review panels and non-researcher clinicians. In addition, after the trial planning process, the participants more strongly believed that the adaptive design would meet the scientific and medical goals of the studies.

CONCLUSION

Introducing the adaptive design at early conceptualization proved critical to successful adoption and implementation of that trial. Involving key stakeholders from several scientific domains early in the process appears to be associated with improved attitudes towards adaptive designs over the life cycle of clinical trial development.

Reducing placebo exposure in trials: Considerations from the Research Roundtable in Epilepsy

The randomized controlled trial is the unequivocal gold standard for demonstrating clinical efficacy and safety of investigational therapies. Recently there have been concerns raised about prolonged exposure to placebo and ineffective therapy during the course of an add-on regulatory trial for new antiepileptic drug approval (typically ∼6 months in duration), due to the potential risks of continued uncontrolled epilepsy for that period. The first meeting of the Research Roundtable in Epilepsy on May 19-20, 2016, focused on "Reducing placebo exposure in epilepsy clinical trials," with a goal of considering new designs for epilepsy regulatory trials that may be added to the overall development plan to make it, as a whole, safer for participants while still providing rigorous evidence of effect. This topic was motivated in part by data from a meta-analysis showing a 3- to 5-fold increased rate of sudden unexpected death in epilepsy in participants randomized to placebo or ineffective doses of new antiepileptic drugs. The meeting agenda included rationale and discussion of different trial designs, including active-control add-on trials, placebo add-on to background therapy with adjustment, time to event designs, adaptive designs, platform trials with pooled placebo control, a pharmacokinetic/pharmacodynamic approach to reducing placebo exposure, and shorter trials when drug tolerance has been ruled out. The merits and limitations of each design were discussed and are reviewed here.

A conceptual model for the development process of confirmatory adaptive clinical trials within an emergency research network

BACKGROUND

Adaptive clinical trials use accumulating data from enrolled subjects to alter trial conduct in pre-specified ways based on quantitative decision rules. In this research, we sought to characterize the perspectives of key stakeholders during the development process of confirmatory-phase adaptive clinical trials within an emergency clinical trials network and to build a model to guide future development of adaptive clinical trials.

METHODS

We used an ethnographic, qualitative approach to evaluate key stakeholders' views about the adaptive clinical trial development process. Stakeholders participated in a series of multidisciplinary meetings during the development of five adaptive clinical trials and completed a Strengths-Weaknesses-Opportunities-Threats questionnaire. In the analysis, we elucidated overarching themes across the stakeholders' responses to develop a conceptual model.

RESULTS

Four major overarching themes emerged during the analysis of stakeholders' responses to questioning: the perceived statistical complexity of adaptive clinical trials and the roles of collaboration, communication, and time during the development process. Frequent and open communication and collaboration were viewed by stakeholders as critical during the development process, as were the careful management of time and logistical issues related to the complexity of planning adaptive clinical trials.

CONCLUSION

The Adaptive Design Development Model illustrates how statistical complexity, time, communication, and collaboration are moderating factors in the adaptive design development process. The intensity and iterative nature of this process underscores the need for funding mechanisms for the development of novel trial proposals in academic settings.

Attitudes and opinions regarding confirmatory adaptive clinical trials: a mixed methods analysis from the Adaptive Designs Accelerating Promising Trials into Treatments (ADAPT-IT) project

Background

Adaptive designs have been increasingly used in the pharmaceutical and device industries, but adoption within the academic setting has been less widespread — particularly for confirmatory phase trials. We sought to understand perceptions about understanding, acceptability, and scientific validity of adaptive clinical trials (ACTs).

Methods

We used a convergent mixed methods design using survey and mini-focus group data collection procedures to elucidate attitudes and opinions among “trial community” stakeholders regarding understanding, acceptability, efficiency, scientific validity, and speed of discovery with adaptive designs. Data were collected about various aspects of ACTs using self-administered surveys (paper or Web-based) with visual analog scales (VASs) with free text responses and with mini-focus groups of key stakeholders. Participants were recruited as part of an ongoing NIH/FDA-funded research project exploring the incorporation of ACTs into an existing NIH network that focuses on confirmatory phase clinical trials in neurological emergencies. “Trial community” representatives, namely, clinical investigators, biostatisticians, NIH officials, and FDA scientists involved in the planning of four clinical trials, were eligible to participate. In addition, recent and current members of a clinical trial-oriented NIH study section were also eligible.

Results

A total of 76 stakeholders completed the survey (out of 91 who were offered it, response rate 84 %). While the VAS attitudinal data showed substantial variability across respondents about acceptability and understanding of ACTs by various constituencies, respondents perceived clinicians to be less likely to understand ACTs and that ACTs probably would increase the efficiency of discovery. Textual and focus group responses emerged into several themes that enhanced understanding of VAS attitudinal data including the following: acceptability of adaptive designs depends on constituency and situation; there is variable understanding of ACTs (limited among clinicians, perceived to be higher at FDA); views about the potential for efficiency depend on the situation and implementation. Participants also frequently mentioned a need for greater education within the academic community. Finally, the empiric, non-quantitative selection of treatments for phase III trials based on limited phase II trials was highlighted as an opportunity for improvement and a potential explanation for the high number of neutral confirmatory trials.

Conclusions

These data show considerable variations in attitudes and beliefs about ACTs among trial community representatives. For adaptive trials to be fully considered when appropriate and for the research enterprise to realize the full potential of adaptive designs will likely require extensive experience and trust building within the trial community.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1493-z) contains supplementary material, which is available to authorized users.

A post hoc evaluation of a sample size re-estimation in the Secondary Prevention of Small Subcortical Strokes study

BACKGROUND/AIMS

The use of adaptive designs has been increasing in randomized clinical trials. Sample size re-estimation is a type of adaptation in which nuisance parameters are estimated at an interim point in the trial and the sample size re-computed based on these estimates. The Secondary Prevention of Small Subcortical Strokes study was a randomized clinical trial assessing the impact of single- versus dual-antiplatelet therapy and control of systolic blood pressure to a higher (130-149 mmHg) versus lower (<130 mmHg) target on recurrent stroke risk in a two-by-two factorial design. A sample size re-estimation was performed during the Secondary Prevention of Small Subcortical Strokes study resulting in an increase from the planned sample size of 2500-3020, and we sought to determine the impact of the sample size re-estimation on the study results.

METHODS

We assessed the results of the primary efficacy and safety analyses with the full 3020 patients and compared them to the results that would have been observed had randomization ended with 2500 patients. The primary efficacy outcome considered was recurrent stroke, and the primary safety outcomes were major bleeds and death. We computed incidence rates for the efficacy and safety outcomes and used Cox proportional hazards models to examine the hazard ratios for each of the two treatment interventions (i.e. the antiplatelet and blood pressure interventions).

RESULTS

In the antiplatelet intervention, the hazard ratio was not materially modified by increasing the sample size, nor did the conclusions regarding the efficacy of mono versus dual-therapy change: there was no difference in the effect of dual- versus monotherapy on the risk of recurrent stroke hazard ratios (n = 3020 HR (95% confidence interval): 0.92 (0.72, 1.2), p = 0.48; n = 2500 HR (95% confidence interval): 1.0 (0.78, 1.3), p = 0.85). With respect to the blood pressure intervention, increasing the sample size resulted in less certainty in the results, as the hazard ratio for higher versus lower systolic blood pressure target approached, but did not achieve, statistical significance with the larger sample (n = 3020 HR (95% confidence interval): 0.81 (0.63, 1.0), p = 0.089; n = 2500 HR (95% confidence interval): 0.89 (0.68, 1.17), p = 0.40). The results from the safety analyses were similar to 3020 and 2500 patients for both study interventions. Other trial-related factors, such as contracts, finances, and study management, were impacted as well.

CONCLUSION

Adaptive designs can have benefits in randomized clinical trials, but do not always result in significant findings. The impact of adaptive designs should be measured in terms of both trial results, as well as practical issues related to trial management. More post hoc analyses of study adaptations will lead to better understanding of the balance between the benefits and the costs.