Efficient group sequential designs when there are several effect sizes under consideration

Comparison of Bayesian and frequentist monitoring boundaries motivated by the Multiplatform Randomized Clinical Trial

BACKGROUND

The coronavirus disease 2019 pandemic highlighted the need to conduct efficient randomized clinical trials with interim monitoring guidelines for efficacy and futility. Several randomized coronavirus disease 2019 trials, including the Multiplatform Randomized Clinical Trial (mpRCT), used Bayesian guidelines with the belief that they would lead to quicker efficacy or futility decisions than traditional "frequentist" guidelines, such as spending functions and conditional power. We explore this belief using an intuitive interpretation of Bayesian methods as translating prior opinion about the treatment effect into imaginary prior data. These imaginary observations are then combined with actual observations from the trial to make conclusions. Using this approach, we show that the Bayesian efficacy boundary used in mpRCT is actually quite similar to the frequentist Pocock boundary.

METHODS

The mpRCT's efficacy monitoring guideline considered stopping if, given the observed data, there was greater than 99% probability that the treatment was effective (odds ratio greater than 1). The mpRCT's futility monitoring guideline considered stopping if, given the observed data, there was greater than 95% probability that the treatment was less than 20% effective (odds ratio less than 1.2). The mpRCT used a normal prior distribution that can be thought of as supplementing the actual patients' data with imaginary patients' data. We explore the effects of varying probability thresholds and the prior-to-actual patient ratio in the mpRCT and compare the resulting Bayesian efficacy monitoring guidelines to the well-known frequentist Pocock and O'Brien-Fleming efficacy guidelines. We also contrast Bayesian futility guidelines with a more traditional 20% conditional power futility guideline.

RESULTS

A Bayesian efficacy and futility monitoring boundary using a neutral, weakly informative prior distribution and a fixed probability threshold at all interim analyses is more aggressive than the commonly used O'Brien-Fleming efficacy boundary coupled with a 20% conditional power threshold for futility. The trade-off is that more aggressive boundaries tend to stop trials earlier, but incur a loss of power. Interestingly, the Bayesian efficacy boundary with 99% probability threshold is very similar to the classic Pocock efficacy boundary.

CONCLUSIONS

In a pandemic where quickly weeding out ineffective treatments and identifying effective treatments is paramount, aggressive monitoring may be preferred to conservative approaches, such as the O'Brien-Fleming boundary. This can be accomplished with either Bayesian or frequentist methods.

Point estimation, confidence intervals, and P-values for optimal adaptive two-stage designs with normal endpoints

Due to the dependency structure in the sampling process, adaptive trial designs create challenges in point and interval estimation and in the calculation of P-values. Optimal adaptive designs, which are designs where the parameters governing the adaptivity are chosen to maximize some performance criterion, suffer from the same problem. Various analysis methods which are able to handle this dependency structure have already been developed. In this work, we aim to give a comprehensive summary of these methods and show how they can be applied to the class of designs with planned adaptivity, of which optimal adaptive designs are an important member. The defining feature of these kinds of designs is that the adaptive elements are completely prespecified. This allows for explicit descriptions of the calculations involved, which makes it possible to evaluate different methods in a fast and accurate manner. We will explain how to do so, and present an extensive comparison of the performance characteristics of various estimators between an optimal adaptive design and its group-sequential counterpart.

Optimization of the two-stage group sequential three-arm gold-standard design for non-inferiority trials

If design parameters are chosen appropriately, group sequential trial designs are known to be able to reduce the expected sample size under the alternative hypothesis compared to single-stage designs. The same holds true for the so-called 'gold-standard' design for non-inferiority trials, a design involving an experimental group, an active control group, and a placebo group. However, choosing design parameters that maximize the advantages of a two-stage approach for the three-arm gold-standard design for non-inferiority trials is not a straightforward task. In particular, optimal choices of futility boundaries for this design have not been thoroughly discussed in existing literature. We present a variation of the hierarchical testing procedure, which allows for the incorporation of binding futility boundaries at interim analyses. We show that this procedure maintains strong control of the family-wise type I error rate. Within this framework, we consider the futility and efficacy boundaries as well as the sample size allocation ratios as optimization parameters. This allows the investigation of the efficiency gain from including the option to stop for futility in addition to the ability to stop for efficacy. To analyze the extended designs, optimality criteria that include the design's performance under the alternative as well as the null hypothesis are introduced. On top of this, we discuss methods to limit the allocation of placebo patients in the trial while maintaining relatively good operating characteristics. The results of our numerical optimization procedure are discussed and a comparison of different approaches to designing a three-arm gold-standard non-inferiority trial is provided.

Improving sample size recalculation in adaptive clinical trials by resampling

Sample size calculations in clinical trials need to be based on profound parameter assumptions. Wrong parameter choices may lead to too small or too high sample sizes and can have severe ethical and economical consequences. Adaptive group sequential study designs are one solution to deal with planning uncertainties. Here, the sample size can be updated during an ongoing trial based on the observed interim effect. However, the observed interim effect is a random variable and thus does not necessarily correspond to the true effect. One way of dealing with the uncertainty related to this random variable is to include resampling elements in the recalculation strategy. In this paper, we focus on clinical trials with a normally distributed endpoint. We consider resampling of the observed interim test statistic and apply this principle to several established sample size recalculation approaches. The resulting recalculation rules are smoother than the original ones and thus the variability in sample size is lower. In particular, we found that some resampling approaches mimic a group sequential design. In general, incorporating resampling of the interim test statistic in existing sample size recalculation rules results in a substantial performance improvement with respect to a recently published conditional performance score.

Optimal planning of adaptive two-stage designs

Adaptive designs are playing an increasingly important role in the planning of clinical trials. While there exists various research on the optimal determination of a two-stage design, non-optimal versions still are frequently applied in clinical research. In this article, we strive to motivate the application of optimal adaptive designs and give guidance on how to determine them. It is demonstrated that optimizing a trial design with respect to particular objective criteria can have a substantial benefit over the application of conventional adaptive sample size recalculation rules. Furthermore, we show that in many practical situations, optimal group-sequential designs show an almost negligible performance loss compared to optimal adaptive designs. Finally, we illustrate how optimal designs can be tailored to specific operational requirements by customizing the underlying optimization problem.

Follow up after sample size re-estimation in a breast cancer randomized trial for disease-free survival

BACKGROUND

While the clinical trials and statistical methodology literature on sample size re-estimation (SSRE) is robust, evaluation of SSRE procedures following the completion of a clinical trial has been sparsely reported. In blinded sample size re-estimation, only nuisance parameters are re-estimated, and the blinding of the current trial treatment effect is preserved. Blinded re-estimation procedures are well-accepted by regulatory agencies and funders. We review our experience of sample size re-estimation in a large international, National Institutes of Health funded clinical trial for adjuvant breast cancer treatment, and evaluate our blinded sample size re-estimation procedure for this time-to-event trial. We evaluated the SSRE procedure by examining assumptions made during the re-estimation process, estimates resulting from re-estimation, and the impact on final trial results with and without the addition of participants, following sample size re-estimation.

METHODS

We compared the control group failure probabilities estimated at the time of SSRE to estimates used in the original planning, to the final un-blinded control group failure probability estimates for those included in the SSRE procedure (SSRE cohort), and to the final total control group failure probability estimates. The impact of re-estimation on the final comparison between randomized treatment groups is evaluated for those in the originally planned cohort (n = 340) and for the combination of those recruited in the originally planned cohort and those added after re-estimation (n = 509).

RESULTS

Very little difference is observed between the originally planned cohort and all randomized patients in the control group failure probabilities over time or in the overall hazard ratio estimating treatment effect (originally planned cohort HR 1.25 (0.86, 1.79); all randomized cohort HR 1.24 95% CI (0.91, 1.68)). At the time of blinded SSRE, the estimated control group failure probabilities at 3 years (0.24) and 5 years (0.40) were similar to those for the SSRE cohort once un-blinded (3 years, 0.22 (0.16, 0.30); 5 years, 0.33 (0.26, 0.41)).

CONCLUSIONS

We found that our re-estimation procedure performed reasonably well in estimating the control group failure probabilities at the time of re-estimation. Particularly for time-to-event outcomes, pre-planned blinded SSRE procedures may be the best option to aid in maintaining power.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT00201851 . Registered on 9 September 2005. Retrospectively registered.

An evaluation of increasing sample size based on conditional power

We evaluate properties of sample size re-estimation (SSR) designs similar to the promising zone design considered by Mehta and Pocock (2011). We evaluate these designs under the assumption of a true effect size of 1.1 down to 0.4 of the protocol-specified effect size by six measures: 1. The probability of a sample size increase, 2. The mean proportional increase in sample size given an increase; 3 and 4. The mean true conditional power with and without a sample size increase; 5 and 6. The expected increase in sample size and power due to the SSR procedure. These measures show the probability of a sample size increase and the cost/benefit for given true effect sizes, particularly when the SSR may either be pursuing a small effect size of little clinical importance or be unnecessary when the true effect size is close to the protocol-specified effect size. The results show the clear superiority of conducting the SSR late in the study and the inefficiency of a mid-study SSR. The results indicate that waiting until late in the study for the SSR yields a smaller, better targeted set of studies with a greater increase in overall power than a mid-study SSR.

Trial Sequential Analysis in systematic reviews with meta-analysis

BACKGROUND

Most meta-analyses in systematic reviews, including Cochrane ones, do not have sufficient statistical power to detect or refute even large intervention effects. This is why a meta-analysis ought to be regarded as an interim analysis on its way towards a required information size. The results of the meta-analyses should relate the total number of randomised participants to the estimated required meta-analytic information size accounting for statistical diversity. When the number of participants and the corresponding number of trials in a meta-analysis are insufficient, the use of the traditional 95% confidence interval or the 5% statistical significance threshold will lead to too many false positive conclusions (type I errors) and too many false negative conclusions (type II errors).

METHODS

We developed a methodology for interpreting meta-analysis results, using generally accepted, valid evidence on how to adjust thresholds for significance in randomised clinical trials when the required sample size has not been reached.

RESULTS

The Lan-DeMets trial sequential monitoring boundaries in Trial Sequential Analysis offer adjusted confidence intervals and restricted thresholds for statistical significance when the diversity-adjusted required information size and the corresponding number of required trials for the meta-analysis have not been reached. Trial Sequential Analysis provides a frequentistic approach to control both type I and type II errors. We define the required information size and the corresponding number of required trials in a meta-analysis and the diversity (D²) measure of heterogeneity. We explain the reasons for using Trial Sequential Analysis of meta-analysis when the actual information size fails to reach the required information size. We present examples drawn from traditional meta-analyses using unadjusted naïve 95% confidence intervals and 5% thresholds for statistical significance. Spurious conclusions in systematic reviews with traditional meta-analyses can be reduced using Trial Sequential Analysis. Several empirical studies have demonstrated that the Trial Sequential Analysis provides better control of type I errors and of type II errors than the traditional naïve meta-analysis.

CONCLUSIONS

Trial Sequential Analysis represents analysis of meta-analytic data, with transparent assumptions, and better control of type I and type II errors than the traditional meta-analysis using naïve unadjusted confidence intervals.

Adaptive designs from a Data Safety Monitoring Board perspective: Some controversies and some case studies

This article describes vignettes concerning interactions with Data Safety Monitoring Boards during the design and monitoring of some clinical trials with an adaptive design. Most reflect personal experiences by the author.

Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

Adaptive sample size modification in clinical trials: start small then ask for more?

We consider sample size re-estimation in a clinical trial, in particular when there is a significant delay before the measurement of patient response. Mehta and Pocock have proposed methods in which sample size is increased when interim results fall in a 'promising zone' where it is deemed worthwhile to increase conditional power by adding more subjects. Our analysis reveals potential pitfalls in applying this approach. Mehta and Pocock use results of Chen, DeMets and Lan to identify when increasing sample size, but applying a conventional level α significance test at the end of the trial does not inflate the type I error rate: we have found the greatest gains in power per additional observation are liable to lie outside the region defined by this method. Mehta and Pocock increase sample size to achieve a particular conditional power, calculated under the current estimate of treatment effect: this leads to high increases in sample size for a small range of interim outcomes, whereas we have found it more efficient to make moderate increases in sample size over a wider range of cases. If the aforementioned pitfalls are avoided, we believe the broad framework proposed by Mehta and Pocock is valuable for clinical trial design. Working in this framework, we propose sample size rules that apply explicitly the principle of adding observations when they are most beneficial. The resulting trial designs are closely related to efficient group sequential tests for a delayed response proposed by Hampson and Jennison.

Adaptive design of confirmatory trials: Advances and challenges

The past decade witnessed major developments in innovative designs of confirmatory clinical trials, and adaptive designs represent the most active area of these developments. We give an overview of the developments and associated statistical methods in several classes of adaptive designs of confirmatory trials. We also discuss their statistical difficulties and implementation challenges, and show how these problems are connected to other branches of mainstream Statistics, which we then apply to resolve the difficulties and bypass the bottlenecks in the development of adaptive designs for the next decade.

Effect of sample size re-estimation in adaptive clinical trials for Alzheimer's disease and mild cognitive impairment

Introduction

The sample size re-estimation (SSR) adaptive design allows interim analyses and resultant modifications of the ongoing trial to preserve or increase power. We investigated the applicability of SSR in Alzheimer's disease (AD) trials using a meta-database of clinical studies.

Methods

Based on six studies, we simulated clinical trials using Alzheimer's Disease Assessment Scale-cognitive subscale (ADAS-Cog) as primary outcome. A single SSR based on effect sizes or based on variances was conducted at 6 months and 12 months. Resultant power improvement and sample size adjustments were evaluated.

Results

SSR resulted in highly variable outcomes for both sample size increases and power improvement. The gain in power after SSR varies by initial sample sizes, trial durations, and effect sizes.

Conclusions

SSR adaptive designs can be effective for trials in AD and mild cognitive impairment with small or medium initial sample sizes. However, SSR in larger trials (>200 subjects per arm) generates no major advantages over the typical randomized trials.

Adaptive designs for comparative effectiveness research trials

CONTEXT

Medical and health policy decision makers require improved design and analysis methods for comparative effectiveness research (CER) trials. In CER trials, there may be limited information to guide initial design choices. In general settings, adaptive designs (ADs) have effectively overcome limits on initial information. However, CER trials have fundamental differences from standard clinical trials including population heterogeneity and a vaguer concept of a "minimum clinically meaningful difference".

OBJECTIVE

To explore the use of a particular form of ADs for comparing treatments within the CER trial context.

METHODS

We review the current state of clinical CER, identify areas of CER as particularly strong candidates for application of novel ADs, and illustrate potential usefulness of the designs and methods for two group comparisons.

RESULTS

ADs can stabilize power. The designs ensure adequate power for true effects are at least at clinically significant preplanned effect size, or when variability is larger than expected. The designs allow for sample size savings when the true effect is larger or when variability is smaller than planned.

CONCLUSION

ADs in CER have great potential to allow trials to successfully and efficiently make important comparisons.

Optimal timing for interim analyses in clinical trials

In clinical trials, interim analyses are often performed before the completion of the trial. The intention is to possibly terminate the trial early or adjust the sample size. The time of conducting an interim analysis affects the probability of the early termination and the number of subjects enrolled until the interim analysis. This influences the expected total number of subjects. In this study, we examine the optimal time for conducting interim analyses with a view to minimizing the expected total sample size. It is found that regardless of the effect size, the optimal time of one interim analysis for the early termination is approximately two-thirds of the planned observations for the O'Brien-Fleming type of spending function and approximately half of the planned observations for the Pocock type when the subject enrollment is halted for the interim analysis. When the subject enrollment is continuous throughout the trial, the optimal time for the interim analysis varies according to the follow-up duration. We also consider the time for one interim analysis including the sample size adjustment in terms of minimizing the expected total sample size.

The thresholds for statistical and clinical significance - a five-step procedure for evaluation of intervention effects in randomised clinical trials

BACKGROUND

Thresholds for statistical significance are insufficiently demonstrated by 95% confidence intervals or P-values when assessing results from randomised clinical trials. First, a P-value only shows the probability of getting a result assuming that the null hypothesis is true and does not reflect the probability of getting a result assuming an alternative hypothesis to the null hypothesis is true. Second, a confidence interval or a P-value showing significance may be caused by multiplicity. Third, statistical significance does not necessarily result in clinical significance. Therefore, assessment of intervention effects in randomised clinical trials deserves more rigour in order to become more valid.

METHODS

Several methodologies for assessing the statistical and clinical significance of intervention effects in randomised clinical trials were considered. Balancing simplicity and comprehensiveness, a simple five-step procedure was developed.

RESULTS

For a more valid assessment of results from a randomised clinical trial we propose the following five-steps: (1) report the confidence intervals and the exact P-values; (2) report Bayes factor for the primary outcome, being the ratio of the probability that a given trial result is compatible with a 'null' effect (corresponding to the P-value) divided by the probability that the trial result is compatible with the intervention effect hypothesised in the sample size calculation; (3) adjust the confidence intervals and the statistical significance threshold if the trial is stopped early or if interim analyses have been conducted; (4) adjust the confidence intervals and the P-values for multiplicity due to number of outcome comparisons; and (5) assess clinical significance of the trial results.

CONCLUSIONS

If the proposed five-step procedure is followed, this may increase the validity of assessments of intervention effects in randomised clinical trials.

Adaptive clinical trial designs with pre-specified rules for modifying the sample size: understanding efficient types of adaptation

Adaptive clinical trial design has been proposed as a promising new approach that may improve the drug discovery process. Proponents of adaptive sample size re-estimation promote its ability to avoid 'up-front' commitment of resources, better address the complicated decisions faced by data monitoring committees, and minimize accrual to studies having delayed ascertainment of outcomes. We investigate aspects of adaptation rules, such as timing of the adaptation analysis and magnitude of sample size adjustment, that lead to greater or lesser statistical efficiency. Owing in part to the recent Food and Drug Administration guidance that promotes the use of pre-specified sampling plans, we evaluate alternative approaches in the context of well-defined, pre-specified adaptation. We quantify the relative costs and benefits of fixed sample, group sequential, and pre-specified adaptive designs with respect to standard operating characteristics such as type I error, maximal sample size, power, and expected sample size under a range of alternatives. Our results build on others' prior research by demonstrating in realistic settings that simple and easily implemented pre-specified adaptive designs provide only very small efficiency gains over group sequential designs with the same number of analyses. In addition, we describe optimal rules for modifying the sample size, providing efficient adaptation boundaries on a variety of scales for the interim test statistic for adaptation analyses occurring at several different stages of the trial. We thus provide insight into what are good and bad choices of adaptive sampling plans when the added flexibility of adaptive designs is desired.

Adaptive trial designs: a review of barriers and opportunities

Adaptive designs allow planned modifications based on data accumulating within a study. The promise of greater flexibility and efficiency stimulates increasing interest in adaptive designs from clinical, academic, and regulatory parties. When adaptive designs are used properly, efficiencies can include a smaller sample size, a more efficient treatment development process, and an increased chance of correctly answering the clinical question of interest. However, improper adaptations can lead to biased studies. A broad definition of adaptive designs allows for countless variations, which creates confusion as to the statistical validity and practical feasibility of many designs. Determining properties of a particular adaptive design requires careful consideration of the scientific context and statistical assumptions. We first review several adaptive designs that garner the most current interest. We focus on the design principles and research issues that lead to particular designs being appealing or unappealing in particular applications. We separately discuss exploratory and confirmatory stage designs in order to account for the differences in regulatory concerns. We include adaptive seamless designs, which combine stages in a unified approach. We also highlight a number of applied areas, such as comparative effectiveness research, that would benefit from the use of adaptive designs. Finally, we describe a number of current barriers and provide initial suggestions for overcoming them in order to promote wider use of appropriate adaptive designs. Given the breadth of the coverage all mathematical and most implementation details are omitted for the sake of brevity. However, the interested reader will find that we provide current references to focused reviews and original theoretical sources which lead to details of the current state of the art in theory and practice.

Sample size re-estimation in an on-going NIH-sponsored clinical trial: the secondary prevention of small subcortical strokes experience

BACKGROUND AND PURPOSE

When planning clinical trials, decisions regarding sample size are often based on educated guesses of parameters, which may in fact prove to be over- or under-estimates. For example, after initiation of the SPS3 study, published data indicated that the recurrent stroke rates might be lower than initially planned for the study. Failure to account for this could result in an under-powered study. Thus, we performed a sample size re-estimation, and describe the experience herein.

METHODS

We evaluated different scenarios based on a re-estimated overall event rate, including increasing the sample size and increasing the follow-up time, to determine their impact on both type I error and the power to detect the initially planned treatment difference.

RESULTS

We found that by increasing the sample size from 2500 to 3000 and by following the patients for one year after the end of recruitment, we would maintain our planned type I error rate, and increase the power to detect the prespecified clinically meaningful difference to between 67% and 87%, depending on the rate of recruitment.

CONCLUSIONS

We successfully implemented this unplanned design modification in the SPS3 study, in order to allow for sufficient power to detect the planned treatment differences.

CLINICAL TRIALS REGISTRATION INFORMATION

Clinical Trials Registration - http://clinicaltrials.gov/show/NCT00059306. Unique identifier: NCT00059306.

Sequential design of phase II-III cancer trials

Although traditional phase II cancer trials are usually single arm, with tumor response as endpoint, and phase III trials are randomized and incorporate interim analyses with progression-free survival or other failure time as endpoint, this paper proposes a new approach that seamlessly expands a randomized phase II study of response rate into a randomized phase III study of time to failure. This approach is based on advances in group sequential designs and joint modeling of the response rate and time to event. The joint modeling is reflected in the primary and secondary objectives of the trial, and the sequential design allows the trial to adapt to increase in information on response and survival patterns during the course of the trial and to stop early either for conclusive evidence on efficacy of the experimental treatment or for the futility in continuing the trial to demonstrate it, on the basis of the data collected so far.

Adaptive trial designs

We review adaptive designs for clinical trials, giving special attention to the control of the Type I error in late-phase confirmatory trials, when the trial planner wishes to adjust the final sample size of the study in response to an unblinded analysis of interim estimates of treatment effects. We point out that there is considerable inefficiency in using the adaptive designs that employ conditional power calculations to reestimate the sample size and that maintain the Type I error by using certain weighted test statistics. Although these adaptive designs have little advantage over familiar group-sequential designs, our review also describes recent developments in adaptive designs that are both flexible and efficient. We also discuss the use of Bayesian designs, when the context of use demands control over operating characteristics (Type I and II errors) and correction of the bias of estimated treatment effects.

Review of draft FDA adaptive design guidance

Recent draft Food and Drug Administration (FDA) guidelines on adaptive clinical trial design and conduct provide a useful background for both academic and industry clinical research leaders. The guidelines help to define the scope of the term "adaptive design" and clarify how adaptive designs can be a critical component of a development program. Adaptive designs are appealing because they hold the promise of conducting trials that can answer the scientific questions of interest with savings of time and resources while exposing fewer subjects to potentially risky therapies. Adaptive designs have been successfully used in clinical trials in all phases of development for many years, enabled by a variety of increasingly flexible and powerful statistical tools. Recent developments that enable adaptations based on emerging treatment differences have demonstrated potential to streamline the research enterprise, but issues remain in their implementation. Proper implementation of adaptive designs requires an adequate understanding of the inherent trade-offs that accompany their use. In exchange for potential efficiencies in resource utilization, adaptive trials suffer from limitations in scientific conclusions, complications and inefficiencies in the statistical analysis, and logistical difficulties relative to fixed sample or fixed duration trials. While scarce resources and ethical imperatives motivate serious consideration of adaptive designs, researchers should be fully aware of the advantages and disadvantages of adaptive designs and adopt them cautiously.

Internal pilots for observational studies

Study planning often involves selecting an appropriate sample size. Power calculations require specifying an effect size and estimating "nuisance" parameters, e.g. the overall incidence of the outcome. For observational studies, an additional source of randomness must be estimated: the rate of the exposure. A poor estimate of any of these parameters will produce an erroneous sample size. Internal pilot (IP) designs reduce the risk of this error - leading to better resource utilization - by using revised estimates of the nuisance parameters at an interim stage to adjust the final sample size. In the clinical trials setting, where allocation to treatment groups is pre-determined, IP designs have been shown to achieve the targeted power without introducing substantial inflation of the type I error rate. It has not been demonstrated whether the same general conclusions hold in observational studies, where exposure-group membership cannot be controlled by the investigator. We extend the IP to observational settings. We demonstrate through simulations that implementing an IP, in which prevalence of the exposure can be re-estimated at an interim stage, helps ensure optimal power for observational research with little inflation of the type I error associated with the final data analysis.

Optimal group-sequential designs for simultaneous testing of superiority and non-inferiority

Confirmatory clinical trials comparing the efficacy of a new treatment with an active control typically aim at demonstrating either superiority or non-inferiority. In the latter case, the objective is to show that the experimental treatment is not worse than the active control by more than a pre-specified non-inferiority margin. We consider two classes of group-sequential designs that combine the superiority and non-inferiority objectives: non-adaptive designs with fixed group sizes and adaptive designs where future group sizes may be based on the observed treatment effect. For both classes, we derive group-sequential designs meeting error probability constraints that have the lowest possible expected sample size averaged over a set of values of the treatment effect. These optimized designs provide an efficient means of reducing expected sample size under a range of treatment effects, even when the separate objectives of proving superiority and non-inferiority would require quite different fixed sample sizes. We also present error spending versions of group-sequential designs that are easily implementable and can handle unpredictable group sizes or information levels. We find the adaptive choice of group sizes to yield some modest efficiency gains; alternatively, expected sample size may be reduced by adding another interim analysis to a non-adaptive group-sequential design.

Adaptive clinical trials: progress and challenges

Adaptive designs promise the flexibility to redesign clinical trials at interim stages. This flexibility would provide greater efficiency in drug development. However, despite this promise, many hesitate to implement such designs. Here we explore three possible reasons for the hesitation: (i) confusion with respect to the definition of an 'adaptive design'; (ii) controversy surrounding the use of sample size re-estimation methods; and (iii) logistical barriers that must be overcome in order to use adaptive designs within existing trial frameworks.The large volume of recent work has created confusion with respect to the definition of an 'adaptive design'. Unfortunately, this has resulted in reduced usage of many acceptable methods because of guilt by association with the more controversial methods. This review attempts to clarify the differences among many common types of proposed adaptive designs. Once the differences are noted, it becomes apparent that some adaptive designs are well accepted while others remain very controversial. In fact, much of the controversy and criticism surrounding adaptive designs has focused on their use for sample size re-estimation. Hence, this review also examines the different types of adaptive designs for sample size re-estimation in order to clarify the controversy surrounding the use of these methods. Specifically, separating the controversial from good practice requires clarifying differences between adaptive designs with sample size re-estimation based on a revised treatment effect and re-estimation based only on nuisance parameters (internal pilot designs). Finally, many logistical barriers must be overcome in order to use adaptive designs within existing trial frameworks.If the promise of adaptive designs is to be achieved, it will be important to bring together large groups of individuals from funding sources and regulatory agencies to address these limitations. Very few discussions of these issues have appeared in journals that are targeted to clinical audiences. In fact, current use of adaptive designs is not really hindered by the lack of statistical methods to accommodate the adaptations. Rather, there is a need for education as to which adaptive designs are acceptable and which are not acceptable. These discussions will require the involvement of many individuals outside the statistical community. In this review, we summarize the existing methods and current controversies with the intent of providing a clarification that will enable these individuals to participate in these much-needed discussions.

Group sequential and adaptive designs - a review of basic concepts and points of discussion

In recent times, group sequential and adaptive designs for clinical trials have attracted great attention from industry, academia and regulatory authorities. These designs allow analyses on accumulating data - as opposed to classical, "fixed-sample" statistics. The rapid development of a great variety of statistical procedures is accompanied by a lively debate on their potential merits and shortcomings. The purpose of this review article is to ease orientation in both respects. First, we provide a concise overview of the essential technical concepts, with special emphasis on their interrelationships. Second, we give a structured review of the current controversial discussion on practical issues, opportunities and challenges of these new designs.

Efficient adaptive designs with mid-course sample size adjustment in clinical trials

Adaptive designs have been proposed for clinical trials in which the nuisance parameters or alternative of interest are unknown or likely to be misspecified before the trial. Although most previous works on adaptive designs and mid-course sample size re-estimation have focused on two-stage or group-sequential designs in the normal case, we consider here a new approach that involves at most three stages and is developed in the general framework of multiparameter exponential families. This approach not only maintains the prescribed type I error probability but also provides a simple but asymptotically efficient sequential test whose finite-sample performance, measured in terms of the expected sample size and power functions, is shown to be comparable to the optimal sequential design, determined by dynamic programming, in the simplified normal mean case with known variance and prespecified alternative, and superior to the existing two-stage designs and also to adaptive group-sequential designs when the alternative or nuisance parameters are unknown or misspecified.