Estimation of offspring production from a limited number of stage-structured censuses

Unbiased estimation for response adaptive clinical trials

Bayesian adaptive trials have the defining feature that the probability of randomization to a particular treatment arm can change as information becomes available as to its true worth. However, there is still a general reluctance to implement such designs in many clinical settings. One area of concern is that their frequentist operating characteristics are poor or, at least, poorly understood. We investigate the bias induced in the maximum likelihood estimate of a response probability parameter, p, for binary outcome by the process of adaptive randomization. We discover that it is small in magnitude and, under mild assumptions, can only be negative - causing one's estimate to be closer to zero on average than the truth. A simple unbiased estimator for p is obtained, but it is shown to have a large mean squared error. Two approaches are therefore explored to improve its precision based on inverse probability weighting and Rao-Blackwellization. We illustrate these estimation strategies using two well-known designs from the literature.

Estimation of treatment effect following a clinical trial with adaptive design

Parameter estimation following an adaptive design or group sequential design has been extremely challenging due to potential random high from its face value estimate. In this paper, we introduce a new framework to model clinical trial data flow based on a marked point process (MPP). The MPP model allows us to use methods of stochastic calculus for analyses of any adaptive clinical trial. As an example, we apply this method to a two stage treatment selection design and derive a procedure to estimate the treatment effect. Numerical examples will be used to evaluate the performance of the proposed procedure.

Adaptive designs: looking for a needle in the haystack-a new challenge in medical research

The statistical principles of fully adaptive designs are outlined. The options of flexibility and the price to be paid in terms of statistical properties of the test procedures are discussed. It is stressed that controlled inference after major design modifications (changing hypotheses) will include a penalty: Intersections among all the hypotheses considered throughout the trial have to be rejected before testing individual hypotheses. Moreover, feasibility in terms of integrity and persuasiveness of the results achieved after adaptations based on unblinded data is considered as the crucial issue in practice. In the second part, sample size adaptive procedures are considered testing a large number of hypotheses under constraints on total sample size as in genetic studies. The advantage of sequential procedures is sketched for the example of two-stage designs with a pilot phase for screening promising hypotheses (markers) and controlling the false discovery rate. Finally, we turn to the clinical problem how to select markers and estimate a score from limited samples, e.g. for predicting the response to therapy of a future patient. The predictive ability of such scores will be rather poor when investigating a large number of hypotheses and truly large marker effects are lacking. An obvious dilemma will show up: More optimistic selection rules may be superior if in fact effective markers exist, but will produce more nuisance prediction if no effective markers exist compared with more cautious strategies, e.g. aiming at some control of type I error probabilities.

Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: general concepts

Traditional drug development consists of a sequence of independent trials organized in different phases. Full development typically involves (i) a learning phase II trial and (ii) one or two confirmatory phase III trial(s). For example, in the phase II trials several doses of the new compound might be compared to a control and/or placebo with the goal of deciding whether to stop or continue development and, in the latter case, selecting one or two "best" doses to carry forward into the confirmatory phase. The phase III trials are then conducted as stand-alone confirmatory studies, not incorporating in their statistical analyses data collected in the previous phases. Seamless phase II/III designs are aimed at interweaving the two phases of full development by combining them into one single, uninterrupted study conducted in two stages. In the dose-finding example above, one (or more) dose(s) are selected after the first stage based on the available data at interim, and are then observed further in the second stage. The final analysis of the selected dose(s) includes patients from both stages and is performed such that the overall type I error rate is controlled at a prespecified level regardless of the dose selection rule used at interim. The adequacy of the dose selection at interim is obviously a critical step for the success of a seamless phase II/III trial. In this paper we focus on the description of flexible test procedures allowing for adaptively selecting hypotheses at interim and thus allowing the combination of learning and confirming in a single seamless trial. We review the statistical background, introduce different test procedures and compare them in a power study. In a subsequent paper (Schmidli et al., 2006) we give several applications from our daily practice and discuss related implementation issues in conducting adaptive seamless designs.

A regulatory view on adaptive/flexible clinical trial design

Recently there is growing interest in use of adaptive or flexible designs for development of pharmaceutical products. Statistical methodology has been greatly advanced in the literature. However, there are still some important issues with the methodology and application. In addition, there are many other challenges with these designs, including efficiency of these designs in the entire development program, trial conduct and logistics, the infrastructure of an adaptive trial, the regulatory evaluation of trial results and trial conduct, etc. Up till now, regulatory experience in these designs is very limited. We share some of the challenges.

Application of Adaptive Designs – a Review

A literature search has been performed to review applications of the adaptive design methodology based on the combination test or conditional error function approach. Some features of the 60 papers identified are summarized, e.g., the specific methodology used, calendar year, country, impact factor of the journal, number of planned and performed stages respectively, stopping for futility boundaries, type of adaptations and others. A selection of the ten recent publications in journals with the highest impact factors is discussed in more detail. Most applications up to now aim at sample size reassessment, the majority of papers is coming from Germany. Although we found that renowned journals allow for sufficient space to present the new statistical methodology in all its necessary details, the general impression is that the presentation of the adaptive designs methodology in applied papers has to be improved. Education and development of standards could help to achieve this.

Estimation in flexible two stage designs

Adaptive test designs for clinical trials allow for a wide range of data driven design adaptations using all information gathered until an interim analysis. The basic principle is to use a test statistics which is invariant with respect to the design adaptations under the null hypothesis. This allows for a control of the type I error rate for the primary hypothesis even for adaptations not specified a priori in the study protocol. Estimation is usually another important part of a clinical trial, however, is more difficult in adaptive designs. In this research paper we give an overview of point and interval estimates for flexible designs and compare methods for typical sample size rules. We also make some proposals for confidence intervals which have nominal coverage probability also after an unforeseen design adaptation and which contain the maximum likelihood estimate and the usual unadjusted confidence interval.

Methodological issues with adaptation of clinical trial design

Adaptation of clinical trial design generates many issues that have not been resolved for practical applications, though statistical methodology has advanced greatly. This paper focuses on some methodological issues. In one type of adaptation such as sample size re-estimation, only the postulated value of a parameter for planning the trial size may be altered. In another type, the originally intended hypothesis for testing may be modified using the internal data accumulated at an interim time of the trial, such as changing the primary endpoint and dropping a treatment arm. For sample size re-estimation, we make a contrast between an adaptive test weighting the two-stage test statistics with the statistical information given by the original design and the original sample mean test with a properly corrected critical value. We point out the difficulty in planning a confirmatory trial based on the crude information generated by exploratory trials. In regards to selecting a primary endpoint, we argue that the selection process that allows switching from one endpoint to the other with the internal data of the trial is not very likely to gain a power advantage over the simple process of selecting one from the two endpoints by testing them with an equal split of alpha (Bonferroni adjustment). For dropping a treatment arm, distributing the remaining sample size of the discontinued arm to other treatment arms can substantially improve the statistical power of identifying a superior treatment arm in the design. A common difficult methodological issue is that of how to select an adaptation rule in the trial planning stage. Pre-specification of the adaptation rule is important for the practicality consideration. Changing the originally intended hypothesis for testing with the internal data generates great concerns to clinical trial researchers.

The reassessment of trial perspectives from interim data—a critical view

If an interim analysis is performed during a trial it is tempting to determine the conditional power to reach a rejection in the trial given the observed results in the interim analysis. Since the true effect size is unknown the conditional power may be calculated by using the effect size, which the study has been powered for in the planning phase or by using an interim estimate of the true size (or a combination of both). In either case the conditional power is a random variable and its density is investigated depending on the analysis time and the true effect size. Under the null hypothesis, in early interim analyses after a small proportion of sample units, the conditional power typically will be close to the overall power when the effect size from the planning stage is used for calculation. In this case the majority of observations must still be made and the small first-stage sample in general will be dominated by the hypothetical second-stage chance based on the wrong parameter value. It is shown that the conditional power in moderately underpowered studies can have a distribution symmetric around 0.5. When using the interim estimate for calculating the conditional power the density in general will be u-shaped. The impact of using conditional power to reassess the sample size using flexible two-stage combination tests is shown for a specific example in terms of overall power and average sample size as compared to the corresponding group sequential design. For small true effect sizes mid-trial sample size recalculation based on an interim estimate may lead to an overly large price to be paid in average sample size in relation to the gain in overall power. Finally, the problem is discussed in terms of estimating the true conditional power.

Increasing the sample size when the unblinded interim result is promising

Increasing the sample size based on unblinded interim result may inflate the type I error rate and appropriate statistical adjustments may be needed to control the type I error rate at the nominal level. We briefly review the existing approaches which allow early stopping due to futility, or change the test statistic by using different weights, or adjust the critical value for final test, or enforce rules for sample size recalculation. The implication of early stopping due to futility and a simple modification to the weighted Z-statistic approach are discussed. In this paper, we show that increasing the sample size when the unblinded interim result is promising will not inflate the type I error rate and therefore no statistical adjustment is necessary. The unblinded interim result is considered promising if the conditional power is greater than 50 per cent or equivalently, the sample size increment needed to achieve a desired power does not exceed an upper bound. The actual sample size increment may be determined by important factors such as budget, size of the eligible patient population and competition in the market. The 50 per cent-conditional-power approach is extended to a group sequential trial with one interim analysis where a decision may be made at the interim analysis to stop the trial early due to a convincing treatment benefit, or to increase the sample size if the interim result is not as good as expected. The type I error rate will not be inflated if the sample size may be increased only when the conditional power is greater than 50 per cent. If there are two or more interim analyses in a group sequential trial, our simulation study shows that the type I error rate is also well controlled.

Issues in designing flexible trials

We outline the general framework of adaptive combination tests and discuss their relationship to flexible group sequential designs. An important field of applications is sample size reassessment. We discuss reassessment rules based on conditional power arguments using either the observed or the prefixed effect size. These rules tend to lead to large expected sample sizes for small actual effects. However, the application of a maximal bound for the second stage sample size leads to more favourable properties. Additionally, we consider an optimized reassessment rule in terms of expected sample sizes. Since the adaptive design does not use the classical test statistics for some types of sample size reassessments, the adaptive test may reject the null hypothesis while the classical one-sample test does not. We characterize sample size reassessment rules, where such inconsistencies are avoided. Finally, the extension of flexibility to the number of stages is explored. In the first interim analysis a second interim analysis is only planned if the chance to achieve a decision there is high. This leads to savings in the average number of interim analysis performed, without paying a noticeable price in terms of expected sample size.

A comparison of methods for adaptive sample size adjustment

In fixed sample size designs, precise knowledge about the magnitude of the outcome variable's variance in the planning phase of a clinical trial is mandatory for an adequate sample size determination. Wittes and Brittain introduced the internal pilot study design that allows recalculation of the sample size during an ongoing trial using the estimated variance obtained from an interim analysis. However, this procedure requires the unblinding of the treatment code. Since unblinding of an ongoing trial should be avoided whenever possible, there should be some benefit of this design compared with blinded sample size recalculation procedures to justify the unveiling of the treatment code. In this paper, we compare several sample size recalculation procedures with and without unblinding. The simulation results indicate that the procedures behave similarly. In particular, breaking of the blind is not required for an efficient sample size adjustment. We also compare these pure sample size adaptation procedures with study designs which additionally allow for early stopping. Evaluation of the cumulative distribution function of the resulting sample sizes shows that the option for early stopping may lead to lower expectation but generally to a higher variability. The procedures are illustrated by an example of a trial in the treatment of depression.