Modification of the sample size and the schedule of interim analyses in survival trials based on data inspections by H. Schäfer and H.-H. Müller,Statistics in Medicine 2001;20: 3741–3751

Simultaneous inference procedures for the comparison of multiple characteristics of two survival functions

Survival time is the primary endpoint of many randomized controlled trials, and a treatment effect is typically quantified by the hazard ratio under the assumption of proportional hazards. Awareness is increasing that in many settings this assumption is a priori violated, for example, due to delayed onset of drug effect. In these cases, interpretation of the hazard ratio estimate is ambiguous and statistical inference for alternative parameters to quantify a treatment effect is warranted. We consider differences or ratios of milestone survival probabilities or quantiles, differences in restricted mean survival times, and an average hazard ratio to be of interest. Typically, more than one such parameter needs to be reported to assess possible treatment benefits, and in confirmatory trials, the according inferential procedures need to be adjusted for multiplicity. A simple Bonferroni adjustment may be too conservative because the different parameters of interest typically show considerable correlation. Hence simultaneous inference procedures that take into account the correlation are warranted. By using the counting process representation of the mentioned parameters, we show that their estimates are asymptotically multivariate normal and we provide an estimate for their covariance matrix. We propose according to the parametric multiple testing procedures and simultaneous confidence intervals. Also, the logrank test may be included in the framework. Finite sample type I error rate and power are studied by simulation. The methods are illustrated with an example from oncology. A software implementation is provided in the R package nph.

A gated group sequential design for seamless Phase II/III trial with subpopulation selection

BACKGROUND

Due to the high cost and high failure rate of Phase III trials where a classical group sequential design (GSD) is usually used, seamless Phase II/III designs are more and more popular to improve trial efficiency. A potential attraction of Phase II/III design is to allow a randomized proof-of-concept stage prior to committing to the full cost of a Phase III trial. Population selection during the trial allows a trial to adapt and focus investment where it is most likely to provide patient benefit. Previous methods have been developed for this problem when there is a single primary endpoint and two possible populations.

METHODS

To find the population that potentially benefits with one or two primary endpoints (e.g., progression free survival (PFS), overall survival (OS)), we propose a gated group sequential design for a seamless Phase II/III trial design with adaptive population selection.

RESULTS

The investigated design controls the familywise error rate and allows multiple interim analyses to enable early stopping for efficacy or futility. Simulations and an illustrative example suggest that the proposed gated group sequential design has more power and requires less time and resources compared to the group sequential design and adaptive design.

CONCLUSIONS

Combining the group sequential design and adaptive design, the gated group sequential design has more power and higher efficiency while controlling for the familywise error rate. It has the potential to save drug development cost and more quickly fulfill unmet medical needs.

Confirmatory adaptive group sequential designs for single-arm phase II studies with multiple time-to-event endpoints

Existing methods concerning the assessment of long-term survival outcomes in one-armed trials are commonly restricted to one primary endpoint. Corresponding adaptive designs suffer from limitations regarding the use of information from other endpoints in interim design changes. Here we provide adaptive group sequential one-sample tests for testing hypotheses on the multivariate survival distribution derived from multi-state models, while making provision for data-dependent design modifications based on all involved time-to-event endpoints. We explicitly elaborate application of the methodology to one-sample tests for the joint distribution of (i) progression-free survival (PFS) and overall survival (OS) in the context of an illness-death model, and (ii) time to toxicity and time to progression while accounting for death as a competing event. Large sample distributions are derived using a counting process approach. Small sample properties are studied by simulation. An already established multi-state model for non-small cell lung cancer is used to illustrate the adaptive procedure.

Predicting outcomes of phase III oncology trials with Bayesian mediation modeling of tumor response

Pivotal cancer trials often fail to yield evidence in support of new therapies thought to offer promising alternatives to standards-of-care. Conducting randomized controlled trials in oncology tends to be considerably more expensive than studies of other diseases with comparable sample size. Moreover, phase III trial design often takes place with a paucity of survival data for experimental therapies. Experts have explained the failures on the basis of design flaws which produce studies with unrealistic expectations. This article presents a framework for predicting outcomes of phase III oncology trials using Bayesian mediation models. Predictions, which arise from interim analyses, derive from multivariate modeling of the relationships among treatment, tumor response, and their conjoint effects on survival. Acting as a safeguard against inaccurate pre-trial design assumptions, the methodology may better facilitate rapid closure of negative studies. Additionally the models can be used to inform re-estimations of sample size for under-powered trials that demonstrate survival benefit via tumor response mediation. The methods are applied to predict the outcomes of two colorectal cancer studies. Simulation is used to evaluate and compare models in the absence versus presence of reliable surrogate markers of survival.

Design aspects of COVID-19 treatment trials: Improving probability and time of favorable events

As a reaction to the pandemic of the severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2), a multitude of clinical trials for the treatment of SARS‐CoV‐2 or the resulting corona disease 2019 (COVID‐19) are globally at various stages from planning to completion. Although some attempts were made to standardize study designs, this was hindered by the ferocity of the pandemic and the need to set up clinical trials quickly. We take the view that a successful treatment of COVID‐19 patients (i) increases the probability of a recovery or improvement within a certain time interval, say 28 days; (ii) aims to expedite favorable events within this time frame; and (iii) does not increase mortality over this time period. On this background, we discuss the choice of endpoint and its analysis. Furthermore, we consider consequences of this choice for other design aspects including sample size and power and provide some guidance on the application of adaptive designs in this particular context.

Adaptive group sequential survival comparisons based on log-rank and pointwise test statistics

Whereas the theory of confirmatory adaptive designs is well understood for uncensored data, implementation of adaptive designs in the context of survival trials remains challenging. Commonly used adaptive survival tests are based on the independent increments structure of the log-rank statistic. This implies some relevant limitations: On the one hand, essentially only the interim log-rank statistic may be used for design modifications (such as data-dependent sample size recalculation). Furthermore, the treatment arm allocation ratio in these classical methods is assumed to be constant throughout the trial period. Here, we propose an extension of the independent increments approach to adaptive survival tests that addresses some of these limitations. We present a confirmatory adaptive two-sample log-rank test that allows rejection regions and sample size recalculation rules to be based not only on the interim log-rank statistic, but also on point-wise survival rate estimates, simultaneously. In addition, the possibility is opened to adapt the treatment arm allocation ratio after each interim analysis in a data-dependent way. The ability to include point-wise survival rate estimators in the rejection region of a test for comparing survival curves might be attractive, e.g., for seamless phase II/III designs. Data-dependent adaptation of the allocation ratio could be helpful in multi-arm trials in order to successively steer recruitment into the study arms with the greatest chances of success. The methodology is motivated by the LOGGIC Europe Trial from pediatric oncology. Distributional properties are derived using martingale techniques in the large sample limit. Small sample properties are studied by simulation.

Sample size re-estimation for pivotal clinical trials

It is well known that if the hypothesis test is left unchanged, the Type I error rate may be inflated for sample size re-estimation (SSR) designs. To address this issue, three main approaches have been proposed in the literature: combination test, conditional error and conventional test with sample size increase in the allowable region (AR) only. These three seemingly different approaches are in fact connected. For each combination test, there is a corresponding conditional error function and AR. Designing adaptation rules in this AR with conventional test guarantees the Type I error rate control but at the same time always leads to smaller power comparing to the corresponding combination test (or conditional error) approach. In cases where conventional test is still preferable, step-wise type adaptation rules that do not fully reside in the AR can be alternatively considered. We believe controversies in the statistical community on the efficiency comparisons between group sequential (GS) and SSR design stem partially from the misalignment of performance metrics and conditional versus unconditional evaluations. We advocate summary metrics, such as median, variance or tail probabilities of the sample size in addition to expectation and personalizing efficiency definition for each trial sponsor. Conditional metrics by favorable, promising and unfavorable zones of the interim results provide additional insights and should always be incorporated into the decision-making process.

Quantifying treatment differences in confirmatory trials under non-proportional hazards

Proportional hazards are a common assumption when designing confirmatory clinical trials in oncology. With the emergence of immunotherapy and novel targeted therapies, departure from the proportional hazard assumption is not rare in nowadays clinical research. Under non-proportional hazards, the hazard ratio does not have a straightforward clinical interpretation, and the log-rank test is no longer the most powerful statistical test even though it is still valid. Nevertheless, the log-rank test and the hazard ratio are still the primary analysis tools, and traditional approaches such as sample size increase are still proposed to account for the impact of non-proportional hazards. The weighed log-rank test and the test based on the restricted mean survival time (RMST) are receiving a lot of attention as a potential alternative to the log-rank test. We conduct a simulation study comparing the performance and operating characteristics of the log-rank test, the weighted log-rank test and the test based on the RMST, including a treatment effect estimation, under different non-proportional hazards patterns. Results show that, under non-proportional hazards, the hazard ratio and weighted hazard ratio have no straightforward clinical interpretation whereas the RMST ratio can be interpreted regardless of the proportional hazards assumption. In terms of power, the RMST achieves a similar performance when compared to the log-rank test.

Clinical Trials Impacted by the COVID-19 Pandemic: Adaptive Designs to the Rescue?

Very recently the new pathogen severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was identified and the coronavirus disease 2019 (COVID-19) declared a pandemic by the World Health Organization. The pandemic has a number of consequences for ongoing clinical trials in non-COVID-19 conditions. Motivated by four current clinical trials in a variety of disease areas we illustrate the challenges faced by the pandemic and sketch out possible solutions including adaptive designs. Guidance is provided on (i) where blinded adaptations can help; (ii) how to achieve Type I error rate control, if required; (iii) how to deal with potential treatment effect heterogeneity; (iv) how to use early read-outs; and (v) how to use Bayesian techniques. In more detail approaches to resizing a trial affected by the pandemic are developed including considerations to stop a trial early, the use of group-sequential designs or sample size adjustment. All methods considered are implemented in a freely available R shiny app. Furthermore, regulatory and operational issues including the role of data monitoring committees are discussed.

Point and interval estimation in two-stage adaptive designs with time to event data and biomarker-driven subpopulation selection

In personalized medicine, it is often desired to determine if all patients or only a subset of them benefit from a treatment. We consider estimation in two-stage adaptive designs that in stage 1 recruit patients from the full population. In stage 2, patient recruitment is restricted to the part of the population, which, based on stage 1 data, benefits from the experimental treatment. Existing estimators, which adjust for using stage 1 data for selecting the part of the population from which stage 2 patients are recruited, as well as for the confirmatory analysis after stage 2, do not consider time to event patient outcomes. In this work, for time to event data, we have derived a new asymptotically unbiased estimator for the log hazard ratio and a new interval estimator with good coverage probabilities and probabilities that the upper bounds are below the true values. The estimators are appropriate for several selection rules that are based on a single or multiple biomarkers, which can be categorical or continuous.

Improving interim decisions in randomized trials by exploiting information on short-term endpoints and prognostic baseline covariates

Conditional power calculations are frequently used to guide the decision whether or not to stop a trial for futility or to modify planned sample size. These ignore the information in short-term endpoints and baseline covariates, and thereby do not make fully efficient use of the information in the data. We therefore propose an interim decision procedure based on the conditional power approach which exploits the information contained in baseline covariates and short-term endpoints. We will realize this by considering the estimation of the treatment effect at the interim analysis as a missing data problem. This problem is addressed by employing specific prediction models for the long-term endpoint which enable the incorporation of baseline covariates and multiple short-term endpoints. We show that the proposed procedure leads to an efficiency gain and a reduced sample size, without compromising the Type I error rate of the procedure, even when the adopted prediction models are misspecified. In particular, implementing our proposal in the conditional power approach enables earlier decisions relative to standard approaches, whilst controlling the probability of an incorrect decision. This time gain results in a lower expected number of recruited patients in case of stopping for futility, such that fewer patients receive the futile regimen. We explain how these methods can be used in adaptive designs with unblinded sample size re-assessment based on the inverse normal P-value combination method to control Type I error. We support the proposal by Monte Carlo simulations based on data from a real clinical trial.

An adaptive population enrichment phase III trial of TRC105 and pazopanib versus pazopanib alone in patients with advanced angiosarcoma (TAPPAS trial)

Background

Major challenges in clinical trials of ultra-orphan oncology diseases include limited patient availability and paucity of reliable prior data for estimating the treatment effect and, therefore, determining optimal sample size. Angiosarcoma (AS), a particularly aggressive form of soft tissue sarcoma with an incidence of about 2000 cases per year in the United States and Europe is poorly addressed by current systemic therapies. Pazopanib, an inhibitor of vascular endothelial growth factor receptor (VEGFR) is approved for the treatment of AS, with modest benefit. TRC105 (carotuximab) is a monoclonal antibody to endoglin, an essential angiogenic target highly expressed on proliferating endothelium and both tumor vessels and tumor cells in AS, that has the potential to complement VEGFR tyrosine kinase inhibitors. In a phase I/II study of soft tissue sarcoma, TRC105 combined safely with pazopanib and the combination demonstrated durable complete responses and encouraging progression-free survival (PFS). In addition, there was a suggestion of superior benefit in patients with cutaneous lesions versus those with the non-cutaneous lesions.

Patients and methods

This article describes the design of a recently initiated phase III trial of TRC105 And Pazopanib versus Pazopanib alone in patients with advanced AngioSarcoma (TAPPAS trial). Given the ultra-orphan status of the disease and the paucity of reliable prior data on PFS or overall survival (end points required for regulatory approval as a pivotal trial), an adaptive design incorporating population enrichment and sample size re-estimation was implemented. The design incorporated regulatory input from the Food and Drug Administration (FDA) and European Medicines Agency and proceeded following special protocol assessment designation by the FDA.

Conclusions

It is shown that the benefit of the adaptive design as compared with a conventional single-look design arises from the learning and subsequent improvements in power that occur after an unblinded analysis of interim data.

Registered on Clinicaltrials.gov

NCT02979899.

Adaptive seamless clinical trials using early outcomes for treatment or subgroup selection: Methods, simulation model and their implementation in R

Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, for example, in terms of sample size and/or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in the form of extensive simulation studies, than conventional approaches. Here, we present a framework for adaptive treatment and subgroup selection using the same notation, which links the somewhat disparate literature on treatment selection on one side and on subgroup selection on the other. Furthermore, we introduce a flexible and efficient simulation model that serves both designs. As primary endpoints often take a long time to observe, interim analyses are frequently informed by early outcomes. Therefore, all methods presented accommodate interim analyses informed by either the primary outcome or an early outcome. The R package asd, previously developed to simulate designs with treatment selection, was extended to include subgroup selection (so‐called adaptive enrichment designs). Here, we describe the functionality of the R package asd and use it to present some worked‐up examples motivated by clinical trials in chronic obstructive pulmonary disease and oncology. The examples both illustrate various features of the R package and provide insights into the operating characteristics of adaptive seamless studies.

Nested combination tests with a time-to-event endpoint using a short-term endpoint for design adaptations

Adaptive trial methodology for multiarmed trials and enrichment designs has been extensively discussed in the past. A general principle to construct test procedures that control the family-wise Type I error rate in the strong sense is based on combination tests within a closed test. Using survival data, a problem arises when using information of patients for adaptive decision making, which are under risk at interim. With the currently available testing procedures, either no testing of hypotheses in interim analyses is possible or there are restrictions on the interim data that can be used in the adaptation decisions as, essentially, only the interim test statistics of the primary endpoint may be used. We propose a general adaptive testing procedure, covering multiarmed and enrichment designs, which does not have these restrictions. An important application are clinical trials, where short-term surrogate endpoints are used as basis for trial adaptations, and we illustrate how such trials can be designed. We propose statistical models to assess the impact of effect sizes, the correlation structure between the short-term and the primary endpoint, the sample size, the timing of interim analyses, and the selection rule on the operating characteristics.

Randomized phase II/III confirmatory treatment selection design with a change of survival end points: Statistical design of Radiation Therapy Oncology Group 1216

BACKGROUND

To confirm the treatment effects of concurrent cetuximab plus docetaxel observed in Radiation Therapy Oncology Group (RTOG) 0234 and single out the effect of cetuximab, we designed RTOG 1216, a randomized phase II/III study, which uses an intermediate end point to select the best regimen for definitive testing of survival benefit.

METHODS

In phase II, the best regimen should demonstrate statistically significant efficacy against the control with predefined advantage over the competing arm regarding disease-free survival (DFS). We evaluate operating characteristics of the randomized II/III group sequential design through simulations and numerical integrations under the null and various alternative hypotheses.

RESULTS

Results show the randomized II/III design yields substantial savings on sample size and time with well-controlled type I and type II error rates.

CONCLUSION

Overall, the proposed randomized II/III design has desirable properties that offer cost effectiveness, operational efficiency, and, most importantly, scientific innovation that can be considered for similar clinical research settings.

Properties of the weighted log-rank test in the design of confirmatory studies with delayed effects

Proportional hazards are a common assumption when designing confirmatory clinical trials in oncology. This assumption not only affects the analysis part but also the sample size calculation. The presence of delayed effects causes a change in the hazard ratio while the trial is ongoing since at the beginning we do not observe any difference between treatment arms, and after some unknown time point, the differences between treatment arms will start to appear. Hence, the proportional hazards assumption no longer holds, and both sample size calculation and analysis methods to be used should be reconsidered. The weighted log-rank test allows a weighting for early, middle, and late differences through the Fleming and Harrington class of weights and is proven to be more efficient when the proportional hazards assumption does not hold. The Fleming and Harrington class of weights, along with the estimated delay, can be incorporated into the sample size calculation in order to maintain the desired power once the treatment arm differences start to appear. In this article, we explore the impact of delayed effects in group sequential and adaptive group sequential designs and make an empirical evaluation in terms of power and type-I error rate of the of the weighted log-rank test in a simulated scenario with fixed values of the Fleming and Harrington class of weights. We also give some practical recommendations regarding which methodology should be used in the presence of delayed effects depending on certain characteristics of the trial.

Adaptive designs in clinical trials: from scientific advice to marketing authorisation to the European Medicine Agency

Background

In recent years, experience on the application of adaptive designs in confirmatory clinical trials has accumulated. Although planning such trials comes at the cost of additional operational complexity, adaptive designs offer the benefit of flexibility to update trial design and objectives as data accrue. In 2007, the European Medicines Agency (EMA) provided guidance on confirmatory clinical trials with adaptive (or flexible) designs. In order to better understand how adaptive trials are implemented in practice and how they may impact medicine approval within the EMA centralised procedure, we followed on 59 medicines for which an adaptive clinical trial had been submitted to the EMA Scientific Advice (SA) and analysed previously in a dedicated EMA survey of scientific advice letters. We scrutinized in particular the submission of the corresponding medicines for a marketing authorisation application (MAA). We also discuss the current regulatory perspective as regards the implementation of adaptive designs in confirmatory clinical trials.

Methods

Using the internal EMA MAA database, the AdisInsight database and related trial registries, we analysed how many of these 59 trials actually started, the completion status, results, the time to trial start, the adaptive elements finally implemented after SA, their possible influence on the success of the trial and corresponding product approval.

Results

Overall 31 trials out of 59 (53%) were retrieved. Thirty of them (97%) have been started and 23 (74%) concluded. Nine of these trials (39% out of 23) demonstrated a significant treatment effect on their primary endpoint and 4 (17% out of 23) supported a marketing authorisation (MA). An additional two trials were stopped using pre-defined criteria for futility, efficiently identifying trials on which further resources should not be spent. Median time to trial start after SA letter was given by EMA was 5 months. In the investigated trial registries, at least 18 trial (58% of 31 retrieved trials) designs were implemented with adaptive elements, which were predominantly dose selection, sample size reassessment (SSR) and stopping for futility (SFF). Among the 11 completed trials including adaptive elements, 6 demonstrated a significant treatment effect on their primary endpoint (55%).

Conclusions

Adaptive designs are now well established in the drug development landscape. If properly pre-planned, adaptations can play a key role in the success of some of these trials, for example to help successfully select the most promising dose regimens for phase II/III trials. Interim analyses can also enable stopping of trials for futility when they do not hold their promises. Type I error rate control, trial integrity and results consistency between the different stages of the analyses are fundamental aspects to be discussed thoroughly. Engaging early dialogue with regulators and implementing the scientific advice received is strongly recommended, since much experience in discussing adaptive designs and assessing their results has been accumulated.

Nonparametric adaptive enrichment designs using categorical surrogate data

Adaptive survival trials are particularly important for enrichment designs in oncology and other life-threatening diseases. Current statistical methodology for adaptive survival trials provide type I error rate control only under restrictions. For instance, if we use stage-wise P values based on increments of the log-rank test, then the information used for the interim decisions need to be restricted to the primary survival endpoint. However, it is often desirable to base interim decisions also on correlated short-term endpoints like tumor response. Alternative statistical approaches based on a patient-wise splitting of the data require unnatural restrictions on the follow-up times and do not permit to efficiently account for an early rejection of the primary null hypothesis. We therefore suggest new approaches that enable us to use discrete surrogate endpoints (like tumor response status) and also to incorporate interim rejection boundaries. The new approaches are based on weighted Kaplan-Meier estimates and thereby have additional advantages. They permit us to account for nonproportional hazards and are robust against informative censoring based on the surrogate endpoint. We will show that nonproportionality is an intrinsic and relevant issue in enrichment designs. Moreover, informative censoring based on the surrogate endpoint is likely because of withdrawals and treatment switches after insufficient treatment response. It is shown and illustrated how nonparametric tests based on weighted Kaplan-Meier estimates can be used in closed combination tests for adaptive enrichment designs, such that type I error rate control is achieved and justified asymptotically.

Optimized adaptive enrichment designs

Based on a Bayesian decision theoretic approach, we optimize frequentist single- and adaptive two-stage trial designs for the development of targeted therapies, where in addition to an overall population, a pre-defined subgroup is investigated. In such settings, the losses and gains of decisions can be quantified by utility functions that account for the preferences of different stakeholders. In particular, we optimize expected utilities from the perspectives both of a commercial sponsor, maximizing the net present value, and also of the society, maximizing cost-adjusted expected health benefits of a new treatment for a specific population. We consider single-stage and adaptive two-stage designs with partial enrichment, where the proportion of patients recruited from the subgroup is a design parameter. For the adaptive designs, we use a dynamic programming approach to derive optimal adaptation rules. The proposed designs are compared to trials which are non-enriched (i.e. the proportion of patients in the subgroup corresponds to the prevalence in the underlying population). We show that partial enrichment designs can substantially improve the expected utilities. Furthermore, adaptive partial enrichment designs are more robust than single-stage designs and retain high expected utilities even if the expected utilities are evaluated under a different prior than the one used in the optimization. In addition, we find that trials optimized for the sponsor utility function have smaller sample sizes compared to trials optimized under the societal view and may include the overall population (with patients from the complement of the subgroup) even if there is substantial evidence that the therapy is only effective in the subgroup.

Sequential tests for non-proportional hazards data

In clinical trials survival endpoints are usually compared using the log-rank test. Sequential methods for the log-rank test and the Cox proportional hazards model are largely reported in the statistical literature. When the proportional hazards assumption is violated the hazard ratio is ill-defined and the power of the log-rank test depends on the distribution of the censoring times. The average hazard ratio was proposed as an alternative effect measure, which has a meaningful interpretation in the case of non-proportional hazards, and is equal to the hazard ratio, if the hazards are indeed proportional. In the present work we prove that the average hazard ratio based sequential test statistics are asymptotically multivariate normal with the independent increments property. This allows for the calculation of group-sequential boundaries using standard methods and existing software. The finite sample characteristics of the new method are examined in a simulation study in a proportional and a non-proportional hazards setting.

Interim decision-making strategies in adaptive designs for population selection using time-to-event endpoints

Adaptive designs in oncology clinical trials with interim analyses for population selection could be used in the development of targeted therapies if a predefined biomarker hypothesis exists. In this article, we consider an interim analysis using overall survival (OS), progression-free survival (PFS), and both OS and PFS, to determine whether the whole population or only the biomarker-positive population should continue into the subsequent stage of the trial, whereas the final decision is made based on OS data only. In order to increase the probability of selecting the most appropriate population at the interim analysis, we propose an interim decision-making strategy in adaptive designs with correlated endpoints considering the post-progression survival (PPS) magnitudes. In our approach, the interim decision is made on the basis of predictive power by incorporating information on OS as well as PFS to supplement the incomplete OS data. Simulation studies assuming a targeted therapy demonstrated that our interim decision-making procedure performs well in terms of selecting the proper population, especially under a scenario in which PPS affects the correlation between OS and PFS.

Sample Size Reassessment and Hypothesis Testing in Adaptive Survival Trials

Mid-study design modifications are becoming increasingly accepted in confirmatory clinical trials, so long as appropriate methods are applied such that error rates are controlled. It is therefore unfortunate that the important case of time-to-event endpoints is not easily handled by the standard theory. We analyze current methods that allow design modifications to be based on the full interim data, i.e., not only the observed event times but also secondary endpoint and safety data from patients who are yet to have an event. We show that the final test statistic may ignore a substantial subset of the observed event times. An alternative test incorporating all event times is found, where a conservative assumption must be made in order to guarantee type I error control. We examine the power of this approach using the example of a clinical trial comparing two cancer therapies.

Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

Methods for identification and confirmation of targeted subgroups in clinical trials: A systematic review

Important objectives in the development of stratified medicines include the identification and confirmation of subgroups of patients with a beneficial treatment effect and a positive benefit-risk balance. We report the results of a literature review on methodological approaches to the design and analysis of clinical trials investigating a potential heterogeneity of treatment effects across subgroups. The identified approaches are classified based on certain characteristics of the proposed trial designs and analysis methods. We distinguish between exploratory and confirmatory subgroup analysis, frequentist, Bayesian and decision-theoretic approaches and, last, fixed-sample, group-sequential, and adaptive designs and illustrate the available trial designs and analysis strategies with published case studies.

Testing multiple primary endpoints in clinical trials with sample size adaptation

In this paper, we propose a design that uses a short-term endpoint for accelerated approval at interim analysis and a long-term endpoint for full approval at final analysis with sample size adaptation based on the long-term endpoint. Two sample size adaptation rules are compared: an adaptation rule to maintain the conditional power at a prespecified level and a step function type adaptation rule to better address the bias issue. Three testing procedures are proposed: alpha splitting between the two endpoints; alpha exhaustive between the endpoints; and alpha exhaustive with improved critical value based on correlation. Family-wise error rate is proved to be strongly controlled for the two endpoints, sample size adaptation, and two analysis time points with the proposed designs. We show that using alpha exhaustive designs greatly improve the power when both endpoints are effective, and the power difference between the two adaptation rules is minimal. The proposed design can be extended to more general settings.

Adaptive seamless designs with interim treatment selection: a case study in oncology

The planning of an oncology clinical trial with a seamless phase II/III adaptive design is discussed. Two regimens of an experimental treatment are compared to a control at an interim analysis, and the most-promising regimen is selected to continue, together with control, until the end of the study. Because the primary endpoint is expected to be immature at the interim regimen selection analysis, designs that incorporate primary as well as surrogate endpoints in the regimen selection process are considered. The final testing of efficacy at the end of the study comparing the selected regimen to the control with respect to the primary endpoint uses all relevant data collected both before and after the regimen selection analysis. Several approaches for testing the primary hypothesis are assessed with regard to power and type I error rate. Because the operating characteristics of these designs depend on the specific regimen selection rules considered, benchmark scenarios are proposed in which a perfect surrogate and no surrogate is used at the regimen selection analysis. The operating characteristics of these benchmark scenarios provide a range where those of the actual study design are expected to lie. A discussion on family-wise error rate control for testing primary and key secondary endpoints as well as an assessment of bias in the final treatment effect estimate for the selected regimen are also presented.

Optimizing the data combination rule for seamless phase II/III clinical trials

We consider seamless phase II/III clinical trials that compare K treatments with a common control in phase II then test the most promising treatment against control in phase III. The final hypothesis test for the selected treatment can use data from both phases, subject to controlling the familywise type I error rate. We show that the choice of method for conducting the final hypothesis test has a substantial impact on the power to demonstrate that an effective treatment is superior to control. To understand these differences in power, we derive decision rules maximizing power for particular configurations of treatment effects. A rule with such an optimal frequentist property is found as the solution to a multivariate Bayes decision problem. The optimal rules that we derive depend on the assumed configuration of treatment means. However, we are able to identify two decision rules with robust efficiency: a rule using a weighted average of the phase II and phase III data on the selected treatment and control, and a closed testing procedure using an inverse normal combination rule and a Dunnett test for intersection hypotheses. For the first of these rules, we find the optimal division of a given total sample size between phases II and III. We also assess the value of using phase II data in the final analysis and find that for many plausible scenarios, between 50% and 70% of the phase II numbers on the selected treatment and control would need to be added to the phase III sample size in order to achieve the same increase in power. © 2014 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

Adaptive clinical trial designs for European marketing authorization: a survey of scientific advice letters from the European Medicines Agency

Background

Since the first methodological publications on adaptive study design approaches in the 1990s, the application of these approaches in drug development has raised increasing interest among academia, industry and regulators. The European Medicines Agency (EMA) as well as the Food and Drug Administration (FDA) have published guidance documents addressing the potentials and limitations of adaptive designs in the regulatory context. Since there is limited experience in the implementation and interpretation of adaptive clinical trials, early interaction with regulators is recommended. The EMA offers such interactions through scientific advice and protocol assistance procedures.

Methods

We performed a text search of scientific advice letters issued between 1 January 2007 and 8 May 2012 that contained relevant key terms. Letters containing questions related to adaptive clinical trials in phases II or III were selected for further analysis. From the selected letters, important characteristics of the proposed design and its context in the drug development program, as well as the responses of the Committee for Human Medicinal Products (CHMP)/Scientific Advice Working Party (SAWP), were extracted and categorized. For 41 more recent procedures (1 January 2009 to 8 May 2012), additional details of the trial design and the CHMP/SAWP responses were assessed. In addition, case studies are presented as examples.

Results

Over a range of 5½ years, 59 scientific advices were identified that address adaptive study designs in phase II and phase III clinical trials. Almost all were proposed as confirmatory phase III or phase II/III studies. The most frequently proposed adaptation was sample size reassessment, followed by dropping of treatment arms and population enrichment. While 12 (20%) of the 59 proposals for an adaptive clinical trial were not accepted, the great majority of proposals were accepted (15, 25%) or conditionally accepted (32, 54%). In the more recent 41 procedures, the most frequent concerns raised by CHMP/SAWP were insufficient justifications of the adaptation strategy, type I error rate control and bias.

Conclusions

For the majority of proposed adaptive clinical trials, an overall positive opinion was given albeit with critical comments. Type I error rate control, bias and the justification of the design are common issues raised by the CHMP/SAWP.

Biomarker driven population enrichment for adaptive oncology trials with time to event endpoints

The development of molecularly targeted therapies for certain types of cancers has led to the consideration of population enrichment designs that explicitly factor in the possibility that the experimental compound might differentially benefit different biomarker subgroups. In such designs, enrollment would initially be open to a broad patient population with the option to restrict future enrollment, following an interim analysis, to only those biomarker subgroups that appeared to be benefiting from the experimental therapy. While this strategy could greatly improve the chances of success for the trial, it poses several statistical and logistical design challenges. Because late-stage oncology trials are typically event driven, one faces a complex trade-off between power, sample size, number of events, and study duration. This trade-off is further compounded by the importance of maintaining statistical independence of the data before and after the interim analysis and of optimizing the timing of the interim analysis. This paper presents statistical methodology that ensures strong control of type 1 error for such population enrichment designs, based on generalizations of the conditional error rate approach. The special difficulties encountered with time-to-event endpoints are addressed by our methods. The crucial role of simulation for guiding the choice of design parameters is emphasized. Although motivated by oncology, the methods are applicable as well to population enrichment designs in other therapeutic areas.

Improving Probabilities of Correct Interim Decision in Population Enrichment Designs

Choice of target population is an essential part at the design stage of clinical trials. Data from earlier clinical development might suggest that the treatment is more effective in a subpopulation, but there might not be enough evidence to restrict the target population upfront. Adaptive designs allow modification of the target population based on interim data. Decision for modification should be based on objective decision rules. The presented decision rules maximize the weighted probability of correct interim decisions based on prior assumptions. Evaluation of decision rules in the planning phase can improve probabilities of correct interim decision and power.

Sample-size calculation and reestimation for a semiparametric analysis of recurrent event data taking robust standard errors into account

In some clinical trials, the repeated occurrence of the same type of event is of primary interest and the Andersen-Gill model has been proposed to analyze recurrent event data. Existing methods to determine the required sample size for an Andersen-Gill analysis rely on the strong assumption that all heterogeneity in the individuals' risk to experience events can be explained by known covariates. In practice, however, this assumption might be violated due to unknown or unmeasured covariates affecting the time to events. In these situations, the use of a robust variance estimate in calculating the test statistic is highly recommended to assure the type I error rate, but this will in turn decrease the actual power of the trial. In this article, we derive a new sample-size formula to reach the desired power even in the presence of unexplained heterogeneity. The formula is based on an inflation factor that considers the degree of heterogeneity and characteristics of the robust variance estimate. Nevertheless, in the planning phase of a trial there will usually be some uncertainty about the size of the inflation factor. Therefore, we propose an internal pilot study design to reestimate the inflation factor during the study and adjust the sample size accordingly. In a simulation study, the performance and validity of this design with respect to type I error rate and power are proven. Our method is applied to the HepaTel trial evaluating a new intervention for patients with cirrhosis of the liver.

An adaptive group sequential phase II design to compare treatments for survival endpoints in rare patient entities

For rare diseases, standard treatments are often not available and essential study parameters are difficult or impossible to obtain. Therefore, designs of clinical trials for these diseases are often based on little information. Adaptive designs allow such trials to be started and to gain information during the study. Motivated by a trial for a rare subtype of renal-cell carcinoma, we present a two-stage adaptive design for right-censored time-to-event data and a two-sided test. After the first stage, one can stop for futility or continue with reestimated sample size. The properties of such designs are analyzed by simulation studies.

Seamless phase II/III designs

In recent years, there has been a drive to save development costs and shorten time-to-market of new therapies. Research into novel trial designs to facilitate this goal has led to, amongst other approaches, the development of methodology for seamless phase II/III designs. Such designs allow treatment or dose selection at an interim analysis and comparative evaluation of efficacy with control, in the same study. Methods have gained much attention because of their potential advantages compared to conventional drug development programmes with separate trials for individual phases. In this article, we review the various approaches to seamless phase II/III designs based upon the group-sequential approach, the combination test approach and the adaptive Dunnett method. The objective of this article is to describe the approaches in a unified framework and highlight their similarities and differences to allow choice of an appropriate methodology by a trialist considering conducting such a trial.

A confirmatory seamless phase II/III clinical trial design incorporating short-term endpoint information

Seamless phase II/III designs allow strong control of the familywise type I error rate when the most promising of a number of experimental treatments is selected at an interim analysis to continue along with the control treatment. If the primary endpoint is observed only after long-term follow-up it may be desirable to use correlated short-term endpoint data available at the interim analysis to inform the treatment selection. If short-term data are available for some patients for whom the primary endpoint is not available, basing treatment selection on these data may, however, lead to inflation of the type I error rate. This paper proposes a method for the adjustment of the usual group-sequential boundaries to maintain strong control of the familywise error rate even when short-term endpoint data are used for the treatment selection at the first interim analysis. This method allows the use of the short-term data, leading to an increase in power when these data are correlated with the primary endpoint data.

Adaptive trial design: a general methodology for censored time to event data

Adaptive designs allow a clinical trial design to be changed according to interim findings without inflating type I error. The Inverse Normal method can be considered as an adaptive generalization of classical group sequential designs. The use of the Inverse Normal method for censored survival data was demonstrated only for the logrank statistic. However, the logrank statistic is inefficient in the presence of nuisance covariates affecting survival. We demonstrate, how the Inverse Normal method can be applied to Cox regression analysis. The required independence between test statistics of the different stages of the trial can be obtained by two different approaches. One is using the independent increment structure of the score process. The other uses right censoring and left truncating to divide individuals follow-up into per-stage data. Simulation studies show, that performance of the adaptive design does not depend on the method used for obtaining independence. Either way, an adaptive Cox regression analyis is more efficient than an adaptive logrank analysis if nuisance covariates affect survival.