On model-based time trend adjustments in platform trials with non-concurrent controls

Examining the bias-efficiency tradeoff from incorporation of nonconcurrent controls in platform trials: A simulation study example from the adaptive COVID-19 treatment trial

BACKGROUND

Platform trials typically feature a shared control arm and multiple experimental treatment arms. Staggered entry and exit of arms splits the control group into two cohorts: those randomized during the same period in which the experimental arm was open (concurrent controls) and those randomized outside that period (nonconcurrent controls). Combining these control groups may offer increased statistical power but can lead to bias if analyses do not account for time trends in the response variable. Proposed methods of adjustment for time may increase type I error rates when time trends impact arms unequally or when large, sudden changes to the response rate occur. However, there has been limited exploration of the degree of type I error inflation one can plausibly expect in real-world scenarios.

METHODS

We use data from the Adaptive COVID-19 Treatment Trial (ACTT) to mimic a realistic platform trial with a remdesivir control arm. We compare four strategies for estimating the effect of interferon beta-1a (the ACTT-3 experimental arm) relative to remdesivir (data from ACTT-1, ACTT-2, and ACTT-3) on recovery and death by day 29: utilizing concurrent controls only (the prespecified analysis), pooling all remdesivir arm data without adjustment (the "unadjusted-pooled" analysis), adjusting for time as a categorical variable, and a Bayesian hierarchical model implementation which adjusts for time trends using smoothing techniques (the "Bayesian time machine"). We compare type I error rates and relative efficiency of each method in simulation settings based on observed ACTT remdesivir arm data.

RESULTS

The unadjusted-pooled approach provided substantially different estimates of the effect of interferon beta-1a relative to remdesivir compared with the concurrent-only and model-based approaches, indicating that changes in recovery and death rates over time were not ignorable across different stages of ACTT. The model-based approaches rely on an assumption of constant treatment effects for each arm in the platform relative to control; error rates more than doubled in settings where this was not satisfied. Relative efficiency of the model-based approaches compared with the concurrent-only analysis was moderate.

CONCLUSIONS

In simulation settings where key model assumptions were not met, potential efficiency gains from incorporation of nonconcurrent controls were outweighed by the risk of substantial type I error rate inflation. This leads us to advise against these strategies for primary analyses in confirmatory clinical trials, aligning with current FDA guidance advising against comparisons to nonconcurrent controls in COVID-19 settings. The model-based adjustment methods may be useful in other settings, but we recommend performing the concurrent-only analysis as a reference for assessing the degree to which nonconcurrent controls drive results.

Why and how should we simulate platform trials? Learnings from EU-PEARL

BACKGROUND

Platform trials are innovative clinical trials governed by a master protocol that allows for the evaluation of multiple investigational treatments that enter and leave the trial over time. Interest in platform trials has been steadily increasing over the last decade. Due to their highly adaptive nature, platform trials provide sufficient flexibility to customize important trial design aspects to the requirements of both the specific disease under investigation and the different stakeholders. The flexibility of platform trials, however, comes with complexities when designing such trials. In the past, we reviewed existing software for simulating clinical trials and found that none of them were suitable for simulating platform trials as they do not accommodate the design features and flexibility inherent to platform trials, such as staggered entry of treatments over time.

RESULTS

We argued that simulation studies are crucial for the design of efficient platform trials. We developed and proposed an iterative, simulation-guided "vanilla and sprinkles" framework, i.e. from a basic to a more complex design, for designing platform trials. We addressed the functionality limitations of existing software as well as the unavailability of the coding therein by developing a suite of open-source software to use in simulating platform trials based on the R programming language. To give some examples, the newly developed software supports simulating staggered entry of treatments throughout the trial, choosing different options for control data sharing, specifying different platform stopping rules and platform-level operating characteristics. The software we developed is available through open-source licensing to enable users to access and modify the code. The separate use of two of these software packages to implement the same platform design by independent teams obtained the same results.

CONCLUSION

We provide a framework, as well as open-source software for the design and simulation of platform trials. The software tools provide the flexibility necessary to capture the complexity of platform trials.

Innovative approaches for vaccine trials as a key component of pandemic preparedness - a white paper

BACKGROUND

WHO postulates the application of adaptive design features in the global clinical trial ecosystem. However, the adaptive platform trial (APT) methodology has not been widely adopted in clinical research on vaccines.

METHODS

The VACCELERATE Consortium organized a two-day workshop to discuss the applicability of APT methodology in vaccine trials under non-pandemic as well as pandemic conditions. Core aspects of the discussions are summarized in this article.

RESULTS

An "ever-warm" APT appears ideally suited to improve efficiency and speed of vaccine research. Continuous learning based on accumulating APT trial data allows for pre-planned adaptations during its course. Given the relative design complexity, alignment of all stakeholders at all stages of an APT is central. Vaccine trial modelling is crucial, both before and in a pandemic emergency. Various inferential paradigms are possible (frequentist, likelihood, or Bayesian). The focus in the interpandemic interval may be on research gaps left by industry trials. For activation in emergency, template Disease X protocols of syndromal design for pathogens yet unknown need to be stockpiled and updated regularly. Governance of a vaccine APT should be fully integrated into supranational pandemic response mechanisms.

DISCUSSION

A broad range of adaptive features can be applied in platform trials on vaccines. Faster knowledge generation comes with increased complexity of trial design. Design complexity should not preclude simple execution at trial sites. Continuously generated evidence represents a return on investment that will garner societal support for sustainable funding. Adaptive design features will naturally find their way into platform trials on vaccines.

Covariate-adjusted response-adaptive designs for semiparametric survival models

Covariate-adjusted response adaptive (CARA) designs are effective in increasing the expected number of patients receiving superior treatment in an ongoing clinical trial, given a patient's covariate profile. There has recently been extensive research on CARA designs with parametric distributional assumptions on patient responses. However, the range of applications for such designs becomes limited in real clinical trials. Sverdlov et al. have pointed out that irrespective of a specific parametric form of the survival outcomes, their proposed CARA designs based on the exponential model provide valid statistical inference, provided the final analysis is performed using the appropriate accelerated failure time (AFT) model. In real survival trials, however, the planned primary analysis is rarely conducted using an AFT model. The proposed CARA designs are developed obviating any distributional assumptions about the survival responses, relying only on the proportional hazards assumption between the two treatment arms. To meet the multiple experimental objectives of a clinical trial, the proposed designs are developed based on an optimal allocation approach. The covariate-adjusted doubly adaptive biased coin design and the covariate-adjusted efficient-randomized adaptive design are used to randomize the patients to achieve the derived targets on expectation. These expected targets are functions of the Cox regression coefficients that are estimated sequentially with the arrival of every new patient into the trial. The merits of the proposed designs are assessed using extensive simulation studies of their operating characteristics and then have been implemented to re-design a real-life confirmatory clinical trial.

Analysis of Nonconcurrent Controls in Adaptive Platform Trials: Separating Randomized and Nonrandomized Information

Adaptive platform trials allow treatments to be added or dropped during the study, meaning that the control arm may be active for longer than the experimental arms. This leads to nonconcurrent controls, which provide nonrandomized information that may increase efficiency but may introduce bias from temporal confounding and other factors. Various methods have been proposed to control confounding from nonconcurrent controls, based on adjusting for time period. We demonstrate that time adjustment is insufficient to prevent bias in some circumstances where nonconcurrent controls are present in adaptive platform trials, and we propose a more general analytical framework that accounts for nonconcurrent controls in such circumstances. We begin by defining nonconcurrent controls using the concept of a concurrently randomized cohort, which is a subgroup of participants all subject to the same randomized design. We then use cohort adjustment rather than time adjustment. Due to flexibilities in platform trials, more than one randomized design may be in force at any time, meaning that cohort-adjusted and time-adjusted analyses may be quite different. Using simulation studies, we demonstrate that time-adjusted analyses may be biased while cohort-adjusted analyses remove this bias. We also demonstrate that the cohort-adjusted analysis may be interpreted as a synthesis of randomized and indirect comparisons analogous to mixed treatment comparisons in network meta-analysis. This allows the use of network meta-analysis methodology to separate the randomized and nonrandomized components and to assess their consistency. Whenever nonconcurrent controls are used in platform trials, the separate randomized and indirect contributions to the treatment effect should be presented.

The impact of heterogeneity on the analysis of platform trials with normally distributed outcomes

BACKGROUND

A platform trial approach allows adding arms to on-going trials to speed up intervention discovery programs. A control arm remains open for recruitment in a platform trial while intervention arms may be added after the onset of the study and could be terminated early for efficacy and/or futility when early stopping is allowed. The topic of utilising non-concurrent control data in the analysis of platform trials has been explored and discussed extensively. A less familiar issue is the presence of heterogeneity, which may exist for example due to modification of enrolment criteria and recruitment strategy.

METHOD

We conduct a simulation study to explore the impact of heterogeneity on the analysis of a two-stage platform trial design. We consider heterogeneity in treatment effects and heteroscedasticity in outcome data across stages for a normally distributed endpoint. We examine the performance of some hypothesis testing procedures and modelling strategies. The use of non-concurrent control data is also considered accordingly. Alongside standard regression analysis, we examine the performance of a novel method that was known as the pairwise trials analysis. It is similar to a network meta-analysis approach but adjusts for treatment comparisons instead of individual studies using fixed effects.

RESULTS

Several testing strategies with concurrent control data seem to control the type I error rate at the required level when there is heteroscedasticity in outcome data across stages and/or a random cohort effect. The main parameter of treatment effects in some analysis models correspond to overall treatment effects weighted by stage wise sample sizes; while others correspond to the effect observed within a single stage. The characteristics of the estimates are not affected significantly by the presence of a random cohort effect and/ or heteroscedasticity.

CONCLUSION

In view of heterogeneity in treatment effect across stages, the specification of null hypotheses in platform trials may need to be more subtle. We suggest employing testing procedure of adaptive design as opposed to testing the statistics from regression models; comparing the estimates from the pairwise trials analysis method and the regression model with interaction terms may indicate if heterogeneity is negligible.

Statistical considerations for the platform trial in COVID-19 vaccine priming and boosting

The Platform trial In COVID-19 priming and BOOsting (PICOBOO) is a multi-site, adaptive platform trial designed to generate evidence of the immunogenicity, reactogenicity, and cross-protection of different booster vaccination strategies against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and its variants, specific for the Australian context. The PICOBOO trial randomises participants to receive one of three COVID-19 booster vaccine brands (Pfizer, Moderna, Novavax) available for use in Australia, where the vaccine brand subtypes vary over time according to the national vaccine roll out strategy, and employs a Bayesian hierarchical modelling approach to efficiently borrow information across consecutive booster doses, age groups and vaccine brand subtypes. Here, we briefly describe the PICOBOO trial structure and report the statistical considerations for the estimands, statistical models and decision making for trial adaptations. This paper should be read in conjunction with the PICOBOO Core Protocol and PICOBOO Sub-Study Protocol 1: Booster Vaccination. PICOBOO was registered on 10 February 2022 with the Australian and New Zealand Clinical Trials Registry ACTRN12622000238774.

How Much More Efficient Are Adaptive Platform Trials Than Multiple Stand-Alone Trials? A Comprehensive Simulation Study for Streamlining Drug Development During a Pandemic

With the coronavirus disease 2019 (COVID-19) pandemic, there is growing interest in utilizing adaptive platform clinical trials (APTs), in which multiple drugs are compared with a single common control group, such as a placebo or standard-of-care group. APTs evaluate several drugs for one disease and accept additions or exclusions of drugs as the trials progress; however, little is known about the efficiency of APTs over multiple stand-alone trials. In this study, we simulated the total development period, total sample size, and statistical operating characteristics of APTs and multiple stand-alone trials in drug development settings for hospitalized patients with COVID-19. Simulation studies using selected scenarios reconfirmed several findings regarding the efficiency of APTs. The APTs without staggered addition of drugs showed a shorter total development period than stand-alone trials, but the difference rapidly diminished if patient's enrollment was accelerated during the trials owing to the spread of infection. APTs with staggered addition of drugs still have the possibility of reducing the total development period compared with multiple stand-alone trials in some cases. Our study demonstrated that APTs could improve efficiency relative to multiple stand-alone trials regarding the total development period and total sample size without undermining statistical validity; however, this improvement varies depending on the speed of patient enrollment, sample size, presence/absence of family-wise error rate adjustment, allocation ratio between drug and placebo groups, and interval of staggered addition of drugs. Given the complexity of planning and implementing APT, the decision to implement APT during a pandemic must be made carefully.

A diagnostic phase III/IV seamless design to investigate the diagnostic accuracy and clinical effectiveness using the example of HEDOS and HEDOS II

The development process of medical devices can be streamlined by combining different study phases. Here, for a diagnostic medical device, we present the combination of confirmation of diagnostic accuracy (phase III) and evaluation of clinical effectiveness regarding patient-relevant endpoints (phase IV) using a seamless design. This approach is used in the Thyroid HEmorrhage DetectOr Study (HEDOS & HEDOS II) investigating a post-operative hemorrhage detector named ISAR-M THYRO® in patients after thyroid surgery. Data from the phase III trial are reused as external controls in the control group of the phase IV trial. An unblinded interim analysis is planned between the two study stages which includes a recalculation of the sample size for the phase IV part after completion of the first stage of the seamless design. The study concept presented here is the first seamless design proposed in the field of diagnostic studies. Hence, the aim of this work is to emphasize the statistical methodology as well as feasibility of the proposed design in relation to the planning and implementation of the seamless design. Seamless designs can accelerate the overall trial duration and increase its efficiency in terms of sample size and recruitment. However, careful planning addressing numerous methodological and procedural challenges is necessary for successful implementation as well as agreement with regulatory bodies.

Current state-of-the-art and gaps in platform trials: 10 things you should know, insights from EU-PEARL

Platform trials bring the promise of making clinical research more efficient and more patient centric. While their use has become more widespread, including their prominent role during the COVID-19 pandemic response, broader adoption of platform trials has been limited by the lack of experience and tools to navigate the critical upfront planning required to launch such collaborative studies. The European Union-Patient-cEntric clinicAl tRial pLatform (EU-PEARL) initiative has produced new methodologies to expand the use of platform trials with an overarching infrastructure and services embedded into Integrated Research Platforms (IRPs), in collaboration with patient representatives and through consultation with U.S. Food and Drug Administration and European Medicines Agency stakeholders. In this narrative review, we discuss the outlook for platform trials in Europe, including challenges related to infrastructure, design, adaptations, data sharing and regulation. Documents derived from the EU-PEARL project, alongside a literature search including PubMed and relevant grey literature (e.g., guidance from regulatory agencies and health technology agencies) were used as sources for a multi-stage collaborative process through which the 10 more important points based on lessons drawn from the EU-PEARL project were developed and summarised as guidance for the setup of platform trials. We conclude that early involvement of critical stakeholder such as regulatory agencies or patients are critical steps in the implementation and later acceptance of platform trials. Addressing these gaps will be critical for attaining the full potential of platform trials for patients.

Funding

Innovative Medicines Initiative 2 Joint Undertaking with support from the European Union's Horizon 2020 research and innovation programme and EFPIA.

A blueprint for a multi-disease, multi-domain Bayesian adaptive platform trial incorporating adult and paediatric subgroups: the Staphylococcus aureus Network Adaptive Platform trial

The Staphylococcus aureus Network Adaptive Platform (SNAP) trial is a multifactorial Bayesian adaptive platform trial that aims to improve the way that S. aureus bloodstream infection, a globally common and severe infectious disease, is treated. In a world first, the SNAP trial will simultaneously investigate the effects of multiple intervention modalities within multiple groups of participants with different forms of S. aureus bloodstream infection. Here, we formalise the trial structure, modelling approach, and decision rules that will be used for the SNAP trial. By summarising the statistical principles governing the design, our hope is that the SNAP trial will serve as an adaptable template that can be used to improve comparative effectiveness research efficiency in other disease areas.Trial registration NCT05137119 . Registered on 30 November 2021.

Transparent reporting of adaptive clinical trials using concurrently randomised cohorts

Adaptive clinical trials have designs that evolve over time because of changes to treatments or changes to the chance that participants will receive these treatments. These changes might introduce confounding that biases crude comparisons of the treatment arms and makes the results from standard reporting methods difficult to interpret for adaptive trials. To deal with this shortcoming, a reporting framework for adaptive trials was developed based on concurrently randomised cohort reporting. A concurrently randomised cohort is a subgroup of participants who all had the same treatments available and the same chance of receiving these treatments. The reporting of pre-randomisation characteristics and post-randomisation outcomes for each concurrently randomised cohort in the study is recommended. This approach provides a transparent and unbiased display of the degree of baseline balance and the randomised treatment comparisons for adaptive trials. The key concepts, terminology, and recommendations underlying concurrently randomised cohort reporting are presented, and its routine use in adaptive trial reporting is advocated.

On the use of non-concurrent controls in platform trials: a scoping review

BACKGROUND

Platform trials gained popularity during the last few years as they increase flexibility compared to multi-arm trials by allowing new experimental arms entering when the trial already started. Using a shared control group in platform trials increases the trial efficiency compared to separate trials. Because of the later entry of some of the experimental treatment arms, the shared control group includes concurrent and non-concurrent control data. For a given experimental arm, non-concurrent controls refer to patients allocated to the control arm before the arm enters the trial, while concurrent controls refer to control patients that are randomised concurrently to the experimental arm. Using non-concurrent controls can result in bias in the estimate in case of time trends if the appropriate methodology is not used and the assumptions are not met.

METHODS

We conducted two reviews on the use of non-concurrent controls in platform trials: one on statistical methodology and one on regulatory guidance. We broadened our searches to the use of external and historical control data. We conducted our review on the statistical methodology in 43 articles identified through a systematic search in PubMed and performed a review on regulatory guidance on the use of non-concurrent controls in 37 guidelines published on the EMA and FDA websites.

RESULTS

Only 7/43 of the methodological articles and 4/37 guidelines focused on platform trials. With respect to the statistical methodology, in 28/43 articles, a Bayesian approach was used to incorporate external/non-concurrent controls while 7/43 used a frequentist approach and 8/43 considered both. The majority of the articles considered a method that downweights the non-concurrent control in favour of concurrent control data (34/43), using for instance meta-analytic or propensity score approaches, and 11/43 considered a modelling-based approach, using regression models to incorporate non-concurrent control data. In regulatory guidelines, the use of non-concurrent control data was considered critical but was deemed acceptable for rare diseases in 12/37 guidelines or was accepted in specific indications (12/37). Non-comparability (30/37) and bias (16/37) were raised most often as the general concerns with non-concurrent controls. Indication specific guidelines were found to be most instructive.

CONCLUSIONS

Statistical methods for incorporating non-concurrent controls are available in the literature, either by means of methods originally proposed for the incorporation of external controls or non-concurrent controls in platform trials. Methods mainly differ with respect to how the concurrent and non-concurrent data are combined and temporary changes handled. Regulatory guidance for non-concurrent controls in platform trials are currently still limited.

Borrowing Concurrent Information from Non-Concurrent Control to Enhance Statistical Efficiency in Platform Trials

A platform trial is a trial involving an innovative adaptive design with a single master protocol to efficiently evaluate multiple interventions. It offers flexible features such as dropping interventions for futility and adding new interventions to be evaluated during the course of a trial. Although there is a consensus that platform trials can identify beneficial interventions with fewer patients, less time, and a higher probability of success than traditional trials, there remains debate on certain issues, one of which is whether (and how) the non-concurrent control (NCC) (i.e., patients in the control group recruited prior to the new interventions) can be combined with the current control (CC) in the analysis, especially if there is a change of standard of care during the trial. Methods: In this paper, considering time-to-event endpoints under the proportional hazard model assumption, we introduce a new concept of NCC concurrent observation time (NCC COT), and propose to borrow NCC COT through left truncation. This assumes that the NCC COT and CC are comparable. If the protocol does not prohibit NCC patients to change the standard of care while on study, NCC COT and CC likely will share the same standard of care. A simulated example is provided to demonstrate the approach. Results: Using exponential distributions, the simulated example assumes that NCC COT and CC have the same hazard, and the treatment group has a lower hazard. The estimated HR comparing treatment to the pooled control group is 0.744 (95% CI 0.575, 0.962), whereas the comparison to the CC group alone is 0.755 (95% CI 0.566, 1.008), with corresponding p-values of 0.024 versus 0.057, respectively. This suggests that borrowing NCC COT can improve statistical efficiency when the exchangeability assumption holds. Conclusion: This article proposes an innovative approach of borrowing NCC COT to enhance statistical inference in platform trials under appropriate scenarios.

Les essais plateformes☆

Les essais plateformes connaissent depuis quelques années un essor important, amplifié récemment par la pandémie de coronavirus disease 2019 (COVID-19). La mise en œuvre d’un essai plateforme s’avère particulièrement utile dans certaines pathologies, notamment lorsqu’il y a un nombre important de candidats médicaments à évaluer, une évolution rapide du traitement de référence ou dans les situations de besoin urgent d’évaluation, au cours desquelles la mutualisation des protocoles et des infrastructures permet d’optimiser le nombre de patients à inclure, les coûts et les délais de réalisation de l’investigation. Toutefois, la spécificité des essais plateformes soulève des problématiques méthodologiques, éthiques et règlementaires, qui ont fait l’objet de la table ronde et qui sont exposées dans cet article. La table ronde a également été l’occasion d’aborder la complexité de la promotion et de la gestion des données liée à la multiplicité des partenaires, le financement et la gouvernance de ces essais, et le niveau d’acceptabilité de leurs résultats par les autorités compétentes.

Platform trials

For the past few years, platform trials have experienced a significant increase, recently amplified by the COVID-19 pandemic. The implementation of a platform trial is particularly useful in certain pathologies, particularly when there is a significant number of drug candidates to be assessed, a rapid evolution of the standard of care or in situations of urgent need for evaluation, during which the pooling of protocols and infrastructure optimizes the number of patients to be enrolled, the costs, and the deadlines for carrying out the investigation. However, the specificity of platform trials raises methodological, ethical, and regulatory issues, which have been the subject of the round table and which are presented in this article. The round table was also an opportunity to discuss the complexity of sponsorship and data management related to the multiplicity of partners, funding, and governance of these trials, and the level of acceptability of their findings by the competent authorities.

Online control of the False Discovery Rate in group-sequential platform trials

When testing multiple hypotheses, a suitable error rate should be controlled even in exploratory trials. Conventional methods to control the False Discovery Rate assume that all p-values are available at the time point of test decision. In platform trials, however, treatment arms enter and leave the trial at different times during its conduct. Therefore, the actual number of treatments and hypothesis tests is not fixed in advance and hypotheses are not tested at once, but sequentially. Recently, for such a setting the concept of online control of the False Discovery Rate was introduced. We propose several heuristic variations of the LOND procedure (significance Levels based On Number of Discoveries) that incorporate interim analyses for platform trials, and study their online False Discovery Rate via simulations. To adjust for the interim looks spending functions are applied with O'Brien-Fleming or Pocock type group-sequential boundaries. The power depends on the prior distribution of effect sizes, for example, whether true alternatives are uniformly distributed over time or not. We consider the choice of design parameters for the LOND procedure to maximize the overall power and investigate the impact on the False Discovery Rate by including both concurrent and non-concurrent control data.