Online error rate control for platform trials

An exhaustive ADDIS principle for online FWER control

In this paper, we consider online multiple testing with familywise error rate (FWER) control, where the probability of committing at least one type I error will remain under control while testing a possibly infinite sequence of hypotheses over time. Currently, adaptive-discard (ADDIS) procedures seem to be the most promising online procedures with FWER control in terms of power. Now, our main contribution is a uniform improvement of the ADDIS principle and thus of all ADDIS procedures. This means, the methods we propose reject as least as much hypotheses as ADDIS procedures and in some cases even more, while maintaining FWER control. In addition, we show that there is no other FWER controlling procedure that enlarges the event of rejecting any hypothesis. Finally, we apply the new principle to derive uniform improvements of the ADDIS-Spending and ADDIS-Graph.

Regulatory Issues of Platform Trials: Learnings from EU-PEARL

Although platform trials have many benefits, the complexity of these designs may result not only in increased methodological but also regulatory and ethical challenges. These aspects were addressed as part of the IMI project EU Patient-Centric Clinical Trial Platforms (EU-PEARL). We reviewed the available guidelines on platform trials in the European Union and the United States. This is supported and complemented by feedback received from regulatory interactions with the European Medicines Agency and the US Food and Drug Administration. Throughout the project we collected the needs of all relevant stakeholders including ethics committees, regulators, and health technology assessment bodies through active dialog and dedicated stakeholder workshops. Furthermore, we focused on methodological aspects and where applicable identified the corresponding guidance. Learnings from the guideline review, regulatory interactions, and workshops are provided. Based on these, a master protocol template was developed. Issues that still need harmonization or clarification in guidelines or where further methodological research is needed are also presented. These include questions around clinical trial submissions in Europe, the need for multiplicity control across the whole master protocol, the use of non-concurrent controls, and the impact of different randomization schemes. Master protocols are an efficient and patient-centered clinical trial design that can expedite drug development. However, they can also introduce additional operational and regulatory complexities. It is important to understand the different requirements of stakeholders upfront and address them in the trial. While relevant guidance is increasing, early dialog with relevant stakeholders can help to further support such designs.

Developing generic templates to shape the future for conducting integrated research platform trials

BACKGROUND

Interventional clinical studies conducted in the regulated drug research environment are designed using International Council for Harmonisation (ICH) regulatory guidance documents: ICH E6 (R2) Good Clinical Practice-scientific guideline, first published in 2002 and last updated in 2016. This document provides an international ethical and scientific quality standard for designing and conducting trials that involve the participation of human subjects. Recently, there has been heightened awareness of the importance of integrated research platform trials (IRPs) designed to evaluate multiple therapies simultaneously. The use of a single master protocol as a key source document to fulfill trial conduct obligations has resulted in a re-examination of the templates used to fulfill the dynamic regulatory and modern drug development environment challenges.

METHODS

Regulatory medical writing, biostatistical, and other members of EU Patient-cEntric clinicAl tRial pLatforms (EU-PEARL) developed the suite of templates for IRPs over a 3.5-year period. Stakeholders contributing expertise included academic hospitals, pharmaceutical companies, non-governmental organizations, patient representative groups, and small and medium-sized enterprises (SMEs).

RESULTS

The suite of templates for IRPs based on TransCelerate's Common Protocol Template (CPT) and statistical analysis plan (SAP) should help authors navigate relevant guidelines as they create study design content relevant for today's IRP studies. It offers practical suggestions for adaptive platform designs which offer flexible features such as dropping treatments for futility or adding new treatments to be tested during a trial. The EU-PEARL suite of templates for IRPs comprises a preface, followed by the actual resource. The preface clarifies the intended use and underlying principles that inform resource utility. The preface lists references contributing to the development of the resource. The resource includes TransCelerate CPT guidance text, and EU-PEARL-derived guidance text, distinguished from one another using shading. Rationale comments are used throughout for clarification purposes. In addition, a user-friendly, functional, and informative Platform Trials Best Practices tool to support the setup, design, planning, implementation, and conduct of complex and innovative trials to support multi-sourced/multi-company platform trials is also provided. Together, the EU-PEARL suite of templates and the Platform Trials Best Practices tool constitute the reference user manual.

CONCLUSIONS

This publication is intended to enhance the use, understanding, and dissemination of the EU-PEARL suite of templates for designing IRPs. The reference user manual and the associated website ( http://www.eu-pearl ) should facilitate the designing of IRP trials.

Current state-of-the-art and gaps in platform trials: 10 things you should know, insights from EU-PEARL

Platform trials bring the promise of making clinical research more efficient and more patient centric. While their use has become more widespread, including their prominent role during the COVID-19 pandemic response, broader adoption of platform trials has been limited by the lack of experience and tools to navigate the critical upfront planning required to launch such collaborative studies. The European Union-Patient-cEntric clinicAl tRial pLatform (EU-PEARL) initiative has produced new methodologies to expand the use of platform trials with an overarching infrastructure and services embedded into Integrated Research Platforms (IRPs), in collaboration with patient representatives and through consultation with U.S. Food and Drug Administration and European Medicines Agency stakeholders. In this narrative review, we discuss the outlook for platform trials in Europe, including challenges related to infrastructure, design, adaptations, data sharing and regulation. Documents derived from the EU-PEARL project, alongside a literature search including PubMed and relevant grey literature (e.g., guidance from regulatory agencies and health technology agencies) were used as sources for a multi-stage collaborative process through which the 10 more important points based on lessons drawn from the EU-PEARL project were developed and summarised as guidance for the setup of platform trials. We conclude that early involvement of critical stakeholder such as regulatory agencies or patients are critical steps in the implementation and later acceptance of platform trials. Addressing these gaps will be critical for attaining the full potential of platform trials for patients.

Funding

Innovative Medicines Initiative 2 Joint Undertaking with support from the European Union's Horizon 2020 research and innovation programme and EFPIA.

Online multiple hypothesis testing

Modern data analysis frequently involves large-scale hypothesis testing, which naturally gives rise to the problem of maintaining control of a suitable type I error rate, such as the false discovery rate (FDR). In many biomedical and technological applications, an additional complexity is that hypotheses are tested in an online manner, one-by-one over time. However, traditional procedures that control the FDR, such as the Benjamini-Hochberg procedure, assume that all p-values are available to be tested at a single time point. To address these challenges, a new field of methodology has developed over the past 15 years showing how to control error rates for online multiple hypothesis testing. In this framework, hypotheses arrive in a stream, and at each time point the analyst decides whether to reject the current hypothesis based both on the evidence against it, and on the previous rejection decisions. In this paper, we present a comprehensive exposition of the literature on online error rate control, with a review of key theory as well as a focus on applied examples. We also provide simulation results comparing different online testing algorithms and an up-to-date overview of the many methodological extensions that have been proposed.