Application of sample size re-estimation in clinical trials: A systematic review

Background

Sample size re-estimation (SSR) is a method used to recalculate sample size during clinical trial conduct to address a lack of adequate information and can have a significant impact on study size, duration, resources, and cost. Few studies to date have summarized the conditions and circumstances under which SSR is applied. We therefore performed a systematic review of the literature related to SSR to better understand its application in clinical trial settings.

Methods

PubMed was used as the primary search source, supplemented with information from ClinicalTrials.gov where necessary details were lacking from PubMed. A systematic review was performed according to a pre-specified search strategy to identify clinical trials using SSR. Features of SSR, such as study phase and study start year, were summarized.

Results

In total, 253 publications met the pre-specified search criteria and 27 clinical trials were subsequently determined as relevant in SSR usage. Among trials where the study phase was provided, 2 (7.4%) trials were Phase I, 5 (18.5%) trials were Phase II, 11 (40.7%) trials were Phase III, and 2 (7.4%) trials were Phase IV.

Conclusion

Our results showed that SSR is also used in Phase I and II, which involve earlier decision making. We expect that SSR will continue to be used in early-phase trials where sufficient prior information may not be available. Furthermore, no major trends were observed in relation to therapy area or type of SSR, meaning that SSR may become a feasible and widely applied method in the future.

Adaptive design clinical trials: current status by disease and trial phase in various perspectives

An adaptive design is a clinical trial design that allows for modification of a structured plan in a clinical trial based on data accumulated during pre-planned interim analyses. This flexible approach to clinical trial design improves the success rate of clinical trials while reducing time, cost, and sample size compared to conventional methods. The purpose of this study is to identify the current status of adaptive design and present key considerations for planning an appropriate adaptive design based on specific circumstances. We searched for clinical trials conducted between January 2006 to July 2021 in the Clinical Trials Registry (ClinicalTrials.gov) using keywords specified in the Food and Drug Administration Adaptive Design Clinical Trial Guidelines. In order to analyze the adaptive designs used in selected cases, we classified the results according to the phase of the clinical trial, type of indication, and the specific adaptation method employed. A total of 267 clinical trials were identified on ClinicalTrials.gov. Among them, 236 clinical trials actually applied adaptive designs and were classified according to phase, indication types, and adaptation methods. Adaptive designs were most frequently used in phase 2 clinical trials and oncology research. The most commonly used adaptation method was the adaptive treatment selection design. In the case of coronavirus disease 2019, the most frequently used designs were adaptive platform design and seamless design. Through this study, we expect to provide valuable insights and considerations for the implementation of adaptive design clinical trials in different diseases and stages.

Sample size re-estimation in Phase 2 dose-finding: Conditional power versus Bayesian predictive power

Unblinded sample size re-estimation (SSR) is often planned in a clinical trial when there is large uncertainty about the true treatment effect. For Proof-of Concept (PoC) in a Phase II dose finding study, contrast test can be adopted to leverage information from all treatment groups. In this article, we propose two-stage SSR designs using frequentist conditional power (CP) and Bayesian predictive power (PP) for both single and multiple contrast tests. The Bayesian SSR can be implemented under a wide range of prior settings to incorporate different prior knowledge. Taking the adaptivity into account, all type I errors of final analysis in this paper are rigorously protected. Simulation studies are carried out to demonstrate the advantages of unblinded SSR in multi-arm trials.

From innovative thinking to pharmaceutical industry implementation: Some success stories

In industry, successful innovation involves not only developing new statistical methodology, but also ensuring that this methodology is implemented successfully. This includes enabling applied statisticians to understand the method, its benefits and limitations and empowering them to implement the new method. This will include advocacy, influencing in-house and external stakeholders, such that these stakeholders are receptive to the new methodology. In this paper, we describe some industry successes and focus on our colleague, Andy Grieve's role in these.

Practical recommendations for implementing a Bayesian adaptive phase I design during a pandemic

BACKGROUND

Modern designs for dose-finding studies (e.g., model-based designs such as continual reassessment method) have been shown to substantially improve the ability to determine a suitable dose for efficacy testing when compared to traditional designs such as the 3 + 3 design. However, implementing such designs requires time and specialist knowledge.

METHODS

We present a practical approach to developing a model-based design to help support uptake of these methods; in particular, we lay out how to derive the necessary parameters and who should input, and when, to these decisions. Designing a model-based, dose-finding trial is demonstrated using a treatment within the AGILE platform trial, a phase I/II adaptive design for novel COVID-19 treatments.

RESULTS

We present discussion of the practical delivery of AGILE, covering what information was found to support principled decision making by the Safety Review Committee, and what could be contained within a statistical analysis plan. We also discuss additional challenges we encountered in the study and discuss more generally what (unplanned) adaptations may be acceptable (or not) in studies using model-based designs.

CONCLUSIONS

This example demonstrates both how to design and deliver an adaptive dose-finding trial in order to support uptake of these methods.

Seven-Year Experience From the National Institute of Neurological Disorders and Stroke-Supported Network for Excellence in Neuroscience Clinical Trials

Importance

One major advantage of developing large, federally funded networks for clinical research in neurology is the ability to have a trial-ready network that can efficiently conduct scientifically rigorous projects to improve the health of people with neurologic disorders.

Observations

National Institute of Neurological Disorders and Stroke Network for Excellence in Neuroscience Clinical Trials (NeuroNEXT) was established in 2011 and renewed in 2018 with the goal of being an efficient network to test between 5 and 7 promising new agents in phase II clinical trials. A clinical coordinating center, data coordinating center, and 25 sites were competitively chosen. Common infrastructure was developed to accelerate timelines for clinical trials, including central institutional review board (a first for the National Institute of Neurological Disorders and Stroke), master clinical trial agreements, the use of common data elements, and experienced research sites and coordination centers. During the first 7 years, the network exceeded the goal of conducting 5 to 7 studies, with 9 funded. High interest was evident by receipt of 148 initial applications for potential studies in various neurologic disorders. Across the first 8 studies (the ninth study was funded at end of initial funding period), the central institutional review board approved the initial protocol in a mean (SD) of 59 (21) days, and additional sites were added a mean (SD) of 22 (18) days after submission. The median time from central institutional review board approval to first site activation was 47.5 days (mean, 102.1; range, 1-282) and from first site activation to first participant consent was 27 days (mean, 37.5; range, 0-96). The median time for database readiness was 3.5 months (mean, 4.0; range, 0-8) from funding receipt. In the 4 completed studies, enrollment met or exceeded expectations with 96% overall data accuracy across all sites. Nine peer-reviewed manuscripts were published, and 22 oral presentations or posters and 9 invited presentations were given at regional, national, and international meetings.

Conclusions and Relevance

NeuroNEXT initiated 8 studies, successfully enrolled participants at or ahead of schedule, collected high-quality data, published primary results in high-impact journals, and provided mentorship, expert statistical, and trial management support to several new investigators. Partnerships were successfully created between government, academia, industry, foundations, and patient advocacy groups. Clinical trial consortia can efficiently and successfully address a range of important neurologic research and therapeutic questions.

Adaptive platform trials using multi-arm, multi-stage protocols: getting fast answers in pandemic settings

Global health pandemics, such as coronavirus disease 2019 (COVID-19), require efficient and well-conducted trials to determine effective interventions, such as treatments and vaccinations. Early work focused on rapid sequencing of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), subsequent in-vitro and in-silico work, along with greater understanding of the different clinical phases of the infection, have helped identify a catalogue of potential therapeutic agents requiring assessment. In a pandemic, there is a need to quickly identify efficacious treatments, and reject those that are non-beneficial or even harmful, using randomised clinical trials. Whilst each potential treatment could be investigated across multiple, separate, competing two-arm trials, this is a very inefficient process. Despite the very large numbers of interventional trials for COVID-19, the vast majority have not used efficient trial designs. Well conducted, adaptive platform trials utilising a multi-arm multi-stage (MAMS) approach provide a solution to overcome limitations of traditional designs. The multi-arm element allows multiple different treatments to be investigated simultaneously against a shared, standard-of-care control arm. The multi-stage element uses interim analyses to assess accumulating data from the trial and ensure that only treatments showing promise continue to recruitment during the next stage of the trial. The ability to test many treatments at once and drop insufficiently active interventions significantly speeds up the rate at which answers can be achieved. This article provides an overview of the benefits of MAMS designs and successes of trials, which have used this approach to COVID-19. We also discuss international collaboration between trial teams, including prospective agreement to synthesise trial results, and identify the most effective interventions. We believe that international collaboration will help provide faster answers for patients, clinicians, and health care systems around the world, including for future waves of COVID-19, and enable preparedness for future global health pandemics.

Adaptive platform trials using multi-arm, multi-stage protocols: getting fast answers in pandemic settings

Global health pandemics, such as coronavirus disease 2019 (COVID-19), require efficient and well-conducted trials to determine effective interventions, such as treatments and vaccinations. Early work focused on rapid sequencing of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), subsequent in-vitro and in-silico work, along with greater understanding of the different clinical phases of the infection, have helped identify a catalogue of potential therapeutic agents requiring assessment. In a pandemic, there is a need to quickly identify efficacious treatments, and reject those that are non-beneficial or even harmful, using randomised clinical trials. Whilst each potential treatment could be investigated across multiple, separate, competing two-arm trials, this is a very inefficient process. Despite the very large numbers of interventional trials for COVID-19, the vast majority have not used efficient trial designs. Well conducted, adaptive platform trials utilising a multi-arm multi-stage (MAMS) approach provide a solution to overcome limitations of traditional designs. The multi-arm element allows multiple different treatments to be investigated simultaneously against a shared, standard-of-care control arm. The multi-stage element uses interim analyses to assess accumulating data from the trial and ensure that only treatments showing promise continue to recruitment during the next stage of the trial. The ability to test many treatments at once and drop insufficiently active interventions significantly speeds up the rate at which answers can be achieved. This article provides an overview of the benefits of MAMS designs and successes of trials, which have used this approach to COVID-19. We also discuss international collaboration between trial teams, including prospective agreement to synthesise trial results, and identify the most effective interventions. We believe that international collaboration will help provide faster answers for patients, clinicians, and health care systems around the world, including for each further wave of COVID-19, and enable preparedness for future global health pandemics.

Adaptive design implementation in confirmatory trials: methods, practical considerations and case studies

The rapidly changing drug development landscapes have brought unique challenges to sponsors in designing clinical trials in a faster and more efficient way. With the ability to accelerate development timeline, reduce redundant sample size, and select the right dose and patient population during the clinical trial, adaptive designs help to increase the probability of success of clinical trials and eventually contribute to bringing the promising drugs to patients earlier and fulfilling their unmet medical needs. Although extensive adaptive design methods have been proposed in recent years, a comprehensive review of how to implement adaptive design in the practical confirmatory trials is still lacking. In this paper, we will review the evolving history of adaptive designs, updates of newly released regulatory guidance and emerging practical adaptive designs, including but not limited to sample size re-estimation, seamless design and surrogate endpoint used in the interim analysis. Furthermore, we will discuss the current practice of adaptive design implementation by demonstrating a complex oncology seamless phase 2/3 adaptive design case study. Through this example, we will introduce the critical roles of each cross disciplinary function, communication process and important documents when adaptive designs are implemented in real-world setting.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

Stakeholder perspectives on adaptive clinical trials: a scoping review

Background

Adaptive clinical trials (ACTs) represent an emerging approach to trial design where accumulating data are used to make decisions about future conduct. Adaptations can include comparisons of multiple dose tiers, response-adaptive randomization, sample size re-estimation, and efficacy/futility stopping rules. The objective of this scoping review is to assess stakeholder attitudes, perspectives, and understanding of adaptive trials.

Methods

We conducted a review of articles examining stakeholders encompassing the broad medical trial community’s perspectives of adaptive designs (ADs). A computerized search was conducted of four electronic databases with relevant search terms. Following screening of articles, the primary findings of each included article were coded for study design, population studied, purpose, and primary implications.

Results

Our team retrieved 167 peer-reviewed titles in total from the database search and 5 additional titles through searching web-based search engines for gray literature. Of those 172 titles, 152 were non-duplicate citations. Of these, 119 were not given full-text reviews, as their titles and abstracts indicated that they did not meet the inclusion criteria. Thirty-three articles were carefully examined for relevance, and of those, 18 were chosen to be part of the analysis; the other 15 were excluded, as they were not relevant upon closer inspection.

Perceived advantages to ADs included limiting ineffective treatments and efficiency in answering the research question; −perceived barriers included insufficient sample size for secondary outcomes, challenges of consent, potential for bias, risk of type 1 error, cost and time to adaptively design trials, unclear rationales for using Ads, and, most importantly, a lack of education regarding ADs among stakeholders within the clinical trial community. Perceptions among different types of stakeholders varied from sector to sector, with patient perspectives being noticeably absent from the literature.

Conclusion

There are diverse perceptions regarding ADs among stakeholders. Further training, guidelines, and toolkits on the proper use of ADs are needed at all levels to overcome many of these perceived barriers. While education for principal investigators is important, it is also crucial to educate other groups in the community, such as patients, as well as clinicians and staff involved in their daily implementation.

A review of available software for adaptive clinical trial design

BACKGROUND/AIMS

The increasing cost of the drug development process has seen interest in the use of adaptive trial designs grow substantially. Accordingly, much research has been conducted to identify barriers to increasing the use of adaptive designs in practice. Several articles have argued that the availability of user-friendly software will be an important step in making adaptive designs easier to implement. Therefore, we present a review of the current state of software availability for adaptive trial design.

METHODS

We review articles from 31 journals published in 2013-2017 that relate to methodology for adaptive trials to assess how often code and software for implementing novel adaptive designs is made available at the time of publication. We contrast our findings against these journals' policies on code distribution. We also search popular code repositories, such as Comprehensive R Archive Network and GitHub, to identify further existing user-contributed software for adaptive designs. From this, we are able to direct interested parties toward solutions for their problem of interest.

RESULTS

Only 30% of included articles made their code available in some form. In many instances, articles published in journals that had mandatory requirements on code provision still did not make code available. There are several areas in which available software is currently limited or saturated. In particular, many packages are available to address group sequential design, but comparatively little code is present in the public domain to determine biomarker-guided adaptive designs.

CONCLUSIONS

There is much room for improvement in the provision of software alongside adaptive design publications. In addition, while progress has been made, well-established software for various types of trial adaptation remains sparsely available.

Development process of a consensus-driven CONSORT extension for randomised trials using an adaptive design

BACKGROUND

Adequate reporting of adaptive designs (ADs) maximises their potential benefits in the conduct of clinical trials. Transparent reporting can help address some obstacles and concerns relating to the use of ADs. Currently, there are deficiencies in the reporting of AD trials. To overcome this, we have developed a consensus-driven extension to the CONSORT statement for randomised trials using an AD. This paper describes the processes and methods used to develop this extension rather than detailed explanation of the guideline.

METHODS

We developed the guideline in seven overlapping stages: 1) Building on prior research to inform the need for a guideline; 2) A scoping literature review to inform future stages; 3) Drafting the first checklist version involving an External Expert Panel; 4) A two-round Delphi process involving international, multidisciplinary, and cross-sector key stakeholders; 5) A consensus meeting to advise which reporting items to retain through voting, and to discuss the structure of what to include in the supporting explanation and elaboration (E&E) document; 6) Refining and finalising the checklist; and 7) Writing-up and dissemination of the E&E document. The CONSORT Executive Group oversaw the entire development process.

RESULTS

Delphi survey response rates were 94/143 (66%), 114/156 (73%), and 79/143 (55%) in rounds 1, 2, and across both rounds, respectively. Twenty-seven delegates from Europe, the USA, and Asia attended the consensus meeting. The main checklist has seven new and nine modified items and six unchanged items with expanded E&E text to clarify further considerations for ADs. The abstract checklist has one new and one modified item together with an unchanged item with expanded E&E text. The E&E document will describe the scope of the guideline, the definition of an AD, and some types of ADs and trial adaptations and explain each reporting item in detail including case studies.

CONCLUSIONS

We hope that making the development processes, methods, and all supporting information that aided decision-making transparent will enhance the acceptability and quick uptake of the guideline. This will also help other groups when developing similar CONSORT extensions. The guideline is applicable to all randomised trials with an AD and contains minimum reporting requirements.

Adaptive designs in clinical trials: why use them, and how to run and report them

Adaptive designs can make clinical trials more flexible by utilising results accumulating in the trial to modify the trial's course in accordance with pre-specified rules. Trials with an adaptive design are often more efficient, informative and ethical than trials with a traditional fixed design since they often make better use of resources such as time and money, and might require fewer participants. Adaptive designs can be applied across all phases of clinical research, from early-phase dose escalation to confirmatory trials. The pace of the uptake of adaptive designs in clinical research, however, has remained well behind that of the statistical literature introducing new methods and highlighting their potential advantages. We speculate that one factor contributing to this is that the full range of adaptations available to trial designs, as well as their goals, advantages and limitations, remains unfamiliar to many parts of the clinical community. Additionally, the term adaptive design has been misleadingly used as an all-encompassing label to refer to certain methods that could be deemed controversial or that have been inadequately implemented.We believe that even if the planning and analysis of a trial is undertaken by an expert statistician, it is essential that the investigators understand the implications of using an adaptive design, for example, what the practical challenges are, what can (and cannot) be inferred from the results of such a trial, and how to report and communicate the results. This tutorial paper provides guidance on key aspects of adaptive designs that are relevant to clinical triallists. We explain the basic rationale behind adaptive designs, clarify ambiguous terminology and summarise the utility and pitfalls of adaptive designs. We discuss practical aspects around funding, ethical approval, treatment supply and communication with stakeholders and trial participants. Our focus, however, is on the interpretation and reporting of results from adaptive design trials, which we consider vital for anyone involved in medical research. We emphasise the general principles of transparency and reproducibility and suggest how best to put them into practice.

Adaptive design clinical trials: a review of the literature and ClinicalTrials.gov

OBJECTIVES

This review investigates characteristics of implemented adaptive design clinical trials and provides examples of regulatory experience with such trials.

DESIGN

Review of adaptive design clinical trials in EMBASE, PubMed, Cochrane Registry of Controlled Clinical Trials, Web of Science and ClinicalTrials.gov. Phase I and seamless Phase I/II trials were excluded. Variables extracted from trials included basic study characteristics, adaptive design features, size and use of independent data monitoring committees (DMCs) and blinded interim analyses. We also examined use of the adaptive trials in new drug submissions to the Food and Drug Administration (FDA) and European Medicines Agency (EMA) and recorded regulators' experiences with adaptive designs.

RESULTS

142 studies met inclusion criteria. There has been a recent growth in publicly reported use of adaptive designs among researchers around the world. The most frequently appearing types of adaptations were seamless Phase II/III (57%), group sequential (21%), biomarker adaptive (20%), and adaptive dose-finding designs (16%). About one-third (32%) of trials reported an independent DMC, while 6% reported blinded interim analysis. We found that 9% of adaptive trials were used for FDA product approval consideration, and 12% were used for EMA product approval consideration. International regulators had mixed experiences with adaptive trials. Many product applications with adaptive trials had extensive correspondence between drug sponsors and regulators regarding the adaptive designs, in some cases with regulators requiring revisions or alterations to research designs.

CONCLUSIONS

Wider use of adaptive designs will necessitate new drug application sponsors to engage with regulatory scientists during planning and conduct of the trials. Investigators need to more consistently report protections intended to preserve confidentiality and minimise potential operational bias during interim analysis.

Practical characteristics of adaptive design in phase 2 and 3 clinical trials

WHAT IS KNOWN AND OBJECTIVE

Adaptive design methods are expected to be ethical, reflect real medical practice, increase the likelihood of research and development success and reduce the allocation of patients into ineffective treatment groups by the early termination of clinical trials. However, the comprehensive details regarding which types of clinical trials will include adaptive designs remain unclear. We examined the practical characteristics of adaptive design used in clinical trials.

METHODS

We conducted a literature search of adaptive design clinical trials published from 2012 to 2015 using PubMed, EMBASE, and the Cochrane Central Register of Controlled Trials, with common search terms related to adaptive design. We systematically assessed the types and characteristics of adaptive designs and disease areas employed in the adaptive design trials.

RESULTS AND DISCUSSION

Our survey identified 245 adaptive design clinical trials. The number of trials by the publication year increased from 2012 to 2013 and did not greatly change afterwards. The most frequently used adaptive design was group sequential design (n = 222, 90.6%), especially for neoplasm or cardiovascular disease trials. Among the other types of adaptive design, adaptive dose/treatment group selection (n = 21, 8.6%) and adaptive sample-size adjustment (n = 19, 7.8%) were frequently used. The adaptive randomization (n = 8, 3.3%) and adaptive seamless design (n = 6, 2.4%) were less frequent. Adaptive dose/treatment group selection and adaptive sample-size adjustment were frequently used (up to 23%) in "certain infectious and parasitic diseases," "diseases of nervous system," and "mental and behavioural disorders" in comparison with "neoplasms" (<6.6%). For "mental and behavioural disorders," adaptive randomization was used in two trials of eight trials in total (25%). Group sequential design and adaptive sample-size adjustment were used frequently in phase 3 trials or in trials where study phase was not specified, whereas the other types of adaptive designs were used more in phase 2 trials. Approximately 82% (202 of 245 trials) resulted in early termination at the interim analysis. Among the 202 trials, 132 (54% of 245 trials) had fewer randomized patients than initially planned. This result supports the motive to use adaptive design to make study durations shorter and include a smaller number of subjects.

WHAT IS NEW AND CONCLUSION

We found that adaptive designs have been applied to clinical trials in various therapeutic areas and interventions. The applications were frequently reported in neoplasm or cardiovascular clinical trials. The adaptive dose/treatment group selection and sample-size adjustment are increasingly common, and these adaptations generally follow the Food and Drug Administration's (FDA's) recommendations.

Embracing model-based designs for dose-finding trials

BACKGROUND

Dose-finding trials are essential to drug development as they establish recommended doses for later-phase testing. We aim to motivate wider use of model-based designs for dose finding, such as the continual reassessment method (CRM).

METHODS

We carried out a literature review of dose-finding designs and conducted a survey to identify perceived barriers to their implementation.

RESULTS

We describe the benefits of model-based designs (flexibility, superior operating characteristics, extended scope), their current uptake, and existing resources. The most prominent barriers to implementation of a model-based design were lack of suitable training, chief investigators' preference for algorithm-based designs (e.g., 3+3), and limited resources for study design before funding. We use a real-world example to illustrate how these barriers can be overcome.

CONCLUSIONS

There is overwhelming evidence for the benefits of CRM. Many leading pharmaceutical companies routinely implement model-based designs. Our analysis identified barriers for academic statisticians and clinical academics in mirroring the progress industry has made in trial design. Unified support from funders, regulators, and journal editors could result in more accurate doses for later-phase testing, and increase the efficiency and success of clinical drug development. We give recommendations for increasing the uptake of model-based designs for dose-finding trials in academia.

A conceptual model for the development process of confirmatory adaptive clinical trials within an emergency research network

BACKGROUND

Adaptive clinical trials use accumulating data from enrolled subjects to alter trial conduct in pre-specified ways based on quantitative decision rules. In this research, we sought to characterize the perspectives of key stakeholders during the development process of confirmatory-phase adaptive clinical trials within an emergency clinical trials network and to build a model to guide future development of adaptive clinical trials.

METHODS

We used an ethnographic, qualitative approach to evaluate key stakeholders' views about the adaptive clinical trial development process. Stakeholders participated in a series of multidisciplinary meetings during the development of five adaptive clinical trials and completed a Strengths-Weaknesses-Opportunities-Threats questionnaire. In the analysis, we elucidated overarching themes across the stakeholders' responses to develop a conceptual model.

RESULTS

Four major overarching themes emerged during the analysis of stakeholders' responses to questioning: the perceived statistical complexity of adaptive clinical trials and the roles of collaboration, communication, and time during the development process. Frequent and open communication and collaboration were viewed by stakeholders as critical during the development process, as were the careful management of time and logistical issues related to the complexity of planning adaptive clinical trials.

CONCLUSION

The Adaptive Design Development Model illustrates how statistical complexity, time, communication, and collaboration are moderating factors in the adaptive design development process. The intensity and iterative nature of this process underscores the need for funding mechanisms for the development of novel trial proposals in academic settings.

CBER's Experience With Adaptive Design Clinical Trials

There is considerable interest among pharmaceutical and other medical product developers in adaptive clinical trials, in which knowledge learned during the course of a trial affects ongoing conduct or analysis of the trial. When the FDA released a draft Guidance document on adaptive design clinical trials in early 2010, expectations were high that it would lead to an increase in regulatory submissions involving adaptive design features, particularly for confirmatory trials. A 6-year (2008-2013) retrospective survey was performed within the Center for Biologics Evaluation and Research (CBER) at the FDA to gather information regarding the submission and evaluation of adaptive design trial proposals. We present an up-to-date summary of adaptive design proposals seen in CBER and provide an overview of our experiences. We share our concerns regarding the statistical issues and operational challenges raised during the review process for adaptive design trials. We also provide general recommendations for developing proposals for such trials. Our motivation in writing this paper was to encourage the best study design proposals to be submitted to CBER. Sometimes these can be adaptive, and sometimes a simpler design is most efficient.

DIA's Adaptive Design Scientific Working Group (ADSWG): Best Practices Case Studies for "Less Well-understood" Adaptive Designs

Adaptive design (AD) clinical trials use accumulating subject data to modify the parameters of the design of an ongoing study, without compromising the validity and integrity of the study. The 2010 US Food and Drug Administration (FDA) Draft Guidance on Adaptive Design Clinical Trials described a subset of 7 primary design types as "less well-understood." FDA defined these designs as those with limited regulatory experience. To better understand the properties of these less well-understood ADs and to promote their use when applicable, the Best Practices Subteam for DIA's Adaptive Design Scientific Working Group conducted an extensive nonsystematic search and reviewed trials from multiple sponsors who had employed these designs. Here, we review 10 specific case studies for which less well-understood ADs were employed and share feedback about their challenges and successes, as well as details about the regulatory interactions from these trials. We learned that these designs and associated statistical methodologies can make difficult research situations more amenable for study and, therefore, are needed in our toolbox. While they can be used to study many diseases, they are particularly valuable for rare diseases, small populations, studies involving terminal illnesses, and vaccine trials, in which it is important to find efficient ways to bring effective treatments to market more rapidly. It is imperative, however, that these methodologies be utilized appropriately, which requires careful planning and precise operational execution.

Attitudes and opinions regarding confirmatory adaptive clinical trials: a mixed methods analysis from the Adaptive Designs Accelerating Promising Trials into Treatments (ADAPT-IT) project

Background

Adaptive designs have been increasingly used in the pharmaceutical and device industries, but adoption within the academic setting has been less widespread — particularly for confirmatory phase trials. We sought to understand perceptions about understanding, acceptability, and scientific validity of adaptive clinical trials (ACTs).

Methods

We used a convergent mixed methods design using survey and mini-focus group data collection procedures to elucidate attitudes and opinions among “trial community” stakeholders regarding understanding, acceptability, efficiency, scientific validity, and speed of discovery with adaptive designs. Data were collected about various aspects of ACTs using self-administered surveys (paper or Web-based) with visual analog scales (VASs) with free text responses and with mini-focus groups of key stakeholders. Participants were recruited as part of an ongoing NIH/FDA-funded research project exploring the incorporation of ACTs into an existing NIH network that focuses on confirmatory phase clinical trials in neurological emergencies. “Trial community” representatives, namely, clinical investigators, biostatisticians, NIH officials, and FDA scientists involved in the planning of four clinical trials, were eligible to participate. In addition, recent and current members of a clinical trial-oriented NIH study section were also eligible.

Results

A total of 76 stakeholders completed the survey (out of 91 who were offered it, response rate 84 %). While the VAS attitudinal data showed substantial variability across respondents about acceptability and understanding of ACTs by various constituencies, respondents perceived clinicians to be less likely to understand ACTs and that ACTs probably would increase the efficiency of discovery. Textual and focus group responses emerged into several themes that enhanced understanding of VAS attitudinal data including the following: acceptability of adaptive designs depends on constituency and situation; there is variable understanding of ACTs (limited among clinicians, perceived to be higher at FDA); views about the potential for efficiency depend on the situation and implementation. Participants also frequently mentioned a need for greater education within the academic community. Finally, the empiric, non-quantitative selection of treatments for phase III trials based on limited phase II trials was highlighted as an opportunity for improvement and a potential explanation for the high number of neutral confirmatory trials.

Conclusions

These data show considerable variations in attitudes and beliefs about ACTs among trial community representatives. For adaptive trials to be fully considered when appropriate and for the research enterprise to realize the full potential of adaptive designs will likely require extensive experience and trust building within the trial community.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1493-z) contains supplementary material, which is available to authorized users.

Adaptive Design Practice at the Center for Devices and Radiological Health (CDRH), January 2007 to May 2013

Adaptive designs have generated great interest in the clinical trial community as a result of their versatility and efficiency. Recently, the Center for Devices and Radiological Health (CDRH) at the US Food and Drug Administration (FDA) surveyed all adaptive design applications submitted between 2007 and May 2013 for regulatory review. In this paper, we discuss the overall results and findings that emerged from an in-depth examination of the submissions. We summarize the current status of adaptive designs used in medical device studies. We also identify some of the lessons learned and common pitfalls that we encountered in our review of the designs.

Adaptive designs undertaken in clinical research: a review of registered clinical trials

Adaptive designs have the potential to improve efficiency in the evaluation of new medical treatments in comparison to traditional fixed sample size designs. However, they are still not widely used in practice in clinical research. Little research has been conducted to investigate what adaptive designs are being undertaken. This review highlights the current state of registered adaptive designs and their characteristics. The review looked at phase II, II/III and III trials registered on ClinicalTrials.gov from 29 February 2000 to 1 June 2014, supplemented with trials from the National Institute for Health Research register and known adaptive trials. A range of adaptive design search terms were applied to the trials extracted from each database. Characteristics of the adaptive designs were then recorded including funder, therapeutic area and type of adaptation. The results in the paper suggest that the use of adaptive designs has increased. They seem to be most often used in phase II trials and in oncology. In phase III trials, the most popular form of adaptation is the group sequential design. The review failed to capture all trials with adaptive designs, which suggests that the reporting of adaptive designs, such as in clinical trials registers, needs much improving. We recommend that clinical trial registers should contain sections dedicated to the type and scope of the adaptation and that the term 'adaptive design' should be included in the trial title or at least in the brief summary or design sections.

Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

Cross-sector surveys assessing perceptions of key stakeholders towards barriers, concerns and facilitators to the appropriate use of adaptive designs in confirmatory trials

Background

Appropriately conducted adaptive designs (ADs) offer many potential advantages over conventional trials. They make better use of accruing data, potentially saving time, trial participants, and limited resources compared to conventional, fixed sample size designs. However, one can argue that ADs are not implemented as often as they should be, particularly in publicly funded confirmatory trials. This study explored barriers, concerns, and potential facilitators to the appropriate use of ADs in confirmatory trials among key stakeholders.

Methods

We conducted three cross-sectional, online parallel surveys between November 2014 and January 2015. The surveys were based upon findings drawn from in-depth interviews of key research stakeholders, predominantly in the UK, and targeted Clinical Trials Units (CTUs), public funders, and private sector organisations. Response rates were as follows: 30(55 %) UK CTUs, 17(68 %) private sector, and 86(41 %) public funders. A Rating Scale Model was used to rank barriers and concerns in order of perceived importance for prioritisation.

Results

Top-ranked barriers included the lack of bridge funding accessible to UK CTUs to support the design of ADs, limited practical implementation knowledge, preference for traditional mainstream designs, difficulties in marketing ADs to key stakeholders, time constraints to support ADs relative to competing priorities, lack of applied training, and insufficient access to case studies of undertaken ADs to facilitate practical learning and successful implementation. Associated practical complexities and inadequate data management infrastructure to support ADs were reported as more pronounced in the private sector. For funders of public research, the inadequate description of the rationale, scope, and decision-making criteria to guide the planned AD in grant proposals by researchers were all viewed as major obstacles.

Conclusions

There are still persistent and important perceptions of individual and organisational obstacles hampering the use of ADs in confirmatory trials research. Stakeholder perceptions about barriers are largely consistent across sectors, with a few exceptions that reflect differences in organisations’ funding structures, experiences and characterisation of study interventions. Most barriers appear connected to a lack of practical implementation knowledge and applied training, and limited access to case studies to facilitate practical learning.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-015-1119-x) contains supplementary material, which is available to authorized users.

An Investigation of the Shortcomings of the CONSORT 2010 Statement for the Reporting of Group Sequential Randomised Controlled Trials: A Methodological Systematic Review

BACKGROUND

It can be argued that adaptive designs are underused in clinical research. We have explored concerns related to inadequate reporting of such trials, which may influence their uptake. Through a careful examination of the literature, we evaluated the standards of reporting of group sequential (GS) randomised controlled trials, one form of a confirmatory adaptive design.

METHODS

We undertook a systematic review, by searching Ovid MEDLINE from the 1st January 2001 to 23rd September 2014, supplemented with trials from an audit study. We included parallel group, confirmatory, GS trials that were prospectively designed using a Frequentist approach. Eligible trials were examined for compliance in their reporting against the CONSORT 2010 checklist. In addition, as part of our evaluation, we developed a supplementary checklist to explicitly capture group sequential specific reporting aspects, and investigated how these are currently being reported.

RESULTS

Of the 284 screened trials, 68(24%) were eligible. Most trials were published in "high impact" peer-reviewed journals. Examination of trials established that 46(68%) were stopped early, predominantly either for futility or efficacy. Suboptimal reporting compliance was found in general items relating to: access to full trials protocols; methods to generate randomisation list(s); details of randomisation concealment, and its implementation. Benchmarking against the supplementary checklist, GS aspects were largely inadequately reported. Only 3(7%) trials which stopped early reported use of statistical bias correction. Moreover, 52(76%) trials failed to disclose methods used to minimise the risk of operational bias, due to the knowledge or leakage of interim results. Occurrence of changes to trial methods and outcomes could not be determined in most trials, due to inaccessible protocols and amendments.

DISCUSSION AND CONCLUSIONS

There are issues with the reporting of GS trials, particularly those specific to the conduct of interim analyses. Suboptimal reporting of bias correction methods could potentially imply most GS trials stopping early are giving biased results of treatment effects. As a result, research consumers may question credibility of findings to change practice when trials are stopped early. These issues could be alleviated through a CONSORT extension. Assurance of scientific rigour through transparent adequate reporting is paramount to the credibility of findings from adaptive trials. Our systematic literature search was restricted to one database due to resource constraints.

Missing steps in a staircase: a qualitative study of the perspectives of key stakeholders on the use of adaptive designs in confirmatory trials

Background

Despite the promising benefits of adaptive designs (ADs), their routine use, especially in confirmatory trials, is lagging behind the prominence given to them in the statistical literature. Much of the previous research to understand barriers and potential facilitators to the use of ADs has been driven from a pharmaceutical drug development perspective, with little focus on trials in the public sector. In this paper, we explore key stakeholders’ experiences, perceptions and views on barriers and facilitators to the use of ADs in publicly funded confirmatory trials.

Methods

Semi-structured, in-depth interviews of key stakeholders in clinical trials research (CTU directors, funding board and panel members, statisticians, regulators, chief investigators, data monitoring committee members and health economists) were conducted through telephone or face-to-face sessions, predominantly in the UK. We purposively selected participants sequentially to optimise maximum variation in views and experiences. We employed the framework approach to analyse the qualitative data.

Results

We interviewed 27 participants. We found some of the perceived barriers to be: lack of knowledge and experience coupled with paucity of case studies, lack of applied training, degree of reluctance to use ADs, lack of bridge funding and time to support design work, lack of statistical expertise, some anxiety about the impact of early trial stopping on researchers’ employment contracts, lack of understanding of acceptable scope of ADs and when ADs are appropriate, and statistical and practical complexities. Reluctance to use ADs seemed to be influenced by: therapeutic area, unfamiliarity, concerns about their robustness in decision-making and acceptability of findings to change practice, perceived complexities and proposed type of AD, among others.

Conclusions

There are still considerable multifaceted, individual and organisational obstacles to be addressed to improve uptake, and successful implementation of ADs when appropriate. Nevertheless, inferred positive change in attitudes and receptiveness towards the appropriate use of ADs by public funders are supportive and are a stepping stone for the future utilisation of ADs by researchers.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-015-0958-9) contains supplementary material, which is available to authorized users.

Reflections on the Adaptive Designs Accelerating Promising Trials Into Treatments (ADAPT-IT) Process-Findings from a Qualitative Study

CONTEXT

The context for this study was the Adaptive Designs Advancing Promising Treatments Into Trials (ADAPT-IT) project, which aimed to incorporate flexible adaptive designs into pivotal clinical trials and to conduct an assessment of the trial development process. Little research provides guidance to academic institutions in planning adaptive trials.

OBJECTIVES

The purpose of this qualitative study was to explore the perspectives and experiences of stakeholders as they reflected back about the interactive ADAPT-IT adaptive design development process, and to understand their perspectives regarding lessons learned about the design of the trials and trial development.

MATERIALS AND METHODS

We conducted semi-structured interviews with ten key stakeholders and observations of the process. We employed qualitative thematic text data analysis to reduce the data into themes about the ADAPT-IT project and adaptive clinical trials.

RESULTS

The qualitative analysis revealed four themes: education of the project participants, how the process evolved with participant feedback, procedures that could enhance the development of other trials, and education of the broader research community.

DISCUSSION AND CONCLUSIONS

While participants became more likely to consider flexible adaptive designs, additional education is needed to both understand the adaptive methodology and articulate it when planning trials.

Adaptive clinical trial designs for European marketing authorization: a survey of scientific advice letters from the European Medicines Agency

Background

Since the first methodological publications on adaptive study design approaches in the 1990s, the application of these approaches in drug development has raised increasing interest among academia, industry and regulators. The European Medicines Agency (EMA) as well as the Food and Drug Administration (FDA) have published guidance documents addressing the potentials and limitations of adaptive designs in the regulatory context. Since there is limited experience in the implementation and interpretation of adaptive clinical trials, early interaction with regulators is recommended. The EMA offers such interactions through scientific advice and protocol assistance procedures.

Methods

We performed a text search of scientific advice letters issued between 1 January 2007 and 8 May 2012 that contained relevant key terms. Letters containing questions related to adaptive clinical trials in phases II or III were selected for further analysis. From the selected letters, important characteristics of the proposed design and its context in the drug development program, as well as the responses of the Committee for Human Medicinal Products (CHMP)/Scientific Advice Working Party (SAWP), were extracted and categorized. For 41 more recent procedures (1 January 2009 to 8 May 2012), additional details of the trial design and the CHMP/SAWP responses were assessed. In addition, case studies are presented as examples.

Results

Over a range of 5½ years, 59 scientific advices were identified that address adaptive study designs in phase II and phase III clinical trials. Almost all were proposed as confirmatory phase III or phase II/III studies. The most frequently proposed adaptation was sample size reassessment, followed by dropping of treatment arms and population enrichment. While 12 (20%) of the 59 proposals for an adaptive clinical trial were not accepted, the great majority of proposals were accepted (15, 25%) or conditionally accepted (32, 54%). In the more recent 41 procedures, the most frequent concerns raised by CHMP/SAWP were insufficient justifications of the adaptation strategy, type I error rate control and bias.

Conclusions

For the majority of proposed adaptive clinical trials, an overall positive opinion was given albeit with critical comments. Type I error rate control, bias and the justification of the design are common issues raised by the CHMP/SAWP.

Maximum type 1 error rate inflation in multiarmed clinical trials with adaptive interim sample size modifications

Sample size modifications in the interim analyses of an adaptive design can inflate the type 1 error rate, if test statistics and critical boundaries are used in the final analysis as if no modification had been made. While this is already true for designs with an overall change of the sample size in a balanced treatment-control comparison, the inflation can be much larger if in addition a modification of allocation ratios is allowed as well. In this paper, we investigate adaptive designs with several treatment arms compared to a single common control group. Regarding modifications, we consider treatment arm selection as well as modifications of overall sample size and allocation ratios. The inflation is quantified for two approaches: a naive procedure that ignores not only all modifications, but also the multiplicity issue arising from the many-to-one comparison, and a Dunnett procedure that ignores modifications, but adjusts for the initially started multiple treatments. The maximum inflation of the type 1 error rate for such types of design can be calculated by searching for the "worst case" scenarios, that are sample size adaptation rules in the interim analysis that lead to the largest conditional type 1 error rate in any point of the sample space. To show the most extreme inflation, we initially assume unconstrained second stage sample size modifications leading to a large inflation of the type 1 error rate. Furthermore, we investigate the inflation when putting constraints on the second stage sample sizes. It turns out that, for example fixing the sample size of the control group, leads to designs controlling the type 1 error rate.