1
|
Robertson DS, Choodari-Oskooei B, Dimairo M, Flight L, Pallmann P, Jaki T. Point estimation for adaptive trial designs II: Practical considerations and guidance. Stat Med 2023; 42:2496-2520. [PMID: 37021359 PMCID: PMC7614609 DOI: 10.1002/sim.9734] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 01/20/2023] [Accepted: 03/18/2023] [Indexed: 04/07/2023]
Abstract
In adaptive clinical trials, the conventional end-of-trial point estimate of a treatment effect is prone to bias, that is, a systematic tendency to deviate from its true value. As stated in recent FDA guidance on adaptive designs, it is desirable to report estimates of treatment effects that reduce or remove this bias. However, it may be unclear which of the available estimators are preferable, and their use remains rare in practice. This article is the second in a two-part series that studies the issue of bias in point estimation for adaptive trials. Part I provided a methodological review of approaches to remove or reduce the potential bias in point estimation for adaptive designs. In part II, we discuss how bias can affect standard estimators and assess the negative impact this can have. We review current practice for reporting point estimates and illustrate the computation of different estimators using a real adaptive trial example (including code), which we use as a basis for a simulation study. We show that while on average the values of these estimators can be similar, for a particular trial realization they can give noticeably different values for the estimated treatment effect. Finally, we propose guidelines for researchers around the choice of estimators and the reporting of estimates following an adaptive design. The issue of bias should be considered throughout the whole lifecycle of an adaptive design, with the estimation strategy prespecified in the statistical analysis plan. When available, unbiased or bias-reduced estimates are to be preferred.
Collapse
Affiliation(s)
| | - Babak Choodari-Oskooei
- MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, UK
| | - Munya Dimairo
- School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK
| | - Laura Flight
- School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK
| | | | - Thomas Jaki
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
- Faculty of Informatics and Data Science, University of Regensburg, Regensburg, Germany
| |
Collapse
|
2
|
Robertson DS, Choodari‐Oskooei B, Dimairo M, Flight L, Pallmann P, Jaki T. Point estimation for adaptive trial designs I: A methodological review. Stat Med 2023; 42:122-145. [PMID: 36451173 PMCID: PMC7613995 DOI: 10.1002/sim.9605] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 10/21/2022] [Accepted: 11/01/2022] [Indexed: 12/02/2022]
Abstract
Recent FDA guidance on adaptive clinical trial designs defines bias as "a systematic tendency for the estimate of treatment effect to deviate from its true value," and states that it is desirable to obtain and report estimates of treatment effects that reduce or remove this bias. The conventional end-of-trial point estimates of the treatment effects are prone to bias in many adaptive designs, because they do not take into account the potential and realized trial adaptations. While much of the methodological developments on adaptive designs have tended to focus on control of type I error rates and power considerations, in contrast the question of biased estimation has received relatively less attention. This article is the first in a two-part series that studies the issue of potential bias in point estimation for adaptive trials. Part I provides a comprehensive review of the methods to remove or reduce the potential bias in point estimation of treatment effects for adaptive designs, while part II illustrates how to implement these in practice and proposes a set of guidelines for trial statisticians. The methods reviewed in this article can be broadly classified into unbiased and bias-reduced estimation, and we also provide a classification of estimators by the type of adaptive design. We compare the proposed methods, highlight available software and code, and discuss potential methodological gaps in the literature.
Collapse
Affiliation(s)
| | | | - Munya Dimairo
- School of Health and Related Research (ScHARR)University of SheffieldSheffieldUK
| | - Laura Flight
- School of Health and Related Research (ScHARR)University of SheffieldSheffieldUK
| | | | - Thomas Jaki
- MRC Biostatistics UnitUniversity of CambridgeCambridgeUK
- Faculty of Informatics and Data ScienceUniversity of RegensburgRegensburgGermany
| |
Collapse
|
3
|
Khan JN, Kimani PK, Glimm E, Stallard N. Adjusting for treatment selection in phase II/III clinical trials with time to event data. Stat Med 2023; 42:146-163. [PMID: 36419206 PMCID: PMC10098876 DOI: 10.1002/sim.9606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Revised: 09/25/2022] [Accepted: 11/01/2022] [Indexed: 11/26/2022]
Abstract
Phase II/III clinical trials are efficient two-stage designs that test multiple experimental treatments. In stage 1, patients are allocated to the control and all experimental treatments, with the data collected from them used to select experimental treatments to continue to stage 2. Patients recruited in stage 2 are allocated to the selected treatments and the control. Combined data of stage 1 and stage 2 are used for a confirmatory phase III analysis. Appropriate analysis needs to adjust for selection bias of the stage 1 data. Point estimators exist for normally distributed outcome data. Extending these estimators to time to event data is not straightforward because treatment selection is based on correlated treatment effects and stage 1 patients who do not get events in stage 1 are followed-up in stage 2. We have derived an approximately uniformly minimum variance conditional unbiased estimator (UMVCUE) and compared its biases and mean squared errors to existing bias adjusted estimators. In simulations, one existing bias adjusted estimator has similar properties as the practically unbiased UMVCUE while the others can have noticeable biases but they are less variable than the UMVCUE. For confirmatory phase II/III clinical trials where unbiased estimators are desired, we recommend the UMVCUE or the existing estimator with which it has similar properties.
Collapse
Affiliation(s)
| | - Peter K Kimani
- Warwick Medical School, University of Warwick, Coventry, UK
| | | | - Nigel Stallard
- Warwick Medical School, University of Warwick, Coventry, UK
| |
Collapse
|
4
|
Collignon O, Schiel A, Burman C, Rufibach K, Posch M, Bretz F. Estimands and Complex Innovative Designs. Clin Pharmacol Ther 2022; 112:1183-1190. [PMID: 35253205 PMCID: PMC9790227 DOI: 10.1002/cpt.2575] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2021] [Accepted: 03/01/2022] [Indexed: 01/31/2023]
Abstract
Since the release of the ICH E9(R1) (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials) document in 2019, the estimand framework has become a fundamental part of clinical trial protocols. In parallel, complex innovative designs have gained increased popularity in drug development, in particular in early development phases or in difficult experimental situations. While the estimand framework is relevant to any study in which a treatment effect is estimated, experience is lacking as regards its application to these designs. In a basket trial for example, should a different estimand be specified for each subpopulation of interest, defined, for example, by cancer site? Or can a single estimand focusing on the general population (defined, for example, by the positivity to a certain biomarker) be used? In the case of platform trials, should a different estimand be proposed for each drug investigated? In this work we discuss possible ways of implementing the estimand framework for different types of complex innovative designs. We consider trials that allow adding or selecting experimental treatment arms, modifying the control arm or the standard of care, and selecting or pooling populations. We also address the potentially data-driven, adaptive selection of estimands in an ongoing trial and disentangle certain statistical issues that pertain to estimation rather than to estimands, such as the borrowing of nonconcurrent information. We hope this discussion will facilitate the implementation of the estimand framework and its description in the study protocol when the objectives of the trial require complex innovative designs.
Collapse
Affiliation(s)
| | | | - Carl‐Fredrik Burman
- Statistical Innovation, Data Science & Artificial IntelligenceAstraZeneca Research & DevelopmentGothenburgSweden
| | - Kaspar Rufibach
- Methods, Collaboration, and Outreach Group, Product Development Data SciencesF.Hoffmann‐La RocheBaselSwitzerland
| | - Martin Posch
- Section for Medical StatisticsCenter for Medical Statistics Informatics, and Intelligent SystemsMedical University of ViennaViennaAustria
| | - Frank Bretz
- Section for Medical StatisticsCenter for Medical Statistics Informatics, and Intelligent SystemsMedical University of ViennaViennaAustria,NovartisBaselSwitzerland
| |
Collapse
|
5
|
Lee KM, Robertson DS, Jaki T, Emsley R. The benefits of covariate adjustment for adaptive multi-arm designs. Stat Methods Med Res 2022; 31:2104-2121. [PMID: 35876412 PMCID: PMC7613816 DOI: 10.1177/09622802221114544] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
Covariate adjustment via a regression approach is known to increase the precision
of statistical inference when fixed trial designs are employed in randomized
controlled studies. When an adaptive multi-arm design is employed with the
ability to select treatments, it is unclear how covariate adjustment affects
various aspects of the study. Consider the design framework that relies on
pre-specified treatment selection rule(s) and a combination test approach for
hypothesis testing. It is our primary goal to evaluate the impact of covariate
adjustment on adaptive multi-arm designs with treatment selection. Our secondary
goal is to show how the Uniformly Minimum Variance Conditionally Unbiased
Estimator can be extended to account for covariate adjustment analytically. We
find that adjustment with different sets of covariates can lead to different
treatment selection outcomes and hence probabilities of rejecting hypotheses.
Nevertheless, we do not see any negative impact on the control of the familywise
error rate when covariates are included in the analysis model. When adjusting
for covariates that are moderately or highly correlated with the outcome, we see
various benefits to the analysis of the design. Conversely, there is negligible
impact when including covariates that are uncorrelated with the outcome.
Overall, pre-specification of covariate adjustment is recommended for the
analysis of adaptive multi-arm design with treatment selection. Having the
statistical analysis plan in place prior to the interim and final analyses is
crucial, especially when a non-collapsible measure of treatment effect is
considered in the trial.
Collapse
Affiliation(s)
- Kim May Lee
- Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | | | - Thomas Jaki
- 47959MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
| | | |
Collapse
|
6
|
Ballarini NM, Burnett T, Jaki T, Jennison C, König F, Posch M. Optimizing subgroup selection in two-stage adaptive enrichment and umbrella designs. Stat Med 2021; 40:2939-2956. [PMID: 33783020 PMCID: PMC8251960 DOI: 10.1002/sim.8949] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2020] [Revised: 01/11/2021] [Accepted: 02/28/2021] [Indexed: 12/11/2022]
Abstract
We design two‐stage confirmatory clinical trials that use adaptation to find the subgroup of patients who will benefit from a new treatment, testing for a treatment effect in each of two disjoint subgroups. Our proposal allows aspects of the trial, such as recruitment probabilities of each group, to be altered at an interim analysis. We use the conditional error rate approach to implement these adaptations with protection of overall error rates. Applying a Bayesian decision‐theoretic framework, we optimize design parameters by maximizing a utility function that takes the population prevalence of the subgroups into account. We show results for traditional trials with familywise error rate control (using a closed testing procedure) as well as for umbrella trials in which only the per‐comparison type 1 error rate is controlled. We present numerical examples to illustrate the optimization process and the effectiveness of the proposed designs.
Collapse
Affiliation(s)
- Nicolás M Ballarini
- Section for Medical Statistics, Medical University of Vienna, Vienna, Austria
| | - Thomas Burnett
- Department of Mathematics and Statistics, Lancaster University, Lancaster, UK
| | - Thomas Jaki
- Department of Mathematics and Statistics, Lancaster University, Lancaster, UK.,MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
| | | | - Franz König
- Section for Medical Statistics, Medical University of Vienna, Vienna, Austria
| | - Martin Posch
- Section for Medical Statistics, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
7
|
Stallard N, Hampson L, Benda N, Brannath W, Burnett T, Friede T, Kimani PK, Koenig F, Krisam J, Mozgunov P, Posch M, Wason J, Wassmer G, Whitehead J, Williamson SF, Zohar S, Jaki T. Efficient Adaptive Designs for Clinical Trials of Interventions for COVID-19. Stat Biopharm Res 2020; 12:483-497. [PMID: 34191981 PMCID: PMC8011600 DOI: 10.1080/19466315.2020.1790415] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Revised: 06/23/2020] [Accepted: 06/24/2020] [Indexed: 02/06/2023]
Abstract
The COVID-19 pandemic has led to an unprecedented response in terms of clinical research activity. An important part of this research has been focused on randomized controlled clinical trials to evaluate potential therapies for COVID-19. The results from this research need to be obtained as rapidly as possible. This presents a number of challenges associated with considerable uncertainty over the natural history of the disease and the number and characteristics of patients affected, and the emergence of new potential therapies. These challenges make adaptive designs for clinical trials a particularly attractive option. Such designs allow a trial to be modified on the basis of interim analysis data or stopped as soon as sufficiently strong evidence has been observed to answer the research question, without compromising the trial's scientific validity or integrity. In this article, we describe some of the adaptive design approaches that are available and discuss particular issues and challenges associated with their use in the pandemic setting. Our discussion is illustrated by details of four ongoing COVID-19 trials that have used adaptive designs.
Collapse
Affiliation(s)
- Nigel Stallard
- Statistics and Epidemiology, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Lisa Hampson
- Advanced Methodology and Data Science, Novartis Pharma AG, Basel, Switzerland
| | - Norbert Benda
- The Federal Institute for Drugs and Medical Devices (BfArM), Bonn, Germany
| | - Werner Brannath
- Institute for Statistics, University of Bremen, Bremen, Germany
| | - Thomas Burnett
- Department of Mathematics and Statistics, Lancaster University, Lancaster, UK
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| | - Peter K. Kimani
- Statistics and Epidemiology, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Franz Koenig
- Section for Medical Statistics, CeMSIIS, Medical University of Vienna, Vienna, Austria
| | - Johannes Krisam
- Institute of Medical Biometry and Informatics, University of Heidelberg, Heidelberg, Germany
| | - Pavel Mozgunov
- Department of Mathematics and Statistics, Lancaster University, Lancaster, UK
| | - Martin Posch
- Section for Medical Statistics, CeMSIIS, Medical University of Vienna, Vienna, Austria
| | - James Wason
- Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
| | | | - John Whitehead
- Department of Mathematics and Statistics, Lancaster University, Lancaster, UK
| | - S. Faye Williamson
- Department of Mathematics and Statistics, Lancaster University, Lancaster, UK
| | - Sarah Zohar
- INSERM, Centre de Recherche des Cordeliers, Sorbonne Université, Université de Paris, Paris, France
| | - Thomas Jaki
- Department of Mathematics and Statistics, Lancaster University, Lancaster, UK
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
| |
Collapse
|
8
|
Kimani PK, Todd S, Renfro LA, Glimm E, Khan JN, Kairalla JA, Stallard N. Point and interval estimation in two-stage adaptive designs with time to event data and biomarker-driven subpopulation selection. Stat Med 2020; 39:2568-2586. [PMID: 32363603 PMCID: PMC7785132 DOI: 10.1002/sim.8557] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2019] [Revised: 03/31/2020] [Accepted: 04/06/2020] [Indexed: 02/02/2023]
Abstract
In personalized medicine, it is often desired to determine if all patients or only a subset of them benefit from a treatment. We consider estimation in two-stage adaptive designs that in stage 1 recruit patients from the full population. In stage 2, patient recruitment is restricted to the part of the population, which, based on stage 1 data, benefits from the experimental treatment. Existing estimators, which adjust for using stage 1 data for selecting the part of the population from which stage 2 patients are recruited, as well as for the confirmatory analysis after stage 2, do not consider time to event patient outcomes. In this work, for time to event data, we have derived a new asymptotically unbiased estimator for the log hazard ratio and a new interval estimator with good coverage probabilities and probabilities that the upper bounds are below the true values. The estimators are appropriate for several selection rules that are based on a single or multiple biomarkers, which can be categorical or continuous.
Collapse
Affiliation(s)
- Peter K Kimani
- Warwick Medical School, University of Warwick, Coventry, UK
| | - Susan Todd
- Department of Mathematics and Statistics, University of Reading, Reading, UK
| | - Lindsay A Renfro
- Division of Biostatistics, University of Southern California, Los Angeles, CA, USA
| | | | | | - John A Kairalla
- Department of Biostatistics, University of Florida, Gainesville, Florida, USA
| | - Nigel Stallard
- Warwick Medical School, University of Warwick, Coventry, UK
| |
Collapse
|
9
|
Ghosh P, Liu L, Mehta C. Adaptive multiarm multistage clinical trials. Stat Med 2020; 39:1084-1102. [PMID: 32048313 PMCID: PMC7065228 DOI: 10.1002/sim.8464] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2018] [Revised: 11/04/2019] [Accepted: 12/12/2019] [Indexed: 11/07/2022]
Abstract
Two methods for designing adaptive multiarm multistage (MAMS) clinical trials, originating from conceptually different group sequential frameworks are presented, and their operating characteristics are compared. In both methods pairwise comparisons are made, stage-by-stage, between each treatment arm and a common control arm with the goal of identifying active treatments and dropping inactive ones. At any stage one may alter the future course of the trial through adaptive changes to the prespecified decision rules for treatment selection and sample size reestimation, and notwithstanding such changes, both methods guarantee strong control of the family-wise error rate. The stage-wise MAMS approach was historically the first to be developed and remains the standard method for designing inferentially seamless phase 2-3 clinical trials. In this approach, at each stage, the data from each treatment comparison are summarized by a single multiplicity adjusted P-value. These stage-wise P-values are combined by a prespecified combination function and the resultant test statistic is monitored with respect to the classical two-arm group sequential efficacy boundaries. The cumulative MAMS approach is a more recent development in which a separate test statistic is constructed for each treatment comparison from the cumulative data at each stage. These statistics are then monitored with respect to multiplicity adjusted group sequential efficacy boundaries. We compared the powers of the two methods for designs with two and three active treatment arms, under commonly utilized decision rules for treatment selection, sample size reestimation and early stopping. In our investigations, which were carried out over a reasonably exhaustive exploration of the parameter space, the cumulative MAMS designs were more powerful than the stage-wise MAMS designs, except for the homogeneous case of equal treatment effects, where a small power advantage was discernable for the stage-wise MAMS designs.
Collapse
Affiliation(s)
| | | | - Cyrus Mehta
- Cytel Inc, Cambridge, Massachusetts.,Harvard T.H. Chan School of Public Health, Boston, Massachusetts
| |
Collapse
|
10
|
Zapf A, Stark M, Gerke O, Ehret C, Benda N, Bossuyt P, Deeks J, Reitsma J, Alonzo T, Friede T. Adaptive trial designs in diagnostic accuracy research. Stat Med 2019; 39:591-601. [PMID: 31773788 DOI: 10.1002/sim.8430] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2019] [Revised: 10/23/2019] [Accepted: 10/26/2019] [Indexed: 11/10/2022]
Abstract
The aim of diagnostic accuracy studies is to evaluate how accurately a diagnostic test can distinguish diseased from nondiseased individuals. Depending on the research question, different study designs and accuracy measures are appropriate. As the prior knowledge in the planning phase is often very limited, modifications of design aspects such as the sample size during the ongoing trial could increase the efficiency of diagnostic trials. In intervention studies, group sequential and adaptive designs are well established. Such designs are characterized by preplanned interim analyses, giving the opportunity to stop early for efficacy or futility or to modify elements of the study design. In contrast, in diagnostic accuracy studies, such flexible designs are less common, even if they are as important as for intervention studies. However, diagnostic accuracy studies have specific features, which may require adaptations of the statistical methods or may lead to specific advantages or limitations of sequential and adaptive designs. In this article, we summarize the current status of methodological research and applications of flexible designs in diagnostic accuracy research. Furthermore, we indicate and advocate future development of adaptive design methodology and their use in diagnostic accuracy trials from an interdisciplinary viewpoint. The term "interdisciplinary viewpoint" describes the collaboration of experts of the academic and nonacademic research.
Collapse
Affiliation(s)
- Antonia Zapf
- Department of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Maria Stark
- Department of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Oke Gerke
- Department of Nuclear Medicine, Odense University Hospital, Odense, Denmark
| | | | - Norbert Benda
- Federal Institute for Drugs and Medical Devices (BfArM), Bonn, Germany.,Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| | - Patrick Bossuyt
- Department of Clinical Epidemiology and Biostatistics, University of Amsterdam, Amsterdam, The Netherlands
| | - Jon Deeks
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK.,NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Trust and the University of Birmingham, Birmingham, UK
| | - Johannes Reitsma
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht & University Utrecht, Utrecht, The Netherlands
| | - Todd Alonzo
- Keck School of Medicine, University of Southern California, Los Angeles, California
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| |
Collapse
|
11
|
Barker KL, Newman M, Stallard N, Leal J, Minns Lowe C, Javaid MK, Noufaily A, Adhikari A, Hughes T, Smith DJ, Gandhi V, Cooper C, Lamb SE. Exercise or manual physiotherapy compared with a single session of physiotherapy for osteoporotic vertebral fracture: three-arm PROVE RCT. Health Technol Assess 2019; 23:1-318. [PMID: 31456562 DOI: 10.3310/hta23440] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022] Open
Abstract
BACKGROUND A total of 25,000 people in the UK have osteoporotic vertebral fracture (OVF). Evidence suggests that physiotherapy may have an important treatment role. OBJECTIVE The objective was to investigate the clinical effectiveness and cost-effectiveness of two different physiotherapy programmes for people with OVF compared with a single physiotherapy session. DESIGN This was a prospective, adaptive, multicentre, assessor-blinded randomised controlled trial (RCT) with nested qualitative and health economic studies. SETTING This trial was based in 21 NHS physiotherapy departments. PARTICIPANTS The participants were people with symptomatic OVF. INTERVENTIONS Seven sessions of either manual outpatient physiotherapy or exercise outpatient physiotherapy compared with the best practice of a 1-hour single session of physiotherapy (SSPT). MAIN OUTCOME MEASURES Outcomes were measured at 4 and 12 months. The primary outcomes were quality of life and muscle endurance, which were measured by the disease-specific QUALEFFO-41 (Quality of Life Questionnaire of the European Foundation for Osteoporosis - 41 items) and timed loaded standing (TLS) test, respectively. Secondary outcomes were (1) thoracic kyphosis angle, (2) balance, evaluated via the functional reach test (FRT), and (3) physical function, assessed via the Short Physical Performance Battery (SPPB), 6-minute walk test (6MWT), Physical Activity Scale for the Elderly, a health resource use and falls diary, and the EuroQol-5 Dimensions, five-level version. RESULTS A total of 615 participants were enrolled, with 216, 203 and 196 randomised by a computer-generated program to exercise therapy, manual therapy and a SSPT, respectively. Baseline data were available for 613 participants, 531 (86.6%) of whom were women; the mean age of these participants was 72.14 years (standard deviation 9.09 years). Primary outcome data were obtained for 69% of participants (429/615) at 12 months: 175 in the exercise therapy arm, 181 in the manual therapy arm and 173 in the SSPT arm. Interim analysis met the criteria for all arms to remain in the study. For the primary outcomes at 12 months, there were no significant benefits over SSPT of exercise [QUALEFFO-41, difference -0.23 points, 95% confidence interval (CI) -3.20 to 1.59 points; p = 1.000; and TLS test, difference 5.77 seconds, 95% CI -4.85 to 20.46 seconds; p = 0.437] or of manual therapy (QUALEFFO-41, difference 1.35 points, 95% CI -1.76 to 2.93 points; p = 0.744; TLS test, difference 9.69 seconds (95% CI 0.09 to 24.86 seconds; p = 0.335). At 4 months, there were significant gains for both manual therapy and exercise therapy over SSPT in the TLS test in participants aged < 70 years. Exercise therapy was superior to a SSPT at 4 months in the SPPB, FRT and 6MWT and manual therapy was superior to a SSPT at 4 months in the TLS test and FRT. Neither manual therapy nor exercise therapy was cost-effective relative to a SSPT using the threshold of £20,000 per quality-adjusted life-year. There were no treatment-related serious adverse events. CONCLUSIONS This is the largest RCT to date assessing physiotherapy in participants with OVFs. At 1 year, neither treatment intervention conferred more benefit than a single 1-hour physiotherapy advice session. The focus of future work should be on the intensity and duration of interventions to determine if changes to these would demonstrate more sustained effects. TRIAL REGISTRATION Current Controlled Trials ISRCTN49117867. FUNDING This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 23, No. 44. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Karen L Barker
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.,Physiotherapy Research Unit, Nuffield Orthopaedic Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Meredith Newman
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.,Physiotherapy Research Unit, Nuffield Orthopaedic Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Nigel Stallard
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Jose Leal
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Catherine Minns Lowe
- Physiotherapy Research Unit, Nuffield Orthopaedic Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Muhammad K Javaid
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Angela Noufaily
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Anish Adhikari
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Tamsin Hughes
- Physiotherapy Research Unit, Nuffield Orthopaedic Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - David J Smith
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Varsha Gandhi
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Cyrus Cooper
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Sarah E Lamb
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
12
|
Chiu YD, Koenig F, Posch M, Jaki T. Design and estimation in clinical trials with subpopulation selection. Stat Med 2018; 37:4335-4352. [PMID: 30088280 PMCID: PMC6282861 DOI: 10.1002/sim.7925] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2017] [Revised: 05/23/2018] [Accepted: 07/06/2018] [Indexed: 11/10/2022]
Abstract
Population heterogeneity is frequently observed among patients' treatment responses in clinical trials because of various factors such as clinical background, environmental, and genetic factors. Different subpopulations defined by those baseline factors can lead to differences in the benefit or safety profile of a therapeutic intervention. Ignoring heterogeneity between subpopulations can substantially impact on medical practice. One approach to address heterogeneity necessitates designs and analysis of clinical trials with subpopulation selection. Several types of designs have been proposed for different circumstances. In this work, we discuss a class of designs that allow selection of a predefined subgroup. Using the selection based on the maximum test statistics as the worst‐case scenario, we then investigate the precision and accuracy of the maximum likelihood estimator at the end of the study via simulations. We find that the required sample size is chiefly determined by the subgroup prevalence and show in simulations that the maximum likelihood estimator for these designs can be substantially biased.
Collapse
Affiliation(s)
- Yi-Da Chiu
- Medical and Pharmaceutical Statistics Research Unit, Department of Mathematics and Statistics, Lancaster University, Lancashire, UK
| | - Franz Koenig
- Center for Medical Statistics, Informatics, and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| | - Martin Posch
- Center for Medical Statistics, Informatics, and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| | - Thomas Jaki
- Medical and Pharmaceutical Statistics Research Unit, Department of Mathematics and Statistics, Lancaster University, Lancashire, UK
| |
Collapse
|
13
|
Pallmann P, Bedding AW, Choodari-Oskooei B, Dimairo M, Flight L, Hampson LV, Holmes J, Mander AP, Odondi L, Sydes MR, Villar SS, Wason JMS, Weir CJ, Wheeler GM, Yap C, Jaki T. Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med 2018; 16:29. [PMID: 29490655 PMCID: PMC5830330 DOI: 10.1186/s12916-018-1017-7] [Citation(s) in RCA: 345] [Impact Index Per Article: 57.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/10/2017] [Accepted: 01/30/2018] [Indexed: 12/11/2022] Open
Abstract
Adaptive designs can make clinical trials more flexible by utilising results accumulating in the trial to modify the trial's course in accordance with pre-specified rules. Trials with an adaptive design are often more efficient, informative and ethical than trials with a traditional fixed design since they often make better use of resources such as time and money, and might require fewer participants. Adaptive designs can be applied across all phases of clinical research, from early-phase dose escalation to confirmatory trials. The pace of the uptake of adaptive designs in clinical research, however, has remained well behind that of the statistical literature introducing new methods and highlighting their potential advantages. We speculate that one factor contributing to this is that the full range of adaptations available to trial designs, as well as their goals, advantages and limitations, remains unfamiliar to many parts of the clinical community. Additionally, the term adaptive design has been misleadingly used as an all-encompassing label to refer to certain methods that could be deemed controversial or that have been inadequately implemented.We believe that even if the planning and analysis of a trial is undertaken by an expert statistician, it is essential that the investigators understand the implications of using an adaptive design, for example, what the practical challenges are, what can (and cannot) be inferred from the results of such a trial, and how to report and communicate the results. This tutorial paper provides guidance on key aspects of adaptive designs that are relevant to clinical triallists. We explain the basic rationale behind adaptive designs, clarify ambiguous terminology and summarise the utility and pitfalls of adaptive designs. We discuss practical aspects around funding, ethical approval, treatment supply and communication with stakeholders and trial participants. Our focus, however, is on the interpretation and reporting of results from adaptive design trials, which we consider vital for anyone involved in medical research. We emphasise the general principles of transparency and reproducibility and suggest how best to put them into practice.
Collapse
Affiliation(s)
- Philip Pallmann
- Department of Mathematics & Statistics, Lancaster University, Lancaster, LA1 4YF UK
| | | | - Babak Choodari-Oskooei
- MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, UK
| | | | - Laura Flight
- Medical Statistics Group, University of Sheffield, Sheffield, UK
| | - Lisa V. Hampson
- Department of Mathematics & Statistics, Lancaster University, Lancaster, LA1 4YF UK
- Statistical Innovation Group, Advanced Analytics Centre, AstraZeneca, Cambridge, UK
| | - Jane Holmes
- Centre for Statistics in Medicine, University of Oxford, Oxford, UK
| | | | - Lang’o Odondi
- Centre for Statistics in Medicine, University of Oxford, Oxford, UK
| | - Matthew R. Sydes
- MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, UK
| | - Sofía S. Villar
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
| | - James M. S. Wason
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
- Institute of Health and Society, Newcastle University, Newcastle, UK
| | - Christopher J. Weir
- Usher Institute of Population Health Sciences and Informatics, University of Edinburgh, Edinburgh, UK
| | - Graham M. Wheeler
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
- Cancer Research UK & UCL Cancer Trials Centre, University College London, London, UK
| | - Christina Yap
- Cancer Research UK Clinical Trials Unit, University of Birmingham, Birmingham, UK
| | - Thomas Jaki
- Department of Mathematics & Statistics, Lancaster University, Lancaster, LA1 4YF UK
| |
Collapse
|
14
|
Bauer P, Bretz F, Dragalin V, König F, Wassmer G. Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls. Stat Med 2016; 35:325-47. [PMID: 25778935 PMCID: PMC6680191 DOI: 10.1002/sim.6472] [Citation(s) in RCA: 130] [Impact Index Per Article: 16.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2014] [Revised: 02/03/2015] [Accepted: 02/19/2015] [Indexed: 12/26/2022]
Abstract
'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.
Collapse
Affiliation(s)
- Peter Bauer
- Section of Medical StatisticsMedical University of ViennaSpitalgasse 231090 WienAustria
| | - Frank Bretz
- Novartis Pharma AGLichtstrasse 354002BaselSwitzerland
- Shanghai University of Finance and EconomicsChina
| | | | - Franz König
- Section of Medical StatisticsMedical University of ViennaSpitalgasse 231090 WienAustria
| | - Gernot Wassmer
- Aptiv Solutions, an ICON plc companyRobert‐Perthel‐Str. 77a50739KölnGermany
- Institute for Medical Statistics, Informatics and EpidemiologyUniversity of Cologne50924KölnGermany
| |
Collapse
|
15
|
Jaki T, Hampson LV. Designing multi-arm multi-stage clinical trials using a risk-benefit criterion for treatment selection. Stat Med 2015; 35:522-33. [PMID: 26456537 PMCID: PMC4950037 DOI: 10.1002/sim.6760] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2014] [Revised: 09/15/2015] [Accepted: 09/24/2015] [Indexed: 01/04/2023]
Abstract
Multi‐arm clinical trials that compare several active treatments to a common control have been proposed as an efficient means of making an informed decision about which of several treatments should be evaluated further in a confirmatory study. Additional efficiency is gained by incorporating interim analyses and, in particular, seamless Phase II/III designs have been the focus of recent research. Common to much of this work is the constraint that selection and formal testing should be based on a single efficacy endpoint, despite the fact that in practice, safety considerations will often play a central role in determining selection decisions. Here, we develop a multi‐arm multi‐stage design for a trial with an efficacy and safety endpoint. The safety endpoint is explicitly considered in the formulation of the problem, selection of experimental arm and hypothesis testing. The design extends group‐sequential ideas and considers the scenario where a minimal safety requirement is to be fulfilled and the treatment yielding the best combined safety and efficacy trade‐off satisfying this constraint is selected for further testing. The treatment with the best trade‐off is selected at the first interim analysis, while the whole trial is allowed to compose of J analyses. We show that the design controls the familywise error rate in the strong sense and illustrate the method through an example and simulation. We find that the design is robust to misspecification of the correlation between the endpoints and requires similar numbers of subjects to a trial based on efficacy alone for moderately correlated endpoints. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.
Collapse
Affiliation(s)
- Thomas Jaki
- Medical and Pharmaceutical Statistics Research Unit, Department of Mathematics and Statistics, Lancaster University, Lancaster, U.K
| | - Lisa V Hampson
- Medical and Pharmaceutical Statistics Research Unit, Department of Mathematics and Statistics, Lancaster University, Lancaster, U.K
| |
Collapse
|
16
|
Kimani PK, Todd S, Stallard N. Estimation after subpopulation selection in adaptive seamless trials. Stat Med 2015; 34:2581-601. [PMID: 25903293 PMCID: PMC4973856 DOI: 10.1002/sim.6506] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2013] [Revised: 12/24/2014] [Accepted: 03/22/2015] [Indexed: 12/30/2022]
Abstract
During the development of new therapies, it is not uncommon to test whether a new treatment works better than the existing treatment for all patients who suffer from a condition (full population) or for a subset of the full population (subpopulation). One approach that may be used for this objective is to have two separate trials, where in the first trial, data are collected to determine if the new treatment benefits the full population or the subpopulation. The second trial is a confirmatory trial to test the new treatment in the population selected in the first trial. In this paper, we consider the more efficient two-stage adaptive seamless designs (ASDs), where in stage 1, data are collected to select the population to test in stage 2. In stage 2, additional data are collected to perform confirmatory analysis for the selected population. Unlike the approach that uses two separate trials, for ASDs, stage 1 data are also used in the confirmatory analysis. Although ASDs are efficient, using stage 1 data both for selection and confirmatory analysis introduces selection bias and consequently statistical challenges in making inference. We will focus on point estimation for such trials. In this paper, we describe the extent of bias for estimators that ignore multiple hypotheses and selecting the population that is most likely to give positive trial results based on observed stage 1 data. We then derive conditionally unbiased estimators and examine their mean squared errors for different scenarios.
Collapse
Affiliation(s)
- Peter K. Kimani
- Warwick Medical SchoolThe University of WarwickCoventryCV4 7ALU.K.
| | - Susan Todd
- Department of Mathematics and StatisticsThe University of ReadingRG6 6AXReadingU.K.
| | - Nigel Stallard
- Warwick Medical SchoolThe University of WarwickCoventryCV4 7ALU.K.
| |
Collapse
|
17
|
Multi-arm clinical trials with treatment selection: what can be gained and at what price? ACTA ACUST UNITED AC 2015. [DOI: 10.4155/cli.15.13] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
18
|
Sugitani T, Bretz F, Maurer W. A simple and flexible graphical approach for adaptive group-sequential clinical trials. J Biopharm Stat 2014; 26:202-16. [PMID: 25372071 DOI: 10.1080/10543406.2014.972509] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
In this article, we introduce a graphical approach to testing multiple hypotheses in group-sequential clinical trials allowing for midterm design modifications. It is intended for structured study objectives in adaptive clinical trials and extends the graphical group-sequential designs from Maurer and Bretz (Statistics in Biopharmaceutical Research 2013; 5: 311-320) to adaptive trial designs. The resulting test strategies can be visualized graphically and performed iteratively. We illustrate the methodology with two examples from our clinical trial practice. First, we consider a three-armed gold-standard trial with the option to reallocate patients to either the test drug or the active control group, while stopping the recruitment of patients to placebo, after having demonstrated superiority of the test drug over placebo at an interim analysis. Second, we consider a confirmatory two-stage adaptive design with treatment selection at interim.
Collapse
Affiliation(s)
- Toshifumi Sugitani
- a Section for Medical Statistics, Medical University of Vienna , Vienna , Austria
| | - Frank Bretz
- b Novartis Pharma AG , Basel , Switzerland.,c Shanghai University of Finance and Economics , Shanghai , Peoples Republic of China
| | | |
Collapse
|
19
|
Magirr D, Stallard N, Jaki T. Flexible sequential designs for multi-arm clinical trials. Stat Med 2014; 33:3269-79. [DOI: 10.1002/sim.6183] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2013] [Revised: 01/30/2014] [Accepted: 04/03/2014] [Indexed: 11/10/2022]
Affiliation(s)
- D. Magirr
- Medical and Pharmaceutical Statistics Research Unit; Lancaster University; Lancaster LA1 4YF U.K
| | - N. Stallard
- Warwick Medical School; University of Warwick; Coventry CV4 7AL U.K
| | - T. Jaki
- Medical and Pharmaceutical Statistics Research Unit; Lancaster University; Lancaster LA1 4YF U.K
| |
Collapse
|
20
|
Kimani PK, Todd S, Stallard N. A comparison of methods for constructing confidence intervals after phase II/III clinical trials. Biom J 2013; 56:107-28. [PMID: 24173686 DOI: 10.1002/bimj.201300036] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2013] [Revised: 08/20/2013] [Accepted: 09/03/2013] [Indexed: 11/10/2022]
Abstract
Recently, in order to accelerate drug development, trials that use adaptive seamless designs such as phase II/III clinical trials have been proposed. Phase II/III clinical trials combine traditional phases II and III into a single trial that is conducted in two stages. Using stage 1 data, an interim analysis is performed to answer phase II objectives and after collection of stage 2 data, a final confirmatory analysis is performed to answer phase III objectives. In this paper we consider phase II/III clinical trials in which, at stage 1, several experimental treatments are compared to a control and the apparently most effective experimental treatment is selected to continue to stage 2. Although these trials are attractive because the confirmatory analysis includes phase II data from stage 1, the inference methods used for trials that compare a single experimental treatment to a control and do not have an interim analysis are no longer appropriate. Several methods for analysing phase II/III clinical trials have been developed. These methods are recent and so there is little literature on extensive comparisons of their characteristics. In this paper we review and compare the various methods available for constructing confidence intervals after phase II/III clinical trials.
Collapse
Affiliation(s)
- Peter K Kimani
- Warwick Medical School, The University of Warwick, Coventry CV4 7AL, UK
| | | | | |
Collapse
|