Methods for identification and confirmation of targeted subgroups in clinical trials: A systematic review

Finding the best subgroup with differential treatment effect with multiple outcomes

Precision medicine aims to identify specific patient subgroups that may benefit the most from a particular treatment than the whole population. Existing definitions for the best subgroup in subgroup analysis are based on a single outcome and do not consider multiple outcomes; specifically, outcomes of different types. In this article, we introduce a definition for the best subgroup under a multiple-outcome setting with continuous, binary, and censored time-to-event outcomes. Our definition provides a trade-off between the subgroup size and the conditional average treatment effects (CATE) in the subgroup with respect to each of the outcomes while taking the relative contribution of the outcomes into account. We conduct simulations to illustrate the proposed definition. By examining the outcomes of urinary tract infection and renal scarring in the RIVUR clinical trial, we identify a subgroup of children that would benefit the most from long-term antimicrobial prophylaxis.

MIDAS-2: an enhanced Bayesian platform design for immunotherapy combinations with subgroup efficacy exploration

Although immunotherapy combinations have revolutionised cancer treatment, the rapid screening of effective and optimal therapies from large numbers of candidate combinations, as well as exploring subgroup efficacy, remains challenging. This necessitates innovative, integrated, and efficient trial designs. In this study, we extend the MIDAS design to include subgroup exploration and propose an enhanced Bayesian information borrowing platform design called MIDAS-2. MIDAS-2 enables quick and continuous screening of promising combination strategies and exploration of their subgroup effects within a unified platform design framework. We use a regression model to characterize the efficacy pattern in subgroups. Information borrowing is applied through Bayesian hierarchical modelling to improve trial efficiency considering the limited sample size in subgroups. Time trend calibration is also employed to avoid potential baseline drifts. Simulation results demonstrate that MIDAS-2 yields high probabilities for identifying the effective drug combinations as well as promising subgroups, facilitating appropriate selection of the best treatments for each subgroup. The proposed design is robust against small time trend drifts, and the type I error is successfully controlled after calibration when a large drift is expected. Overall, MIDAS-2 provides an adaptive drug screening and subgroup exploring framework to accelerate immunotherapy development in an efficient, accurate, and integrated fashion.

Using biomarkers to allocate patients in a response-adaptive clinical trial

In this paper, we discuss a response adaptive randomization method, and why it should be used in clinical trials for rare diseases compared to a randomized controlled trial with equal fixed randomization. The developed method uses a patient's biomarkers to alter the allocation probability to each treatment, in order to emphasize the benefit to the trial population. The method starts with an initial burn-in period of a small number of patients, who with equal probability, are allocated to each treatment. We then use a regression method to predict the best outcome of the next patient, using their biomarkers and the information from the previous patients. This estimated best treatment is assigned to the next patient with high probability. A completed clinical trial for the effect of catumaxomab on the survival of cancer patients is used as an example to demonstrate the use of the method and the differences to a controlled trial with equal allocation. Different regression procedures are investigated and compared to a randomized controlled trial, using efficacy and ethical measures.

The impact of misclassification errors on the performance of biomarkers based on next-generation sequencing, a simulation study

The development of next-generation sequencing (NGS) opens opportunities for new applications such as liquid biopsy, in which tumor mutation genotypes can be determined by sequencing circulating tumor DNA after blood draws. However, with highly diluted samples like those obtained with liquid biopsy, NGS invariably introduces a certain level of misclassification, even with improved technology. Recently, there has been a high demand to use mutation genotypes as biomarkers for predicting prognosis and treatment selection. Many methods have also been proposed to build classifiers based on multiple loci with machine learning algorithms as biomarkers. How the higher misclassification rate introduced by liquid biopsy will affect the performance of these biomarkers has not been thoroughly investigated. In this paper, we report the results from a simulation study focused on the clinical utility of biomarkers when misclassification is present due to the current technological limit of NGS in the liquid biopsy setting. The simulation covers a range of performance profiles for current NGS platforms with different machine learning algorithms and uses actual patient genotypes. Our results show that, at the high end of the performance spectrum, the misclassification introduced by NGS had very little effect on the clinical utility of the biomarker. However, in more challenging applications with lower accuracy, misclassification could have a notable effect on clinical utility. The pattern of this effect can be complex, especially for machine learning-based classifiers. Our results show that simulation can be an effective tool for assessing different scenarios of misclassification.

New insights into adaptive enrichment designs

The transition towards personalized medicine is happening and the new experimental framework is raising several challenges, from a clinical, ethical, logistical, regulatory, and statistical perspective. To face these challenges, innovative study designs with increasing complexity have been proposed. In particular, adaptive enrichment designs are becoming more attractive for their flexibility. However, these procedures rely on an increasing number of parameters that are unknown at the planning stage of the clinical trial, so the study design requires particular care. This review is dedicated to adaptive enrichment studies with a focus on design aspects. While many papers deal with methods for the analysis, the sample size determination and the optimal allocation problem have been overlooked. We discuss the multiple aspects involved in adaptive enrichment designs that contribute to their advantages and disadvantages. The decision-making process of whether or not it is worth enriching should be driven by clinical and ethical considerations as well as scientific and statistical concerns.

Adaptive enrichment designs with a continuous biomarker

A popular design for clinical trials assessing targeted therapies is the two-stage adaptive enrichment design with recruitment in stage 2 limited to a biomarker-defined subgroup chosen based on data from stage 1. The data-dependent selection leads to statistical challenges if data from both stages are used to draw inference on treatment effects in the selected subgroup. If subgroups considered are nested, as when defined by a continuous biomarker, treatment effect estimates in different subgroups follow the same distribution as estimates in a group-sequential trial. This result is used to obtain tests controlling the familywise type I error rate (FWER) for six simple subgroup selection rules, one of which also controls the FWER for any selection rule. Two approaches are proposed: one based on multivariate normal distributions suitable if the number of possible subgroups, k, is small, and one based on Brownian motion approximations suitable for large k. The methods, applicable in the wide range of settings with asymptotically normal test statistics, are illustrated using survival data from a breast cancer trial.

Blinded sample size recalculation in adaptive enrichment designs

In the precision medicine era, (prespecified) subgroup analyses are an integral part of clinical trials. Incorporating multiple populations and hypotheses in the design and analysis plan, adaptive designs promise flexibility and efficiency in such trials. Adaptations include (unblinded) interim analyses (IAs) or blinded sample size reviews. An IA offers the possibility to select promising subgroups and reallocate sample size in further stages. Trials with these features are known as adaptive enrichment designs. Such complex designs comprise many nuisance parameters, such as prevalences of the subgroups and variances of the outcomes in the subgroups. Additionally, a number of design options including the timepoint of the sample size review and timepoint of the IA have to be selected. Here, for normally distributed endpoints, we propose a strategy combining blinded sample size recalculation and adaptive enrichment at an IA, that is, at an early timepoint nuisance parameters are reestimated and the sample size is adjusted while subgroup selection and enrichment is performed later. We discuss implications of different scenarios concerning the variances as well as the timepoints of blinded review and IA and investigate the design characteristics in simulations. The proposed method maintains the desired power if planning assumptions were inaccurate and reduces the sample size and variability of the final sample size when an enrichment is performed. Having two separate timepoints for blinded sample size review and IA improves the timing of the latter and increases the probability to correctly enrich a subgroup.

AN OMNIBUS TEST FOR DETECTION OF SUBGROUP TREATMENT EFFECTS VIA DATA PARTITIONING

Late-stage clinical trials have been conducted primarily to establish the efficacy of a new treatment in an intended population. A corollary of population heterogeneity in clinical trials is that a treatment might be effective for one or more subgroups, rather than for the whole population of interest. As an example, the phase III clinical trial of panitumumab in metastatic colorectal cancer patients failed to demonstrate its efficacy in the overall population, but a subgroup associated with tumor KRAS status was found to be promising (Peeters et al. (Am. J. Clin. Oncol. 28 (2010) 4706-4713)). As we search for such subgroups via data partitioning based on a large number of biomarkers, we need to guard against inflated type I error rates due to multiple testing. Commonly-used multiplicity adjustments tend to lose power for the detection of subgroup treatment effects. We develop an effective omnibus test to detect the existence of, at least, one subgroup treatment effect, allowing a large number of possible subgroups to be considered and possibly censored outcomes. Applied to the panitumumab trial data, the proposed test would confirm a significant subgroup treatment effect. Empirical studies also show that the proposed test is applicable to a variety of outcome variables and maintains robust statistical power.

A framework for assessing the impact of accelerated approval

The FDA's Accelerated Approval program (AA) is a regulatory program to expedite availability of products to treat serious or life-threatening illnesses that lack effective treatment alternatives. Ideally, all of the many stakeholders such as patients, physicians, regulators, and health technology assessment [HTA] agencies that are affected by AA should benefit from it. In practice, however, there is intense debate over whether evidence supporting AA is sufficient to meet the needs of the stakeholders who collectively bring an approved product into routine clinical care. As AAs have become more common, it becomes essential to be able to determine their impact objectively and reproducibly in a way that provides for consistent evaluation of therapeutic decision alternatives. We describe the basic features of an approach for evaluating AA impact that accommodates stakeholder-specific views about potential benefits, risks, and costs. The approach is based on a formal decision-analytic framework combining predictive distributions for therapeutic outcomes (efficacy and safety) based on statistical models that incorporate findings from AA trials with stakeholder assessments of various actions that might be taken. The framework described here provides a starting point for communicating the value of a treatment granted AA in the context of what is important to various stakeholders.

Selecting predictive biomarkers from genomic data

Recently there have been tremendous efforts to develop statistical procedures which allow to determine subgroups of patients for which certain treatments are effective. This article focuses on the selection of prognostic and predictive genetic biomarkers based on a relatively large number of candidate Single Nucleotide Polymorphisms (SNPs). We consider models which include prognostic markers as main effects and predictive markers as interaction effects with treatment. We compare different high-dimensional selection approaches including adaptive lasso, a Bayesian adaptive version of the Sorted L-One Penalized Estimator (SLOBE) and a modified version of the Bayesian Information Criterion (mBIC2). These are compared with classical multiple testing procedures for individual markers. Having identified predictive markers we consider several different approaches how to specify subgroups susceptible to treatment. Our main conclusion is that selection based on mBIC2 and SLOBE has similar predictive performance as the adaptive lasso while including substantially fewer biomarkers.

Dynamic contrast-enhanced MRI in malignant pleural mesothelioma: prediction of outcome based on DCE-MRI measurements in patients undergoing cytotoxic chemotherapy

Background

The malignant pleural mesothelioma (MPM) response rate to chemotherapy is low. The identification of imaging biomarkers that could help guide the most effective therapy approach for individual patients is highly desirable. Our aim was to investigate the dynamic contrast-enhanced (DCE) MR parameters as predictors for progression-free (PFS) and overall survival (OS) in patients with MPM treated with cisplatin-based chemotherapy.

Methods

Thirty-two consecutive patients with MPM were enrolled in this prospective study. Pretreatment and intratreatment DCE-MRI were scheduled in each patient. The DCE parameters were analyzed using the extended Tofts (ET) and the adiabatic approximation tissue homogeneity (AATH) model. Comparison analysis, logistic regression and ROC analysis were used to identify the predictors for the patient’s outcome.

Results

Patients with higher pretreatment ET and AATH-calculated K^trans and v_e values had longer OS (P≤.006). Patients with a more prominent reduction in ET-calculated K^trans and k_ep values during the early phase of chemotherapy had longer PFS (P =.008). No parameter was identified to predict PFS. Pre-treatment ET-calculated K^trans was found to be an independent predictive marker for longer OS (P=.02) demonstrating the most favourable discrimination performance compared to other DCE parameters with an estimated sensitivity of 89% and specificity of 78% (AUC 0.9, 95% CI 0.74-0.98, cut off > 0.08 min^-1).

Conclusions

In the present study, higher pre-treatment ET-calculated K^trans values were associated with longer OS. The results suggest that DCE-MRI might provide additional information for identifying MPM patients that may respond to chemotherapy.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-022-09277-x.

A targeted simulation-extrapolation method for evaluating biomarkers based on new technologies in precision medicine

New technologies for novel biomarkers have transformed the field of precision medicine. However, in applications such as liquid biopsy for early tumor detection, the misclassification rates of next generation sequencing and other technologies have become an unavoidable feature of biomarker development. Because initial experiments are usually confined to specific technology choices and application settings, a statistical method that can project the performance metrics of other scenarios with different misclassification rates would be very helpful for planning further biomarker development and future trials. In this article, we describe an approach based on an extended version of simulation extrapolation (SIMEX) to project the performance of biomarkers measured with varying misclassification rates due to different technological or application settings when experimental results are only available from one specific setting. Through simulation studies for logistic regression and proportional hazards models, we show that our proposed method can be used to project the biomarker performance with good precision when switching from one to anther technology or application setting. Similar to the original SIMEX model, the proposed method can be implemented with existing software in a straightforward manner. A data analysis example is also presented using a lung cancer data set and performance metrics for two gene panel based biomarkers. Results demonstrate that it is feasible to infer the potential implications of using a range of technologies or application scenarios for biomarkers with limited human trial data.

Subgroup identification in individual participant data meta-analysis using model-based recursive partitioning

Model-based recursive partitioning (MOB) can be used to identify subgroups with differing treatment effects. The detection rate of treatment-by-covariate interactions and the accuracy of identified subgroups using MOB depend strongly on the sample size. Using data from multiple randomized controlled clinical trials can overcome the problem of too small samples. However, naively pooling data from multiple trials may result in the identification of spurious subgroups as differences in study design, subject selection and other sources of between-trial heterogeneity are ignored. In order to account for between-trial heterogeneity in individual participant data (IPD) meta-analysis random-effect models are frequently used. Commonly, heterogeneity in the treatment effect is modelled using random effects whereas heterogeneity in the baseline risks is modelled by either fixed effects or random effects. In this article, we propose metaMOB, a procedure using the generalized mixed-effects model tree (GLMM tree) algorithm for subgroup identification in IPD meta-analysis. Although the application of metaMOB is potentially wider, e.g. randomized experiments with participants in social sciences or preclinical experiments in life sciences, we focus on randomized controlled clinical trials. In a simulation study, metaMOB outperformed GLMM trees assuming a random intercept only and model-based recursive partitioning (MOB), whose algorithm is the basis for GLMM trees, with respect to the false discovery rates, accuracy of identified subgroups and accuracy of estimated treatment effect. The most robust and therefore most promising method is metaMOB with fixed effects for modelling the between-trial heterogeneity in the baseline risks.

Data-Driven Subgroup Identification in Confirmatory Clinical Trials

Data-driven subgroup analysis plays an important role in clinical trials. This paper focuses on practical considerations in post-hoc subgroup investigations in the context of confirmatory clinical trials. The analysis is aimed at assessing the heterogeneity of treatment effects across the trial population and identifying patient subgroups with enhanced treatment benefit. The subgroups are defined using baseline patient characteristics, including demographic and clinical factors. Much progress has been made in the development of reliable statistical methods for subgroup investigation, including methods based on global models and recursive partitioning. The paper provides a review of principled approaches to data-driven subgroup identification and illustrates subgroup analysis strategies using a family of recursive partitioning methods known as the SIDES (subgroup identification based on differential effect search) methods. These methods are applied to a Phase III trial in patients with metastatic colorectal cancer. The paper discusses key considerations in subgroup exploration, including the role of covariate adjustment, subgroup analysis at early decision points and interpretation of subgroup search results in trials with a positive overall effect.

Frequentist rules for regulatory approval of subgroups in phase III trials: A fresh look at an old problem

Background

The number of Phase III trials that include a biomarker in design and analysis has increased due to interest in personalised medicine. For genetic mutations and other predictive biomarkers, the trial sample comprises two subgroups, one of which, say B+ is known or suspected to achieve a larger treatment effect than the other B−. Despite treatment effect heterogeneity, trials often draw patients from both subgroups, since the lower responding B− subgroup may also gain benefit from the intervention. In this case, regulators/commissioners must decide what constitutes sufficient evidence to approve the drug in the B− population.

Methods and Results

Assuming trial analysis can be completed using generalised linear models, we define and evaluate three frequentist decision rules for approval. For rule one, the significance of the average treatment effect in B− should exceed a pre-defined minimum value, say ZB−>L. For rule two, the data from the low-responding group B− should increase statistical significance. For rule three, the subgroup-treatment interaction should be non-significant, using type I error chosen to ensure that estimated difference between the two subgroup effects is acceptable. Rules are evaluated based on conditional power, given that there is an overall significant treatment effect. We show how different rules perform according to the distribution of patients across the two subgroups and when analyses include additional (stratification) covariates in the analysis, thereby conferring correlation between subgroup effects.

Conclusions

When additional conditions are required for approval of a new treatment in a lower response subgroup, easily applied rules based on minimum effect sizes and relaxed interaction tests are available. Choice of rule is influenced by the proportion of patients sampled from the two subgroups but less so by the correlation between subgroup effects.

Urgently seeking efficiency and sustainability of clinical trials in global health

This paper shows the scale of global health research and the context in which we frame the subsequent papers in the Series. In this Series paper, we provide a historical perspective on clinical trial research by revisiting the 1948 streptomycin trial for pulmonary tuberculosis, which was the first documented randomised clinical trial in the English language, and we discuss its close connection with global health. We describe the current state of clinical trial research globally by providing an overview of clinical trials that have been registered in the WHO International Clinical Trial Registry since 2010. We discuss challenges with current trial planning and designs that are often used in clinical trial research undertaken in low-income and middle-income countries, as an overview of the global health trials landscape. Finally, we discuss the importance of collaborative work in global health research towards generating sustainable and culturally appropriate research environments.

Optimizing subgroup selection in two-stage adaptive enrichment and umbrella designs

We design two‐stage confirmatory clinical trials that use adaptation to find the subgroup of patients who will benefit from a new treatment, testing for a treatment effect in each of two disjoint subgroups. Our proposal allows aspects of the trial, such as recruitment probabilities of each group, to be altered at an interim analysis. We use the conditional error rate approach to implement these adaptations with protection of overall error rates. Applying a Bayesian decision‐theoretic framework, we optimize design parameters by maximizing a utility function that takes the population prevalence of the subgroups into account. We show results for traditional trials with familywise error rate control (using a closed testing procedure) as well as for umbrella trials in which only the per‐comparison type 1 error rate is controlled. We present numerical examples to illustrate the optimization process and the effectiveness of the proposed designs.

Adaptive Enrichment Designs in Clinical Trials

Adaptive enrichment designs for clinical trials may include rules that use interim data to identify treatment-sensitive patient subgroups, select or compare treatments, or change entry criteria. A common setting is a trial to compare a new biologically targeted agent to standard therapy. An enrichment design's structure depends on its goals, how it accounts for patient heterogeneity and treatment effects, and practical constraints. This article first covers basic concepts, including treatment-biomarker interaction, precision medicine, selection bias, and sequentially adaptive decision making, and briefly describes some different types of enrichment. Numerical illustrations are provided for qualitatively different cases involving treatment-biomarker interactions. Reviews are given of adaptive signature designs; a Bayesian design that uses a random partition to identify treatment-sensitive biomarker subgroups and assign treatments; and designs that enrich superior treatment sample sizes overall or within subgroups, make subgroup-specific decisions, or include outcome-adaptive randomization.

A Bayesian Decision-Theoretic Design for Simultaneous Biomarker-Based Subgroup Selection and Efficacy Evaluation

The success of drug development of targeted therapy often hinges on an appropriate selection of the sensitive patient population, mostly based on patients' biomarker levels. At the planning stage of a phase II study, although a potential biomarker may have been identified, a threshold value for defining sensitive patient population is often unavailable for adopting many existing biomarker-guided designs. To address this issue, we propose a two-stage design that allows for simultaneous biomarker threshold selection and efficacy evaluation while accommodating situations where the drug is efficacious in the entire patient population. The design uses a Bayesian decision-theoretic approach and incorporates the benefit and cost considerations of the study into a utility function. The operating characteristics of the proposed design under different scenarios are investigated via simulations. We also provide a discussion on the choice of the benefit and cost parameters in practice.

Bayesian Decision-Theoretic Methods for Survival Data using Stochastic Optimization

We introduce a principled method for Bayesian subgroup analysis. The approach is based on casting subgroup analysis as a Bayesian decision problem. The two main innovations are: (1) the explicit consideration of a "subgroup report," comprising multiple subpopulations; and (2) adapting an inhomogeneous Markov chain Monte Carlo simulation scheme to implement stochastic optimization. The latter makes the search for "subgroup reports" practically feasible.

Subgroup-adaptive randomization for subgroup confirmation in clinical trials

A well-known issue when testing for treatment-by-subgroup interaction is its low power, as clinical trials are generally powered for establishing efficacy claims for the overall population, and they are usually not adequately powered for detecting interaction (Alosh, Huque, & Koch [2015] Journal of Biopharmaceutical Statistics, 25, 1161-1178). Hence, it is necessary to develop an adaptive design to improve the efficiency of detecting heterogeneous treatment effects within subgroups. Considering Neyman allocation can maximize the power of usual Z-test (see p. 194 of the book edited by Rosenberger and Lachin), we propose a subgroup-adaptive randomization procedure aiming to achieve Neyman allocation in both predefined subgroups and overall study population in this paper. To verify whether the proposed randomization procedure works as intended, relevant theoretical results are derived and displayed . Numerical studies show that the proposed randomization procedure has obvious advantages in power of tests compared with complete randomization and Pocock and Simon's minimization method.

Multi-stage adaptive enrichment trial design with subgroup estimation

We consider the problem of estimating the best subgroup and testing for treatment effect in a clinical trial. We define the best subgroup as the subgroup that maximizes a utility function that reflects the trade-off between the subgroup size and the treatment effect. For moderate effect sizes and sample sizes, simpler methods for subgroup estimation worked better than more complex tree-based regression approaches. We propose a three-stage design with a weighted inverse normal combination test to test the hypothesis of no treatment effect across the three stages.

Exposure-response modelling approaches for determining optimal dosing rules in children

Within paediatric populations, there may be distinct age groups characterised by different exposure-response relationships. Several regulatory guidance documents have suggested general age groupings. However, it is not clear whether these categorisations will be suitable for all new medicines and in all disease areas. We consider two model-based approaches to quantify how exposure-response model parameters vary over a continuum of ages: Bayesian penalised B-splines and model-based recursive partitioning. We propose an approach for deriving an optimal dosing rule given an estimate of how exposure-response model parameters vary with age. Methods are initially developed for a linear exposure-response model. We perform a simulation study to systematically evaluate how well the various approaches estimate linear exposure-response model parameters and the accuracy of recommended dosing rules. Simulation scenarios are motivated by an application to epilepsy drug development. Results suggest that both bootstrapped model-based recursive partitioning and Bayesian penalised B-splines can estimate underlying changes in linear exposure-response model parameters as well as (and in many scenarios, better than) a comparator linear model adjusting for a categorical age covariate with levels following International Conference on Harmonisation E11 groupings. Furthermore, the Bayesian penalised B-splines approach consistently estimates the intercept and slope more accurately than the bootstrapped model-based recursive partitioning. Finally, approaches are extended to estimate Emax exposure-response models and are illustrated with an example motivated by an in vitro study of cyclosporine.

A pragmatic adaptive enrichment design for selecting the right target population for cancer immunotherapies

One of the challenges in the design of confirmatory trials is to deal with uncertainties regarding the optimal target population for a novel drug. Adaptive enrichment designs (AED) which allow for a data-driven selection of one or more prespecified biomarker subpopulations at an interim analysis have been proposed in this setting but practical case studies of AEDs are still relatively rare. We present the design of an AED with a binary endpoint in the highly dynamic setting of cancer immunotherapy. The trial was initiated as a conventional trial in early triple-negative breast cancer but amended to an AED based on emerging data external to the trial suggesting that PD-L1 status could be a predictive biomarker. Operating characteristics are discussed including the concept of a minimal detectable difference, that is, the smallest observed treatment effect that would lead to a statistically significant result in at least one of the target populations at the interim or the final analysis, respectively, in the setting of AED.

Efficient Adaptive Designs for Clinical Trials of Interventions for COVID-19

The COVID-19 pandemic has led to an unprecedented response in terms of clinical research activity. An important part of this research has been focused on randomized controlled clinical trials to evaluate potential therapies for COVID-19. The results from this research need to be obtained as rapidly as possible. This presents a number of challenges associated with considerable uncertainty over the natural history of the disease and the number and characteristics of patients affected, and the emergence of new potential therapies. These challenges make adaptive designs for clinical trials a particularly attractive option. Such designs allow a trial to be modified on the basis of interim analysis data or stopped as soon as sufficiently strong evidence has been observed to answer the research question, without compromising the trial's scientific validity or integrity. In this article, we describe some of the adaptive design approaches that are available and discuss particular issues and challenges associated with their use in the pandemic setting. Our discussion is illustrated by details of four ongoing COVID-19 trials that have used adaptive designs.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

A semiparametric Bayesian approach to population finding with time-to-event and toxicity data in a randomized clinical trial

A utility-based Bayesian population finding (BaPoFi) method was proposed by Morita and Müller to analyze data from a randomized clinical trial with the aim of identifying good predictive baseline covariates for optimizing the target population for a future study. The approach casts the population finding process as a formal decision problem together with a flexible probability model using a random forest to define a regression mean function. BaPoFi is constructed to handle a single continuous or binary outcome variable. In this paper, we develop BaPoFi-TTE as an extension of the earlier approach for clinically important cases of time-to-event (TTE) data with censoring, and also accounting for a toxicity outcome. We model the association of TTE data with baseline covariates using a semiparametric failure time model with a Pólya tree prior for an unknown error term and a random forest for a flexible regression mean function. We define a utility function that addresses a trade-off between efficacy and toxicity as one of the important clinical considerations for population finding. We examine the operating characteristics of the proposed method in extensive simulation studies. For illustration, we apply the proposed method to data from a randomized oncology clinical trial. Concerns in a preliminary analysis of the same data based on a parametric model motivated the proposed more general approach.

Pretreatment Prevotella-to-Bacteroides ratio and salivary amylase gene copy number as prognostic markers for dietary weight loss

BACKGROUND

The inconsistent link observed between salivary amylase gene copy number (AMY1 CN) and weight management is likely modified by diet and microbiome.

OBJECTIVE

Based on analysis of a previously published study, we investigated the hypothesis that interaction between diet, Prevotella-to-Bacteriodes ratio (P/B ratio), and AMY1 CN influence weight change.

METHODS

Sixty-two people with increased waist circumference were randomly assigned to receive an ad libitum New Nordic Diet (NND) high in dietary fiber, whole grain, intrinsic sugars, and starch or an Average Danish (Western) Diet (ADD) for 26 weeks. All foods were provided free of charge. Before subjects were randomly assigned to receive the NND or ADD diet, blood and fecal samples were collected, from which AMY1 CN and P/B ratio, respectively, were determined. Body weight change was described by using linear mixed models, including biomarker [log10(P/B ratio) and/or AMY1 CN] diet-group interactions.

RESULTS

Baseline means ± SDs of log10(P/B ratio) and AMY1 CN were -2.1 ± 1.8 and 6.6 ± 2.4, respectively. Baseline P/B ratio predicted a 0.99-kg/unit (95% CI: 0.40, 1.57; n = 54; P < 0.001) higher weight loss for those subjects on the NND compared with those on the ADD diet, whereas AMY1 CN was not found to predict weight loss differences between the NND and ADD groups [0.05 kg/CN (95% CI: -0.40, 0.51; n = 54; P = 0.83)]. However, among subjects with low AMY1 CN (<6.5 copies), baseline P/B ratio predicted a 2.12-kg/unit (95% CI: 1.37, 2.88; n = 30; P < 0.001) higher weight loss for the NND group than the ADD group. No such differences in weight loss were found among subjects in both groups with high AMY1 CN [-0.17 kg/unit (95% CI: -1.01, 0.66; n = 24; P = 0.68)].

CONCLUSIONS

The combined use of low AMY1 CN and pretreatment P/B ratio for weight loss prediction led to highly individualized weight loss results with the introduction of more fiber, whole grain, intrinsic sugars, and starch in the diet. These preliminary observations suggest that more undigested starch reaches the colon in individuals with low AMY1 CN, and that the fate of this starch depends on the gut microbiota composition. This trial was registered at clinicaltrials.gov as NCT01195610.

Optimized multiple testing procedures for nested sub-populations based on a continuous biomarker

An important step in the development of targeted therapies is the identification and confirmation of sub-populations where the treatment has a positive treatment effect compared to a control. These sub-populations are often based on continuous biomarkers, measured at baseline. For example, patients can be classified into biomarker low and biomarker high subgroups, which are defined via a threshold on the continuous biomarker. However, if insufficient information on the biomarker is available, the a priori choice of the threshold can be challenging and it has been proposed to consider several thresholds and to apply appropriate multiple testing procedures to test for a treatment effect in the corresponding subgroups controlling the family-wise type 1 error rate. In this manuscript we propose a framework to select optimal thresholds and corresponding optimized multiple testing procedures that maximize the expected power to identify at least one subgroup with a positive treatment effect. Optimization is performed over a prior on a family of models, modelling the relation of the biomarker with the expected outcome under treatment and under control. We find that for the considered scenarios 3 to 4 thresholds give the optimal power. If there is a prior belief on a small subgroup where the treatment has a positive effect, additional optimization of the spacing of thresholds may result in a large benefit. The procedure is illustrated with a clinical trial example in depression.

A critical review of graphics for subgroup analyses in clinical trials

Subgroup analyses are a routine part of clinical trials to investigate whether treatment effects are homogeneous across the study population. Graphical approaches play a key role in subgroup analyses to visualise effect sizes of subgroups, to aid the identification of groups that respond differentially, and to communicate the results to a wider audience. Many existing approaches do not capture the core information and are prone to lead to a misinterpretation of the subgroup effects. In this work, we critically appraise existing visualisation techniques, propose useful extensions to increase their utility and attempt to develop an effective visualisation approach. We focus on forest plots, UpSet plots, Galbraith plots, subpopulation treatment effect pattern plot, and contour plots, and comment on other approaches whose utility is more limited. We illustrate the methods using data from a prostate cancer study.

Adaptive seamless clinical trials using early outcomes for treatment or subgroup selection: Methods, simulation model and their implementation in R

Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, for example, in terms of sample size and/or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in the form of extensive simulation studies, than conventional approaches. Here, we present a framework for adaptive treatment and subgroup selection using the same notation, which links the somewhat disparate literature on treatment selection on one side and on subgroup selection on the other. Furthermore, we introduce a flexible and efficient simulation model that serves both designs. As primary endpoints often take a long time to observe, interim analyses are frequently informed by early outcomes. Therefore, all methods presented accommodate interim analyses informed by either the primary outcome or an early outcome. The R package asd, previously developed to simulate designs with treatment selection, was extended to include subgroup selection (so‐called adaptive enrichment designs). Here, we describe the functionality of the R package asd and use it to present some worked‐up examples motivated by clinical trials in chronic obstructive pulmonary disease and oncology. The examples both illustrate various features of the R package and provide insights into the operating characteristics of adaptive seamless studies.

Confidence interval estimation for the changepoint of treatment stratification in the presence of a qualitative covariate-treatment interaction

The goal in stratified medicine is to administer the "best" treatment to a patient. Not all patients might benefit from the same treatment; the choice of best treatment can depend on certain patient characteristics. In this article, it is assumed that a time-to-event outcome is considered as a patient-relevant outcome and a qualitative interaction between a continuous covariate and treatment exists, ie, that patients with different values of one specific covariate should be treated differently. We suggest and investigate different methods for confidence interval estimation for the covariate value, where the treatment recommendation should be changed based on data collected in a randomized clinical trial. An adaptation of Fieller's theorem, the delta method, and different bootstrap approaches (normal, percentile-based, wild bootstrap) are investigated and compared in a simulation study. Extensions to multivariable problems are presented and evaluated. We observed appropriate confidence interval coverage following Fieller's theorem irrespective of sample size but at the cost of very wide or even infinite confidence intervals. The delta method and the wild bootstrap approach provided the smallest intervals but inadequate coverage for small to moderate event numbers, also depending on the location of the true changepoint. For the percentile-based bootstrap, wide intervals were observed, and it was slightly conservative regarding coverage, whereas the normal bootstrap did not provide acceptable results for many scenarios. The described methods were also applied to data from a randomized clinical trial comparing two treatments for patients with symptomatic, severe carotid artery stenosis, considering patient's age as predictive marker.

Comparison of crossover and parallel-group designs for the identification of a binary predictive biomarker of the treatment effect

Pros and cons of crossover design are well known for estimating the treatment effect compared to parallel-group design, but remain unclear for identifying and estimating an interaction between a potential biomarker and the treatment effect. Such 'predictive' biomarkers, or 'effect modifiers', help to predict the response to specific treatments. The purpose of this report was to better characterize the advantages and disadvantages of crossover versus parallel-group design to identify predictive biomarkers. The treatment effect, the effect of a binary biomarker and their interaction were modelled using a linear model. The intra-subject correlation in the crossover design was taken into account through an intra-class correlation coefficient. The variance-covariance matrix of the parameters was derived and compared. For both trial designs, the variance of the parameter estimating an interaction between the treatment effect and a potential predictive biomarker corresponds to the variance of the parameter estimating the treatment effect, multiplied by the inverse of the frequency of the candidate biomarker. The ratio of the variance of the interaction parameter in the crossover to the variance estimated in the parallel-group design depends on the complement of the intra-class correlation coefficient. When planning a clinical trial including a search for candidate biomarker, the frequency of the candidate biomarker helps design the sample size, and the intra-subject correlation of the outcome should be taken into account for choosing between parallel-group and crossover designs.

Bayesian Approaches to Subgroup Analysis and Related Adaptive Clinical Trial Designs

We review Bayesian and Bayesian decision theoretic approaches to subgroup analysis and applications to subgroup-based adaptive clinical trial designs. Subgroup analysis refers to inference about subpopulations with significantly distinct treatment effects. The discussion mainly focuses on inference for a benefiting subpopulation, that is, a characterization of a group of patients who benefit from the treatment under consideration more than the overall population. We introduce alternative approaches and demonstrate them with a small simulation study. Then, we turn to clinical trial designs. When the selection of the interesting subpopulation is carried out as the trial proceeds, the design becomes an adaptive clinical trial design, using subgroup analysis to inform the randomization and assignment of treatments to patients. We briefly review some related designs. There are a variety of approaches to Bayesian subgroup analysis. Practitioners should consider the type of subpopulations in which they are interested and choose their methods accordingly. We demonstrate how subgroup analysis can be carried out by different Bayesian methods and discuss how they identify slightly different subpopulations.

A comparison of subgroup identification methods in clinical drug development: Simulation study and regulatory considerations

With advancement of technologies such as genomic sequencing, predictive biomarkers have become a useful tool for the development of personalized medicine. Predictive biomarkers can be used to select subsets of patients, which are most likely to benefit from a treatment. A number of approaches for subgroup identification were proposed over the last years. Although overviews of subgroup identification methods are available, systematic comparisons of their performance in simulation studies are rare. Interaction trees (IT), model-based recursive partitioning, subgroup identification based on differential effect, simultaneous threshold interaction modeling algorithm (STIMA), and adaptive refinement by directed peeling were proposed for subgroup identification. We compared these methods in a simulation study using a structured approach. In order to identify a target population for subsequent trials, a selection of the identified subgroups is needed. Therefore, we propose a subgroup criterion leading to a target subgroup consisting of the identified subgroups with an estimated treatment difference no less than a pre-specified threshold. In our simulation study, we evaluated these methods by considering measures for binary classification, like sensitivity and specificity. In settings with large effects or huge sample sizes, most methods perform well. For more realistic settings in drug development involving data from a single trial only, however, none of the methods seems suitable for selecting a target population. Using the subgroup criterion as alternative to the proposed pruning procedures, STIMA and IT can improve their performance in some settings. The methods and the subgroup criterion are illustrated by an application in amyotrophic lateral sclerosis.

A multiple comparison procedure for dose-finding trials with subpopulations

Identifying subgroups of patients with an enhanced response to a new treatment has become an area of increased interest in the last few years. When there is knowledge about possible subpopulations with an enhanced treatment effect before the start of a trial it might be beneficial to set up a testing strategy, which tests for a significant treatment effect not only in the full population, but also in these prespecified subpopulations. In this paper, we present a parametric multiple testing approach for tests in multiple populations for dose-finding trials. Our approach is based on the MCP-Mod methodology, which uses multiple comparison procedures (MCPs) to test for a dose-response signal, while considering multiple possible candidate dose-response shapes. Our proposed methods allow for heteroscedastic error variances between populations and control the family-wise error rate over tests in multiple populations and for multiple candidate models. We show in simulations that the proposed multipopulation testing approaches can increase the power to detect a significant dose-response signal over the standard single-population MCP-Mod, when the specified subpopulation has an enhanced treatment effect.

A conditional error function approach for adaptive enrichment designs with continuous endpoints

Adaptive enrichment designs offer an efficient and flexible way to demonstrate the efficacy of a treatment in a clinically defined full population or in, eg, biomarker-defined subpopulations while controlling the family-wise Type I error rate in the strong sense. Frequently used testing strategies in designs with two or more stages include the combination test and the conditional error function approach. Here, we focus on the latter and present some extensions. In contrast to previous work, we allow for multiple subgroups rather than one subgroup only. For nested as well as nonoverlapping subgroups with normally distributed endpoints, we explore the effect of estimating the variances in the subpopulations. Instead of using a normal approximation, we derive new t-distribution-based methods for two different scenarios. First, in the case of equal variances across the subpopulations, we present exact results using a multivariate t-distribution. Second, in the case of potentially varying variances across subgroups, we provide some improved approximations compared to the normal approximation. The performance of the proposed conditional error function approaches is assessed and compared to the combination test in a simulation study. The proposed methods are motivated by an example in pulmonary arterial hypertension.

A Simulation Study Comparing Different Statistical Approaches for the Identification of Predictive Biomarkers

Identification of relevant biomarkers that are associated with a treatment effect is one requirement for adequate treatment stratification and consequently to improve health care by administering the best available treatment to an individual patient. Various statistical approaches were proposed that allow assessing the interaction between a continuous covariate and treatment. Nevertheless, categorization of a continuous covariate, e.g., by splitting the data at the observed median value, appears to be very prevalent in practice. In this article, we present a simulation study considering data as observed in a randomized clinical trial with a time-to-event outcome performed to compare properties of such approaches, namely, Cox regression with linear interaction, Multivariable Fractional Polynomials for Interaction (MFPI), Local Partial-Likelihood Bootstrap (LPLB), and the Subpopulation Treatment Effect Pattern Plot (STEPP) method, and of strategies based on categorization of continuous covariates (splitting the covariate at the median, splitting at quartiles, and using an "optimal" split by maximizing a corresponding test statistic). In different scenarios with no interactions, linear interactions or nonlinear interactions, type I error probability and the power for detection of a true covariate-treatment interaction were estimated. The Cox regression approach was more efficient than the other methods for scenarios with monotonous interactions, especially when the number of observed events was small to moderate. When patterns of the biomarker-treatment interaction effect were more complex, MFPI and LPLB performed well compared to the other approaches. Categorization of data generally led to a loss of power, but for very complex patterns, splitting the data into multiple categories might help to explore the nature of the interaction effect. Consequently, we recommend application of statistical methods developed for assessment of interactions between continuous biomarkers and treatment instead of arbitrary or data-driven categorization of continuous covariates.

A proof-of-concept application of a novel scoring approach for personalized medicine in multiple sclerosis

BACKGROUND

Stratified medicine methodologies based on subgroup analyses are often insufficiently powered. More powerful personalized medicine approaches are based on continuous scores.

OBJECTIVE

We deployed a patient-specific continuous score predicting treatment response in patients with relapsing-remitting multiple sclerosis (RRMS).

METHODS

Data from two independent randomized controlled trials (RCTs) were used to build and validate an individual treatment response (ITR) score, regressing annualized relapse rates (ARRs) on a set of baseline predictors.

RESULTS

The ITR score for the combined treatment groups versus placebo detected differential clinical response in both RCTs. High responders in one RCT had a cross-validated ARR ratio of 0.29 (95% confidence interval (CI) = 0.13-0.55) versus 0.62 (95% CI = 0.47-0.83) for all other responders (heterogeneity p = 0.038) and were validated in the other RCT, with the corresponding ARR ratios of 0.31 (95% CI = 0.18-0.56) and 0.61 (95% CI = 0.47-0.79; heterogeneity p = 0.036). The strongest treatment effect modifiers were the Short Form-36 Physical Component Summary, age, Visual Function Test 2.5%, prior MS treatment and Expanded Disability Status Scale.

CONCLUSION

Our modelling strategy detects and validates an ITR score and opens up avenues for building treatment response calculators that are also applicable in routine clinical practice.

Machine learning methods for developing precision treatment rules with observational data

Clinical trials have identified a variety of predictor variables for use in precision treatment protocols, ranging from clinical biomarkers and symptom profiles to self-report measures of various sorts. Although such variables are informative collectively, none has proven sufficiently powerful to guide optimal treatment selection individually. This has prompted growing interest in the development of composite precision treatment rules (PTRs) that are constructed by combining information across a range of predictors. But this work has been hampered by the generally small samples in randomized clinical trials and the use of suboptimal analysis methods to analyze the resulting data. In this paper, we propose to address the sample size problem by: working with large observational electronic medical record databases rather than controlled clinical trials to develop preliminary PTRs; validating these preliminary PTRs in subsequent pragmatic trials; and using ensemble machine learning methods rather than individual algorithms to carry out statistical analyses to develop the PTRs. The major challenges in this proposed approach are that treatment are not randomly assigned in observational databases and that these databases often lack measures of key prescriptive predictors and mental disorder treatment outcomes. We proposed a tiered case-cohort design approach that uses innovative methods for measuring and balancing baseline covariates and estimating PTRs to address these challenges.

Studying treatment-effect heterogeneity in precision medicine through induced subgroups

Precision medicine, in the sense of tailoring the choice of medical treatment to patients' pretreatment characteristics, is nowadays gaining a lot of attention. Preferably, this tailoring should be realized in an evidence-based way, with key evidence in this regard pertaining to subgroups of patients that respond differentially to treatment (i.e., to subgroups involved in treatment-subgroup interactions). Often a-priori hypotheses on subgroups involved in treatment-subgroup interactions are lacking or are incomplete at best. Therefore, methods are needed that can induce such subgroups from empirical data on treatment effectiveness in a post hoc manner. Recently, quite a few such methods have been developed. So far, however, there is little empirical experience in their usage. This may be problematic for medical statisticians and statistically minded medical researchers, as many (nontrivial) choices have to be made during the data-analytic process. The main purpose of this paper is to discuss the major concepts and considerations when using these methods. This discussion will be based on a systematic, conceptual, and technical analysis of the type of research questions at play, and of the type of data that the methods can handle along with the available software, and a review of available empirical evidence. We will illustrate all this with the analysis of a dataset comparing several anti-depressant treatments.

Nested combination tests with a time-to-event endpoint using a short-term endpoint for design adaptations

Adaptive trial methodology for multiarmed trials and enrichment designs has been extensively discussed in the past. A general principle to construct test procedures that control the family-wise Type I error rate in the strong sense is based on combination tests within a closed test. Using survival data, a problem arises when using information of patients for adaptive decision making, which are under risk at interim. With the currently available testing procedures, either no testing of hypotheses in interim analyses is possible or there are restrictions on the interim data that can be used in the adaptation decisions as, essentially, only the interim test statistics of the primary endpoint may be used. We propose a general adaptive testing procedure, covering multiarmed and enrichment designs, which does not have these restrictions. An important application are clinical trials, where short-term surrogate endpoints are used as basis for trial adaptations, and we illustrate how such trials can be designed. We propose statistical models to assess the impact of effect sizes, the correlation structure between the short-term and the primary endpoint, the sample size, the timing of interim analyses, and the selection rule on the operating characteristics.