Bayesian design of clinical trials using the scale transformed power prior

There are many Bayesian design methods allowing for the incorporation of historical data for sample size determination (SSD) in situations where the outcome in the historical data is the same as the outcome of a new study. However, there is a dearth of methods supporting the incorporation of data from a previously completed clinical trial that investigated the same or similar treatment as the new trial but had a primary outcome that is different. We propose a simulation-based Bayesian SSD framework using the partial-borrowing scale transformed power prior (straPP). The partial-borrowing straPP is developed by applying a novel scale transformation to a traditional power prior on the parameters from the historical data model to make the information better align with the new data model. The scale transformation is based on the assumption that the standardized parameters (i.e., parameters multiplied by the square roots of their respective Fisher information matrices) are equal. To illustrate the method, we present results from simulation studies that use real data from a previously completed clinical trial to design a new clinical trial with a primary time-to-event endpoint.

Effects of sceptical priors on the performance of adaptive clinical trials with binary outcomes

It is unclear how sceptical priors impact adaptive trials. We assessed the influence of priors expressing a spectrum of scepticism on the performance of several Bayesian, multi-stage, adaptive clinical trial designs using binary outcomes under different clinical scenarios. Simulations were conducted using fixed stopping rules and stopping rules calibrated to keep type 1 error rates at approximately 5%. We assessed total sample sizes, event rates, event counts, probabilities of conclusiveness and selecting the best arm, root mean squared errors (RMSEs) of the estimated treatment effect in the selected arms, and ideal design percentages (IDPs; which combines arm selection probabilities, power, and consequences of selecting inferior arms), with RMSEs and IDPs estimated in conclusive trials only and after selecting the control arm in inconclusive trials. Using fixed stopping rules, increasingly sceptical priors led to larger sample sizes, more events, higher IDPs in simulations ending in superiority, and lower RMSEs, lower probabilities of conclusiveness/selecting the best arm, and lower IDPs when selecting controls in inconclusive simulations. With calibrated stopping rules, the effects of increased scepticism on sample sizes and event counts were attenuated, and increased scepticism increased the probabilities of conclusiveness/selecting the best arm and IDPs when selecting controls in inconclusive simulations without substantially increasing sample sizes. Results from trial designs with gentle adaptation and non-informative priors resembled those from designs with more aggressive adaptation using weakly-to-moderately sceptical priors. In conclusion, the use of somewhat sceptical priors in adaptive trial designs with binary outcomes seems reasonable when considering multiple performance metrics simultaneously.

A dynamic power prior approach to non-inferiority trials for normal means

Non-inferiority trials compare new experimental therapies to standard ones (active control). In these experiments, historical information on the control treatment is often available. This makes Bayesian methodology appealing since it allows a natural way to exploit information from past studies. In the present paper, we suggest the use of previous data for constructing the prior distribution of the control effect parameter. Specifically, we consider a dynamic power prior that possibly allows to discount the level of borrowing in the presence of heterogeneity between past and current control data. The discount parameter of the prior is based on the Hellinger distance between the posterior distributions of the control parameter based, respectively, on historical and current data. We develop the methodology for comparing normal means and we handle the unknown variance assumption using MCMC. We also provide a simulation study to analyze the proposed test in terms of frequentist size and power, as it is usually requested by regulatory agencies. Finally, we investigate comparisons with some existing methods and we illustrate an application to a real case study.

Simulating and reporting frequentist operating characteristics of clinical trials that borrow external information: Towards a fair comparison in case of one-arm and hybrid control two-arm trials

Borrowing information from historical or external data to inform inference in a current trial is an expanding field in the era of precision medicine, where trials are often performed in small patient cohorts for practical or ethical reasons. Even though methods proposed for borrowing from external data are mainly based on Bayesian approaches that incorporate external information into the prior for the current analysis, frequentist operating characteristics of the analysis strategy are often of interest. In particular, type I error rate and power at a prespecified point alternative are the focus. We propose a procedure to investigate and report the frequentist operating characteristics in this context. The approach evaluates type I error rate of the test with borrowing from external data and calibrates the test without borrowing to this type I error rate. On this basis, a fair comparison of power between the test with and without borrowing is achieved. We show that no power gains are possible in one-sided one-arm and two-arm hybrid control trials with normal endpoint, a finding proven in general before. We prove that in one-arm fixed-borrowing situations, unconditional power (i.e., when external data is random) is reduced. The Empirical Bayes power prior approach that dynamically borrows information according to the similarity of current and external data avoids the exorbitant type I error inflation occurring with fixed borrowing. In the hybrid control two-arm trial we observe power reductions as compared to the test calibrated to borrowing that increase when considering unconditional power.

Robust incorporation of historical information with known type I error rate inflation

Bayesian clinical trials can benefit from available historical information through the specification of informative prior distributions. Concerns are however often raised about the potential for prior-data conflict and the impact of Bayes test decisions on frequentist operating characteristics, with particular attention being assigned to inflation of type I error (TIE) rates. This motivates the development of principled borrowing mechanisms, that strike a balance between frequentist and Bayesian decisions. Ideally, the trust assigned to historical information defines the degree of robustness to prior-data conflict one is willing to sacrifice. However, such relationship is often not directly available when explicitly considering inflation of TIE rates. We build on available literature relating frequentist and Bayesian test decisions, and investigate a rationale for inflation of TIE rate which explicitly and linearly relates the amount of borrowing and the amount of TIE rate inflation in one-arm studies. A novel dynamic borrowing mechanism tailored to hypothesis testing is additionally proposed. We show that, while dynamic borrowing prevents the possibility to obtain a simple closed-form TIE rate computation, an explicit upper bound can still be enforced. Connections with the robust mixture prior approach, particularly in relation to the choice of the mixture weight and robust component, are made. Simulations are performed to show the properties of the approach for normal and binomial outcomes, and an exemplary application is demonstrated in a case study.

Power priors for replication studies

The ongoing replication crisis in science has increased interest in the methodology of replication studies. We propose a novel Bayesian analysis approach using power priors: The likelihood of the original study's data is raised to the power of α , and then used as the prior distribution in the analysis of the replication data. Posterior distribution and Bayes factor hypothesis tests related to the power parameter α quantify the degree of compatibility between the original and replication study. Inferences for other parameters, such as effect sizes, dynamically borrow information from the original study. The degree of borrowing depends on the conflict between the two studies. The practical value of the approach is illustrated on data from three replication studies, and the connection to hierarchical modeling approaches explored. We generalize the known connection between normal power priors and normal hierarchical models for fixed parameters and show that normal power prior inferences with a beta prior on the power parameter α align with normal hierarchical model inferences using a generalized beta prior on the relative heterogeneity variance I 2 . The connection illustrates that power prior modeling is unnatural from the perspective of hierarchical modeling since it corresponds to specifying priors on a relative rather than an absolute heterogeneity scale.

Adaptively leveraging external data with robust meta-analytical-predictive prior using empirical Bayes

The robust meta-analytical-predictive (rMAP) prior is a popular method to robustly leverage external data. However, a mixture coefficient would need to be pre-specified based on the anticipated level of prior-data conflict. This can be very challenging at the study design stage. We propose a novel empirical Bayes robust MAP (EB-rMAP) prior to address this practical need and adaptively leverage external/historical data. Built on Box's prior predictive p-value, the EB-rMAP prior framework balances between model parsimony and flexibility through a tuning parameter. The proposed framework can be applied to binomial, normal, and time-to-event endpoints. Implementation of the EB-rMAP prior is also computationally efficient. Simulation results demonstrate that the EB-rMAP prior is robust in the presence of prior-data conflict while preserving statistical power. The proposed EB-rMAP prior is then applied to a clinical dataset that comprises 10 oncology clinical trials, including the prospective study.

Bayesian borrowing from historical control data in a vaccine efficacy trial

In the context of vaccine efficacy trial where the incidence rate is very low and a very large sample size is usually expected, incorporating historical data into a new trial is extremely attractive to reduce sample size and increase estimation precision. Nevertheless, for some infectious diseases, seasonal change in incidence rates poses a huge challenge in borrowing historical data and a critical question is how to properly take advantage of historical data borrowing with acceptable tolerance to between-trials heterogeneity commonly from seasonal disease transmission. In this article, we extend a probability-based power prior which determines the amount of information to be borrowed based on the agreement between the historical and current data, to make it applicable for either a single or multiple historical trials available, with constraint on the amount of historical information to be borrowed. Simulations are conducted to compare the performance of the proposed method with other methods including modified power prior (MPP), meta-analytic-predictive (MAP) prior and the commensurate prior methods. Furthermore, we illustrate the application of the proposed method for trial design in a practical setting.

Alone, together: On the benefits of Bayesian borrowing in a meta-analytic setting

It is common practice to use hierarchical Bayesian model for the informing of a pediatric randomized controlled trial (RCT) by adult data, using a prespecified borrowing fraction parameter (BFP). This implicitly assumes that the BFP is intuitive and corresponds to the degree of similarity between the populations. Generalizing this model to any K ≥ 1 historical studies, naturally leads to empirical Bayes meta-analysis. In this paper we calculate the Bayesian BFPs and study the factors that drive them. We prove that simultaneous mean squared error reduction relative to an uninformed model is always achievable through application of this model. Power and sample size calculations for a future RCT, designed to be informed by multiple external RCTs, are also provided. Potential applications include inference on treatment efficacy from independent trials involving either heterogeneous patient populations or different therapies from a common class.

On the use of non-concurrent controls in platform trials: a scoping review

BACKGROUND

Platform trials gained popularity during the last few years as they increase flexibility compared to multi-arm trials by allowing new experimental arms entering when the trial already started. Using a shared control group in platform trials increases the trial efficiency compared to separate trials. Because of the later entry of some of the experimental treatment arms, the shared control group includes concurrent and non-concurrent control data. For a given experimental arm, non-concurrent controls refer to patients allocated to the control arm before the arm enters the trial, while concurrent controls refer to control patients that are randomised concurrently to the experimental arm. Using non-concurrent controls can result in bias in the estimate in case of time trends if the appropriate methodology is not used and the assumptions are not met.

METHODS

We conducted two reviews on the use of non-concurrent controls in platform trials: one on statistical methodology and one on regulatory guidance. We broadened our searches to the use of external and historical control data. We conducted our review on the statistical methodology in 43 articles identified through a systematic search in PubMed and performed a review on regulatory guidance on the use of non-concurrent controls in 37 guidelines published on the EMA and FDA websites.

RESULTS

Only 7/43 of the methodological articles and 4/37 guidelines focused on platform trials. With respect to the statistical methodology, in 28/43 articles, a Bayesian approach was used to incorporate external/non-concurrent controls while 7/43 used a frequentist approach and 8/43 considered both. The majority of the articles considered a method that downweights the non-concurrent control in favour of concurrent control data (34/43), using for instance meta-analytic or propensity score approaches, and 11/43 considered a modelling-based approach, using regression models to incorporate non-concurrent control data. In regulatory guidelines, the use of non-concurrent control data was considered critical but was deemed acceptable for rare diseases in 12/37 guidelines or was accepted in specific indications (12/37). Non-comparability (30/37) and bias (16/37) were raised most often as the general concerns with non-concurrent controls. Indication specific guidelines were found to be most instructive.

CONCLUSIONS

Statistical methods for incorporating non-concurrent controls are available in the literature, either by means of methods originally proposed for the incorporation of external controls or non-concurrent controls in platform trials. Methods mainly differ with respect to how the concurrent and non-concurrent data are combined and temporary changes handled. Regulatory guidance for non-concurrent controls in platform trials are currently still limited.

On efficient posterior inference in normalized power prior Bayesian analysis

The power prior has been widely used to discount the amount of information borrowed from historical data in the design and analysis of clinical trials. It is realized by raising the likelihood function of the historical data to a power parameterδ ∈ [ 0 , 1 ] $\delta \in [0, 1]$ , which quantifies the heterogeneity between the historical and the new study. In a fully Bayesian approach, a natural extension is to assign a hyperprior to δ such that the posterior of δ can reflect the degree of similarity between the historical and current data. To comply with the likelihood principle, an extra normalizing factor needs to be calculated and such prior is known as the normalized power prior. However, the normalizing factor involves an integral of a prior multiplied by a fractional likelihood and needs to be computed repeatedly over different δ during the posterior sampling. This makes its use prohibitive in practice for most elaborate models. This work provides an efficient framework to implement the normalized power prior in clinical studies. It bypasses the aforementioned efforts by sampling from the power prior withδ = 0 $\delta = 0$ andδ = 1 $\delta = 1$ only. Such a posterior sampling procedure can facilitate the use of a random δ with adaptive borrowing capability in general models. The numerical efficiency of the proposed method is illustrated via extensive simulation studies, a toxicological study, and an oncology study.

A model-based approach for historical borrowing, with an application to neovascular age-related macular degeneration

Bayesian historical borrowing has recently attracted growing interest due to the increasing availability of historical control data, as well as improved computational methodology and software. In this article, we argue that the statistical models used for borrowing may be suboptimal when they do not adjust for differing factors across historical studies such as covariates, dosing regimen, etc. We propose an alternative approach to address these shortcomings. We start by constructing a historical model based on subject-level historical data to accurately characterize the control treatment by adjusting for known between trials differences. This model is subsequently used to predict the control arm response in the current trial, enabling the derivation of a model-informed prior for the treatment effect parameter of another (potentially simpler) model used to analyze the trial efficacy (i.e. the trial model). Our approach is applied to neovascular age-related macular degeneration trials, employing a cross-sectional regression trial model, and a longitudinal non-linear mixed-effects drug-disease-trial historical model. The latter model characterizes the relationship between clinical response, drug exposure and baseline covariates so that the derived model-informed prior seamlessly adapts to the trial population and can be extrapolated to a different dosing regimen. This approach can yield a more accurate prior for borrowing, thus optimizing gains in efficiency (e.g. increasing power or reducing the sample size) in future trials.

Challenges of using external data in clinical trials- an illustration in patients with COVID-19

BACKGROUND

To improve the efficiency of clinical trials, leveraging external data on control and/or treatment effects, which is almost always available, appears to be a promising approach.

METHODS

We used data from the experimental arm of the Covidicus trial evaluating high-dose dexamethasone in severely ill and mechanically ventilated COVID-19 patients, using published data from the Recovery trial as external data, to estimate the 28-day mortality rate. Primary approaches to deal with external data were applied.

RESULTS

Estimates ranged from 0.241 ignoring the external data up to 0.294 using hierarchical Bayesian models. Some evidence of differences in mortality rates between the Covidicus and Recovery trials were observed, with an matched adjusted odds ratio of death in the Covidicus arm of 0.41 compared to the Recovery arm.

CONCLUSIONS

These indirect comparisons appear sensitive to the method used. None of those approaches appear robust enough to overcome randomized clinical trial data.

TRIAL REGISTRATION

Covidicus Trial: NCT04344730, First Posted: 14/04/2020; Recovery trial: NCT04381936.

Power prior models for estimating response rates in a small n, sequential, multiple assignment, randomized trial

A small n, sequential, multiple assignment, randomized trial (snSMART) is a small sample, two-stage design where participants receive up to two treatments sequentially, but the second treatment depends on response to the first treatment. The parameters of interest in an snSMART are the first-stage response rates of the treatments, but outcomes from both stages can be used to obtain more information from a small sample. A novel way to incorporate the outcomes from both stages uses power prior models, in which first stage outcomes from an snSMART are regarded as the primary (internal) data and second stage outcomes are regarded as supplemental data (co-data). We apply existing power prior models to snSMART data, and we also develop new extensions of power prior models. All methods are compared to each other and to the Bayesian joint stage model (BJSM) via simulation studies. By comparing the biases and the efficiency of the response rate estimates among all proposed power prior methods, we suggest application of Fisher's Exact Test or the Bhattacharyya's overlap measure to an snSMART to estimate the response rates in an snSMART, which both have performance mostly as good or better than the BJSM. We describe the situations where each of these suggested approaches is preferred.

Borrowing historical information for non-inferiority trials on Covid-19 vaccines

Non-inferiority vaccine trials compare new candidates to active controls that provide clinically significant protection against a disease. Bayesian statistics allows to exploit pre-experimental information available from previous studies to increase precision and reduce costs. Here, historical knowledge is incorporated into the analysis through a power prior that dynamically regulates the degree of information-borrowing. We examine non-inferiority tests based on credible intervals for the unknown effects-difference between two vaccines on the log odds ratio scale, with an application to new Covid-19 vaccines. We explore the frequentist properties of the method and we address the sample size determination problem.

Incorporating historical controls in clinical trials with longitudinal outcomes using the modified power prior

Several dynamic borrowing methods, such as the modified power prior (MPP), the commensurate prior, have been proposed to increase statistical power and reduce the required sample size in clinical trials where comparable historical controls are available. Most methods have focused on cross‐sectional endpoints, and appropriate methodology for longitudinal outcomes is lacking. In this study, we extend the MPP to the linear mixed model (LMM). An important question is whether the MPP should use the conditional version of the LMM (given the random effects) or the marginal version (averaged over the distribution of the random effects), which we refer to as the conditional MPP and the marginal MPP, respectively. We evaluated the MPP for one historical control arm via a simulation study and an analysis of the data of Alzheimer's Disease Cooperative Study (ADCS) with the commensurate prior as the comparator. The conditional MPP led to inflated type I error rate when there existed moderate or high between‐study heterogeneity. The marginal MPP and the commensurate prior yielded a power gain (3.6%–10.4% vs. 0.6%–4.6%) with the type I error rates close to 5% (5.2%–6.2% vs. 3.8%–6.2%) when the between‐study heterogeneity is not excessively high. For the ADCS data, all the borrowing methods improved the precision of estimates and provided the same clinical conclusions. The marginal MPP and the commensurate prior are useful for borrowing historical controls in longitudinal data analysis, while the conditional MPP is not recommended due to inflated type I error rates.

New approaches for testing non-inferiority for three-arm trials with Poisson distributed outcomes

With the availability of limited resources, innovation for improved statistical method for the design and analysis of randomized controlled trials (RCTs) is of paramount importance for newer and better treatment discovery for any therapeutic area. Although clinical efficacy is almost always the primary evaluating criteria to measure any beneficial effect of a treatment, there are several important other factors (e.g., side effects, cost burden, less debilitating, less intensive, etc.), which can permit some less efficacious treatment options favorable to a subgroup of patients. This leads to non-inferiority (NI) testing. The objective of NI trial is to show that an experimental treatment is not worse than an active reference treatment by more than a pre-specified margin. Traditional NI trials do not include a placebo arm for ethical reason; however, this necessitates stringent and often unverifiable assumptions. On the other hand, three-arm NI trials consisting of placebo, reference, and experimental treatment, can simultaneously test the superiority of the reference over placebo and NI of experimental treatment over the reference. In this article, we proposed both novel Frequentist and Bayesian procedures for testing NI in the three-arm trial with Poisson distributed count outcome. RCTs with count data as the primary outcome are quite common in various disease areas such as lesion count in cancer trials, relapses in multiple sclerosis, dermatology, neurology, cardiovascular research, adverse event count, etc. We first propose an improved Frequentist approach, which is then followed by it's Bayesian version. Bayesian methods have natural advantage in any active-control trials, including NI trial when substantial historical information is available for placebo and established reference treatment. In addition, we discuss sample size calculation and draw an interesting connection between the two paradigms.

Machine Learning for Supplementing Behavioral Assessment

The Questions About Behavioral Function (QABF) has a high degree of convergent validity, but there is still a lack of agreement between the results of the assessment and the results of experimental function analysis. Machine learning (ML) may improve the validity of assessments by using data to build a mathematical model for more accurate predictions. We used published QABF and subsequent functional analyses to train ML models to identify the function of behavior. With ML models, predictions can be made from indirect assessment results based on learning from results of past experimental functional analyses. In Experiment 1, we compared the results of five algorithms to the QABF criteria using a leave-one-out cross-validation approach. All five outperformed the QABF assessment on multilabel accuracy (i.e., percentage of predictions with the presence or absence of each function indicated correctly), but false negatives remained an issue. In Experiment 2, we augmented the data with 1,000 artificial samples to train and test an artificial neural network. The artificial network outperformed other models on all measures of accuracy. The results indicated that ML could be used to inform conditions that should be present in a functional analysis. Therefore, this study represents a proof-of-concept for the application of machine learning to functional assessment.

Dynamic borrowing from a single prior data source using the conditional power prior

The conditional power prior is a popular method to borrow information from a single prior data source. The amount of borrowing is controlled by the power parameter which is fixed before running the new study. However, fixing this parameter before running a new study is often difficult and may be unwise because if the outcomes in the current study are much different from the prior data outcomes, the power parameter cannot be changed to reflect a more appropriate degree of borrowing. On the other hand, treating the power parameter as a random variable to be updated via Bayes theorem may relinquish control over how much to borrow in cases where regulatory oversight recommends a conservative approach.Previous authors have determined the power parameter at the end of the current study based on "stochastic" similarity in the outcomes between the current study and the prior data. In this paper, we introduce some modifications to those methods. First, we determine the power parameter based on similarity between a percentage of the current study outcome data available at an interim look and the prior outcome data. This may limit potential for operational bias resulting from the determination of the power parameter after the current study is complete. Next, we introduce a new measure of similarity between the current (interim) and prior data that limits similarity by a pre-specified clinical margin. The proposed clinical similarity region may be readily understood by clinicians who need to assess when such borrowing is clinically appropriate. Through simulations, we show that our approach has low bias and good power, while reducing type I error rate in areas outside of the "similarity region". An example of a hypothetical medical device study illustrates its potential use in practice.

Hi3 + 3: A model-assisted dose-finding design borrowing historical data

BACKGROUND

In phase I clinical trials, historical data may be available through multi-regional programs, reformulation of the same drug, or previous trials for a drug under the same class. Statistical designs that borrow information from historical data can reduce cost, speed up drug development, and maintain safety.

PURPOSE

Based on a hybrid design that partly uses probability models and partly uses algorithmic rules for decision making, we aim to improve the efficiency of the dose-finding trials in the presence of historical data, maintain safety for patients, and achieve a level of simplicity for practical applications.

METHODS

We propose the Hi3 + 3 design, in which the letter "H" represents "historical data". We apply the idea in power prior to borrow historical data and define the effective sample size (ESS) of the prior. Dose-finding decision rules follow the idea in the i3 + 3 design (Liu et al., 2020 [1]) while incorporating the historical data via the power prior and ESS. The proposed Hi3 + 3 design pretabulates the dosing decisions before the trial starts, a desirable feature for ease of application in practice.

RESULTS

In most cases we investigated, the Hi3 + 3 design is superior than the i3 + 3 design due to information borrow from historical data. Even when the historical data is incompatible with the current data, it is capable of maintaining a high level of safety for trial patients and comparable performances without sacrificing the ability to identify the correct MTD too much. Ilustration of this feature are found in the simulation results.

CONCLUSION

With the demonstrated safety, efficiency, and simplicity, the Hi3 + 3 design could be a desirable choice for dose-finding trials borrowing historical data.

Power prior for borrowing the real-world data in bioequivalence test with a parallel design

Recently, real-world study has attracted wide attention for drug development. In bioequivalence study, the reference drug often has been marketed for many years and accumulated abundant real-world data. It is therefore appealing to incorporate these data in the design to improve trial efficiency. In this paper, we propose a Bayesian method to include real-world data of the reference drug in a current bioequivalence trial, with the aim to increase the power of analysis and reduce sample size for long half-life drugs. We adopt the power prior method for incorporating real-world data and use the average bioequivalence posterior probability to evaluate the bioequivalence between the test drug and the reference drug. Simulations were conducted to investigate the performance of the proposed method in different scenarios. The simulation results show that the proposed design has higher power than the traditional design without borrowing real-world data, while controlling the type I error. Moreover, the proposed method saves sample size and reduces costs for the trial.

A novel equivalence probability weighted power prior for using historical control data in an adaptive clinical trial design: A comparison to standard methods

A standard two-arm randomised controlled trial usually compares an intervention to a control treatment with equal numbers of patients randomised to each treatment arm and only data from within the current trial are used to assess the treatment effect. Historical data are used when designing new trials and have recently been considered for use in the analysis when the required number of patients under a standard trial design cannot be achieved. Incorporating historical control data could lead to more efficient trials, reducing the number of controls required in the current study when the historical and current control data agree. However, when the data are inconsistent, there is potential for biased treatment effect estimates, inflated type I error and reduced power. We introduce two novel approaches for binary data which discount historical data based on the agreement with the current trial controls, an equivalence approach and an approach based on tail area probabilities. An adaptive design is used where the allocation ratio is adapted at the interim analysis, randomising fewer patients to control when there is agreement. The historical data are down-weighted in the analysis using the power prior approach with a fixed power. We compare operating characteristics of the proposed design to historical data methods in the literature: the modified power prior; commensurate prior; and robust mixture prior. The equivalence probability weight approach is intuitive and the operating characteristics can be calculated exactly. Furthermore, the equivalence bounds can be chosen to control the maximum possible inflation in type I error.

Combined assessment of early and late-phase outcomes in orphan drug development

In drug development programs, proof‐of‐concept Phase II clinical trials typically have a biomarker as a primary outcome, or an outcome that can be observed with relatively short follow‐up. Subsequently, the Phase III clinical trials aim to demonstrate the treatment effect based on a clinical outcome that often needs a longer follow‐up to be assessed. Early‐phase outcomes or biomarkers are typically associated with late‐phase outcomes and they are often included in Phase III trials. The decision to proceed to Phase III development is based on analysis of the early‐Phase II outcome data. In rare diseases, it is likely that only one Phase II trial and one Phase III trial are available. In such cases and before drug marketing authorization requests, positive results of the early‐phase outcome of Phase II trials are then likely seen as supporting (or even replicating) positive Phase III results on the late‐phase outcome, without a formal retrospective combined assessment and without accounting for between‐study differences. We used double‐regression modeling applied to the Phase II and Phase III results to numerically mimic this informal retrospective assessment. We provide an analytical solution for the bias and mean square error of the overall effect that leads to a corrected double‐regression. We further propose a flexible Bayesian double‐regression approach that minimizes the bias by accounting for between‐study differences via discounting the Phase II early‐phase outcome when they are not in line with the Phase III biomarker outcome results. We illustrate all methods with an orphan drug example for Fabry disease.

Bayesian adaptive decision-theoretic designs for multi-arm multi-stage clinical trials

Multi-arm multi-stage clinical trials in which more than two drugs are simultaneously investigated provide gains over separate single- or two-arm trials. In this paper we propose a generic Bayesian adaptive decision-theoretic design for multi-arm multi-stage clinical trials with K (K≥2) arms. The basic idea is that after each stage a decision about continuation of the trial and accrual of patients for an additional stage is made on the basis of the expected reduction in loss. For this purpose, we define a loss function that incorporates the patient accrual costs as well as costs associated with an incorrect decision at the end of the trial. An attractive feature of our loss function is that its estimation is computationally undemanding, also when K > 2. We evaluate the frequentist operating characteristics for settings with a binary outcome and multiple experimental arms. We consider both the situation with and without a control arm. In a simulation study, we show that our design increases the probability of making a correct decision at the end of the trial as compared to nonadaptive designs and adaptive two-stage designs.

A decision-theoretic approach to Bayesian clinical trial design and evaluation of robustness to prior-data conflict

Bayesian clinical trials allow taking advantage of relevant external information through the elicitation of prior distributions, which influence Bayesian posterior parameter estimates and test decisions. However, incorporation of historical information can have harmful consequences on the trial’s frequentist (conditional) operating characteristics in case of inconsistency between prior information and the newly collected data. A compromise between meaningful incorporation of historical information and strict control of frequentist error rates is therefore often sought. Our aim is thus to review and investigate the rationale and consequences of different approaches to relaxing strict frequentist control of error rates from a Bayesian decision-theoretic viewpoint. In particular, we define an integrated risk which incorporates losses arising from testing, estimation, and sampling. A weighted combination of the integrated risk addends arising from testing and estimation allows moving smoothly between these two targets. Furthermore, we explore different possible elicitations of the test error costs, leading to test decisions based either on posterior probabilities, or solely on Bayes factors. Sensitivity analyses are performed following the convention which makes a distinction between the prior of the data-generating process, and the analysis prior adopted to fit the data. Simulation in the case of normal and binomial outcomes and an application to a one-arm proof-of-concept trial, exemplify how such analysis can be conducted to explore sensitivity of the integrated risk, the operating characteristics, and the optimal sample size, to prior-data conflict. Robust analysis prior specifications, which gradually discount potentially conflicting prior information, are also included for comparison. Guidance with respect to cost elicitation, particularly in the context of a Phase II proof-of-concept trial, is provided.

A Bayesian decision-theoretic approach to incorporate preclinical information into phase I oncology trials

Leveraging preclinical animal data for a phase I oncology trial is appealing yet challenging. In this paper, we use animal data to improve decision-making in a model-based dose-escalation procedure. We make a proposal for how to measure and address a prior-data conflict in a sequential study with a small sample size. Animal data are incorporated via a robust two-component mixture prior for the parameters of the human dose-toxicity relationship. The weights placed on each component of the prior are chosen empirically and updated dynamically as the trial progresses and more data accrue. After completion of each cohort, we use a Bayesian decision-theoretic approach to evaluate the predictive utility of the animal data for the observed human toxicity outcomes, reflecting the degree of agreement between dose-toxicity relationships in animals and humans. The proposed methodology is illustrated through several data examples and an extensive simulation study.

Incorporating historical two-arm data in clinical trials with binary outcome: A practical approach

The feasibility of a new clinical trial may be increased by incorporating historical data of previous trials. In the particular case where only data from a single historical trial are available, there exists no clear recommendation in the literature regarding the most favorable approach. A main problem of the incorporation of historical data is the possible inflation of the type I error rate. A way to control this type of error is the so-called power prior approach. This Bayesian method does not "borrow" the full historical information but uses a parameter 0 ≤ δ ≤ 1 to determine the amount of borrowed data. Based on the methodology of the power prior, we propose a frequentist framework that allows incorporation of historical data from both arms of two-armed trials with binary outcome, while simultaneously controlling the type I error rate. It is shown that for any specific trial scenario a value δ > 0 can be determined such that the type I error rate falls below the prespecified significance level. The magnitude of this value of δ depends on the characteristics of the data observed in the historical trial. Conditionally on these characteristics, an increase in power as compared to a trial without borrowing may result. Similarly, we propose methods how the required sample size can be reduced. The results are discussed and compared to those obtained in a Bayesian framework. Application is illustrated by a clinical trial example.

Critical appraisal of Bayesian dynamic borrowing from an imperfectly commensurate historical control

Bayesian dynamic borrowing designs facilitate borrowing information from historical studies. Historical data, when perfectly commensurate with current data, have been shown to reduce the trial duration and the sample size, while inflation in the type I error and reduction in the power have been reported, when imperfectly commensurate. These results, however, were obtained without considering that Bayesian designs are calibrated to meet regulatory requirements in practice and even no-borrowing designs may use information from historical data in the calibration. The implicit borrowing of historical data suggests that imperfectly commensurate historical data may similarly impact no-borrowing designs negatively. We will provide a fair appraiser of Bayesian dynamic borrowing and no-borrowing designs. We used a published selective adaptive randomization design and real clinical trial setting and conducted simulation studies under varying degrees of imperfectly commensurate historical control scenarios. The type I error was inflated under the null scenario of no intervention effect, while larger inflation was noted with borrowing. The larger inflation in type I error under the null setting can be offset by the greater probability to stop early correctly under the alternative. Response rates were estimated more precisely and the average sample size was smaller with borrowing. The expected increase in bias with borrowing was noted, but was negligible. Using Bayesian dynamic borrowing designs may improve trial efficiency by stopping trials early correctly and reducing trial length at the small cost of inflated type I error.

Power gains by using external information in clinical trials are typically not possible when requiring strict type I error control

In the era of precision medicine, novel designs are developed to deal with flexible clinical trials that incorporate many treatment strategies for multiple diseases in one trial setting. This situation often leads to small sample sizes in disease-treatment combinations and has fostered the discussion about the benefits of borrowing of external or historical information for decision-making in these trials. Several methods have been proposed that dynamically discount the amount of information borrowed from historical data based on the conformity between historical and current data. Specifically, Bayesian methods have been recommended and numerous investigations have been performed to characterize the properties of the various borrowing mechanisms with respect to the gain to be expected in the trials. However, there is common understanding that the risk of type I error inflation exists when information is borrowed and many simulation studies are carried out to quantify this effect. To add transparency to the debate, we show that if prior information is conditioned upon and a uniformly most powerful test exists, strict control of type I error implies that no power gain is possible under any mechanism of incorporation of prior information, including dynamic borrowing. The basis of the argument is to consider the test decision function as a function of the current data even when external information is included. We exemplify this finding in the case of a pediatric arm appended to an adult trial and dichotomous outcome for various methods of dynamic borrowing from adult information to the pediatric arm. In conclusion, if use of relevant external data is desired, the requirement of strict type I error control has to be replaced by more appropriate metrics.

An adaptive power prior for sequential clinical trials - Application to bridging studies

During drug evaluation trials, information from clinical trials previously conducted on another population, indications or schedules may be available. In these cases, it might be desirable to share information by efficiently using the available resources. In this work, we developed an adaptive power prior with a commensurability parameter for using historical or external information. It allows, at each stage, full borrowing when the data are not in conflict, no borrowing when the data are in conflict or "tuned" borrowing when the data are in between. We propose to apply our adaptive power prior method to bridging studies between Caucasians and Asians, and we focus on the sequential adaptive allocation design, although other design settings can be used. We weight the prior information in two steps: the effective sample size approach is used to set the maximum desirable amount of information to be shared from historical data at each step of the trial; then, in a sort of Empirical Bayes approach, a commensurability parameter is chosen using a measure of distribution distance. This approach avoids elicitation and computational issues regarding the usual Empirical Bayes approach. We propose several versions of our method, and we conducted an extensive simulation study evaluating the robustness and sensitivity to prior choices.

Optimizing the Design and Analysis of Clinical Trials for Antibacterials Against Multidrug-resistant Organisms: A White Paper From COMBACTE's STAT-Net

Innovations are urgently required for clinical development of antibacterials against multidrug-resistant organisms. Therefore, a European, public-private working group (STAT-Net; part of Combatting Bacterial Resistance in Europe [COMBACTE]), has reviewed and tested several innovative trials designs and analytical methods for randomized clinical trials, which has resulted in 8 recommendations. The first 3 focus on pharmacokinetic and pharmacodynamic modeling, emphasizing the pertinence of population-based pharmacokinetic models, regulatory procedures for the reassessment of old antibiotics, and rigorous quality improvement. Recommendations 4 and 5 address the need for more sensitive primary end points through the use of rank-based or time-dependent composite end points. Recommendation 6 relates to the applicability of hierarchical nested-trial designs, and the last 2 recommendations propose the incorporation of historical or concomitant trial data through Bayesian methods and/or platform trials. Although not all of these recommendations are directly applicable, they provide a solid, evidence-based approach to develop new, and established, antibacterials and address this public health challenge.

Quantification of prior impact in terms of effective current sample size

Bayesian methods allow borrowing of historical information through prior distributions. The concept of prior effective sample size (prior ESS) facilitates quantification and communication of such prior information by equating it to a sample size. Prior information can arise from historical observations; thus, the traditional approach identifies the ESS with such a historical sample size. However, this measure is independent of newly observed data, and thus would not capture an actual "loss of information" induced by the prior in case of prior-data conflict. We build on a recent work to relate prior impact to the number of (virtual) samples from the current data model and introduce the effective current sample size (ECSS) of a prior, tailored to the application in Bayesian clinical trial designs. Special emphasis is put on robust mixture, power, and commensurate priors. We apply the approach to an adaptive design in which the number of recruited patients is adjusted depending on the effective sample size at an interim analysis. We argue that the ECSS is the appropriate measure in this case, as the aim is to save current (as opposed to historical) patients from recruitment. Furthermore, the ECSS can help overcome lack of consensus in the ESS assessment of mixture priors and can, more broadly, provide further insights into the impact of priors. An R package accompanies the paper.

Incorporating individual historical controls and aggregate treatment effect estimates into a Bayesian survival trial: a simulation study

Background

Performing well-powered randomised controlled trials (RCTs) of new treatments for rare diseases is often infeasible. However, with the increasing availability of historical data, incorporating existing information into trials with small sample sizes is appealing in order to increase the power. Bayesian approaches enable one to incorporate historical data into a trial’s analysis through a prior distribution.

Methods

Motivated by a RCT intended to evaluate the impact on event-free survival of mifamurtide in patients with osteosarcoma, we performed a simulation study to evaluate the impact on trial operating characteristics of incorporating historical individual control data and aggregate treatment effect estimates. We used power priors derived from historical individual control data for baseline parameters of Weibull and piecewise exponential models, while we used a mixture prior to summarise aggregate information obtained on the relative treatment effect. The impact of prior-data conflicts, both with respect to the parameters and survival models, was evaluated for a set of pre-specified weights assigned to the historical information in the prior distributions.

Results

The operating characteristics varied according to the weights assigned to each source of historical information, the variance of the informative and vague component of the mixture prior and the level of commensurability between the historical and new data. When historical and new controls follow different survival distributions, we did not observe any advantage of choosing a piecewise exponential model compared to a Weibull model for the new trial analysis. However, we think that it remains appealing given the uncertainty that will often surround the shape of the survival distribution of the new data.

Conclusion

In the setting of Sarcome-13 trial, and other similar studies in rare diseases, the gains in power and accuracy made possible by incorporating different types of historical information commensurate with the new trial data have to be balanced against the risk of biased estimates and a possible loss in power if data are not commensurate. The weights allocated to the historical data have to be carefully chosen based on this trade-off. Further simulation studies investigating methods for incorporating historical data are required to generalise the findings.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0714-z) contains supplementary material, which is available to authorized users.

The assessment of intrinsic credibility and a new argument for p < 0.005

The concept of intrinsic credibility has been recently introduced to check the credibility of 'out of the blue' findings without any prior support. A significant result is deemed intrinsically credible if it is in conflict with a sceptical prior derived from the very same data that would make the effect just non-significant. In this paper, I propose to use Bayesian prior-predictive tail probabilities to assess intrinsic credibility. For the standard 5% significance level, this leads to a new p-value threshold that is remarkably close to the recently proposed p < 0.005 standard. I also introduce the credibility ratio, the ratio of the upper to the lower limit (or vice versa) of a confidence interval for a significant effect size. I show that the credibility ratio has to be smaller than 5.8 such that a significant finding is also intrinsically credible. Finally, a p-value for intrinsic credibility is proposed that is a simple function of the ordinary p-value and has a direct frequentist interpretation in terms of the probability of replicating an effect. An application to data from the Open Science Collaboration study on the reproducibility of psychological science suggests that intrinsic credibility of the original experiment is better suited to predict the success of a replication experiment than standard significance.

Monitoring futility and efficacy in phase II trials with Bayesian posterior distributions-A calibration approach

A multistage single arm phase II trial with binary endpoint is considered. Bayesian posterior probabilities are used to monitor futility in interim analyses and efficacy in the final analysis. For a beta-binomial model, decision rules based on Bayesian posterior probabilities are converted to "traditional" decision rules in terms of number of responders among patients observed so far. Analytical derivations are given for the probability of stopping for futility and for the probability to declare efficacy. A workflow is presented on how to select the parameters specifying the Bayesian design, and the operating characteristics of the design are investigated. It is outlined how the presented approach can be transferred to statistical models other than the beta-binomial model.

Bayesian sample size re-estimation using power priors

The sample size of a randomized controlled trial is typically chosen in order for frequentist operational characteristics to be retained. For normally distributed outcomes, an assumption for the variance needs to be made which is usually based on limited prior information. Especially in the case of small populations, the prior information might consist of only one small pilot study. A Bayesian approach formalizes the aggregation of prior information on the variance with newly collected data. The uncertainty surrounding prior estimates can be appropriately modelled by means of prior distributions. Furthermore, within the Bayesian paradigm, quantities such as the probability of a conclusive trial are directly calculated. However, if the postulated prior is not in accordance with the true variance, such calculations are not trustworthy. In this work we adapt previously suggested methodology to facilitate sample size re-estimation. In addition, we suggest the employment of power priors in order for operational characteristics to be controlled.

Dynamic borrowing through empirical power priors that control type I error

In order for historical data to be considered for inclusion in the design and analysis of clinical trials, prospective rules are essential. Incorporation of historical data may be of particular interest in the case of small populations where available data is scarce and heterogeneity is not as well understood, and thus conventional methods for evidence synthesis might fall short. The concept of power priors can be particularly useful for borrowing evidence from a single historical study. Power priors employ a parameter <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>γ</mml:mi> <mml:mo>∈</mml:mo> <mml:mo>[</mml:mo> <mml:mn>0</mml:mn> <mml:mo>,</mml:mo> <mml:mn>1</mml:mn> <mml:mo>]</mml:mo></mml:math> that quantifies the heterogeneity between the historical study and the new study. However, the possibility of borrowing data from a historical trial will usually be associated with an inflation of the type I error. We suggest a new, simple method of estimating the power parameter suitable for the case when only one historical dataset is available. The method is based on predictive distributions and parameterized in such a way that the type I error can be controlled by calibrating to the degree of similarity between the new and historical data. The method is demonstrated for normal responses in a one or two group setting. Generalization to other models is straightforward.