A decision theoretic approach to sample size determination in clinical trials

Bayesian statistics in the design and analysis of cluster randomised controlled trials and their reporting quality: a methodological systematic review

BACKGROUND

In a cluster randomised controlled trial (CRCT), randomisation units are "clusters" such as schools or GP practices. This has methodological implications for study design and statistical analysis, since clustering often leads to correlation between observations which, if not accounted for, can lead to spurious conclusions of efficacy/effectiveness. Bayesian methodology offers a flexible, intuitive framework to deal with such issues, but its use within CRCT design and analysis appears limited. This review aims to explore and quantify the use of Bayesian methodology in the design and analysis of CRCTs, and appraise the quality of reporting against CONSORT guidelines.

METHODS

We sought to identify all reported/published CRCTs that incorporated Bayesian methodology and papers reporting development of new Bayesian methodology in this context, without restriction on publication date or location. We searched Medline and Embase and the Cochrane Central Register of Controlled Trials (CENTRAL). Reporting quality metrics according to the CONSORT extension for CRCTs were collected, as well as demographic data, type and nature of Bayesian methodology used, journal endorsement of CONSORT guidelines, and statistician involvement.

RESULTS

Twenty-seven publications were included, six from an additional hand search. Eleven (40.7%) were reports of CRCT results: seven (25.9%) were primary results papers and four (14.8%) reported secondary results. Thirteen papers (48.1%) reported Bayesian methodological developments, the remaining three (11.1%) compared different methods. Four (57.1%) of the primary results papers described the method of sample size calculation; none clearly accounted for clustering. Six (85.7%) clearly accounted for clustering in the analysis. All results papers reported use of Bayesian methods in the analysis but none in the design or sample size calculation.

CONCLUSIONS

The popularity of the CRCT design has increased rapidly in the last twenty years but this has not been mirrored by an uptake of Bayesian methodology in this context. Of studies using Bayesian methodology, there were some differences in reporting quality compared to CRCTs in general, but this study provided insufficient data to draw firm conclusions. There is an opportunity to further develop Bayesian methodology for the design and analysis of CRCTs in order to expand the accessibility, availability, and, ultimately, use of this approach.

Bayesian sample size determination for a Phase III clinical trial with diluted treatment effect

When Phase III treatment effect is diluted from what was observed from Phase II results, we propose to determine the Bayesian sample size for a Phase III clinical trial based on the normal, uniform, and truncated normal prior distributions of the treatment effects on an interval, which starts from an acceptable treatment effect to the observed treatment effect from Phase II. After incorporating the prior information of the treatment effects, the Bayesian sample size is the number of patients of the Phase III trial for a given Bayesian Predictive Power (BPP) or Bayesian Historical Predictive Power (BHPP). After that, the numerical simulations are carried out to determine the Bayesian sample size for the Phase III clinical trial. In particular, there exists a hook phenomenon for the BHPP when the number of patients of the Phase II trial equals 70 assuming the normal, uniform, or truncated normal treatment effect. Moreover, we add some sensitivity analysis of the Bayesian sample size about the parameters in the simulations. Finally, we determine the Bayesian sample size (number of events or deaths) of the Phase III trial for a fixed power, Bayesian Historical Power (BHP), and BHPP in the axitinib example.

Bayesian approach to determine the number of subsequent users of a new treatment

The aim of this article is to discuss the distribution function of the number of subsequent users of a new treatment. A Bayesian approach is applied. Using the fact that the number of subsequent users of the new treatment will not be high, unless it is, in the statistical and also in the clinical sense, significantly better than the existing one, we obtain the distribution function of the number of subsequent users of a new treatment for which we assume the data have come from a normal distribution.

Decision-theoretic designs for small trials and pilot studies: A review

Pilot studies and other small clinical trials are often conducted but serve a variety of purposes and there is little consensus on their design. One paradigm that has been suggested for the design of such studies is Bayesian decision theory. In this article, we review the literature with the aim of summarizing current methodological developments in this area. We find that decision-theoretic methods have been applied to the design of small clinical trials in a number of areas. We divide our discussion of published methods into those for trials conducted in a single stage, those for multi-stage trials in which decisions are made through the course of the trial at a number of interim analyses, and those that attempt to design a series of clinical trials or a drug development programme. In all three cases, a number of methods have been proposed, depending on the decision maker’s perspective being considered and the details of utility functions that are used to construct the optimal design.

Methods for specifying the target difference in a randomised controlled trial: the Difference ELicitation in TriAls (DELTA) systematic review

BACKGROUND

Randomised controlled trials (RCTs) are widely accepted as the preferred study design for evaluating healthcare interventions. When the sample size is determined, a (target) difference is typically specified that the RCT is designed to detect. This provides reassurance that the study will be informative, i.e., should such a difference exist, it is likely to be detected with the required statistical precision. The aim of this review was to identify potential methods for specifying the target difference in an RCT sample size calculation.

METHODS AND FINDINGS

A comprehensive systematic review of medical and non-medical literature was carried out for methods that could be used to specify the target difference for an RCT sample size calculation. The databases searched were MEDLINE, MEDLINE In-Process, EMBASE, the Cochrane Central Register of Controlled Trials, the Cochrane Methodology Register, PsycINFO, Science Citation Index, EconLit, the Education Resources Information Center (ERIC), and Scopus (for in-press publications); the search period was from 1966 or the earliest date covered, to between November 2010 and January 2011. Additionally, textbooks addressing the methodology of clinical trials and International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) tripartite guidelines for clinical trials were also consulted. A narrative synthesis of methods was produced. Studies that described a method that could be used for specifying an important and/or realistic difference were included. The search identified 11,485 potentially relevant articles from the databases searched. Of these, 1,434 were selected for full-text assessment, and a further nine were identified from other sources. Fifteen clinical trial textbooks and the ICH tripartite guidelines were also reviewed. In total, 777 studies were included, and within them, seven methods were identified-anchor, distribution, health economic, opinion-seeking, pilot study, review of the evidence base, and standardised effect size.

CONCLUSIONS

A variety of methods are available that researchers can use for specifying the target difference in an RCT sample size calculation. Appropriate methods may vary depending on the aim (e.g., specifying an important difference versus a realistic difference), context (e.g., research question and availability of data), and underlying framework adopted (e.g., Bayesian versus conventional statistical approach). Guidance on the use of each method is given. No single method provides a perfect solution for all contexts.

Value of information methods for assessing a new diagnostic test

Value-of-information methods are applied to assess the evidence in support of a new diagnostic test and, where the evidence is insufficient for decision making, to determine the optimal sample size for future studies. Net benefit formulations are derived under various diagnostic and treatment scenarios. The expressions for the expected opportunity loss of adopting strategies that include the new test are given. Expressions for the expected value of information from future studies are derived. One-sample and two-sample designs, with or without known prevalence, are considered. An example is given.

Determining optimal sample sizes for multistage adaptive randomized clinical trials from an industry perspective using value of information methods

Background

 Most often, sample size determinations for randomized clinical trials are based on frequentist approaches that depend on somewhat arbitrarily chosen factors, such as type I and II error probabilities and the smallest clinically important difference. As an alternative, many authors have proposed decision-theoretic (full Bayesian) approaches, often referred to as value of information methods that attempt to determine the sample size that maximizes the difference between the trial’s expected utility and its expected cost, referred to as the expected net gain. Taking an industry perspective, Willan proposes a solution in which the trial’s utility is the increase in expected profit. Furthermore, Willan and Kowgier, taking a societal perspective, show that multistage designs can increase expected net gain.

Purpose

 The purpose of this article is to determine the optimal sample size using value of information methods for industry-based, multistage adaptive randomized clinical trials, and to demonstrate the increase in expected net gain realized. At the end of each stage, the trial’s sponsor must decide between three actions: continue to the next stage, stop the trial and seek regulatory approval, or stop the trial and abandon the drug.

Methods

 A model for expected total profit is proposed that includes consideration of per-patient profit, disease incidence, time horizon, trial duration, market share, and the relationship between trial results and probability of regulatory approval. The proposed method is extended to include multistage designs with a solution provided for a two-stage design. An example is given.

Results

 Significant increases in the expected net gain are realized by using multistage designs.

Limitations

 The complexity of the solutions increases with the number of stages, although far simpler near-optimal solutions exist. The method relies on the central limit theorem, assuming that the sample size is sufficiently large so that the relevant statistics are normally distributed.

Conclusion

 From a value of information perspective, the use of multistage designs in industry trials leads to significant gains in the expected net gain.

Sample size determination for cost-effectiveness trials

Methods for determining sample size requirements for cost-effectiveness studies are reviewed and illustrated. Traditional methods based on tests of hypothesis and power arguments are given for the incremental cost-effectiveness ratio and incremental net benefit (INB). In addition, a full Bayesian approach using decision theory to determine optimal sample size is given for INB. The full Bayesian approach, based on the value of information, is proposed in reaction to concerns that traditional methods rely on arbitrarily chosen error probabilities and an ill-defined notion of the smallest clinically important difference. Furthermore, the results of cost-effectiveness studies are used for decision making (e.g. should a new intervention be adopted or the old one retained), and employing decision theory, which permits optimal use of current information and the optimal design of new studies, provides a more consistent approach.

Bayesian sample size calculation for estimation of the difference between two binomial proportions

In this study, we discuss a decision theoretic or fully Bayesian approach to the sample size question in clinical trials with binary responses. Data are assumed to come from two binomial distributions. A Dirichlet distribution is assumed to describe prior knowledge of the two success probabilities p1 and p2. The parameter of interest is p = p1 - p2. The optimal size of the trial is obtained by maximising the expected net benefit function. The methodology presented in this article extends previous work by the assumption of dependent prior distributions for p1 and p2.

Statisticians and pharmacokineticists: what they can still learn from each other

Examples are given of how the practice of statistics could be improved if statisticians showed a greater awareness of pharmacokinetic and pharmacodynamic modeling. Some examples are also given where a wider appreciation of statistical theory would improve current approaches to pharmacometrics. Areas in which the two disciplines are in agreement but have failed to have as much influence on others in drug development as they ought are also considered. It is concluded that there would be much benefit in increasing collaboration between these disciplines.

A behavioural Bayes approach to the determination of sample size for clinical trials considering efficacy and safety: imbalanced sample size in treatment groups

The behavioural Bayes approach to sample size determination for clinical trials assumes that the number of subsequent patients switching to a new drug from the current drug depends on the strength of the evidence for efficacy and safety that was observed in the clinical trials. The optimal sample size is the one which maximises the expected net benefit of the trial. The approach has been developed in a series of papers by Pezeshk and the present authors (Gittins JC, Pezeshk H. A behavioral Bayes method for determining the size of a clinical trial. Drug Information Journal 2000; 34: 355-63; Gittins JC, Pezeshk H. How Large should a clinical trial be? The Statistician 2000; 49(2): 177-87; Gittins JC, Pezeshk H. A decision theoretic approach to sample size determination in clinical trials. Journal of Biopharmaceutical Statistics 2002; 12(4): 535-51; Gittins JC, Pezeshk H. A fully Bayesian approach to calculating sample sizes for clinical trials with binary responses. Drug Information Journal 2002; 36: 143-50; Kikuchi T, Pezeshk H, Gittins J. A Bayesian cost-benefit approach to the determination of sample size in clinical trials. Statistics in Medicine 2008; 27(1): 68-82; Kikuchi T, Gittins J. A behavioral Bayes method to determine the sample size of a clinical trial considering efficacy and safety. Statistics in Medicine 2009; 28(18): 2293-306; Kikuchi T, Gittins J. A Bayesian procedure for cost-benefit evaluation of a new drug in multi-national clinical trials. Statistics in Medicine 2009 (Submitted)). The purpose of this article is to provide a rationale for experimental designs which allocate more patients to the new treatment than to the control group. The model uses a logistic weight function, including an interaction term linking efficacy and safety, which determines the number of patients choosing the new drug, and hence the resulting benefit. A Monte Carlo simulation is employed for the calculation. Having a larger group of patients on the new drug in general makes it easier to recruit patients to the trial and may also be ethically desirable. Our results show that this can be done with very little if any reduction in expected net benefit.

A behavioral Bayes method to determine the sample size of a clinical trial considering efficacy and safety

It is necessary for the calculation of sample size to achieve the best balance between the cost of a clinical trial and the possible benefits from a new treatment. Gittins and Pezeshk developed an innovative (behavioral Bayes) approach, which assumes that the number of users is an increasing function of the difference in performance between the new treatment and the standard treatment. The better a new treatment, the more the number of patients who want to switch to it. The optimal sample size is calculated in this framework. This BeBay approach takes account of three decision-makers, a pharmaceutical company, the health authority and medical advisers. Kikuchi, Pezeshk and Gittins generalized this approach by introducing a logistic benefit function, and by extending to the more usual unpaired case, and with unknown variance. The expected net benefit in this model is based on the efficacy of the new drug but does not take account of the incidence of adverse reactions. The present paper extends the model to include the costs of treating adverse reactions and focuses on societal cost-effectiveness as the criterion for determining sample size. The main application is likely to be to phase III clinical trials, for which the primary outcome is to compare the costs and benefits of a new drug with a standard drug in relation to national health-care.

Screening designs for drug development

We propose drug screening designs based on a Bayesian decision-theoretic approach. The discussion is motivated by screening designs for phase II studies. The proposed screening designs allow consideration of multiple treatments simultaneously. In each period, new treatments can arise and currently considered treatments can be dropped. Once a treatment is removed from the phase II screening trial, a terminal decision is made about abandoning the treatment or recommending it for a future confirmatory phase III study. The decision about dropping treatments from the active set is a sequential stopping decision. We propose a solution based on decision boundaries in the space of marginal posterior moments for the unknown parameter of interest that relates to each treatment. We present a Monte Carlo simulation algorithm to implement the proposed approach. We provide an implementation of the proposed method as an easy to use R library available for public domain download (http://www.stat.rice.edu/~rusi/ or http://odin.mdacc.tmc.edu/~pm/).

Bayesian techniques for sample size determination in clinical trials: a short review

The aim of this paper is to review some key techniques of Bayesian methods of sample size determination. The approach is to cover a small number of simple problems, such as estimating the mean of a normal distribution. The methods considered are in two groups: inferential and decision theoretic. In the inferential Bayesian methods of sample size determination, we are solely concerned with the inference about the parameter(s) of interest. The fully Bayesian or decision theoretic approach treats the problem as a decision problem and employs a loss or utility function.