Robust Bayesian methods for monitoring clinical trials

Before/after Bayes: A comparison of frequentist and Bayesian mixed-effects models in applied psychological research

Bayesian methods are becoming increasingly used in applied psychological research. Previous researchers have thoroughly written about much of the details already, including the philosophy underlying Bayesian methods, computational issues associated with Bayesian model estimation, Bayesian model development and summary, and the role of Bayesian methods in the so-called replication crisis. In this paper, we seek to provide case studies comparing the use of frequentist methods to the use of Bayesian methods in applied psychological research. These case studies are intended to 'illustrate by example' the ways that Bayesian modelling differs from frequentist modelling and the differing conclusions that one may arrive at using the two methods. The intended audience is applied psychological researchers who have been trained in the traditional frequentist framework, who are familiar with mixed-effects models and who are curious about how statistical results might look in a Bayesian context. Along with our case studies, we provide general opinions and guidance on the use of Bayesian methods in applied psychological research.

Bayesian Design of Superiority Trials: Methods and Applications

In this paper, we lay out the basic elements of Bayesian sample size determination (SSD) for the Bayesian design of a two-arm superiority clinical trial. We develop a flowchart of the Bayesian SSD that highlights the critical components of a Bayesian design and provides a practically useful roadmap for designing a Bayesian clinical trial in real world applications. We empirically examine the amount of borrowing, the choice of noninformative priors, and the impact of model misspecification on the Bayesian type I error and power. A formal and statistically rigorous formulation of conditional borrowing within the decision rule framework is developed. Moreover, by extending the partial borrowing power priors, a new borrowing-by-parts power prior for incorporating historical data is proposed. Computational algorithms are also developed to calculate the Bayesian type I error and power. Extensive simulation studies are carried out to explore the operating characteristics of the proposed Bayesian design of a superiority trial.

Prior Elicitation for Use in Clinical Trial Design and Analysis: A Literature Review

Bayesian inference is increasingly popular in clinical trial design and analysis. The subjective knowledge derived from an expert elicitation procedure may be useful to define a prior probability distribution when no or limited data is available. This work aims to investigate the state-of-the-art Bayesian prior elicitation methods with a focus on clinical trial research. A literature search on the Current Index to Statistics (CIS), PubMed, and Web of Science (WOS) databases, considering “prior elicitation” as a search string, was run on 1 November 2020. Summary statistics and trend of publications over time were reported. Finally, a Latent Dirichlet Allocation (LDA) model was developed to recognise latent topics in the pertinent papers retrieved. A total of 460 documents pertinent to the Bayesian prior elicitation were identified. Of these, 213 (45.4%) were published in the “Probability and Statistics” area. A total of 42 articles pertain to clinical trial and the majority of them (81%) reports parametric techniques as elicitation method. The last decade has seen an increased interest in prior elicitation and the gap between theory and application getting narrower and narrower. Given the promising flexibility of non-parametric approaches to the experts’ elicitation, more efforts are needed to ensure their diffusion also in applied settings.

A Bayesian approach in design and analysis of pediatric cancer clinical trials

It is well recognized that cancer drug development for children and adolescents has many challenges, from biological and societal to economic. Pediatric cancer consists of a diverse group of rare diseases, and the relatively small population of children with multiple, disparate tumor types across various age groups presents a significant challenge for drug development programs as compared to oncology drug development programs for adults. Due to the different types of cancers, limited opportunities exist for extrapolation of efficacy from adult cancer indications to children. Thus, innovative study designs including Bayesian statistical approaches should be considered. A Bayesian approach can be a flexible tool to formally leverage prior knowledge of adult or external controls in pediatric cancer trials. In this article, we provide in a case example of how Bayesian approaches can be used to design, monitor, and analyze pediatric trials. Particularly, Bayesian sequential monitoring can be useful to monitor pediatric trial results as data accumulate. In addition, designing a pediatric trial with both skeptical and enthusiastic priors with Bayesian sequential monitoring can be an efficient mechanism for early trial cessation for both efficacy and futility. The interpretation of efficacy using a Bayesian approach is based on posterior probability and is intuitive and interpretable for patients, parents and prescribers given limited data.

Using prior distributions to synthesize historical evidence: comments on the Goodman–Sladky case study of IVIg in Guillain–Barré syndrome

One feature of the Bayesian approach is that it provides methods for synthesizing what is known about a question of interest and provides a formalism based on the laws of probability for incorporating this auxiliary knowledge into the planning and the analysis of the next study. In this comment, we use elements of the Goodman-Sladky case study to illustrate (1) the use of Bayesian methods to quantify historical information about an intervention through the specification of a prior distribution, (2) an approach to the analysis of the sensitivity of the conclusions of a Bayesian analysis to the specification of the prior distribution, and (3) we comment on the role of research synthesis for combining information about an intervention from different data sources as a tool to help summarize evidence about the intervention.

Twin-to-Twin Transfusion Syndrome: From Observational Evidence to Randomized Controlled Trials

Fetoscopic surgery is widely accepted as the preferred first-line treatment for twin-twin transfusion syndrome (TTTS). Nonetheless, the broad diffusion of this technique relies on a single multicentric-randomized trial. We hereby question this trial in a post-hoc Bayesian analysis, submitting its results to several scenarios comprising the alternative published non-randomized literature and pessimistic opinions regarding this surgery. Furthermore, we also discuss further refinements in indications, questioning potential alternatives in early stages of the disease.

A two-stage Bayesian design for co-development of new drugs and companion diagnostics

Most new drug development in oncology is based on targeting specific molecules. Genomic profiles and deregulated drug targets vary from patient to patient making new treatments likely to benefit only a subset of patients traditionally grouped in the same clinical trials. Predictive biomarkers are being developed to identify patients who are most likely to benefit from a particular treatment; however, their biological basis is not always conclusive. The inclusion of marker-negative patients in a trial is therefore sometimes necessary for a more informative evaluation of the therapy. In this paper, we present a two-stage Bayesian design that includes both marker-positive and marker-negative patients in a clinical trial. We formulate a family of prior distributions that represent the degree of a priori confidence in the predictive biomarker. To avoid exposing patients to a treatment to which they may not be expected to benefit, we perform an interim analysis that may stop accrual of marker-negative patients or accrual of all patients. We demonstrate with simulations that the design and priors used control type I errors, give adequate power, and enable the early futility analysis of test-negative patients to be based on prior specification on the strength of evidence in the biomarker.

Summarizing historical information on controls in clinical trials

BACKGROUND

Historical information is always relevant when designing clinical trials, but it might also be incorporated in the analysis. It seems appropriate to exploit past information on comparable control groups.

PURPOSE

Phase IV and proof-of-concept trials are used to discuss aspects of summarizing historical control data as prior information in a new trial. The importance of a fair assessment of the similarity of control parameters is emphasized.

METHODS

The methodology is meta-analytic-predictive. Heterogeneity of control parameters is expressed via the between-trial variation, which is the key parameter determining the prior effective sample size and its upper bound (prior maximum sample size).

RESULTS

For a Phase IV trial (930 control patients in 11 historical trials) between-trial heterogeneity was fairly small, resulting in a prior effective sample size of approximately 90 patients. For a proof-of-concept trial (363 patients in four historical trials) heterogeneity was moderate to substantial, resulting in a prior effective sample size of approximately 20. For another proof-of-concept trial (14 patients in one historical trial), assuming substantial heterogeneity implied a prior effective sample size of 7. The prior effective sample size can only be large if the amount of historical data is large and between-trial heterogeneity is small. The prior effective sample size is bounded by the prior maximum sample size (ratio of within- to between-trial variance), irrespective of the amount of historical data.

LIMITATIONS

The meta-analytic-predictive approach assumes exchangeability of control parameters across trials. Due to the difficulty to quantify between-trial variability, sensitivity of conclusions regarding assumptions and type of inference should be assessed.

CONCLUSIONS

The use of historical control information is a valuable option and may lead to more efficient clinical trials. The proposed approach is attractive for nonconfirmatory trials, but under certain circumstances extensions to the confirmatory setting could be envisaged as well.

Determining the effective sample size of a parametric prior

We present a definition for the effective sample size of a parametric prior distribution in a Bayesian model, and propose methods for computing the effective sample size in a variety of settings. Our approach first constructs a prior chosen to be vague in a suitable sense, and updates this prior to obtain a sequence of posteriors corresponding to each of a range of sample sizes. We then compute a distance between each posterior and the parametric prior, defined in terms of the curvature of the logarithm of each distribution, and the posterior minimizing the distance defines the effective sample size of the prior. For cases where the distance cannot be computed analytically, we provide a numerical approximation based on Monte Carlo simulation. We provide general guidelines for application, illustrate the method in several standard cases where the answer seems obvious, and then apply it to some nonstandard settings.

Bayesian statistics in medicine: a 25 year review

This review examines the state of Bayesian thinking as Statistics in Medicine was launched in 1982, reflecting particularly on its applicability and uses in medical research. It then looks at each subsequent five-year epoch, with a focus on papers appearing in Statistics in Medicine, putting these in the context of major developments in Bayesian thinking and computation with reference to important books, landmark meetings and seminal papers. It charts the growth of Bayesian statistics as it is applied to medicine and makes predictions for the future. From sparse beginnings, where Bayesian statistics was barely mentioned, Bayesian statistics has now permeated all the major areas of medical statistics, including clinical trials, epidemiology, meta-analyses and evidence synthesis, spatial modelling, longitudinal modelling, survival modelling, molecular genetics and decision-making in respect of new technologies.

Prior distributions for the intracluster correlation coefficient, based on multiple previous estimates, and their application in cluster randomized trials

Numerous estimates for the intracluster correlation coefficient (ICC) are available in research databases and publications. When planning a cluster randomized trial, an anticipated value for the ICC is required; currently, researchers base their choice informally on the magnitude of previous ICC estimates. In this paper, we make use of the wealth of ICC information by formally constructing informative prior distributions, while acknowledging the varying relevance and precision of the estimates available. Typically, for a planned trial in a given clinical setting, multiple relevant ICC estimates are available from each of several completed studies. Our preferred model allows for the imprecision in each ICC estimate around its underlying true value and, separately, allows for the similarity of ICC values from the same study. The relevance of each previous estimate to the planned clinical setting is considered, and estimates corresponding to less relevant outcomes or population types are given less influence. We find that such downweighting can increase the precision of the anticipated ICC. In trial design, the prior distribution constructed allows uncertainty about the ICC to be acknowledged, and we describe how to choose a design that provides adequate power across the range of likely ICC values. Prior information on the ICC can also be incorporated in analysis of the trial data, when taking a Bayesian approach. The methods proposed enable available ICC information to be summarised appropriately by an informative prior distribution, which is of direct practical use in cluster randomized trials.

Application of an adaptive design to a randomized phase II selection trial in gastric cancer: a report of the study design

Randomized phase II selection trials seek to provide unbiased comparisons for the selection of the most promising treatment arm for evaluation in a future phase III trial. In this paper, we present an application of an adaptive design to a randomized phase II selection trial comparing three experimental treatments with a control arm in patients with advanced gastric cancer. The trial design continuously monitors multiple patient outcomes to protect future patients from treatments with unacceptably high toxicity and/or unacceptably low efficacy. We use a Bayesian approach to monitor the trial and carry out simulations to investigate operating characteristics of the trial design. The simulation study also evaluates the sensitivity of the design to the prior distribution by considering two alternative priors.

Use of a Bayesian approach to decide when to stop a therapeutic trial: the case of a chemoprophylaxis trial in human immunodeficiency virus infection

From 1996 to 1998, a phase III, placebo-controlled, therapeutic trial was conducted in Abidjan, Ivory Coast, to assess the efficacy of cotrimoxazole prophylaxis in reducing severe morbidity in adults at early stages of human immunodeficiency virus infection. The authors used the real data from this trial to simulate three Bayesian interim analyses. Three prior distributions were considered: a noninformative one, a skeptical one, and one based on external information. The posterior distribution was calculated by using directed acyclic graphs and Gibbs sampling. This Bayesian approach showed different results according to the prior distribution chosen. Although use of the noninformative prior would have led to stopping the trial at the same time that the frequentist approach would have, the skeptical prior would have led to continuing it, and the prior based on external data would have led to stopping it 1 year earlier.

Prior convictions

Large, randomized clinical trials ("megatrials") are key drivers of modern cardiovascular practice, since they are cited frequently as the authoritative foundation for evidence-based management policies. Nevertheless, fundamental limitations in the conventional approach to statistical hypothesis testing undermine the scientific basis of the conclusions drawn from these trials. This review describes the conventional approach to statistical inference, highlights its limitations, and proposes an alternative approach based on Bayes' theorem. Despite its inherent subjectivity, the Bayesian approach possesses a number of practical advantages over the conventional approach: 1). it allows the explicit integration of previous knowledge with new empirical data; 2). it avoids the inevitable misinterpretations of p values derived from megatrial populations; and 3). it replaces the misleading p value with a summary statistic having a natural, clinically relevant interpretation-the probability that the study hypothesis is true given the observations. This posterior probability thereby quantifies the likelihood of various magnitudes of therapeutic benefit rather than the single null magnitude to which the p value refers, and it lends itself to graphical sensitivity analyses with respect to its underlying assumptions. Accordingly, the Bayesian approach should be employed more widely in the design, analysis, and interpretation of clinical megatrials.

Regulatory perspectives on data monitoring

Data monitoring is a critical component of the conduct of clinical trials that provide the evidence of efficacy and safety of investigational drugs. These trials may be conducted either by a pharmaceutical sponsor or by the government, especially those large trials that assess the impact of therapies on serious morbidity and/or mortality. While not extensive, I will review a regulatory history of FDA's evolving concerns and positions on data monitoring. I will review the key aspects of data monitoring and interim analysis of clinical trials contained in the recently published International Conference on Harmonization's statistical guidance as well as some other issues being considered for a draft guidance on data monitoring. Finally, some suggestions for improving and enhancing tools and statistical methods for monitoring clinical trials for safety assessment will be offered. This latter area deserves more consideration by statisticians than it has received to date.

Using elicitation techniques to estimate the value of ambulatory treatments for major depression

Estimating the value of spending on medical treatments in a health care system involves relating output, measured in terms of effectiveness, to cost, measured in terms of spending. Although information on spending at the system level often exists in administrative data, such as insurance claims, information on effectiveness is not always available. An inferential tool available to researchers in this context is elicitation. The authors develop an approach to elicit effectiveness parameters and apply it to a panel of 10 experts to estimate predictive Hamilton Depression Rating Scale scores representing postambulatory treatment outcomes. The elicited parameters are used to estimate outcomes associated with 120 acute phase treatments for major depression within a privately insured health insurance system. The outcome-adjusted price per full remission episode is estimated for each acute treatment, and corresponding 95% percentile bootstrap intervals are calculated. The average spending for all observed treatments was $473 (SE = 478), with a depression-free adjusted price per case of $5,995 (95% confidence interval = $5,959-$6,031).

Quantifying and documenting prior beliefs in clinical trials

Collecting and documenting subjective prior beliefs from knowledgeable clinicians about the potential results of a clinical trial has many advantages. Two large trials of prophylactic treatments in an HIV-positive population are used as examples. The trials recruited patients of primary care physicians and compared treatments which were in use in clinical practice. Opinions about these trials were elicited from 58 practising HIV clinicians. It is shown how the documented opinions can be used to augment the monitoring process; the prior opinions are updated with interim data using approximate Bayesian methods to give posterior opinions incorporating interim results. These posterior opinions can be used by the monitoring board to anticipate the clinicians' reaction to the results. Eliciting prior beliefs is also ethically important for documenting the nature of the uncertainty or equipoise. Important information is provided for the informed consent process and Institutional Review Board (IRB).

Bayesian statistics in medical research: an intuitive alternative to conventional data analysis

Statistical analysis of both experimental and observational data is central to medical research. Unfortunately, the process of conventional statistical analysis is poorly understood by many medical scientists. This is due, in part, to the counter-intuitive nature of the basic tools of traditional (frequency-based) statistical inference. For example, the proper definition of a conventional 95% confidence interval is quite confusing. It is based upon the imaginary results of a series of hypothetical repetitions of the data generation process and subsequent analysis. Not surprisingly, this formal definition is often ignored and a 95% confidence interval is widely taken to represent a range of values that is associated with a 95% probability of containing the true value of the parameter being estimated. Working within the traditional framework of frequency-based statistics, this interpretation is fundamentally incorrect. It is perfectly valid, however, if one works within the framework of Bayesian statistics and assumes a 'prior distribution' that is uniform on the scale of the main outcome variable. This reflects a limited equivalence between conventional and Bayesian statistics that can be used to facilitate a simple Bayesian interpretation based on the results of a standard analysis. Such inferences provide direct and understandable answers to many important types of question in medical research. For example, they can be used to assist decision making based upon studies with unavoidably low statistical power, where non-significant results are all too often, and wrongly, interpreted as implying 'no effect'. They can also be used to overcome the confusion that can result when statistically significant effects are too small to be clinically relevant. This paper describes the theoretical basis of the Bayesian-based approach and illustrates its application with a practical example that investigates the prevalence of major cardiac defects in a cohort of children born using the assisted reproduction technique known as ICSI (intracytoplasmic sperm injection).

Robustness considerations in Bayesian analysis

All statistical analyses demand uncertain inputs or assumptions. This is especially true of Bayesian analyses. In addition to the usual concerns about the agreement of the data and model, a Bayesian must contemplate the effect of an uncertain prior specification. The degree to which inferences are robust to changes in the prior is of primary interest. This article discusses some robust techniques that have been suggested in the literature. One goal is to make apparent the relevance of some of these techniques to biostatistical work.