Detecting treatment-by-centre interaction in multi-centre clinical trials

Homogeneity assessment for pivotal medical device clinical studies

A question that routinely arises in medical device clinical studies is the homogeneity across demographic subgroups, geographical regions, or investigational sites of the enrolled patients in terms of treatment effects or outcome variables. The main objective of this paper is to discuss statistical concepts and methods for the assessment of such homogeneity and to provide the practitioner a statistical framework and points to consider in conducting homogeneity assessment. Demographic subgroups, geographical regions, and investigational sites are discussed separately as each has its unique issues. Specific considerations are also given to randomized controlled trials, non-randomized comparative studies, and single-arm studies. We point out that judicious use of statistical methods, in conjunction with sound clinical judgment, is essential in handling the issue of homogeneity of treatment effect in medical device clinical studies.

Studying treatment-effect heterogeneity in precision medicine through induced subgroups

Precision medicine, in the sense of tailoring the choice of medical treatment to patients' pretreatment characteristics, is nowadays gaining a lot of attention. Preferably, this tailoring should be realized in an evidence-based way, with key evidence in this regard pertaining to subgroups of patients that respond differentially to treatment (i.e., to subgroups involved in treatment-subgroup interactions). Often a-priori hypotheses on subgroups involved in treatment-subgroup interactions are lacking or are incomplete at best. Therefore, methods are needed that can induce such subgroups from empirical data on treatment effectiveness in a post hoc manner. Recently, quite a few such methods have been developed. So far, however, there is little empirical experience in their usage. This may be problematic for medical statisticians and statistically minded medical researchers, as many (nontrivial) choices have to be made during the data-analytic process. The main purpose of this paper is to discuss the major concepts and considerations when using these methods. This discussion will be based on a systematic, conceptual, and technical analysis of the type of research questions at play, and of the type of data that the methods can handle along with the available software, and a review of available empirical evidence. We will illustrate all this with the analysis of a dataset comparing several anti-depressant treatments.

Investigating covariate-by-centre interaction in survival data

In survival analysis, assessing the existence of potential centre effects on the baseline hazard or on the effect of fixed covariates on the baseline hazard, such as treatment-by-centre interaction, is a frequent clinical concern in multicentre studies. Survival models with random effects on the baseline hazard and/or on the effect of the covariates of interest have been largely applied, for instance, to investigate potential centre effects. We aimed to develop a procedure to routinely test for multiple random effects in survival analyses. We propose a statistic and a permutation approach to test whether all or a subset of components of the variance-covariance matrix of random effects are non-zero in a mixed-effects Cox model framework. Performances of the proposed permutation tests are examined under different null hypotheses corresponding to the different components of the variance-covariance matrix, i.e ., to the different random effects considered on the baseline hazard and/or on the covariates effects. Several alternative hypotheses are evaluated using simulations. The results indicate that the permutation tests have valid type I error rates under the null and achieve satisfactory power under all alternatives. The procedure is applied to two European cohorts of haematological stem cell transplants in acute leukaemia to investigate the heterogeneity across centres in leukaemia-free survival and the potential heterogeneity in prognostic factors effects across centres.

Subpopulation Treatment Effect Pattern Plot (STEPP) analysis for continuous, binary, and count outcomes

BACKGROUND

For the past few decades, randomized clinical trials have provided evidence for effective treatments by comparing several competing therapies. Their successes have led to numerous new therapies to combat many diseases. However, since their conclusions are based on the entire cohort in the trial, the treatment recommendation is for everyone, and may not be the best option for an individual. Medical research is now focusing more on providing personalized care for patients, which requires investigating how patient characteristics, including novel biomarkers, modify the effect of current treatment modalities. This is known as heterogeneity of treatment effects. A better understanding of the interaction between treatment and patient-specific prognostic factors will enable practitioners to expand the availability of tailored therapies, with the ultimate goal of improving patient outcomes. The Subpopulation Treatment Effect Pattern Plot (STEPP) approach was developed to allow researchers to investigate the heterogeneity of treatment effects on survival outcomes across values of a (continuously measured) covariate, such as a biomarker measurement.

METHODS

Here, we extend the Subpopulation Treatment Effect Pattern Plot approach to continuous, binary, and count outcomes, which can be easily modeled using generalized linear models. With this extension of Subpopulation Treatment Effect Pattern Plot, these additional types of treatment effects within subpopulations defined with respect to a covariate of interest can be estimated, and the statistical significance of any observed heterogeneity of treatment effect can be assessed using permutation tests. The desirable feature that commonly used models are applied to well-defined patient subgroups to estimate treatment effects is retained in this extension.

RESULTS

We describe a simulation study to confirm that the proper Type I error rate is maintained when there is no treatment heterogeneity, and a power study to show that the statistics have power to detect treatment heterogeneity under alternative scenarios. As an illustration, we apply the methods to data from the Aspirin/Folate Polyp Prevention Study, a clinical trial evaluating the effect of oral aspirin, folic acid, or both as a chemoprevention agent against colorectal adenomas. The pre-existing R software package stepp has been extended to handle continuous, binary, and count data using Gaussian, Bernoulli, and Poisson models, and it is available on the Comprehensive R Archive Network.

CONCLUSION

The extension of the method and the availability of new software now permit STEPP to be applied to the full range of clinical trial end points.

Identifying treatment effect heterogeneity in clinical trials using subpopulations of events: STEPP

BACKGROUND

Investigators conducting randomized clinical trials often explore treatment effect heterogeneity to assess whether treatment efficacy varies according to patient characteristics. Identifying heterogeneity is central to making informed personalized healthcare decisions. Treatment effect heterogeneity can be investigated using subpopulation treatment effect pattern plot (STEPP), a non-parametric graphical approach that constructs overlapping patient subpopulations with varying values of a characteristic. Procedures for statistical testing using subpopulation treatment effect pattern plot when the endpoint of interest is survival remain an area of active investigation.

METHODS

A STEPP analysis was used to explore patterns of absolute and relative treatment effects for varying levels of a breast cancer biomarker, Ki-67, in the phase III Breast International Group 1-98 randomized clinical trial, comparing letrozole to tamoxifen as adjuvant therapy for postmenopausal women with hormone receptor-positive breast cancer. Absolute treatment effects were measured by differences in 4-year cumulative incidence of breast cancer recurrence, while relative effects were measured by the subdistribution hazard ratio in the presence of competing risks using O-E (observed-minus-expected) methodology, an intuitive non-parametric method. While estimation of hazard ratio values based on O-E methodology has been shown, a similar development for the subdistribution hazard ratio has not. Furthermore, we observed that the subpopulation treatment effect pattern plot analysis may not produce results, even with 100 patients within each subpopulation. After further investigation through simulation studies, we observed inflation of the type I error rate of the traditional test statistic and sometimes singular variance-covariance matrix estimates that may lead to results not being produced. This is due to the lack of sufficient number of events within the subpopulations, which we refer to as instability of the subpopulation treatment effect pattern plot analysis. We introduce methodology designed to improve stability of the subpopulation treatment effect pattern plot analysis and generalize O-E methodology to the competing risks setting. Simulation studies were designed to assess the type I error rate of the tests for a variety of treatment effect measures, including subdistribution hazard ratio based on O-E estimation. This subpopulation treatment effect pattern plot methodology and standard regression modeling were used to evaluate heterogeneity of Ki-67 in the Breast International Group 1-98 randomized clinical trial.

RESULTS

We introduce methodology that generalizes O-E methodology to the competing risks setting and that improves stability of the STEPP analysis by pre-specifying the number of events across subpopulations while controlling the type I error rate. The subpopulation treatment effect pattern plot analysis of the Breast International Group 1-98 randomized clinical trial showed that patients with high Ki-67 percentages may benefit most from letrozole, while heterogeneity was not detected using standard regression modeling.

CONCLUSION

The STEPP methodology can be used to study complex patterns of treatment effect heterogeneity, as illustrated in the Breast International Group 1-98 randomized clinical trial. For the subpopulation treatment effect pattern plot analysis, we recommend a minimum of 20 events within each subpopulation.

Permutation Testing for Treatment-Covariate Interactions and Subgroup Identification

We consider the problem of using permutation-based methods to test for treatment-covariate interactions from randomized clinical trial data. Testing for interactions is common in the field of personalized medicine, as subgroups with enhanced treatment effects arise when treatment-by-covariate interactions exist. Asymptotic tests can often be performed for simple models, but in many cases, more complex methods are used to identify subgroups, and non-standard test statistics proposed, and asymptotic results may be difficult to obtain. In such cases, it is natural to consider permutation-based tests, which shuffle selected parts of the data in order to remove one or more associations of interest; however, in the case of interactions, it is generally not possible to remove only the associations of interest by simple permutations of the data. We propose a number of alternative permutation-based methods, designed to remove only the associations of interest, but preserving other associations. These methods estimate the interaction term in a model, then create data that "looks like" the original data except that the interaction term has been permuted. The proposed methods are shown to outperform traditional permutation methods in a simulation study. In addition, the proposed methods are illustrated using data from a randomized clinical trial of patients with hypertension.

Multiple testing of treatment-effect-modifying biomarkers in a randomized clinical trial with a survival endpoint

The recent revolution in genomics and the advent of targeted therapies have increased interest in biomarker-defined subgroups of patients who respond to therapy or exhibit specific toxicities. Such biomarker-defined subgroups are also being investigated for non-targeted therapies (e.g. chemotherapy and statins). However, even when the targeting pathway has been identified, a broadly available test to identify the appropriate subgroup will rarely exist prior to the launch of the pivotal phase III trial. Our aim in this paper is to provide guidance for the analysis of a phase III clinical trial with a survival endpoint, in order to ascertain whether a therapy is more effective in the biomarker-positive patients as compared with biomarker-negative patients, when the trial is conducted on the entire population and when there are multiple candidate biomarkers. We studied treatment-by-biomarker interactions in a Weibull regression model. Different permutation procedures, using single-biomarker statistics and novel composite statistics, are proposed in order to control the family-wise error rate accounting for dependence structures among the biomarkers. A simulation study was performed to compare the operational characteristics of the permutation tests under different scenarios. The tests were applied to a phase III trial of adjuvant chemotherapy in early breast cancer, for which 10 biomarkers were measured in tumor samples from 798 patients. These permutation tests can be applied to retrospective biomarker studies and to prospective phase III trials of new drugs for which a few clues are known about the targeting pathway at the start of the trial.

A small sample study of the STEPP approach to assessing treatment-covariate interactions in survival data

A new, intuitive method has recently been proposed to explore treatment-covariate interactions in survival data arising from two treatment arms of a clinical trial. The method is based on constructing overlapping subpopulations of patients with respect to one (or more) covariates of interest and in observing the pattern of the treatment effects estimated across the subpopulations. A plot of these treatment effects is called a subpopulation treatment effect pattern plot. Here, we explore the small sample characteristics of the asymptotic results associated with the method and develop an alternative permutation distribution-based approach to inference that should be preferred for smaller sample sizes. We then describe an extension of the method to the case in which the pattern of estimated quantiles of survivor functions is of interest.

MULTI-CENTER CLINICAL TRIALS: RANDOMIZATION AND ANCILLARY STATISTICS

The purpose of this paper is to investigate and develop methods for analysis of multi-center randomized clinical trials which only rely on the randomization process as a basis of inference. Our motivation is prompted by the fact that most current statistical procedures used in the analysis of randomized multi-center studies are model based. The randomization feature of the trials is usually ignored. An important characteristic of model based analysis is that it is straightforward to model covariates. Nevertheless in nearly all model based analyses, the effects due to different centers and, in general, the design of the clinical trials are ignored. An alternative to a model based analysis is to have analyses guided by the design of the trial. Our development of design based methods allows the incorporation of centers as well as other features of the trial design. The methods make use of conditioning on the ancillary statistics in the sample space generated by the randomization process. We have investigated the power of the methods and have found that, in the presence of center variation, there is a significant increase in power. The methods have been extended to group sequential trials with similar increases in power.

Modelling clustered survival data from multicentre clinical trials

In randomized clinical trials, subjects are recruited at multiple study centres. Factors that vary across centres may exert a powerful independent influence on study outcomes. A common problem is how to incorporate these centre effects into the analysis of censored time-to-event data. We survey various methods and find substantial advantages in the gamma frailty model. This approach compares favourably with competing methods and appears minimally affected by violation of the assumption of a gamma-distributed frailty. Recent computational advances make use of the gamma frailty model a practical and appealing tool for addressing centre effects in the analysis of multicentre trials.