Statistical considerations in evaluating pharmacogenomics-based clinical effect for confirmatory trials

Pharmacogenetic interactions in amyotrophic lateral sclerosis: a step closer to a cure?

Genetic mutations related to amyotrophic lateral sclerosis (ALS) act through distinct pathophysiological pathways, which may lead to varying treatment responses. Here we assess the genetic interaction between C9orf72, UNC13A, and MOBP with creatine and valproic acid treatment in two clinical trials. Genotypic data was available for 309 of the 338 participants (91.4%). The UNC13A genotype affected mortality (p = 0.012), whereas C9orf72 repeat-expansion carriers exhibited a faster rate of decline in overall (p = 0.051) and bulbar functioning (p = 0.005). A dose-response pharmacogenetic interaction was identified between creatine and the A allele of the MOBP genotype (p = 0.027), suggesting a qualitative interaction in a recessive model (HR 3.96, p = 0.015). Not taking genetic information into account may mask evidence of response to treatment or be an unrecognized source of bias. Incorporating genetic data could help investigators to identify critical treatment clues in patients with ALS.

Identifying treatment effect heterogeneity in clinical trials using subpopulations of events: STEPP

BACKGROUND

Investigators conducting randomized clinical trials often explore treatment effect heterogeneity to assess whether treatment efficacy varies according to patient characteristics. Identifying heterogeneity is central to making informed personalized healthcare decisions. Treatment effect heterogeneity can be investigated using subpopulation treatment effect pattern plot (STEPP), a non-parametric graphical approach that constructs overlapping patient subpopulations with varying values of a characteristic. Procedures for statistical testing using subpopulation treatment effect pattern plot when the endpoint of interest is survival remain an area of active investigation.

METHODS

A STEPP analysis was used to explore patterns of absolute and relative treatment effects for varying levels of a breast cancer biomarker, Ki-67, in the phase III Breast International Group 1-98 randomized clinical trial, comparing letrozole to tamoxifen as adjuvant therapy for postmenopausal women with hormone receptor-positive breast cancer. Absolute treatment effects were measured by differences in 4-year cumulative incidence of breast cancer recurrence, while relative effects were measured by the subdistribution hazard ratio in the presence of competing risks using O-E (observed-minus-expected) methodology, an intuitive non-parametric method. While estimation of hazard ratio values based on O-E methodology has been shown, a similar development for the subdistribution hazard ratio has not. Furthermore, we observed that the subpopulation treatment effect pattern plot analysis may not produce results, even with 100 patients within each subpopulation. After further investigation through simulation studies, we observed inflation of the type I error rate of the traditional test statistic and sometimes singular variance-covariance matrix estimates that may lead to results not being produced. This is due to the lack of sufficient number of events within the subpopulations, which we refer to as instability of the subpopulation treatment effect pattern plot analysis. We introduce methodology designed to improve stability of the subpopulation treatment effect pattern plot analysis and generalize O-E methodology to the competing risks setting. Simulation studies were designed to assess the type I error rate of the tests for a variety of treatment effect measures, including subdistribution hazard ratio based on O-E estimation. This subpopulation treatment effect pattern plot methodology and standard regression modeling were used to evaluate heterogeneity of Ki-67 in the Breast International Group 1-98 randomized clinical trial.

RESULTS

We introduce methodology that generalizes O-E methodology to the competing risks setting and that improves stability of the STEPP analysis by pre-specifying the number of events across subpopulations while controlling the type I error rate. The subpopulation treatment effect pattern plot analysis of the Breast International Group 1-98 randomized clinical trial showed that patients with high Ki-67 percentages may benefit most from letrozole, while heterogeneity was not detected using standard regression modeling.

CONCLUSION

The STEPP methodology can be used to study complex patterns of treatment effect heterogeneity, as illustrated in the Breast International Group 1-98 randomized clinical trial. For the subpopulation treatment effect pattern plot analysis, we recommend a minimum of 20 events within each subpopulation.

Stratification and partial ascertainment of biomarker value in biomarker-driven clinical trials

This article examines the role of stratification of treatment assignment with regard to biomarker value in clinical trials that accept biomarker-positive and -negative patients but have a primary objective of evaluating treatment effect separately for the marker-positive subset. It also examines the issue of incomplete ascertainment of biomarker value and how this affects inference about treatment effect for the biomarker-positive subset of patients. I find that stratifying the randomization for the biomarker ensures that all patients will have tissue collected but is not necessary for the validity of inference for the biomarker-positive subset if there is complete ascertainment. If there is not complete ascertainment of biomarker values, it is important to establish that ascertainment is independent of treatment assignment. Having a large proportion of cases with biomarker ascertainment is not necessary for establishing internal validity of the treatment evaluation in biomarker-positive patients; independence of ascertainment and treatment is the important factor. Having a large proportion of cases with biomarker ascertainment makes it more likely that biomarker-positive patients with ascertainment are representative of the biomarker-positive patients in the clinical trial (with and without ascertainment), but since the patients in the clinical trial are a convenience sample of the population of patients potentially eligible for the trial, requiring a large proportion of cases with ascertainment does not facilitate generalizability of conclusions.

A composite model for subgroup identification and prediction via bicluster analysis

Background

A major challenges in the analysis of large and complex biomedical data is to develop an approach for 1) identifying distinct subgroups in the sampled populations, 2) characterizing their relationships among subgroups, and 3) developing a prediction model to classify subgroup memberships of new samples by finding a set of predictors. Each subgroup can represent different pathogen serotypes of microorganisms, different tumor subtypes in cancer patients, or different genetic makeups of patients related to treatment response.

Methods

This paper proposes a composite model for subgroup identification and prediction using biclusters. A biclustering technique is first used to identify a set of biclusters from the sampled data. For each bicluster, a subgroup-specific binary classifier is built to determine if a particular sample is either inside or outside the bicluster. A composite model, which consists of all binary classifiers, is constructed to classify samples into several disjoint subgroups. The proposed composite model neither depends on any specific biclustering algorithm or patterns of biclusters, nor on any classification algorithms.

Results

The composite model was shown to have an overall accuracy of 97.4% for a synthetic dataset consisting of four subgroups. The model was applied to two datasets where the sample’s subgroup memberships were known. The procedure showed 83.7% accuracy in discriminating lung cancer adenocarcinoma and squamous carcinoma subtypes, and was able to identify 5 serotypes and several subtypes with about 94% accuracy in a pathogen dataset.

Conclusion

The composite model presents a novel approach to developing a biclustering-based classification model from unlabeled sampled data. The proposed approach combines unsupervised biclustering and supervised classification techniques to classify samples into disjoint subgroups based on their associated attributes, such as genotypic factors, phenotypic outcomes, efficacy/safety measures, or responses to treatments. The procedure is useful for identification of unknown species or new biomarkers for targeted therapy.

Pharmacogenomics in asthma therapy: where are we and where do we go?

The response to drug treatment in asthma is a complex trait and is markedly variable even in patients with apparently similar clinical features. Pharmaco-genomics, which is the study of variations of human genome characteristics as related to drug response, can play a role in asthma therapy. Both a traditional candidate-gene approach to conducting genetic association studies and genome-wide association studies have provided an increasing list of genes and variants associated with the three major classes of asthma medications: β2-agonists, inhaled corticosteroids, and leukotriene modifiers. Moreover, a recent integrative, systems-level approach has offered a promising opportunity to identify important pharmacogenomics loci in asthma treatment. However, we are still a long way away from making this discipline directly relevant to patients. The combination of network modeling, functional validation, and integrative omics technologies will likely be needed to move asthma pharmacogenomics closer to clinical relevance.

Statistical analysis of big data on pharmacogenomics

This paper discusses statistical methods for estimating complex correlation structure from large pharmacogenomic datasets. We selectively review several prominent statistical methods for estimating large covariance matrix for understanding correlation structure, inverse covariance matrix for network modeling, large-scale simultaneous tests for selecting significantly differently expressed genes and proteins and genetic markers for complex diseases, and high dimensional variable selection for identifying important molecules for understanding molecule mechanisms in pharmacogenomics. Their applications to gene network estimation and biomarker selection are used to illustrate the methodological power. Several new challenges of Big data analysis, including complex data distribution, missing data, measurement error, spurious correlation, endogeneity, and the need for robust statistical methods, are also discussed.

Characteristics of pharmacogenomics/biomarker-guided clinical trials for regulatory approval of anti-cancer drugs in Japan

Pharmacogenomics (PGx) or biomarker (BM) has the potential to facilitate the development of safer and more effective drugs in terms of their benefit/risk profiles by stratifying population into categories such as responders/non-responders and high-/low-risks to drug-induced serious adverse reactions. In the past decade, practical use of PGx or BM has advanced the field of anti-cancer drug development. To identify the characteristics of the PGx/BM-guided clinical trials for regulatory approval of anti-cancer drugs in Japan, we collected information on design features of 'key trials' in the review reports of anti-cancer drugs that were approved after the implementation of the 'Revised Guideline for the Clinical Evaluation of Anti-cancer drugs' in April 2006. On the basis of the information available on the regulatory review data for the newly approved anti-cancer drugs in Japan, this article aims to explain the limitations and points to consider in the study design of PGx/BM-guided clinical trials.

Regulatory perspective on remaining challenges for utilization of pharmacogenomics-guided drug developments

Pharmacogenomics-guided drug development has been implemented in practice in the last decade, resulting in increased labeling of drugs with pharmacogenomic information. However, there are still many challenges remaining in utilizing this process. Here, we describe such remaining challenges from the regulatory perspective, specifically focusing on sample collection, biomarker qualification, ethnic factors, codevelopment of companion diagnostics and means to provide drugs for off-target patients. To improve the situation, it is important to strengthen international harmonization and collaboration among academia, industries and regulatory agencies, followed by the establishment of an international guideline on this topic. Communication with a regulatory agency from an early stage of drug development is also a key to success.

Stratification of randomization is not required for a pre-specified subgroup analysis

Published literature and regulatory agency guidance documents provide conflicting recommendations as to whether a pre-specified subgroup analysis also requires for its validity that the study employ randomization that is stratified on subgroup membership. This is an important issue, as subgroup analyses are often required to demonstrate efficacy in the development of drugs with a companion diagnostic. Here, it is shown, for typical randomization methods, that the fraction of patients in the subgroup given experimental treatment matches, on average, the target fraction in the entire study. Also, mean covariate values are balanced, on average, between treatment arms in the subgroup, and it is argued that the variance in covariate imbalance between treatment arms in the subgroup is at worst only slightly increased versus a subgroup-stratified randomization method. Finally, in an analysis of variance setting, a least-squares treatment effect estimator within the subgroup is shown to be unbiased whether or not the randomization is stratified on subgroup membership. Thus, a requirement that a study be stratified on subgroup membership would place an artificial roadblock to innovation and the goals of personalized healthcare.

Personalized medicine using DNA biomarkers: a review

Biomarkers are of increasing importance for personalized medicine, with applications including diagnosis, prognosis, and selection of targeted therapies. Their use is extremely diverse, ranging from pharmacodynamics to treatment monitoring. Following a concise review of terminology, we provide examples and current applications of three broad categories of biomarkers—DNA biomarkers, DNA tumor biomarkers, and other general biomarkers. We outline clinical trial phases for identifying and validating diagnostic and prognostic biomarkers. Predictive biomarkers, more generally termed companion diagnostic tests predict treatment response in terms of efficacy and/or safety. We consider suitability of clinical trial designs for predictive biomarkers, including a detailed discussion of validation study designs, with emphasis on interpretation of study results. We specifically discuss the interpretability of treatment effects if a large set of DNA biomarker profiles is available and the number of therapies is identical to the number of different profiles.

Assessment and implication of prognostic imbalance in randomized controlled trials with a binary outcome--a simulation study

BACKGROUND

Chance imbalance in baseline prognosis of a randomized controlled trial can lead to over or underestimation of treatment effects, particularly in trials with small sample sizes. Our study aimed to (1) evaluate the probability of imbalance in a binary prognostic factor (PF) between two treatment arms, (2) investigate the impact of prognostic imbalance on the estimation of a treatment effect, and (3) examine the effect of sample size (n) in relation to the first two objectives.

METHODS

We simulated data from parallel-group trials evaluating a binary outcome by varying the risk of the outcome, effect of the treatment, power and prevalence of the PF, and n. Logistic regression models with and without adjustment for the PF were compared in terms of bias, standard error, coverage of confidence interval and statistical power.

RESULTS

For a PF with a prevalence of 0.5, the probability of a difference in the frequency of the PF≥5% reaches 0.42 with 125/arm. Ignoring a strong PF (relative risk = 5) leads to underestimating the strength of a moderate treatment effect, and the underestimate is independent of n when n is >50/arm. Adjusting for such PF increases statistical power. If the PF is weak (RR = 2), adjustment makes little difference in statistical inference. Conditional on a 5% imbalance of a powerful PF, adjustment reduces the likelihood of large bias. If an absolute measure of imbalance ≥5% is deemed important, including 1000 patients/arm provides sufficient protection against such an imbalance. Two thousand patients/arm may provide an adequate control against large random deviations in treatment effect estimation in the presence of a powerful PF.

CONCLUSIONS

The probability of prognostic imbalance in small trials can be substantial. Covariate adjustment improves estimation accuracy and statistical power, and hence should be performed when strong PFs are observed.

Systems pharmacology, pharmacogenetics, and clinical trial design in network medicine

The rapidly growing disciplines of systems biology and network science are now poised to meet the fields of clinical medicine and pharmacology. Principles of systems pharmacology can be applied to drug design and, ultimately, testing in human clinical trials. Rather than focusing exclusively on single drug targets, systems pharmacology examines the holistic response of a phenotype-dependent pathway or pathways to drug perturbation. Knowledge of individual pharmacogenetic profiles further modulates the responses to these drug perturbations, moving the field toward more individualized ('personalized') drug development. The speed with which the information required to assess these system responses and their genomic underpinnings is changing and the importance of identifying the optimal drug or drug combinations for maximal benefit and minimal risk require that clinical trial design strategies be adaptable. In this paper, we review the tenets of adaptive clinical trial design as they may apply to an era of expanding knowledge of systems pharmacology and pharmacogenomics, and clinical trail design in network medicine.

Pharmacogenetics in randomized controlled trials: considerations for trial design

Pharmacogenetic analyses of clinical trials aim to either detect whether a subgroup of patients identified by genetic characteristics responds differently to the treatment or to verify whether a proposed genotype-guided treatment is beneficial over standard care. This article describes three different trial designs, differing in the timing of randomization and genotyping. Each design has its own advantages, and the objectives and conditions under which each one is most suited are discussed.

Pharmacogenetics: practices and opportunities for study design and data analysis

Pharmacogenetics (PGx) is increasingly used as a way to target treatment to patients who are most likely to benefit. To date, PGx has shown clinical significance across a few applications but widespread use has been limited by the need for further technical, methodological and practical advances and for educating clinical researchers on the value of PGx. Here, I describe the current scope of PGx research, including recent contributions to prospective study design. A case study is included to demonstrate the limitations of current practice and to describe some practical steps for improving the chances of identifying genetic effects. Additionally, I describe some opportunities for the integration and application of disparate data sources in exploratory PGx research.

Integrating biomarkers in clinical trials

Biomarkers have a growing role in clinical trials. With the advent of the targeted therapy era, molecular biomarkers in particular are becoming increasingly important within both clinical research and clinical practice. This article focuses on biomarkers that anticipate the prognosis of individual patients ('prognostic' biomarkers) and on biomarkers that predict how individual patients will respond to specific treatments ('predictive' biomarkers, also called 'effect modifiers'). Specific Phase II and III clinical trial designs are discussed in detail for their ability to validate the biomarker and/or to establish the effect of an experimental therapy in patient populations defined by the presence or absence of the biomarker. Contemporary biomarker-based clinical trials in oncology are used as examples.

Designing pharmacogenomic studies to be fit for purpose

The 2010 US FDA–Drug Industry Association (DIA) Pharmacogenomics Workshop, the fifth in a series of meetings that begun in 2002, brought together multidisciplinary experts from regulatory authorities, medical research, healthcare and drug development. This article summarizes the ‘Designing Pharmacogenomic Studies to be Fit for Purpose’ track in which considerations regarding the use of retrospective and prospective studies were examined in relation to their ability to influence treatment decisions and labeling for drugs. The aim of the session, using real-world examples (KRAS/panitumumab and HLA-B*5701/abacavir), was to identify good scientific principles that would guide the design of studies to identify subgroups of responders during development programs (including marketed drugs), which could subsequently be used to guide treatment decisions.