Subgroup Analyses in Randomized Clinical Trials: Statistical and Regulatory Issues

Pragmatic Trial Design to Compare Real-world Effectiveness of Different Treatments for Inflammatory Bowel Diseases: The PRACTICE-IBD European Consensus

BACKGROUND AND AIMS

Pragmatic studies designed to test interventions in everyday clinical settings can successfully complement the evidence from registration and explanatory clinical trials. The European consensus project PRACTICE-IBD was developed to identify essential criteria and address key methodological issues needed to design valid, comparative, pragmatic studies in inflammatory bowel diseases [BDs].

METHODS

Statements were issued by a panel of 11 European experts in IBD management and trial methodology, on four main topics: [I] study design; [II] eligibility, recruitment and organisation, flexibility; [III] outcomes; [IV] analysis. The consensus process followed a modified Delphi approach, involving two rounds of assessment and rating of the level of agreement [1 to 9; cut-off ≥7 for approval] with the statements by 18 additional European experts in IBD.

RESULTS

At the first voting round, 25 out of the 26 statements reached a mean score ≥7. Following the discussion that preceded the second round of voting, it was decided to eliminate two statements and to split one into two. At the second voting round, 25 final statements were approved: seven for study design; six for eligibility, recruitment and organisation, flexibility; eight for outcomes; and four for analysis.

CONCLUSIONS

Pragmatic, randomised, clinical trials can address important questions in IBD clinical practice, and may provide complementary, high-level evidence, as long as they follow a methodologically rigorous approach. These 25 statements intend to offer practical guidance in the design of high-quality, pragmatic, clinical trials that can aid decision making in choosing a management strategy for IBDs.

Designing and testing treatments for alcohol use disorder

This chapter provides a succinct overview of several recommendations for the design and analysis of treatments for AUD with a specific focus on increasing rigor and generalizability of treatment studies in order to increase the reach of AUD treatment. We recommend that researchers always register their trials in a clinical trial registry and make the protocol accessible so that the trial can be replicated in future work, follow CONSORT reporting guidelines when reporting the results of the trial, carefully describe all inclusion and exclusion criteria as well as the randomization scheme, and always use an intent to treat design with attention to analysis of missing data. In addition, we recommend that researchers pay closer attention to recruitment and engagement strategies that increase enrollment and retention of historically marginalized and understudied populations, and we end with a plea for more consideration of implementation science approaches to increase the dissemination and implementation of AUD treatment in real-world settings.

A platform for comparing subgroup identification methodologies

Since the advent of the phrase "subgroup identification," there has been an explosion of methodologies that seek to identify meaningful subgroups of patients with exceptional response in order to further the realization of personalized medicine. However, to perform fair comparison and understand what methods work best under different clinical trials situations, a common platform is needed for comparative effectiveness of these various approaches. In this paper, we describe a comprehensive project that created an extensive platform for evaluating subgroup identification methods as well as a publicly posted challenge that was used to elicit new approaches. We proposed a common data-generating model for creating virtual clinical trial datasets that contain subgroups of exceptional responders encompassing the many dimensions of the problem or null scenarios in which there are no such subgroups. Furthermore, we created a common scoring system for evaluating performance of purported methods for identifying subgroups. This makes it possible to benchmark methodologies in order to understand what methods work best under different clinical trial situations. The findings from this project produced considerable insights and allow us to make recommendations for how the statistical community can better compare and contrast old and new subgroup identification methodologies.

A shrinkage estimator for subgroup analysis without the exchangeability assumption

Shrinkage estimators for exploratory subgroup analyses are intuitively appealing and can greatly improve estimation over standard analysis approaches; however, adoption of these estimators has been limited by reliance on the exchangeability assumption. This paper describes a new shrinkage estimator that does not rely on this assumption. Rather than assuming that treatment effect sizes within subgroups are randomly distributed around an overall mean, this new estimator assumes that the difference between the effect sizes in any given pair of subgroups is randomly distributed around zero. The estimator is illustrated using data from a clinical trial in which the treatment effect size in one region was substantially different from the sizes in other regions. Simulation results show that the estimator has properties that are comparable to or superior to a standard shrinkage estimator when exchangeability is assumed, while allowing the flexibility to handle situations where exchangeability cannot be assumed.

Accounting for Heterogeneity in Resource Allocation Decisions: Methods and Practice in UK Cancer Technology Appraisals

OBJECTIVES

The availability of novel, more efficacious and expensive cancer therapies is increasing, resulting in significant treatment effect heterogeneity and complicated treatment and disease pathways. The aim of this study is to review the extent to which UK cancer technology appraisals (TAs) consider the impact of patient and treatment effect heterogeneity.

METHODS

A systematic search of National Institute for Health and Care Excellence TAs of colorectal, lung and ovarian cancer was undertaken for the period up to April 2020. For each TA, the pivotal clinical studies and economic evaluations were reviewed for considerations of patient and treatment effect heterogeneity. The study critically reviews the use of subgroup analysis and real-world translation in economic evaluations, alongside specific attributes of the economic modeling framework.

RESULTS

The search identified 49 TAs including 49 economic models. In total, 804 subgroup analyses were reported across 69 clinical studies. The most common stratification factors were age, gender, and Eastern Cooperative Oncology Group performance score, with 15% (119 of 804) of analyses demonstrating significantly different clinical outcomes to the main population; economic subgroup analyses were undertaken in only 17 TAs. All economic models were cohort-level with the majority described as partitioned survival models (39) or Markov/semi-Markov models. The impact of real-world heterogeneity on disease progression estimates was only explored in 2 models.

CONCLUSION

The ability of current modeling approaches to capture patient and treatment effect heterogeneity is constrained by their limited flexibility and simplistic nature. This study highlights a need for the use of more sophisticated modeling methods that enable greater consideration of real-world heterogeneity.

Effects of Combined Resistance and Power Training on Cognitive Function in Older Women: A Randomized Controlled Trial

The present study compared the effects of traditional resistance training (TRT) and combined power training (PT) and TRT (PTRT) on cognitive parameters and serum brain-derived neurotrophic factor (BDNF) levels in non-demented, well-functioning, community-dwelling older women. Forty-five older women were randomized into one of three experimental groups: TRT, PTRT, and control group (CG). Cognitive tests explored global cognitive function, short-term memory, and dual-task performance. Serum BDNF levels were assessed at baseline and after the intervention. Exercise sessions were performed twice a week over 22 weeks. In TRT, exercise sessions were based on three sets of 8-10 repetitions at "difficult" intensity. In PTRT, the first session was based on PT (three sets of 8-10 repetitions at "moderate" intensity), while the second session was similar to the TRT. Our analyses indicated that overall cognitive function, short-term memory, and dual-task performance were similarly improved after TRT and PTRT. Serum BDNF concentrations were not altered by any training protocol. In conclusion, the two RT programs tested in the present trial improved global cognitive function, short-term memory and dual task performance in non-demented, well-functioning, community-dwelling older women. In addition, our findings suggest that mechanisms other than BDNF may be associated with such improvements.

Bayesian hierarchical meta-analytic methods for modeling surrogate relationships that vary across treatment classes using aggregate data

Surrogate endpoints play an important role in drug development when they can be used to measure treatment effect early compared to the final clinical outcome and to predict clinical benefit or harm. Such endpoints are assessed for their predictive value of clinical benefit by investigating the surrogate relationship between treatment effects on the surrogate and final outcomes using meta‐analytic methods. When surrogate relationships vary across treatment classes, such validation may fail due to limited data within each treatment class. In this paper, two alternative Bayesian meta‐analytic methods are introduced which allow for borrowing of information from other treatment classes when exploring the surrogacy in a particular class. The first approach extends a standard model for the evaluation of surrogate endpoints to a hierarchical meta‐analysis model assuming full exchangeability of surrogate relationships across all the treatment classes, thus facilitating borrowing of information across the classes. The second method is able to relax this assumption by allowing for partial exchangeability of surrogate relationships across treatment classes to avoid excessive borrowing of information from distinctly different classes. We carried out a simulation study to assess the proposed methods in nine data scenarios and compared them with subgroup analysis using the standard model within each treatment class. We also applied the methods to an illustrative example in colorectal cancer which led to obtaining the parameters describing the surrogate relationships with higher precision.

Reporting of health equity considerations in cluster and individually randomized trials

Background

The randomized controlled trial (RCT) is considered the gold standard study design to inform decisions about the effectiveness of interventions. However, a common limitation is inadequate reporting of the applicability of the intervention and trial results for people who are “socially disadvantaged” and this can affect policy-makers’ decisions. We previously developed a framework for identifying health-equity-relevant trials, along with a reporting guideline for transparent reporting. In this study, we provide a descriptive assessment of health-equity considerations in 200 randomly sampled equity-relevant trials.

Methods

We developed a search strategy to identify health-equity-relevant trials published between 2013 and 2015. We randomly sorted the 4316 records identified by the search and screened studies until 100 individually randomized (RCTs) and 100 cluster randomized controlled trials (CRTs) were identified. We developed and pilot-tested a data extraction form based on our initial work, to inform the development of our reporting guideline for equity-relevant randomized trials.

Results

In total, 39 trials (20%) were conducted in a low- and middle-income country and 157 trials (79%) in a high-income country focused on socially disadvantaged populations (78% CRTs, 79% RCTs). Seventy-four trials (37%) reported a subgroup analysis across a population characteristic associated with disadvantage (25% CRT, 49% RCTs), with 19% of included studies reporting subgroup analyses across sex, 9% across race/ethnicity/culture, and 4% across socioeconomic status. No subgroup analyses were reported for place of residence, occupation, religion, education, or social capital. One hundred and forty-one trials (71%) discussed the applicability of their results to one or more socially disadvantaged populations (68% of CRT, 73% of RCT).

Discussion

In this set of trials, selected for their relevance to health equity, data that were disaggregated for socially disadvantaged populations were rarely reported. We found that even when the data are available, opportunities to analyze health-equity considerations are frequently missed. The recently published equity extension of the Consolidated Reporting Standards for Randomized Trials (CONSORT-Equity) may help improve delineation of hypotheses related to socially disadvantaged populations, and transparency and completeness of reporting of health-equity considerations in RCTs. This study can serve as a baseline assessment of the reporting of equity considerations.

Cost-utility analysis of learning and coping versus standard education in cardiac rehabilitation: a randomised controlled trial with 3 years of follow-up

Objectives

To enhance adherence to cardiac rehabilitation (CR), a patient education programme called 'learning and coping' (LC-programme) was implemented in three hospitals in Denmark. The aim of this study was to investigate the cost-utility of the LC-programme compared with the standard CR-programme.

Methods

825 patients with ischaemic heart disease or heart failure were randomised to the LC-programme or the standard CR-programme and were followed for 3 years.A societal cost perspective was applied and quality-adjusted life years (QALY) were based on SF-6D measurements. Multiple imputation technique was used to handle missing data on the SF-6D. The statistical analyses were based on means and bootstrapped SEs. Regression framework was employed to estimate the net benefit and to illustrate cost-effectiveness acceptability curves.

Results

No statistically significant differences were found between the two programmes in total societal costs (4353 Euros; 95% CI -3828 to 12 533) or in QALY (-0.006; 95% CI -0.053 to 0.042). At a threshold of 40 000 Euros, the LC-programme was found to be cost-effective at 15% probability; however, for patients with heart failure, due to increased cost savings, the probability of cost-effectiveness increased to 91%.

Conclusions

While the LC-programme did not appear to be cost-effective in CR, important heterogeneity was noted for subgroups of patients. The LC-programme was demonstrated to increase adherence to the rehabilitation programme and to be cost-effective among patients with heart failure. However, further research is needed to study the dynamic value of heterogeneity due to the small sample size in this subgroup.

A systematic survey identified 36 criteria for assessing effect modification claims in randomized trials or meta-analyses

OBJECTIVE

The objective of the study was to systematically survey the methodological literature and collect suggested criteria for assessing the credibility of effect modification and associated rationales.

STUDY DESIGN AND SETTING

We searched MEDLINE, Embase, and WorldCat up to March 2018 for publications providing guidance for assessing the credibility of effect modification identified in randomized trials or meta-analyses. Teams of two investigators independently identified eligible publications and extracted credibility criteria and authors' rationale, reaching consensus through discussion. We created a taxonomy of criteria that we iteratively refined during data abstraction.

RESULTS

We identified 150 eligible publications that provided 36 criteria and associated rationales. Frequent criteria included significant test for interaction (n = 54), a priori hypothesis (n = 49), providing a causal explanation (n = 47), accounting for multiplicity (n = 45), testing a small number of effect modifiers (n = 38), and prespecification of analytic details (n = 39). For some criteria, we found more than one rationale; some criteria were connected through a common rationale. For some criteria, experts disagreed regarding their suitability (e.g., added value of stratified randomization; trustworthiness of biologic rationales).

CONCLUSION

Methodologists have expended substantial intellectual energy providing criteria for critical appraisal of apparent effect modification. Our survey highlights popular criteria, expert agreement and disagreement, and where more work is needed, including testing criteria in practice.

Towards an appropriate framework to facilitate responsible inclusion of pregnant women in drug development programs

Evidence-based treatment for pregnant women will ultimately require research conducted in the population of pregnant women. Currently, few scholars have addressed the issue of responsible inclusion of pregnant women in drug research. Because of additional risks associated with including pregnant women in drug research and the altered ways in which drugs are processed by the pregnant body, pregnant women cannot be treated as an ordinary subgroup in the various phases of drug development. Instead, responsible inclusion of pregnant women requires careful design and planning of research for pregnant women specifically. Knowledge about these aspects is virtually nonexistent.

In this article, we present a practical framework for the responsible inclusion of pregnant women in drug development. We suggest that the framework consists of using a question-based approach with five key questions in combination with three prerequisites which should be addressed when considering inclusion of pregnant women in drug research. The five questions are:A.

Can we consider the drug safe (enough) for first exposure in pregnant women and fetuses?

B.

In which dose range (potentially depending on gestational age) can the drug be considered to remain safe in pregnant women?

C.

At what dose (regimen, within the range considered safe) can we expect efficacy in pregnant women?

D.

Can efficacy be confirmed at the target dose, either similar to the initial population or different?

E.

Can clinical safety be confirmed at a sufficiently acceptable level at the target dose for pregnant women and fetuses, so as to conclude a positive benefit–risk ratio?

Combining questions and prerequisites leads to a scheme for appropriate timing of responsible inclusion of pregnant women in drug research. Accordingly, we explore several research design options for including pregnant women in drug trials that are feasible within the framework. Ultimately, the framework may lead to (i) earlier inclusion of pregnant women in drug development, (ii) ensuring that key prerequisites, such as proper dosing, are addressed before more substantial numbers of pregnant women are included in trials, and (iii) optimal use of safety and efficacy data from the initial (nonpregnant) population throughout the drug development process.

General guidance on exploratory and confirmatory subgroup analysis in late-stage clinical trials

This article focuses on a broad class of statistical and clinical considerations related to the assessment of treatment effects across patient subgroups in late-stage clinical trials. This article begins with a comprehensive review of clinical trial literature and regulatory guidelines to help define scientifically sound approaches to evaluating subgroup effects in clinical trials. All commonly used types of subgroup analysis are considered in the article, including different variations of prospectively defined and post-hoc subgroup investigations. In the context of confirmatory subgroup analysis, key design and analysis options are presented, which includes conventional and innovative trial designs that support multi-population tailoring approaches. A detailed summary of exploratory subgroup analysis (with the purpose of either consistency assessment or subgroup identification) is also provided. The article promotes a more disciplined approach to post-hoc subgroup identification and formulates key principles that support reliable evaluation of subgroup effects in this setting.

Subgroup analyses in confirmatory clinical trials: time to be specific about their purposes

Background

It is well recognized that treatment effects may not be homogeneous across the study population. Subgroup analyses constitute a fundamental step in the assessment of evidence from confirmatory (Phase III) clinical trials, where conclusions for the overall study population might not hold. Subgroup analyses can have different and distinct purposes, requiring specific design and analysis solutions. It is relevant to evaluate methodological developments in subgroup analyses against these purposes to guide health care professionals and regulators as well as to identify gaps in current methodology.

Methods

We defined four purposes for subgroup analyses: (1) Investigate the consistency of treatment effects across subgroups of clinical importance, (2) Explore the treatment effect across different subgroups within an overall non-significant trial, (3) Evaluate safety profiles limited to one or a few subgroup(s), (4) Establish efficacy in the targeted subgroup when included in a confirmatory testing strategy of a single trial. We reviewed the methodology in line with this “purpose-based” framework. The review covered papers published between January 2005 and April 2015 and aimed to classify them in none, one or more of the aforementioned purposes.

Results

In total 1857 potentially eligible papers were identified. Forty-eight papers were selected and 20 additional relevant papers were identified from their references, leading to 68 papers in total. Nineteen were dedicated to purpose 1, 16 to purpose 4, one to purpose 2 and none to purpose 3. Seven papers were dedicated to more than one purpose, the 25 remaining could not be classified unambiguously. Purposes of the methods were often not specifically indicated, methods for subgroup analysis for safety purposes were almost absent and a multitude of diverse methods were developed for purpose (1).

Conclusions

It is important that researchers developing methodology for subgroup analysis explicitly clarify the objectives of their methods in terms that can be understood from a patient’s, health care provider’s and/or regulator’s perspective. A clear operational definition for consistency of treatment effects across subgroups is lacking, but is needed to improve the usability of subgroup analyses in this setting. Finally, methods to particularly explore benefit-risk systematically across subgroups need more research.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0122-6) contains supplementary material, which is available to authorized users.

Recommendations for the Design and Analysis of Treatment Trials for Alcohol Use Disorders

BACKGROUND

Over the past 60 years, the view that "alcoholism" is a disease for which the only acceptable goal of treatment is abstinence has given way to the recognition that alcohol use disorders (AUDs) occur on a continuum of severity, for which a variety of treatment options are appropriate. However, because the available treatments for AUDs are not effective for everyone, more research is needed to develop novel and more efficacious treatments to address the range of AUD severity in diverse populations. Here we offer recommendations for the design and analysis of alcohol treatment trials, with a specific focus on the careful conduct of randomized clinical trials of medications and nonpharmacological interventions for AUDs.

METHODS

This paper provides a narrative review of the quality of published clinical trials and recommendations for the optimal design and analysis of treatment trials for AUDs.

RESULTS

Despite considerable improvements in the design of alcohol clinical trials over the past 2 decades, many studies of AUD treatments have used faulty design features and statistical methods that are known to produce biased estimates of treatment efficacy.

CONCLUSIONS

The published statistical and methodological literatures provide clear guidance on methods to improve clinical trial design and analysis. Consistent use of state-of-the-art design features and analytic approaches will enhance the internal and external validity of treatment trials for AUDs across the spectrum of severity. The ultimate result of this attention to methodological rigor is that better treatment options will be identified for patients with an AUD.

The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application

This article develops a general framework to guide the use of subgroup cost-effectiveness analysis for decision making in a collectively funded health system. In doing so, it addresses 2 key policy questions, namely, the identification and selection of subgroups, while distinguishing 2 sources of potential value associated with heterogeneity. These are 1) the value of revealing the factors associated with heterogeneity in costs and outcomes using existing evidence (static value) and 2) the value of acquiring further subgroup-related evidence to resolve the uncertainty given the current understanding of heterogeneity (dynamic value). Consideration of these 2 sources of value can guide subgroup-specific treatment decisions and inform whether further research should be conducted to resolve uncertainty to explain variability in costs and outcomes. We apply the proposed methods to a cost-effectiveness analysis for the management of patients with acute coronary syndrome. This study presents the expected net benefits under current and perfect information when subgroups are defined based on the use and combination of 6 binary covariates. The results of the case study confirm the theoretical expectations. As more subgroups are considered, the marginal net benefit gains obtained under the current information show diminishing marginal returns, and the expected value of perfect information shows a decreasing trend. We present a suggested algorithm that synthesizes the results to guide policy.

Subgroups: time to go back to basic statistical principles?

Exploratory subgroup analyses are an increasing source of controversy as part of the interpretation of the results of clinical trials. In this article, we review the major challenges of multiplicity, statistical methods available to assess consistency of effect, and the part appropriate design plays in mitigating the risk of false conclusions from subgroup analyses. We discuss the problems associated with using definitions of consistency based on effect sizes in specific subgroups. We argue that what is required is a return to basic statistical principles, including more use of modeling techniques.

Level of evidence for promising subgroup findings in an overall non-significant trial

In drug development and drug licensing, it sometimes occurs that a new drug does not demonstrate effectiveness for the full study population, but there appears to be benefit in a relevant, pre-defined subgroup. This raises the question, how strong the evidence from such a subgroup is, and which confirmatory testing strategies are the most appropriate ones. Hence, we considered the type I error and the power of a subgroup result in a trial with non-significant overall results and of suitable replication strategies. In the case of a single trial, the inflation of the overall type I error is substantial and can be up to twice as large, especially in relatively small subgroups. This also increases to the risk of starting a replication trial that should not be done, if such a second trial is not already available. The overall type I error is almost controlled by using an appropriate replication strategy. This confirms the required cautious interpretation of promising subgroups, even in the case that overall trial results were perceived to be close to significance.

Adaptive designs for confirmatory clinical trials with subgroup selection

Growing interest in stratified medicine is leading to increasing importance of subgroup analyses in confirmatory clinical trials. Conventionally, confirmatory clinical trials either focus on a subgroup identified in advance or assess subgroup effects once the trial is completed. The focus of this article is methodology for adaptive clinical trials that both identify whether a treatment is particularly effective in a predefined subgroup, potentially enabling alteration of recruitment, and assess the effectiveness in the subgroup and/or whole population. Methods for such adaptive trials are described and compared, and the logistical and regulatory issues associated with such approaches are discussed.

Multiplicity considerations for subgroup analysis subject to consistency constraint

A significant heterogeneity in response across subgroups of a clinical trial implies that the average response from the overall population might not characterize the treatment effect; and as noted by different regulatory guidances, can cause concerns in interpreting study findings and might lead to restricting treatment labeling. However, along with the challenges raised by the heterogeneity, recently there has been growing interest in taking advantage of the expected variability in response across subgroups to increase the chance of success of a trial by designing the trial with objectives of establishing efficacy claims for the total population and a targeted subgroup. For such trials, there have been several approaches to address the multiplicity issue with the two paths of success. This manuscript advocates the utility of setting a threshold on the treatment effect for the subgroups at the design stage to guide determination of the population labeling when significant findings for the total population have been established. Specifically, it proposes that licensing treatment for the total population requires, in addition to significant findings for this population, that the treatment effect in the least benefited (complementary) subgroup meets the treatment effect threshold at a minimum; otherwise, the treatment would be restricted to the targeted subgroup only. Setting such a threshold can be based on clinical considerations, including toxicity and adverse events, in addition to treatment effect in the subgroup. This manuscript expands some of the multiplicity approaches to account for the threshold requirement and investigates the impact of the threshold requirement on study power.

A comparison of subgroup analyses in grant applications and publications

In this paper, the authors compare subgroup analyses as outlined in grant applications and their related publications. Grants awarded by the Netherlands Organization for Health Research and Development (ZonMw) from 2001 onward that were finalized before March 1, 2010, were studied. Of the 79 grant proposals, 50 (63%) were intervention studies, 18 (23%) were diagnostic studies, and 6 (8%) were prognostic studies. Subgroups were mentioned in 49 (62%) grant applications and in 53 (67%) publications. In 20 of the 79 projects (25%), the publications were completely in agreement with the grant proposal; that is, subgroups that were prespecified in the grant proposal were reported and no new subgroup analyses were introduced in the publications. Of the 149 prespecified subgroups, 46 (31%) were reported in the final report or scientific publications, and 143 of the 189 (76%) reported subgroups were based on post-hoc findings. For 77% of the subgroup analyses in the publications, there was no mention of whether these were prespecified or post hoc. Justification for subgroup analysis and methods to study subgroups were rarely reported. The authors conclude that there is a large discrepancy between grant applications and final publications regarding subgroup analyses. Both nonreporting prespecified subgroup analyses and reporting post-hoc subgroup analyses are common. More guidance is clearly needed.

Bayesian models for subgroup analysis in clinical trials

BACKGROUND

In a pharmaceutical drug development setting, possible interactions between the treatment and particular baseline clinical or demographic factors are often of interest. However, the subgroup analysis required to investigate such associations remains controversial. Concerns with classical hypothesis testing approaches to the problem include low power, multiple testing, and the possibility of data dredging.

PURPOSE

As an alternative to hypothesis testing, the use of shrinkage estimation techniques is investigated in the context of an exploratory post hoc subgroup analysis. A range of models that have been suggested in the literature are reviewed. Building on this, we explore a general modeling strategy, considering various options for shrinkage of effect estimates. This is applied to a case-study, in which evidence was available from seven-phase II-III clinical trials examining a novel therapy, and also to two artificial datasets with the same structure.

METHODS

Emphasis is placed on hierarchical modeling techniques, adopted within a Bayesian framework using freely available software. A range of possible subgroup model structures are applied, each incorporating shrinkage estimation techniques.

RESULTS

The investigation of the case-study showed little evidence of subgroup effects. Because inferences appeared to be consistent across a range of well-supported models, and model diagnostic checks showed no obvious problems, it seemed this conclusion was robust. It is reassuring that the structured shrinkage techniques appeared to work well in a situation where deeper inspection of the data suggested little evidence of subgroup effects.

LIMITATIONS

The post hoc examination of subgroups should be seen as an exploratory analysis, used to help make better informed decisions regarding potential future studies examining specific subgroups. To a certain extent, the degree of understanding provided by such assessments will be limited by the quality and quantity of available data.

CONCLUSIONS

In light of recent interest by health authorities into the use of subgroup analysis in the context of drug development, it appears that Bayesian approaches involving shrinkage techniques could play an important role in this area. Hopefully, the developments outlined here provide useful methodology for tackling such a problem, in-turn leading to better informed decisions regarding subgroups.

Targeting population entering phase III trials: a new stratified adaptive phase II design

The primary goal of phase II studies is to assess the efficacy of the new treatment in order to decide whether it has sufficient activity to warrant further evaluation in a phase III comparative trial. However, many adequately conducted phase II trials are negative leading to termination of drug development. Heterogeneity of the population is often considered to be a cause of treatment effect dilution. One approach to determine the sensitive subpopulation is to conduct several phase II trials, one in each specific subset of patients. This option might unethically increase the number of non-sensitive patients under evaluation. Adaptive two-stage designs have been recently proposed. London and Chang proposed a global one-sample test for response rates for stratified phase II clinical trials, whereas Jones and Holmgren proposed an adaptive design that allows preliminary determination of efficacy that may be restricted to a specific subpopulation defined by biomarker status. These two methods do not allow early termination for efficacy in one or several subgroups as they are extensions of the Simon design. The authors propose an alternative method to deal with stratification in phase II clinical trials and identification of the best target population. This method is based on the multiple-stage Fleming design allowing for early stopping rules for either efficacy or inefficacy. It also integrates a procedure testing whether treatment effects are similar or heterogeneous between the two groups. The operating characteristics of this method were compared with those of a standard Fleming design using exact binomial probabilities.

Inconsistent trial assessments by the National Institute for Health and Clinical Excellence and IQWiG: standards for the performance and interpretation of subgroup analyses are needed

OBJECTIVES

The methodology for the critical assessment of medical interventions is well established. Regulatory agencies and institutions adhere, in principle, to the same standards. This consistency, however, is not always the case in practice.

STUDY DESIGN AND SETTING

Using the evaluation of the CAPRIE (Clopidogrel versus Aspirin in Patients at risk of Ischemic Events) trial by the British National Institute for Health and Clinical Excellence (NICE) and the German Institute for Quality and Efficiency in Health Care (IQWiG), we illustrate that there was no consensus for the interpretation of possible heterogeneity in treatment comparisons across subgroups.

RESULTS

The NICE concluded that CAPRIE demonstrated clinical benefit for the overall intention-to-treat (ITT) population with sufficient robustness to possible sources of heterogeneity. The IQWiG interpreted the alleged heterogeneity as implying that the clinical benefit only applied to the subgroup of patients with a statistically significant result irrespective of the results of the ITT analysis.

CONCLUSION

International standards for the performance and interpretation of subgroup analyses as well as for the assessment of heterogeneity between strata are needed.

Virological and immunological responses to efavirenz or boosted lopinavir as first-line therapy for patients with HIV

BACKGROUND

Efavirenz and lopinavir boosted with ritonavir are both recommended as first-line therapies for patients with HIV when combined with two nucleoside reverse transcriptase inhibitors. It is uncertain which therapy is more effective for patients starting therapy with an advanced infection.

METHODS

We estimated the relative effect of these two therapies on rates of virological and immunological failure within the Swiss HIV Cohort Study and considered whether estimates depended on the CD4(+) T-cell count when starting therapy. We defined virological failure as either an incomplete virological response or viral rebound after viral suppression and immunological failure as failure to achieve an expected CD4(+) T-cell increase calculated from EuroSIDA statistics.

RESULTS

Patients starting efavirenz (n=660) and lopinavir (n=541) were followed for a median of 4.5 and 3.1 years, respectively. Virological failure was less likely for patients on efavirenz, with the adjusted hazard ratio (95% confidence interval) of 0.63 (0.50-0.78) then multiplied by a factor of 1.00 (0.90-1.12) for each 100 cells/mm(3) decrease in CD4(+) T-cell count below the mean when starting therapy. Immunological failure was also less likely for patients on efavirenz, with the adjusted hazard ratio of 0.68 (0.51-0.91) then multiplied by a factor of 1.29 (1.14-1.46) for each 100 cells/mm(3) decrease in CD4(+) T-cell count below the mean when starting therapy.

CONCLUSIONS

Virological failure is less likely with efavirenz regardless of the CD4(+) T-cell count when starting therapy. Immunological failure is also less likely with efavirenz; however, this advantage disappears if patients start therapy with a low CD4(+) T-cell count.

A method for testing a prespecified subgroup in clinical trials

In clinical trials, investigators are often interested in the effect of a given study treatment on a subgroup of patients with certain clinical or biological attributes in addition to its effect on the overall study population. Such a subgroup analysis would become even more important to the study sponsor if an efficacy claim can be made for the subgroup when the test for the overall study population fails at a prespecified alpha level. In practice, such a claim is often dependent on prespecification of the subgroup and certain implicit or explicit requirements placed on the study results due to ethical or regulatory concerns. By carefully considering these requirements, we propose a general statistical methodology for testing both the overall and subgroup hypotheses, which has optimal power and strongly controls the familywise Type I error rate.

Bayesian sample size determination in non-sequential clinical trials: Statistical aspects and some regulatory considerations

The most common Bayesian methods for sample size determination (SSD) are reviewed in the non-sequential context of a confirmatory phase III trial in drug development. After recalling the regulatory viewpoint on SSD, we discuss the relevance of the various priors applied to the planning of clinical trials. We then investigate whether these Bayesian methods could compete with the usual frequentist approach to SSD and be considered as acceptable from a regulatory viewpoint.