Why ITT analysis is not always the answer for estimating treatment effects in clinical trials

Demystifying estimands in cluster-randomised trials

Estimands can help clarify the interpretation of treatment effects and ensure that estimators are aligned with the study's objectives. Cluster-randomised trials require additional attributes to be defined within the estimand compared to individually randomised trials, including whether treatment effects are marginal or cluster-specific, and whether they are participant- or cluster-average. In this paper, we provide formal definitions of estimands encompassing both these attributes using potential outcomes notation and describe differences between them. We then provide an overview of estimators for each estimand, describe their assumptions, and show consistency (i.e. asymptotically unbiased estimation) for a series of analyses based on cluster-level summaries. Then, through a re-analysis of a published cluster-randomised trial, we demonstrate that the choice of both estimand and estimator can affect interpretation. For instance, the estimated odds ratio ranged from 1.38 (p = 0.17) to 1.83 (p = 0.03) depending on the target estimand, and for some estimands, the choice of estimator affected the conclusions by leading to smaller treatment effect estimates. We conclude that careful specification of the estimand, along with an appropriate choice of estimator, is essential to ensuring that cluster-randomised trials address the right question.

Adherence, per-protocol effects, and the estimands framework

In the statistical literature, treatment effects in clinical trials are frequently described as either ITT or per-protocol effects. The estimand given for the per-protocol effect is the effect in adherers, where adherers are typically defined as adhering to the intervention as specified in the trial protocol. This dichotomy of treatment effects is unhelpful when there are in reality multiple treatment effects that can be of clinical interest and relevance. The terms "per-protocol" and "adherence" are confusing to non-statisticians. Protocols always allow for discontinuation of randomized treatment so participants discontinuing have actually followed the protocol. When rescue or additional medication is available, the effect in adherers could mean the effect regardless of use of these medications or the effect in a counterfactual world where the participant did not take the medication. Adherence can mean continuing to be prescribed a treatment or some arbitrary level of compliance with a medication that has been prescribed. The ICH E9 (R1) estimands framework provides an improved alternative for the description of treatment effects in clinical trials. Identification of important intercurrent events and the strategy used to handle these events is key to determining the treatment effect. When designing a trial, estimands should be properly defined according to this framework. It is time the statistical literature abandoned describing treatment effects as the effect in adherers or the per-protocol effect.

Why estimands are needed to define treatment effects in clinical trials

BACKGROUND

The estimand for a clinical trial is a precise definition of the treatment effect to be estimated. Traditionally, estimates of treatment effects are based on either an ITT analysis or a per-protocol analysis. However, there are important clinical questions which are not addressed by either of these analyses. For example, consider a trial where patients take a rescue medication. The ITT analysis includes data after use of rescue, while the per-protocol analysis excludes these patients altogether. Neither of these analyses addresses the important question of what the treatment effect would have been if patients did not take rescue medication.

MAIN TEXT

Trial estimands provide a broader perspective compared to the limitations of ITT and per-protocol analysis. Trial treatment effects depend on how events occurring after treatment initiation such as use of alternative medication or discontinuation of the intervention are included in the definition. These events can be accounted for in different ways, depending on the clinical question of interest.

CONCLUSION

The estimand framework is an important step forward in improving the clarity and transparency of clinical trials. The centrality of estimands to clinical trials is currently not reflected in methods recommended by the Cochrane group or the CONSORT statement, the current standard for reporting clinical trials in medical journals. We encourage revisions to these guidelines.

Is Intention to Treat Still the Gold Standard or Should Health Technology Assessment Agencies Embrace a Broader Estimands Framework?: Insights and Perspectives From the National Institute for Health and Care Excellence and Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen on the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use E9 (R1) Addendum

OBJECTIVES

The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) E9 (R1) addendum will have an important impact on the design and analysis of randomized controlled clinical trials, which represent crucial sources of evidence in health technology assessments, and on the intention-to-treat (ITT) principle in particular. This article brings together a task force of health economists and statisticians in academic institutes and the pharmaceutical industry, to examine the implications of the addendum from the perspective of the National Institute for Health and Care Excellence (NICE) and the Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG) and to address the question of whether the ITT principle should be considered the gold standard for estimating treatment effects.

METHODS

We review the ITT principle, as introduced in the ICH E9 guideline. We then present an overview of the ICH E9 (R1) addendum and its estimand framework, highlighting its premise and the proposed strategies for handling intercurrent events, and examine some cases among submissions to IQWiG and NICE.

RESULTS

IQWiG and NICE appear to have diverging perspectives around the relevance of the ITT principle and, in particular, the acceptance of hypothetical strategies for estimating treatment effects, as suggested by examples where the sponsor proposed an alternative approach to the ITT principle when accounting for treatment switching for interventional oncology trials.

CONCLUSIONS

The ICH E9 (R1) addendum supports the use of methods that depart from the ITT principle. The relevance of estimands using these methods depends on the perspectives and objectives of payers. It is challenging to design a study that meets all stakeholders' research questions. Different estimands may serve to answer different relevant questions or decision problems.

Finerenone and effects on mortality in chronic kidney disease and type 2 diabetes: a FIDELITY analysis

AIMS

Finerenone reduces the risk of cardiovascular events in patients with chronic kidney disease (CKD) and type 2 diabetes (T2D). We investigated the causes of mortality in the FIDELITY population.

METHODS AND RESULTS

The FIDELITY prespecified pooled data analysis from FIDELIO-DKD and FIGARO-DKD excluded patients with heart failure and reduced ejection fraction. Outcomes included intention-to-treat and prespecified on-treatment analyses of the risk of all-cause and cardiovascular mortality. Of 13 026 patients [mean age, 64.8 years; mean estimated glomerular filtration rate (eGFR), 57.6 mL/min/1.73 m2], 99.8% were on renin-angiotensin system inhibitors. Finerenone reduced the incidence of all-cause and cardiovascular mortality vs. placebo (8.5% vs. 9.4% and 4.9% vs. 5.6%, respectively) and demonstrated significant on-treatment reductions [hazard ratio (HR), 0.82; 95% confidence interval (CI), 0.70-0.96; P = 0.014 and HR, 0.82; 95% CI, 0.67-0.99; P = 0.040, respectively]. Cardiovascular-related mortality was most common, and finerenone lowered the incidence of sudden cardiac death vs. placebo [1.3% (incidence rate 0.44/100 patient-years) vs. 1.8% (0.58/100 patient-years), respectively; HR, 0.75; 95% CI, 0.57-0.996; P = 0.046]. The effects of finerenone on mortality were similar across all Kidney Disease: Improving Global Outcomes risk groups. Event probability with finerenone at 4 years was consistent irrespective of baseline urine albumin-to-creatinine ratio, but seemingly more pronounced in patients with higher baseline eGFR.

CONCLUSION

In FIDELITY, finerenone significantly reduced the risk of all-cause and cardiovascular mortality vs. placebo in patients with T2D across a broad spectrum of CKD stages while on treatment, as well as sudden cardiac death in the intention-to-treat population.

CLINICAL TRIALS REGISTRATION

FIDELIO-DKD and FIGARO-DKD are registered with ClinicalTrials.gov, numbers NCT02540993 and NCT02545049, respectively (funded by Bayer AG).

Evaluating how clear the questions being investigated in randomised trials are: systematic review of estimands

OBJECTIVES

To evaluate how often the precise research question being addressed about an intervention (the estimand) is stated or can be determined from reported methods, and to identify what types of questions are being investigated in phase 2-4 randomised trials.

DESIGN

Systematic review of the clarity of research questions being investigated in randomised trials in 2020 in six leading general medical journals.

DATA SOURCE

PubMed search in February 2021.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Phase 2-4 randomised trials, with no restrictions on medical conditions or interventions. Cluster randomised, crossover, non-inferiority, and equivalence trials were excluded.

MAIN OUTCOME MEASURES

Number of trials that stated the precise primary question being addressed about an intervention (ie, the primary estimand), or for which the primary estimand could be determined unambiguously from the reported methods using statistical knowledge. Strategies used to handle post-randomisation events that affect the interpretation or existence of patient outcomes, such as intervention discontinuations or uses of additional drug treatments (known as intercurrent events), and the corresponding types of questions being investigated.

RESULTS

255 eligible randomised trials were identified. No trials clearly stated all the attributes of the estimand. In 117 (46%) of 255 trials, the primary estimand could be determined from the reported methods. Intercurrent events were reported in 242 (95%) of 255 trials; but the handling of these could only be determined in 125 (49%) of 255 trials. Most trials that provided this information considered the occurrence of intercurrent events as irrelevant in the calculation of the treatment effect and assessed the effect of the intervention regardless (96/125, 77%)-that is, they used a treatment policy strategy. Four (4%) of 99 trials with treatment non-adherence owing to adverse events estimated the treatment effect in a hypothetical setting (ie, the effect as if participants continued treatment despite adverse events), and 19 (79%) of 24 trials where some patients died estimated the treatment effect in a hypothetical setting (ie, the effect as if participants did not die).

CONCLUSIONS

The precise research question being investigated in most trials is unclear, mainly because of a lack of clarity on the approach to handling intercurrent events. Clear reporting of estimands is necessary in trial reports so that all stakeholders, including clinicians, patients and policy makers, can make fully informed decisions about medical interventions.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42021238053.

Marking 2-Years of New Thinking in Clinical Trials: The Estimand Journey

The ICH E9(R1) addendum on Estimands and Sensitivity Analyses in Clinical Trials has introduced a new estimand framework for the design, conduct, analysis, and interpretation of clinical trials. We share Pharmaceutical Industry experiences of implementing the estimand framework in the first two years since the final guidance became available with key lessons learned and highlight what else needs to be done to continue the journey in embedding the estimand framework in clinical trials. Emerging best practices and points to consider on strategies for implementing a new estimand thinking process are provided. Whilst much of the focus of implementing ICH E9(R1) to date has been on defining estimands, we highlight some of the important aspects relating to the choice of statistical analysis methods and sensitivity analyses to ensure estimands can be estimated robustly with minimal bias. In particular, we discuss the implications if complete follow-up is not possible when the treatment policy strategy is being used to handle intercurrent events. ICH E9(R1) was introduced just before the start of the COVID-19 pandemic, but a positive outcome from the pandemic has been an acceleration in the adoption of the estimand framework, including differentiating intercurrent events related or not related to the pandemic. In summary, much has been learned on the estimand journey and continued sharing of case studies will help to further advance the understanding and increase awareness across all clinical researchers of the estimand framework.

A likely responder approach for the analysis of randomized controlled trials

OBJECTIVE

To further the precision medicine goal of tailoring medical treatment to individual patient characteristics by providing a method of analysis of the effect of test treatment, T, compared to a reference treatment, R, in participants in a RCT who are likely responders to T.

METHODS

Likely responders to T are individuals whose expected response at baseline exceeds a prespecified minimum. A prognostic score, the expected response predicted as a function of baseline covariates, is obtained at trial completion. It is a balancing score that can be used to match likely responders randomized to T with those randomized to R; the result is comparable treatment groups that have a common covariance distribution. Treatments are compared based on observed outcomes in this enriched sample. The approach is illustrated in a RCT comparing two treatments for opioid use disorder.

RESULTS

A standard statistical analysis of the opioid use disorder RCT found no treatment difference in the total sample. However, a subset of likely responders to T were identified and in this group, T was statistically superior to R.

CONCLUSION

The causal treatment effect of T relative to R among likely responders may be more important than the effect in the whole target population. The prognostic score function provides quantitative information to support patient specific treatment decisions regarding T furthering the goal of precision medicine.

The therapeutic effect of Taijiquan combined with acupoint pressing on the treatment of anxiety insomnia in college students: A study protocol for a randomized controlled trial

INTRODUCTION

Sleep health is an important part of health and has become a common concern of society. For anxiety insomnia, the commonly used clinical therapies have limitations. Alternative and complementary therapy is gradually rising and showing remarkable effect in clinical practice. This is the first study to evaluate the therapeutic effect of Taijiquan combined with acupoint pressing in the treatment of anxiety insomnia in college students and to compare the difference in intervention before and after sleep, to choose the best treatment time.

METHODS AND ANALYSIS

This is a multicenter, single-blind, randomized controlled trial. A total of 126 eligible subjects who have passed the psychological evaluation and met inclusion criteria by completing a psychometric scale will be randomly divided into treatment group A (treat before sleep), treatment group B (treat after sleep) and control group C (waiting list group) in a ratio of 1:1:1. All the three groups will receive regular psychological counseling during the trial, and the treatment groups will practice 24-style Taijiquan and do meridian acupuncture at Baihui (DU20), Shenting (DU24), Yintang (EX-HN3), Shenmen (HT7) and Sanyinjiao (SP6). This RCT includes a 2-week baseline period, a 12-week intervention period, and a 12-week follow-up period. The main results will be measured by changes in the Pittsburgh sleep quality index (PSQI) and Hamilton anxiety scale (HAMA). The secondary results will be measured by the generalized anxiety scale (GAD-7) and insomnia severity index (ISI). The safety of the intervention will be evaluated at each assessment. The statistical analysis of data will be carried out by SPSSV.26.0 software.

DISCUSSION

We expect this trial to explore the effectiveness of Taijiquan combined with acupoint pressing in the treatment of anxiety insomnia in college students and choose the best treatment time by comparison.

CLINICAL TRIAL REGISTRATION

[www.ClinicalTrials.gov], identifier [ChiCTR2200057003].

Estimands-a Basic Element for Clinical Trials. Part 29 of a Series on Evaluation of Scientific Publications

BACKGROUND

Clinical trials are of central importance for the evaluation and comparison of treatments. The transparency and intelligibility of the treatment effect under investigation is an essential matter for physicians, patients, and health-care authorities. The estimand framework has been introduced because many trials are deficient in this respect.

METHODS

Introduction, definition, and application of the estimand framework on the basis of an example and a selective review of the literature.

RESULTS

The estimand framework provides a systematic approach to the definition of the treatment effect under investigation in a clinical trial. An estimand consists of five attributes: treatment, population, variable, population-level summary, and handling of intercurrent events. Each of these attributes is defined in an interdisciplinary discussion during the trial planning phase, based on the clinical question being asked. Special attention is given to the handling of intercurrent events (ICEs): these are events-e.g., discontinuation or modification of treatment or the use of emergency medication-that can occur once the treatment has begun and might affect the possibility of observing the endpoints or their interpretability. There are various strategies for the handling of ICEs; these can, for example, also reflect the existing intentionto- treat (ITT) principle. Per-protocol analyses, in contrast, are prone to bias and cannot be represented in a sensible manner by an estimand, although they may be performed as a supplementary analysis. The discussion of potential intercurrent events and how they should appropriately be handled in view of the aim of the trial must already take place in the planning phase.

CONCLUSION

Use of the estimand framework should make it easier for both physicians and patients to understand what trials reveal about the efficacy of treatment, and to compare the results of different trials.