Improved two-stage estimation to adjust for treatment switching in randomised trials: g-estimation to address time-dependent confounding

Challenges and Opportunities in Interdisciplinary Research and Real-World Data for Treatment Sequences in Health Technology Assessments

With an ever-increasing number of treatment options, the assessment of treatment sequences has become crucial in health technology assessment (HTA). This review systematically explores the multifaceted challenges inherent in evaluating sequences, delving into their interplay and nuances that go beyond economic model structures. We synthesised a 'roadmap' of literature from key methodological studies, highlighting the evolution of recent advances and emerging research themes. These insights were compared against HTA guidelines to identify potential avenues for future research. Our findings reveal a spectrum of challenges in sequence evaluation, encompassing selecting appropriate decision-analytic modelling approaches and comparators, deriving appropriate clinical effectiveness evidence in the face of data scarcity, scrutinising effectiveness assumptions and statistical adjustments, considering treatment displacement, and optimising model computations. Integrating methodologies from diverse disciplines-statistics, epidemiology, causal inference, operational research and computer science-has demonstrated promise in addressing these challenges. An updated review of application studies is warranted to provide detailed insights into the extent and manner in which these methodologies have been implemented. Data scarcity on the effectiveness of treatment sequences emerged as a dominant concern, especially because treatment sequences are rarely compared in clinical trials. Real-world data (RWD) provide an alternative means for capturing evidence on effectiveness and future research should prioritise harnessing causal inference methods, particularly Target Trial Emulation, to evaluate treatment sequence effectiveness using RWD. This approach is also adaptable for analysing trials harbouring sequencing information and adjusting indirect comparisons when collating evidence from heterogeneous sources. Such investigative efforts could lend support to reviews of HTA recommendations and contribute to synthesising external control arms involving treatment sequences.

A cautionary tale: an evaluation of the performance of treatment switching adjustment methods in a real world case study

BACKGROUND

Treatment switching in randomised controlled trials (RCTs) is a problem for health technology assessment when substantial proportions of patients switch onto effective treatments that would not be available in standard clinical practice. Often statistical methods are used to adjust for switching: these can be applied in different ways, and performance has been assessed in simulation studies, but not in real-world case studies. We assessed the performance of adjustment methods described in National Institute for Health and Care Excellence Decision Support Unit Technical Support Document 16, applying them to an RCT comparing panitumumab to best supportive care (BSC) in colorectal cancer, in which 76% of patients randomised to BSC switched onto panitumumab. The RCT resulted in intention-to-treat hazard ratios (HR) for overall survival (OS) of 1.00 (95% confidence interval [CI] 0.82-1.22) for all patients, and 0.99 (95% CI 0.75-1.29) for patients with wild-type KRAS (Kirsten rat sarcoma virus).

METHODS

We tested several applications of inverse probability of censoring weights (IPCW), rank preserving structural failure time models (RPSFTM) and simple and complex two-stage estimation (TSE) to estimate treatment effects that would have been observed if BSC patients had not switched onto panitumumab. To assess the performance of these analyses we ascertained the true effectiveness of panitumumab based on: (i) subsequent RCTs of panitumumab that disallowed treatment switching; (ii) studies of cetuximab that disallowed treatment switching, (iii) analyses demonstrating that only patients with wild-type KRAS benefit from panitumumab. These sources suggest the true OS HR for panitumumab is 0.76-0.77 (95% CI 0.60-0.98) for all patients, and 0.55-0.73 (95% CI 0.41-0.93) for patients with wild-type KRAS.

RESULTS

Some applications of IPCW and TSE provided treatment effect estimates that closely matched the point-estimates and CIs of the expected truths. However, other applications produced estimates towards the boundaries of the expected truths, with some TSE applications producing estimates that lay outside the expected true confidence intervals. The RPSFTM performed relatively poorly, with all applications providing treatment effect estimates close to 1, often with extremely wide confidence intervals.

CONCLUSIONS

Adjustment analyses may provide unreliable results. How each method is applied must be scrutinised to assess reliability.

Nature-based social interventions to address loneliness among vulnerable populations: a common study protocol for three related randomized controlled trials in Barcelona, Helsinki, and Prague within the RECETAS European project

BACKGROUND

The negative effects of loneliness on population health and wellbeing requires interventions that transcend the medical system and leverage social, cultural, and public health system resources. Group-based social interventions are a potential method to alleviate loneliness. Moreover, nature, as part of our social and health infrastructure, may be an important part of the solutions that are needed to address loneliness. The RECETAS European project H2020 (Re-imagining Environments for Connection and Engagement: Testing Actions for Social Prescribing in Natural Spaces) is an international research project aiming to develop and test the effectiveness of nature-based social interventions to reduce loneliness and increase health-related quality of life.

METHODS

This article describes the three related randomized controlled trials (RCTs) that will be implemented: the RECETAS-BCN Trial in Barcelona (Spain) is targeting people 18+ from low socio-economic urban areas; the RECETAS-PRG Trial in Prague (Czech Republic) is addressing community-dwelling older adults over 60 years of age, and the RECETAS-HLSNK trial is reaching older people in assisted living facilities. Each trial will recruit 316 adults suffering from loneliness at least sometimes and randomize them to nature-based social interventions called "Friends in Nature" or to the control group. "Friends in Nature" uses modifications of the "Circle of Friends" methodology based on group processes of peer support and empowerment but including activities in nature. Participants will be assessed at baseline, at post-intervention (3 months), and at 6- and 12-month follow-up after baseline. Primary outcomes are the health-related quality-of-life according to 15D measure and The De Jong Gierveld 11-item loneliness scale. Secondary outcomes are health and psychosocial variables tailored to the specific target population. Nature exposure will be collected throughout the intervention period. Process evaluation will explore context, implementation, and mechanism of impact. Additionally, health economic evaluations will be performed.

DISCUSSION

The three RECETAS trials will explore the effectiveness of nature-based social interventions among lonely people from various ages, social, economic, and cultural backgrounds. RECETAS meets the growing need of solid evidence for programs addressing loneliness by harnessing the beneficial impact of nature on enhancing wellbeing and social connections.

TRIAL REGISTRATION

Barcelona (Spain) trial: ClinicalTrials.gov, ID: NCT05488496. Registered 29 July 2022. Prague (Czech Republic) trial: ClinicalTrials.gov, ID: NCT05522140. Registered August 25, 2022. Helsinki (Finland) trial: ClinicalTrials.gov, ID: NCT05507684. Registered August 12, 2022.

Estimation of treatment effect in presence of noncompliance and competing risks: a simulation study

A randomized controlled trial is commonly designed to assess the treatment effect in survival studies, in which patients are randomly assigned to the standard or the experimental treatment group. Upon disease progression, patients who have been randomized to standard treatment are allowed to switch to the experimental treatment. Treatment switching in a randomized controlled trial refers to a situation in which patients switch from their randomized treatment to another treatment. Often, the switchis from the control group to the experimental treatment. In this case, the treatment effect estimate is adjusted using either convenient naive methods such as intention-to-treat, per-protocol or advanced methods such as rank preserving structural failure time (RPSFT) models. In previous simulation studies performed so far, there was only one possible outcome for patients. However, in oncology in particular, multiple outcomes are potentially possible. These outcomes are called competing risks. This aspect has not been considered in previous studies when determining the effect of a treatment in the presence of noncompliance. This study aimed to extend the RPSFT method using a two-dimensional G-estimation in the presence of competing risks. The RPSFT method was extended for two events, the event of interest and the competing event. For this purpose, the RPSFT method was applied based on the cause-specific hazard approach, the result of which is compared to the naive methods used in simulation studies. The results show that the proposed method has a good performance compared to other methods.

Causal evidence in health decision making: methodological approaches of causal inference and health decision science

Objectives

Public health decision making is a complex process based on thorough and comprehensive health technology assessments involving the comparison of different strategies, values and tradeoffs under uncertainty. This process must be based on best available evidence and plausible assumptions. Causal inference and health decision science are two methodological approaches providing information to help guide decision making in health care. Both approaches are quantitative methods that use statistical and modeling techniques and simplifying assumptions to mimic the complexity of the real world. We intend to review and lay out both disciplines with their aims, strengths and limitations based on a combination of textbook knowledge and expert experience.

Methods

To help understanding and differentiating the methodological approaches of causal inference and health decision science, we reviewed both methods with the focus on aims, research questions, methods, assumptions, limitations and challenges, and software. For each methodological approach, we established a group of four experts from our own working group to carefully review and summarize each method, followed by structured discussion rounds and written reviews, in which the experts from all disciplines including HTA and medicine were involved. The entire expert group discussed objectives, strengths and limitations of both methodological areas, and potential synergies. Finally, we derived recommendations for further research and provide a brief outlook on future trends.

Results

Causal inference methods aim for drawing causal conclusions from empirical data on the relationship of pre-specified interventions on a specific target outcome and apply a counterfactual framework and statistical techniques to derive causal effects of exposures or interventions from these data. Causal inference is based on a causal diagram, more specifically, a directed acyclic graph (DAG), which encodes the assumptions regarding the causal relations between variables. Depending on the type of confounding and selection bias, traditional statistical methods or more complex g-methods are needed to derive valid causal effects. Besides the correct specification of the DAG and the statistical model, assumptions such as consistency, positivity, and exchangeability must be checked when aiming at causal inference. Health decision science aims for guiding policy decision making regarding health interventions considering and balancing multiple competing objectives of a decision based on data from multiple sources and studies, for example prevalence studies, clinical trials and long-term observational routine effectiveness studies, and studies on preferences and costs. It involves decision analysis, a systematic, explicit and quantitative framework to guide decisions under uncertainty. Decision analyses are based on decision-analytic models to mimic the course of disease as well as aspects and consequences of the intervention in order to quantitatively optimize the decision. Depending on the type of decision problem, decision trees, state-transition models, discrete event simulation models, dynamic transmission models, or other model types are applied. Models must be validated against observed data, and comprehensive sensitivity analyses must be performed to assess uncertainty. Besides the appropriate choice of the model type and the valid specification of the model structure, it must be checked if input parameters of effects can be interpreted as causal parameters in the model. Otherwise results will be biased.

Conclusions

Both causal inference and health decision science aim for providing best causal evidence for informed health decision making. The strengths and limitations of both methods differ and a good understanding of both methods is essential for correct application but also for correct interpretation of findings from the described methods. Importantly, decision-analytic modeling should be combined with causal inference when developing guidance and recommendations regarding decisions on health care interventions.

Causal analyses with target trial emulation for real-world evidence removed large self-inflicted biases: systematic bias assessment of ovarian cancer treatment effectiveness

BACKGROUND AND OBJECTIVES

Drawing causal conclusions from real-world data (RWD) poses methodological challenges and risk of bias. We aimed to systematically assess the type and impact of potential biases that may occur when analyzing RWD using the case of progressive ovarian cancer.

METHODS

We retrospectively compared overall survival with and without second-line chemotherapy (LOT2) using electronic medical records. Potential biases were determined using directed acyclic graphs. We followed a stepwise analytic approach ranging from crude analysis and multivariable-adjusted Cox model up to a full causal analysis using a marginal structural Cox model with replicates emulating a reference randomized controlled trial (RCT). To assess biases, we compared effect estimates (hazard ratios [HRs]) of each approach to the HR of the reference trial.

RESULTS

The reference trial showed an HR for second line vs. delayed therapy of 1.01 (95% confidence interval [95% CI]: 0.82-1.25). The corresponding HRs from the RWD analysis ranged from 0.51 for simple baseline adjustments to 1.41 (95% CI: 1.22-1.64) accounting for immortal time bias with time-varying covariates. Causal trial emulation yielded an HR of 1.12 (95% CI: 0.96-1.28).

CONCLUSION

Our study, using ovarian cancer as an example, shows the importance of a thorough causal design and analysis if one is expecting RWD to emulate clinical trial results.

Synthesizing studies for comparing different treatment sequences in clinical trials

With advances in cancer treatments and improved patient survival, more patients may go through multiple lines of treatment. It is of clinical importance to choose a sequence of effective treatments (eg, lines of treatment) for individual patients with the goal of optimizing their long-term clinical outcome (eg, survival). Several important issues arise in cancer studies. First, cancer clinical trials are usually conducted by each line of treatment. For a treatment sequence, we may have first line and second line treatment data from two different studies. Second, there is typically a treatment initiation period varying from patient to patient between progression of disease and the start of the second line treatment due to administrative reasons. Additionally, the choice of the second line treatment for patients with progression of disease may depend on their characteristics. We address all these issues and develop semiparametric methods under the potential outcome framework for the estimation of the overall survival probability for a treatment sequence and for comparing different treatment sequences. We establish the large sample properties of the proposed inferential procedures. Simulation studies and an application to a colorectal clinical trial are provided.

Estimating Treatment-Switching Bias in a Randomized Clinical Trial of Ovarian Cancer Treatment: Combining Causal Inference with Decision-Analytic Modeling

BACKGROUND

Bevacizumab is efficacious in delaying ovarian cancer progression and controlling ascites. The ICON7 trial showed a significant benefit in overall survival for bevacizumab, whereas the GOG-218 trial did not. GOG-218 allowed control group patients to switch to bevacizumab upon progression, which may have biased the results. Lack of data on switching behavior prevented the application of g-methods to adjust for switching. The objective of this study was to apply decision-analytic modeling to estimate the impact of switching bias on causal treatment-effect estimates.

METHODS

We developed a causal decision-analytic Markov model (CDAMM) to emulate the GOG-218 trial and estimate overall survival. CDAMM input parameters were based on data from randomized clinical trials and the published literature. Overall switching proportion was based on GOG-218 trial information, whereas the proportion switching with and without ascites was estimated using calibration. We estimated the counterfactual treatment effect that would have been observed had no switching occurred by denying switching in the CDAMM.

RESULTS

The survival curves generated by the CDAMM matched well with the ones reported in the GOG-218 trial. The survival curve correcting for switching showed an estimated bias such that 79% of the true treatment effect could not be observed in the GOG-218 trial. Results were most sensitive to changes in the proportion progressing with severe ascites and mortality.

LIMITATIONS

We used a simplified model structure and based model parameters on published data and assumptions. Robustness of the CDAMM was tested and model assumptions transparently reported.

CONCLUSIONS

Medical-decision science methods may be merged with empirical methods of causal inference to integrate data from other sources where empirical data are not sufficient. We recommend collecting sufficient information on switching behavior when switching cannot be avoided.

Estimands in published protocols of randomised trials: urgent improvement needed

BACKGROUND

An estimand is a precise description of the treatment effect to be estimated from a trial (the question) and is distinct from the methods of statistical analysis (how the question is to be answered). The potential use of estimands to improve trial research and reporting has been underpinned by the recent publication of the ICH E9(R1) Addendum on the use of estimands in clinical trials in 2019. We set out to assess how well estimands are described in published trial protocols.

METHODS

We reviewed 50 trial protocols published in October 2020 in Trials and BMJ Open. For each protocol, we determined whether the estimand for the primary outcome was explicitly stated, not stated but inferable (i.e. could be constructed from the information given), or not inferable.

RESULTS

None of the 50 trials explicitly described the estimand for the primary outcome, and in 74% of trials, it was impossible to infer the estimand from the information included in the protocol. The population attribute of the estimand could not be inferred in 36% of trials, the treatment condition attribute in 20%, the population-level summary measure in 34%, and the handling of intercurrent events in 60% (the strategy for handling non-adherence was not inferable in 32% of protocols, and the strategy for handling mortality was not inferable in 80% of the protocols for which it was applicable). Conversely, the outcome attribute was stated for all trials. In 28% of trials, three or more of the five estimand attributes could not be inferred.

CONCLUSIONS

The description of estimands in published trial protocols is poor, and in most trials, it is impossible to understand exactly what treatment effect is being estimated. Given the utility of estimands to improve clinical research and reporting, this urgently needs to change.

Estimands for overall survival in clinical trials with treatment switching in oncology

An addendum of the ICH E9 guideline on Statistical Principles for Clinical Trials was released in November 2019 introducing the estimand framework. This new framework aims to align trial objectives and statistical analyses by requiring a precise definition of the inferential quantity of interest, that is, the estimand. This definition explicitly accounts for intercurrent events, such as switching to new anticancer therapies for the analysis of overall survival (OS), the gold standard in oncology. Traditionally, OS in confirmatory studies is analyzed using the intention-to-treat (ITT) approach comparing treatment groups as they were initially randomized regardless of whether treatment switching occurred and regardless of any subsequent therapy (treatment-policy strategy). Regulatory authorities and other stakeholders often consider ITT results as most relevant. However, the respective estimand only yields a clinically meaningful comparison of two treatment arms if subsequent therapies are already approved and reflect clinical practice. We illustrate different scenarios where subsequent therapies are not yet approved drugs and thus do not reflect clinical practice. In such situations the hypothetical strategy could be more meaningful from patient's and prescriber's perspective. The cross-industry Oncology Estimand Working Group (www.oncoestimand.org) was initiated to foster a common understanding and consistent implementation of the estimand framework in oncology clinical trials. This paper summarizes the group's recommendations for appropriate estimands in the presence of treatment switching, one of the key intercurrent events in oncology clinical trials. We also discuss how different choices of estimands may impact study design, data collection, trial conduct, analysis, and interpretation.

Exploring the Impact of Treatment Switching on Overall Survival from the PROfound Study in Homologous Recombination Repair (HRR)-Mutated Metastatic Castration-Resistant Prostate Cancer (mCRPC)

Background

In oncology trials, treatment switching from the comparator to the experimental regimen is often allowed but may lead to underestimating overall survival (OS) of an experimental therapy.

Objective

This study evaluates the impact of treatment switching from control to olaparib on OS using the final survival data from the PROfound study and compares validated adjustment methods to estimate the magnitude of OS benefit with olaparib.

Patients and methods

The primary population from PROfound (Cohort A) was included, alongside two populations approved for treatment with olaparib by the European Medicines Agency and US Food and Drug Administration: BRCAm and Cohort A+B (excluding the PPP2R2A gene). Five methods were explored to adjust for switching: excluding or censoring patients in the control arm who receive subsequent olaparib, Rank Preserving Structural Failure Time Model (RPSFTM), Inverse Probability of Censoring Weights, and Two-Stage Estimation.

Results

The RPSFTM was considered the most appropriate approach for PROfound as the results were robust to sensitivity analysis testing of the common treatment effect assumption. For Cohort A, the final OS hazard ratio reduced from 0.69 (95% CI 0.5–0.97) to between 0.42 (0.18–0.90) and 0.52 (0.31–1.00) for olaparib versus control, depending on the RPSFTM selected. Median OS reduced from 14.7 months to between 11.73 and 12.63 months for control.

Conclusions

The magnitude of the statistically significant (P < 0.05) survival benefit of olaparib versus control observed in Cohort A of PROfound is likely to be underestimated if adjustment for treatment switching from control to olaparib is not conducted. The RPSFTM was considered the most plausible method, although further development and validation of robust methods to estimate the magnitude of impact of treatment switching is needed.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11523-021-00837-y.