Patients and investigators prefer measures of absolute risk in subgroups for pragmatic randomized trials

Using the counterfactual framework to estimate non-intention-to-treat estimands in randomised controlled trials: A methodological scoping review

BACKGROUND

Randomised controlled trials (RCTs) commonly estimate intention-to-treat (ITT) estimands. However, when nonadherence to assigned treatment occurs, ITT estimands reflect the effect of being offered treatment, rather than adhering to it and thus are less useful for clinical decision-making.

OBJECTIVE

Summarise current literature on non-ITT estimands used in RCTs to estimate the effect of adhering to treatment and counterfactual framework estimators that are employed to obtain these non-ITT estimands.

STUDY DESIGN AND SETTING

We conducted a methodological scoping review and searched MEDLINE and EMBASE from inception to March 2024, with forward and backward citation searching. Eligible records discussed counterfactual framework estimators to obtain non-ITT estimands in RCTs or simulation studies based on empirical RCTs.

RESULTS

From 746 records screened, our search identified 56 eligible records. 47 (84 %) described specific estimators for addressing nonadherence and 9 (16 %) described frameworks for overall methodological approach. In the 47 estimator records, 51 non-ITT estimands were reported, including poorly defined estimands (n = 21, 41 %), complier average causal effects (n = 17, 33 %), switching-adjusted estimands (n = 7, 14 %), and per-protocol estimands (n = 6, 12 %). There were 83 estimator applications, including inverse probability weighting (n = 22, 27 %), instrumental variables (IVs) for time-varying treatments (n = 15, 18 %), and standard IVs (n = 14, 17 %). Other estimators included doubly-robust estimators using machine learning.

CONCLUSION

Non-ITT estimands in RCTs tended to be poorly defined and lacked relevance for clinical decision-making. Further research on using estimators from the counterfactual framework to estimate well-defined estimands, particularly per-protocol estimands, is needed to support greater uptake in practice and policy decision making.

Separable effects for adherence

Comparing different medications is complicated when adherence to these medications differs. We can overcome the adherence issue by assessing effectiveness under sustained use, as in usual causal "per-protocol" estimands. However, when sustained use is challenging to satisfy in practice, the usefulness of these estimands can be limited. Here we propose a different class of estimands: separable effects for adherence. These estimands compare modified medications, holding fixed a component responsible for nonadherence. Under assumptions about treatment components' mechanisms of effect, a separable effects estimand can quantify the effectiveness of medication initiation strategies on an outcome of interest under the adherence mechanism of one of the medications. These assumptions are amenable to interrogation by subject-matter experts and can be evaluated using causal graphs. We describe an algorithm for constructing causal graphs for separable effects, illustrate how these graphs can be used to reason about assumptions required for identification, and provide semi-parametric weighted estimators. This article is part of a Special Collection on Pharmacoepidemiology.

Real-world effectiveness and cardiovascular safety of abiraterone versus enzalutamide amongst older patients diagnosed with metastatic castration-resistant prostate cancer

INTRODUCTION

Abiraterone and enzalutamide are both approved in the United States for the treatment of metastatic castration-resistant prostate cancer (mCRPC). The objective of this study was to compare the real-world effectiveness and cardiovascular safety of these agents, drawing from a cohort of older adult patients diagnosed with mCRPC.

MATERIALS AND METHODS

The Surveillance, Epidemiology, and End Results-Medicare database was used to conduct an observational study comparing three-year overall survival and one-year risk of major adverse cardiovascular events (MACE) between initiators of abiraterone or enzalutamide between September 2012 and June 2017. Inverse-probability-of-treatment weighting was used to balance measured confounders. MACE was defined as a hospitalization for myocardial infarction, heart failure, or ischemic event (stroke or transient attack). Results were additionally stratified by levels of a claims frailty index (robust, prefrail, frail) and the presence of baseline cardiovascular comorbidities.

RESULTS

The study population consisted of 4622 male adults 66 years of age and older diagnosed with mCRPC, of which 2430 initiated abiraterone and 2192 enzalutamide. The adjusted three-year overall survival was lower in patients initiating abiraterone (27.9 %) than enzalutamide (31.5 %) (adjusted survival difference [aSD] = -3.6 %, 95 % CI: -6.2 %, -0.9 %). In frailty-stratified analysis, no survival difference was found for the robust (aSD = 0.6 %, 95 % CI: -5.0 %, 6.3 %) or frail (aSD = -1.2 %, 95 % CI: -6.1 %, 3.7 %) subgroups, but there was lower survival with abiraterone for the prefrail group (aSD = -5.9 %, 95 % CI: -9.6, -2.3). The adjusted one-year risk of MACE was higher in abiraterone initiators (5.5 %) than enzalutamide initiators (3.6 %) (adjusted risk difference [aRD] = 1.8 %, 95 % CI: 0.6 %, 3.1 %); the increase was significant in the frail (aRD = 4.8 %, 95 % CI = 1.4 %, 8.3 %) and pre-frail subgroups (aRD =1.9 %, 95 % CI: 0.1 %, 3.6 %) but not the robust subgroup (aRD = -0.3 %, 95 % CI: -1.8 %, 1.2 %).

DISCUSSION

The three-year survival of abiraterone initiators was slightly lower than that of enzalutamide initiators, though the agents showed similar survival for patients with robust fitness. A one-year increase in MACE risk was observed in abiraterone initiators, especially amongst frail individuals, highlighting the importance of assessing frailty during therapy selection.

Moving beyond the prevalent exposure design for causal inference in dementia research

As randomised trials are not always feasible or practical, observational studies remain crucial for addressing many causal questions in the dementia prevention field. Through a systematic search, we found that 84 (72%) of the 116 latest observational cohort studies that investigated factors hypothesised to reduce the risk of dementia (hearing aids, physical activity, antihypertensives, antidiabetics, and antidepressants) used a prevalent exposure design. The approach of using a prevalent exposure design, which compares dementia risk between individuals with and without prevalent exposure at the start of follow-up, has several limitations, including ill-defined interventions, selection biases, and challenges in adjusting for confounders. This Personal View discusses these limitations using physical activity as a case study and describes an alternative approach based on the target trial framework that can help to overcome such limitations. This approach aligns observational analyses with the design and analysis principles of randomised trials and can, thereby, improve the robustness and relevance of evidence for dementia prevention, which is the ultimate goal.

Causal Inference in Oncology: Why, What, How and When

Oncologists are faced with choosing the best treatment for each patient, based on the available evidence from randomized controlled trials (RCTs) and observational studies. RCTs provide estimates of the average effects of treatments on groups of patients, but they may not apply in many real-world scenarios where for example patients have different characteristics than the RCT participants, or where different treatment variants are considered. Causal inference defines what a treatment effect is and how it may be estimated with RCTs or outside of RCTs with observational - or 'real-world' - data. In this review, we introduce the field of causal inference, explain what a treatment effect is and what important challenges are with treatment effect estimation with observational data. We then provide a framework for conducting causal inference studies and describe when in oncology causal inference from observational data may be particularly valuable. Recognizing the strengths and limitations of both RCTs and observational causal inference provides a way for more informed and individualized treatment decision-making in oncology.

Clinical Effectiveness and Cost-Effectiveness of Multigene Panel Sequencing in Advanced Melanoma: A Population-Level Real-World Target Trial Emulation

PURPOSE

Targeted therapy and immunotherapy promise improved survival in patients with advanced melanoma, yet the effectiveness and cost-effectiveness of multigene panel sequencing compared with single-gene BRAF testing to guide therapeutic decisions is unknown.

METHODS

Our population-based quasi-experimental retrospective target trial emulation used comprehensive patient-level data for 364 British Columbia, Canada, adults with an advanced melanoma diagnosis receiving multigene panel sequencing or single-gene BRAF testing between September 1, 2016, and December 31, 2018. We 1:1 matched multigene panel patients to controls using genetic algorithm-based matching. Outcomes included 3-year overall survival (OS) and health care costs (2021 Canadian dollars [CAD]) with incremental net monetary benefit for life-years gained (LYG). Outcomes were analyzed using inverse probability of censoring weighted linear regression for the intention-to-treat (ITT) effect. The per-protocol (PP) effect estimation also included stabilized inverse probability of treatment weights. We then used Weibull regression and Kaplan-Meier survival analysis.

RESULTS

We matched 147 multigene panel patients to controls, achieving balance for all covariates. After matching, ITT incremental costs were $19,447 CAD (95% CI, -$18,516 to $76,006) and incremental LYG were 0.22 (95% CI, -0.05 to 0.49). We found uncertainty in differences on OS using Kaplan-Meier (P = .11) and Weibull regression (hazard ratio [HR], 0.73 [95% CI, 0.51 to 1.03]) in the ITT. PP incremental costs were $36,367 CAD (95% CI, -$6,653 to $120,216]) and incremental LYG were 0.56 (95% CI, 0.39 to 1.24), with corresponding differences in OS using Kaplan-Meier (P = .02) and Weibull regression (HR, 0.56 [95% CI, 0.36 to 0.87]). The probability of multigene panels being cost-effective at $100,000/LYG CAD was 55% for ITT and 65% for PP.

CONCLUSION

The cost-effectiveness of multigene panels was evenly poised at higher thresholds, even when accounting for treatment initiation. Health systems reimbursing multigene panels and expensive therapies may be confronted with a value tradeoff, in which there may be improved survival albeit with a modest change in cost-effectiveness.

De-Mystifying the Clone-Censor-Weight Method for Causal Research Using Observational Data: A Primer for Cancer Researchers

BACKGROUND

Regulators and oncology healthcare providers are increasingly interested in using observational studies of real-world data (RWD) to complement clinical evidence from randomized controlled trials for informed decision-making. To generate valid evidence, RWD studies must be carefully designed to avoid systematic biases. The clone-censor-weight (CCW) method has been proposed to address immortal time and other time-related biases.

METHODS

The objective of this manuscript is to de-mystify the CCW method for cancer researchers by describing and presenting its core components in an accessible and digestible format, using visualizations and examples from cancer-relevant studies. The CCW method has been applied in diverse settings, including investigations of the effects of surgery within a certain time after cancer diagnosis, the continuation of annual screening mammography, and chemotherapy duration on survival.

RESULTS

The method handles complex data wherein the treatment group to which an individual belongs is unknown at the start of follow-up. The three steps of the CCW method involve cloning or duplicating the patient population and assigning one clone to each treatment strategy, artificially censoring the clones when their observed data are inconsistent with the assigned strategy and weighting the cloned and censored population to address selection bias created by the artificial censoring.

CONCLUSIONS

The CCW method is a powerful tool for designing RWD studies in cancer that are free from time-related biases and successfully, to the extent possible, emulate features of a randomized clinical trial.

Comments, suggestions, and criticisms of the Pragmatic Explanatory Continuum Indicator Summary-2 design tool: a citation analysis

INTRODUCTION

The pragmatic explanatory continuum indicator summary (PRECIS) tool, initially published in 2009 and revised in 2015, was created to assist trialists to align their design choices with the intended purpose of their randomised controlled trial (RCT): either to guide real-world decisions between alternative interventions (pragmatic) or to test hypotheses about intervention mechanisms by minimising sources of variation (explanatory). There have been many comments, suggestions, and criticisms of PRECIS-2. This summary will be used to facilitate the development of to the next revision, which is PRECIS-3.

METHODS

We used Web of Science to identify all publication types citing PRECIS-2, published between May 2015 and July 2023. Citations were eligible if they contained 'substantive' suggestions, comments, or criticism of the PRECIS-2 tool. We defined 'substantive' as comments explicitly referencing at least one PRECIS-2 domain or a concept directly linked to an existing or newly proposed domain. Two reviewers independently extracted comments, suggestions, and criticisms, noting their implications for the update. These were discussed among authors to achieve consensus on the interpretation of each comment and its implications for PRECIS-3.

RESULTS

The search yielded 885 publications, and after full-text review, 89 articles met the inclusion criteria. Comments pertained to new domains, changes in existing domains, or were relevant across several or all domains. Proposed new domains included assessment of the comparator arm and a domain to describe blinding. There were concerns about scoring eligibility and recruitment domains for cluster trials. Suggested areas for improvement across domains included the need for more scoring guidance for explanatory design choices.

DISCUSSION

Published comments recognise PRECIS-2's success in aiding trialists with pragmatic or explanatory design choices. Enhancing its implementation and widespread use will involve adding new domains, refining domain definitions, and addressing overall tool issues. This citation review offers valuable user feedback, pivotal for shaping the upcoming version of the PRECIS tool, PRECIS-3.

Glucagon-like Peptide 1 Receptor Agonists and Asthma Exacerbations: Which Patients Benefit Most?

Rationale: Although recent evidence suggested that glucagon-like peptide 1 receptor agonists (GLP1RAs) might reduce the risk of asthma exacerbations, it remains unclear which subpopulations might derive the most benefit from GLP1RA treatment. Objectives: To identify characteristics of patients with asthma that predict who might benefit the most from GLP1RA treatment using real-world data. Methods: We implemented an active-comparator, new-user design analysis using commercially ensured patients 18-65 years of age from MarketScan data for 2007-2019 and identified two cohorts: GLP1RAs versus thiazolidinediones and GLP1RAs versus sulfonylureas. The outcome was acute exacerbation of asthma (hospital admission or emergency department visit for asthma) within 180 days after initiation. We applied iterative causal forest, a novel causal machine learning subgrouping algorithm, to assess heterogeneous treatment effects. In identified subgroups, we predicted propensity score, conducted propensity score trimming, and then estimated adjusted risk differences for the effect of GLP1RAs relative to comparators on asthma exacerbation using inverse probability treatment weighting in the propensity score-trimmed subpopulation. Results: Among 10,989 patients initiating GLP1RAs or thiazolidinediones and 17,088 patients initiating GLP1RAs versus sulfonylurea, GLP1RA initiators had fewer exacerbations, with adjusted risk differences of -0.5% (95% confidence interval [CI], -1.1% to 0.1%) and -1.6% (95% CI, -2.2% to -1.1%), respectively. In the GLP1RA versus sulfonylurea cohort, in which we observed a beneficial effect, our iterative causal forest analysis identified five subgroups with different treatment effects, defined by the number of emergency department visits, the number of prescriptions for short-acting β2-agonists, the number of prescriptions for inhaled steroids and long-acting β-agonists (either combination therapy or concurrent use), and age ≥ 50 years. Among these, patients with two or more emergency department visits during the 12-month baseline period had the largest absolute exacerbation risk reduction, with a decrease of 2.8% for GLP1RAs (95% CI, -4.8% to -0.9%). Conclusions: GLP1RAs demonstrated a beneficial effect on reducing asthma exacerbation relative to sulfonylureas. Patients with asthma with two or more emergency department visits (a proxy for disease severity) benefit most from GLP1RAs. Emergency department visit frequency, the number of maintenance and reliever inhalers, and age might help individualize prediction of the short-term benefit of GLP1RAs on asthma exacerbation.

Comparative effectiveness of extended-release naltrexone and sublingual buprenorphine for treatment of opioid use disorder among Medicaid patients

BACKGROUND AND AIMS

Extended-release naltrexone (XR-NTX) and sublingual buprenorphine (SL-BUP) are both approved for opioid use disorder (OUD) treatment in any medical setting. We aimed to compare the real-world effectiveness of XR-NTX and SL-BUP.

DESIGN AND SETTING

This was an observational active comparator, new user cohort study of Medicaid claims records for patients in New Jersey and California, USA, 2016-19.

PARTICIPANTS/CASES

The participants were adult Medicaid patients aged 18-64 years who initiated XR-NTX or SL-BUP for maintenance treatment of OUD and did not use medications for OUD in the 90 days before initiation. Our cohort included 1755 XR-NTX and 9886 SL-BUP patients.

MEASUREMENTS

We examined two outcomes up to 180 days after medication initiation: (1) composite of medication discontinuation and death and (2) composite of overdose and death.

FINDINGS

In adjusted analyses, treatment with XR-NTX was more likely to result in discontinuation or death by the end of follow-up than treatment with SL-BUP: cumulative risk 75.9% [95% confidence interval (CI) = 73.9%, 77.9%] versus 62.2% (95% CI = 61.2%, 63.2%), respectively (risk difference = 13.7 percentage points, 95% CI = 11.4, 16.0). There was minimal difference in the cumulative risk of overdose or death by the end of follow-up: XR-NTX 3.9% (95% CI = 3.0%, 4.8%) versus SL-BUP 3.3% (95% CI = 2.9%, 3.7%); risk difference = 0.5 percentage points, 95% CI = -0.4, 1.5. Results were consistent across sensitivity analyses.

CONCLUSIONS

Medicaid patients in California and New Jersey, USA, receiving treatment for opioid use disorder stayed in treatment longer on sublingual buprenorphine than on extended-release naltrexone, but the risk of overdose was similar. Most patients in this study discontinued medication within 6 months, regardless of which medication was initiated.

Evaluating individualized treatment effect predictions: A model-based perspective on discrimination and calibration assessment

In recent years, there has been a growing interest in the prediction of individualized treatment effects. While there is a rapidly growing literature on the development of such models, there is little literature on the evaluation of their performance. In this paper, we aim to facilitate the validation of prediction models for individualized treatment effects. The estimands of interest are defined based on the potential outcomes framework, which facilitates a comparison of existing and novel measures. In particular, we examine existing measures of discrimination for benefit (variations of the c-for-benefit), and propose model-based extensions to the treatment effect setting for discrimination and calibration metrics that have a strong basis in outcome risk prediction. The main focus is on randomized trial data with binary endpoints and on models that provide individualized treatment effect predictions and potential outcome predictions. We use simulated data to provide insight into the characteristics of the examined discrimination and calibration statistics under consideration, and further illustrate all methods in a trial of acute ischemic stroke treatment. The results show that the proposed model-based statistics had the best characteristics in terms of bias and accuracy. While resampling methods adjusted for the optimism of performance estimates in the development data, they had a high variance across replications that limited their accuracy. Therefore, individualized treatment effect models are best validated in independent data. To aid implementation, a software implementation of the proposed methods was made available in R.

Iterative Causal Forest: A Novel Algorithm for Subgroup Identification

Precisely and efficiently identifying subgroups with heterogeneous treatment effects (HTEs) in real-world evidence studies remains a challenge. Based on the causal forest (CF) method, we developed an iterative CF (iCF) algorithm to identify HTEs in subgroups defined by important variables. Our method iteratively grows different depths of the CF with important effect modifiers, performs plurality votes to obtain decision trees (subgroup decisions) for a family of CFs with different depths, and then finds the cross-validated subgroup decision that best predicts the treatment effect as a final subgroup decision. We simulated 12 different scenarios and showed that the iCF outperformed other machine learning methods for interaction/subgroup identification in the majority of scenarios assessed. Using a 20% random sample of fee-for-service Medicare beneficiaries initiating sodium-glucose cotransporter-2 inhibitors or glucagon-like peptide-1 receptor agonists, we implemented the iCF to identify subgroups with HTEs for hospitalized heart failure. Consistent with previous studies suggesting patients with heart failure benefit more from sodium-glucose cotransporter-2 inhibitors, iCF successfully identified such a subpopulation with HTEs and additive interactions. The iCF is a promising method for identifying subgroups with HTEs in real-world data where the potential for unmeasured confounding can be limited by study design.

Comparative effectiveness of extended release naltrexone and sublingual buprenorphine for treatment of opioid use disorder among Medicaid patients

Aims

To compare the real-world effectiveness of extended release naltrexone (XR-NTX) and sublingual buprenorphine (SL-BUP) for the treatment of opioid use disorder (OUD).

Design

An observational active comparator, new user cohort study.

Setting

Medicaid claims records for patients in New Jersey and California, 2016-2019.

Participants/Cases

Adult Medicaid patients aged 18-64 years who initiated XR-NTX or SL-BUP for maintenance treatment of OUD and did not use medications for OUD in the 90-days before initiation.

Comparators

New initiation with XR-NTX versus SL-BUP for the treatment of OUD.

Measurements

We examined two outcomes up to 180 days after medication initiation, 1) composite of medication discontinuation and death, and 2) composite of overdose and death.

Findings

Our cohort included 1,755 XR-NTX and 9,886 SL-BUP patients. In adjusted analyses, treatment with XR-NTX was more likely to result in discontinuation or death by the end of follow-up than treatment with SL-BUP: cumulative risk 76% (95% confidence interval [CI] 75%, 78%) versus 62% (95% CI 61%, 63%), respectively (risk difference 14 percentage points, 95% CI 13, 16). There was minimal difference in the cumulative risk of overdose or death by the end of follow-up: XR-NTX 3.8% (95% CI 2.9%, 4.7%) versus SL-BUP 3.3% (95% 2.9%, 3.7%); risk difference 0.5 percentage points, 95%CI -0.5, 1.5. Results were consistent across sensitivity analyses.

Conclusions

Longer medication retention is important because risks of negative outcomes are elevated after discontinuation. Our results support selection of SL-BUP over XR-NTX. However, most patients discontinued medication by 6 months indicating that more effective tools are needed to improve medication retention, particularly after initiation with XR-NTX, and to identify which patients do best on which medication.

Causal relationships between pain, medical treatments, and knee osteoarthritis: A graphical causal model to guide analyses

OBJECTIVE

Randomized controlled trials (RCTs) are a gold standard for estimating the benefits of clinical interventions, but their decision-making utility can be limited by relatively short follow-up time. Longer-term follow-up of RCT participants is essential to support treatment decisions. However, as time from randomization accrues, loss to follow-up and competing events can introduce biases and require covariate adjustment even for intention-to-treat effects. We describe a process for synthesizing expert knowledge and apply this to long-term follow-up of an RCT of treatments for meniscal tears in patients with knee osteoarthritis (OA).

METHODS

We identified 2 post-randomization events likely to impact accurate assessment of pain outcomes beyond 5 years in trial participants: loss to follow-up and total knee replacement (TKR). We conducted literature searches for covariates related to pain and TKR in individuals with knee OA and combined these with expert input. We synthesized the evidence into graphical models.

RESULTS

We identified 94 potential covariates potentially related to pain and/or TKR among individuals with knee OA. Of these, 46 were identified in the literature review and 48 by expert panelists. We determined that adjustment for 50 covariates may be required to estimate the long-term effects of knee OA treatments on pain.

CONCLUSION

We present a process for combining literature reviews with expert input to synthesize existing knowledge and improve covariate selection. We apply this process to the long-term follow-up of a randomized trial and show that expert input provides additional information not obtainable from literature reviews alone.

Starting a conversation about estimands with public partners involved in clinical trials: a co-developed tool

BACKGROUND

Clinical trials aim to draw conclusions about the effects of treatments, but a trial can address many different potential questions. For example, does the treatment work well for patients who take it as prescribed? Or does it work regardless of whether patients take it exactly as prescribed? Since different questions can lead to different conclusions on treatment benefit, it is important to clearly understand what treatment effect a trial aims to investigate-this is called the 'estimand'. Using estimands helps to ensure trials are designed and analysed to answer the questions of interest to different stakeholders, including patients and public. However, there is uncertainty about whether patients and public would like to be involved in defining estimands and how to do so. Public partners are patients and/or members of the public who are part of, or advise, the research team. We aimed to (i) co-develop a tool with public partners that helps explain what an estimand is and (ii) explore public partner's perspectives on the importance of discussing estimands during trial design.

METHODS

An online consultation meeting was held with 5 public partners of mixed age, gender and ethnicities, from various regions of the UK. Public partner opinions were collected and a practical tool describing estimands, drafted before the meeting by the research team, was developed. Afterwards, the tool was refined, and additional feedback sought via email.

RESULTS

Public partners want to be involved in estimand discussions. They found an introductory tool, to be presented and described to them by a researcher, helpful for starting a discussion about estimands in a trial design context. They recommended storytelling, analogies and visual aids within the tool. Four topics related to public partners' involvement in defining estimands were identified: (i) the importance of addressing questions that are relevant to patients and public in trials, (ii) involving public partners early on, (iii) a need for education and communication for all stakeholders and (iv) public partners and researchers working together.

CONCLUSIONS

We co-developed a tool for researchers and public partners to use to facilitate the involvement of public partners in estimand discussions.

A randomized trial of web-based fertility-tracking software and fecundability

OBJECTIVE

To assess the effect of randomization to FertilityFriend.com, a mobile computing fertility-tracking app, on fecundability.

DESIGN

Parallel non-blinded randomized controlled trial nested within the Pregnancy Study Online (PRESTO), a North American preconception cohort.

PATIENT(S)

Female participants aged 21 to 45 years attempting conception for ≤6 menstrual cycles at enrolment (2013-2019).

INTERVENTION

Randomization (1:1) of 5532 participants to receive a premium Fertility Friend (FF) subscription.

MAIN OUTCOME MEASURE(S)

Fecundability (per-cycle probability of conception). Participants completed bimonthly follow-up questionnaires until pregnancy or a censoring event, whichever came first. We first performed an intent-to-treat analysis of the effect of FF randomization on fecundability. In secondary analyses, we used a per-protocol approach that accounted for adherence in each trial arm. In both analyses, we used proportional probabilities regression models to estimate fecundability ratios (FR) and 95% confidence intervals (CI) comparing those randomized vs. not randomized and applied inverse probability weights to account for loss-to-follow-up (intent-to-treat and per-protocol analyses) and adherence (per-protocol analyses only).

RESULTS

Using life-table methods, 64% of the 2775 participants randomized to FF and 63% of the 2767 participants not randomized to FF conceived during 12 cycles; these respective percentages were each 70% among those with 0-1 cycles of attempt time at enrolment. Of those randomized to FF, 72% were defined as adherent (68% of observed menstrual cycles). In intent-to-treat analyses, there was no appreciable association overall (FR = 0.97; 95% CI, 0.90-1.04) or within strata of pregnancy attempt time at enrolment, age, education, or other characteristics. In per-protocol analyses, we observed little association overall (FR = 1.06; 95% CI, 0.99-1.14), but weak-to-moderate positive associations among participants who had longer attempt times at enrolment (FR = 1.15; 95% CI, 0.98-1.35 for 3-4 cycles; FR = 1.14; 95% CI, 0.87-1.48 for 5-6 cycles), were aged <25 years (FR = 1.29; 95% CI, 1.01-1.66), had ≤12 years of education (FR = 1.32; 95% CI, 0.92-1.89), or were non-users of hormonal contraception within 3 months before enrolment (FR = 1.10; 95% CI, 1.02-1.19).

CONCLUSION

No appreciable associations were observed in intent-to-treat analyses. In secondary per-protocol analyses that accounted for adherence, randomization to FF was associated with slightly greater fecundability among selected subgroups of participants; however, these results are susceptible to unmeasured confounding.

Early time-restricted eating affects weight, metabolic health, mood, and sleep in adherent completers: A secondary analysis

OBJECTIVE

Data are mixed on whether intermittent fasting improves weight loss and cardiometabolic health. Here, the effects of time-restricted eating (TRE) in participants who consistently adhered ≥5 d/wk every week were analyzed.

METHODS

Ninety patients aged 25 to 75 years old with obesity were randomized to early TRE (eTRE; 8-hour eating window from 07:00 to 15:00) or a control schedule (≥12-hour window) for 14 weeks. A per-protocol analysis of weight loss, body composition, cardiometabolic health, and other end points was performed.

RESULTS

Participants who adhered to eTRE ≥5 d/wk every week had greater improvements in body weight (-3.7 ± 1.2 kg; p = 0.003), body fat (-2.8 ± 1.3 kg; p = 0.04), heart rate (-7 ± 3 beats/min; p = 0.02), insulin resistance (-2.80 ± 1.36; p = 0.047), and glucose (-9 ± 5 mg/dL; p = 0.047) relative to adherers in the control group. They also experienced greater improvements in mood, including fatigue and anger; however, they self-reported sleeping less and taking longer to fall asleep.

CONCLUSIONS

For those who can consistently adhere at least 5 d/wk, eTRE is a valuable approach for improving body weight, body fat, cardiometabolic health, and mood. Further research is needed to determine whether eTRE's effects of shortening sleep but reducing fatigue are healthful or not.

Key considerations for choosing a statistical method to deal with incomplete treatment adherence in pragmatic trials

Pragmatic trials offer practical means of obtaining real-world evidence to help improve decision-making in comparative effectiveness settings. Unfortunately, incomplete adherence is a common problem in pragmatic trials. The commonly used methods in randomized control trials often cannot handle the added complexity imposed by incomplete adherence, resulting in biased estimates. Several naive methods and advanced causal inference methods (e.g., inverse probability weighting and instrumental variable-based approaches) have been used in the literature to deal with incomplete adherence. Practitioners and applied researchers are often confused about which method to consider under a given setting. This current work is aimed to review commonly used statistical methods to deal with non-adherence along with their key assumptions, advantages, and limitations, with a particular focus on pragmatic trials. We have listed the applicable settings for these methods and provided a summary of available software. All methods were applied to two hypothetical datasets to demonstrate how these methods perform in a given scenario, along with the R codes. The key considerations include the type of intervention strategy (point treatment settings, where treatment is administered only once versus sustained treatment settings, where treatment has to be continued over time) and availability of data (e.g., the extent of measured or unmeasured covariates that are associated with adherence, dependent confounding impacted by past treatment, and potential violation of assumptions). This study will guide practitioners and applied researchers to use the appropriate statistical method to address incomplete adherence in pragmatic trial settings for both the point and sustained treatment strategies.

Early introduction of peanut reduces peanut allergy across risk groups in pooled and causal inference analyses

BACKGROUND

The Learning Early About Peanut allergy (LEAP) study has shown the effectiveness of early peanut introduction in prevention of peanut allergy (PA). In the Enquiring About Tolerance (EAT) study, a statistically significant reduction in PA was present only in per-protocol (PP) analyses, which can be subject to bias.

OBJECTIVE

The aim of this study was to combine individual-level data from the LEAP and EAT trials and provide robust evidence on the bias-corrected, causal effect of early peanut introduction.

METHOD

As part of the European Union-funded iFAAM project, this pooled analysis of individual pediatric patient data combines and compares effectiveness and efficacy estimates of oral tolerance induction among different risk strata and analysis methods.

RESULTS

An intention-to-treat (ITT) analysis of pooled data showed a 75% reduction in PA (p < .0001) among children randomized to consume peanut from early infancy. A protective effect was present across all eczema severity groups, irrespective of enrollment sensitization to peanut, and across different ethnicities. Earlier age of introduction was associated with improved effectiveness of the intervention. In the pooled PP analysis, peanut consumption reduced the risk of PA by 98% (p < .0001). A causal inference analysis confirmed the strong PP effect (89% average treatment effect relative risk reduction p < .0001). A multivariable causal inference analysis approach estimated a large (100%) reduction in PA in children without eczema (p = .004).

CONCLUSION

We demonstrate a significant reduction in PA with early peanut introduction in a large group of pooled, randomized participants. This significant reduction was demonstrated across all risk subgroups, including children with no eczema. Furthermore, our results point to increased efficacy of the intervention with earlier age of introduction.

Invited Commentary: Conducting and Emulating Trials to Study Effects of Social Interventions

All else being equal, if we had 1 causal effect we wished to estimate, we would conduct a randomized trial with a protocol that mapped onto that causal question, or we would attempt to emulate that target trial with observational data. However, studying the social determinants of health often means there are not just 1 but several causal contrasts of simultaneous interest and importance, and each of these related but distinct causal questions may have varying degrees of feasibility in conducting trials. With this in mind, we discuss challenges and opportunities that arise when conducting and emulating such trials. We describe designing trials with the simultaneous goals of estimating the intention-to-treat effect, the per-protocol effect, effects of alternative protocols or joint interventions, effects within subgroups, and effects under interference, and we describe ways to make the most of all feasible randomized trials and emulated trials using observational data. Our comments are grounded in the study results of Courtin et al. (Am J Epidemiol. 2022;191(8):1444-1452).

Large-scale evidence generation and evaluation across a network of databases for type 2 diabetes mellitus (LEGEND-T2DM): a protocol for a series of multinational, real-world comparative cardiovascular effectiveness and safety studies

INTRODUCTION

Therapeutic options for type 2 diabetes mellitus (T2DM) have expanded over the last decade with the emergence of cardioprotective novel agents, but without such data for older drugs, leaving a critical gap in our understanding of the relative effects of T2DM agents on cardiovascular risk.

METHODS AND ANALYSIS

The large-scale evidence generations across a network of databases for T2DM (LEGEND-T2DM) initiative is a series of systematic, large-scale, multinational, real-world comparative cardiovascular effectiveness and safety studies of all four major second-line anti-hyperglycaemic agents, including sodium-glucose co-transporter-2 inhibitor, glucagon-like peptide-1 receptor agonist, dipeptidyl peptidase-4 inhibitor and sulfonylureas. LEGEND-T2DM will leverage the Observational Health Data Sciences and Informatics (OHDSI) community that provides access to a global network of administrative claims and electronic health record data sources, representing 190 million patients in the USA and about 50 million internationally. LEGEND-T2DM will identify all adult, patients with T2DM who newly initiate a traditionally second-line T2DM agent. Using an active comparator, new-user cohort design, LEGEND-T2DM will execute all pairwise class-versus-class and drug-versus-drug comparisons in each data source, producing extensive study diagnostics that assess reliability and generalisability through cohort balance and equipoise to examine the relative risk of cardiovascular and safety outcomes. The primary cardiovascular outcomes include a composite of major adverse cardiovascular events and a series of safety outcomes. The study will pursue data-driven, large-scale propensity adjustment for measured confounding, a large set of negative control outcome experiments to address unmeasured and systematic bias.

ETHICS AND DISSEMINATION

The study ensures data safety through a federated analytic approach and follows research best practices, including prespecification and full disclosure of results. LEGEND-T2DM is dedicated to open science and transparency and will publicly share all analytic code from reproducible cohort definitions through turn-key software, enabling other research groups to leverage our methods, data and results to verify and extend our findings.

Can patient-led surveillance detect subsequent new primary or recurrent melanomas and reduce the need for routinely scheduled follow up? Statistical analysis plan for the MEL-SELF randomised controlled trial

BACKGROUND

The MEL-SELF trial is a randomised controlled trial of patient-led surveillance compared to clinician-led surveillance in people treated for localised cutaneous melanoma (stage 0, I, II). The primary trial aim is to determine if patient led-surveillance compared to clinician-led surveillance increases the proportion of participants who are diagnosed with a new primary or recurrent melanoma at a fast-tracked unscheduled clinic visit. The secondary outcomes include time to diagnosis of any skin cancer, psychosocial outcomes, acceptability, and resource use.

OBJECTIVE

The objective of this report is to outline and publish the pre-determined statistical analysis plan before the database lock and the start of analysis.

METHODS/DESIGN

The statistical analysis plan describes the overall analysis principles, including how participants will be included in each analysis, the presentation of the results, adjustments for covariates, the primary and secondary outcomes, and their respective analyses. In addition, we present the planned sensitivity and subgroup analyses. A separate analysis plan will be published for health economic outcomes.

RESULTS

The MEL-SELF statistical analysis plan has been designed to minimize bias in estimating effects of the intervention on primary and secondary outcomes. By pre-specifying analyses, we ensure the study's integrity and believability while enabling the reproducibility of the final analysis.

CONCLUSION

This detailed statistical analysis plan will help to ensure transparency of reporting of results from the MEL-SELF trial.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry (ANZCTR): ACTRN12621000176864. Registered 18 February 2021, https://www.anzctr.org.au/ACTRN12621000176864.aspx.

Harrell Jr F, Moons KGM, Debray TPA, Reitsma JB. A tutorial on individualized treatment effect prediction from randomized trials with a binary endpoint

Randomized trials typically estimate average relative treatment effects, but decisions on the benefit of a treatment are possibly better informed by more individualized predictions of the absolute treatment effect. In case of a binary outcome, these predictions of absolute individualized treatment effect require knowledge of the individual's risk without treatment and incorporation of a possibly differential treatment effect (ie, varying with patient characteristics). In this article, we lay out the causal structure of individualized treatment effect in terms of potential outcomes and describe the required assumptions that underlie a causal interpretation of its prediction. Subsequently, we describe regression models and model estimation techniques that can be used to move from average to more individualized treatment effect predictions. We focus mainly on logistic regression-based methods that are both well-known and naturally provide the required probabilistic estimates. We incorporate key components from both causal inference and prediction research to arrive at individualized treatment effect predictions. While the separate components are well known, their successful amalgamation is very much an ongoing field of research. We cut the problem down to its essentials in the setting of a randomized trial, discuss the importance of a clear definition of the estimand of interest, provide insight into the required assumptions, and give guidance with respect to modeling and estimation options. Simulated data illustrate the potential of different modeling options across scenarios that vary both average treatment effect and treatment effect heterogeneity. Two applied examples illustrate individualized treatment effect prediction in randomized trial data.

The estimand framework and its application in substance use disorder clinical trials: a case study

Relapse rates among individuals with substance use disorder (SUD) remain high and new treatment approaches are needed, which require evaluation in randomized controlled trials (RCTs). Measurement and interpretation challenges for SUD RCT data are often ignored or presented only in statistical analysis plans. Since different analytic approaches may result in different estimates and thus interpretations of the treatment effect, it is important to present this clearly throughout the trial. Inconsistencies between study analyses and objectives present further challenges for interpretation and cross-study comparisons. The recent International Council for Harmonization (ICH) addendum provides standardized language and a common framework for aligning trial objectives, design, conduct, and analysis. The framework focuses on estimands, which describe the treatment effect and link the trial objective with the scientific question and the analytic approach. The use of estimands offers SUD researchers and clinicians the opportunity to explicitly address events that affect measurement and interpretation at the outset of the trial. Furthermore, the use of standard terminology can lead to clearer interpretations of SUD trials and the treatments evaluated in SUD trials. Resources for understanding and applying estimands are needed to optimize the use of this new, helpful framework. This Perspective provides this resource for SUD researchers. Specifically, it highlights the relevance of estimands for SUD trials. Furthermore, it demonstrates how estimands can be used to develop clinically relevant analyses to address challenges in SUD trials. It also shows how a standardized framework can be employed to improve the interpretation and presentation of SUD study findings.

A review of pragmatic trials found a high degree of diversity in design and scope, deficiencies in reporting and trial registry data, and poor indexing

OBJECTIVE

We established a large database of trials to serve as a resource for future methodological and ethical analyses. Here, we use meta-data to describe the broad landscape of pragmatic trials including research areas, identification as pragmatic, quality of trial registry data and enrolment.

STUDY DESIGN AND SETTING

Trials were identified by a validated search filter and included if a primary report of a health-related randomized trial published January 2014-April 2019. Data were collated from MEDLINE, Web of Science, ClinicalTrials.gov, and full text.

RESULTS

4337 eligible trials were identified from 13,065 records, of which 1988 were registered in ClinicalTrials.gov. Research areas were diverse, with the most common being general and internal medicine; public, environmental and occupational health; and health care sciences and services. The term "pragmatic" was seldom used in titles or abstracts. Several domains in ClinicalTrials.gov had questionable data quality. We estimated that one-fifth of trials under-accrued by at least 15%.

CONCLUSION

There is a need to improve reporting of pragmatic trials and quality of trial registry data. Under accrual remains a challenge in pragmatic RCTs despite calls for more streamlined recruitment approaches. The diversity of pragmatic trials should be reflected in future ethical analyses.

Smoking cessation and survival among people diagnosed with non-metastatic cancer

Background

We aimed to estimate the effects of smoking cessation on survival among people diagnosed with cancer.

Methods

We used data from a Comprehensive Community Cancer Program that is part of a large urban safety-net hospital system. Eligible patients were diagnosed with primary invasive solid tumors between 2013 and 2015, and were current smokers at time of diagnosis. Our exposure of interest was initiation of smoking cessation within 6 months of cancer diagnosis. We estimated inverse probability weighted restricted mean survival time (RMST) differences and risk ratio (RR) for all cause 3-year mortality.

Results

Our study population comprised 369 patients, of whom 42% were aged < 55 years, 59% were male, 44% were racial/ethnic minorities, and 59% were uninsured. The 3-year RMST was 1.8 (95% CL: − 1.5, 5.1) months longer for individuals who initiated smoking cessation within 6 months of cancer diagnosis. The point estimate for risk of 3-year mortality was lower for initiation of smoking cessation within 6 months of diagnosis compared with no initiation within 6 months (RR = 0.72, 95% CL: 0.37, 1.4).

Conclusions

Our point estimates suggest longer 3-year survival, but the results are compatible with 1.5 month shorter or 5.1 longer 3-year overall survival after smoking cessation within 6 months of cancer diagnosis. Future studies with larger sample sizes that test the comparative effectiveness of different smoking cessation strategies are needed for more detailed evidence to inform decision-making about the effect of smoking cessation on survival among cancer patients.

Implications for Cancer survivors

The benefits of smoking cessation after cancer diagnosis may include longer survival, but the magnitude of benefit is unclear.

The importance of decision intent within descriptions of pragmatic trials

OBJECTIVE

It is now more than 50 years since the concepts of explanatory and pragmatic attitudes toward trials were first discussed by Schwartz and Lellouch in their influential 1967 paper. Since then, there has been increasing focus on design aspects that may be consistent with more pragmatic attitudes within clinical trials, and a number of tools developed to assist investigators prospectively think about their trial design. Researchers have subsequently expressed interest in using these tools retrospectively to characterize trials as pragmatic or explanatory.

RESULTS

We suggest that recent attempts to retrospectively dichotomize trials solely on the basis of quantitative scoring of trial design features are flawed. Instead, we argue that there is a need to consider both the intent and design when assessing the degree of pragmatism within a trial.

CONCLUSION

The practical implication of our suggestion for trial reporting is that investigators should explicitly state the intent of the trial through a clear articulation of the decision that they hope will be informed by the trial results. This should be coupled with a completed PRagmatic-Explanatory Continuum Indicator Summary 2 assessment (or similar) with an explanation of study design choices to appropriately assess whether the study design is consistent with the study intent. We believe this will assist reviewers and knowledge users in making assessments of trials.

Adherence-adjustment in placebo-controlled randomized trials: An application to the candesartan in heart failure randomized trial

BACKGROUND

The per-protocol effect provides important information in randomized trials with incomplete adherence. Yet, because valid estimation typically requires adjustment for prognostic factors that predict adherence, per-protocol effect estimates are often met with skepticism. In placebo-controlled trials, however, the validity of adjustment can be indirectly verified by demonstrating no association between adherence and the outcome among the placebo arm. Here, we describe a two-stage procedure in which we first adjust for time-varying adherence in the placebo arm and then use a similar procedure to estimate the per-protocol effect.

METHODS

We use the Candesartan in Heart Failure: Assessment of Reduction in Mortality and Morbidity (CHARM) randomized trial. First, we compare adherers versus non-adherers in the placebo arm, adjusting for pre- and post-randomization variables. Second, we use models validated in the placebo arm to estimate the per-protocol effect of adherence to candesartan versus placebo in the full trial.

FINDINGS

We successfully estimated no association between adherence and mortality in the placebo arm; hazard ratio: 0.91 (95% CI: 0.51, 2.52). We then estimated the per-protocol effect under two sets of protocol-defined stopping criteria after adjustment for post-randomization confounders. The mortality hazard ratio estimates ranged from 0.91 to 0.93 for the per-protocol effect estimates, similar to the intention-to-treat effect estimates.

INTERPRETATION

Adherence adjustment in the CHARM trial is feasible when appropriate assumptions about missing data and confounding are made. These assumptions cannot be verified but can be supported through the use of placebo-arm adherence assessment.

Local average treatment effects estimation via substantive model compatible multiple imputation

Nonadherence to assigned treatment is common in randomized controlled trials (RCTs). Recently, there has been increased interest in estimating causal effects of treatment received, for example, the so-called local average treatment effect (LATE). Instrumental variables (IV) methods can be used for identification, with estimation proceeding either via fully parametric mixture models or two-stage least squares (TSLS). TSLS is popular but can be problematic for binary outcomes where the estimand of interest is a causal odds ratio. Mixture models are rarely used in practice, perhaps because of their perceived complexity and need for specialist software. Here, we propose using multiple imputation (MI) to impute the latent compliance class appearing in the mixture models. Since such models include an interaction term between the latent compliance class and randomized treatment, we use "substantive model compatible" MI (SMC MIC), which can additionally handle missing data in outcomes and other variables in the model, before fitting the mixture models via maximum likelihood to the MI data sets and combining results via Rubin's rules. We use simulations to compare the performance of SMC MIC to existing approaches and also illustrate the methods by reanalyzing an RCT in UK primary health. We show that SMC MIC can be more efficient than full Bayesian estimation when auxiliary variables are incorporated, and is superior to two-stage methods, especially for binary outcomes.

Estimating cluster-level local average treatment effects in cluster randomised trials with non-adherence

Non-adherence to assigned treatment is a common issue in cluster randomised trials. In these settings, the efficacy estimand may also be of interest. Many methodological contributions in recent years have advocated using instrumental variables to identify and estimate the local average treatment effect. However, the clustered nature of randomisation in cluster randomised trials adds to the complexity of such analyses. In this paper, we show that the local average treatment effect can be estimated via two-stage least squares regression using cluster-level summaries of the outcome and treatment received under certain assumptions. We propose the use of baseline variables to adjust the cluster-level summaries before performing two-stage least squares in order to improve efficiency. Implementation needs to account for the reduced sample size, as well as the possible heteroscedasticity, to obtain valid inferences. Simulations are used to assess the performance of two-stage least squares of cluster-level summaries under cluster-level or individual-level non-adherence, with and without weighting and robust standard errors. The impact of adjusting for baseline covariates and of appropriate degrees of freedom correction for inference is also explored. The methods are then illustrated by re-analysing a cluster randomised trial carried out in a specific UK primary care setting. Two-stage least squares estimation using cluster-level summaries provides estimates with small to negligible bias and coverage close to nominal level, provided the appropriate small sample degrees of freedom correction and robust standard errors are used for inference.