Reporting of Observational Studies Explicitly Aiming to Emulate Randomized Trials: A Systematic Review

Strengthening health technology assessment for cancer treatments in Europe by integrating causal inference and target trial emulation

Health Technology Assessment (HTA) for reimbursement of all new cancer drugs in the European Union (EU) will be evaluated for all member states by a central European HTA starting in 2025. EU HTA guidelines for applicants under these new regulations put the focus on meta-analysis of aggregated randomized trial data and are in contrast with the growing number of cancer drug approvals by drug regulators that are based on single arm studies and the needs in the rapidly evolving field of oncological drug development. We advocate to broaden the methodological approaches for HTA by including observational data based causal inference methodology and target trial emulation into the assessments of comparative effectiveness. Causal inference estimates causal estimands, effect measures that reflect a population level effect in terms of contrasts of counterfactual outcomes in the same patients and are directly measured in the target population by modeling of hypothetical intervention. Target trial emulation allows with the use of causal inference to estimate causal effects by mimicking pragmatic trials that evolve apart from randomization like a trial. We illustrate the potential of causal inference for HTA and provide an introduction into causal inference methodology for health scientists involved in HTA.

Effectiveness of total hip arthroplasty versus non-surgery on patient-reported hip function at 3 months: a target trial emulation study of patients with osteoarthritis

BACKGROUND AND PURPOSE

 This study introduces an innovative research design in the field of orthopedics, using a target trial emulation approach. We aimed to assess the causal effects of total hip arthroplasty (THA) compared with nonoperative treatment in reducing patient-reported hip disability at 3 months in patients with osteoarthritis, using real-world data.

METHODS

 We emulated a target trial using real-world data of 2 Dutch hospitals between April 2020 and January 2022. Patients diagnosed with hip osteoarthritis and eligible for primary THA were included in the study. During the COVID-19 pandemic, THA was often cancelled due to external factors (i.e., limited operating room capacity, or surgeon unavailable due to quarantine rules), resulting in an arbitrary allocation of patients to THA (n = 132) or non-THA (n = 60). We compared changes in hip disability, measured using the Hip disability and Osteoarthritis Outcome Score Physical function Short form (HOOS-PS), between the THA group at 3 months postoperatively and the non-THA group at ≥3 months post waiting-list. Linear regression analysis, adjusting for potential confounders, was used to compare between-group differences.

RESULTS

 THA showed preferable outcomes compared with non-THA, indicated by a difference of -33 points (95% confidence interval [CI] -37 to -28) on the HOOS-PS. Patients in the THA group demonstrated a clinically significant improvement in hip function, with a mean change of -27 points (CI -31 to -24), while the control group showed no improvement with a mean change of 7 points (CI 3-11) on the HOOS-PS.

CONCLUSION

 THA significantly improves hip function in osteoarthritis patients, surpassing the outcomes observed in the non-surgery group.

Enhancing knowledge of treatment effects by analysing observational data as a randomised trial

PURPOSE AND BACKGROUND

Randomised controlled trials are the gold standard design to obtain knowledge of causal effects of treatments. However, not all causal questions can be answered by randomised trials for practical or ethical reasons. Target trial emulation is a conceptual framework and set of analysis methods to obtain knowledge of causal effects of treatments using observational data, and is particularly valuable when randomised trials are not feasible.

OBJECTIVES

In this Perspective, we outline the concepts and application of target trial emulation, showing how it can be used to minimise bias due to confounding at baseline and over the course of sustained treatments, and how causal per-protocol analysis can be performed to provide informative yet unbiased estimates of treatment effects.

SUMMARY AND CONCLUSIONS

We summarise a recent target trial emulation that examined the effects of intensity of early rehabilitation for hip fracture on performance in activities of daily living as a worked example. We encourage investigators to explore the target trial framework, and apply it to answer causal questions when using observational data.

Improving the validity of noninterventional comparative effectiveness research by basing study design on a specified existing randomized controlled trial

This issue of the American Journal of Epidemiology includes 3 articles (2 reporting original analyses and 1 systematic review) in which noninterventional studies used an existing randomized controlled trial (RCT) as a reference standard to inform noninterventional study design and against which to benchmark results. This commentary provides a brief background on the challenges of noninterventional comparative effectiveness research, before elaborating on (1) the potential benefits and challenges of basing noninterventional study design on a specified existing RCT and (2) the distinction between designing analysis based upon a specified existing RCT and studies based solely upon a hypothetical target trial. Finally, several recommendations for the conduct and reporting of noninterventional studies based on existing RCTs are provided. This article is part of a Special Collection on Pharmacoepidemiology.

Emulating Target Trials to Study Perioperative and Critical Care Interventions with Observational Data: Promise and Limitations

Estimating effects of interventions is a central task in perioperative and critical care outcomes research. While randomized trials remain the accepted standard for causal inference, trial data are not always available to inform clinical decisions, and some questions cannot be answered feasibly or efficiently with trials. In these settings, studies using observational healthcare data may be used to inform practice. Causal inference from observational data has been reconsidered in recent years, challenging the prevailing notion among clinical researchers that causal conclusions cannot be drawn from observational studies. The "target trial framework" is one contribution within a growing methodologic field that helps investigators avoid common pitfalls in observational study design and analysis. Importantly, researchers must understand which biases this framework can-and cannot-help avoid. The authors present an overview of target trial emulation and describe the promise and limitations of this framework for improving observational perioperative and critical care outcomes research.

Bridging ideal studies and real-world practice: the potential role of real-world evidence in reproductive medicine

Evidence-based medicine is central to reproductive health care, guiding interventions such as ovarian stimulation protocols and fertility preservation strategies. While randomized controlled trials (RCT) remain the gold standard for providing evidence and for establishing causality, their limitations, including restricted generalizability and high costs, highlight the need for complementary methodologies. Real-world evidence (RWE), derived from real-world data (RWD) such as electronic health records and assisted reproductive technology (ART) registries, has the potential to bridge the gap between controlled research settings and routine clinical practice, particularly for evaluating long-term and rare outcomes. However, RWE faces considerable challenges, including bias, variability in data quality, and difficulties in establishing causality. This paper explores the role of RWE in ART. Despite its promise, RWE cannot replace RCT, particularly in scenarios where the signal-to-noise ratio is low, such as detecting small effect differences in ovarian stimulation protocols or treatment add-ons. Instead, in most instances, RWE is positioned to complement RCT, supported by emerging regulatory frameworks such as the guidelines of the European Medicine Agency and the European Union's tissue directives, which emphasize the generation of RWE from ART practice to strengthen patient follow-up and safety monitoring. For ART stakeholders, a comprehensive understanding of the strengths and limitations of RWE is essential, as is the target trial emulation framework on RWD, for advancing the field and improving patient outcomes.

The target trial framework in global health research: barriers and opportunities

A randomised trial is the best way to make causal inferences when evaluating the effectiveness and safety of health interventions in global health research. Trials, however, are inherently expensive, unfeasible in many scenarios, and may raise ethical issues. In these scenarios, we must turn to analyses of observational data to learn what works. The target trial framework provides an organising principle for the design of observational studies that can lead to clinically interpretable results and analytic approaches that can reduce common biases. In this analysis, we describe the global distribution of data sources used in applications of the target trial framework and discuss barriers to its increased use in global health research, such as limited access to high-quality observational data. We then suggest a cost-effective solution of incorporating the collection of additional high-quality observational data into the implementation of large randomised trials in low- and middle-income countries. We found that the target trial framework is underutilised in observational studies conducted in most low- and middle-income countries. The main barriers are little available data and few trained researchers, which can be overcome by incorporating high-quality observational data collection into the data collection phase of large randomised trials, and by introducing small adjustments to the teaching curriculum.

The R.O.A.D. to clinical trial emulation

Observational studies provide the only evidence on the effectiveness of interventions when randomized controlled trials (RCTs) - apart from the initial RCT that establishes the efficacy of a treatment compared to a placebo - are impractical due to cost, ethical concerns, or time constraints. While many methodologies aim to draw causal inferences from observational data, there is a growing trend to model observational study designs after hypothetical or existing RCTs, a strategy known as "target trial emulation." Despite its potential, causal inference through target trial emulation is challenging because it cannot fully address the confounding bias inherent in real-world data due to the lack of randomization. In this work, we present a novel framework for target trial emulation that aims to overcome several key limitations, including confounding bias. The framework proceeds as follows: First, we apply the eligibility criteria of a specific trial to an observational cohort derived from real-world data. We then "correct" this cohort by extracting a subset that, through optimization techniques, matches both the distribution of covariates and baseline prognoses (i.e., the prognosis in the trial's control group) of the target RCT. Next, we address unmeasured confounding by adjusting the prognosis estimates of the treated group to align with those observed in the trial, using cost-sensitive counterfactual models. Following trial emulation, we go a step further by leveraging the emulated cohort to train optimal decision trees, developed by our team, to identify subgroups of patients exhibiting heterogeneity in treatment effects (HTE). The absence of confounding is verified using two external models, and the validity of the treatment effects estimated by our framework is independently confirmed by the team responsible for the original trial we emulate. To our knowledge, this is the first framework to successfully address both observed and unobserved confounding, a challenge that has historically limited the use of randomized trial emulation and causal inference in general since the 1950s. Additionally, our framework holds promise in advancing precision or personalized medicine by identifying patient subgroups that benefit most from specific treatments.

Target Trial Emulation: Improving the Quality of Observational Studies in Inflammatory Bowel Disease Using the Principles of Randomized Trials

The past decade has seen a substantial increase in the number of randomized controlled trials (RCTs) conducted in inflammatory bowel disease (IBD). Randomized controlled trials are the gold standard method for generating robust evidence of drug safety and efficacy but are expensive, time-consuming, and may have ethical implications. Observational studies in IBD are often used to fill the gaps in evidence but are typically hindered by significant bias. There are several approaches for making statistical inferences from observational data with some that focus on study design and others on statistical techniques. Target trial emulation is an emerging methodological process that aims to bridge this gap and improve the quality of observational studies by applying the principles of an ideal, or "target," randomized trial to routinely collected clinical data. There has been a rapid expansion of observational studies that have emulated trials over the past 5 years in other medical fields, but this has yet to be adopted in gastroenterology and IBD. The wealth of nonrandomized clinical data available through electronic health records, patient registries, and administrative health databases afford innumerable hypothesis-generating opportunities for IBD research. This review outlines the principles of target trial emulation, discusses the merits to IBD observational studies in reducing the most common biases and improving confidence in causality, and details the caveats of using this approach.

The Target Trial Framework for Causal Inference From Observational Data: Why and When Is It Helpful?

When randomized trials are not available to answer a causal question about the comparative effectiveness or safety of interventions, causal inferences are drawn using observational data. A helpful 2-step framework for causal inference from observational data is 1) specifying the protocol of the hypothetical randomized pragmatic trial that would answer the causal question of interest (the target trial), and 2) using the observational data to attempt to emulate that trial. The target trial framework can improve the quality of observational analyses by preventing some common biases. In this article, we discuss the utility and scope of applications of the framework. We clarify that target trial emulation resolves problems related to incorrect design but not those related to data limitations. We also describe some settings in which adopting this approach is advantageous to generate effect estimates that can close the gaps that randomized trials have not filled. In these settings, the target trial framework helps reduce the ambiguity of causal questions.

Effectiveness of Ampicillin-Sulbactam Versus Ceftriaxone for the Initial Treatment of Community-Acquired Pneumonia in Older Adults: A Target Trial Emulation Study

Background

Current guidelines for community-acquired pneumonia (CAP) include ampicillin-sulbactam as an initial treatment option, though they do not mandate routine coverage of anaerobic organisms. This study aimed to compare the effectiveness of ampicillin-sulbactam with that of ceftriaxone as initial treatment for CAP in older adults.

Methods

This study was conducted using the target trial emulation framework, using a nationwide Japanese database (May 2010-June 2023). The study included patients aged ≥65 years, admitted to Diagnosis Procedure Combination hospitals for CAP, who received either ampicillin-sulbactam or ceftriaxone as the initial treatment. The exposure group received ampicillin-sulbactam, while the control group received ceftriaxone, both on the day of hospitalization. The primary outcome was in-hospital mortality; the secondary outcome was the development of Clostridioides difficile infection during hospitalization.

Results

The study included 26 633 older patients hospitalized with CAP, with 14 906 receiving ampicillin-sulbactam and 11 727 receiving ceftriaxone as initial treatment. After inverse probability of treatment weighting, the ampicillin-sulbactam group was associated with a higher in-hospital mortality rate than the ceftriaxone group (10.5% vs 9.0%, respectively; adjusted risk difference, 1.5% [95% confidence interval, .7%-2.4%]; adjusted odds ratio, 1.19 [1.08-1.31]). The incidence of C difficile infection was numerically higher in the ampicillin-sulbactam group (0.6% vs 0.4%; adjusted risk difference, 0.2% [95% confidence interval, .0%-.4%]; adjusted odds ratio, 1.45 [.99-2.11]). These results were consistent among patients with risk factors for aspiration.

Conclusions

In older patients with CAP, initial treatment with ampicillin-sulbactam was associated with higher mortality compared to treatment with ceftriaxone.

Transitions to Nursing Homes among Residents of Assisted Living and Community-Dwelling Home Care Recipients

OBJECTIVES

To examine transitions to a nursing home among residents of assisted living relative to community-dwelling home care recipients.

DESIGN

Population-based retrospective cohort study emulating a target trial.

SETTING AND PARTICIPANTS

Linked, individual-level health system data were obtained from older adults (aged ≥65 years) who made an incident application for a bed in a nursing home in Ontario, Canada, between April 1, 2014, and March 31, 2019, and were followed until December 31, 2019.

METHODS

Residency in assisted living was compared with only community-dwelling home care. Any long-stay (≥90 days) and short-stay (<90 days) transitions to a nursing home were examined. Inverse probability weighted pooled logistic regression models were used to generate marginal cumulative incidence curves under each exposure status that were standardized by the covariates.

RESULTS

This study included 10,012 residents of assisted living [mean (SD) aged 88.7 (6.26) years, 75% female] and 131,679 home care recipients [mean (SD) aged 84.8 (7.43) years, 63% female] who applied for a bed in a nursing home (N = 141,691; 95,744.6 person-years). There were 6049 transitions among applicants from assisted living and 85,190 transitions among applicants who were home care recipients to a nursing home. The 5-year absolute risk reduction was 110 transitions to a nursing home per 1000 older adult applicants if all applicants resided in assisted living (95% CI, 71-148). Residency in assisted living resulted in a 12.7% relative decrease in the 5-year risk of any transition to a nursing home had all applicants resided in assisted living (95% CI, 8.3%-17.1%).

CONCLUSIONS AND IMPLICATIONS

Residents of assisted living were less likely to transition to a nursing home, despite equivalent clinical complexity and health care needs. The integration of assisted living into the continuum of care from the community to institutionalized nursing homes would better inform health system capacity and planning.

Patient use of pulse oximetry to support management of COVID-19 in Greater Manchester: A non-randomised evaluation using a target trial approach

INTRODUCTION

The pandemic saw widespread use of home pulse oximeters to patients diagnosed with COVID-19 to support early detection of low oxygen saturation levels and appropriate care. Rapid implementation made conventional evaluation challenging, highlighting the need for rigorous non-randomised methods to support decision-making about future use of these technologies. We used routine data to explore the benefits of pulse oximetry in Greater Manchester, under the 'COVID-19 oximetry at home' (CO@h) programme.

METHODS

We used data from the Greater Manchester Secure Data Environment and defined study parameters using a 'target trial' model to compare patients receiving pulse oximetry under the CO@h programme, with matched controls using various comparator groups. Primary outcomes were unplanned hospitalisation and all-cause mortality. This study is based on data from the Greater Manchester Care Record (GMCR), using anonymised, routinely collected data provided in a de-identified format for research. Informed written consent is needed for primary care patient data to be collected for service improvement and research, before data extraction to the GMCR. The study was approved under protocol GMCR RQ-048, on 12/05/2022. As indicated by the University of Manchester ethics decision tool, formal ethical approval was not required for this study.

RESULTS

The adjusted odds ratios for an unplanned hospitalisation were higher among patients receiving pulse oximetry: OR 1.86 (95% CI 1.54-2.25) at 28 days, 1.5 (95% CI 1.3-1.74) at 90 days and 1.63 (95% CI 1.44-1.83) at 1 year. Overall odds of mortality were lower among patients receiving pulse oximetry: adjusted ORs of 0.5 (95% CI 0.25-0.98) at 28 days, 0.5 (95% CI 0.32-0.78) at 90 days and 0.58 (95% CI 0.44-0.76) at 1 year. The results were robust to different comparison groups.

CONCLUSION

Use of pulse oximetry at home under the CO@h programme, through the resulting prioritisation for appropriate care, was associated with a higher frequency of unplanned admissions and a reduction in the risk of mortality up to 1 year later. Therefore, it is likely effective for early detection of clinical deterioration and timely intervention among patients with COVID-19. Further research is needed to understand whether this is a cost-effective use of healthcare resources.

Conducting observational analyses with the target trial emulation approach: a methodological systematic review

OBJECTIVES

Target trial emulation is an approach that is increasingly used to improve transparency in observational studies and help mitigate biases. For studies declaring that they emulated a target trial, we aimed to evaluate the specification of the target trial, examine its consistency with the observational emulation and assess the risk of bias in the observational analysis.

DESIGN

Methodological systematic review reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.

DATA SOURCES

The database MEDLINE (Medical Literature Analysis and Retrieval System Online) was interrogated for all studies published from 1 January 2021 to 3 July 2022. We performed an additional manual search of 20 general medical and specialised journals that spanned the same period.

ELIGIBILITY CRITERIA

All studies that declared emulating a hypothetical or real randomised trial were eligible.

DATA EXTRACTION AND SYNTHESIS

Two independent reviewers performed the whole systematic review process (screening and selection of studies, data extraction and risk of bias assessment). The main outcomes were the definition of the key protocol components of the target trial and its emulation, consistency between the target trial and its emulation and risk of bias according to the ROBINS-I (Risk Of Bias In Non-randomised Studies - of Interventions) tool.

RESULTS

Among the selected sample of 100 studies, 24 (24%) did not specify the target trial. Only 40 studies (40%) provided detailed information on all components of the target trial protocol. Eligibility criteria, intervention strategies and outcomes were consistent between the target trial and its emulation in 35 studies (46% of those specifying the target trial). Overall, 28 studies (28%) exhibited serious risk of bias and 41 (41%) had misalignments in the timing of eligibility assessment, treatment assignment and the start of follow-up (time-zero). As compared with studies that did not specify the target trial, those that did specify the trial less frequently seemed to have both time-zero issues (39% vs 52%) and serious risk of bias (26% vs 33%).

CONCLUSIONS

One-quarter of studies declaring that they emulated a target trial did not specify the trial. Target trials and their emulations were particularly inconsistent for studies emulating a real randomised trial. Risk of methodological issues seemed lower in observational analyses that specified versus did not specify the target trial.

Ethical challenges for Health Technology Assessment (HTA) in the evolving evidence landscape

Since its inception, Health Technology Assessment (HTA) has typically determined the value of a technology by collecting information derived from randomized clinical trials (RCTs), in line with the principles of evidence-based medicine (EBM). However, data from RCTs did not constitute the sole source of information, as other types of evidence (such as primary qualitative research) have often been utilized. Recent advances in both generating and collecting other types of evidence are broadening the landscape of evidence, adding complexity to the discussion of "robustness of evidence." What are the consequences of these recent developments for the methodology and conduct of HTA, the HTA community, and its ethical commitments? The aim of this article is to explore some ethical challenges that are emerging in the current evolving evidence landscape, particularly changes in evidence generation and collection (e.g., diversification of data sources), and shifting standards of evidence in the field of HTA (e.g., increasing acceptability of evidence that is thought of as lower quality). Our conclusion is that deciding how to best maintain trustworthiness is common to all these issues.

Statistical analysis of observational studies in disability research

Observational studies have a critical role in disability research, providing the opportunity to address a range of research questions. Over the past decades, there have been substantial shifts and developments in statistical methods for observational studies, most notably for causal inference. In this review, we provide an overview of modern design and analysis concepts critical for observational studies, drawing examples from the field of disability research and highlighting the challenges in this field, to inform the readership on important statistical considerations for their studies. WHAT THIS PAPER ADDS: Descriptive research questions have specific analytical complexities, so careful statistical design before analysis is critical. Prediction research aims to produce a model with good predictive ability and requires thorough statistical design prior to analysis. Causal research requires careful statistical analysis planning, facilitated by modern causal inference concepts and analytical methods. Adopting these approaches will strengthen the quality of observational studies addressing a range of research questions in the disability space.

Target Trial Emulation for Evaluating Health Policy

Target trial emulation is an approach to designing rigorous nonexperimental studies by "emulating" key features of a clinical trial. Most commonly used outside of policy contexts, this approach is also valuable for policy evaluation as policies typically are not randomly assigned. In this article, we discuss the application of the target trial emulation framework in a policy evaluation context. The policy trial emulation framework includes 7 components: the units and eligibility criteria, definitions of the exposure and comparison conditions, assignment mechanism, baseline ("time zero") and follow-up, outcomes, causal estimand, and statistical analysis and assumptions. Policy evaluations that emulate a randomized trial across these dimensions can yield estimates of the causal effects of the policy on outcomes. Using the policy trial emulation framework to conduct and report on research design and methods supports transparent assessment of threats to causal inference in nonexperimental studies intended to assess the effect of a health policy on clinical or population health outcomes.

Fusion versus decompression alone for lumbar degenerative spondylolisthesis and spinal stenosis: a target trial emulation with index trial benchmarking

PURPOSE

The value of adding fusion to decompression surgery for lumbar degenerative spondylolisthesis and spinal canal stenosis remains debated. Therefore, the comparative effectiveness and selected healthcare resource utilization of patients undergoing decompression with or without fusion surgery at 3 years follow-up was assessed.

METHODS

Using observational data from the Lumbar Stenosis Outcome Study and a target trial emulation with index trial benchmarking approach, our study assessed the comparative effectiveness of the two main surgical interventions for lumbar degenerative spondylolisthesis-fusion and decompression alone in patients with lumbar degenerative spondylolisthesis and spinal canal stenosis. The primary outcome-measure was change in health-related quality of life (EuroQol Health Related Quality of Life 5-Dimension 3-Level questionnaire [EQ-5D-3L]); secondary outcome measures were change in back/leg pain intensity (Numeric Rating Scale), change in satisfaction (Spinal Stenosis Measure satisfaction subscale), physical therapy and oral analgesic use (healthcare utilization).

RESULTS

153 patients underwent decompression alone and 62 had decompression plus fusion. After inverse probability weighting, 137 patients were included in the decompression alone group (mean age, 73.9 [7.5] years; 77 female [56%]) and 36 in the decompression plus fusion group (mean age, 70.1 [6.7] years; 18 female [50%]). Our findings were compatible with no standardized mean differences in EQ-5D-3L summary index change score at 3 years (EQ-5D-3L German: 0.07 [95% confidence interval (CI), - 0.25 to 0.39]; EQ-5D-3L French: 0.18 [95% CI, - 0.14 to 0.50]). No between-group differences in change in back/leg pain intensity or satisfaction were found. Decompression plus fusion was associated with greater physical therapy utilization at 3 years follow-up.

CONCLUSION

Decompression alone should be considered the primary option for patients with lumbar degenerative spondylolisthesis and spinal stenosis.

Characteristics of non-randomised studies of drug treatments: cross sectional study

ABSTRACT

Objective

To examine the characteristics of comparative non-randomised studies that assess the effectiveness or safety, or both, of drug treatments.

Design

Cross sectional study.

Data sources

Medline (Ovid), for reports published from 1 June 2022 to 31 August 2022.

Eligibility criteria for selecting studies

Reports of comparative non-randomised studies that assessed the effectiveness or safety, or both, of drug treatments were included. A randomly ordered sample was screened until 200 eligible reports were found. Data on general characteristics, reporting characteristics, and time point alignment were extracted, and possible related biases, with a piloted form inspired by reporting guidelines and the target trial emulation framework.

Results

Of 462 reports of non-randomised studies identified, 262 studies were excluded (32% had no comparator and 25% did not account for confounding factors). To assess time point alignment and possible related biases, three study time points were considered: eligibility, treatment assignment, and start of follow-up. Of the 200 included reports, 70% had one possible bias, related to: inclusion of prevalent users in 24%, post-treatment eligibility criteria in 32%, immortal time periods in 42%, and classification of treatment in 23%. Reporting was incomplete, and only 2% reported all six of the key elements considered: eligibility criteria (87%), description of treatment (46%), deviations in treatment (27%), causal contrast (11%), primary outcomes (90%), and confounding factors (88%). Most studies used routinely collected data (67%), but only 7% reported using validation studies of the codes or algorithms applied to select the population. Only 7% of reports mentioned registration on a trial registry and 3% had an available protocol.

Conclusions

The findings of the study suggest that although access to real world evidence could be valuable, the robustness and transparency of non-randomised studies need to be improved.

Associations Between Reductions in Depressive Symptoms and Reductions in Pain and Anxiety Symptoms and Substance Use: Emulation of a Randomized Trial

Introduction

Depressive symptoms are linked with pain, anxiety, and substance use. Research estimating whether a reduction in depressive symptoms is linked to subsequent reductions in pain and anxiety symptoms and substance use is limited.

Methods

Using data from the Veterans Aging Cohort Study, a multisite observational study of U.S. veterans, the authors used a target trial emulation framework to compare individuals with elevated depressive symptoms (Patient Health Questionnaire-9 score ≥ 10) who experienced reductions in depressive symptoms (Patient Health Questionnaire-9 score < 10) with those whose symptoms persisted (Patient Health Questionnaire-9 score ≥ 10) at the next follow-up visit (on average, 1 year later). Using inverse probability of treatment weighting, the authors estimated ORs and 95% CIs for associations between depressive symptom reduction status and improvement on the following: anxiety symptoms, pain symptoms, unhealthy alcohol use, and use of tobacco, cannabis, cocaine, and/or illicit opioids.

Results

Reductions in depressive symptoms were associated with reductions in pain symptoms (OR=1.43, 95% CI=1.01, 2.02), anxiety symptoms (OR=2.50, 95% CI=1.63, 3.83), and illicit opioid use (OR=2.07, 95% CI=1.13, 3.81). Depressive symptom reductions were not associated with reductions in unhealthy alcohol use (OR=0.85, 95% CI=0.48, 1.52) or use of tobacco (OR=1.49, 95% CI=0.89, 2.48), cannabis (OR=1.07, 95% CI=0.63, 1.83), or cocaine (OR=1.28, 95% CI=0.73, 2.24).

Conclusions

Reducing depressive symptoms may potentially reduce pain and anxiety symptoms and illicit opioid use. Future work should determine whether reductions achieved through antidepressant medications, behavioral therapy, or other means have comparable impact.

Treatment Effects in Randomized and Nonrandomized Studies of Pharmacological Interventions: A Meta-Analysis

Importance

Randomized clinical trials (RCTs) are widely regarded as the methodological benchmark for assessing clinical efficacy and safety of health interventions. There is growing interest in using nonrandomized studies to assess efficacy and safety of new drugs.

Objective

To determine how treatment effects for the same drug compare when evaluated in nonrandomized vs randomized studies.

Data Sources

Meta-analyses published between 2009 and 2018 were identified in MEDLINE via PubMed and the Cochrane Database of Systematic Reviews. Data analysis was conducted from October 2019 to July 2024.

Study Selection

Meta-analyses of pharmacological interventions were eligible for inclusion if both randomized and nonrandomized studies contributed to a single meta-analytic estimate.

Data Extraction and Synthesis

For this meta-analysis using a meta-epidemiological framework, separate summary effect size estimates were calculated for nonrandomized and randomized studies within each meta-analysis using a random-effects model and then these estimates were compared. The reporting of this study followed the Guidelines for Reporting Meta-Epidemiological Methodology Research and relevant portions of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline.

Main Outcome and Measures

The primary outcome was discrepancies in treatment effects obtained from nonrandomized and randomized studies, as measured by the proportion of meta-analyses where the 2 study types disagreed about the direction or magnitude of effect, disagreed beyond chance about the effect size estimate, and the summary ratio of odds ratios (ROR) obtained from nonrandomized vs randomized studies combined across all meta-analyses.

Results

A total of 346 meta-analyses with 2746 studies were included. Statistical conclusions about drug benefits and harms were different for 130 of 346 meta-analyses (37.6%) when focusing solely on either nonrandomized or randomized studies. Disagreements were beyond chance for 54 meta-analyses (15.6%). Across all meta-analyses, there was no strong evidence of consistent differences in treatment effects obtained from nonrandomized vs randomized studies (summary ROR, 0.95; 95% credible interval [CrI], 0.89-1.02). Compared with experimental nonrandomized studies, randomized studies produced on average a 19% smaller treatment effect (ROR, 0.81; 95% CrI, 0.68-0.97). There was increased heterogeneity in effect size estimates obtained from nonrandomized compared with randomized studies.

Conclusions and Relevance

In this meta-analysis of treatment effects of pharmacological interventions obtained from randomized and nonrandomized studies, there was no overall difference in effect size estimates between study types on average, but nonrandomized studies both overestimated and underestimated treatment effects observed in randomized studies and introduced additional uncertainty. These findings suggest that relying on nonrandomized studies as substitutes for RCTs may introduce additional uncertainty about the therapeutic effects of new drugs.

Replacement Strategies for Tunneled Hemodialysis Catheters with Complications: A Nationwide Cohort Study

Key Points

Replacement strategies for hemodialysis catheters with mechanical failure differed widely between services, which suggests clinical equipoise. For mechanical hemodialysis catheter failure, exchange did not result in more dysfunction or infection than removal and separate replacement. In Australia, infected catheters were almost universally removed and then replaced through a separate tunnel tract.

Background

Tunneled hemodialysis catheters often have infectious or mechanical complications that require unplanned removal and replacement, but the optimal replacement strategy is unknown. This study described the real-world use of two strategies in Australia and compared the survival of replacement catheters inserted by either strategy.

Methods

Observational data from the REDUcing the burden of dialysis Catheter ComplicaTIOns: a National approach trial, which enrolled a nationwide cohort of 6400 adults who received an incident hemodialysis catheter (2016–2020), was used for this secondary analysis. Tunneled catheters were replaced by either catheter exchange through the existing tunnel tract or removal and replacement through a new tract. The effect of the replacement strategy on the time to catheter removal because of infection or dysfunction was estimated by emulating a hypothetical pragmatic randomized trial among a subset of 434 patients with mechanical tunneled catheter failure.

Results

Of 9974 tunneled hemodialysis catheters inserted during the trial, 380 had infectious and 945 had mechanical complications that required replacement. Almost all infected hemodialysis catheters (97%) were removed and separately replaced through a new tunnel tract, whereas nephrology services differed widely in their replacement practices for catheters with mechanical failure (median=50% guidewire exchanged, interquartile range=30%–67%). Service-level differences accounted for 29% of the residual variation after adjusting for patient factors. In the target trial emulation cohort of patients with mechanical catheter failure (n =434 patients), catheter exchange was not associated with lower complication-free survival at 1, 6, or 12 months (counterfactual survival difference at 1 month=5.9%; 95% confidence interval, −2% to 14%).

Conclusions

Guidewire exchange for mechanical failure of catheter was not associated with lower catheter survival and may be preferable for patients.

Trial registration and protocol:

The trial was registered in the Australia and New Zealand clinical trials registry on the June 23, 2016 (ACTRN12616000830493).

Zoledronic acid for hip fracture during initial hospitalization

Inpatient zoledronic acid (IP-ZA) administered during the initial fracture hospitalization significantly improves the osteoporosis treatment rate. Clinical outcomes of IP-ZA after hip fracture remain uncertain. Here we report a cohort study that emulated a randomized controlled trial using real-world data and evaluated the risk of all-cause-mortality and radiologically confirmed subsequent new fractures among patients hospitalized for a hip fracture who had received IP-ZA as compared with propensity-matched controls. A total of 654 patients who had received IP-ZA and 6877 controls (for whom anti-osteoporosis treatment was indicated but no IP-ZA started during index hospitalization) were included in the study. The primary cohort comprised 652 IP-ZA patients (IP-ZA group) and 1926 matched controls (untreated group), with 71.7% female 92.1% White participants, with a mean age of 80.9 years. Cumulative all-cause mortality over the 24-month follow-up for the IP-ZA group was 12.3% and 20.7% for the untreated group (hazard ratio [HR], 0.62; 95% CI, 0.49-0.78, p < .001). A total of 585 (89.7%) patients in IP-ZA group received only a single dose of ZA during the 24 months, and the death rate of this single dose group was 13.3%, which was significantly lower than that of the untreated group (HR, 0.70; 95% CI, 0.55-0.89, p = .003). Rates of radiologically confirmed cumulative subsequent new vertebral fractures were 2.0% in the IP-ZA group and 5.4% in the untreated group (HR, 0.40; 95% CI, 0.22-0.71, p = .001). A similarly lower rate of new vertebral fractures was seen in the single dose subgroup (1.9% vs 5.4%; HR, 0.44; 95% 0.24-0.82, p = .008). IP-ZA, administered during the initial hospitalization for hip fracture, was associated with lower all-cause-mortality and risk of radiologically confirmed subsequent new vertebral fractures, and thus offers a mechanism to narrow the treatment gap in patients having sustained a hip fragility fracture.

Designing target trials using electronic health records: A case study of second-line disease-modifying anti-rheumatic drugs and cardiovascular disease outcomes in patients with rheumatoid arthritis

BACKGROUND

Emulation of the "target trial" (TT), a hypothetical pragmatic randomized controlled trial (RCT), using observational data can be used to mitigate issues commonly encountered in comparative effectiveness research (CER) when randomized trials are not logistically, ethically, or financially feasible. However, cardiovascular (CV) health research has been slow to adopt TT emulation. Here, we demonstrate the design and analysis of a TT emulation using electronic health records to study the comparative effectiveness of the addition of a disease-modifying anti-rheumatic drug (DMARD) to a regimen of methotrexate on CV events among rheumatoid arthritis (RA) patients.

METHODS

We used data from an electronic medical records-based cohort of RA patients from Northwestern Medicine to emulate the TT. Follow-up began 3 months after initial prescription of MTX (2000-2020) and included all available follow-up through June 30, 2020. Weighted pooled logistic regression was used to estimate differences in CVD risk and survival. Cloning was used to handle immortal time bias and weights to improve baseline and time-varying covariate imbalance.

RESULTS

We identified 659 eligible people with RA with average follow-up of 46 months and 31 MACE events. The month 24 adjusted risk difference for MACE comparing initiation vs non-initiation of a DMARD was -1.47% (95% confidence interval [CI]: -4.74, 1.95%), and the marginal hazard ratio (HR) was 0.72 (95% CI: 0.71, 1.23). In analyses subject to immortal time bias, the HR was 0.62 (95% CI: 0.29-1.44).

CONCLUSION

In this sample, we did not observe evidence of differences in risk of MACE, a finding that is compatible with previously published meta-analyses of RCTs. Thoughtful application of the TT framework provides opportunities to conduct CER in observational data. Benchmarking results of observational analyses to previously published RCTs can lend credibility to interpretation.

Comparative effectiveness of second line oral antidiabetic treatments among people with type 2 diabetes mellitus: emulation of a target trial using routinely collected health data

OBJECTIVE

To compare the effectiveness of three commonly prescribed oral antidiabetic drugs added to metformin for people with type 2 diabetes mellitus requiring second line treatment in routine clinical practice.

DESIGN

Cohort study emulating a comparative effectiveness trial (target trial).

SETTING

Linked primary care, hospital, and death data in England, 2015-21.

PARTICIPANTS

75 739 adults with type 2 diabetes mellitus who initiated second line oral antidiabetic treatment with a sulfonylurea, DPP-4 inhibitor, or SGLT-2 inhibitor added to metformin.

MAIN OUTCOME MEASURES

Primary outcome was absolute change in glycated haemoglobin A1c (HbA1c) between baseline and one year follow-up. Secondary outcomes were change in body mass index (BMI), systolic blood pressure, and estimated glomerular filtration rate (eGFR) at one year and two years, change in HbA1c at two years, and time to ≥40% decline in eGFR, major adverse kidney event, hospital admission for heart failure, major adverse cardiovascular event (MACE), and all cause mortality. Instrumental variable analysis was used to reduce the risk of confounding due to unobserved baseline measures.

RESULTS

75 739 people initiated second line oral antidiabetic treatment with sulfonylureas (n=25 693, 33.9%), DPP-4 inhibitors (n=34 464 ,45.5%), or SGLT-2 inhibitors (n=15 582, 20.6%). SGLT-2 inhibitors were more effective than DPP-4 inhibitors or sulfonylureas in reducing mean HbA1c values between baseline and one year. After the instrumental variable analysis, the mean differences in HbA1c change between baseline and one year were -2.5 mmol/mol (95% confidence interval (CI) -3.7 to -1.3) for SGLT-2 inhibitors versus sulfonylureas and -3.2 mmol/mol (-4.6 to -1.8) for SGLT-2 inhibitors versus DPP-4 inhibitors. SGLT-2 inhibitors were more effective than sulfonylureas or DPP-4 inhibitors in reducing BMI and systolic blood pressure. For some secondary endpoints, evidence for SGLT-2 inhibitors being more effective was lacking-the hazard ratio for MACE, for example, was 0.99 (95% CI 0.61 to 1.62) versus sulfonylureas and 0.91 (0.51 to 1.63) versus DPP-4 inhibitors. SGLT-2 inhibitors had reduced hazards of hospital admission for heart failure compared with DPP-4 inhibitors (0.32, 0.12 to 0.90) and sulfonylureas (0.46, 0.20 to 1.05). The hazard ratio for a ≥40% decline in eGFR indicated a protective effect versus sulfonylureas (0.42, 0.22 to 0.82), with high uncertainty in the estimated hazard ratio versus DPP-4 inhibitors (0.64, 0.29 to 1.43).

CONCLUSIONS

This emulation study of a target trial found that SGLT-2 inhibitors were more effective than sulfonylureas or DPP-4 inhibitors in lowering mean HbA1c, BMI, and systolic blood pressure and in reducing the hazards of hospital admission for heart failure (v DPP-4 inhibitors) and kidney disease progression (v sulfonylureas), with no evidence of differences in other clinical endpoints.

Prospective benchmarking of an observational analysis in the SWEDEHEART registry against the REDUCE-AMI randomized trial

Prospective benchmarking of an observational analysis against a randomized trial increases confidence in the benchmarking process as it relies exclusively on aligning the protocol of the trial and the observational analysis, while the trials findings are unavailable. The Randomized Evaluation of Decreased Usage of Betablockers After Myocardial Infarction (REDUCE-AMI, ClinicalTrials.gov ID: NCT03278509) trial started recruitment in September 2017 and results are expected in 2024. REDUCE-AMI aimed to estimate the effect of long-term use of beta blockers on the risk of death and myocardial following a myocardial infarction with preserved left ventricular systolic ejection fraction. We specified the protocol of a target trial as similar as possible to that of REDUCE-AMI, then emulated the target trial using observational data from Swedish healthcare registries. Had everyone followed the treatment strategy as specified in the target trial protocol, the observational analysis estimated a reduction in the 5-year risk of death or myocardial infarction of 0.8 percentage points for beta blockers compared with no beta blockers; effects ranging from an absolute reduction of 4.5 percentage points to an increase of 2.8 percentage points in the risk of death or myocardial infarction were compatible with our data under conventional statistical criteria. Once results of REDUCE-AMI are published, we will compare the results of our observational analysis against those from the trial. If this prospective benchmarking is successful, it supports the credibility of additional analyses using these observational data, which can rapidly deliver answers to questions that could not be answered by the initial trial. If benchmarking proves unsuccessful, we will conduct a "postmortem" analysis to identify the reasons for the discrepancy. Prospective benchmarking shifts the investigator focus away from an endeavour to use observational data to obtain similar results as a completed randomized trial, to a systematic attempt to align the design and analysis of the trial and the observational analysis.

Methodological biases in observational hospital studies of COVID-19 treatment effectiveness: pitfalls and potential

Introduction

This study aims to discuss and assess the impact of three prevalent methodological biases: competing risks, immortal-time bias, and confounding bias in real-world observational studies evaluating treatment effectiveness. We use a demonstrative observational data example of COVID-19 patients to assess the impact of these biases and propose potential solutions.

Methods

We describe competing risks, immortal-time bias, and time-fixed confounding bias by evaluating treatment effectiveness in hospitalized patients with COVID-19. For our demonstrative analysis, we use observational data from the registry of patients with COVID-19 who were admitted to the Bellvitge University Hospital in Spain from March 2020 to February 2021 and met our predefined inclusion criteria. We compare estimates of a single-dose, time-dependent treatment with the standard of care. We analyze the treatment effectiveness using common statistical approaches, either by ignoring or only partially accounting for the methodological biases. To address these challenges, we emulate a target trial through the clone-censor-weight approach.

Results

Overlooking competing risk bias and employing the naïve Kaplan-Meier estimator led to increased in-hospital death probabilities in patients with COVID-19. Specifically, in the treatment effectiveness analysis, the Kaplan-Meier estimator resulted in an in-hospital mortality of 45.6% for treated patients and 59.0% for untreated patients. In contrast, employing an emulated trial framework with the weighted Aalen-Johansen estimator, we observed that in-hospital death probabilities were reduced to 27.9% in the "X"-treated arm and 40.1% in the non-"X"-treated arm. Immortal-time bias led to an underestimated hazard ratio of treatment.

Conclusion

Overlooking competing risks, immortal-time bias, and confounding bias leads to shifted estimates of treatment effects. Applying the naïve Kaplan-Meier method resulted in the most biased results and overestimated probabilities for the primary outcome in analyses of hospital data from COVID-19 patients. This overestimation could mislead clinical decision-making. Both immortal-time bias and confounding bias must be addressed in assessments of treatment effectiveness. The trial emulation framework offers a potential solution to address all three methodological biases.