Effect Estimates in Randomized Trials and Observational Studies: Comparing Apples With Apples

Acceptability of Using Real-World Data to Estimate Relative Treatment Effects in Health Technology Assessments: Barriers and Future Steps

OBJECTIVES

Evidence about the comparative effects of new treatments is typically collected in randomized controlled trials (RCTs). In some instances, RCTs are not possible, or their value is limited by an inability to capture treatment effects over the longer term or in all relevant population subgroups. In these cases, nonrandomized studies (NRS) using real-world data (RWD) are increasingly used to complement trial evidence on treatment effects for health technology assessment (HTA). However, there have been concerns over a lack of acceptability of this evidence by HTA agencies. This article aims to identify the barriers to the acceptance of NRS and steps that may facilitate increases in the acceptability of NRS in the future.

METHODS

Opinions of the authorship team based on their experience in real-world evidence research in academic, HTA, and industry settings, supported by a critical assessment of existing studies.

RESULTS

Barriers were identified that are applicable to key stakeholder groups, including HTA agencies (eg, the lack of comprehensive methodological guidelines for using RWD), evidence generators (eg, avoidable deviations from best practices), and external stakeholders (eg, data controllers providing timely access to high-quality RWD). Future steps that may facilitate future acceptability of NRS include improvements in the quality, integration, and accessibility of RWD, wider use of demonstration projects to highlight the value and applicability of nonrandomized designs, living, and more detailed HTA guidelines, and improvements in HTA infrastructure relating to RWD.

CONCLUSION

NRS can represent a crucial source of evidence on treatment effects for use in HTA when RCT evidence is limited.

Prospective benchmarking of an observational analysis in the SWEDEHEART registry against the REDUCE-AMI randomized trial

Prospective benchmarking of an observational analysis against a randomized trial increases confidence in the benchmarking process as it relies exclusively on aligning the protocol of the trial and the observational analysis, while the trials findings are unavailable. The Randomized Evaluation of Decreased Usage of Betablockers After Myocardial Infarction (REDUCE-AMI, ClinicalTrials.gov ID: NCT03278509) trial started recruitment in September 2017 and results are expected in 2024. REDUCE-AMI aimed to estimate the effect of long-term use of beta blockers on the risk of death and myocardial following a myocardial infarction with preserved left ventricular systolic ejection fraction. We specified the protocol of a target trial as similar as possible to that of REDUCE-AMI, then emulated the target trial using observational data from Swedish healthcare registries. Had everyone followed the treatment strategy as specified in the target trial protocol, the observational analysis estimated a reduction in the 5-year risk of death or myocardial infarction of 0.8 percentage points for beta blockers compared with no beta blockers; effects ranging from an absolute reduction of 4.5 percentage points to an increase of 2.8 percentage points in the risk of death or myocardial infarction were compatible with our data under conventional statistical criteria. Once results of REDUCE-AMI are published, we will compare the results of our observational analysis against those from the trial. If this prospective benchmarking is successful, it supports the credibility of additional analyses using these observational data, which can rapidly deliver answers to questions that could not be answered by the initial trial. If benchmarking proves unsuccessful, we will conduct a "postmortem" analysis to identify the reasons for the discrepancy. Prospective benchmarking shifts the investigator focus away from an endeavour to use observational data to obtain similar results as a completed randomized trial, to a systematic attempt to align the design and analysis of the trial and the observational analysis.

Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data

Objective

To explore how design emulation and population differences relate to variation in results between randomised controlled trials (RCT) and non-randomised real world evidence (RWE) studies, based on the RCT-DUPLICATE initiative (Randomised, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology).

Design

Meta-analysis of RCT-DUPLICATE data.

Data sources

Trials included in RCT-DUPLICATE, a demonstration project that emulated 32 randomised controlled trials using three real world data sources: Optum Clinformatics Data Mart, 2004-19; IBM MarketScan, 2003-17; and subsets of Medicare parts A, B, and D, 2009-17.

Eligibility criteria for selecting studies

Trials where the primary analysis resulted in a hazard ratio; 29 RCT-RWE study pairs from RCT-DUPLICATE.

Results

Differences and variation in effect sizes between the results from randomised controlled trials and real world evidence studies were investigated. Most of the heterogeneity in effect estimates between the RCT-RWE study pairs in this sample could be explained by three emulation differences in the meta-regression model: treatment started in hospital (which does not appear in health insurance claims data), discontinuation of some baseline treatments at randomisation (which would have been an unusual care decision in clinical practice), and delayed onset of drug effects (which would be under-reported in real world clinical practice because of the relatively short persistence of the treatment). Adding the three emulation differences to the meta-regression reduced heterogeneity from 1.9 to almost 1 (absence of heterogeneity).

Conclusions

This analysis suggests that a substantial proportion of the observed variation between results from randomised controlled trials and real world evidence studies can be attributed to differences in design emulation.

Post-hospitalization remote monitoring for patients with heart failure or chronic obstructive pulmonary disease in an accountable care organization

BACKGROUND

Post-hospitalization remote patient monitoring (RPM) has potential to improve health outcomes for high-risk patients with chronic medical conditions. The purpose of this study is to determine the extent to which RPM for patients with congestive heart failure (CHF) and chronic obstructive pulmonary disease (COPD) is associated with reductions in post-hospitalization mortality, hospital readmission, and ED visits within an Accountable Care Organization (ACO).

METHODS

Nonrandomized prospective study of patients in an ACO offered enrollment in RPM upon hospital discharge between February 2021 and December 2021. RPM comprised of vital sign monitoring equipment (blood pressure monitor, scale, pulse oximeter), tablet device with symptom tracking software and educational material, and nurse-provided oversight and triage. Expected enrollment was for at least 30-days of monitoring, and outcomes were followed for 6 months following enrollment. The co-primary outcomes were (a) the composite of death, hospital admission, or emergency care visit within 180 days of eligibility, and (b) time to occurrence of this composite. Secondary outcomes were each component individually, the composite of death or hospital admission, and outpatient office visits. Adjusted analyses involved doubly robust estimation to address confounding by indication.

RESULTS

Of 361 patients offered remote monitoring (251 with CHF and 110 with COPD), 140 elected to enroll (106 with CHF and 34 with COPD). The median duration of RPM-enrollment was 54 days (IQR 34-85). Neither the 6-month frequency of the co-primary composite outcome (59% vs 66%, FDR p-value = 0.47) nor the time to this composite (median 29 vs 38 days, FDR p-value = 0.60) differed between the groups, but 6-month mortality was lower in the RPM group (6.4% vs 17%, FDR p-value = 0.02). After adjustment for confounders, RPM enrollment was associated with nonsignificantly decreased odds for the composite outcome (adjusted OR [aOR] 0.68, 99% CI 0.25-1.34, FDR p-value 0.30) and lower 6-month mortality (aOR 0.41, 99% CI 0.00-0.86, FDR p-value 0.20).

CONCLUSIONS

RPM enrollment may be associated with improved health outcomes, including 6-month mortality, for selected patient populations.

Producing Change to Understand the Social Determinants of Health: The Promise of Experiments for Social Epidemiology

In this commentary, invited for the 100th anniversary of the Journal, we discuss the addition of randomized experiments, along with natural experiments that emulate randomized trials using observational data, as designs in the social epidemiologist's toolbox. These approaches transform the way we define and ask questions about social exposures. They compel us to ask questions about how well-defined interventions change a social exposure that might lead to changes in health. As such, experiments are of unique public health and policy significance. We argue that they are a powerful approach to advance our understanding of how well-defined changes in social exposures impact health, and how credible social policy reforms may be instrumental to address health inequalities. We focus on two research designs. The first is a "pure" randomized controlled trial (RCT) in which the investigator defines and randomly assigns the intervention. The second is a natural experiment, which exploits the fact that policies or interventions in the real world often involve an element of random assignment, emulating an RCT. To give the reader our bottom line: While acknowledging their limits, we continue to be very excited about the promise of RCTs and natural experiments to advance social epidemiology.

Target trial emulation: Do antimicrobials or gastrointestinal nutraceuticals prescribed at first presentation for acute diarrhoea cause a better clinical outcome in dogs under primary veterinary care in the UK?

Target trial emulation applies design principles from randomised controlled trials to the analysis of observational data for causal inference and is increasingly used within human epidemiology. Veterinary electronic clinical records represent a potentially valuable source of information to estimate real-world causal effects for companion animal species. This study employed the target trial framework to evaluate the usefulness on veterinary observational data. Acute diarrhoea in dogs was used as a clinical exemplar. Inclusion required dogs aged ≥ 3 months and < 10 years, presenting for veterinary primary care with acute diarrhoea during 2019. Treatment strategies were: 1. antimicrobial prescription compared to no antimicrobial prescription and 2. gastrointestinal nutraceutical prescription compared to no gastrointestinal nutraceutical prescription. The primary outcome was clinical resolution (defined as no revisit with ongoing diarrhoea within 30 days from the date of first presentation). Informed from a directed acyclic graph, data on the following covariates were collected: age, breed, bodyweight, insurance status, comorbidities, vomiting, reduced appetite, haematochezia, pyrexia, duration, additional treatment prescription and veterinary group. Inverse probability of treatment weighting was used to balance covariates between the treatment groups for each of the two target trials. The risk difference (RD) of 0.4% (95% CI -4.5% to 5.3%) was non-significant for clinical resolution in dogs treated with antimicrobials compared with dogs not treated with antimicrobials. The risk difference (RD) of 0.3% (95% CI -4.5% to 5.0%) was non-significant for clinical resolution in dogs treated with gastrointestinal nutraceuticals compared with dogs not treated with gastrointestinal nutraceuticals. This study successfully applied the target trial framework to veterinary observational data. The findings show that antimicrobial or gastrointestinal prescription at first presentation of acute diarrhoea in dogs causes no difference in clinical resolution. The findings support the recommendation for veterinary professionals to limit antimicrobial use for acute diarrhoea in dogs.

Emulating Target Trials With Real-World Data to Inform Health Technology Assessment: Findings and Lessons From an Application to Emergency Surgery

OBJECTIVES

International health technology assessment (HTA) agencies recommend that real-world data (RWD) are used in some circumstances to add to the evidence base about the effectiveness and cost-effectiveness of health interventions. The target trial framework applies the design principles of randomized-controlled trials to RWD and can help alleviate inevitable concerns about bias and design flaws with nonrandomized studies. This article aimed to tackle the lack of guidance and exemplar applications on how this methodology can be applied to RWD to inform HTA decision making.

METHODS

We use Hospital Episode Statistics data from England on emergency hospital admissions from 2010 to 2019 to evaluate the cost-effectiveness of emergency surgery for 2 acute gastrointestinal conditions. We draw on the case study to describe the main challenges in applying the target trial framework alongside RWD and provide recommendations for how these can be addressed in practice.

RESULTS

The 4 main challenges when applying the target trial framework to RWD are (1) defining the study population, (2) defining the treatment strategies, (3) establishing time zero (baseline), and (4) adjusting for unmeasured confounding. The recommendations for how to address these challenges, mainly around the incorporation of expert judgment and use of appropriate methods for handling unmeasured confounding, are illustrated within the case study.

CONCLUSIONS

The recommendations outlined in this study could help future studies seeking to inform HTA decision processes. These recommendations can complement checklists for economic evaluations and design tools for estimating treatment effectiveness in nonrandomized studies.

Target Trial Emulation to Improve Causal Inference from Observational Data: What, Why, and How?

ABSTRACT

Target trial emulation has drastically improved the quality of observational studies investigating the effects of interventions. Its ability to prevent avoidable biases that have plagued many observational analyses has contributed to its recent popularity. This review explains what target trial emulation is, why it should be the standard approach for causal observational studies that investigate interventions, and how to do a target trial emulation analysis. We discuss the merits of target trial emulation compared with often used, but biased analyses, as well as potential caveats, and provide clinicians and researchers with the tools to better interpret results from observational studies investigating the effects of interventions.

Where next for partial randomisation of research funding? The feasibility of RCTs and alternatives

We outline essential considerations for any study of partial randomisation of research funding, and consider scenarios in which randomised controlled trials (RCTs) would be feasible and appropriate. We highlight the interdependence of target outcomes, sample availability and statistical power for determining the cost and feasibility of a trial. For many choices of target outcome, RCTs may be less practical and more expensive than they at first appear (in large part due to issues pertaining to sample size and statistical power). As such, we briefly discuss alternatives to RCTs. It is worth noting that many of the considerations relevant to experiments on partial randomisation may also apply to other potential experiments on funding processes (as described in The Experimental Research Funder's Handbook. RoRI, June 2022).

DESIGN DIFFERENCES EXPLAIN VARIATION IN RESULTS BETWEEN RANDOMIZED TRIALS AND THEIR NON-RANDOMIZED EMULATIONS

Objectives

While randomized controlled trials (RCTs) are considered a standard for evidence on the efficacy of medical treatments, non-randomized real-world evidence (RWE) studies using data from health insurance claims or electronic health records can provide important complementary evidence. The use of RWE to inform decision-making has been questioned because of concerns regarding confounding in non-randomized studies and the use of secondary data. RCT-DUPLICATE was a demonstration project that emulated the design of 32 RCTs with non-randomized RWE studies. We sought to explore how emulation differences relate to variation in results between the RCT-RWE study pairs.

Methods

We include all RCT-RWE study pairs from RCT-DUPLICATE where the measure of effect was a hazard ratio and use exploratory meta-regression methods to explain differences and variation in the effect sizes between the results from the RCT and the RWE study. The considered explanatory variables are related to design and population differences.

Results

Most of the observed variation in effect estimates between RCT-RWE study pairs in this sample could be explained by three emulation differences in the meta-regression model: (i) in-hospital start of treatment (not observed in claims data), (ii) discontinuation of certain baseline therapies at randomization (not part of clinical practice), (iii) delayed onset of drug effects (missed by short medication persistence in clinical practice).

Conclusions

This analysis suggests that a substantial proportion of the observed variation between results from RCTs and RWE studies can be attributed to design emulation differences. (238 words).

Use of Real-World Data in Population Science to Improve the Prevention and Care of Diabetes-Related Outcomes

The past decade of population research for diabetes has seen a dramatic proliferation of the use of real-world data (RWD) and real-world evidence (RWE) generation from non-research settings, including both health and non-health sources, to influence decisions related to optimal diabetes care. A common attribute of these new data is that they were not collected for research purposes yet have the potential to enrich the information around the characteristics of individuals, risk factors, interventions, and health effects. This has expanded the role of subdisciplines like comparative effectiveness research and precision medicine, new quasi-experimental study designs, new research platforms like distributed data networks, and new analytic approaches for clinical prediction of prognosis or treatment response. The result of these developments is a greater potential to progress diabetes treatment and prevention through the increasing range of populations, interventions, outcomes, and settings that can be efficiently examined. However, this proliferation also carries an increased threat of bias and misleading findings. The level of evidence that may be derived from RWD is ultimately a function of the data quality and the rigorous application of study design and analysis. This report reviews the current landscape and applications of RWD in clinical effectiveness and population health research for diabetes and summarizes opportunities and best practices in the conduct, reporting, and dissemination of RWD to optimize its value and limit its drawbacks.

Current practices in studies applying the target trial emulation framework: a protocol for a systematic review

INTRODUCTION

Observational studies represent an alternative to estimate real-world causal effects in the absence of available randomised controlled trials (RCTs). Target trial emulation is a framework for the application of RCT design principles to emulate a hypothetical open-label RCT (the hypothetical target trial) using existing observational data as the primary data source as opposed to the prospective recruitment and measurement of randomised units. The aim of this systematic review is to investigate the practices of studies applying the target trial emulation framework to evaluate the effectiveness of interventions.

METHODS AND ANALYSIS

We will systematically search in Medline (via Ovid), Embase (via Ovid, entries from medRxiv are included), PsycINFO (via Ovid), SCOPUS, Web of Science, Cochrane Library, the ISRCTN registry and ClinicalTrials.gov for all study reports and protocols which used the trial emulation framework (without time restriction). We will extract information concerning study design, data source, analysis, results, interpretation and dissemination. Two reviewers will perform study selection, data extraction and quality assessment. Disagreements between reviewers will be resolved by a third reviewer. A narrative approach will be used to synthesise and report qualitative and quantitative data. Reporting of the review will be informed by Preferred Reporting Items for Systematic Review and Meta-Analysis guidance (PRISMA).

ETHICS AND DISSEMINATION

Ethical approval is not required as it is a protocol for a systematic review. Findings will be disseminated through peer-reviewed publications and conference presentations.

Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses: Results of 32 Clinical Trials

Importance

Nonrandomized studies using insurance claims databases can be analyzed to produce real-world evidence on the effectiveness of medical products. Given the lack of baseline randomization and measurement issues, concerns exist about whether such studies produce unbiased treatment effect estimates.

Objective

To emulate the design of 30 completed and 2 ongoing randomized clinical trials (RCTs) of medications with database studies using observational analogues of the RCT design parameters (population, intervention, comparator, outcome, time [PICOT]) and to quantify agreement in RCT-database study pairs.

Design, Setting, and Participants

New-user cohort studies with propensity score matching using 3 US claims databases (Optum Clinformatics, MarketScan, and Medicare). Inclusion-exclusion criteria for each database study were prespecified to emulate the corresponding RCT. RCTs were explicitly selected based on feasibility, including power, key confounders, and end points more likely to be emulated with real-world data. All 32 protocols were registered on ClinicalTrials.gov before conducting analyses. Emulations were conducted from 2017 through 2022.

Exposures

Therapies for multiple clinical conditions were included.

Main Outcomes and Measures

Database study emulations focused on the primary outcome of the corresponding RCT. Findings of database studies were compared with RCTs using predefined metrics, including Pearson correlation coefficients and binary metrics based on statistical significance agreement, estimate agreement, and standardized difference.

Results

In these highly selected RCTs, the overall observed agreement between the RCT and the database emulation results was a Pearson correlation of 0.82 (95% CI, 0.64-0.91), with 75% meeting statistical significance, 66% estimate agreement, and 75% standardized difference agreement. In a post hoc analysis limited to 16 RCTs with closer emulation of trial design and measurements, concordance was higher (Pearson r, 0.93; 95% CI, 0.79-0.97; 94% meeting statistical significance, 88% estimate agreement, 88% standardized difference agreement). Weaker concordance occurred among 16 RCTs for which close emulation of certain design elements that define the research question (PICOT) with data from insurance claims was not possible (Pearson r, 0.53; 95% CI, 0.00-0.83; 56% meeting statistical significance, 50% estimate agreement, 69% standardized difference agreement).

Conclusions and Relevance

Real-world evidence studies can reach similar conclusions as RCTs when design and measurements can be closely emulated, but this may be difficult to achieve. Concordance in results varied depending on the agreement metric. Emulation differences, chance, and residual confounding can contribute to divergence in results and are difficult to disentangle.

Improving efficiency of inference in clinical trials with external control data

Suppose we are interested in the effect of a treatment in a clinical trial. The efficiency of inference may be limited due to small sample size. However, external control data are often available from historical studies. Motivated by an application to Helicobacter pylori infection, we show how to borrow strength from such data to improve efficiency of inference in the clinical trial. Under an exchangeability assumption about the potential outcome mean, we show that the semiparametric efficiency bound for estimating the average treatment effect can be reduced by incorporating both the clinical trial data and external controls. We then derive a doubly robust and locally efficient estimator. The improvement in efficiency is prominent especially when the external control data set has a large sample size and small variability. Our method allows for a relaxed overlap assumption, and we illustrate with the case where the clinical trial only contains a treated group. We also develop doubly robust and locally efficient approaches that extrapolate the causal effect in the clinical trial to the external population and the overall population. Our results also offer a meaningful implication for trial design and data collection. We evaluate the finite-sample performance of the proposed estimators via simulation. In the Helicobacter pylori infection application, our approach shows that the combination treatment has potential efficacy advantages over the triple therapy.

Target Trial Emulation: A Design Tool for Cancer Clinical Trials

PURPOSE

To apply target trial emulation to explore the potential impact of eligibility criteria on the primary outcome of a randomized controlled trial.

METHODS

Simulations of a real-world explanatory trial of transarterial radioembolization for advanced unresectable hepatocellular carcinoma with portal vein invasion were performed to examine the effects of cohort specification on survival outcomes and patient sample size. Simulations comprised 24 different permutations of the trial varied on three disease nonspecific eligibility parameters. Treatment and control arms for these emulated trials were drawn from the National Cancer Database and matched by treatment propensity. Target trial emulation served as the causal framework for this analysis, allowing the architecture of a true controlled experiment to address forms of bias routinely encountered in comparative effectiveness studies involving real-world observational data.

RESULTS

Twenty-four propensity score-matched cohorts comprising a wider clinical spectrum of patients than specified by the original target trial were successfully generated using the National Cancer Database. The arms for each of the emulated trials demonstrated exchangeability across all eligibility criteria and other clinical covariates. Significant treatment benefits were associated with only a narrow range of eligibility criteria, indicating that the original target trial was well specified.

CONCLUSION

The impact of patient selection on treatment outcomes can be studied using target trial emulation. This analytical framework can furthermore serve to leverage existing real-world data to inform the task of cohort specification for a randomized controlled trial, facilitating a more data-driven approach for this important step in clinical trial design.

Emulation of a target trial with sustained treatment strategies: an application to prostate cancer using both inverse probability weighting and the g-formula

As with many chronic illnesses, recurrent prostate cancer generally requires sustained treatment to prolong survival. However, initiating treatment immediately after recurrence may negatively impact quality of life without any survival gains. Therefore, we consider sustained strategies for initiating treatment based on specific characteristics of prostate-specific antigen (PSA), which can indicate disease progression. We define the protocol for a target trial comparing treatment strategies based on PSA doubling time, in which androgen deprivation therapy is initiated only after doubling time decreases below a certain threshold. Such a treatment strategy means the timing of treatment initiation (if ever) is not known at baseline, and the target trial protocol must explicitly specify the frequency of PSA monitoring until the threshold is met, as well as the duration of treatment. We describe these and other components of a target trial that need to be specified in order for such a trial to be emulated in observational data. We then use the parametric g-formula and inverse-probability weighted dynamic marginal structural models to emulate our target trial in a cohort of prostate cancer patients from clinics across the United States.

Causal analyses with target trial emulation for real-world evidence removed large self-inflicted biases: systematic bias assessment of ovarian cancer treatment effectiveness

BACKGROUND AND OBJECTIVES

Drawing causal conclusions from real-world data (RWD) poses methodological challenges and risk of bias. We aimed to systematically assess the type and impact of potential biases that may occur when analyzing RWD using the case of progressive ovarian cancer.

METHODS

We retrospectively compared overall survival with and without second-line chemotherapy (LOT2) using electronic medical records. Potential biases were determined using directed acyclic graphs. We followed a stepwise analytic approach ranging from crude analysis and multivariable-adjusted Cox model up to a full causal analysis using a marginal structural Cox model with replicates emulating a reference randomized controlled trial (RCT). To assess biases, we compared effect estimates (hazard ratios [HRs]) of each approach to the HR of the reference trial.

RESULTS

The reference trial showed an HR for second line vs. delayed therapy of 1.01 (95% confidence interval [95% CI]: 0.82-1.25). The corresponding HRs from the RWD analysis ranged from 0.51 for simple baseline adjustments to 1.41 (95% CI: 1.22-1.64) accounting for immortal time bias with time-varying covariates. Causal trial emulation yielded an HR of 1.12 (95% CI: 0.96-1.28).

CONCLUSION

Our study, using ovarian cancer as an example, shows the importance of a thorough causal design and analysis if one is expecting RWD to emulate clinical trial results.

Timing of dialysis in acute kidney injury using routinely collected data and dynamic treatment regimes

Background and objectives

Defining the optimal moment to start renal replacement therapy (RRT) in acute kidney injury (AKI) remains challenging. Multiple randomized controlled trials (RCTs) addressed this question whilst using absolute criteria such as pH or serum potassium. However, there is a need for identification of the most optimal cut-offs of these criteria. We conducted a causal analysis on routinely collected data (RCD) to compare the impact of different pre-specified dynamic treatment regimes (DTRs) for RRT initiation based on time-updated levels of potassium, pH, and urinary output on 30-day ICU mortality.

Design, setting, participants, and measurements

Patients in the ICU of Ghent University Hospital were included at the time they met KDIGO-AKI-stage ≥ 2. We applied inverse-probability-of-censoring-weighted Aalen–Johansen estimators to evaluate 30-day survival under 81 DTRs prescribing RRT initiation under different thresholds of potassium, pH, or persisting oliguria.

Results

Out of 13,403 eligible patients (60.8 ± 16.8 years, SOFA 7.0 ± 4.1), 5622 (63.4 ± 15.3 years, SOFA 8.2 ± 4.2) met KDIGO-AKI-stage ≥ 2. The DTR that delayed RRT until potassium ≥ 7 mmol/l, persisting oliguria for 24–36 h, and/or pH < 7.0 (non-oliguric) or < 7.2 (oliguric) despite maximal conservative treatment resulted in a reduced 30-day ICU mortality (from 12.7% [95% CI 11.9–13.6%] under current standard of care to 10.5% [95% CI 9.5–11.7%]; risk difference 2.2% [95% CI 1.3–3.8%]) with no increase in patients starting RRT (from 471 [95% CI 430–511] to 475 [95% CI 342–572]). The fivefold cross-validation benchmark for the optimal DTR resulted in 30-day ICU mortality of 10.7%.

Conclusions

Our causal analysis of RCD to compare RRT initiation at different thresholds of refractory low pH, high potassium, and persisting oliguria identified a DTR that resulted in a decrease in 30-day ICU mortality without increase in number of RRTs. Our results suggest that the current criteria to start RRT as implemented in most RCTs may be suboptimal. However, as our analysis is hypothesis generating, this optimal DTR should ideally be validated in a multicentric RCT.

Real World-Big Data Analytics in Healthcare

The term Big Data is used to describe extremely large datasets that are complex, multi-dimensional, unstructured, and heterogeneous and that are accumulating rapidly and may be analyzed with appropriate informatic and statistical methodologies to reveal patterns, trends, and associations [...].

A Review of Causal Inference for External Comparator Arm Studies

Randomized controlled trials (RCTs) are the gold standard design to establish the efficacy of new drugs and to support regulatory decision making. However, a marked increase in the submission of single-arm trials (SATs) has been observed in recent years, especially in the field of oncology due to the trend towards precision medicine contributing to the rise of new therapeutic interventions for rare diseases. SATs lack results for control patients, and information from external sources can be compiled to provide context for better interpretability of study results. External comparator arm (ECA) studies are defined as a clinical trial (most commonly a SAT) and an ECA of a comparable cohort of patients-commonly derived from real-world settings including registries, natural history studies, or medical records of routine care. This publication aims to provide a methodological overview, to sketch emergent best practice recommendations and to identify future methodological research topics. Specifically, existing scientific and regulatory guidance for ECA studies is reviewed and appropriate causal inference methods are discussed. Further topics include sample size considerations, use of estimands, handling of different data sources regarding differential baseline covariate definitions, differential endpoint measurements and timings. In addition, unique features of ECA studies are highlighted, specifically the opportunity to address bias caused by unmeasured ECA covariates, which are available in the SAT.

Combined Conventional Synthetic Disease Modifying Therapy vs. Infliximab for Rheumatoid Arthritis: Emulating a Randomized Trial in Observational Data

Observational studies are often considered unreliable for evaluating relative treatment effectiveness, but it has been suggested that following target trial protocols could reduce bias. Using observational data from patients with rheumatoid arthritis (RA) in the Swedish Rheumatology Quality Register (SRQ), between 2006 and 2020, we emulated the protocol of the Swedish Farmacotherapy trial (SWEFOT) and compared the results. SWEFOT was a pragmatic trial nested in SRQ, between 2002 and 2005, where methotrexate (MTX) insufficient responders were randomized to receive additional infliximab or sulfasalazine (SSZ) + hydroxychloroquine (HCQ). Patients with RA initiating infliximab (N = 313) or SSZ + HCQ (N = 196) after MTX were identified in SRQ and the Prescribed Drugs Register, mimicking the SWEFOT eligibility criteria. The primary outcome was the proportion of European Alliance of Associations for Rheumatology (EULAR) good responders at 9 months, classifying patients who discontinued treatment as “nonresponders.” Through sensitivity analyses, we assessed the impact of relaxing eligibility criteria. The observed proportions reaching EULAR good response were close to those reported in SWEFOT: 39% (vs. 39% in SWEFOT) for infliximab and 28% (vs. 25%) for SSZ + HCQ. The crude observed response ratio was 1.39 (95% confidence interval (CI) 1.04–1.86), increasing to 1.48 (95% CI 0.98–2.24) after confounding adjustment, compared to 1.59 (95% CI 1.10–2.30) in SWEFOT. Results remained close to SWEFOT when relaxing eligibility criteria until allowing prior disease‐modifying anti‐rheumatic drug (DMARD) use which reduced the observed difference between treatments. By applying a prespecified trial emulation protocol to observational clinical registry data, we could replicate the results of SWEFOT, favoring infliximab over SSZ + HCQ combination therapy at 9 months.

Target Trial Emulation for Transparent and Robust Estimation of Treatment Effects for Health Technology Assessment Using Real-World Data: Opportunities and Challenges

Evidence about the relative effects of new treatments is typically collected in randomised controlled trials (RCTs). In many instances, evidence from RCTs falls short of the needs of health technology assessment (HTA). For example, RCTs may not be able to capture longer-term treatment effects, or include all relevant comparators and outcomes required for HTA purposes. Information routinely collected about patients and the care they receive have been increasingly used to complement RCT evidence on treatment effects. However, such routine (or real-world) data are not collected for research purposes, so investigators have little control over the way patients are selected into the study or allocated to the different treatment groups, introducing biases for example due to selection or confounding. A promising approach to minimise common biases in non-randomised studies that use real-world data (RWD) is to apply design principles from RCTs. This approach, known as 'target trial emulation' (TTE), involves (1) developing the protocol with respect to core study design and analysis components of the hypothetical RCT that would answer the question of interest, and (2) applying this protocol to the RWD so that it mimics the data that would have been gathered for the RCT. By making the 'target trial' explicit, TTE helps avoid common design flaws and methodological pitfalls in the analysis of non-randomised studies, keeping each step transparent and accessible. It provides a coherent framework that embeds existing analytical methods to minimise confounding and helps identify potential limitations of RWD and the extent to which these affect the HTA decision. This paper provides a broad overview of TTE and discusses the opportunities and challenges of using this approach in HTA. We describe the basic principles of trial emulation, outline some areas where TTE using RWD can help complement RCT evidence in HTA, identify potential barriers to its adoption in the HTA setting and highlight some priorities for future work.

Benchmarking Observational Analyses Before Using Them to Address Questions Trials Do Not Answer: An Application to Coronary Thrombus Aspiration

To increase confidence in the use of observational analyses when addressing effectiveness questions beyond those addressed by randomized trials, one can first benchmark the observational analyses against existing trial results. We used Swedish registry data to emulate a target trial similar to the Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE) randomized trial, which found no difference in the risk of death or myocardial infarction by 1 year with or without thrombus aspiration among individuals with ST-elevation myocardial infarction. We benchmarked the emulation against the trial at 1 year and then extended the emulation's follow-up to 3 years and estimated effects in subpopulations underrepresented in the trial. As in the TASTE trial, the observational analysis found no differences in risk of outcomes by 1 year between groups (risk difference = 0.7 (confidence interval, -0.7, 2.0) and -0.2 (confidence interval, -1.3, 1.0) for death and myocardial infarction, respectively), so benchmarking was considered successful. We additionally showed no difference in risk of death or myocardial infarction by 3 years, or within subpopulations by 1 year. Benchmarking against an index trial before using observational analyses to answer questions beyond those the trial could address allowed us to explore whether the observational data can be trusted to deliver valid estimates of treatment effects.

Effects of ACE inhibitors and angiotensin receptor blockers: protocol for a UK cohort study using routinely collected electronic health records with validation against the ONTARGET trial

INTRODUCTION

Cardiovascular disease is a leading cause of death globally, responsible for nearly 18 million deaths worldwide in 2017. Medications to reduce the risk of cardiovascular events are prescribed based on evidence from clinical trials which explore treatment effects in an indicated sample of the general population. However, these results may not be fully generalisable because of trial eligibility criteria that generally restrict to younger patients with fewer comorbidities. Therefore, evidence of effectiveness of medications for groups underrepresented in clinical trials such as those aged ≥75 years, from ethnic minority backgrounds or with low kidney function may be limited.Using individual anonymised data from the Ongoing Telmisartan Alone and the Ramipril Global Endpoint Trial (ONTARGET) trial, in collaboration with the original trial investigators, we aim to investigate clinical trial replicability within a real-world setting in the area of cardiovascular disease. If the original trial results are replicable, we will estimate treatment effects and risk in groups underrepresented and excluded from the original clinical trial.

METHODS AND ANALYSIS

We will develop a cohort analogous to the ONTARGET trial within the Clinical Practice Research Datalink between 1 January 2001 and 31 July 2019 using the trial eligibility criteria and propensity score matching. The primary outcome is a composite of cardiovascular death, non-fatal myocardial infarction, non-fatal stroke and hospitalisation for congestive heart failure. If results from the cohort study fall within pre-specified limits, we will expand the cohort to include under represented and excluded groups.

ETHICS AND DISSEMINATION

Ethical approval has been granted by the London School of Hygiene & Tropical Medicine Ethics Committee (Ref: 22658). The study has been approved by the Independent Scientific Advisory Committee of the UK Medicines and Healthcare Products Regulatory Agency (protocol no. 20_012). Access to the individual patient data from the ONTARGET trial was obtained by the trial investigators. Findings will be submitted to peer-reviewed journals and presented at conferences.

Integrative Analysis of Randomized Clinical Trial and Observational Study Data to Inform Post-marketing Safety Decision-Making

Safety evaluation is a continual and iterative process throughout the drug development life cycle and requires long time horizons and large amounts of data to fully understand the safety profile of a medical product. Although randomized clinical trials (RCT) provide high-quality data for an initial assessment of safety signals, the safety signals may not all have been known at the time of approval because safety data collected from RCT only involve a relatively small number of subjects during a relatively short follow-up period. The increased accumulation of post-marketing real-world data (RWD) presents an opportunity to utilize them for safety decision-making; these include identifying new safety signals, further characterization of safety concerns that are raised in pre-marketing RCT, and further generalization of RCT findings to the broader patient populations not previously studied in RCT. In this paper, we use cardiovascular safety outcome trial for antidiabetic therapies as an illustrative example and discuss how integrative analysis of RCT and observational study data can answer regulatory concerns about cardiovascular risk in a post-marketing setting. A novel statistical analysis strategy is proposed to combine both sources of safety data in a data fusion approach. The proposed approach includes three stages: (1) feasibility analysis that uses an RCT to validate an observational study, applying estimand framework and emulating RCT with RWD; (2) integrative analysis that combines evidence from the RCT and observational study data cooperatively; and (3) sensitivity analysis that examines the consistency of the previous analyses. Two potential utilities of the proposed integrative analysis for the cardiovascular safety outcome trial are discussed.

The Effect of Facility Volume on Survival Following Proctectomy for Rectal Cancer

BACKGROUND

Prior studies assessing colorectal cancer survival have reported better outcomes when operations are performed at high-volume centers. These studies have largely been cross-sectional, making it difficult to interpret their estimates. We aimed to assess the effect of facility volume on survival following proctectomy for rectal cancer.

METHODS

Using data from the National Cancer Database, we included all patients with complete baseline information who underwent proctectomy for non-metastatic rectal cancer between 2004 and 2016. Facility volume was defined as the number of rectal cancer cases managed at the treating center in the calendar year prior to the patient's surgery. Overall survival estimates were obtained for facility volumes ranging from 10 to 100 cases/year. Follow-up began on the day of surgery and continued until loss to follow-up or death.

RESULTS

A total of 52,822 patients were eligible. Patients operated on at hospitals with volumes of 10, 30, and 50 cases/year had similar distributions of grade, clinical stage, and neoadjuvant therapies. 1-, 3-, and 5-year survival all improved with increasing facility volume. One-year survival was 94.0% (95% CI: 93.7, 94.3) for hospitals that performed 10 cases/year, 94.5% (95% CI: 94.2, 94.7) for 30 cases/year, and 94.8% (95% CI: 94.5, 95.0) for 50 cases/year. Five-year survival was 68.9% (95% CI: 68.0, 69.7) for hospitals that performed 10 cases/year, 70.8% (95% CI: 70.1, 71.5) for 30 cases/year, and 72.0% (95% CI: 71.2, 72.8) for 50 cases/year.

CONCLUSIONS

Treatment at a higher volume facility results in improved survival following proctectomy for rectal cancer, though the small benefits are less profound than previously reported.

Crosstalk between Statins and Cancer Prevention and Therapy: An Update

The importance of statins in cancer has been discussed in many studies. They are known for their anticancer properties against solid tumors of the liver or lung, as well as diffuse cancers, such as multiple myeloma or leukemia. Currently, the most commonly used statins are simvastatin, rosuvastatin and atorvastatin. The anti-tumor activity of statins is largely related to their ability to induce apoptosis by targeting cancer cells with high selectivity. Statins are also involved in the regulation of the histone acetylation level, the disturbance of which can lead to abnormal activity of genes involved in the regulation of proliferation, differentiation and apoptosis. As a result, tumor growth and its invasion may be promoted, which is associated with a poor prognosis. High levels of histone deacetylases are observed in many cancers; therefore, one of the therapeutic strategies is to use their inhibitors. Combining statins with histone deacetylase inhibitors can induce a synergistic anticancer effect.

Signals were broadly positive for months, but never definitive: the tocilizumab story

Background

Most treatment guidelines for coronavirus disease 2019 (COVID-19) currently recommend tocilizumab in combination with dexamethasone in critically ill patients who are exhibiting rapid respiratory decompensation.

Aims

To produce a critical review and summary of the pathway which led to the repurposing of tocilizumab for COVID-19 treatment, from in vitro observations to guidelines recommendations.

Sources

All studies evaluating the effectiveness of tocilizumab to treat COVID-19 disease published between July 2020 and July 2021.

Content

Two large and methodologically well conducted observational studies, the TESEO and the STOP COVID cohorts, showed a reduction in the risk of invasive mechanical ventilation or death in patients treated with tocilizumab as compared to standard of care in 2020. Concomitantly, and up to February 2021, a number of randomized trials (RCTs) with small sample sizes were showing discrepant results. These RCTs had a number of issues: small sample size, various designs and inclusion criteria, and different dosages of tocilizumab used. The confidence interval of the meta-analytic estimate for the RCT results was consistent with the hypothesis of no efficacy of tocilizumab. In our opinion, this was mainly because the meta-analysis included small and heterogeneous studies. These results led to a delay in the inclusion of tocilizumab in guidelines which occurred only in the summer of 2021.

Implications

Although observational studies are unable to control for unmeasured confounding, they can be put together quickly during a pandemic and promptly provide important information. The large sample size allows us to investigate effect measure modifiers and to better target interventions. It is key that the effect size is somewhat large (RR > 2), all sources of bias are properly accounted for, and the direct evidence is weighted against these factors. It appears to us that for tocilizumab, not having dismissed the results of carefully designed and analysed observational studies in 2020 could have prevented many deaths over those months.

Estimating Treatment-Switching Bias in a Randomized Clinical Trial of Ovarian Cancer Treatment: Combining Causal Inference with Decision-Analytic Modeling

BACKGROUND

Bevacizumab is efficacious in delaying ovarian cancer progression and controlling ascites. The ICON7 trial showed a significant benefit in overall survival for bevacizumab, whereas the GOG-218 trial did not. GOG-218 allowed control group patients to switch to bevacizumab upon progression, which may have biased the results. Lack of data on switching behavior prevented the application of g-methods to adjust for switching. The objective of this study was to apply decision-analytic modeling to estimate the impact of switching bias on causal treatment-effect estimates.

METHODS

We developed a causal decision-analytic Markov model (CDAMM) to emulate the GOG-218 trial and estimate overall survival. CDAMM input parameters were based on data from randomized clinical trials and the published literature. Overall switching proportion was based on GOG-218 trial information, whereas the proportion switching with and without ascites was estimated using calibration. We estimated the counterfactual treatment effect that would have been observed had no switching occurred by denying switching in the CDAMM.

RESULTS

The survival curves generated by the CDAMM matched well with the ones reported in the GOG-218 trial. The survival curve correcting for switching showed an estimated bias such that 79% of the true treatment effect could not be observed in the GOG-218 trial. Results were most sensitive to changes in the proportion progressing with severe ascites and mortality.

LIMITATIONS

We used a simplified model structure and based model parameters on published data and assumptions. Robustness of the CDAMM was tested and model assumptions transparently reported.

CONCLUSIONS

Medical-decision science methods may be merged with empirical methods of causal inference to integrate data from other sources where empirical data are not sufficient. We recommend collecting sufficient information on switching behavior when switching cannot be avoided.

Disease progression of 213 patients hospitalized with Covid-19 in the Czech Republic in March-October 2020: An exploratory analysis

We collected a multi-centric retrospective dataset of patients (N = 213) who were admitted to ten hospitals in Czech Republic and tested positive for SARS-CoV-2 during the early phases of the pandemic in March-October 2020. The dataset contains baseline patient characteristics, breathing support required, pharmacological treatment received and multiple markers on daily resolution. Patients in the dataset were treated with hydroxychloroquine (N = 108), azithromycin (N = 72), favipiravir (N = 9), convalescent plasma (N = 7), dexamethasone (N = 4) and remdesivir (N = 3), often in combination. To explore association between treatments and patient outcomes we performed multiverse analysis, observing how the conclusions change between defensible choices of statistical model, predictors included in the model and other analytical degrees of freedom. Weak evidence to constrain the potential efficacy of azithromycin and favipiravir can be extracted from the data. Additionally, we performed external validation of several proposed prognostic models for Covid-19 severity showing that they mostly perform unsatisfactorily on our dataset.

A Framework for Methodological Choice and Evidence Assessment for Studies Using External Comparators from Real-World Data

Several approaches have been proposed recently to accelerate the pathway from drug discovery to patient access. These include novel designs such as using controls external to the clinical trial where standard randomised controls are not feasible. In parallel, there has been rapid growth in the application of routinely collected healthcare ‘real-world’ data for post-market safety and effectiveness studies. Thus, using real-world data to establish an external comparator arm in clinical trials is a natural next step. Regulatory authorities have begun to endorse the use of external comparators in certain circumstances, with some positive outcomes for new drug approvals. Given the potential to introduce bias associated with observational studies, there is a need for recommendations on how external comparators should be best used. In this article, we propose an evaluation framework for real-world data external comparator studies that enables full assessment of available evidence and related bias. We define the principle of exchangeability and discuss the applicability of criteria described by Pocock for consideration of the exchangeability of the external and trial populations. We explore how trial designs using real-world data external comparators fit within the evidence hierarchy and propose a four-step process for good conduct of external comparator studies. This process is intended to maximise the quality of evidence based on careful study design and the combination of covariate balancing, bias analysis and combining outcomes.

Emulating Target Trials to Improve Causal Inference From Agent-Based Models

Agent-based models are a key tool for investigating the emergent properties of population health settings, such as infectious disease transmission, where the exposure often violates the key "no interference" assumption of traditional causal inference under the potential outcomes framework. Agent-based models and other simulation-based modeling approaches have generally been viewed as a separate knowledge-generating paradigm from the potential outcomes framework, but this can lead to confusion about how to interpret the results of these models in real-world settings. By explicitly incorporating the target trial framework into the development of an agent-based or other simulation model, we can clarify the causal parameters of interest, as well as make explicit the assumptions required for valid causal effect estimation within or between populations. In this paper, we describe the use of the target trial framework for designing agent-based models when the goal is estimation of causal effects in the presence of interference, or spillover.

The Impact of Same-Day Antiretroviral Therapy Initiation Under the World Health Organization Treat-All Policy

Rapid initiation of antiretroviral therapy (ART) is recommended for people living with human immunodeficiency virus (HIV), with the option to start treatment on the day of diagnosis (same-day ART). However, the effect of same-day ART remains unknown in realistic public sector settings. We established a cohort of ≥16-year-old patients who initiated first-line ART under a treat-all policy in Nhlangano (Eswatini) during 2014-2016, either on the day of HIV care enrollment (same-day ART) or 1-14 days thereafter (early ART). Directed acyclic graphs, flexible parametric survival analysis, and targeted maximum likelihood estimation (TMLE) were used to estimate the effect of same-day-ART initiation on a composite unfavorable treatment outcome (loss to follow-up, death, viral failure, treatment switch). Of 1,328 patients, 839 (63.2%) initiated same-day ART. The adjusted hazard ratio of the unfavorable outcome was higher, 1.48 (95% confidence interval: 1.16, 1.89), for same-day ART compared with early ART. TMLE suggested that after 1 year, 28.9% of patients would experience the unfavorable outcome under same-day ART compared with 21.2% under early ART (difference: 7.7%; 1.3%-14.1%). This estimate was driven by loss to follow-up and varied over time, with a higher hazard during the first year after HIV care enrollment and a similar hazard thereafter. We found an increased risk with same-day ART. A limitation was that possible silent transfers that were not captured.

Transarterial Radioembolization for Hepatocellular Carcinoma with Major Vascular Invasion: A Nationwide Propensity Score-Matched Analysis with Target Trial Emulation

PURPOSE

To examine National Cancer Database (NCDB) data to comparatively evaluate overall survival (OS) between patients undergoing transarterial radioembolization (TARE) and those undergoing systemic therapy for hepatocellular carcinoma with major vascular invasion (HCC-MVI).

METHODS

One thousand five hundred fourteen patients with HCC-MVI undergoing first-line TARE or systemic therapy were identified from the NCDB. OS was compared using propensity score-matched Cox regression and landmark analysis. Efficacy was also compared within a target trial framework.

RESULTS

TARE usage doubled between 2010 and 2015. Intervals before treatment were longer for TARE than for systemic therapy (mean [median], 66.5 [60] days vs 46.8 (35) days, respectively, P < .0001). In propensity-score-matched and landmark-time-adjusted analyses, TARE was found to be associated with a hazard ratio of 0.74 (95 % CI, 0.60-0.91; P = .005) and median OS of 7.1 months (95 % CI, 5.0-10.5) versus 4.9 months (95 % CI, 3.9-6.5) for systemically treated patients. In an emulated target trial involving 236 patients with unilobular HCC-MVI, a low number of comorbidities, creatinine levels <2.0 mg/dL, bilirubin levels <2.0 mg/dL, and international normalized ratio <1.7, TARE was found to be associated with a hazard ratio of 0.57 (95 % CI, 0.39-0.83; P = .004) and a median OS of 12.9 months (95 % CI, 7.6-19.2) versus 6.5 months (95 % CI, 3.6-11.1) for the systemic therapy arm.

CONCLUSIONS

In propensity-score-matched analyses involving pragmatic and target trial HCC-MVI cohorts, TARE was found to be associated with significant survival benefits compared with systemic therapy. Although not a substitute for prospective trials, these findings suggest that the increasing use of TARE for HCC-MVI is accompanied by improved OS. Further trials of TARE in patients with HCC-MVI are needed, especially to compare with newer systemic therapies.

Population intervention effects in observational studies to emulate target trial results: reconciling the effects of improved sanitation on child growth

BACKGROUND

Improved sanitation has been associated with improved child growth in observational studies, but multiple randomized trials that delivered improved sanitation found no effect on child growth. We assessed to what extent differences in the effect estimated in the two study designs (the effect of treatment in observational studies and the effect of treatment assignment in trials) could explain the contradictory results.

METHODS

We used parametric g-computation in five prospective studies (n = 21 524) and 59 cross-sectional Demographic and Health Surveys (DHS; n = 158 439). We compared the average treatment effect (ATE) for improved sanitation on mean length-for-age z-score (LAZ) among children aged <2 years to population intervention effects (PIEs), which are the observational analogue of the effect estimated in trials in which some participants are already exposed.

RESULTS

The ATE was >0.15 z-scores, a clinically meaningful difference, in most prospective studies but in <20% of DHS surveys. The PIE was always smaller than the ATE, and the magnitude of difference depended on the baseline prevalence of the improved sanitation. Interventions with suboptimal coverage and interventions delivered in populations with higher mean LAZ had a smaller effect on population-level LAZ.

CONCLUSIONS

Estimates of PIEs corresponding to anticipated trial results were often smaller than clinically meaningful effects. Incongruence between observational associations and null trial results may in part be explained by expected differences between the effects estimated. Using observational ATEs to set expectations for trials may overestimate the impact that sanitation interventions can achieve. PIEs predict realistic effects and should be more routinely estimated.

Emulating a Randomised Controlled Trial With Observational Data: An Introduction to the Target Trial Framework

Randomised controlled trials (RCTs) are often considered to be the highest quality of evidence owing to the absence of baseline confounding, the simplicity of analyses, and direct estimation of causal effects. However, observational studies can be designed to mimic RCTs and estimate causal treatment effects. In this review, we describe the target trial framework to illustrate how observational studies can successfully emulate RCTs. We focus on key design elements of RCTs and how to emulate them with observational data. These elements include 1) eligibility criteria, 2) treatment assignment and randomisation, 3) specification of "time zero", 4) outcomes, 5) follow-up, 6) causal contrasts (intention-to-treat vs per-protocol), and 7) statistical analyses. In addition, we describe the design of an example target trial created to emulate the Trial to Assess Improvement in Therapeutic Outcomes by Optimizing Platelet Inhibition With Prasugrel Thrombolysis in Myocardial Infarction (TRITON-TIMI) 38 trial and compare effect estimates. Overall, careful design of a target trial using observational data can produce causal effect estimates that are often comparable to RCTs.

Comparing Effect Estimates in Randomized Trials and Observational Studies From the Same Population: An Application to Percutaneous Coronary Intervention

Background

To understand when results from observational studies and randomized trials are comparable, we performed an observational emulation of a target trial designed to ask similar questions as the VALIDATE (Bivalirudin Versus Heparin in ST‐Segment and Non–ST‐Segment Elevation Myocardial Infarction in Patients on Modern Antiplatelet Therapy) randomized trial. The VALIDATE trial compared the effect of bivalirudin and heparin during percutaneous coronary intervention on the risk of death, myocardial infarction, and bleeding across Sweden.

Methods and Results

We specified the protocol of a target trial similar to the VALIDATE trial, then emulated the target trial in the period before the VALIDATE trial took place using data from the SWEDEHEART (Swedish Web System for Enhancement and Development of Evidence‐Based Care in Heart Disease Evaluated According to Recommended Therapies) registry—the same registry in which the trial was undertaken. The target trial emulation and the VALIDATE trial both estimated little or no effect of bivalirudin versus heparin on the risk of death or myocardial infarction by 180 days (target trial emulation risk ratio for death, 1.21 [95% CI, 0.88 – 1.54]; VALIDATE trial hazard ratio for death, 1.05 [95% CI, 0.78 – 1.41]). The observational data, however, could not capture less severe cases of bleeding, resulting in an inability to define a bleeding outcome like the trial, and could not accurately estimate the comparative risk of death by 14 days, which may be the result of intractable confounding early in follow‐up or the inability to precisely emulate the trial’s eligibility criteria.

Conclusions

Using real‐world data to emulate a target trial can deliver accurate effect estimates. Yet, even with rich observational data, it is not always possible to estimate the short‐term effect of interventions or the effect on outcomes for which data are not routinely collected.

Chronic use of renin-angiotensin-aldosterone system blockers and mortality in COVID-19: A multicenter prospective cohort and literature review

Aims

The role of renin‐angiotensin‐aldosterone system (RAAS) blockers on the course of coronavirus disease 2019 (COVID‐19) is debated. We assessed the association between chronic use of RAAS blockers and mortality among inpatients with COVID‐19 and explored reasons for discrepancies in the literature.

Methods and results

We included adult hypertensive patients from a prospective nationwide cohort of 3512 inpatients with COVID‐19 up to June 30, 2020. Cox proportional hazard models with various adjustment or propensity weighting methods were used to estimate the hazard ratios (HR) of 30‐day mortality for chronic users versus non‐users of RAAS blockers. We analyzed data of 1160 hypertensive patients: 719 (62%) were male and 777 (67%) were older than 65 years. The main comorbidities were diabetes (n = 416, 36%), chronic cardiac disease (n = 401, 35%), and obesity (n = 340, 29%); 705 (61%) received oxygen therapy. We recorded 135 (11.6%) deaths within 30 days of diagnosis. We found no association between chronic use of RAAS blockers and mortality (unadjusted HR = 1.13, 95% CI [0.8–1.6]; propensity inverse probability treatment weighted HR = 1.09 [0.86‐1.39]; propensity standardized mortality ratio weighted HR = 1.08 [0.79–1.47]). Our comprehensive review of previous studies highlighted that significant associations were mostly found in unrestricted populations with inappropriate adjustment, or with biased in‐hospital exposure measurement.

Conclusion

Our results do not support previous concerns regarding these drugs, nor a potential protective effect as reported in previous poorly designed studies and meta‐analyses. RAAS blockers should not be discontinued during the pandemic, while in‐hospital management of these drugs will be clarified by randomized trials. NCT04262921.

New Perspective for Soft Tissue Closure in Medication-Related Osteonecrosis of the Jaw (MRONJ) Using Barbed Sutures

The aim of this study was to compare the effectiveness of barbed versus smooth sutures for soft tissue closure of exposed jawbone sites in medication-related osteonecrosis of the jaw (MRONJ) patients. Exposed necrotic jawbone sites surgically managed by intraoral soft tissue closure were evaluated. Either barbed sutures (Stratafix™ or V-Loc™) together with Prolene^® or Vicryl^® sutures were used. We estimated the effect of barbed sutures (BS) with Prolene^® compared to smooth sutures (Vicryl^®) on the hazard rate of intraoral soft tissue dehiscence using a multivariate Cox regression model within a target trial framework, adjusting for relevant confounders. In total, 306 operations were performed in 188 sites. In the primary analysis 182 sites without prior surgery were included. Of these, 113 sites developed a dehiscence during follow-up. 84 sites were operated using BS and Prolene^®. A total of 222 sites were operated with Vicryl^® (control group). In the BS group, the median time to event (i.e., dehiscence) was 148 days (interquartile range (IQR), 42–449 days) compared to 15 days (IQR, 12–52 days) in the control group. The hazard rate of developing intraoral dehiscence was 0.03 times (95%-confidence interval (CI): 0.01; 0.14, p < 0.001) lower for BS patients compared to the control group. Within the limits of a retrospective study, BS showed a high success rate and are therefore recommended for soft tissue closure of exposed jawbone sites in MRONJ patients. Additional studies are warranted to further evaluate this novel application of BS.

Per-protocol analysis of the ZINC trial for HIV disease among alcohol users

BACKGROUND

The Zinc for INflammation and Chronic disease in HIV (ZINC) trial randomized person who live with HIV (PLWH) who engage in heavy drinking to either daily zinc supplementation or placebo. The primary outcome was change in the Veterans Aging Cohort Study (VACS) index, a predictor of mortality, between baseline and 18 months. Because adherence and follow-up were suboptimal, the intention-to-treat analysis, which was not statistically significant, may have underestimated the effect of the zinc supplementation.

OBJECTIVE

We estimated the per-protocol effect of zinc versus placebo in the ZINC trial (i.e., the effect that would have been observed if all participants had had high adherence and none was lost to follow-up).

METHODS

Adherence was measured as the self-reported percentage of pills taken in the previous 6 weeks and assessed at all post-baseline visits. We used inverse probability weighting to estimate and compare the change in the VACS index at 18 months in the zinc and placebo groups, had all the trial participants had high adherence (i.e., cumulative adherence ≥80% at 18 months). To examine trends by level of adherence, we rerun the analyses using thresholds for high adherence of 70% and 90% of average self-reported pill coverage.

RESULTS

The estimated (95% confidence interval) change in the VACS index was - 2.16 (- 8.07, 3.59) and 5.84 (0.73, 11.80) under high adherence and no loss to follow-up in the zinc and placebo groups, respectively. The per-protocol effect estimate of the mean difference in the change between the zinc and placebo groups was - 8.01 (- 16.42, 0.01), somewhat larger than the intention-to-treat effect difference in change (- 4.68 (- 9.62, 0.25)), but it was still not statistically significant. The mean difference in the change between individuals in the zinc and placebo groups was - 4.07 (- 11.5, 2.75) and -12.34 (- 20.14, -4.14) for high adherence defined as 70% and 90% of pill coverage, respectively.

CONCLUSIONS

Overall, high adherence to zinc was associated with a lower VACS score, but confidence intervals were wide and crossed 0. Further studies with a larger sample size are needed to quantify the benefits of zinc supplementation in this population.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01934803 . Registered on August 30, 2013.

Predictors of retention in the prospective HIV prevention OKAPI cohort in Kinshasa

Retention is a key element in HIV prevention programs. In Sub-Saharan Africa most data on retention come from HIV clinical trials or people living with HIV attending HIV treatment and control programs. Data from observational cohorts are less frequent. Retention at 6-/12-month follow-up and its predictors were analyzed in OKAPI prospective cohort. From April 2016 to April 2018, 797 participants aged 15-59 years attending HIV Voluntary Counseling and Testing in Kinshasa were interviewed about HIV-related knowledge and behaviors at baseline and at 6- and 12-month follow-ups. Retention rates were 57% and 27% at 6- and 12-month follow up; 22% of participants attended both visits. Retention at 6-month was significantly associated with 12-month retention. Retention was associated with low economic status, being studying, daily/weekly Internet access, previous HIV tests and aiming to share HIV test with partner. Contrarily, perceiving a good health, living far from an antiretroviral center, daily/weekly alcohol consumption and perceiving frequent HIV information were inversely associated with retention. In conclusion, a high attrition was found among people attending HIV testing participating in a prospective cohort in Kinshasa. Considering the low retention rates and the predictors found in this study, more HIV cohort studies in Kinshasa need to be evaluated to identify local factors and strategies that could improve retention if needed.

Contextualizing single-arm trials with real-world data: An emulated target trial comparing therapies for neovascular age-related macular degeneration

Abstract

One‐in‐four ophthalmology trials are single‐armed, which poses challenges to their interpretation. We demonstrate how real‐world cohorts used as external/synthetic control arms can contextualize such trials. We herein emulated a target trial on the intention‐to‐treat efficacy of off‐label bevacizumab (q6w) pro re nata relative to fixed‐interval aflibercept (q8w) for improving week 54 visual acuity of eyes affected by neovascular age‐related macular degeneration. The bevacizumab arm (n = 65) was taken from the ABC randomized controlled trial. A total of 4,471 aflibercept‐treated eyes aligning with the ABC trial eligibility were identified from electronic health records and synthetic control arms were created by emulating randomization conditional on age, sex, and baseline visual read via exact matching and propensity score methods. We undertook an inferiority analysis on mean difference at 54 weeks; outcomes regression on achieving a change in visual acuity of greater than or equal to 15, greater than or equal to 10, and less than or equal to −15 Early Treatment Diabetic Retinopathy (ETDRS) letters at week 54; and a time‐to‐event analysis on achieving a change in visual acuity of greater than or equal to 15, greater than or equal to 10, and less than or equal to −15 ETDRS letters by week 54. The findings suggest off‐label bevacizumab to be neither inferior nor superior to licensed aflibercept. Our study highlights how real‐world cohorts representing the counterfactual intervention could aid the interpretation of single‐armed trials when analyzed in accord to the target trial framework.

Statins as Potential Chemoprevention or Therapeutic Agents in Cancer: a Model for Evaluating Repurposed Drugs

PURPOSE OF REVIEW

Repurposing established medicines for a new therapeutic indication potentially has important global and societal impact. The high costs and slow pace of new drug development have increased interest in more cost-effective repurposed drugs, particularly in the cancer arena. The conventional drug development pathway and evidence framework are not designed for drug repurposing and there is currently no consensus on establishing the evidence base before embarking on a large, resource intensive, potential practice changing phase III randomised controlled trial (RCT). Numerous observational studies have suggested a potential role for statins as a repurposed drug for cancer chemoprevention and therapy, and we review the strength of the cumulative evidence here.

RECENT FINDINGS

In the setting of cancer, a potential repurposed drug, like statins, typically goes through a cyclical history, with initial use for several years in another disease setting, prior to epidemiological research identifying a possible chemo-protective effect. However, further information is required, including review of RCT data in the initial disease setting with exploration of cancer outcomes. Additionally, more contemporary methods should be considered, such as Mendelian randomization and pharmaco-epidemiological research with "target" trial design emulation using electronic health records. Pre-clinical and traditional observational data potentially support the role of statins in the treatment of cancer; however, randomised trial evidence is not supportive. Evaluation of contemporary methods provides little added support for the use of statin therapy in cancer. We provide complementary evidence of alternative study designs to enable a robust critical appraisal from a number of sources of the go/no-go decision for a prospective phase III RCT of statins in the treatment of cancer.