Comparative Effectiveness of Oxaliplatin Versus 5-flourouricil in Older Adults: An Instrumental Variable Analysis

Visualizing External Validity: Graphical Displays to Inform the Extension of Treatment Effects from Trials to Clinical Practice

BACKGROUND

In the presence of effect measure modification, estimates of treatment effects from randomized controlled trials may not be valid in clinical practice settings. The development and application of quantitative approaches for extending treatment effects from trials to clinical practice settings is an active area of research.

METHODS

In this article, we provide researchers with a practical roadmap and four visualizations to assist in variable selection for models to extend treatment effects observed in trials to clinical practice settings and to assess model specification and performance. We apply this roadmap and visualizations to an example extending the effects of adjuvant chemotherapy (5-fluorouracil vs. plus oxaliplatin) for colon cancer from a trial population to a population of individuals treated in community oncology practices in the United States.

RESULTS

The first visualization screens for potential effect measure modifiers to include in models extending trial treatment effects to clinical practice populations. The second visualization displays a measure of covariate overlap between the clinical practice populations and the trial population. The third and fourth visualizations highlight considerations for model specification and influential observations. The conceptual roadmap describes how the output from the visualizations helps interrogate the assumptions required to extend treatment effects from trials to target populations.

CONCLUSIONS

The roadmap and visualizations can inform practical decisions required for quantitatively extending treatment effects from trials to clinical practice settings.

Noninterventional studies in the COVID-19 era: methodological considerations for study design and analysis

The global COVID-19 pandemic has generated enormous morbidity and mortality, as well as large health system disruptions including changes in use of prescription medications, outpatient encounters, emergency department admissions, and hospitalizations. These pandemic-related disruptions are reflected in real-world data derived from electronic medical records, administrative claims, disease or medication registries, and mobile devices. We discuss how pandemic-related disruptions in healthcare utilization may impact the conduct of noninterventional studies designed to characterize the utilization and estimate the effects of medical interventions on health-related outcomes. Using hypothetical studies, we highlight consequences that the pandemic may have on study design elements including participant selection and ascertainment of exposures, outcomes, and covariates. We discuss the implications of these pandemic-related disruptions on possible threats to external validity (participant selection) and internal validity (for example, confounding, selection bias, missing data bias). These concerns may be amplified in populations disproportionately impacted by COVID-19, such as racial/ethnic minorities, rural residents, or people experiencing poverty. We propose a general framework for researchers to carefully consider during the design and analysis of noninterventional studies that use real-world data from the COVID-19 era.

Effectiveness of adjuvant FOLFOX vs 5FU/LV in adults over age 65 with stage II and III colon cancer using a novel hybrid approach

PURPOSE

Estimates of cancer therapy effects can differ in clinical trials and clinical practice, partly due to underrepresentation of certain patient subgroups in trials. We utilize a hybrid approach, combining clinical trial and real-world data, to estimate the comparative effectiveness of two adjuvant chemotherapy regimens for colon cancer.

METHODS

We identified patients aged 66 and older enrolled in the Multicenter International Study of Oxaliplatin/5FU-LV in the Adjuvant Treatment of Colon Cancer. Similar patients were identified in the Surveillance, Epidemiology, and End Results (SEER)-Medicare database, initiating adjuvant chemotherapy with either 5-fluorouracil (5FU) alone or in combination with oxaliplatin (FOLFOX). We used logistic regression to estimate the likelihood of trial enrollment as a function of age, sex, and substage. Using inverse odds of sampling weights (IOSW), we compared 5-year mortality in patients randomized to FOLFOX vs 5FU using weighted Cox proportional hazards regression, the Nelson-Aalen estimator for cumulative hazards, and bootstrapping for 95% confidence intervals (CIs).

RESULTS

There were 690 trial participants and 3834 SEER-Medicare patients. The SEER-Medicare population was older and had a higher proportion of stage IIIB and IIIC patients than the trial. After controlling for differences between populations, the IOSW 5-year HR was 1.21 (0.89, 1.65), slightly farther from the null than the trial estimate (HR = 1.14, 95%CI: 0.87, 1.49).

CONCLUSIONS

This study supports mounting evidence of little to no incremental reduction in 5-year mortality for FOLFOX vs 5FU in older adults with stage II-III colon cancer, emphasizing the importance of combining clinical trial and real-world data to support such conclusions.

Inference without randomization or ignorability: A stability-controlled quasi-experiment on the prevention of tuberculosis

The stability-controlled quasi-experiment (SCQE) is an approach to study the effects of nonrandomized, newly adopted treatments. While covariate adjustment techniques rely on a "no unobserved confounding" assumption, SCQE imposes an assumption on the change in the average nontreatment outcome between successive cohorts (the "baseline trend"). We provide inferential tools for SCQE and its first application, examining whether isoniazid preventive therapy (IPT) reduced tuberculosis (TB) incidence among 26 715 HIV patients in Tanzania. After IPT became available, 16% of untreated patients developed TB within a year, compared with only 0.5% of patients under treatment. Thus, a simple difference in means suggests a 15.5 percentage point (pp) lower risk (p ≪ .001). Adjusting for covariates using numerous techniques leaves this effectively unchanged. Yet, due to confounding biases, such estimates can be misleading regardless of their statistical strength. By contrast, SCQE reveals valid causal effect estimates for any chosen assumption on the baseline trend. For example, assuming a baseline trend near 0 (no change in TB incidence over time, absent this treatment) implies a small and insignificant effect. To argue IPT was beneficial requires arguing that the nontreatment incidence would have risen by at least 0.7 pp per year, which is plausible but far from certain. SCQE may produce narrow estimates when the plausible range of baseline trends can be sufficiently constrained, while in every case it tells us what baseline trends must be believed in order to sustain a given conclusion, protecting against inferences that rely upon infeasible assumptions.

Long-term use of hydrocodone vs. oxycodone in primary care

BACKGROUND

Hydrocodone and oxycodone are the Schedule II opioids most often prescribed in primary care. Notwithstanding the dangers of prescription opioid use, the likelihood of long-term use with either drug is presently unknown.

METHODS

Using a retrospective cohort design and data from a commerical healthcare claims repository, we compared the likelihood of long-term use of hydrocodone and oxycodone in primary care patients presenting with acute back pain. Treatment was categorized as long-term if the prescription dates spanned ≥90 days from initial prescription to the run-out date of the last prescription, and included ≥120 days' supply or ≥10 fills. Instrumental variable methods and probit regression were used to model the effect of drug choice on long-term use, estimate the average treatment effect, and correct for confounding by indication.

RESULTS

A total of 3,983 patients who were prescribed only hydrocodone or only oxycodone were followed for 270 days in 2016. Long-term opioid use was observed in 320 patients (8%). Controlling for potential confounders including morphine milligram equivalents and dosage, an estimated 12% (95 CI, 10%-14%) treated with hydrocodone transitioned to long-term use vs. 2% (95 CI, 1%-3%) on oxycodone. Among patients who received more than one prescription (n = 1,866), an estimated 23% (95 CI, 19%-26%) treated with hydrocodone transitioned to long-term use vs. 5% (95 CI, 3%-7%) on oxycodone. The difference between drugs was supported in sensitivity and subgroup analyses. Sample selection bias was not detected.

CONCLUSIONS

Long-term use was substantially greater for patients treated with hydrocodone than oxycodone, despite equianalgesia.

Estimating Causal Effects of New Treatments Despite Self-Selection: The Case of Experimental Medical Treatments

Providing terminally ill patients with access to experimental treatments, as allowed by recent "right to try" laws and "expanded access" programs, poses a variety of ethical questions. While practitioners and investigators may assume it is impossible to learn the effects of these treatment without randomized trials, this paper describes a simple tool to estimate the effects of these experimental treatments on those who take them, despite the problem of selection into treatment, and without assumptions about the selection process. The key assumption is that the average outcome, such as survival, would remain stable over time in the absence of the new treatment. Such an assumption is unprovable, but can often be credibly judged by reference to historical data and by experts familiar with the disease and its treatment. Further, where this assumption may be violated, the result can be adjusted to account for a hypothesized change in the non-treatment outcome, or to conduct a sensitivity analysis. The method is simple to understand and implement, requiring just four numbers to form a point estimate. Such an approach can be used not only to learn which experimental treatments are promising, but also to warn us when treatments are actually harmful - especially when they might otherwise appear to be beneficial, as illustrated by example here. While this note focuses on experimental medical treatments as a motivating case, more generally this approach can be employed where a new treatment becomes available or has a large increase in uptake, where selection bias is a concern, and where an assumption on the change in average non-treatment outcome over time can credibly be imposed.

Surveillance, Epidemiology, and End Results (SEER) and SEER-Medicare Databases: Use in Clinical Research for Improving Colorectal Cancer Outcomes

The Surveillance, Epidemiology, and End Results (SEER) program is a clinical database, funded by the National Cancer Institute (NCI), which was created to collect cancer incidence, prevalence, and survival data from U.S. cancer registries. By capturing approximately 30% of the U.S. population, it serves as a powerful resource for researchers focused on understanding the natural history of colorectal cancer and improvement in patient care. The linked SEER-Medicare database is a robust database allowing investigators to perform studies focusing on health disparities, quality of care, and cost of treatment in oncologic disease. Since its infancy in the early 1970s, the database has been utilized for thousands of studies resulting in novel publications that have shaped our management of colorectal cancer among other malignancies.

Calendar time as an instrumental variable in assessing the risk of heart failure with antihyperglycemic drugs

OBJECTIVE

In recent years, second-line diabetes treatment with dipeptidyl peptidase-4 inhibitors (DPP-4i) increased with a corresponding decrease in thiazolidinediones (TZDs). Using hospitalization for heart failure (HF) as a positive control outcome, we explored the use of calendar time as an instrumental variable (IV) and compared this approach to an active comparator new-user study.

METHODS

We identified DPP-4i or TZD initiators after a 6-month washout using Medicare claims 2006-2013. The IV was defined as a binary variable comparing initiators during October 2010 to December 2013 (postperiod) versus January 2008 to May 2010 (preperiod). We examined IV strength and estimated risk differences (RDs) for HF using Kaplan-Meier curves, which were compared with propensity score (PS)-weighted RD for DPP-4i versus TZD.

RESULTS

The IV compared 22 696 initiators (78% DPP-4i) in the postperiod versus 20 283 initiators (38% DPP-4i) in the preperiod, resulting in 40% compliance. The active-comparator (PS-weighted) approach compared 26 198 DPP-4i and 18 842 TZD initiators. Covariate balance across IV levels was slightly better than across treatments (standardized difference, 3% vs 4.5%). The 1- and 2-year local average treatment effects of RD of HF per 100 patients in the "compliers" (95% confidence intervals) were -0.62 (-0.99 to -0.25) and -0.88 (-1.46 to -0.25). Corresponding PS-weighted results were -0.20 (-0.33 to -0.05) and -0.18 (-0.30 to 0.03).

CONCLUSION

Both approaches indicated lesser risk of HF hospitalizations among DPP-4i vs TZD initiators. The magnitude of the estimated effects may differ due to differences in the target populations and assumptions. Calendar time can be leveraged as an IV when market dynamics lead to profound changes in treatments.

Methodologic limitations of prescription opioid safety research and recommendations for improving the evidence base

PURPOSE

The ongoing opioid epidemic has claimed more than a quarter million Americans' lives over the past 15 years. The epidemic began with an escalation of prescription opioid deaths and has now evolved to include secondary waves of illicit heroin and fentanyl deaths, while the deaths due to prescription opioid overdoses are still increasing. In response, the Centers for Disease Control and Prevention (CDC) moved to limit opioid prescribing with the release of opioid prescribing guidelines for chronic noncancer pain in March 2016. The guidelines represent a logical and timely federal response to this growing crisis. However, CDC acknowledged that the evidence base linking opioid prescribing to opioid use disorders and overdose was grades 3 and 4.

METHODS

Motivated by the need to strengthen the evidence base, this review details limitations of the opioid safety studies cited in the CDC guidelines with a focus on methodological limitations related to internal and external validity.

RESULTS

Internal validity concerns were related to poor confounding control, variable misclassification, selection bias, competing risks, and potential competing interventions. External validity concerns arose from the use of limited source populations, historical data (in a fast-changing epidemic), and issues with handling of cancer and acute pain patients' data. We provide a nonexhaustive list of 7 recommendations to address these limitations in future opioid safety studies.

CONCLUSION

Strengthening the opioid safety evidence base will aid any future revisions of the CDC guidelines and enhance their prevention impact.

On two-stage estimation of structural instrumental variable models

Two-stage least squares estimation is popular for structural equation models with unmeasured confounders. In such models, both the outcome and the exposure are assumed to follow linear models conditional on the measured confounders and instrumental variable, which is related to the outcome only via its relation with the exposure. We consider data where both the outcome and the exposure may be incompletely observed, with particular attention to the case where both are censored event times. A general class of two-stage minimum distance estimators is proposed that separately fits linear models for the outcome and exposure and then uses a minimum distance criterion based on the reduced-form model for the outcome to estimate the regression parameters of interest. An optimal minimum distance estimator is identified which may be superior to the usual two-stage least squares estimator with fully observed data. Simulation studies demonstrate that the proposed methods perform well with realistic sample sizes. Their practical utility is illustrated in a study of the comparative effectiveness of colon cancer treatments, where the effect of chemotherapy on censored survival times may be confounded with patient status.

Instrumental Variable Analyses in Pharmacoepidemiology: What Target Trials Do We Emulate?

Purpose of review

When leveraging observational data to estimate treatment effects, it is useful to explicitly specify the “target trial” the investigators aspire to emulate. One concern is whether a proposed analysis plan can address the realities of the differences between the available non-randomized observational study and the target trial. When large or unknown sources of unmeasured confounding are suspected, investigators might consider turning to instrumental variable (IV) methods. Of course, the interpretation and appropriateness of IV analyses need to be considered carefully. The purpose of this review is to summarize recent methodologic advancements in how epidemiologists weigh the validity of an IV analysis and to place these methodologic advancements in the context of the feasible target trial’s protocol components.

Recent findings

There have been increased development and application of tools for sensitivity analyses, falsification strategies, and the identification of previously overlooked problems with IV analyses as applied in pharmacoepidemiology. Many of these recent insights can be seen as articulating restrictions on or tradeoffs between the types of target trials that can be validly emulated when using a classical IV analysis.

Summary

Putting classical IV methods in the context of target trials underscores the importance of recent methodologic developments and, more generally, when and how an IV analysis would be appropriate. We see that some tradeoffs in defining the target trials are unavoidable, that some tradeoffs may be offset or explored via sensitivity analyses, and that this serves as a framework for scientific discourse regarding IV and non-IV results emulating potentially different trials with different tradeoffs.

How Does Outcomes Research Help Advance Our Knowledge of Patient Outcomes in Hepatopancreaticobiliary Surgery?

Randomized controlled trials have historically been regarded as the gold standard of modern clinical research tools, allowing us to elucidate the efficacy of novel therapeutics in an unparalleled manner. However, when attempting to generalize trial results to broader populations, it becomes apparent that the unexplained outcome variability exists among treatment recipients, suggesting that randomized controlled trials harbor inherent limitations. Herein, we explore the benefits of health services (outcomes) research utilization in addressing variation in patient outcomes following surgical intervention in the non-randomized setting, with a specific focus on hepatopancreaticobiliary surgery outcomes. To achieve this, we have constructed a framework that outlines the complex interactions existing between therapeutic, patient, and provider factors that subsequently lead to variation in outcomes. By exploring examples in the current literature, we have highlighted the areas where the knowledge is currently lacking and can be further strengthened through the application of outcomes research. Furthermore, we have attempted to demonstrate the utility of alternative study designs in the investigation of novel clinical questions currently unanswered in the field of hepatopancreaticobiliary surgery.