An omnibus non-parametric test of equality in distribution for unknown functions

Bridging Differences in Cohort Analyses of the Relationship between Secondhand Smoke Exposure during Pregnancy and Birth Weight: The Transportability Framework in the ECHO Program

BACKGROUND

Estimates for the effects of environmental exposures on health outcomes, including secondhand smoke (SHS) exposure, often present considerable variability across studies. Knowledge of the reasons behind these differences can aid our understanding of effects in specific populations as well as inform practices of combining data from multiple studies.

OBJECTIVES

This study aimed to assess the presence of effect modification by measured sociodemographic characteristics on the effect of SHS exposure during pregnancy on birth weights that may drive differences observed across cohorts. We also aimed to quantify the extent to which differences in the cohort mean effects observed across cohorts in the Environmental influences on Child Health Outcomes (ECHO) consortium are due to differing distributions of these characteristics.

METHODS

We assessed the presence of effect modification and transportability of effect estimates across five ECHO cohorts in a total of 6,771 mother-offspring dyads. We assessed the presence of effect modification via gradient boosting of regression trees based on the H-statistic. We estimated individual cohort effects using linear models and targeted maximum likelihood estimation (TMLE). We then estimated transported effects from one cohort to each of the remaining cohorts using a robust nonparametric estimation approach relying on TMLE estimators and compared them to the original effect estimates for these cohorts.

RESULTS

Observed effect estimates varied across the five cohorts, ranging from significantly lower birth weight associated with exposure [- 167.3 g ; 95% confidence interval (CI): - 270.4 , - 64.1 ] to higher birth weight with wide CIs, including the null (42.4 g ; 95% CI: - 15.0 , 99.8). Transported effect estimates only minimally explained differences in the point estimates for two out of the four cohort pairs.

DISCUSSION

Our findings of weak to moderate evidence of effect modification and transportability indicate that unmeasured individual-level and contextual factors and sources of bias may be responsible for differences in the effect estimates observed across ECHO cohorts. https://doi.org/10.1289/EHP13961.

Using instrumental variables to address unmeasured confounding in causal mediation analysis

Mediation analysis is a strategy for understanding the mechanisms by which interventions affect later outcomes. However, unobserved confounding concerns may be compounded in mediation analyses, as there may be unobserved exposure-outcome, exposure-mediator, and mediator-outcome confounders. Instrumental variables (IVs) are a popular identification strategy in the presence of unobserved confounding. However, in contrast to the rich literature on the use of IV methods to identify and estimate a total effect of a non-randomized exposure, there has been almost no research into using IV as an identification strategy to identify mediational indirect effects. In response, we define and nonparametrically identify novel estimands-double complier interventional direct and indirect effects-when 2, possibly related, IVs are available, one for the exposure and another for the mediator. We propose nonparametric, robust, efficient estimators for these effects and apply them to a housing voucher experiment.

Efficient Estimation under Data Fusion

We aim to make inferences about a smooth, finite-dimensional parameter by fusing data from multiple sources together. Previous works have studied the estimation of a variety of parameters in similar data fusion settings, including in the estimation of the average treatment effect and average reward under a policy, with the majority of them merging one historical data source with covariates, actions, and rewards and one data source of the same covariates. In this work, we consider the general case where one or more data sources align with each part of the distribution of the target population, for example, the conditional distribution of the reward given actions and covariates. We describe potential gains in efficiency that can arise from fusing these data sources together in a single analysis, which we characterize by a reduction in the semiparametric efficiency bound. We also provide a general means to construct estimators that achieve these bounds. In numerical simulations, we illustrate marked improvements in efficiency from using our proposed estimators rather than their natural alternatives. Finally, we illustrate the magnitude of efficiency gains that can be realized in vaccine immunogenicity studies by fusing data from two HIV vaccine trials.

Estimating the area under the ROC curve when transporting a prediction model to a target population

We propose methods for estimating the area under the receiver operating characteristic (ROC) curve (AUC) of a prediction model in a target population that differs from the source population that provided the data used for original model development. If covariates that are associated with model performance, as measured by the AUC, have a different distribution in the source and target populations, then AUC estimators that only use data from the source population will not reflect model performance in the target population. Here, we provide identification results for the AUC in the target population when outcome and covariate data are available from the sample of the source population, but only covariate data are available from the sample of the target population. In this setting, we propose three estimators for the AUC in the target population and show that they are consistent and asymptotically normal. We evaluate the finite-sample performance of the estimators using simulations and use them to estimate the AUC in a nationally representative target population from the National Health and Nutrition Examination Survey for a lung cancer risk prediction model developed using source population data from the National Lung Screening Trial.

Elastic integrative analysis of randomised trial and real-world data for treatment heterogeneity estimation

We propose a test-based elastic integrative analysis of the randomised trial and real-world data to estimate treatment effect heterogeneity with a vector of known effect modifiers. When the real-world data are not subject to bias, our approach combines the trial and real-world data for efficient estimation. Utilising the trial design, we construct a test to decide whether or not to use real-world data. We characterise the asymptotic distribution of the test-based estimator under local alternatives. We provide a data-adaptive procedure to select the test threshold that promises the smallest mean square error and an elastic confidence interval with a good finite-sample coverage property.

Efficient and robust methods for causally interpretable meta-analysis: Transporting inferences from multiple randomized trials to a target population

We present methods for causally interpretable meta-analyses that combine information from multiple randomized trials to draw causal inferences for a target population of substantive interest. We consider identifiability conditions, derive implications of the conditions for the law of the observed data, and obtain identification results for transporting causal inferences from a collection of independent randomized trials to a new target population in which experimental data may not be available. We propose an estimator for the potential outcome mean in the target population under each treatment studied in the trials. The estimator uses covariate, treatment, and outcome data from the collection of trials, but only covariate data from the target population sample. We show that it is doubly robust in the sense that it is consistent and asymptotically normal when at least one of the models it relies on is correctly specified. We study the finite sample properties of the estimator in simulation studies and demonstrate its implementation using data from a multicenter randomized trial.

Optimally Choosing Medication Type for Patients With Opioid Use Disorder

Patients with opioid use disorder (OUD) tend to get assigned to one of 3 medications based on the treatment program to which the patient presents (e.g., opioid treatment programs tend to treat patients with methadone, while office-based practices tend to prescribe buprenorphine). It is possible that optimally matching patients with treatment type would reduce the risk of return to regular opioid use (RROU). We analyzed data from 3 comparative effectiveness trials from the US National Institute on Drug Abuse Clinical Trials Network (CTN0027, 2006-2010; CTN0030, 2006-2009; and CTN0051 2014-2017), in which patients with OUD (n = 1,459) were assigned to treatment with either injection extended-release naltrexone (XR-NTX), sublingual buprenorphine-naloxone (BUP-NX), or oral methadone. We learned an individualized rule by which to assign medication type such that risk of RROU during 12 weeks of treatment would be minimized, and then estimated the amount by which RROU risk could be reduced if the rule were applied. Applying our estimated treatment rule would reduce risk of RROU compared with treating everyone with methadone (relative risk (RR) = 0.79, 95% confidence interval (CI): 0.60, 0.97) or treating everyone with XR-NTX (RR = 0.71, 95% CI: 0.47, 0.96). Applying the estimated treatment rule would have resulted in a similar risk of RROU to that of with treating everyone with BUP-NX (RR = 0.92, 95% CI: 0.73, 1.11).

Improving efficiency of inference in clinical trials with external control data

Suppose we are interested in the effect of a treatment in a clinical trial. The efficiency of inference may be limited due to small sample size. However, external control data are often available from historical studies. Motivated by an application to Helicobacter pylori infection, we show how to borrow strength from such data to improve efficiency of inference in the clinical trial. Under an exchangeability assumption about the potential outcome mean, we show that the semiparametric efficiency bound for estimating the average treatment effect can be reduced by incorporating both the clinical trial data and external controls. We then derive a doubly robust and locally efficient estimator. The improvement in efficiency is prominent especially when the external control data set has a large sample size and small variability. Our method allows for a relaxed overlap assumption, and we illustrate with the case where the clinical trial only contains a treated group. We also develop doubly robust and locally efficient approaches that extrapolate the causal effect in the clinical trial to the external population and the overall population. Our results also offer a meaningful implication for trial design and data collection. We evaluate the finite-sample performance of the proposed estimators via simulation. In the Helicobacter pylori infection application, our approach shows that the combination treatment has potential efficacy advantages over the triple therapy.

Estimating the impact of stimulant use on initiation of buprenorphine and extended-release naltrexone in two clinical trials and real-world populations

BACKGROUND

Co-use of stimulants and opioids is rapidly increasing. Randomized clinical trials (RCTs) have established the efficacy of medications for opioid use disorder (MOUD), but stimulant use may decrease the likelihood of initiating MOUD treatment. Furthermore, trial participants may not represent "real-world" populations who would benefit from treatment.

METHODS

We conducted a two-stage analysis. First, associations between stimulant use (time-varying urine drug screens for cocaine, methamphetamine, or amphetamines) and initiation of buprenorphine or extended-release naltrexone (XR-NTX) were estimated across two RCTs (CTN-0051 X:BOT and CTN-0067 CHOICES) using adjusted Cox regression models. Second, results were generalized to three target populations who would benefit from MOUD: Housed adults identifying the need for OUD treatment, as characterized by the National Survey on Drug Use and Health (NSDUH); adults entering OUD treatment, as characterized by Treatment Episodes Dataset (TEDS); and adults living in rural regions of the U.S. with high rates of injection drug use, as characterized by the Rural Opioids Initiative (ROI). Generalizability analyses adjusted for differences in demographic characteristics, substance use, housing status, and depression between RCT and target populations using inverse probability of selection weighting.

RESULTS

Analyses included 673 clinical trial participants, 139 NSDUH respondents (weighted to represent 661,650 people), 71,751 TEDS treatment episodes, and 1,933 ROI participants. The majority were aged 30-49 years, male, and non-Hispanic White. In RCTs, stimulant use reduced the likelihood of MOUD initiation by 32% (adjusted HR [aHR] = 0.68, 95% CI 0.49-0.94, p = 0.019). Stimulant use associations were slightly attenuated and non-significant among housed adults needing treatment (25% reduction, aHR = 0.75, 0.48-1.18, p = 0.215) and adults entering OUD treatment (28% reduction, aHR = 0.72, 0.51-1.01, p = 0.061). The association was more pronounced, but still non-significant among rural people injecting drugs (39% reduction, aHR = 0.61, 0.35-1.06, p = 0.081). Stimulant use had a larger negative impact on XR-NTX initiation compared to buprenorphine, especially in the rural population (76% reduction, aHR = 0.24, 0.08-0.69, p = 0.008).

CONCLUSIONS

Stimulant use is a barrier to buprenorphine or XR-NTX initiation in clinical trials and real-world populations that would benefit from OUD treatment. Interventions to address stimulant use among patients with OUD are urgently needed, especially among rural people injecting drugs, who already suffer from limited access to MOUD.

Transporting stochastic direct and indirect effects to new populations

Transported mediation effects may contribute to understanding how interventions work differently when applied to new populations. However, we are not aware of any estimators for such effects. Thus, we propose two doubly robust, efficient estimators of transported stochastic (also called randomized interventional) direct and indirect effects. We demonstrate their finite sample properties in a simulation study. We then apply the preferred substitution estimator to longitudinal data from the Moving to Opportunity Study, a large-scale housing voucher experiment, to transport stochastic indirect effect estimates of voucher receipt in childhood on subsequent risk of mental health or substance use disorder mediated through parental employment across sites, thereby gaining understanding of drivers of the site differences.

Efficiently transporting causal direct and indirect effects to new populations under intermediate confounding and with multiple mediators

The same intervention can produce different effects in different sites. Existing transport mediation estimators can estimate the extent to which such differences can be explained by differences in compositional factors and the mechanisms by which mediating or intermediate variables are produced; however, they are limited to consider a single, binary mediator. We propose novel nonparametric estimators of transported interventional (in)direct effects that consider multiple, high-dimensional mediators and a single, binary intermediate variable. They are multiply robust, efficient, asymptotically normal, and can incorporate data-adaptive estimation of nuisance parameters. They can be applied to understand differences in treatment effects across sites and/or to predict treatment effects in a target site based on outcome data in source sites.

Using Transportability to Understand Differences in Mediation Mechanisms Across Trial Sites of a Housing Voucher Experiment

BACKGROUND

Randomized trials may have different effects in different settings. Moving to Opportunity (MTO), a housing experiment, is one such example. Previously, we examined the extent to which MTO's overall effects on adolescent substance use and mental health outcomes were transportable across the sites to disentangle the contributions of differences in population composition versus differences in contextual factors to site differences. However, to further understand reasons for different site effects, it may be beneficial to examine mediation mechanisms and the degree to which they too are transportable across sites.

METHODS

We used longitudinal data from MTO youth. We examined mediators summarizing aspects of the school environment over the 10-15 year follow-up. Outcomes of past-year substance use, mental health, and risk behavior were assessed at the final timepoint when participants were 10-20 years old. We used doubly robust and efficient substitution estimators to estimate (1) indirect effects by MTO site and (2) transported indirect effects from one site to another.

RESULTS

Differences in indirect effect estimates were most pronounced between Chicago and Los Angeles. Using transport estimators to account for differences in baseline covariates, likelihood of using the voucher to move, and mediator distributions partially to fully accounted for site differences in indirect effect estimates in 10 of the 12 pathways examined.

CONCLUSIONS

Using transport estimators can provide an evidence-based approach for understanding the extent to which differences in compositional factors contribute to differences in indirect effect estimates across sites, and ultimately, to understanding why interventions may have different effects when applied to new populations.

Detecting Anomalies Among Practice Sites Within Multicenter Trials

BACKGROUND

Recent multisite trials reveal striking heterogeneities in results between trial sites. These may be because of population differences indicating different treatment benefits among different types of participants or site anomalies, such as failures to adhere to study protocols that could negatively affect study validity. We sought to determine whether a new data analysis strategy-transportability methods-could suggest site anomalies not readily identified through standard methods.

METHODS AND RESULTS

We applied transportability methods to 2 large, multicenter cardiovascular disease treatment trials: the TOPCAT trial (Treatment of Preserved Cardiac Function Heart Failure With an Aldosterone Antagonist; n=3445) comparing spironolactone to placebo for heart failure (for which site anomalies were suspected) and the ACCORD BP trial (Action to Control Cardiovascular Risk in Diabetes-Blood Pressure; n=4733) comparing intensive-to-standard blood pressure treatment (for which site anomalies were not suspected). The transportability methods give expected results by standardizing from one site to another using data on participant covariates. The difference between the expected and observed results was assessed using calibration tests to identify whether treatment-effect differences between sites could be explained by participant population characteristics. Standard regression methods did not detect heterogeneities in TOPCAT between Russia/Georgia study sites suspected of study protocol violations and sites in the Americas ( P=0.12 for difference in primary cardiovascular outcome; P=0.20 for difference in total mortality). The transportability methods, however, detected the difference between Russia/Georgia sites and sites in the Americas ( P<0.001) and found that measured participant characteristics did not explain the between-site discrepancies. The transport methods found no such discrepancies between sites in ACCORD BP, suggesting participant characteristics explained between-site differences.

CONCLUSIONS

Transportability methods may be superior to standard approaches for detecting anomalies within multicenter randomized trials and assist data monitoring boards to determine whether important treatment-effect heterogeneities can be attributed to participant differences or potentially to site performance differences requiring further investigation.