Targeted Learning with an Undersmoothed Lasso Propensity Score Model for Large-Scale Covariate Adjustment in Healthcare Database Studies

Lasso regression is widely used for large-scale propensity score (PS) estimation in healthcare database studies. In these settings, previous work has shown that undersmoothing (overfitting) Lasso PS models can improve confounding control, but it can also cause problems of non-overlap in covariate distributions. It remains unclear how to select the degree of undersmoothing when fitting large-scale Lasso PS models to improve confounding control while avoiding issues that can result from reduced covariate overlap. Here, we used simulations to evaluate the performance of using collaborative-controlled targeted learning to data-adaptively select the degree of undersmoothing when fitting large-scale PS models within both singly and doubly robust frameworks to reduce bias in causal estimators. Simulations showed that collaborative learning can data-adaptively select the degree of undersmoothing to reduce bias in estimated treatment effects. Results further showed that when fitting undersmoothed Lasso PS-models, the use of cross-fitting was important for avoiding non-overlap in covariate distributions and reducing bias in causal estimates.

Targeted malaria elimination interventions reduce Plasmodium falciparum infections up to 3 kilometers away

Malaria elimination interventions in low-transmission settings aim to extinguish hot spots and prevent transmission to nearby areas. In malaria elimination settings, the World Health Organization recommends reactive, focal interventions targeted to the area near malaria cases shortly after they are detected. A key question is whether these interventions reduce transmission to nearby uninfected or asymptomatic individuals who did not receive interventions. Here, we measured direct effects (among intervention recipients) and spillover effects (among non-recipients) of reactive, focal interventions delivered within 500m of confirmed malaria index cases in a cluster-randomized trial in Namibia. The trial delivered malaria chemoprevention (artemether lumefantrine) and vector control (indoor residual spraying with Actellic) separately and in combination using a factorial design. We compared incidence, infection prevalence, and seroprevalence between study arms among intervention recipients (direct effects) and non-recipients (spillover effects) up to 3 km away from index cases. We calculated incremental cost-effectiveness ratios accounting for spillover effects. The combined chemoprevention and vector control intervention produced direct effects and spillover effects. In the primary analysis among non-recipients within 1 km from index cases, the combined intervention reduced malaria incidence by 43% (95% CI 20%, 59%). In secondary analyses among non-recipients 500m-3 km from interventions, the combined intervention reduced infection by 79% (6%, 95%) and seroprevalence 34% (20%, 45%). Accounting for spillover effects increased the cost-effectiveness of the combined intervention by 37%. Our findings provide the first evidence that targeting hot spots with combined chemoprevention and vector control interventions can indirectly benefit non-recipients up to 3 km away.

Case study of semaglutide and cardiovascular outcomes: An application of the Causal Roadmap to a hybrid design for augmenting an RCT control arm with real-world data

Introduction

Increasing interest in real-world evidence has fueled the development of study designs incorporating real-world data (RWD). Using the Causal Roadmap, we specify three designs to evaluate the difference in risk of major adverse cardiovascular events (MACE) with oral semaglutide versus standard-of-care: (1) the actual sequence of non-inferiority and superiority randomized controlled trials (RCTs), (2) a single RCT, and (3) a hybrid randomized-external data study.

Methods

The hybrid design considers integration of the PIONEER 6 RCT with RWD controls using the experiment-selector cross-validated targeted maximum likelihood estimator. We evaluate 95% confidence interval coverage, power, and average patient time during which participants would be precluded from receiving a glucagon-like peptide-1 receptor agonist (GLP1-RA) for each design using simulations. Finally, we estimate the effect of oral semaglutide on MACE for the hybrid PIONEER 6-RWD analysis.

Results

In simulations, Designs 1 and 2 performed similarly. The tradeoff between decreased coverage and patient time without the possibility of a GLP1-RA for Designs 1 and 3 depended on the simulated bias. In real data analysis using Design 3, external controls were integrated in 84% of cross-validation folds, resulting in an estimated risk difference of -1.53%-points (95% CI -2.75%-points to -0.30%-points).

Conclusions

The Causal Roadmap helps investigators to minimize potential bias in studies using RWD and to quantify tradeoffs between study designs. The simulation results help to interpret the level of evidence provided by the real data analysis in support of the superiority of oral semaglutide versus standard-of-care for cardiovascular risk reduction.

A causal roadmap for generating high-quality real-world evidence

Increasing emphasis on the use of real-world evidence (RWE) to support clinical policy and regulatory decision-making has led to a proliferation of guidance, advice, and frameworks from regulatory agencies, academia, professional societies, and industry. A broad spectrum of studies use real-world data (RWD) to produce RWE, ranging from randomized trials with outcomes assessed using RWD to fully observational studies. Yet, many proposals for generating RWE lack sufficient detail, and many analyses of RWD suffer from implausible assumptions, other methodological flaws, or inappropriate interpretations. The Causal Roadmap is an explicit, itemized, iterative process that guides investigators to prespecify study design and analysis plans; it addresses a wide range of guidance within a single framework. By supporting the transparent evaluation of causal assumptions and facilitating objective comparisons of design and analysis choices based on prespecified criteria, the Roadmap can help investigators to evaluate the quality of evidence that a given study is likely to produce, specify a study to generate high-quality RWE, and communicate effectively with regulatory agencies and other stakeholders. This paper aims to disseminate and extend the Causal Roadmap framework for use by clinical and translational researchers; three companion papers demonstrate applications of the Causal Roadmap for specific use cases.

Evaluating and improving real-world evidence with Targeted Learning

BACKGROUND

The Targeted Learning roadmap provides a systematic guide for generating and evaluating real-world evidence (RWE). From a regulatory perspective, RWE arises from diverse sources such as randomized controlled trials that make use of real-world data, observational studies, and other study designs. This paper illustrates a principled approach to assessing the validity and interpretability of RWE.

METHODS

We applied the roadmap to a published observational study of the dose-response association between ritodrine hydrochloride and pulmonary edema among women pregnant with twins in Japan. The goal was to identify barriers to causal effect estimation beyond unmeasured confounding reported by the study's authors, and to explore potential options for overcoming the barriers that robustify results.

RESULTS

Following the roadmap raised issues that led us to formulate alternative causal questions that produced more reliable, interpretable RWE. The process revealed a lack of information in the available data to identify a causal dose-response curve. However, under explicit assumptions the effect of treatment with any amount of ritodrine versus none, albeit a less ambitious parameter, can be estimated from data.

CONCLUSIONS

Before RWE can be used in support of clinical and regulatory decision-making, its quality and reliability must be systematically evaluated. The TL roadmap prescribes how to carry out a thorough, transparent, and realistic assessment of RWE. We recommend this approach be a routine part of any decision-making process.

Two-Stage TMLE to reduce bias and improve efficiency in cluster randomized trials

Cluster randomized trials (CRTs) randomly assign an intervention to groups of individuals (e.g., clinics or communities) and measure outcomes on individuals in those groups. While offering many advantages, this experimental design introduces challenges that are only partially addressed by existing analytic approaches. First, outcomes are often missing for some individuals within clusters. Failing to appropriately adjust for differential outcome measurement can result in biased estimates and inference. Second, CRTs often randomize limited numbers of clusters, resulting in chance imbalances on baseline outcome predictors between arms. Failing to adaptively adjust for these imbalances and other predictive covariates can result in efficiency losses. To address these methodological gaps, we propose and evaluate a novel two-stage targeted minimum loss-based estimator to adjust for baseline covariates in a manner that optimizes precision, after controlling for baseline and postbaseline causes of missing outcomes. Finite sample simulations illustrate that our approach can nearly eliminate bias due to differential outcome measurement, while existing CRT estimators yield misleading results and inferences. Application to real data from the SEARCH community randomized trial demonstrates the gains in efficiency afforded through adaptive adjustment for baseline covariates, after controlling for missingness on individual-level outcomes.

Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learning

Empirical risk minimization (ERM) is the workhorse of machine learning, whether for classification and regression or for off-policy policy learning, but its model-agnostic guarantees can fail when we use adaptively collected data, such as the result of running a contextual bandit algorithm. We study a generic importance sampling weighted ERM algorithm for using adaptively collected data to minimize the average of a loss function over a hypothesis class and provide first-of-their-kind generalization guarantees and fast convergence rates. Our results are based on a new maximal inequality that carefully leverages the importance sampling structure to obtain rates with the good dependence on the exploration rate in the data. For regression, we provide fast rates that leverage the strong convexity of squared-error loss. For policy learning, we provide regret guarantees that close an open gap in the existing literature whenever exploration decays to zero, as is the case for bandit-collected data. An empirical investigation validates our theory.

Post-Contextual-Bandit Inference

Contextual bandit algorithms are increasingly replacing non-adaptive A/B tests in e-commerce, healthcare, and policymaking because they can both improve outcomes for study participants and increase the chance of identifying good or even best policies. To support credible inference on novel interventions at the end of the study, nonetheless, we still want to construct valid confidence intervals on average treatment effects, subgroup effects, or value of new policies. The adaptive nature of the data collected by contextual bandit algorithms, however, makes this difficult: standard estimators are no longer asymptotically normally distributed and classic confidence intervals fail to provide correct coverage. While this has been addressed in non-contextual settings by using stabilized estimators, the contextual setting poses unique challenges that we tackle for the first time in this paper. We propose the Contextual Adaptive Doubly Robust (CADR) estimator, the first estimator for policy value that is asymptotically normal under contextual adaptive data collection. The main technical challenge in constructing CADR is designing adaptive and consistent conditional standard deviation estimators for stabilization. Extensive numerical experiments using 57 OpenML datasets demonstrate that confidence intervals based on CADR uniquely provide correct coverage.

Nonparametric bootstrap inference for the targeted highly adaptive least absolute shrinkage and selection operator (LASSO) estimator

The Highly-Adaptive least absolute shrinkage and selection operator (LASSO) Targeted Minimum Loss Estimator (HAL-TMLE) is an efficient plug-in estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance functions (i.e., relevant part of data distribution) are finite. It relies on an initial estimator (HAL-MLE) of the nuisance functions by minimizing the empirical risk over the parameter space under the constraint that the sectional variation norm of the candidate functions are bounded by a constant, where this constant can be selected with cross-validation. In this article we establish that the nonparametric bootstrap for the HAL-TMLE, fixing the value of the sectional variation norm at a value larger or equal than the cross-validation selector, provides a consistent method for estimating the normal limit distribution of the HAL-TMLE. In order to optimize the finite sample coverage of the nonparametric bootstrap confidence intervals, we propose a selection method for this sectional variation norm that is based on running the nonparametric bootstrap for all values of the sectional variation norm larger than the one selected by cross-validation, and subsequently determining a value at which the width of the resulting confidence intervals reaches a plateau. We demonstrate our method for 1) nonparametric estimation of the average treatment effect when observing a covariate vector, binary treatment, and outcome, and for 2) nonparametric estimation of the integral of the square of the multivariate density of the data distribution. In addition, we also present simulation results for these two examples demonstrating the excellent finite sample coverage of bootstrap-based confidence intervals.

HIV Testing and Treatment with the Use of a Community Health Approach in Rural Africa

BACKGROUND

Universal antiretroviral therapy (ART) with annual population testing and a multidisease, patient-centered strategy could reduce new human immunodeficiency virus (HIV) infections and improve community health.

METHODS

We randomly assigned 32 rural communities in Uganda and Kenya to baseline HIV and multidisease testing and national guideline-restricted ART (control group) or to baseline testing plus annual testing, eligibility for universal ART, and patient-centered care (intervention group). The primary end point was the cumulative incidence of HIV infection at 3 years. Secondary end points included viral suppression, death, tuberculosis, hypertension control, and the change in the annual incidence of HIV infection (which was evaluated in the intervention group only).

RESULTS

A total of 150,395 persons were included in the analyses. Population-level viral suppression among 15,399 HIV-infected persons was 42% at baseline and was higher in the intervention group than in the control group at 3 years (79% vs. 68%; relative prevalence, 1.15; 95% confidence interval [CI], 1.11 to 1.20). The annual incidence of HIV infection in the intervention group decreased by 32% over 3 years (from 0.43 to 0.31 cases per 100 person-years; relative rate, 0.68; 95% CI, 0.56 to 0.84). However, the 3-year cumulative incidence (704 incident HIV infections) did not differ significantly between the intervention group and the control group (0.77% and 0.81%, respectively; relative risk, 0.95; 95% CI, 0.77 to 1.17). Among HIV-infected persons, the risk of death by year 3 was 3% in the intervention group and 4% in the control group (0.99 vs. 1.29 deaths per 100 person-years; relative risk, 0.77; 95% CI, 0.64 to 0.93). The risk of HIV-associated tuberculosis or death by year 3 among HIV-infected persons was 4% in the intervention group and 5% in the control group (1.19 vs. 1.50 events per 100 person-years; relative risk, 0.79; 95% CI, 0.67 to 0.94). At 3 years, 47% of adults with hypertension in the intervention group and 37% in the control group had hypertension control (relative prevalence, 1.26; 95% CI, 1.15 to 1.39).

CONCLUSIONS

Universal HIV treatment did not result in a significantly lower incidence of HIV infection than standard care, probably owing to the availability of comprehensive baseline HIV testing and the rapid expansion of ART eligibility in the control group. (Funded by the National Institutes of Health and others; SEARCH ClinicalTrials.gov number, NCT01864603.).

Double Robust Efficient Estimators of Longitudinal Treatment Effects: Comparative Performance in Simulations and a Case Study

A number of sophisticated estimators of longitudinal effects have been proposed for estimating the intervention-specific mean outcome. However, there is a relative paucity of research comparing these methods directly to one another. In this study, we compare various approaches to estimating a causal effect in a longitudinal treatment setting using both simulated data and data measured from a human immunodeficiency virus cohort. Six distinct estimators are considered: (i) an iterated conditional expectation representation, (ii) an inverse propensity weighted method, (iii) an augmented inverse propensity weighted method, (iv) a double robust iterated conditional expectation estimator, (v) a modified version of the double robust iterated conditional expectation estimator, and (vi) a targeted minimum loss-based estimator. The details of each estimator and its implementation are presented along with nuisance parameter estimation details, which include potentially pooling the observed data across all subjects regardless of treatment history and using data adaptive machine learning algorithms. Simulations are constructed over six time points, with each time point steadily increasing in positivity violations. Estimation is carried out for both the simulations and applied example using each of the six estimators under both stratified and pooled approaches of nuisance parameter estimation. Simulation results show that double robust estimators remained without meaningful bias as long as at least one of the two nuisance parameters were estimated with a correctly specified model. Under full misspecification, the bias of the double robust estimators remained better than that of the inverse propensity estimator under misspecification, but worse than the iterated conditional expectation estimator. Weighted estimators tended to show better performance than the covariate estimators. As positivity violations increased, the mean squared error and bias of all estimators considered became worse, with covariate-based double robust estimators especially susceptible. Applied analyses showed similar estimates at most time points, with the important exception of the inverse propensity estimator which deviated markedly as positivity violations increased. Given its efficiency, ability to respect the parameter space, and observed performance, we recommend the pooled and weighted targeted minimum loss-based estimator.

Smoking Is Associated with Higher Disease Activity in Rheumatoid Arthritis: A Longitudinal Study Controlling for Time-varying Covariates

OBJECTIVE

Prior studies around the relationship between smoking and rheumatoid arthritis (RA) disease activity have reported inconsistent findings, which may be ascribed to heterogeneous study designs or biases in statistical analyses. We examined the association between smoking and RA outcomes using statistical methods that account for time-varying confounding and loss to followup.

METHODS

We included 282 individuals with an RA diagnosis using electronic health record data collected at a public hospital between 2013 and 2017. Current smoking status and disease activity were assessed at each visit; covariates included sex, race/ethnicity, age, obesity, and medication use. We used longitudinal targeted maximum likelihood estimation to estimate the causal effect of smoking on disease activity measures at 27 months, and compared results to conventional longitudinal methods.

RESULTS

Smoking was associated with an increase of 0.64 units in the patient global score compared to nonsmoking (p = 0.01), and with 2.58 more swollen joints (p < 0.001). While smoking was associated with a higher clinical disease activity score (2.11), the difference was not statistically significant (p = 0.22). We found no association between smoking and physician global score, or C-reactive protein levels, and an inverse association between smoking and tender joint count (p = 0.05). Analyses using conventional methods showed a null relationship for all outcomes.

CONCLUSION

Smoking is associated with higher levels of disease activity in RA. Causal methods may be useful for investigations of additional exposures on longitudinal outcome measures in rheumatologic disease.

Using Super Learner Prediction Modeling to Improve High-dimensional Propensity Score Estimation

The high-dimensional propensity score is a semiautomated variable selection algorithm that can supplement expert knowledge to improve confounding control in nonexperimental medical studies utilizing electronic healthcare databases. Although the algorithm can be used to generate hundreds of patient-level variables and rank them by their potential confounding impact, it remains unclear how to select the optimal number of variables for adjustment. We used plasmode simulations based on empirical data to discuss and evaluate data-adaptive approaches for variable selection and prediction modeling that can be combined with the high-dimensional propensity score to improve confounding control in large healthcare databases. We considered approaches that combine the high-dimensional propensity score with Super Learner prediction modeling, a scalable version of collaborative targeted maximum-likelihood estimation, and penalized regression. We evaluated performance using bias and mean squared error (MSE) in effect estimates. Results showed that the high-dimensional propensity score can be sensitive to the number of variables included for adjustment and that severe overfitting of the propensity score model can negatively impact the properties of effect estimates. Combining the high-dimensional propensity score with Super Learner was the most consistent strategy, in terms of reducing bias and MSE in the effect estimates, and may be promising for semiautomated data-adaptive propensity score estimation in high-dimensional covariate datasets.

The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification

Artificial neural networks have been successfully applied to a variety of machine learning tasks, including image recognition, semantic segmentation, and machine translation. However, few studies fully investigated ensembles of artificial neural networks. In this work, we investigated multiple widely used ensemble methods, including unweighted averaging, majority voting, the Bayes Optimal Classifier, and the (discrete) Super Learner, for image recognition tasks, with deep neural networks as candidate algorithms. We designed several experiments, with the candidate algorithms being the same network structure with different model checkpoints within a single training process, networks with same structure but trained multiple times stochastically, and networks with different structure. In addition, we further studied the over-confidence phenomenon of the neural networks, as well as its impact on the ensemble methods. Across all of our experiments, the Super Learner achieved best performance among all the ensemble methods in this study.

Constrained binary classification using ensemble learning: an application to cost-efficient targeted PrEP strategies

Binary classification problems are ubiquitous in health and social sciences. In many cases, one wishes to balance two competing optimality considerations for a binary classifier. For instance, in resource-limited settings, an human immunodeficiency virus prevention program based on offering pre-exposure prophylaxis (PrEP) to select high-risk individuals must balance the sensitivity of the binary classifier in detecting future seroconverters (and hence offering them PrEP regimens) with the total number of PrEP regimens that is financially and logistically feasible for the program. In this article, we consider a general class of constrained binary classification problems wherein the objective function and the constraint are both monotonic with respect to a threshold. These include the minimization of the rate of positive predictions subject to a minimum sensitivity, the maximization of sensitivity subject to a maximum rate of positive predictions, and the Neyman-Pearson paradigm, which minimizes the type II error subject to an upper bound on the type I error. We propose an ensemble approach to these binary classification problems based on the Super Learner methodology. This approach linearly combines a user-supplied library of scoring algorithms, with combination weights and a discriminating threshold chosen to minimize the constrained optimality criterion. We then illustrate the application of the proposed classifier to develop an individualized PrEP targeting strategy in a resource-limited setting, with the goal of minimizing the number of PrEP offerings while achieving a minimum required sensitivity. This proof of concept data analysis uses baseline data from the ongoing Sustainable East Africa Research in Community Health study. Copyright © 2017 John Wiley & Sons, Ltd.

A Generally Efficient Targeted Minimum Loss Based Estimator based on the Highly Adaptive Lasso

Suppose we observe n $n$ independent and identically distributed observations of a finite dimensional bounded random variable. This article is concerned with the construction of an efficient targeted minimum loss-based estimator (TMLE) of a pathwise differentiable target parameter of the data distribution based on a realistic statistical model. The only smoothness condition we will enforce on the statistical model is that the nuisance parameters of the data distribution that are needed to evaluate the canonical gradient of the pathwise derivative of the target parameter are multivariate real valued cadlag functions (right-continuous and left-hand limits, (G. Neuhaus. On weak convergence of stochastic processes with multidimensional time parameter. Ann Stat 1971;42:1285-1295.) and have a finite supremum and (sectional) variation norm. Each nuisance parameter is defined as a minimizer of the expectation of a loss function over over all functions it its parameter space. For each nuisance parameter, we propose a new minimum loss based estimator that minimizes the loss-specific empirical risk over the functions in its parameter space under the additional constraint that the variation norm of the function is bounded by a set constant. The constant is selected with cross-validation. We show such an MLE can be represented as the minimizer of the empirical risk over linear combinations of indicator basis functions under the constraint that the sum of the absolute value of the coefficients is bounded by the constant: i.e., the variation norm corresponds with this L1 $L_1$-norm of the vector of coefficients. We will refer to this estimator as the highly adaptive Lasso (HAL)-estimator. We prove that for all models the HAL-estimator converges to the true nuisance parameter value at a rate that is faster than n-1/4 $n^{-1/4}$ w.r.t. square-root of the loss-based dissimilarity. We also show that if this HAL-estimator is included in the library of an ensemble super-learner, then the super-learner will at minimal achieve the rate of convergence of the HAL, but, by previous results, it will actually be asymptotically equivalent with the oracle (i.e., in some sense best) estimator in the library. Subsequently, we establish that a one-step TMLE using such a super-learner as initial estimator for each of the nuisance parameters is asymptotically efficient at any data generating distribution in the model, under weak structural conditions on the target parameter mapping and model and a strong positivity assumption (e.g., the canonical gradient is uniformly bounded). We demonstrate our general theorem by constructing such a one-step TMLE of the average causal effect in a nonparametric model, and establishing that it is asymptotically efficient.

Longitudinal Mediation Analysis with Time-varying Mediators and Exposures, with Application to Survival Outcomes

In this paper, we study the effect of a time-varying exposure mediated by a time-varying intermediate variable. We consider general longitudinal settings, including survival outcomes. At a given time point, the exposure and mediator of interest are influenced by past covariates, mediators and exposures, and affect future covariates, mediators and exposures. Right censoring, if present, occurs in response to past history. To address the challenges in mediation analysis that are unique to these settings, we propose a formulation in terms of random interventions based on conditional distributions for the mediator. This formulation, in particular, allows for well-defined natural direct and indirect effects in the survival setting, and natural decomposition of the standard total effect. Upon establishing identifiability and the corresponding statistical estimands, we derive the efficient influence curves and establish their robustness properties. Applying Targeted Maximum Likelihood Estimation, we use these efficient influence curves to construct multiply robust and efficient estimators. We also present an inverse probability weighted estimator and a nested non-targeted substitution estimator for these parameters.

Association of Implementation of a Universal Testing and Treatment Intervention With HIV Diagnosis, Receipt of Antiretroviral Therapy, and Viral Suppression in East Africa

IMPORTANCE

Antiretroviral treatment (ART) is now recommended for all HIV-positive persons. UNAIDS has set global targets to diagnose 90% of HIV-positive individuals, treat 90% of diagnosed individuals with ART, and suppress viral replication among 90% of treated individuals, for a population-level target of 73% of all HIV-positive persons with HIV viral suppression.

OBJECTIVE

To describe changes in the proportions of HIV-positive individuals with HIV viral suppression, HIV-positive individuals who had received a diagnosis, diagnosed individuals treated with ART, and treated individuals with HIV viral suppression, following implementation of a community-based testing and treatment program in rural East Africa.

DESIGN, SETTING, AND PARTICIPANTS

Observational analysis based on interim data from 16 rural Kenyan (n = 6) and Ugandan (n = 10) intervention communities in the SEARCH Study, an ongoing cluster randomized trial. Community residents who were 15 years or older (N = 77 774) were followed up for 2 years (2013-2014 to 2015-2016). HIV serostatus and plasma HIV RNA level were measured annually at multidisease health campaigns followed by home-based testing for nonattendees. All HIV-positive individuals were offered ART using a streamlined delivery model designed to reduce structural barriers, improve patient-clinician relationships, and enhance patient knowledge and attitudes about HIV.

MAIN OUTCOMES AND MEASURES

Primary outcome was viral suppression (plasma HIV RNA<500 copies/mL) among all HIV-positive individuals, assessed at baseline and after 1 and 2 years. Secondary outcomes included HIV diagnosis, ART among previously diagnosed individuals, and viral suppression among those who had initiated ART.

RESULTS

Among 77 774 residents (male, 45.3%; age 15-24 years, 35.1%), baseline HIV prevalence was 10.3% (7108 of 69 283 residents). The proportion of HIV-positive individuals with HIV viral suppression at baseline was 44.7% (95% CI, 43.5%-45.9%; 3464 of 7745 residents) and after 2 years of intervention was 80.2% (95% CI, 79.1%-81.2%; 5666 of 7068 residents), an increase of 35.5 percentage points (95% CI, 34.4-36.6). After 2 years, 95.9% of HIV-positive individuals had been previously diagnosed (95% CI, 95.3%-96.5%; 6780 of 7068 residents); 93.4% of those previously diagnosed had received ART (95% CI, 92.8%-94.0%; 6334 of 6780 residents); and 89.5% of those treated had achieved HIV viral suppression (95% CI, 88.6%-90.3%; 5666 of 6334 residents).

CONCLUSIONS AND RELEVANCE

Among individuals with HIV in rural Kenya and Uganda, implementation of community-based testing and treatment was associated with an increased proportion of HIV-positive adults who achieved viral suppression, along with increased HIV diagnosis and initiation of antiretroviral therapy. In these communities, the UNAIDS population-level viral suppression target was exceeded within 2 years after program implementation.

TRIAL REGISTRATION

clinicaltrials.gov Identifier: NCT01864683.

Online cross-validation-based ensemble learning

Online estimators update a current estimate with a new incoming batch of data without having to revisit past data thereby providing streaming estimates that are scalable to big data. We develop flexible, ensemble-based online estimators of an infinite-dimensional target parameter, such as a regression function, in the setting where data are generated sequentially by a common conditional data distribution given summary measures of the past. This setting encompasses a wide range of time-series models and, as special case, models for independent and identically distributed data. Our estimator considers a large library of candidate online estimators and uses online cross-validation to identify the algorithm with the best performance. We show that by basing estimates on the cross-validation-selected algorithm, we are asymptotically guaranteed to perform as well as the true, unknown best-performing algorithm. We provide extensions of this approach including online estimation of the optimal ensemble of candidate online estimators. We illustrate excellent performance of our methods using simulations and a real data example where we make streaming predictions of infectious disease incidence using data from a large database. Copyright © 2017 John Wiley & Sons, Ltd.

Use of a machine learning framework to predict substance use disorder treatment success

There are several methods for building prediction models. The wealth of currently available modeling techniques usually forces the researcher to judge, a priori, what will likely be the best method. Super learning (SL) is a methodology that facilitates this decision by combining all identified prediction algorithms pertinent for a particular prediction problem. SL generates a final model that is at least as good as any of the other models considered for predicting the outcome. The overarching aim of this work is to introduce SL to analysts and practitioners. This work compares the performance of logistic regression, penalized regression, random forests, deep learning neural networks, and SL to predict successful substance use disorders (SUD) treatment. A nationwide database including 99,013 SUD treatment patients was used. All algorithms were evaluated using the area under the receiver operating characteristic curve (AUC) in a test sample that was not included in the training sample used to fit the prediction models. AUC for the models ranged between 0.793 and 0.820. SL was superior to all but one of the algorithms compared. An explanation of SL steps is provided. SL is the first step in targeted learning, an analytic framework that yields double robust effect estimation and inference with fewer assumptions than the usual parametric methods. Different aspects of SL depending on the context, its function within the targeted learning framework, and the benefits of this methodology in the addiction field are discussed.

The Highly Adaptive Lasso Estimator

Estimation of a regression functions is a common goal of statistical learning. We propose a novel nonparametric regression estimator that, in contrast to many existing methods, does not rely on local smoothness assumptions nor is it constructed using local smoothing techniques. Instead, our estimator respects global smoothness constraints by virtue of falling in a class of right-hand continuous functions with left-hand limits that have variation norm bounded by a constant. Using empirical process theory, we establish a fast minimal rate of convergence of our proposed estimator and illustrate how such an estimator can be constructed using standard software. In simulations, we show that the finite-sample performance of our estimator is competitive with other popular machine learning techniques across a variety of data generating mechanisms. We also illustrate competitive performance in real data examples using several publicly available data sets.

Estimating Effects with Rare Outcomes and High Dimensional Covariates: Knowledge is Power

Many of the secondary outcomes in observational studies and randomized trials are rare. Methods for estimating causal effects and associations with rare outcomes, however, are limited, and this represents a missed opportunity for investigation. In this article, we construct a new targeted minimum loss-based estimator (TMLE) for the effect or association of an exposure on a rare outcome. We focus on the causal risk difference and statistical models incorporating bounds on the conditional mean of the outcome, given the exposure and measured confounders. By construction, the proposed estimator constrains the predicted outcomes to respect this model knowledge. Theoretically, this bounding provides stability and power to estimate the exposure effect. In finite sample simulations, the proposed estimator performed as well, if not better, than alternative estimators, including a propensity score matching estimator, inverse probability of treatment weighted (IPTW) estimator, augmented-IPTW and the standard TMLE algorithm. The new estimator yielded consistent estimates if either the conditional mean outcome or the propensity score was consistently estimated. As a substitution estimator, TMLE guaranteed the point estimates were within the parameter range. We applied the estimator to investigate the association between permissive neighborhood drunkenness norms and alcohol use disorder. Our results highlight the potential for double robust, semiparametric efficient estimation with rare events and high dimensional covariates.

Doubly Robust and Efficient Estimation of Marginal Structural Models for the Hazard Function

In social and health sciences, many research questions involve understanding the causal effect of a longitudinal treatment on mortality (or time-to-event outcomes in general). Often, treatment status may change in response to past covariates that are risk factors for mortality, and in turn, treatment status may also affect such subsequent covariates. In these situations, Marginal Structural Models (MSMs), introduced by Robins (1997. Marginal structural models Proceedings of the American Statistical Association. Section on Bayesian Statistical Science, 1-10), are well-established and widely used tools to account for time-varying confounding. In particular, a MSM can be used to specify the intervention-specific counterfactual hazard function, i. e. the hazard for the outcome of a subject in an ideal experiment where he/she was assigned to follow a given intervention on their treatment variables. The parameters of this hazard MSM are traditionally estimated using the Inverse Probability Weighted estimation Robins (1999. Marginal structural models versus structural nested models as tools for causal inference. In: Statistical models in epidemiology: the environment and clinical trials. Springer-Verlag, 1999:95-134), Robins et al. (2000), (IPTW, van der Laan and Petersen (2007. Causal effect models for realistic individualized treatment and intention to treat rules. Int J Biostat 2007;3:Article 3), Robins et al. (2008. Estimaton and extrapolation of optimal treatment and testing strategies. Statistics in Medicine 2008;27(23):4678-721)). This estimator is easy to implement and admits Wald-type confidence intervals. However, its consistency hinges on the correct specification of the treatment allocation probabilities, and the estimates are generally sensitive to large treatment weights (especially in the presence of strong confounding), which are difficult to stabilize for dynamic treatment regimes. In this paper, we present a pooled targeted maximum likelihood estimator (TMLE, van der Laan and Rubin (2006. Targeted maximum likelihood learning. The International Journal of Biostatistics 2006;2:1-40)) for MSM for the hazard function under longitudinal dynamic treatment regimes. The proposed estimator is semiparametric efficient and doubly robust, offering bias reduction over the incumbent IPTW estimator when treatment probabilities may be misspecified. Moreover, the substitution principle rooted in the TMLE potentially mitigates the sensitivity to large treatment weights in IPTW. We compare the performance of the proposed estimator with the IPTW and a on-targeted substitution estimator in a simulation study.

One-Step Targeted Minimum Loss-based Estimation Based on Universal Least Favorable One-Dimensional Submodels

Consider a study in which one observes n independent and identically distributed random variables whose probability distribution is known to be an element of a particular statistical model, and one is concerned with estimation of a particular real valued pathwise differentiable target parameter of this data probability distribution. The targeted maximum likelihood estimator (TMLE) is an asymptotically efficient substitution estimator obtained by constructing a so called least favorable parametric submodel through an initial estimator with score, at zero fluctuation of the initial estimator, that spans the efficient influence curve, and iteratively maximizing the corresponding parametric likelihood till no more updates occur, at which point the updated initial estimator solves the so called efficient influence curve equation. In this article we construct a one-dimensional universal least favorable submodel for which the TMLE only takes one step, and thereby requires minimal extra data fitting to achieve its goal of solving the efficient influence curve equation. We generalize these to universal least favorable submodels through the relevant part of the data distribution as required for targeted minimum loss-based estimation. Finally, remarkably, given a multidimensional target parameter, we develop a universal canonical one-dimensional submodel such that the one-step TMLE, only maximizing the log-likelihood over a univariate parameter, solves the multivariate efficient influence curve equation. This allows us to construct a one-step TMLE based on a one-dimensional parametric submodel through the initial estimator, that solves any multivariate desired set of estimating equations.

Optimizing matching and analysis combinations for estimating causal effects

Matching methods are common in studies across many disciplines. However, there is limited evidence on how to optimally combine matching with subsequent analysis approaches to minimize bias and maximize efficiency for the quantity of interest. We conducted simulations to compare the performance of a wide variety of matching methods and analysis approaches in terms of bias, variance, and mean squared error (MSE). We then compared these approaches in an applied example of an employment training program. The results indicate that combining full matching with double robust analysis performed best in both the simulations and the applied example, particularly when combined with machine learning estimation methods. To reduce bias, current guidelines advise researchers to select the technique with the best post-matching covariate balance, but this work finds that such an approach does not always minimize mean squared error (MSE). These findings have important implications for future research utilizing matching. To minimize MSE, investigators should consider additional diagnostics, and use of simulations tailored to the study of interest to identify the optimal matching and analysis combination.

Computationally efficient confidence intervals for cross-validated area under the ROC curve estimates

In binary classification problems, the area under the ROC curve (AUC) is commonly used to evaluate the performance of a prediction model. Often, it is combined with cross-validation in order to assess how the results will generalize to an independent data set. In order to evaluate the quality of an estimate for cross-validated AUC, we obtain an estimate of its variance. For massive data sets, the process of generating a single performance estimate can be computationally expensive. Additionally, when using a complex prediction method, the process of cross-validating a predictive model on even a relatively small data set can still require a large amount of computation time. Thus, in many practical settings, the bootstrap is a computationally intractable approach to variance estimation. As an alternative to the bootstrap, we demonstrate a computationally efficient influence curve based approach to obtaining a variance estimate for cross-validated AUC.

Improving propensity score estimators' robustness to model misspecification using super learner

The consistency of propensity score (PS) estimators relies on correct specification of the PS model. The PS is frequently estimated using main-effects logistic regression. However, the underlying model assumptions may not hold. Machine learning methods provide an alternative nonparametric approach to PS estimation. In this simulation study, we evaluated the benefit of using Super Learner (SL) for PS estimation. We created 1,000 simulated data sets (n = 500) under 4 different scenarios characterized by various degrees of deviance from the usual main-term logistic regression model for the true PS. We estimated the average treatment effect using PS matching and inverse probability of treatment weighting. The estimators' performance was evaluated in terms of PS prediction accuracy, covariate balance achieved, bias, standard error, coverage, and mean squared error. All methods exhibited adequate overall balancing properties, but in the case of model misspecification, SL performed better for highly unbalanced variables. The SL-based estimators were associated with the smallest bias in cases of severe model misspecification. Our results suggest that use of SL to estimate the PS can improve covariate balance and reduce bias in a meaningful manner in cases of serious model misspecification for treatment assignment.

A novel targeted learning method for quantitative trait loci mapping

We present a novel semiparametric method for quantitative trait loci (QTL) mapping in experimental crosses. Conventional genetic mapping methods typically assume parametric models with Gaussian errors and obtain parameter estimates through maximum-likelihood estimation. In contrast with univariate regression and interval-mapping methods, our model requires fewer assumptions and also accommodates various machine-learning algorithms. Estimation is performed with targeted maximum-likelihood learning methods. We demonstrate our semiparametric targeted learning approach in a simulation study and a well-studied barley data set.

Targeted Maximum Likelihood Estimation for Dynamic and Static Longitudinal Marginal Structural Working Models

This paper describes a targeted maximum likelihood estimator (TMLE) for the parameters of longitudinal static and dynamic marginal structural models. We consider a longitudinal data structure consisting of baseline covariates, time-dependent intervention nodes, intermediate time-dependent covariates, and a possibly time-dependent outcome. The intervention nodes at each time point can include a binary treatment as well as a right-censoring indicator. Given a class of dynamic or static interventions, a marginal structural model is used to model the mean of the intervention-specific counterfactual outcome as a function of the intervention, time point, and possibly a subset of baseline covariates. Because the true shape of this function is rarely known, the marginal structural model is used as a working model. The causal quantity of interest is defined as the projection of the true function onto this working model. Iterated conditional expectation double robust estimators for marginal structural model parameters were previously proposed by Robins (2000, 2002) and Bang and Robins (2005). Here we build on this work and present a pooled TMLE for the parameters of marginal structural working models. We compare this pooled estimator to a stratified TMLE (Schnitzer et al. 2014) that is based on estimating the intervention-specific mean separately for each intervention of interest. The performance of the pooled TMLE is compared to the performance of the stratified TMLE and the performance of inverse probability weighted (IPW) estimators using simulations. Concepts are illustrated using an example in which the aim is to estimate the causal effect of delayed switch following immunological failure of first line antiretroviral therapy among HIV-infected patients. Data from the International Epidemiological Databases to Evaluate AIDS, Southern Africa are analyzed to investigate this question using both TML and IPW estimators. Our results demonstrate practical advantages of the pooled TMLE over an IPW estimator for working marginal structural models for survival, as well as cases in which the pooled TMLE is superior to its stratified counterpart.

A double robust approach to causal effects in case-control studies

In a recent issue of the Journal, VanderWeele and Vansteelandt (Am J Epidemiol. 2011;174(10):1197-1203) discussed an inverse probability weighting method for case-control studies that could be used to estimate an additive interaction effect, referred to as the "relative excess risk due to interaction." In this article, we reinforce the well-known disadvantages of inverse probability weighting and comment on the desirability of the described parameter. Further, we review an existing double robust estimator not considered by VanderWeele and Vansteelandt, the case-control-weighted targeted maximum likelihood estimator, which has improved properties in comparison with a previously described inverse-probability-weighted estimator. This targeted maximum likelihood estimator can be used to target various parameters of interest, and its implementation has been described previously for the risk difference, relative risk, and odds ratio.

Cravings as a mediator and moderator of drinking outcomes in the COMBINE study

AIMS

Investigators of the COMBINE (Combining Medications and Behavioral Interventions for Alcoholism) study examined whether combining medications with a behavioral intervention would improve outcomes over monotherapies. Unexpectedly, the combination did not offer any advantage over either treatment alone. This study aimed to explain the lack of incremental benefit offered by the combination over either monotherapy by assessing the role of cravings as a treatment mediator and moderator.

DESIGN

Secondary mediation and moderation analyses of COMBINE study data.

SETTING

Eleven United States academic sites.

PARTICIPANTS

A total of 863 patients randomized to one of four treatment groups: naltrexone (100 mg/day; n = 209), the combined behavioral intervention (CBI, n = 236), naltrexone and CBI combined (n = 213) and placebo naltrexone (n = 205).

MEASUREMENTS

Percentage of days abstinent (PDA) measured between 13 and 16 weeks post-baseline. Cravings, the potential mediator/moderator, were measured at baseline, weeks 4 and 12 using the Obsessive-Compulsive Drinking Scale.

FINDINGS

Compared with placebo, naltrexone, CBI and the combination all increased PDA by an additional 6-10 percentage points for those with high cravings (P < 0.05 for all three treatment groups). None had significant effects on PDA for those with low cravings. The effects of all three treatments were mediated at least partially by cravings; craving reduction explained 48-53% of treatment effects (P < 0.05 for all three treatment groups). Furthermore, naltrexone appeared to reduce cravings at 4 weeks, while CBI did not reduce cravings until 12 weeks.

CONCLUSIONS

The Combining Medications and Behavioral Interventions for Alcoholism (COMBINE) naltrexone + CBI combination may not be more beneficial than either monotherapy because craving reduction is a common mechanism of both.

Population intervention causal effects based on stochastic interventions

Estimating the causal effect of an intervention on a population typically involves defining parameters in a nonparametric structural equation model (Pearl, 2000, Causality: Models, Reasoning, and Inference) in which the treatment or exposure is deterministically assigned in a static or dynamic way. We define a new causal parameter that takes into account the fact that intervention policies can result in stochastically assigned exposures. The statistical parameter that identifies the causal parameter of interest is established. Inverse probability of treatment weighting (IPTW), augmented IPTW (A-IPTW), and targeted maximum likelihood estimators (TMLE) are developed. A simulation study is performed to demonstrate the properties of these estimators, which include the double robustness of the A-IPTW and the TMLE. An application example using physical activity data is presented.

Accounting for heterogeneous treatment effects in the FDA approval process

The FDA employs an average-patient standard when reviewing drugs: it approves a drug only if is safe and effective for the average patient in a clinical trial. It is common, however, for patients to respond differently to a drug. Therefore, the average-patient standard can reject a drug that benefits certain patient subgroups (false negatives) and even approve a drug that harms other patient subgroups (false positives). These errors increase the cost of drug development - and thus health care - by wasting research on unproductive or unapproved drugs. The reason why the FDA sticks with an average patient standard is concern about opportunism by drug companies. With enough data dredging, a drug company can always find some subgroup of patients that appears to benefit from its drug, even if the subgroup truly does not. In this paper we offer alternatives to the average patient standard that reduce the risk of false negatives without increasing false positives from drug company opportunism. These proposals combine changes to institutional design - evaluation of trial data by an independent auditor - with statistical tools to reinforce the new institutional design - specifically, to ensure the auditor is truly independent of drug companies. We illustrate our proposals by applying them to the results of a recent clinical trial of a cancer drug (motexafin gadolinium). Our analysis suggests that the FDA may have made a mistake in rejecting that drug.

Colon cancer survival with herbal medicine and vitamins combined with standard therapy in a whole-systems approach: ten-year follow-up data analyzed with marginal structural models and propensity score methods

Although localized colon cancer is often successfully treated with surgery, advanced disease requires aggressive systemic therapy that has lower effectiveness. Approximately 30% to 75% of patients with colon cancer use complementary and alternative medicine (CAM), but there is limited formal evidence of survival efficacy. In a consecutive case series with 10-year follow-up of all colon cancer patients (n = 193) presenting at a San Francisco Bay-Area center for Chinese medicine (Pine Street Clinic, San Anselmo, CA), the authors compared survival in patients choosing short-term treatment lasting the duration of chemotherapy/radiotherapy with those continuing long-term. To put these data into the context of treatment responses seen in conventional medical practice, they also compared survival with Pan-Asian medicine + vitamins (PAM+V) with that of concurrent external controls from Kaiser Permanente Northern California and California Cancer Registries. Kaplan-Meier, traditional Cox regression, and more modern methods were used for causal inference-namely, propensity score and marginal structural models (MSMs), which have not been used before in studies of cancer survival and Chinese herbal medicine. PAM+V combined with conventional therapy, compared with conventional therapy alone, reduced the risk of death in stage I by 95%, stage II by 64%, stage III by 29%, and stage IV by 75%. There was no significant difference between short-term and long-term PAM+V. Combining PAM+V with conventional therapy improved survival, compared with conventional therapy alone, suggesting that prospective trials combining PAM+V with conventional therapy are justified.

Lung Cancer Survival With Herbal Medicine and Vitamins in a Whole-Systems Approach

Complementary and alternative medicines are used by up to 48% of lung cancer patients but have seen little formal assessment of survival efficacy. In this 10-year retrospective survival study, the authors investigated Pan-Asian medicine + vitamins (PAM+V) therapy in a consecutive case series of all non-small-cell lung cancer patients (n = 239) presenting at a San Francisco Bay Area Chinese medicine center (Pine Street Clinic). They compared short-term treatment lasting the duration of chemotherapy/radiotherapy with long-term therapy continuing beyond conventional therapy. They also compared PAM+V plus conventional therapy with conventional therapy alone, using concurrent controls from the Kaiser Permanente Northern California and California Cancer Registries. They adjusted for confounding with Kaplan-Meier, Cox regression, and newer methods – propensity score and marginal structural models (MSMs), which when analyzing data from observational studies or clinical practice records can provide results comparable with randomized trials. Long-term use of PAM+V beyond completion of chemotherapy reduced stage IIIB deaths by 83% and stage IV by 72% compared with short-term use only for the duration of chemotherapy. Long-term PAM+V combined with conventional therapy reduced stage IIIA deaths by 46%, stage IIIB by 62%, and stage IV by 69% compared with conventional therapy alone. Survival rates for stage IV patients treated with PAM+V were 82% at 1 year, 68% at 2 years, and 14% at 5 years. PAM+V combined with conventional therapy improved survival in stages IIIA, IIIB, and IV, compared with conventional therapy alone. Prospective trials using PAM+V with conventional therapy for lung cancer patients are justified.

partDSA: deletion/substitution/addition algorithm for partitioning the covariate space in prediction

MOTIVATION

Until now, much of the focus in cancer has been on biomarker discovery and generating lists of univariately significant genes, as well as epidemiological and clinical measures. These approaches, although significant on their own, are not effective for elucidating the synergistic qualities of the numerous components in complex diseases. These components do not act one at a time, but rather in concert with numerous others. A compelling need exists to develop analytically sound and computationally advanced methods that elucidate a more biologically meaningful understanding of the mechanisms of cancer initiation and progression by taking these interactions into account.

RESULTS

We propose a novel algorithm, partDSA, for prediction when several variables jointly affect the outcome. In such settings, piecewise constant estimation provides an intuitive approach by elucidating interactions and correlation patterns in addition to main effects. As well as generating 'and' statements similar to previously described methods, partDSA explores and chooses the best among all possible 'or' statements. The immediate benefit of partDSA is the ability to build a parsimonious model with 'and' and 'or' conjunctions that account for the observed biological phenomena. Importantly, partDSA is capable of handling categorical and continuous explanatory variables and outcomes. We evaluate the effectiveness of partDSA in comparison to several adaptive algorithms in simulations; additionally, we perform several data analyses with publicly available data and introduce the implementation of partDSA as an R package.

AVAILABILITY

http://cran.r-project.org/web/packages/partDSA/index.html

CONTACT

annette.molinaro@yale.edu

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

The risk of virologic failure decreases with duration of HIV suppression, at greater than 50% adherence to antiretroviral therapy

Background

We hypothesized that the percent adherence to antiretroviral therapy necessary to maintain HIV suppression would decrease with longer duration of viral suppression.

Methodology

Eligible participants were identified from the REACH cohort of marginally housed HIV infected adults in San Francisco. Adherence to antiretroviral therapy was measured through pill counts obtained at unannounced visits by research staff to each participant's usual place of residence. Marginal structural models and targeted maximum likelihood estimation methodologies were used to determine the effect of adherence to antiretroviral therapy on the probability of virologic failure during early and late viral suppression.

Principal Findings

A total of 221 subjects were studied (median age 44.1 years; median CD4+ T cell nadir 206 cells/mm³). Most subjects were taking the following types of antiretroviral regimens: non-nucleoside reverse transcriptase inhibitor based (37%), ritonavir boosted protease inhibitor based (28%), or unboosted protease inhibitor based (25%). Comparing the probability of failure just after achieving suppression vs. after 12 consecutive months of suppression, there was a statistically significant decrease in the probability of virologic failure for each range of adherence proportions we considered, as long as adherence was greater than 50%. The estimated risk difference, comparing the probability of virologic failure after 1 month vs. after 12 months of continuous viral suppression was 0.47 (95% CI 0.23–0.63) at 50–74% adherence, 0.29 (CI 0.03–0.50) at 75–89% adherence, and 0.36 (CI 0.23–0.48) at 90–100% adherence.

Conclusions

The risk of virologic failure for adherence greater than 50% declines with longer duration of continuous suppression. While high adherence is required to maximize the probability of durable viral suppression, the range of adherence capable of sustaining viral suppression is wider after prolonged periods of viral suppression.

Influenza vaccination and mortality: differentiating vaccine effects from bias

It is widely believed that influenza (flu) vaccination of the elderly reduces all-cause mortality, yet randomized trials for assessing vaccine effectiveness are not feasible and the observational research has been controversial. Efforts to differentiate vaccine effectiveness from selection bias have been problematic. The authors examined mortality before, during, and after 9 flu seasons in relation to time-varying vaccination status in an elderly California population in which 115,823 deaths occurred from 1996 to 2005, including 20,484 deaths during laboratory-defined flu seasons. Vaccine coverage averaged 63%; excess mortality when the flu virus was circulating averaged 7.8%. In analyses that omitted weeks when flu circulated, the odds ratio measuring the vaccination-mortality association increased monotonically from 0.34 early in November to 0.56 in January, 0.67 in April, and 0.76 in August. This reflects the trajectory of selection effects in the absence of flu. In analyses that included weeks with flu and adjustment for selection effects, flu season multiplied the odds ratio by 0.954. The corresponding vaccine effectiveness estimate was 4.6% (95% confidence interval: 0.7, 8.3). To differentiate vaccine effects from selection bias, the authors used logistic regression with a novel case-centered specification that may be useful in other population-based studies when the exposure-outcome association varies markedly over time.

Analyzing Direct Effects in Randomized Trials with Secondary Interventions: An Application to HIV Prevention Trials

The Methods for Improving Reproductive Health in Africa (MIRA) trial is a recently completed randomized trial that investigated the effect of diaphragm and lubricant gel use in reducing HIV infection among susceptible women. 5,045 women were randomly assigned to either the active treatment arm or not. Additionally, all subjects in both arms received intensive condom counselling and provision, the "gold standard" HIV prevention barrier method. There was much lower reported condom use in the intervention arm than in the control arm, making it difficult to answer important public health questions based solely on the intention-to-treat analysis. We adapt an analysis technique from causal inference to estimate the "direct effects" of assignment to the diaphragm arm, adjusting for condom use in an appropriate sense. Issues raised in the MIRA trial apply to other trials of HIV prevention methods, some of which are currently being conducted or designed.