Targeted learning ensembles for optimal individualized treatment rules with time-to-event outcomes

AUGMENTED DOUBLY ROBUST POST-IMPUTATION INFERENCE FOR PROTEOMIC DATA

Quantitative measurements produced by mass spectrometry proteomics experiments offer a direct way to explore the role of proteins in molecular mechanisms. However, analysis of such data is challenging due to the large proportion of missing values. A common strategy to address this issue is to utilize an imputed dataset, which often introduces systematic bias into downstream analyses if the imputation errors are ignored. In this paper, we propose a statistical framework inspired by doubly robust estimators that offers valid and efficient inference for proteomic data. Our framework combines powerful machine learning tools, such as variational autoencoders, to augment the imputation quality with high-dimensional peptide data, and a parametric model to estimate the propensity score for debiasing imputed outcomes. Our estimator is compatible with the double machine learning framework and has provable properties. Simulation studies verify its empirical superiority over other existing procedures. In application to both single-cell proteomic data and bulk-cell Alzheimer's Disease data our method utilizes the imputed data to gain additional, meaningful discoveries and yet maintains good control of false positives.

Learning optimal dynamic treatment regimes from longitudinal data

Investigators often report estimates of the average treatment effect (ATE). While the ATE summarizes the effect of a treatment on average, it does not provide any information about the effect of treatment within any individual. A treatment strategy that uses an individual's information to tailor treatment to maximize benefit is known as an optimal dynamic treatment rule (ODTR). Treatment, however, is typically not limited to a single point in time; consequently, learning an optimal rule for a time-varying treatment may involve not just learning the extent to which the comparative treatments' benefits vary across the characteristics of individuals, but also learning the extent to which the comparative treatments' benefits vary as relevant circumstances evolve within an individual. The goal of this paper is to provide a tutorial for estimating ODTR from longitudinal observational and clinical trial data for applied researchers. We describe an approach that uses a doubly robust unbiased transformation of the conditional ATE. We then learn a time-varying ODTR for when to increase buprenorphine-naloxone dose to minimize a return to regular opioid use among patients with opioid use disorder. Our analysis highlights the utility of ODTRs in the context of sequential decision-making: The learned ODTR outperforms a clinically defined strategy. This article is part of a Special Collection on Pharmacoepidemiology.

Smoothed Estimation on Optimal Treatment Regime Under Semisupervised Setting in Randomized Trials

A treatment regime refers to the process of assigning the most suitable treatment to a patient based on their observed information. However, prevailing research on treatment regimes predominantly relies on labeled data, which may lead to the omission of valuable information contained within unlabeled data, such as historical records and healthcare databases. Current semisupervised works for deriving optimal treatment regimes either rely on model assumptions or struggle with high computational burdens for even moderate-dimensional covariates. To address this concern, we propose a semisupervised framework that operates within a model-free context to estimate the optimal treatment regime by leveraging the abundant unlabeled data. Our proposed approach encompasses three key steps. First, we employ a single-index model to achieve dimension reduction, followed by kernel regression to impute the missing outcomes in the unlabeled data. Second, we propose various forms of semisupervised value functions based on the imputed values, incorporating both labeled and unlabeled data components. Lastly, the optimal treatment regimes are derived by maximizing the semisupervised value functions. We establish the consistency and asymptotic normality of the estimators proposed in our framework. Furthermore, we introduce a perturbation resampling procedure to estimate the asymptotic variance. Simulations confirm the advantageous properties of incorporating unlabeled data in the estimation for optimal treatment regimes. A practical data example is also provided to illustrate the application of our methodology. This work is rooted in the framework of randomized trials, with additional discussions extending to observational studies.

An Approach for Combining Clinical Judgment with Machine Learning to Inform Medical Decision Making: Analysis of Nonemergency Surgery Strategies for Acute Appendicitis in Patients with Multiple Long-Term Conditions

BACKGROUND

Machine learning (ML) methods can identify complex patterns of treatment effect heterogeneity. However, before ML can help to personalize decision making, transparent approaches must be developed that draw on clinical judgment. We develop an approach that combines clinical judgment with ML to generate appropriate comparative effectiveness evidence for informing decision making.

METHODS

We motivate this approach in evaluating the effectiveness of nonemergency surgery (NES) strategies, such as antibiotic therapy, for people with acute appendicitis who have multiple long-term conditions (MLTCs) compared with emergency surgery (ES). Our 4-stage approach 1) draws on clinical judgment about which patient characteristics and morbidities modify the relative effectiveness of NES; 2) selects additional covariates from a high-dimensional covariate space (P > 500) by applying an ML approach, least absolute shrinkage and selection operator (LASSO), to large-scale administrative data (N = 24,312); 3) generates estimates of comparative effectiveness for relevant subgroups; and 4) presents evidence in a suitable form for decision making.

RESULTS

This approach provides useful evidence for clinically relevant subgroups. We found that overall NES strategies led to increases in the mean number of days alive and out-of-hospital compared with ES, but estimates differed across subgroups, ranging from 21.2 (95% confidence interval: 1.8 to 40.5) for patients with chronic heart failure and chronic kidney disease to -10.4 (-29.8 to 9.1) for patients with cancer and hypertension. Our interactive tool for visualizing ML output allows for findings to be customized according to the specific needs of the clinical decision maker.

CONCLUSIONS

This principled approach of combining clinical judgment with an ML approach can improve trust, relevance, and usefulness of the evidence generated for clinical decision making.

HIGHLIGHTS

Machine learning (ML) methods have many potential applications in medical decision making, but the lack of model interpretability and usability constitutes an important barrier for the wider adoption of ML evidence in practice.We develop a 4-stage approach for integrating clinical judgment into the way an ML approach is used to estimate and report comparative effectiveness.We illustrate the approach in undertaking an evaluation of nonemergency surgery (NES) strategies for acute appendicitis in patients with multiple long-term conditions and find that NES strategies lead to better outcomes compared with emergency surgery and that the effects differ across subgroups.We develop an interactive tool for visualizing the results of this study that allows findings to be customized according to the user's preferences.

Modern approaches for evaluating treatment effect heterogeneity from clinical trials and observational data

In this paper, we review recent advances in statistical methods for the evaluation of the heterogeneity of treatment effects (HTE), including subgroup identification and estimation of individualized treatment regimens, from randomized clinical trials and observational studies. We identify several types of approaches using the features introduced in Lipkovich et al (Stat Med 2017;36: 136-196) that distinguish the recommended principled methods from basic methods for HTE evaluation that typically rely on rules of thumb and general guidelines (the methods are often referred to as common practices). We discuss the advantages and disadvantages of various principled methods as well as common measures for evaluating their performance. We use simulated data and a case study based on a historical clinical trial to illustrate several new approaches to HTE evaluation.

Optimally Choosing Medication Type for Patients With Opioid Use Disorder

Patients with opioid use disorder (OUD) tend to get assigned to one of 3 medications based on the treatment program to which the patient presents (e.g., opioid treatment programs tend to treat patients with methadone, while office-based practices tend to prescribe buprenorphine). It is possible that optimally matching patients with treatment type would reduce the risk of return to regular opioid use (RROU). We analyzed data from 3 comparative effectiveness trials from the US National Institute on Drug Abuse Clinical Trials Network (CTN0027, 2006-2010; CTN0030, 2006-2009; and CTN0051 2014-2017), in which patients with OUD (n = 1,459) were assigned to treatment with either injection extended-release naltrexone (XR-NTX), sublingual buprenorphine-naloxone (BUP-NX), or oral methadone. We learned an individualized rule by which to assign medication type such that risk of RROU during 12 weeks of treatment would be minimized, and then estimated the amount by which RROU risk could be reduced if the rule were applied. Applying our estimated treatment rule would reduce risk of RROU compared with treating everyone with methadone (relative risk (RR) = 0.79, 95% confidence interval (CI): 0.60, 0.97) or treating everyone with XR-NTX (RR = 0.71, 95% CI: 0.47, 0.96). Applying the estimated treatment rule would have resulted in a similar risk of RROU to that of with treating everyone with BUP-NX (RR = 0.92, 95% CI: 0.73, 1.11).

When the Ends do not Justify the Means: Learning Who is Predicted to Have Harmful Indirect Effects

There is a growing literature on finding rules by which to assign treatment based on an individual's characteristics such that a desired outcome under the intervention is maximized. A related goal entails identifying a subpopulation of individuals predicted to have a harmful indirect effect (the effect of treatment on an outcome through mediators), perhaps even in the presence of a predicted beneficial total treatment effect. In some cases, the implications of a likely harmful indirect effect may outweigh an anticipated beneficial total treatment effect, and would motivate further discussion of whether to treat identified individuals. We build on the mediation and optimal treatment rule literatures to propose a method of identifying a subgroup for which the treatment effect through the mediator is expected to be harmful. Our approach is nonparametric, incorporates post-treatment confounders of the mediator-outcome relationship, and does not make restrictions on the distribution of baseline covariates, mediating variables, or outcomes. We apply the proposed approach to identify a subgroup of boys in the MTO housing voucher experiment who are predicted to have a harmful indirect effect of housing voucher receipt on subsequent psychiatric disorder incidence through aspects of their school and neighborhood environments.

Semiparametric single-index models for optimal treatment regimens with censored outcomes

There is a growing interest in precision medicine, where a potentially censored survival time is often the most important outcome of interest. To discover optimal treatment regimens for such an outcome, we propose a semiparametric proportional hazards model by incorporating the interaction between treatment and a single index of covariates through an unknown monotone link function. This model is flexible enough to allow non-linear treatment-covariate interactions and yet provides a clinically interpretable linear rule for treatment decision. We propose a sieve maximum likelihood estimation approach, under which the baseline hazard function is estimated nonparametrically and the unknown link function is estimated via monotone quadratic B-splines. We show that the resulting estimators are consistent and asymptotically normal with a covariance matrix that attains the semiparametric efficiency bound. The optimal treatment rule follows naturally as a linear combination of the maximum likelihood estimators of the model parameters. Through extensive simulation studies and an application to an AIDS clinical trial, we demonstrate that the treatment rule derived from the single-index model outperforms the treatment rule under the standard Cox proportional hazards model.

Doubly-robust methods for differences in restricted mean lifetimes using pseudo-observations

In clinical studies or trials comparing survival times between two treatment groups, the restricted mean lifetime (RML), defined as the expectation of the survival from time 0 to a prespecified time-point, is often the quantity of interest that is readily interpretable to clinicians without any modeling restrictions. It is well known that if the treatments are not randomized (as in observational studies), covariate adjustment is necessary to account for treatment imbalances due to confounding factors. In this article, we propose a simple doubly-robust pseudo-value approach to effectively estimate the difference in the RML between two groups (akin to a metric for estimating average causal effects), while accounting for confounders. The proposed method combines two general approaches: (a) group-specific regression models for the time-to-event and covariate information, and (b) inverse probability of treatment assignment weights, where the RMLs are replaced by the corresponding pseudo-observations for survival outcomes, thereby mitigating the estimation complexities in presence of censoring. The proposed estimator is double-robust, in the sense that it is consistent if at least one of the two working models remains correct. In addition, we explore the potential of available machine learning algorithms in causal inference to reduce possible bias of the causal estimates in presence of a complex association between the survival outcome and covariates. We conduct extensive simulation studies to assess the finite-sample performance of the pseudo-value causal effect estimators. Furthermore, we illustrate our methodology via application to a dataset from a breast cancer cohort study. The proposed method is implementable using the R package drRML, available in GitHub.

Machine learning in the estimation of causal effects: targeted minimum loss-based estimation and double/debiased machine learning

In recent decades, the fields of statistical and machine learning have seen a revolution in the development of data-adaptive regression methods that have optimal performance under flexible, sometimes minimal, assumptions on the true regression functions. These developments have impacted all areas of applied and theoretical statistics and have allowed data analysts to avoid the biases incurred under the pervasive practice of parametric model misspecification. In this commentary, I discuss issues around the use of data-adaptive regression in estimation of causal inference parameters. To ground ideas, I focus on two estimation approaches with roots in semi-parametric estimation theory: targeted minimum loss-based estimation (TMLE; van der Laan and Rubin, 2006) and double/debiased machine learning (DML; Chernozhukov and others, 2018). This commentary is not comprehensive, the literature on these topics is rich, and there are many subtleties and developments which I do not address. These two frameworks represent only a small fraction of an increasingly large number of methods for causal inference using machine learning. To my knowledge, they are the only methods grounded in statistical semi-parametric theory that also allow unrestricted use of data-adaptive regression techniques.

Resampling-based confidence intervals for model-free robust inference on optimal treatment regimes

We propose a new procedure for inference on optimal treatment regimes in the model-free setting, which does not require to specify an outcome regression model. Existing model-free estimators for optimal treatment regimes are usually not suitable for the purpose of inference, because they either have nonstandard asymptotic distributions or do not necessarily guarantee consistent estimation of the parameter indexing the Bayes rule due to the use of surrogate loss. We first study a smoothed robust estimator that directly targets the parameter corresponding to the Bayes decision rule for optimal treatment regimes estimation. This estimator is shown to have an asymptotic normal distribution. Furthermore, we verify that a resampling procedure provides asymptotically accurate inference for both the parameter indexing the optimal treatment regime and the optimal value function. A new algorithm is developed to calculate the proposed estimator with substantially improved speed and stability. Numerical results demonstrate the satisfactory performance of the new methods.

Joint modeling and multiple comparisons with the best of data from a SMART with survival outcomes

A dynamic treatment regimen (DTR) is a sequence of decision rules that can alter treatments or doses based on outcomes from prior treatment. In the case of two lines of treatment, a DTR specifies first-line treatment, and second-line treatment for responders and treatment for non-responders to the first-line treatment. A sequential, multiple assignment, randomized trial (SMART) is one such type of trial that has been designed to assess DTRs. The primary goal of our project is to identify the treatments, covariates, and their interactions result in the best overall survival rate. Many previously proposed methods to analyze data with survival outcomes from a SMART use inverse probability weighting and provide non-parametric estimation of survival rates, but no other information. Other methods have been proposed to identify and estimate the optimal DTR, but inference issues were seldom addressed. We apply a joint modeling approach to provide unbiased survival estimates as a mechanism to quantify baseline and time-varying covariate effects, treatment effects, and their interactions within regimens. The issue of multiple comparisons at specific time points is addressed using multiple comparisons with the best method.