Doubly Robust Learning for Estimating Individualized Treatment with Censored Data

A matching-based machine learning approach to estimating optimal dynamic treatment regimes with time-to-event outcomes

Observational data (e.g. electronic health records) has become increasingly important in evidence-based research on dynamic treatment regimes, which tailor treatments over time to patients based on their characteristics and evolving clinical history. It is of great interest for clinicians and statisticians to identify an optimal dynamic treatment regime that can produce the best expected clinical outcome for each individual and thus maximize the treatment benefit over the population. Observational data impose various challenges for using statistical tools to estimate optimal dynamic treatment regimes. Notably, the task becomes more sophisticated when the clinical outcome of primary interest is time-to-event. Here, we propose a matching-based machine learning method to identify the optimal dynamic treatment regime with time-to-event outcomes subject to right-censoring using electronic health record data. In contrast to the established inverse probability weighting-based dynamic treatment regime methods, our proposed approach provides better protection against model misspecification and extreme weights in the context of treatment sequences, effectively addressing a prevalent challenge in the longitudinal analysis of electronic health record data. In simulations, the proposed method demonstrates robust performance across a range of scenarios. In addition, we illustrate the method with an application to estimate optimal dynamic treatment regimes for patients with advanced non-small cell lung cancer using a real-world, nationwide electronic health record database from Flatiron Health.

Estimating individualized treatment effects using an individual participant data meta-analysis

BACKGROUND

One key aspect of personalized medicine is to identify individuals who benefit from an intervention. Some approaches have been developed to estimate individualized treatment effects (ITE) with a single randomized control trial (RCT) or observational data, but they are often underpowered for the ITE estimation. Using individual participant data meta-analyses (IPD-MA) might solve this problem. Few studies have investigated how to develop risk prediction models with IPD-MA, and it remains unclear how to combine those methods with approaches used for ITE estimation. In this article, we compared different approaches using both simulated and real data with binary and time-to-event outcomes to estimate the individualized treatment effects from an IPD-MA in a one-stage approach.

METHODS

We compared five one-stage models: naive model (NA), random intercept (RI), stratified intercept (SI), rank-1 (R1), and fully stratified (FS), built with two different strategies, the S-learner and the T-learner constructed with a Monte Carlo simulation study in which we explored different scenarios with a binary or a time-to-event outcome. To evaluate the performance of the models, we used the c-statistic for benefit, the calibration of predictions, and the mean squared error. The different models were also used on the INDANA IPD-MA, comparing an anti-hypertensive treatment to no treatment or placebo ( N = 40 237 , 836 events).

RESULTS

Simulation results showed that using the S-learner led to better ITE estimation performances for both binary and time-to-event outcomes. None of the risk models stand out and had significantly better results. For the INDANA dataset with a binary outcome, the naive and the random intercept models had the best performances.

CONCLUSIONS

For the choice of the strategy, using interactions with treatment (the S-learner) is preferable. For the choice of the method, no approach is better than the other.

Estimation of optimal treatment regimes with electronic medical record data using the residual life value estimator

Clinicians and patients must make treatment decisions at a series of key decision points throughout disease progression. A dynamic treatment regime is a set of sequential decision rules that return treatment decisions based on accumulating patient information, like that commonly found in electronic medical record (EMR) data. When applied to a patient population, an optimal treatment regime leads to the most favorable outcome on average. Identifying optimal treatment regimes that maximize residual life is especially desirable for patients with life-threatening diseases such as sepsis, a complex medical condition that involves severe infections with organ dysfunction. We introduce the residual life value estimator (ReLiVE), an estimator for the expected value of cumulative restricted residual life under a fixed treatment regime. Building on ReLiVE, we present a method for estimating an optimal treatment regime that maximizes expected cumulative restricted residual life. Our proposed method, ReLiVE-Q, conducts estimation via the backward induction algorithm Q-learning. We illustrate the utility of ReLiVE-Q in simulation studies, and we apply ReLiVE-Q to estimate an optimal treatment regime for septic patients in the intensive care unit using EMR data from the Multiparameter Intelligent Monitoring Intensive Care database. Ultimately, we demonstrate that ReLiVE-Q leverages accumulating patient information to estimate personalized treatment regimes that optimize a clinically meaningful function of residual life.

Imputation-based Q-learning for optimizing dynamic treatment regimes with right-censored survival outcome

Q-learning has been one of the most commonly used methods for optimizing dynamic treatment regimes (DTRs) in multistage decision-making. Right-censored survival outcome poses a significant challenge to Q-Learning due to its reliance on parametric models for counterfactual estimation which are subject to misspecification and sensitive to missing covariates. In this paper, we propose an imputation-based Q-learning (IQ-learning) where flexible nonparametric or semiparametric models are employed to estimate optimal treatment rules for each stage and then weighted hot-deck multiple imputation (MI) and direct-draw MI are used to predict optimal potential survival times. Missing data are handled using inverse probability weighting and MI, and the nonrandom treatment assignment among the observed is accounted for using a propensity-score approach. We investigate the performance of IQ-learning via extensive simulations and show that it is more robust to model misspecification than existing Q-Learning methods, imputes only plausible potential survival times contrary to parametric models and provides more flexibility in terms of baseline hazard shape. Using IQ-learning, we developed an optimal DTR for leukemia treatment based on a randomized trial with observational follow-up that motivated this study.

Estimation in optimal treatment regimes based on mean residual lifetimes with right-censored data

An optimal individualized treatment regime (ITR) is a decision rule in allocating the best treatment to each patient and, hence, maximizing overall benefits. In this paper, we propose a novel framework based on nonparametric inverse probability weighting (IPW) and augmented inverse probability weighting (AIPW) estimators of the value function when the data are subject to right censoring. In contrast to most existing approaches that are designed to maximize the expected survival time under a binary treatment framework, the proposed method targets maximizing the mean residual lifetime of patients. Specifically, the proposed IPW method searches the optimal ITR by maximizing an estimator for the overall population outcome directly, without specifying the regression model for the conditional mean residual lifetime, whereas the AIPW method integrates the model information of the mean residual lifetime to improve the robustness. Furthermore, to overcome the computational difficulty in a nonsmooth value estimator, smoothed IPW and AIPW estimators are constructed. In theory, we establish the asymptotic properties of the proposed method under suitable regularity conditions. The empirical performances of the proposed IPW and AIPW estimators are evaluated using simulation studies and are further illustrated with an application to the real-world data set from the Acquired Immunodeficiency Syndrome Clinical Trial Group Protocol 175 (ACTG175).

Estimating optimal individualized treatment rules with multistate processes

Multistate process data are common in studies of chronic diseases such as cancer. These data are ideal for precision medicine purposes as they can be leveraged to improve more refined health outcomes, compared to standard survival outcomes, as well as incorporate patient preferences regarding quantity versus quality of life. However, there are currently no methods for the estimation of optimal individualized treatment rules with such data. In this paper, we propose a nonparametric outcome weighted learning approach for this problem in randomized clinical trial settings. The theoretical properties of the proposed methods, including Fisher consistency and asymptotic normality of the estimated expected outcome under the estimated optimal individualized treatment rule, are rigorously established. A consistent closed-form variance estimator is provided and methodology for the calculation of simultaneous confidence intervals is proposed. Simulation studies show that the proposed methodology and inference procedures work well even with small-sample sizes and high rates of right censoring. The methodology is illustrated using data from a randomized clinical trial on the treatment of metastatic squamous-cell carcinoma of the head and neck.

A general framework for subgroup detection via one-step value difference estimation

Recent statistical methodology for precision medicine has focused on either identification of subgroups with enhanced treatment effects or estimating optimal treatment decision rules so that treatment is allocated in a way that maximizes, on average, predefined patient outcomes. Less attention has been given to subgroup testing, which involves evaluation of whether at least a subgroup of the population benefits from an investigative treatment, compared to some control or standard of care. In this work, we propose a general framework for testing for the existence of a subgroup with enhanced treatment effects based on the difference of the estimated value functions under an estimated optimal treatment regime and a fixed regime that assigns everyone to the same treatment. Our proposed test does not require specification of the parametric form of the subgroup and allows heterogeneous treatment effects within the subgroup. The test applies to cases when the outcome of interest is either a time-to-event or a (uncensored) scalar, and is valid at the exceptional law. To demonstrate the empirical performance of the proposed test, we study the type I error and power of the test statistics in simulations and also apply our test to data from a Phase III trial in patients with hematological malignancies.

Optimizing Treatment for Human Immunodeficiency Virus to Improve Clinical Outcomes Using Precision Medicine

In first-line antiretroviral therapy (ART) for human immunodeficiency virus (HIV) treatment, some subgroups of patients may respond better to an efavirenz-based regimen than an integrase strand transfer inhibitor (InSTI)-based regimen, or vice versa, due to patient characteristics modifying treatment effects. Using data based on nearly 16,000 patients from the North American AIDS Cohort Collaboration on Research and Design from 2009-2016, statistical methods for precision medicine were employed to estimate an optimal treatment rule that minimizes the 5-year risk of the composite outcome of acquired immune deficiency syndrome (AIDS)-defining illnesses, serious non-AIDS events, and all-cause mortality. The treatment rules considered were functions that recommend either an efavirenz- or InSTI-based regimen conditional on baseline patient characteristics such as demographic information, laboratory results, and health history. The estimated 5-year risk under the estimated optimal treatment rule was 10.0% (95% confidence interval (CI): 8.6, 11.3), corresponding to an absolute risk reduction of 2.3% (95% CI: 0.9, 3.8) when compared with recommending an efavirenz-based regimen for all patients and 2.6% (95% CI: 1.0, 4.2) when compared with recommending an InSTI-based regimen for all. Tailoring ART to individual patient characteristics may reduce 5-year risk of the composite outcome compared with assigning all patients the same drug regimen.

Continuous-Time Decision Transformer for Healthcare Applications

Offline reinforcement learning (RL) is a promising approach for training intelligent medical agents to learn treatment policies and assist decision making in many healthcare applications, such as scheduling clinical visits and assigning dosages for patients with chronic conditions. In this paper, we investigate the potential usefulness of Decision Transformer (Chen et al., 2021)-a new offline RL paradigm-in medical domains where decision making in continuous time is desired. As Decision Transformer only handles discrete-time (or turn-based) sequential decision making scenarios, we generalize it to Continuous-Time Decision Transformer that not only considers the past clinical measurements and treatments but also the timings of previous visits, and learns to suggest the timings of future visits as well as the treatment plan at each visit. Extensive experiments on synthetic datasets and simulators motivated by real-world medical applications demonstrate that Continuous-Time Decision Transformer is able to outperform competitors and has clinical utility in terms of improving patients' health and prolonging their survival by learning high-performance policies from logged data generated using policies of different levels of quality.

A novel approach to assess dynamic treatment regimes embedded in a SMART with an ordinal outcome

Sequential multiple assignment randomized trials (SMARTs) are used to construct data-driven optimal intervention strategies for subjects based on their intervention and covariate histories in different branches of health and behavioral sciences where a sequence of interventions is given to a participant. Sequential intervention strategies are often called dynamic treatment regimes (DTR). In the existing literature, the majority of the analysis methodologies for SMART data assume a continuous primary outcome. However, ordinal outcomes are also quite common in clinical practice. In this work, first, we introduce the notion of generalized odds ratio ( G O R $$ GOR $$ ) to compare two DTRs embedded in a SMART with an ordinal outcome and discuss some combinatorial properties of this measure. Next, we propose a likelihood-based approach to estimate G O R $$ GOR $$ from SMART data, and derive the asymptotic properties of its estimate. We discuss alternative ways to estimate G O R $$ GOR $$ using concordant-discordant pairs and two-sample U $$ U $$ -statistic. We derive the required sample size formula for designing SMARTs with ordinal outcomes based on G O R $$ GOR $$ . A simulation study shows the performance of the estimated G O R $$ GOR $$ in terms of the estimated power corresponding to the derived sample size. The methodology is applied to analyze data from the SMART+ study, conducted in the UK, to improve carbohydrate periodization behavior in athletes using a menu planner mobile application, Hexis Performance. A freely available Shiny web app using R is provided to make the proposed methodology accessible to other researchers and practitioners.

Accountable survival contrast-learning for optimal dynamic treatment regimes

Dynamic treatment regime (DTR) is an emerging paradigm in recent medical studies, which searches a series of decision rules to assign optimal treatments to each patient by taking into account individual features such as genetic, environmental, and social factors. Although there is a large and growing literature on statistical methods to estimate optimal treatment regimes, most methodologies focused on complete data. In this article, we propose an accountable contrast-learning algorithm for optimal dynamic treatment regime with survival endpoints. Our estimating procedure is originated from a doubly-robust weighted classification scheme, which is a model-based contrast-learning method that directly characterizes the interaction terms between predictors and treatments without main effects. To reflect the censorship, we adopt the pseudo-value approach that replaces survival quantities with pseudo-observations for the time-to-event outcome. Unlike many existing approaches, mostly based on complicated outcome regression modeling or inverse-probability weighting schemes, the pseudo-value approach greatly simplifies the estimating procedure for optimal treatment regime by allowing investigators to conveniently apply standard machine learning techniques to censored survival data without losing much efficiency. We further explore a SCAD-penalization to find informative clinical variables and modified algorithms to handle multiple treatment options by searching upper and lower bounds of the objective function. We demonstrate the utility of our proposal via extensive simulations and application to AIDS data.

Model selection for survival individualized treatment rules using the jackknife estimator

BACKGROUND

Precision medicine is an emerging field that involves the selection of treatments based on patients' individual prognostic data. It is formalized through the identification of individualized treatment rules (ITRs) that maximize a clinical outcome. When the type of outcome is time-to-event, the correct handling of censoring is crucial for estimating reliable optimal ITRs.

METHODS

We propose a jackknife estimator of the value function to allow for right-censored data for a binary treatment. The jackknife estimator or leave-one-out-cross-validation approach can be used to estimate the value function and select optimal ITRs using existing machine learning methods. We address the issue of censoring in survival data by introducing an inverse probability of censoring weighted (IPCW) adjustment in the expression of the jackknife estimator of the value function. In this paper, we estimate the optimal ITR by using random survival forest (RSF) and Cox proportional hazards model (COX). We use a Z-test to compare the optimal ITRs learned by RSF and COX with the zero-order model (or one-size-fits-all). Through simulation studies, we investigate the asymptotic properties and the performance of our proposed estimator under different censoring rates. We illustrate our proposed method on a phase III clinical trial of non-small cell lung cancer data.

RESULTS

Our simulations show that COX outperforms RSF for small sample sizes. As sample sizes increase, the performance of RSF improves, in particular when the expected log failure time is not linear in the covariates. The estimator is fairly normally distributed across different combinations of simulation scenarios and censoring rates. When applied to a non-small-cell lung cancer data set, our method determines the zero-order model (ZOM) as the best performing model. This finding highlights the possibility that tailoring may not be needed for this cancer data set.

CONCLUSION

The jackknife approach for estimating the value function in the presence of right-censored data shows satisfactory performance when there is small to moderate censoring. Winsorizing the upper and lower percentiles of the estimated survival weights for computing the IPCWs stabilizes the estimator.

Semiparametric single-index models for optimal treatment regimens with censored outcomes

There is a growing interest in precision medicine, where a potentially censored survival time is often the most important outcome of interest. To discover optimal treatment regimens for such an outcome, we propose a semiparametric proportional hazards model by incorporating the interaction between treatment and a single index of covariates through an unknown monotone link function. This model is flexible enough to allow non-linear treatment-covariate interactions and yet provides a clinically interpretable linear rule for treatment decision. We propose a sieve maximum likelihood estimation approach, under which the baseline hazard function is estimated nonparametrically and the unknown link function is estimated via monotone quadratic B-splines. We show that the resulting estimators are consistent and asymptotically normal with a covariance matrix that attains the semiparametric efficiency bound. The optimal treatment rule follows naturally as a linear combination of the maximum likelihood estimators of the model parameters. Through extensive simulation studies and an application to an AIDS clinical trial, we demonstrate that the treatment rule derived from the single-index model outperforms the treatment rule under the standard Cox proportional hazards model.

Doubly robust estimation of optimal dynamic treatment regimes with multicategory treatments and survival outcomes

Patients with chronic diseases, such as cancer or epilepsy, are often followed through multiple stages of clinical interventions. Dynamic treatment regimes (DTRs) are sequences of decision rules that assign treatments at each stage based on measured covariates for each patient. A DTR is said to be optimal if the expectation of the desirable clinical benefit reaches a maximum when applied to a population. When there are three or more options for treatments at each decision point and the clinical outcome of interest is a time-to-event variable, estimating an optimal DTR can be complicated. We propose a doubly robust method to estimate optimal DTRs with multicategory treatments and survival outcomes. A novel blip function is defined to measure the difference in expected outcomes among treatments, and a doubly robust weighted least squares algorithm is designed for parameter estimation. Simulations using various weight functions and scenarios support the advantages of the proposed method in estimating optimal DTRs over existing approaches. We further illustrate the practical value of our method by applying it to data from the Standard and New Antiepileptic Drugs study. In this analysis, the proposed method supports the use of the new drug lamotrigine over the standard option carbamazepine. When the actual treatments match the estimated optimal treatments, survival outcomes tend to be better. The newly developed method provides a practical approach for clinicians that is not limited to cases of binary treatment options.

On Robustness of Individualized Decision Rules

With the emergence of precision medicine, estimating optimal individualized decision rules (IDRs) has attracted tremendous attention in many scientific areas. Most existing literature has focused on finding optimal IDRs that can maximize the expected outcome for each individual. Motivated by complex individualized decision making procedures and the popular conditional value at risk (CVaR) measure, we propose a new robust criterion to estimate optimal IDRs in order to control the average lower tail of the individuals' outcomes. In addition to improving the individualized expected outcome, our proposed criterion takes risks into consideration, and thus the resulting IDRs can prevent adverse events. The optimal IDR under our criterion can be interpreted as the decision rule that maximizes the "worst-case" scenario of the individualized outcome when the underlying distribution is perturbed within a constrained set. An efficient non-convex optimization algorithm is proposed with convergence guarantees. We investigate theoretical properties for our estimated optimal IDRs under the proposed criterion such as consistency and finite sample error bounds. Simulation studies and a real data application are used to further demonstrate the robust performance of our methods. Several extensions of the proposed method are also discussed.

Novel Use of Patient-Specific Covariates From Oncology Studies in the Era of Biomedical Data Science: A Review of Latest Methodologies

In this article, we review different applications of how to incorporate individual patient variables into clinical research within oncology. These methodologies range from the more traditional use of baseline covariates from randomized clinical trials, as well as observational studies, to using covariates to generalize the results of randomized clinical trials to other populations. Individual patient variables also allow for the consideration of heterogeneity in treatment effects and individualized treatment rules. We primarily consider two treatment groups and mostly focus on time-to-event outcomes where such methodologies have been well established and widely applied. We also discuss more conceptually newer statistical research that has not been widely applied in clinical oncology, but is likely to make an impact in future oncology research. With the increasing amount of biomedical data available for analysis, it is inevitable that more methods are developed to make best use of information, to advance oncology research.

Individualized treatment rules under stochastic treatment cost constraints

Estimation and evaluation of individualized treatment rules have been studied extensively, but real-world treatment resource constraints have received limited attention in existing methods. We investigate a setting in which treatment is intervened upon based on covariates to optimize the mean counterfactual outcome under treatment cost constraints when the treatment cost is random. In a particularly interesting special case, an instrumental variable corresponding to encouragement to treatment is intervened upon with constraints on the proportion receiving treatment. For such settings, we first develop a method to estimate optimal individualized treatment rules. We further construct an asymptotically efficient plug-in estimator of the corresponding average treatment effect relative to a given reference rule.

Restricted survival benefit with right-censored data

The hazard ratio is widely used to quantify treatment effects. However, it may be difficult to interpret for patients and practitioners, especially when the hazard ratio is not constant over time. Alternative measures of the treatment effects have been proposed such as the difference of the restricted mean survival times, the difference in survival proportions at some fixed follow-up time, or the net chance of a longer survival. In this paper, we propose the restricted survival benefit (RSB), a quantity that can incorporate multiple useful measurements of treatment effects. Hence, it provides a framework for a comprehensive assessment of the treatment effects. We provide estimation and inference procedures for the RSB that accommodate censored survival outcomes, using methods of the inverse-probability-censoring-weighted U -statistic and the jackknife empirical likelihood. We conduct extensive simulation studies to examine the numerical performance of the proposed method, and we analyze data from a randomized Phase III clinical trial (SWOG S0777) using the proposed method.

Risk controlled decision trees and random forests for precision Medicine

Statistical methods generating individualized treatment rules (ITRs) often focus on maximizing expected benefit, but these rules may expose patients to excess risk. For instance, aggressive treatment of type 2 diabetes (T2D) with insulin therapies may result in an ITR which controls blood glucose levels but increases rates of hypoglycemia, diminishing the appeal of the ITR. This work proposes two methods to identify risk-controlled ITRs (rcITR), a class of ITR which maximizes a benefit while controlling risk at a prespecified threshold. A novel penalized recursive partitioning algorithm is developed which optimizes an unconstrained, penalized value function. The final rule is a risk-controlled decision tree (rcDT) that is easily interpretable. A natural extension of the rcDT model, risk controlled random forests (rcRF), is also proposed. Simulation studies demonstrate the robustness of rcRF modeling. Three variable importance measures are proposed to further guide clinical decision-making. Both rcDT and rcRF procedures can be applied to data from randomized controlled trials or observational studies. An extensive simulation study interrogates the performance of the proposed methods. A data analysis of the DURABLE diabetes trial in which two therapeutics were compared is additionally presented. An R package implements the proposed methods ( https://github.com/kdoub5ha/rcITR).

Survival Augmented Patient Preference Incorporated Reinforcement Learning to Evaluate Tailoring Variables for Personalized Healthcare

In this paper, we consider personalized treatment decision strategies in the management of chronic diseases, such as chronic kidney disease, which typically consists of sequential and adaptive treatment decision making. We investigate a two-stage treatment setting with a survival outcome that could be right censored. This can be formulated through a dynamic treatment regime (DTR) framework, where the goal is to tailor treatment to each individual based on their own medical history in order to maximize a desirable health outcome. We develop a new method, Survival Augmented Patient Preference incorporated reinforcement Q-Learning (SAPP-Q-Learning) to decide between quality of life and survival restricted at maximal follow-up. Our method incorporates the latent patient preference into a weighted utility function that balances between quality of life and survival time, in a Q-learning model framework. We further propose a corresponding m-out-of-n Bootstrap procedure to accurately make statistical inferences and construct confidence intervals on the effects of tailoring variables, whose values can guide personalized treatment strategies.

A parsimonious personalized dose-finding model via dimension reduction

Learning an individualized dose rule in personalized medicine is a challenging statistical problem. Existing methods often suffer from the curse of dimensionality, especially when the decision function is estimated nonparametrically. To tackle this problem, we propose a dimension reduction framework that effectively reduces the estimation to a lower-dimensional subspace of the covariates. We exploit that the individualized dose rule can be defined in a subspace spanned by a few linear combinations of the covariates, leading to a more parsimonious model. Also, our framework does not require the inverse probability of the propensity score under observational studies due to a direct maximization of the value function. This distinguishes us from the outcome weighted learning framework, which also solves decision rules directly. Under the same framework, we further propose a pseudo-direct learning approach focuses more on estimating the dimensionality-reduced subspace of the treatment outcome. Parameters in both approaches can be estimated efficiently using an orthogonality constrained optimization algorithm on the Stiefel manifold. Under mild regularity assumptions, the asymptotic normality results of the proposed estimators can are established, respectively. We also derive the consistency and convergence rate for the value function under the estimated optimal dose rule. We evaluate the performance of the proposed approaches through extensive simulation studies and a warfarin pharmacogenetic dataset.

Optimal treatment regimes for competing risk data using doubly robust outcome weighted learning with bi-level variable selection

The goal of the optimal treatment regime is maximizing treatment benefits via personalized treatment assignments based on the observed patient and treatment characteristics. Parametric regression-based outcome learning approaches require exploring complex interplay between the outcome and treatment assignments adjusting for the patient and treatment covariates, yet correctly specifying such relationships is challenging. Thus, a robust method against misspecified models is desirable in practice. Parsimonious models are also desired to pursue a concise interpretation and to avoid including spurious predictors of the outcome or treatment benefits. These issues have not been comprehensively addressed in the presence of competing risks. Recognizing that competing risks and group variables are frequently present, we propose a doubly robust estimation with adaptive L 1 penalties to select important variables at both group and within-group levels for competing risks data. The proposed method is applied to hematopoietic cell transplantation data to personalize the graft source choice for treatment-related mortality (TRM). While the existing medical literature attempts to find a uniform solution ignoring the heterogeneity of the graft source effects on TRM, the analysis results show the effect of the graft source on TRM could be different depending on the patient-specific characteristics.

A nonparametric method for value function guided subgroup identification via gradient tree boosting for censored survival data

In randomized clinical trials with survival outcome, there has been an increasing interest in subgroup identification based on baseline genomic, proteomic markers, or clinical characteristics. Some of the existing methods identify subgroups that benefit substantially from the experimental treatment by directly modeling outcomes or treatment effect. When the goal is to find an optimal treatment for a given patient rather than finding the right patient for a given treatment, methods under the individualized treatment regime framework estimate an individualized treatment rule that would lead to the best expected clinical outcome as measured by a value function. Connecting the concept of value function to subgroup identification, we propose a nonparametric method that searches for subgroup membership scores by maximizing a value function that directly reflects the subgroup-treatment interaction effect based on restricted mean survival time. A gradient tree boosting algorithm is proposed to search for the individual subgroup membership scores. We conduct simulation studies to evaluate the performance of the proposed method and an application to an AIDS clinical trial is performed for illustration.

Ascertaining properties of weighting in the estimation of optimal treatment regimes under monotone missingness

Dynamic treatment regimes operationalize precision medicine as a sequence of decision rules, one per stage of clinical intervention, that map up-to-date patient information to a recommended intervention. An optimal treatment regime maximizes the mean utility when applied to the population of interest. Methods for estimating an optimal treatment regime assume the data to be fully observed, which rarely occurs in practice. A common approach is to first use multiple imputation and then pool the estimators across imputed datasets. However, this approach requires estimating the joint distribution of patient trajectories, which can be high-dimensional, especially when there are multiple stages of intervention. We examine the application of inverse probability weighted estimating equations as an alternative to multiple imputation in the context of monotonic missingness. This approach applies to a broad class of estimators of an optimal treatment regime including both Q-learning and a generalization of outcome weighted learning. We establish consistency under mild regularity conditions and demonstrate its advantages in finite samples using a series of simulation experiments and an application to a schizophrenia study.

Predictive approaches to heterogeneous treatment effects: a scoping review

BACKGROUND

Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial.

METHODS

We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel.

RESULTS

The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers).

CONCLUSIONS

Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

Resampling-based confidence intervals for model-free robust inference on optimal treatment regimes

We propose a new procedure for inference on optimal treatment regimes in the model-free setting, which does not require to specify an outcome regression model. Existing model-free estimators for optimal treatment regimes are usually not suitable for the purpose of inference, because they either have nonstandard asymptotic distributions or do not necessarily guarantee consistent estimation of the parameter indexing the Bayes rule due to the use of surrogate loss. We first study a smoothed robust estimator that directly targets the parameter corresponding to the Bayes decision rule for optimal treatment regimes estimation. This estimator is shown to have an asymptotic normal distribution. Furthermore, we verify that a resampling procedure provides asymptotically accurate inference for both the parameter indexing the optimal treatment regime and the optimal value function. A new algorithm is developed to calculate the proposed estimator with substantially improved speed and stability. Numerical results demonstrate the satisfactory performance of the new methods.

A semiparametric instrumental variable approach to optimal treatment regimes under endogeneity

There is a fast-growing literature on estimating optimal treatment regimes based on randomized trials or observational studies under a key identifying condition of no unmeasured confounding. Because confounding by unmeasured factors cannot generally be ruled out with certainty in observational studies or randomized trials subject to noncompliance, we propose a general instrumental variable approach to learning optimal treatment regimes under endogeneity. Specifically, we establish identification of both value function E [ Y D ( L ) ] for a given regime D and optimal regimes arg max D E [ Y D ( L ) ] with the aid of a binary instrumental variable, when no unmeasured confounding fails to hold. We also construct novel multiply robust classification-based estimators. Furthermore, we propose to identify and estimate optimal treatment regimes among those who would comply to the assigned treatment under a monotonicity assumption. In this latter case, we establish the somewhat surprising result that complier optimal regimes can be consistently estimated without directly collecting compliance information and therefore without the complier average treatment effect itself being identified. Our approach is illustrated via extensive simulation studies and a data application on the effect of child rearing on labor participation.

Joint modeling and multiple comparisons with the best of data from a SMART with survival outcomes

A dynamic treatment regimen (DTR) is a sequence of decision rules that can alter treatments or doses based on outcomes from prior treatment. In the case of two lines of treatment, a DTR specifies first-line treatment, and second-line treatment for responders and treatment for non-responders to the first-line treatment. A sequential, multiple assignment, randomized trial (SMART) is one such type of trial that has been designed to assess DTRs. The primary goal of our project is to identify the treatments, covariates, and their interactions result in the best overall survival rate. Many previously proposed methods to analyze data with survival outcomes from a SMART use inverse probability weighting and provide non-parametric estimation of survival rates, but no other information. Other methods have been proposed to identify and estimate the optimal DTR, but inference issues were seldom addressed. We apply a joint modeling approach to provide unbiased survival estimates as a mechanism to quantify baseline and time-varying covariate effects, treatment effects, and their interactions within regimens. The issue of multiple comparisons at specific time points is addressed using multiple comparisons with the best method.

Estimation and Validation of Ratio-based Conditional Average Treatment Effects Using Observational Data

While sample sizes in randomized clinical trials are large enough to estimate the average treatment effect well, they are often insufficient for estimation of treatment-covariate interactions critical to studying data-driven precision medicine. Observational data from real world practice may play an important role in alleviating this problem. One common approach in trials is to predict the outcome of interest with separate regression models in each treatment arm, and estimate the treatment effect based on the contrast of the predictions. Unfortunately, this simple approach may induce spurious treatment-covariate interaction in observational studies when the regression model is misspecified. Motivated by the need of modeling the number of relapses in multiple sclerosis patients, where the ratio of relapse rates is a natural choice of the treatment effect, we propose to estimate the conditional average treatment effect (CATE) as the ratio of expected potential outcomes, and derive a doubly robust estimator of this CATE in a semiparametric model of treatment-covariate interactions. We also provide a validation procedure to check the quality of the estimator on an independent sample. We conduct simulations to demonstrate the finite sample performance of the proposed methods, and illustrate their advantages on real data by examining the treatment effect of dimethyl fumarate compared to teriflunomide in multiple sclerosis patients.

Adaptive treatment and robust control

A control theory perspective on determination of optimal dynamic treatment regimes is considered. The aim is to adapt statistical methodology that has been developed for medical or other biostatistical applications to incorporate powerful control techniques that have been designed for engineering or other technological problems. Data tend to be sparse and noisy in the biostatistical area and interest has tended to be in statistical inference for treatment effects. In engineering fields, experimental data can be more easily obtained and reproduced and interest is more often in performance and stability of proposed controllers rather than modeling and inference per se. We propose that modeling and estimation should be based on standard statistical techniques but subsequent treatment policy should be obtained from robust control. To bring focus, we concentrate on A-learning methodology as developed in the biostatistical literature and H ∞ -synthesis from control theory. Simulations and two applications demonstrate robustness of the H ∞ strategy compared to standard A-learning in the presence of model misspecification or measurement error.

Near-optimal Individualized Treatment Recommendations

The individualized treatment recommendation (ITR) is an important analytic framework for precision medicine. The goal of ITR is to assign the best treatments to patients based on their individual characteristics. From the machine learning perspective, the solution to the ITR problem can be formulated as a weighted classification problem to maximize the mean benefit from the recommended treatments given patients' characteristics. Several ITR methods have been proposed in both the binary setting and the multicategory setting. In practice, one may prefer a more flexible recommendation that includes multiple treatment options. This motivates us to develop methods to obtain a set of near-optimal individualized treatment recommendations alternative to each other, called alternative individualized treatment recommendations (A-ITR). We propose two methods to estimate the optimal A-ITR within the outcome weighted learning (OWL) framework. Simulation studies and a real data analysis for Type 2 diabetic patients with injectable antidiabetic treatments are conducted to show the usefulness of the proposed A-ITR framework. We also show the consistency of these methods and obtain an upper bound for the risk between the theoretically optimal recommendation and the estimated one. An R package aitr has been developed, found at https://github.com/menghaomiao/aitr.

Quantifying treatment effects using the personalized chance of longer survival

The hazard ratio is widely used to measure or to summarize the magnitude of treatment effects, but it is justifiably difficult to interpret in a meaningful way to patients and perhaps for clinicians as well. In addition, it is most meaningful when the hazard functions are approximately proportional over time. We propose a new measure, termed personalized chance of longer survival. The measure, which quantifies the probability of living longer with one treatment over the another, accounts for individualized characteristics to directly address personalized treatment effects. Hence, the measure is patient focused, which can be used to evaluate subgroups easily. We believe it is intuitive to understand and clinically interpretable in the presence of nonproportionality. Furthermore, because it estimates the probability of living longer by some fixed amount of time, it encodes the probabilistic part of treatment effect estimation. We provide nonparametric estimation and inference procedures that can accommodate censored survival outcomes. We conduct extensive simulation studies, which characterize performance of the proposed method, and data from a large randomized Phase III clinical trial (SWOG S0819) are analyzed using the proposed method.

Noninferiority and equivalence tests in sequential, multiple assignment, randomized trials (SMARTs)

Adaptive interventions (AIs) are increasingly popular in the behavioral sciences. An AI is a sequence of decision rules that specify for whom and under what conditions different intervention options should be offered, in order to address the changing needs of individuals as they progress over time. The sequential, multiple assignment, randomized trial (SMART) is a novel trial design that was developed to aid in empirically constructing effective AIs. The sequential randomizations in a SMART often yield multiple AIs that are embedded in the trial by design. Many SMARTs are motivated by scientific questions pertaining to the comparison of such embedded AIs. Existing data analytic methods and sample size planning resources for SMARTs are suitable only for superiority testing, namely for testing whether one embedded AI yields better primary outcomes on average than another. This calls for noninferiority/equivalence testing methods, because AIs are often motivated by the need to deliver support/care in a less costly or less burdensome manner, while still yielding benefits that are equivalent or noninferior to those produced by a more costly/burdensome standard of care. Here, we develop data-analytic methods and sample-size formulas for SMARTs testing the noninferiority or equivalence of one AI over another. Sample size and power considerations are discussed with supporting simulations, and online resources for sample size planning are provided. A simulated data analysis shows how to test noninferiority and equivalence hypotheses with SMART data. For illustration, we use an example from a SMART in the area of health psychology aiming to develop an AI for promoting weight loss among overweight/obese adults. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

On restricted optimal treatment regime estimation for competing risks data

It is well accepted that individualized treatment regimes may improve the clinical outcomes of interest. However, positive treatment effects are often accompanied by certain side effects. Therefore, when choosing the optimal treatment regime for a patient, we need to consider both efficacy and safety issues. In this article, we propose to model time to a primary event of interest and time to severe side effects of treatment by a competing risks model and define a restricted optimal treatment regime based on cumulative incidence functions. The estimation approach is derived using a penalized value search method and investigated through extensive simulations. The proposed method is applied to an HIV dataset obtained from Health Sciences South Carolina, where we minimize the risk of treatment or virologic failures while controlling the risk of serious drug-induced side effects.

Robustifying Trial-Derived Optimal Treatment Rules for A Target Population

Treatment rules based on individual patient characteristics that are easy to interpret and disseminate are important in clinical practice. Properly planned and conducted randomized clinical trials are used to construct individualized treatment rules. However, it is often a concern that trial participants lack representativeness, so it limits the applicability of the derived rules to a target population. In this work, we use data from a single trial study to propose a two-stage procedure to derive a robust and parsimonious rule to maximize the benefit in the target population. The procedure allows a wide range of possible covariate distributions in the target population, with minimal assumptions on the first two moments of the covariate distribution. The practical utility and favorable performance of the methodology are demonstrated using extensive simulations and a real data application.

Multi-Armed Angle-Based Direct Learning for Estimating Optimal Individualized Treatment Rules With Various Outcomes

Estimating an optimal individualized treatment rule (ITR) based on patients' information is an important problem in precision medicine. An optimal ITR is a decision function that optimizes patients' expected clinical outcomes. Many existing methods in the literature are designed for binary treatment settings with the interest of a continuous outcome. Much less work has been done on estimating optimal ITRs in multiple treatment settings with good interpretations. In this article, we propose angle-based direct learning (AD-learning) to efficiently estimate optimal ITRs with multiple treatments. Our proposed method can be applied to various types of outcomes, such as continuous, survival, or binary outcomes. Moreover, it has an interesting geometric interpretation on the effect of different treatments for each individual patient, which can help doctors and patients make better decisions. Finite sample error bounds have been established to provide a theoretical guarantee for AD-learning. Finally, we demonstrate the superior performance of our method via an extensive simulation study and real data applications. Supplementary materials for this article are available online.

Bagging and deep learning in optimal individualized treatment rules

An ENsemble Deep Learning Optimal Treatment (EndLot) approach is proposed for personalized medicine problems. The statistical framework of the proposed method is based on the outcome weighted learning (OWL) framework which transforms the optimal decision rule problem into a weighted classification problem. We further employ an ensemble of deep neural networks (DNNs) to learn the optimal decision rule. Utilizing the flexibility of DNNs and the stability of bootstrap aggregation, the proposed method achieves a considerable improvement over existing methods. An R package "ITRlearn" is developed to implement the proposed method. Numerical performance is demonstrated via simulation studies and a real data analysis of the Cancer Cell Line Encyclopedia data.

Look before you leap: systematic evaluation of tree-based statistical methods in subgroup identification

Subgroup analysis, as the key component of personalized medicine development, has attracted a lot of interest in recent years. While a number of exploratory subgroup searching approaches have been proposed, informative evaluation criteria and scenario-based systematic comparison of these methods are still underdeveloped topics. In this article, we propose two evaluation criteria in connection with traditional type I error and power concepts, and another criterion to directly assess recovery performance of the underlying treatment effect structure. Extensive simulation studies are carried out to investigate empirical performance of a variety of tree-based exploratory subgroup methods under the proposed criteria. A real data application is also included to illustrate the necessity and importance of method evaluation.

Precision Medicine

Precision medicine seeks to maximize the quality of healthcare by individualizing the healthcare process to the uniquely evolving health status of each patient. This endeavor spans a broad range of scientific areas including drug discovery, genetics/genomics, health communication, and causal inference all in support of evidence-based, i.e., data-driven, decision making. Precision medicine is formalized as a treatment regime which comprises a sequence of decision rules, one per decision point, which map up-to-date patient information to a recommended action. The potential actions could be the selection of which drug to use, the selection of dose, timing of administration, specific diet or exercise recommendation, or other aspects of treatment or care. Statistics research in precision medicine is broadly focused on methodological development for estimation of and inference for treatment regimes which maximize some cumulative clinical outcome. In this review, we provide an overview of this vibrant area of research and present important and emerging challenges.

Studying treatment-effect heterogeneity in precision medicine through induced subgroups

Precision medicine, in the sense of tailoring the choice of medical treatment to patients' pretreatment characteristics, is nowadays gaining a lot of attention. Preferably, this tailoring should be realized in an evidence-based way, with key evidence in this regard pertaining to subgroups of patients that respond differentially to treatment (i.e., to subgroups involved in treatment-subgroup interactions). Often a-priori hypotheses on subgroups involved in treatment-subgroup interactions are lacking or are incomplete at best. Therefore, methods are needed that can induce such subgroups from empirical data on treatment effectiveness in a post hoc manner. Recently, quite a few such methods have been developed. So far, however, there is little empirical experience in their usage. This may be problematic for medical statisticians and statistically minded medical researchers, as many (nontrivial) choices have to be made during the data-analytic process. The main purpose of this paper is to discuss the major concepts and considerations when using these methods. This discussion will be based on a systematic, conceptual, and technical analysis of the type of research questions at play, and of the type of data that the methods can handle along with the available software, and a review of available empirical evidence. We will illustrate all this with the analysis of a dataset comparing several anti-depressant treatments.