Estimating optimal dynamic treatment strategies under resource constraints using dynamic marginal structural models

Methods in dynamic treatment regimens using observational healthcare data: A systematic review

We present a systematic review of methods used to estimate Dynamic Treatment Regimens (DTR) using observational healthcare data and provide a brief summary of their strengths and weaknesses, evaluation metrics, and suitable research problem settings. We considered all observational studies identified in PubMed or EMBASE between January 1950 until January 2022, including only studies that evaluated medical treatments or interventions as exposure and/or outcome in patients and where DTRs were estimated. 83 studies met our inclusion criteria; 44.6% estimating DTR utilizing reinforcement learning, 18.1% utilizing counterfactual-based models, 12.1% utilizing classification-based methods, and 9.6% utilized g-methods. Among the studies analyzed, 28.9% aimed to replicate human expert DTRs, while 71.1% aimed to refine and improve existing DTRs. Approximately two-thirds of studies (65.1%) reported the assumptions required for their applied methods, such as exchangeability, positivity, consistency, and Markov property. Most of the studies (83.1%) estimated DTRs with more than two treatment options; 50.6% mentioned time-varying confounders, only a few estimated conditional average treatment effects (7.2%). Most (85.5%) validated their methods, with 32.5% using expected outcomes (e.g., survival rates), 26.5% employing simulated data, and 25.3% conducting direct comparisons with observational data.

Transportability of causal inference under random dynamic treatment regimes for kidney-pancreas transplantation

A difficult decision for patients in need of kidney-pancreas transplant is whether to seek a living kidney donor or wait to receive both organs from one deceased donor. The framework of dynamic treatment regimes (DTRs) can inform this choice, but a patient-relevant strategy such as "wait for deceased-donor transplant" is ill-defined because there are multiple versions of treatment (i.e., wait times, organ qualities). Existing DTR methods average over the distribution of treatment versions in the data, estimating survival under a "representative intervention." This is undesirable if transporting inferences to a target population such as patients today, who experience shorter wait times thanks to evolutions in allocation policy. We, therefore, propose the concept of a generalized representative intervention (GRI): a random DTR that assigns treatment version by drawing from the distribution among strategy compliers in the target population (e.g., patients today). We describe an inverse-probability-weighted product-limit estimator of survival under a GRI that performs well in simulations and can be implemented in standard statistical software. For continuous treatments (e.g., organ quality), weights are reformulated to depend on probabilities only, not densities. We apply our method to a national database of kidney-pancreas transplant candidates from 2001-2020 to illustrate that variability in transplant rate across years and centers results in qualitative differences in the optimal strategy for patient survival.

Longitudinal incremental propensity score interventions for limited resource settings

Many real-life treatments are of limited supply and cannot be provided to all individuals in the population. For example, patients on the liver transplant waiting list usually cannot be assigned a liver transplant immediately at the time they reach highest priority because a suitable organ is not immediately available. In settings with limited supply, investigators are often interested in the effects of treatment strategies in which a limited proportion of patients receive an organ at a given time, that is, treatment regimes satisfying resource constraints. Here, we describe an estimand that allows us to define causal effects of treatment strategies that satisfy resource constraints: incremental propensity score interventions (IPSIs) for limited resources. IPSIs flexibly constrain time-varying resource utilization through proportional scaling of patients' natural propensities for treatment, thereby preserving existing propensity rank ordering compared to the status quo. We derive a simple class of inverse-probability-weighted estimators, and we apply one such estimator to evaluate the effect of restricting or expanding utilization of "increased risk" liver organs to treat patients with end-stage liver disease.