Bayesian Nonparametric Policy Search with Application to Periodontal Recall Intervals

Methods in dynamic treatment regimens using observational healthcare data: A systematic review

We present a systematic review of methods used to estimate Dynamic Treatment Regimens (DTR) using observational healthcare data and provide a brief summary of their strengths and weaknesses, evaluation metrics, and suitable research problem settings. We considered all observational studies identified in PubMed or EMBASE between January 1950 until January 2022, including only studies that evaluated medical treatments or interventions as exposure and/or outcome in patients and where DTRs were estimated. 83 studies met our inclusion criteria; 44.6% estimating DTR utilizing reinforcement learning, 18.1% utilizing counterfactual-based models, 12.1% utilizing classification-based methods, and 9.6% utilized g-methods. Among the studies analyzed, 28.9% aimed to replicate human expert DTRs, while 71.1% aimed to refine and improve existing DTRs. Approximately two-thirds of studies (65.1%) reported the assumptions required for their applied methods, such as exchangeability, positivity, consistency, and Markov property. Most of the studies (83.1%) estimated DTRs with more than two treatment options; 50.6% mentioned time-varying confounders, only a few estimated conditional average treatment effects (7.2%). Most (85.5%) validated their methods, with 32.5% using expected outcomes (e.g., survival rates), 26.5% employing simulated data, and 25.3% conducting direct comparisons with observational data.

Bayesian semiparametric model for sequential treatment decisions with informative timing

We develop a Bayesian semiparametric model for the impact of dynamic treatment rules on survival among patients diagnosed with pediatric acute myeloid leukemia (AML). The data consist of a subset of patients enrolled in a phase III clinical trial in which patients move through a sequence of four treatment courses. At each course, they undergo treatment that may or may not include anthracyclines (ACT). While ACT is known to be effective at treating AML, it is also cardiotoxic and can lead to early death for some patients. Our task is to estimate the potential survival probability under hypothetical dynamic ACT treatment strategies, but there are several impediments. First, since ACT is not randomized, its effect on survival is confounded over time. Second, subjects initiate the next course depending on when they recover from the previous course, making timing potentially informative of subsequent treatment and survival. Third, patients may die or drop out before ever completing the full treatment sequence. We develop a generative Bayesian semiparametric model based on Gamma Process priors to address these complexities. At each treatment course, the model captures subjects' transition to subsequent treatment or death in continuous time. G-computation is used to compute a posterior over potential survival probability that is adjusted for time-varying confounding. Using our approach, we estimate the efficacy of hypothetical treatment rules that dynamically modify ACT based on evolving cardiac function.

Bayesian inference for optimal dynamic treatment regimes in practice

In this work, we examine recently developed methods for Bayesian inference of optimal dynamic treatment regimes (DTRs). DTRs are a set of treatment decision rules aimed at tailoring patient care to patient-specific characteristics, thereby falling within the realm of precision medicine. In this field, researchers seek to tailor therapy with the intention of improving health outcomes; therefore, they are most interested in identifying optimal DTRs. Recent work has developed Bayesian methods for identifying optimal DTRs in a family indexed by ψ via Bayesian dynamic marginal structural models (MSMs) (Rodriguez Duque D, Stephens DA, Moodie EEM, Klein MB. Semiparametric Bayesian inference for dynamic treatment regimes via dynamic regime marginal structural models. Biostatistics; 2022. (In Press)); we review the proposed estimation procedure and illustrate its use via the new BayesDTR R package. Although methods in Rodriguez Duque D, Stephens DA, Moodie EEM, Klein MB. (Semiparametric Bayesian inference for dynamic treatment regimes via dynamic regime marginal structural models. Biostatistics; 2022. (In Press)) can estimate optimal DTRs well, they may lead to biased estimators when the model for the expected outcome if everyone in a population were to follow a given treatment strategy, known as a value function, is misspecified or when a grid search for the optimum is employed. We describe recent work that uses a Gaussian process ( G P ) prior on the value function as a means to robustly identify optimal DTRs (Rodriguez Duque D, Stephens DA, Moodie EEM. Estimation of optimal dynamic treatment regimes using Gaussian processes; 2022. Available from: https://doi.org/10.48550/arXiv.2105.12259). We demonstrate how a G P approach may be implemented with the BayesDTR package and contrast it with other value-search approaches to identifying optimal DTRs. We use data from an HIV therapeutic trial in order to illustrate a standard analysis with these methods, using both the original observed trial data and an additional simulated component to showcase a longitudinal (two-stage DTR) analysis.

A multiple controlled-release hydrophilicity minocycline hydrochloride delivery system for the efficient treatment of periodontitis

The complexity of periodontitis, including the complex formation mechanisms and the complex periodontium physiological environment, as well as the complex association with multiple complications, often results in poor therapy effects. Herein, we aimed to design a nanosystem with a controlled release of minocycline hydrochloride (MH) and good retention to effectively treat periodontitis by inhibiting inflammation and repairing the alveolar bone. Firstly, insoluble ion-pairing (IIP) complexes were constructed to improve the encapsulation efficiency of hydrophilic MH in PLGA nanoparticles. Then, a nanogenerator was constructed and combined with a double emulsion method to encapsulate the complexes into PLGA nanoparticles (MH-NPs). The average particle size of MH-NPs was about 100 nm as observed by AFM and TEM, and the drug loading and encapsulation efficiency were 9.59% and 95.58%, respectively. Finally, a multifunctional system (MH-NPs-in-gels) was prepared by dispersing MH-NPs into thermosensitive gels, which could continue to release drug for 21 days in vitro. And the release mechanism showed that this controlled release behavior for MH was influenced by the insoluble ion-pairing complex, PLGA nanoparticles, and gels. In addition, the periodontitis rat model was established to investigate the pharmacodynamic effects. After 4 weeks of treatment, changes in the alveolar bone were assessed by Micro-CT (BV/TV: 70.88%; BMD: 0.97 g/cm³; TB.Th: 0.14 mm; Tb.N: 6.39 mm^-1; Tb.Sp: 0.07 mm). The mechanism of MH-NPs-in-gels in vivo was clarified by the analysis of pharmacodynamic results, which showed that insoluble ion-pairing complexes with the aid of PLGA nanoparticles and gels achieved significant anti-inflammatory effects and bone repair capabilities. In conclusion, the multiple controlled-release hydrophilicity MH delivery system would have good prospects for the effective treatment of periodontitis.

CAPITAL: Optimal subgroup identification via constrained policy tree search

Personalized medicine, a paradigm of medicine tailored to a patient's characteristics, is an increasingly attractive field in health care. An important goal of personalized medicine is to identify a subgroup of patients, based on baseline covariates, that benefits more from the targeted treatment than other comparative treatments. Most of the current subgroup identification methods only focus on obtaining a subgroup with an enhanced treatment effect without paying attention to subgroup size. Yet, a clinically meaningful subgroup learning approach should identify the maximum number of patients who can benefit from the better treatment. In this article, we present an optimal subgroup selection rule (SSR) that maximizes the number of selected patients, and in the meantime, achieves the pre‐specified clinically meaningful mean outcome, such as the average treatment effect. We derive two equivalent theoretical forms of the optimal SSR based on the contrast function that describes the treatment‐covariates interaction in the outcome. We further propose a constrained policy tree search algorithm (CAPITAL) to find the optimal SSR within the interpretable decision tree class. The proposed method is flexible to handle multiple constraints that penalize the inclusion of patients with negative treatment effects, and to address time to event data using the restricted mean survival time as the clinically interesting mean outcome. Extensive simulations, comparison studies, and real data applications are conducted to demonstrate the validity and utility of our method.

Estimating optimal dynamic treatment strategies under resource constraints using dynamic marginal structural models

Methods for estimating optimal treatment strategies typically assume unlimited access to resources. However, when a health system has resource constraints, such as limited funds, access to medication, or monitoring capabilities, medical decisions must account for competition between individuals in resource usage. The problem of incorporating resource constraints into optimal treatment strategies has been solved for point exposures (1), that is, treatment strategies entailing a decision at just one time point. However, attempts to directly generalize the point exposure solution to dynamic time-varying treatment strategies run into complications. We sidestep these complications by targeting the optimal strategy within a clinically defined subclass. Our approach is to employ dynamic marginal structural models to estimate (counterfactual) resource usage under the class of candidate treatment strategies and solve a constrained optimization problem to choose the optimal strategy for which expected resource usage is within acceptable limits. We apply this method to determine the optimal dynamic monitoring strategy for people living with HIV when resource limits on monitoring exist using observational data from the HIV-CAUSAL Collaboration.

A spatiotemporal recommendation engine for malaria control

Malaria is an infectious disease affecting a large population across the world, and interventions need to be efficiently applied to reduce the burden of malaria. We develop a framework to help policy-makers decide how to allocate limited resources in realtime for malaria control. We formalize a policy for the resource allocation as a sequence of decisions, one per intervention decision, that map up-to-date disease related information to a resource allocation. An optimal policy must control the spread of the disease while being interpretable and viewed as equitable to stakeholders. We construct an interpretable class of resource allocation policies that can accommodate allocation of resources residing in a continuous domain and combine a hierarchical Bayesian spatiotemporal model for disease transmission with a policy-search algorithm to estimate an optimal policy for resource allocation within the pre-specified class. The estimated optimal policy under the proposed framework improves the cumulative long-term outcome compared with naive approaches in both simulation experiments and application to malaria interventions in the Democratic Republic of the Congo.

A scoping review of studies using observational data to optimise dynamic treatment regimens

BACKGROUND

Dynamic treatment regimens (DTRs) formalise the multi-stage and dynamic decision problems that clinicians often face when treating chronic or progressive medical conditions. Compared to randomised controlled trials, using observational data to optimise DTRs may allow a wider range of treatments to be evaluated at a lower cost. This review aimed to provide an overview of how DTRs are optimised with observational data in practice.

METHODS

Using the PubMed database, a scoping review of studies in which DTRs were optimised using observational data was performed in October 2020. Data extracted from eligible articles included target medical condition, source and type of data, statistical methods, and translational relevance of the included studies.

RESULTS

From 209 PubMed abstracts, 37 full-text articles were identified, and a further 26 were screened from the reference lists, totalling 63 articles for inclusion in a narrative data synthesis. Observational DTR models are a recent development and their application has been concentrated in a few medical areas, primarily HIV/AIDS (27, 43%), followed by cancer (8, 13%), and diabetes (6, 10%). There was substantial variation in the scope, intent, complexity, and quality between the included studies. Statistical methods that were used included inverse-probability weighting (26, 41%), the parametric G-formula (16, 25%), Q-learning (10, 16%), G-estimation (4, 6%), targeted maximum likelihood/minimum loss-based estimation (4, 6%), regret regression (3, 5%), and other less common approaches (10, 16%). Notably, studies that were primarily intended to address real-world clinical questions (18, 29%) tended to use inverse-probability weighting and the parametric G-formula, relatively well-established methods, along with a large amount of data. Studies focused on methodological developments (45, 71%) tended to be more complicated and included a demonstrative real-world application only.

CONCLUSIONS

As chronic and progressive conditions become more common, the need will grow for personalised treatments and methods to estimate the effects of DTRs. Observational DTR studies will be necessary, but so far their use to inform clinical practice has been limited. Focusing on simple DTRs, collecting large and rich clinical datasets, and fostering tight partnerships between content experts and data analysts may result in more clinically relevant observational DTR studies.