What to do after smoking relapse? A sequential multiple assignment randomized trial of chronic care smoking treatments

AIM

To compare effects of three post-relapse interventions on smoking abstinence.

DESIGN

Sequential three-phase multiple assignment randomized trial (SMART).

SETTING

Eighteen Wisconsin, USA, primary care clinics.

PARTICIPANTS

A total of 1154 primary care patients (53.6% women, 81.2% White) interested in quitting smoking enrolled from 2015 to 2019; 582 relapsed and were randomized to relapse recovery treatment.

INTERVENTIONS

In phase 1, patients received cessation counseling and 8 weeks nicotine patch. Those who relapsed and agreed were randomized to a phase 2 relapse recovery group: (1) reduction counseling + nicotine mini-lozenges + encouragement to quit starting 1 month post-randomization (preparation); (2) repeated encouragement to quit starting immediately post-randomization (recycling); or (3) advice to call the tobacco quitline (control). The first two groups could opt into phase 3 new quit treatment [8 weeks nicotine patch + mini-lozenges plus randomization to two treatment factors (skill training and supportive counseling) in a 2 × 2 design]. Phase 2 and 3 interventions lasted ≤ 15 months.

MEASUREMENTS

The study was powered to compare each active phase 2 treatment with the control on the primary outcome: biochemically confirmed 7-day point-prevalence abstinence 14 months post initiating phase 2 relapse recovery treatment. Exploratory analyses tested for phase 3 counseling factor effects.

FINDINGS

Neither skill training nor supportive counseling (each on versus off) increased 14-month abstinence rates; skills on versus off 9.3% (14/151) versus 5.2% (8/153), P = 0.19; support on versus off 6.6% (10/152) versus 7.9% (12/152), P = 0.73. Phase 2 preparation did not produce higher 14-month abstinence rates than quitline referral; 3.6% (8/220) versus 2.1% [3/145; risk difference = 1.5%, 95% confidence interval (CI) = -1.8-5.0%, odds ratio (OR) = 1.8, 95% CI = 0.5-6.9]. Recycling, however, produced higher abstinence rates than quitline referral; 6.9% (15/217) versus 2.1% (three of 145; risk difference, 4.8%, 95% CI = 0.7-8.9%, OR = 3.5, 95% CI = 1.0-12.4). Recycling produced greater entry into new quit treatment than preparation: 83.4% (181/217) versus 55.9% (123/220), P < 0.0001.

CONCLUSIONS

Among people interested in quitting smoking, immediate encouragement post-relapse to enter a new round of smoking cessation treatment ('recycling') produced higher probability of abstinence than tobacco quitline referral. Recycling produced higher rates of cessation treatment re-engagement than did preparation/cutting down using more intensive counseling and pharmacotherapy.

SMART Binary: New Sample Size Planning Resources for SMART Studies with Binary Outcome Measurements

Sequential Multiple-Assignment Randomized Trials (SMARTs) play an increasingly important role in psychological and behavioral health research. This experimental approach enables researchers to answer scientific questions about how to sequence and match interventions to the unique, changing needs of individuals. A variety of sample size planning resources for SMART studies have been developed, enabling researchers to plan SMARTs for addressing different types of scientific questions. However, relatively limited attention has been given to planning SMARTs with binary (dichotomous) outcomes, which often require higher sample sizes relative to continuous outcomes. Existing resources for estimating sample size requirements for SMARTs with binary outcomes do not consider the potential to improve power by including a baseline measurement and/or multiple repeated outcome measurements. The current paper addresses this issue by providing sample size planning simulation procedures and approximate formulas for two-wave repeated measures binary outcomes (i.e., two measurement times for the outcome variable, before and after intervention delivery). The simulation results agree well with the formulas. We also discuss how to use simulations to calculate power for studies with more than two outcome measurement occasions. Results show that having at least one repeated measurement of the outcome can substantially improve power under certain conditions.

Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication

The promise of adaptation and adaptive designs in implementation science has been hindered by the lack of clarity and precision in defining what it means to adapt, especially regarding the distinction between adaptive study designs and adaptive implementation strategies. To ensure a common language for science and practice, authors reviewed the implementation science literature and found that the term adaptive was used to describe interventions, implementation strategies, and trial designs. To provide clarity and offer recommendations for reporting and strengthening study design, we propose a taxonomy that describes fixed versus adaptive implementation strategies and implementation trial designs. To improve impact, (a) future implementation studies should prespecify implementation strategy core functions that in turn can be taught to and replicated by health system/community partners, (b) funders should support exploratory studies that refine and specify implementation strategies, and (c) investigators should systematically address design requirements and ethical considerations (e.g., randomization, blinding/masking) with health system/community partners. Expected final online publication date for the Annual Review of Public Health, Volume 45 is April 2024. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Estimating tree-based dynamic treatment regimes using observational data with restricted treatment sequences

A dynamic treatment regime (DTR) is a sequence of decision rules that provide guidance on how to treat individuals based on their static and time-varying status. Existing observational data are often used to generate hypotheses about effective DTRs. A common challenge with observational data, however, is the need for analysts to consider "restrictions" on the treatment sequences. Such restrictions may be necessary for settings where (1) one or more treatment sequences that were offered to individuals when the data were collected are no longer considered viable in practice, (2) specific treatment sequences are no longer available, or (3) the scientific focus of the analysis concerns a specific type of treatment sequences (eg, "stepped-up" treatments). To address this challenge, we propose a restricted tree-based reinforcement learning (RT-RL) method that searches for an interpretable DTR with the maximum expected outcome, given a (set of) user-specified restriction(s), which specifies treatment options (at each stage) that ought not to be considered as part of the estimated tree-based DTR. In simulations, we evaluate the performance of RT-RL versus the standard approach of ignoring the partial data for individuals not following the (set of) restriction(s). The method is illustrated using an observational data set to estimate a two-stage stepped-up DTR for guiding the level of care placement for adolescents with substance use disorder.

Optimizing Longitudinal Tobacco Cessation Treatment in Lung Cancer Screening: A Sequential, Multiple Assignment, Randomized Trial

Importance

Nearly half of the 14.8 million US adults eligible for lung cancer screening (LCS) smoke cigarettes. The optimal smoking cessation program components for the LCS setting are unclear.

Objective

To assess the effect of adding a referral to prescription medication therapy management (MTM) to the tobacco longitudinal care (TLC) program among patients eligible for LCS who smoke and do not respond to early tobacco treatment and to assess the effect of decreasing the intensity of TLC among participants who do respond to early treatment.

Design, Setting, and Participants

This randomized clinical trial included patients who currently smoked cigarettes daily and were eligible for LCS. Recruitment took place at primary care centers and LCS programs at 3 large health systems in the US and began in October 2016, and 18-month follow-up was completed April 2021.

Interventions

(1) TLC comprising intensive telephone coaching and combination nicotine replacement therapy for 1 year with at least monthly contact; (2) TLC with MTM, MTM offered pharmacist-referral for prescription medications; and (3) Quarterly TLC, intensity of TLC was decreased to quarterly contact. Intervention assignments were based on early response to tobacco treatment (abstinence) that was assessed either 4 weeks or 8 weeks after treatment initiation.

Main outcomes and Measures

Self-reported, 6-month prolonged abstinence at 18-month.

Results

Of 636 participants, 228 (35.9%) were female, 564 (89.4%) were White individuals, and the median (IQR) age was 64.3 (59.6-68.8) years. Four weeks or 8 weeks after treatment initiation, 510 participants (80.2%) continued to smoke (ie, early treatment nonresponders) and 126 participants (19.8%) had quit (ie, early treatment responders). The 18 month follow-up survey response rate was 83.2% (529 of 636). Across TLC groups at 18 months follow-up, the overall 6-month prolonged abstinence rate was 24.4% (129 of 529). Among the 416 early treatment nonresponders, 6-month prolonged abstinence for TLC with MTM vs TLC was 17.8% vs 16.4% (adjusted odds ratio [aOR] 1.13; 95% CI, 0.67-1.89). In TLC with MTM, 98 of 254 participants (39%) completed at least 1 MTM visit. Among 113 early treatment responders, 6-month prolonged abstinence for Quarterly TLC vs TLC was 24 of 55 (43.6%) vs 34 of 58 (58.6%) (aOR, 0.54; 95% CI, 0.25-1.17).

Conclusions and Relevance

In this randomized clinical trial, adding referral to MTM with TLC for participants who did not respond to early treatment did not improve smoking abstinence. Stepping down to Quarterly TLC among early treatment responders is not recommended. Integrating longitudinal tobacco cessation care with LCS is feasible and associated with clinically meaningful quit rates.

Trial Registration

ClinicalTrials.gov Identifier: NCT02597491.

Self-relevant appeals to engage in self-monitoring of alcohol use: A microrandomized trial

OBJECTIVE

While self-monitoring can help mitigate alcohol misuse in young adults, engagement with digital self-monitoring is suboptimal. The present study investigates the utility of two types of digital prompts (reminders) to encourage young adults to self-monitor their alcohol use. These prompts leverage information that is self-relevant (i.e., represents and is valuable) to the person.

METHOD

Five hundred ninety-one college students (Mage = 18; 61% = female, 76% = White) were enrolled in an 8-week intervention study involving biweekly digital self-monitoring of their alcohol use. At baseline, participants selected an item they would like to purchase for themselves and their preferred charitable organization. Then, biweekly, participants were microrandomized to a prompt highlighting the opportunity to either (a) win their preferred item (self-interest prompt); or (b) donate to their preferred charity (prosocial prompt). Following self-monitoring completion, participants allocated reward points toward lottery drawings for their preferred item or charity.

RESULTS

The self-interest (vs. prosocial) prompt was significantly more effective in promoting proximal self-monitoring at the beginning of the study, Est = exp(.14) = 1.15; 95% confidence interval (CI) [1.01, 1.29], whereas the prosocial (vs. self-interest) prompt was significantly more effective at the end, Est = exp(-.17) = 0.84; 95% CI [0.70, 0.98]. Further, the prosocial (vs. self-interest) prompt was significantly more effective among participants who previously allocated all their reward points to drawings for their preferred item, Est = exp(-.15) = 0.86; 95% CI [.75, .97].

CONCLUSIONS

These results suggest that the advantage of prompts that appeal to a person's self-interest (vs. prosocial) motives varies over time and based on what reward options participants prioritized in previous decisions. Theoretical and practical implications for intervention design are discussed. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Sequential, Multiple Assignment, Randomized Trial Designs

This JAMA Guide to Statistics and Methods explains sequential, multiple assignment, randomized trial (SMART) study designs, in which some or all participants are randomized at 2 or more decision points depending on the participant’s response to prior treatment.

Accounting for Context in Randomized Trials after Assignment

Many preventive trials randomize individuals to intervention condition which is then delivered in a group setting. Other trials randomize higher levels, say organizations, and then use learning collaboratives comprised of multiple organizations to support improved implementation or sustainment. Other trials randomize or expand existing social networks and use key opinion leaders to deliver interventions through these networks. We use the term contextually driven to refer generally to such trials (traditionally referred to as clustering, where groups are formed either pre-randomization or post-randomization - i.e., a cluster-randomized trial), as these groupings or networks provide fixed or time-varying contexts that matter both theoretically and practically in the delivery of interventions. While such contextually driven trials can provide efficient and effective ways to deliver and evaluate prevention programs, they all require analytical procedures that take appropriate account of non-independence, something not always appreciated. Published analyses of many prevention trials have failed to take this into account. We discuss different types of contextually driven designs and then show that even small amounts of non-independence can inflate actual Type I error rates. This inflation leads to rejecting the null hypotheses too often, and erroneously leading us to conclude that there are significant differences between interventions when they do not exist. We describe a procedure to account for non-independence in the important case of a two-arm trial that randomizes units of individuals or organizations in both arms and then provides the active treatment in one arm through groups formed after assignment. We provide sample code in multiple programming languages to guide the analyst, distinguish diverse contextually driven designs, and summarize implications for multiple audiences.

Primary aim results of a clustered SMART for developing a school-level, adaptive implementation strategy to support CBT delivery at high schools in Michigan

Background

Schools increasingly provide mental health services to students, but often lack access to implementation strategies to support school-based (and school professional [SP]) delivery of evidence-based practices. Given substantial heterogeneity in implementation barriers across schools, development of adaptive implementation strategies that guide which implementation strategies to provide to which schools and when may be necessary to support scale-up.

Methods

A clustered, sequential, multiple-assignment randomized trial (SMART) of high schools across Michigan was used to inform the development of a school-level adaptive implementation strategy for supporting SP-delivered cognitive behavioral therapy (CBT). All schools were first provided with implementation support informed by Replicating Effective Programs (REP) and then were randomized to add in-person Coaching or not (phase 1). After 8 weeks, schools were assessed for response based on SP-reported frequency of CBT delivered to students and/or barriers reported. Responder schools continued with phase 1 implementation strategies. Slower-responder schools (not providing ≥ 3 CBT components to ≥10 students or >2 organizational barriers identified) were re-randomized to add Facilitation to current support or not (phase 2). The primary aim hypothesis was that SPs at schools receiving the REP + Coaching + Facilitation adaptive implementation strategy would deliver more CBT sessions than SPs at schools receiving REP alone. Secondary aims compared four implementation strategies (Coaching vs no Coaching × Facilitation vs no Facilitation) on CBT sessions delivered, including by type (group, brief and full individual). Analyses used a marginal, weighted least squares approach developed for clustered SMARTs.

Results

SPs (n = 169) at 94 high schools entered the study. N = 83 schools (88%) were slower-responders after phase 1. Contrary to the primary aim hypothesis, there was no evidence of a significant difference in CBT sessions delivered between REP + Coaching + Facilitation and REP alone (111.4 vs. 121.1 average total CBT sessions; p = 0.63). In secondary analyses, the adaptive strategy that offered REP + Facilitation resulted in the highest average CBT delivery (154.1 sessions) and the non-adaptive strategy offering REP + Coaching the lowest (94.5 sessions).

Conclusions

The most effective strategy in terms of average SP-reported CBT delivery is the adaptive implementation strategy that (i) begins with REP, (ii) augments with Facilitation for slower-responder schools (schools where SPs identified organizational barriers or struggled to deliver CBT), and (iii) stays the course with REP for responder schools.

Trial registration

ClinicalTrials.gov, NCT03541317, May 30, 2018.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01211-w.

Expanding access to medications for opioid use disorder in primary care clinics: an evaluation of common implementation strategies and outcomes

BACKGROUND

To combat the opioid epidemic in the USA, unprecedented federal funding has been directed to states and territories to expand access to prevention, overdose rescue, and medications for opioid use disorder (MOUD). Similar to other states, California rapidly allocated these funds to increase reach and adoption of MOUD in safety-net, primary care settings such as Federally Qualified Health Centers. Typical of current real-world implementation endeavors, a package of four implementation strategies was offered to all clinics. The present study examines (i) the pre-post effect of the package of strategies, (ii) whether/how this effect differed between new (start-up) versus more established (scale-up) MOUD practices, and (iii) the effect of clinic engagement with each of the four implementation strategies.

METHODS

Forty-one primary care clinics were offered access to four implementation strategies: (1) Enhanced Monitoring and Feedback, (2) Learning Collaboratives, (3) External Facilitation, and (4) Didactic Webinars. Using linear mixed effects models, RE-AIM guided outcomes of reach, adoption, and implementation quality were assessed at baseline and at 9 months follow-up.

RESULTS

Of the 41 clinics, 25 (61%) were at MOUD start-up and 16 (39%) were at scale-up phases. Pre-post difference was observed for the primary outcome of percent of patient prescribed MOUD (reach) (β_time = 3.99; 0.73 to 7.26; p = 0.02). The largest magnitude of change occurred in implementation quality (ES = 0.68; 95% CI = 0.66 to 0.70). Baseline MOUD capability moderated the change in reach (start-ups 22.60%, 95% CI = 16.05 to 29.15; scale-ups -4.63%, 95% CI = -7.87 to -1.38). Improvement in adoption and implementation quality were moderately associated with early prescriber engagement in Learning Collaboratives (adoption: ES = 0.61; 95% CI = 0.25 to 0.96; implementation quality: ES = 0.55; 95% CI = 0.41 to 0.69). Improvement in adoption was also associated with early prescriber engagement in Didactic Webinars (adoption: ES = 0.61; 95% CI = 0.20 to 1.05).

CONCLUSIONS

Rather than providing an all-clinics-get-all-components package of implementation strategies, these data suggest that it may be more efficient and effective to tailor the provision of implementation strategies based on the needs of clinic. Future implementation endeavors could benefit from (i) greater precision in the provision of implementation strategies based on contextual determinants, and (ii) the inclusion of strategies targeting engagement.

Estimating time-varying causal excursion effect in mobile health with binary outcomes

Advances in wearables and digital technology now make it possible to deliver behavioral mobile health interventions to individuals in their everyday life. The micro-randomized trial is increasingly used to provide data to inform the construction of these interventions. In a micro-randomized trial, each individual is repeatedly randomized among multiple intervention options, often hundreds or even thousands of times, over the course of the trial. This work is motivated by multiple micro-randomized trials that have been conducted or are currently in the field, in which the primary outcome is a longitudinal binary outcome. The primary aim of such micro-randomized trials is to examine whether a particular time-varying intervention has an effect on the longitudinal binary outcome, often marginally over all but a small subset of the individual's data. We propose the definition of causal excursion effect that can be used in such primary aim analysis for micro-randomized trials with binary outcomes. Under rather restrictive assumptions one can, based on existing literature, derive a semiparametric, locally efficient estimator of the causal effect. Starting from this estimator, we develop an estimator that can be used as the basis of a primary aim analysis under more plausible assumptions. Simulation studies are conducted to compare the estimators. We illustrate the developed methods using data from the micro-randomized trial, BariFit. In BariFit, the goal is to support weight maintenance for individuals who received bariatric surgery.

Multitiered Systems of Support, Adaptive Interventions, and SMART Designs

This article introduces the special section on adaptive interventions and sequential multiple-assignment randomized trial (SMART) research designs. In addition to describing the two accompanying articles, we discuss features of adaptive interventions (AIs) and describe the use of SMART design to optimize AIs in the context of multitiered systems of support (MTSS) and integrated MTSS. AI is a treatment delivery model that explicitly specifies how information about individuals should be used to decide which treatment to provide in practice. Principles that apply to the design of AIs may help to more clearly operationalize MTSS-based programs, improve their implementation in school settings, and increase their efficacy when used according to evidence-based decision rules. A SMART is a research design for developing and optimizing MTSS-based programs. We provide a running example of a SMART design to optimize an MTSS-aligned AI that integrates academic and behavioral interventions.

Main outcomes of M-bridge: A sequential multiple assignment randomized trial (SMART) for developing an adaptive preventive intervention for college drinking

Objective: The goal was to develop a universal and resource-efficient adaptive preventive intervention (API) for incoming first-year students as a bridge to indicated interventions to address alcohol-related risks. The aims were to examine: (a) API versus assessment-only control, (b) the different APIs (i.e., 4 intervention sequences) embedded in the study design, and (c) moderators of intervention effects on binge drinking. Method: A sequential multiple assignment randomized trial (SMART) included two randomizations: timing (summer before vs. first semester) of universal personalized normative feedback and biweekly self-monitoring and, for heavy drinkers, bridging strategy (resource email vs. health coaching invitation). Participants (N = 891, 62.4% female, 76.8% White) were surveyed at the end of first and second semesters. The primary outcome was binge drinking frequency (4+/5+ drinks for females/males); secondary outcomes were alcohol consequences and health services utilization. Results: API (vs. control) was not significantly associated with outcomes. There were no differences between embedded APIs. Among heavy drinkers, the resource email (vs. health coach invitation) led to greater health services utilization. Moderator analyses suggested students intending to pledge into Greek life benefited more from any API (vs. control; 42% smaller increase from precollege in binge drinking frequency). Conclusions: Although overall effects were not significant, students at high risk (i.e., entering fraternities/sororities) did benefit more from the intervention. Furthermore, the resource email was effective for heavier drinkers. A technology-based strategy to deliver targeted resource-light interventions for heavy drinkers may be effective for reducing binge drinking during the transition to college. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

A sequential multiple assignment randomized trial (SMART) protocol for empirically developing an adaptive preventive intervention for college student drinking reduction

College student alcohol use and associated negative consequences are clear public health problems with consequences including damage to self, others, and institutions. This paper describes the protocol of a research study designed to answer a number of important questions in the development of an adaptive preventive intervention (API) to reduce high-risk drinking among first-year college students. The API is designed to educate students and to motivate heavy-drinking college students to engage in existing resources to support reducing high-risk alcohol use, by leveraging technology-based intervention modalities. The primary outcome is a reduction in binge drinking, with secondary outcomes of reducing negative alcohol-related consequences and increasing health services utilization. Adaptive preventive interventions have the potential to reduce the acute and long-term negative health consequences of young adult alcohol use.

Identifying optimal level-of-care placement decisions for adolescent substance use treatment

BACKGROUND

Adolescents respond differentially to substance use treatment based on their individual needs and goals. Providers may benefit from guidance (via decision rules) for personalizing aspects of treatment, such as level-of-care (LOC) placements, like choosing between outpatient or inpatient care. The field lacks an empirically-supported foundation to inform the development of an adaptive LOC-placement protocol. This work begins to build the evidence base for adaptive protocols by estimating them from a large observational dataset.

METHODS

We estimated two-stage LOC-placement protocols adapted to individual adolescent characteristics collected from the Global Appraisal of Individual Needs assessment tool (n = 10,131 adolescents). We used a modified version of Q-learning, a regression-based method for estimating personalized treatment rules over time, to estimate four protocols, each targeting a potentially distinct treatment goal: one primary outcome (a composite of ten positive treatment outcomes) and three secondary (substance frequency, substance problems, and emotional problems). We compared the adaptive protocols to non-adaptive protocols using an independent dataset.

RESULTS

Intensive outpatient was recommended for all adolescents at intake for the primary outcome, while low-risk adolescents were recommended for no further treatment at followup while higher-risk patients were recommended to inpatient. Our adaptive protocols outperformed static protocols by an average of 0.4 standard deviations (95 % confidence interval 0.2-0.6) of the primary outcome.

CONCLUSIONS

Adaptive protocols provide a simple one-to-one guide between adolescents' needs and recommended treatment which can be used as decision support for clinicians making LOC-placement decisions.

The Balanced Opioid Initiative: protocol for a clustered, sequential, multiple-assignment randomized trial to construct an adaptive implementation strategy to improve guideline-concordant opioid prescribing in primary care

BACKGROUND

Rates of opioid prescribing tripled in the USA between 1999 and 2015 and were associated with significant increases in opioid misuse and overdose death. Roughly half of all opioids are prescribed in primary care. Although clinical guidelines describe recommended opioid prescribing practices, implementing these guidelines in a way that balances safety and effectiveness vs. risk remains a challenge. The literature offers little help about which implementation strategies work best in different clinical settings or how strategies could be tailored to optimize their effectiveness in different contexts. Systems consultation consists of (1) educational/engagement meetings with audit and feedback reports, (2) practice facilitation, and (3) prescriber peer consulting. The study is designed to discover the most cost-effective sequence and combination of strategies for improving opioid prescribing practices in diverse primary care clinics.

METHODS/DESIGN

The study is a hybrid type 3 clustered, sequential, multiple-assignment randomized trial (SMART) that randomizes clinics from two health systems at two points, months 3 and 9, of a 21-month intervention. Clinics are provided one of four sequences of implementation strategies: a condition consisting of educational/engagement meetings and audit and feedback alone (EM/AF), EM/AF plus practice facilitation (PF), EM/AF + prescriber peer consulting (PPC), and EM/AF + PF + PPC. The study's primary outcome is morphine-milligram equivalent (MME) dose by prescribing clinicians within clinics. The study's primary aim is the comparison of EM/AF + PF + PPC versus EM/AF alone on change in MME from month 3 to month 21. The secondary aim is to derive cost estimates for each of the four sequences and compare them. The exploratory aim is to examine four tailoring variables that can be used to construct an adaptive implementation strategy to meet the needs of different primary care clinics.

DISCUSSION

Systems consultation is a practical blend of implementation strategies used in this case to improve opioid prescribing practices in primary care. The blend offers a range of strategies in sequences from minimally to substantially intensive. The results of this study promise to help us understand how to cost effectively improve the implementation of evidence-based practices.

TRIAL REGISTRATION

NCT04044521 (ClinicalTrials.gov). Registered 05 August 2019.

Sample size considerations for comparing dynamic treatment regimens in a sequential multiple-assignment randomized trial with a continuous longitudinal outcome

Clinicians and researchers alike are increasingly interested in how best to personalize interventions. A dynamic treatment regimen is a sequence of prespecified decision rules which can be used to guide the delivery of a sequence of treatments or interventions that is tailored to the changing needs of the individual. The sequential multiple-assignment randomized trial is a research tool which allows for the construction of effective dynamic treatment regimens. We derive easy-to-use formulae for computing the total sample size for three common two-stage sequential multiple-assignment randomized trial designs in which the primary aim is to compare mean end-of-study outcomes for two embedded dynamic treatment regimens which recommend different first-stage treatments. The formulae are derived in the context of a regression model which leverages information from a longitudinal outcome collected over the entire study. We show that the sample size formula for a sequential multiple-assignment randomized trial can be written as the product of the sample size formula for a standard two-arm randomized trial, a deflation factor that accounts for the increased statistical efficiency resulting from a longitudinal analysis, and an inflation factor that accounts for the design of a sequential multiple-assignment randomized trial. The sequential multiple-assignment randomized trial design inflation factor is typically a function of the anticipated probability of response to first-stage treatment. We review modeling and estimation for dynamic treatment regimen effect analyses using a longitudinal outcome from a sequential multiple-assignment randomized trial, as well as the estimation of standard errors. We also present estimators for the covariance matrix for a variety of common working correlation structures. Methods are motivated using the ENGAGE study, a sequential multiple-assignment randomized trial aimed at developing a dynamic treatment regimen for increasing motivation to attend treatments among alcohol- and cocaine-dependent patients.

A Data Analysis Method for Using Longitudinal Binary Outcome Data from a SMART to Compare Adaptive Interventions

Sequential multiple assignment randomized trials (SMARTs) are a useful and increasingly popular approach for gathering information to inform the construction of adaptive interventions to treat psychological and behavioral health conditions. Until recently, analysis methods for data from SMART designs considered only a single measurement of the outcome of interest when comparing the efficacy of adaptive interventions. Lu et al. proposed a method for considering repeated outcome measurements to incorporate information about the longitudinal trajectory of change. While their proposed method can be applied to many kinds of outcome variables, they focused mainly on linear models for normally distributed outcomes. Practical guidelines and extensions are required to implement this methodology with other types of repeated outcome measures common in behavioral research. In this article, we discuss implementation of this method with repeated binary outcomes. We explain how to compare adaptive interventions in terms of various summaries of repeated binary outcome measures, including average outcome (area under the curve) and delayed effects. The method is illustrated using an empirical example from a SMART study to develop an adaptive intervention for engaging alcohol- and cocaine-dependent patients in treatment. Monte Carlo simulations are provided to demonstrate the good performance of the proposed technique.

SMART longitudinal analysis: A tutorial for using repeated outcome measures from SMART studies to compare adaptive interventions

In recent years, there has been increased interest in the development of adaptive interventions across various domains of health and psychological research. An adaptive intervention is a protocolized sequence of individualized treatments that seeks to address the unique and changing needs of individuals as they progress through an intervention program. The sequential, multiple assignment, randomized trial (SMART) is an experimental study design that can be used to build the empirical basis for the construction of effective adaptive interventions. A SMART involves multiple stages of randomizations; each stage of randomization is designed to address scientific questions concerning the best intervention option to employ at that point in the intervention. Several adaptive interventions are embedded in a SMART by design; many SMARTs are motivated by scientific questions that concern the comparison of these embedded adaptive interventions. Until recently, analysis methods available for the comparison of adaptive interventions were limited to end-of-study outcomes. The current article provides an accessible and comprehensive tutorial to a new methodology for using repeated outcome data from SMART studies to compare adaptive interventions. We discuss how existing methods for comparing adaptive interventions in terms of end-of-study outcome data from a SMART can be extended for use with longitudinal outcome data. We also highlight the scientific utility of using longitudinal data from a SMART to compare adaptive interventions. A SMART study aiming to develop an adaptive intervention to engage alcohol- and cocaine-dependent individuals in treatment is used to demonstrate the application of this new methodology. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Change in Patient Outcomes After Augmenting a Low-level Implementation Strategy in Community Practices That Are Slow to Adopt a Collaborative Chronic Care Model: A Cluster Randomized Implementation Trial

BACKGROUND

Implementation strategies are essential for promoting the uptake of evidence-based practices and for patients to receive optimal care. Yet strategies differ substantially in their intensity and feasibility. Lower-intensity strategies (eg, training and technical support) are commonly used but may be insufficient for all clinics. Limited research has examined the comparative effectiveness of augmentations to low-level implementation strategies for nonresponding clinics.

OBJECTIVES

To compare 2 augmentation strategies for improving uptake of an evidence-based collaborative chronic care model (CCM) on 18-month outcomes for patients with depression at community-based clinics nonresponsive to lower-level implementation support.

RESEARCH DESIGN

Providers initially received support using a low-level implementation strategy, Replicating Effective Programs (REP). After 6 months, nonresponsive clinics were randomized to add either external facilitation (REP+EF) or external and internal facilitation (REP+EF/IF).

MEASURES

The primary outcome was patient 12-item short form survey (SF-12) mental health score at month 18. Secondary outcomes were patient health questionnaire (PHQ-9) depression score at month 18 and receipt of the CCM during months 6 through 18.

RESULTS

Twenty-seven clinics were nonresponsive after 6 months of REP. Thirteen clinics (N=77 patients) were randomized to REP+EF and 14 (N=92) to REP+EF/IF. At 18 months, patients in the REP+EF/IF arm had worse SF-12 [diff, 8.38; 95% confidence interval (CI), 3.59-13.18] and PHQ-9 scores (diff, 1.82; 95% CI, -0.14 to 3.79), and lower odds of CCM receipt (odds ratio, 0.67; 95% CI, 0.30-1.49) than REP+EF patients.

CONCLUSIONS

Patients at sites receiving the more intensive REP+EF/IF saw less improvement in mood symptoms at 18 months than those receiving REP+EF and were no more likely to receive the CCM. For community-based clinics, EF augmentation may be more feasible than EF/IF for implementing CCMs.

Piloting a Sequential, Multiple Assignment, Randomized Trial for Mothers with Attention-Deficit/Hyperactivity Disorder and Their At-Risk Young Children

Objective: Parental attention-deficit/hyperactivity disorder (ADHD) is associated with suboptimal parenting and reduces the effectiveness of child ADHD treatments. We conducted a Pilot Sequential, Multiple Assignment, Randomized Trial (SMART Pilot) to evaluate the feasibility and acceptability of sequencing medication and behavioral treatments for mothers with ADHD to target outcomes, including maternal ADHD, parenting, and child ADHD symptoms/impairment in multiplex ADHD families. Methods: Thirty-five mothers with ADHD and their 5- to 8-year-old child with ADHD symptoms were enrolled. Mothers were randomized to 8 weeks of individually titrated stimulant medication (MSM) or behavioral parent training (BPT), followed by rerandomization to 8 weeks of continued first-line treatment (with as-needed modifications) or combined treatment, leading to four treatment sequences (MSM-MSM, MSM-BPT, BPT-MSM, and BPT-BPT). Results: Recruitment of multiplex ADHD families came primarily from child providers. Mothers were adherent to medication and had high therapy session attendance. Mothers and clinicians found both treatments to be acceptable and preferred combination treatment, especially receiving medication before BPT. Monotherapy treatment visits were viewed as more burdensome (MSM-MSM, BPT-BPT). Conclusions: Maternal stimulant medication and BPT are acceptable and feasible interventions for families in which both the mother and child have ADHD symptoms. Mothers with concerns about their children's ADHD symptoms are receptive to receiving treatment themselves as an initial strategy for improving their children's health and functioning. Fully powered SMART designs show promise in evaluating the sequencing of interventions and helping clinicians develop algorithms for treating multiplex families in real-world practice settings.

Lung Cancer Screening and Smoking Cessation Clinical Trials. SCALE (Smoking Cessation within the Context of Lung Cancer Screening) Collaboration

National recommendations for lung cancer screening for former and current smokers aged 55-80 years with a 30-pack-year smoking history create demand to implement efficient and effective systems to offer smoking cessation on a large scale. These older, high-risk smokers differ from participants in past clinical trials of behavioral and pharmacologic interventions for tobacco dependence. There is a gap in knowledge about how best to design systems to extend reach and treatments to maximize smoking cessation in the context of lung cancer screening. Eight clinical trials, seven funded by the National Cancer Institute and one by the Veterans Health Administration, address this gap and form the SCALE (Smoking Cessation within the Context of Lung Cancer Screening) collaboration. This paper describes methodological issues related to the design of these clinical trials: clinical workflow, participant eligibility criteria, screening indication (baseline or annual repeat screen), assessment content, interest in stopping smoking, and treatment delivery method and dose, all of which will affect tobacco treatment outcomes. Tobacco interventions consider the "teachable moment" offered by lung cancer screening, how to incorporate positive and negative screening results, and coordination of smoking cessation treatment with clinical events associated with lung cancer screening. Unique data elements, such as perceived risk of lung cancer and costs of tobacco treatment, are of interest. Lung cancer screening presents a new and promising opportunity to reduce morbidity and mortality resulting from lung cancer that can be amplified by effective smoking cessation treatment. SCALE teamwork and collaboration promise to maximize knowledge gained from the clinical trials.

To Prompt or Not to Prompt? A Microrandomized Trial of Time-Varying Push Notifications to Increase Proximal Engagement With a Mobile Health App

Background

Mobile health (mHealth) apps provide an opportunity for easy, just-in-time access to health promotion and self-management support. However, poor user engagement with these apps remains a significant unresolved challenge.

Objective

This study aimed to assess the effect of sending versus not sending a push notification containing a contextually tailored health message on proximal engagement, measured here as self-monitoring via the app. Secondary aims were to examine whether this effect varies by the number of weeks enrolled in the program or by weekday versus weekend. An exploratory aim was to describe how the effect on proximal engagement differs between weekday versus weekend by the time of day.

Methods

The study analyzes the causal effects of push notifications on proximal engagement in 1255 users of a commercial workplace well-being intervention app over 89 days. The app employs a microrandomized trial (MRT) design to send push notifications. At 1 of 6 times per day (8:30 am, 12:30 pm, 5:30 pm, 6:30 pm, 7:30 pm, and 8:30 pm; selected randomly), available users were randomized with equal probability to be sent or not sent a push notification containing a tailored health message. The primary outcome of interest was whether the user self-monitored behaviors and feelings at some time during the next 24 hours via the app. A generalization of log-linear regression analysis, adapted for use with data arising from an MRT, was used to examine the effect of sending a push notification versus not sending a push notification on the probability of engagement over the next 24 hours.

Results

Users were estimated to be 3.9% more likely to engage with the app in the next 24 hours when a tailored health message was sent versus when it was not sent (risk ratio 1.039; 95% CI 1.01 to 1.08; P<.05). The effect of sending the message attenuated over the course of the study, but this effect was not statistically significant (P=.84). The effect of sending the message was greater on weekends than on weekdays, but the difference between these effects was not statistically significant (P=.18). When sent a tailored health message on weekends, the users were 8.7% more likely to engage with the app (95% CI 1.01 to 1.17), whereas on weekdays, the users were 2.5% more likely to engage with the app (95% CI 0.98 to 1.07). The effect of sending a tailored health message was greatest at 12:30 pm on weekends, when the users were 11.8% more likely to engage (90% CI 1.02 to 1.13).

Conclusions

Sending a push notification containing a tailored health message was associated with greater engagement in an mHealth app. Results suggested that users are more likely to engage with the app within 24 hours when push notifications are sent at mid-day on weekends.

Adaptive School-based Implementation of CBT (ASIC): clustered-SMART for building an optimized adaptive implementation intervention to improve uptake of mental health interventions in schools

Background

Depressive and anxiety disorders affect 20–30% of school-age youth, most of whom do not receive adequate services, contributing to poor developmental and academic outcomes. Evidence-based practices (EBPs) such as cognitive behavioral therapy (CBT) can improve outcomes, but numerous barriers limit access among affected youth. Many youth try to access mental health services in schools, but school professionals (SPs: counselors, psychologists, social workers) are rarely trained adequately in CBT methods. Further, SPs face organizational barriers to providing CBT, such as lack of administrative support. Three promising implementation strategies to address barriers to school-based CBT delivery include (1) Replicating Effective Programs (REP), which deploys customized CBT packaging, didactic training in CBT, and technical assistance; (2) coaching, which extends training via live supervision to improve SP competence in CBT delivery; and (3) facilitation, which employs an organizational expert who mentors SPs in strategic thinking to promote self-efficacy in garnering administrative support. REP is a relatively low-intensity/low-cost strategy, whereas coaching and facilitation require additional resources. However, not all schools will require all three strategies. The primary aim of this study is to compare the effectiveness of a school-level adaptive implementation intervention involving REP, coaching, and facilitation versus REP alone on the frequency of CBT delivered to students by SPs and student mental health outcomes. Secondary and exploratory aims examine cost-effectiveness, moderators, and mechanisms of implementation strategies.

Methods

Using a clustered, sequential multiple-assignment, randomized trial (SMART) design, ≥ 200 SPs from 100 schools across Michigan will be randomized initially to receive REP vs. REP+coaching. After 8 weeks, schools that do not meet a pre-specified implementation benchmark are re-randomized to continue with the initial strategy or to augment with facilitation.

Discussion

EBPs need to be implemented successfully and efficiently in settings where individuals are most likely to seek care in order to gain large-scale impact on public health. Adaptive implementation interventions hold the promise of providing cost-effective implementation support. This is the first study to test an adaptive implementation of CBT for school-age youth, at a statewide level, delivered by school staff, taking an EBP to large populations with limited mental health care access.

Trial registration

NCT03541317—Registered on 29 May 2018 on ClinicalTrials.gov PRS

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0808-8) contains supplementary material, which is available to authorized users.

TREE-BASED REINFORCEMENT LEARNING FOR ESTIMATING OPTIMAL DYNAMIC TREATMENT REGIMES

Dynamic treatment regimes (DTRs) are sequences of treatment decision rules, in which treatment may be adapted over time in response to the changing course of an individual. Motivated by the substance use disorder (SUD) study, we propose a tree-based reinforcement learning (T-RL) method to directly estimate optimal DTRs in a multi-stage multi-treatment setting. At each stage, T-RL builds an unsupervised decision tree that directly handles the problem of optimization with multiple treatment comparisons, through a purity measure constructed with augmented inverse probability weighted estimators. For the multiple stages, the algorithm is implemented recursively using backward induction. By combining semiparametric regression with flexible tree-based learning, T-RL is robust, efficient and easy to interpret for the identification of optimal DTRs, as shown in the simulation studies. With the proposed method, we identify dynamic SUD treatment regimes for adolescents.

Organizational culture and climate as moderators of enhanced outreach for persons with serious mental illness: results from a cluster-randomized trial of adaptive implementation strategies

Background

Organizational culture and climate are considered key factors in implementation efforts but have not been examined as moderators of implementation strategy comparative effectiveness. We investigated organizational culture and climate as moderators of comparative effectiveness of two sequences of implementation strategies (Immediate vs. Delayed Enhanced Replicating Effective Programs [REP]) combining Standard REP and REP enhanced with facilitation on implementation of an outreach program for Veterans with serious mental illness lost to care at Veterans Health Administration (VA) facilities nationwide.

Methods

This study is a secondary analysis of the cluster-randomized Re-Engage implementation trial that assigned 3075 patients at 89 VA facilities to either the Immediate or Delayed Enhanced REP sequences. We hypothesized that sites with stronger entrepreneurial culture, task, or relational climate would benefit more from Enhanced REP than Standard REP. Veteran- and site-level data from the Re-Engage trial were combined with site-aggregated measures of entrepreneurial culture and task and relational climate from the 2012 VA All Employee Survey. Longitudinal mixed-effects logistic models examined whether the comparative effectiveness of the Immediate vs. Delayed Enhanced REP sequences were moderated by culture or climate measures at 6 and 12 months post-randomization. Three Veteran-level outcomes related to the engagement with the VA system were assessed: updated documentation, attempted contact by coordinator, and completed contact.

Results

For updated documentation and attempted contact, Veterans at sites with higher entrepreneurial culture and task climate scores benefitted more from Enhanced REP compared to Standard REP than Veterans at sites with lower scores. Few culture or climate moderation effects were detected for the comparative effectiveness of the full sequences of implementation strategies.

Conclusions

Implementation strategy effectiveness is highly intertwined with contextual factors, and implementation practitioners may use knowledge of contextual moderation to tailor strategy deployment. We found that facilitation strategies provided with Enhanced REP were more effective at improving uptake of a mental health outreach program at sites with stronger entrepreneurial culture and task climate; Veterans at sites with lower levels of these measures saw more similar improvement under Standard and Enhanced REP. Within resource-constrained systems, practitioners may choose to target more intensive implementation strategies to sites that will most benefit from them.

Trial registration

ISRCTN: ISRCTN21059161. Date registered: April 11, 2013.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0787-9) contains supplementary material, which is available to authorized users.

Developing Optimized Adaptive Interventions in Education

Hedges (2018) encourages us to consider asking new scientific questions concerning the optimization of adaptive interventions in education. In this commentary, we have expanded on this (albeit briefly) by providing concrete examples of scientific questions and associated experimental designs to optimize adaptive interventions, and commenting on some of the ways such designs might challenge us to think differently. A great deal of methodological work remains to be done. For example, we have only begun to consider experimental design and analysis methods for developing "cluster-level adaptive interventions" (NeCamp, Kilbourne, & Almirall, 2017), or to extend methods for comparing the marginal mean trajectories between the adaptive interventions embedded in a SMART (Lu et al., 2016) to accommodate random effects. These methodological advances, among others, will propel educational research concerning the construction of more complex, yet meaningful, interventions that are necessary for improving student and teacher outcomes.

The Right Tool for the Job: Choosing Between Covariate-balancing and Generalized Boosted Model Propensity Scores

Estimating the causal effect of an exposure (vs. some control) on an outcome using observational data often requires addressing the fact that exposed and control groups differ on pre-exposure characteristics that may be related to the outcome (confounders). Propensity score methods have long been used as a tool for adjusting for observed confounders in order to produce more valid causal effect estimates under the strong ignorability assumption. In this article, we compare two promising propensity score estimation methods (for time-invariant binary exposures) when assessing the average treatment effect on the treated: the generalized boosted models and covariate-balancing propensity scores, with the main objective to provide analysts with some rules-of-thumb when choosing between these two methods. We compare the methods across different dimensions including the presence of extraneous variables, the complexity of the relationship between exposure or outcome and covariates, and the residual variance in outcome and exposure. We found that when noncomplex relationships exist between outcome or exposure and covariates, the covariate-balancing method outperformed the boosted method, but under complex relationships, the boosted method performed better. We lay out criteria for when one method should be expected to outperform the other with no blanket statement on whether one method is always better than the other.

Design and Analysis Considerations for Comparing Dynamic Treatment Regimens with Binary Outcomes from Sequential Multiple Assignment Randomized Trials

In behavioral, educational and medical practice, interventions are often personalized over time using strategies that are based on individual behaviors and characteristics and changes in symptoms, severity, or adherence that are a result of one's treatment. Such strategies that more closely mimic real practice, are known as dynamic treatment regimens (DTRs). A sequential multiple assignment randomized trial (SMART) is a multi-stage trial design that can be used to construct effective DTRs. This article reviews a simple to use 'weighted and replicated' estimation technique for comparing DTRs embedded in a SMART design using logistic regression for a binary, end-of-study outcome variable. Based on a Wald test that compares two embedded DTRs of interest from the 'weighted and replicated' regression model, a sample size calculation is presented with a corresponding user-friendly applet to aid in the process of designing a SMART. The analytic models and sample size calculations are presented for three of the more commonly used two-stage SMART designs. Simulations for the sample size calculation show the empirical power reaches expected levels. A data analysis example with corresponding code is presented in the appendix using data from a SMART developing an effective DTR in autism.

Estimating Moderated Causal Effects with Time-varying Treatments and Time-varying Moderators: Structural Nested Mean Models and Regression with Residuals

Individuals differ in how they respond to a particular treatment or exposure, and social scientists are often interested in understanding how treatment effects are moderated by observed characteristics of individuals. Effect moderation occurs when individual covariates dampen or amplify the effect of some exposure. This article focuses on estimating moderated causal effects in longitudinal settings where both the treatment and effect moderator vary over time. Effect moderation is typically examined using covariate by treatment interactions in regression analyses, but in the longitudinal setting, this approach may be problematic because time-varying moderators of future treatment may be affected by prior treatment-for example, moderators may also be mediators-and naively conditioning on an outcome of treatment in a conventional regression model can lead to bias. This article introduces to sociology moderated intermediate causal effects and the structural nested mean model for analyzing effect moderation in the longitudinal setting. It discusses problems with conventional regression and presents a new approach to estimation that avoids these problems (regression-with-residuals). The method is illustrated using longitudinal data from the PSID to examine whether the effects of time-varying exposures to poor neighborhoods on the risk of adolescent childbearing are moderated by time-varying family income.

Comparing cluster-level dynamic treatment regimens using sequential, multiple assignment, randomized trials: Regression estimation and sample size considerations

Cluster-level dynamic treatment regimens can be used to guide sequential treatment decision-making at the cluster level in order to improve outcomes at the individual or patient-level. In a cluster-level dynamic treatment regimen, the treatment is potentially adapted and re-adapted over time based on changes in the cluster that could be impacted by prior intervention, including aggregate measures of the individuals or patients that compose it. Cluster-randomized sequential multiple assignment randomized trials can be used to answer multiple open questions preventing scientists from developing high-quality cluster-level dynamic treatment regimens. In a cluster-randomized sequential multiple assignment randomized trial, sequential randomizations occur at the cluster level and outcomes are observed at the individual level. This manuscript makes two contributions to the design and analysis of cluster-randomized sequential multiple assignment randomized trials. First, a weighted least squares regression approach is proposed for comparing the mean of a patient-level outcome between the cluster-level dynamic treatment regimens embedded in a sequential multiple assignment randomized trial. The regression approach facilitates the use of baseline covariates which is often critical in the analysis of cluster-level trials. Second, sample size calculators are derived for two common cluster-randomized sequential multiple assignment randomized trial designs for use when the primary aim is a between-dynamic treatment regimen comparison of the mean of a continuous patient-level outcome. The methods are motivated by the Adaptive Implementation of Effective Programs Trial which is, to our knowledge, the first-ever cluster-randomized sequential multiple assignment randomized trial in psychiatry.

A SMART data analysis method for constructing adaptive treatment strategies for substance use disorders

AIMS

To demonstrate how Q-learning, a novel data analysis method, can be used with data from a sequential, multiple assignment, randomized trial (SMART) to construct empirically an adaptive treatment strategy (ATS) that is more tailored than the ATSs already embedded in a SMART.

METHOD

We use Q-learning with data from the Extending Treatment Effectiveness of Naltrexone (ExTENd) SMART (N = 250) to construct empirically an ATS employing naltrexone, behavioral intervention, and telephone disease management to reduce alcohol consumption over 24 weeks in alcohol dependent individuals.

RESULTS

Q-learning helped to identify a subset of individuals who, despite showing early signs of response to naltrexone, require additional treatment to maintain progress.

CONCLUSIONS

Q-learning can inform the development of more cost-effective, adaptive treatment strategies for treating substance use disorders.

Assessing Time-Varying Causal Effect Moderation in Mobile Health

In mobile health interventions aimed at behavior change and maintenance, treatments are provided in real time to manage current or impending high risk situations or promote healthy behaviors in near real time. Currently there is great scientific interest in developing data analysis approaches to guide the development of mobile interventions. In particular data from mobile health studies might be used to examine effect moderators-individual characteristics, time-varying context or past treatment response that moderate the effect of current treatment on a subsequent response. This paper introduces a formal definition for moderated effects in terms of potential outcomes, a definition that is particularly suited to mobile interventions, where treatment occasions are numerous, individuals are not always available for treatment, and potential moderators might be influenced by past treatment. Methods for estimating moderated effects are developed and compared. The proposed approach is illustrated using BASICS-Mobile, a smartphone-based intervention designed to curb heavy drinking and smoking among college students.

Chasing balance and other recommendations for improving nonparametric propensity score models

In this article, we carefully examine two important implementation issues when estimating propensity scores using generalized boosted models (GBM), a promising machine learning technique. First, we examine which of the following methods for tuning GBM lead to better covariate balance and inferences about causal effects: pursuing covariate balance between the treatment groups or tuning the propensity score model on the basis of a model fit criterion. Second, we examine how well GBM can handle irrelevant covariates that are included in the estimation model. We find that chasing balance rather than model fit when estimating propensity scores yielded better covariate balance and more accurate treatment effect estimates. Additionally, we find that adding irrelevant covariates to GBM increased imbalance and bias in the treatment effects. The findings from this paper have useful implications for other work focused on improving methods for estimating propensity scores.

Improving Physical Health in Patients With Chronic Mental Disorders: Twelve-Month Results From a Randomized Controlled Collaborative Care Trial

OBJECTIVE

Persons with chronic mental disorders are disproportionately burdened with physical health conditions. We determined whether Life Goals Collaborative Care compared to usual care improves physical health in patients with mental disorders within 12 months.

METHODS

This single-blind randomized controlled effectiveness study of a collaborative care model was conducted at a midwestern Veterans Affairs urban outpatient mental health clinic. Patients (N = 293 out of 474 eligible approached) with an ICD-9-CM diagnosis of schizophrenia, bipolar disorder, or major depressive disorder and at least 1 cardiovascular disease risk factor provided informed consent and were randomized (February 24, 2010, to April 29, 2015) to Life Goals (n = 146) or usual care (n = 147). A total of 287 completed baseline assessments, and 245 completed 12-month follow-up assessments. Life Goals included 5 weekly sessions that provided semistructured guidance on managing physical and mental health symptoms through healthy behavior changes, augmented by ongoing care coordination. The primary outcome was change in physical health-related quality of life score (Veterans RAND 12-item Short Form Health Survey [VR-12] physical health component score). Secondary outcomes included control of cardiovascular risk factors from baseline to 12 months (blood pressure, lipids, weight), mental health-related quality of life, and mental health symptoms.

RESULTS

Among patients completing baseline and 12-month outcomes assessments (N = 245), the mean age was 55.3 years (SD = 10.8; range, 25-78 years), and 15.4% were female. Intent-to-treat analysis revealed that compared to those in usual care, patients randomized to Life Goals had slightly increased VR-12 physical health scores (coefficient = 3.21; P = .01).

CONCLUSIONS

Patients with chronic mental disorders and cardiovascular disease risk who received Life Goals had improved physical health-related quality of life.

TRIAL REGISTRATION

ClinicalTrials.gov identifiers: NCT01487668 and NCT01244854.

Microrandomized trials: An experimental design for developing just-in-time adaptive interventions

OBJECTIVE

This article presents an experimental design, the microrandomized trial, developed to support optimization of just-in-time adaptive interventions (JITAIs). JITAIs are mHealth technologies that aim to deliver the right intervention components at the right times and locations to optimally support individuals' health behaviors. Microrandomized trials offer a way to optimize such interventions by enabling modeling of causal effects and time-varying effect moderation for individual intervention components within a JITAI.

METHOD

The article describes the microrandomized trial design, enumerates research questions that this experimental design can help answer, and provides an overview of the data analyses that can be used to assess the causal effects of studied intervention components and investigate time-varying moderation of those effects.

RESULTS

Microrandomized trials enable causal modeling of proximal effects of the randomized intervention components and assessment of time-varying moderation of those effects.

CONCLUSION

Microrandomized trials can help researchers understand whether their interventions are having intended effects, when and for whom they are effective, and what factors moderate the interventions' effects, enabling creation of more effective JITAIs.

A sample size calculator for SMART pilot studies

In clinical practice, as well as in other areas where interventions are provided, a sequential individualized approach to treatment is often necessary, whereby each treatment is adapted based on the object's response. An adaptive intervention is a sequence of decision rules which formalizes the provision of treatment at critical decision points in the care of an individual. In order to inform the development of an adaptive intervention, scientists are increasingly interested in the use of sequential multiple assignment randomized trials (SMART), which is a type of multi-stage randomized trial where individuals are randomized repeatedly at critical decision points to a set treatment options. While there is great interest in the use of SMART and in the development of adaptive interventions, both are relatively new to the medical and behavioral sciences. As a result, many clinical researchers will first implement a SMART pilot study (i.e., a small-scale version of a SMART) to examine feasibility and acceptability considerations prior to conducting a full-scale SMART study. A primary aim of this paper is to introduce a new methodology to calculate minimal sample size necessary for conducting a SMART pilot.

Adaptive Interventions in Child and Adolescent Mental Health

The treatment or prevention of child and adolescent mental health (CAMH) disorders often requires an individualized, sequential approach to intervention, whereby treatments (or prevention efforts) are adapted over time based on the youth's evolving status (e.g., early response, adherence). Adaptive interventions are intended to provide a replicable guide for the provision of individualized sequences of interventions in actual clinical practice. Recently, there has been great interest in the development of adaptive intervenions by investigators working in CAMH. The development of such replicable, real-world, individualized sequences of decision rules to guide the treatment or prevention of CAMH disorders represents an important "next step" in interventions research. The primary purpose of this special issue is to showcase some recent work on the science of adaptive interventions in CAMH. In this overview article, we review why individualized sequences of interventions are needed in CAMH, provide an introduction to adaptive interventions, briefly describe each of the articles included in this special issue, and describe some exciting areas of ongoing and future research. A hopeful outcome of this special issue is that it encourages other researchers in CAMH to pursue creative and significant research on adaptive interventions.

Comparing dynamic treatment regimes using repeated-measures outcomes: modeling considerations in SMART studies

A dynamic treatment regime (DTR) is a sequence of decision rules, each of which recommends a treatment based on a patient's past and current health status. Sequential, multiple assignment, randomized trials (SMARTs) are multi-stage trial designs that yield data specifically for building effective DTRs. Modeling the marginal mean trajectories of a repeated-measures outcome arising from a SMART presents challenges, because traditional longitudinal models used for randomized clinical trials do not take into account the unique design features of SMART. We discuss modeling considerations for various forms of SMART designs, emphasizing the importance of considering the timing of repeated measures in relation to the treatment stages in a SMART. For illustration, we use data from three SMART case studies with increasing level of complexity, in autism, child attention deficit hyperactivity disorder, and adult alcoholism. In all three SMARTs, we illustrate how to accommodate the design features along with the timing of the repeated measures when comparing DTRs based on mean trajectories of the repeated-measures outcome.

Longitudinal Effects of Adaptive Interventions With a Speech-Generating Device in Minimally Verbal Children With ASD

There are limited data on the effects of adaptive social communication interventions with a speech-generating device in autism. This study is the first to compare growth in communications outcomes among three adaptive interventions in school-age children with autism spectrum disorder (ASD) who are minimally verbal. Sixty-one children, ages 5-8 years, participated in a sequential, multiple-assignment randomized trial (SMART). All children received a developmental behavioral communication intervention: joint attention, symbolic play, engagement and regulation (JASP) with enhanced milieu teaching (EMT). The SMART included three 2-stage, 24-week adaptive interventions with different provisions of a speech-generating device (SGD) in the context of JASP+EMT. The first adaptive intervention, with no SGD, initially assigned JASP+EMT alone, then intensified JASP+EMT for slow responders. In the second adaptive intervention, slow responders to JASP+EMT were assigned JASP+EMT+SGD. The third adaptive intervention initially assigned JASP+EMT+SGD; then intensified JASP+EMT+SGD for slow responders. Analyses examined between-group differences in change in outcomes from baseline to Week 36. Verbal outcomes included spontaneous communicative utterances and novel words. Nonlinguistic communication outcomes included initiating joint attention and behavior regulation, and play. The adaptive intervention beginning with JASP+EMT+SGD was estimated as superior. There were significant (p < .05) between-group differences in change in spontaneous communicative utterances and initiating joint attention. School-age children with ASD who are minimally verbal make significant gains in communication outcomes with an adaptive intervention beginning with JASP+EMT+SGD. Future research should explore mediators and moderators of the adaptive intervention effects and second-stage intervention options that further capitalize on early gains in treatment.

Personalized Treatment of Mothers With ADHD and Their Young At-Risk Children: A SMART Pilot

Young children of mothers with adult attention-deficit/hyperactivity disorder (ADHD) are at risk for ADHD by virtue of genetics and environmental factors. Moreover, parent ADHD is associated with maladaptive parenting and poor child behavioral treatment response. Thus, a combined approach consisting of behavioral parent training (BPT) and maternal stimulant medication (MSM) may be needed to effectively treat ADHD within families. However, providing combined BPT+MSM initially to all families may be unnecessarily burdensome because not all families likely need combined treatment. The purpose of this study is to examine how to combine, sequence, and personalize treatment for these multiplex families in order to yield benefits to both the parent and child, thereby impacting the course of child ADHD and disruptive behavior symptoms. This article presents our rationale for, design of, and preliminary experiences (based on 26 participants) with an ongoing pilot Sequential Multiple Assessment Randomized Trial (SMART) designed to answer questions regarding the feasibility and acceptability of study protocols and interventions. This article also describes how the subsequent full-scale SMART might change based on what is learned in the SMART pilot and illustrates how the full-scale SMART could be used to inform clinical decision making about how to combine, sequence, and personalize treatment for complex children and families in which a parent has ADHD.

A Pilot SMART for Developing an Adaptive Treatment Strategy for Adolescent Depression

This pilot study was conducted to assess the feasibility and acceptability of 4 adaptive treatment strategies (ATSs) for adolescent depression to plan for a subsequent full-scale clinical trial. The ATSs aim to address 2 questions that arise when personalizing treatment: (a) For adolescents treated with Interpersonal Psychotherapy for depressed adolescents (IPT-A; Mufson et al., 2004 ), at what time point should therapists make the determination that the adolescent is not likely to respond if the initial treatment plan is continued (week 4 or week 8)? (b) For adolescents who are judged to need their treatment augmented, should the therapist increase the number of IPT-A sessions or add pharmacotherapy (fluoxetine)? A 16-week pilot sequential multiple assignment randomized trial (SMART) was conducted with 32 adolescents (M age = 14.9) who had a diagnosis of major depressive disorder, dysthymic disorder, or depressive disorder not otherwise specified. Adolescents were primarily female (75%) and Caucasian (84.4%). Data regarding the feasibility and acceptability of the study and treatment procedures and treatment response rates were collected. Week 4 was the more feasible and acceptable decision point for assessing need for a change to treatment. Adolescents, parents, and therapists reported a range of attitudes about medication and more intensive therapy as treatment options. Results from the pilot study have yielded additional research questions for the full-scale SMART and will improve our ability to successfully conduct the trial.

Reengaging veterans with serious mental illness into care: preliminary results from a national randomized trial

OBJECTIVE

This study compared effectiveness of an enhanced versus standard implementation strategy (Replicating Effective Programs [REP]) on site-level uptake of Re-Engage, a national program for veterans with serious mental illness.

METHODS

Mental health providers at 158 Veterans Affairs (VA) facilities were given REP-based manuals and training in Re-Engage, which involved identifying veterans who had not been seen in VA care for at least one year, documenting their clinical status, and coordinating further health care. After six months, facilities not responding to REP (N=88) were randomized to receive six months of facilitation (enhanced REP) or continued standard REP. Site-level uptake was defined as percentage of patients (N=1,531) with updated documentation or with whom contact was attempted.

RESULTS

Rate of Re-Engage uptake was greater for enhanced REP sites compared with standard REP sites (41% versus 31%, p=.01). Total REP facilitation time was 7.3 hours per site for six months.

CONCLUSIONS

Added facilitation improved short-term uptake of a national mental health program.

Enhancing outreach for persons with serious mental illness: 12-month results from a cluster randomized trial of an adaptive implementation strategy

Background

Few implementation strategies have been empirically tested for their effectiveness in improving uptake of evidence-based treatments or programs. This study compared the effectiveness of an immediate versus delayed enhanced implementation strategy (Enhanced Replicating Effective Programs (REP)) for providers at Veterans Health Administration (VA) outpatient facilities (sites) on improved uptake of an outreach program (Re-Engage) among sites not initially responding to a standard implementation strategy.

Methods

One mental health provider from each U.S. VA site (N = 158) was initially given a REP-based package and training program in Re-Engage. The Re-Engage program involved giving each site provider a list of patients with serious mental illness who had not been seen at their facility for at least a year, requesting that providers contact these patients, assessing patient clinical status, and where appropriate, facilitating appointments to VA health services. At month 6, sites considered non-responsive (N = 89, total of 3,075 patients), defined as providers updating documentation for less than <80% of patients on their list, were randomized to two adaptive implementation interventions: Enhanced REP (provider coaching; N = 40 sites) for 6 months followed by Standard REP for 6 months; versus continued Standard REP (N = 49 sites) for 6 months followed by 6 months of Enhanced REP for sites still not responding. Outcomes included patient-level Re-Engage implementation and utilization.

Results

Patients from sites that were randomized to receive Enhanced REP immediately compared to Standard REP were more likely to have a completed contact (adjusted OR = 2.13; 95% CI: 1.09–4.19, P = 0.02). There were no differences in patient-level utilization between Enhanced and Standard REP sites.

Conclusions

Enhanced REP was associated with greater Re-Engage program uptake (completed contacts) among sites not responding to a standard implementation strategy. Further research is needed to determine whether national implementation of Facilitation results in tangible changes in patient-level outcomes.

Trial registration

ISRCTN: ISRCTN21059161

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-014-0163-3) contains supplementary material, which is available to authorized users.

Trajectories of change in youth anxiety during cognitive-behavior therapy

OBJECTIVE

To evaluate changes in the trajectory of youth anxiety following the introduction of specific cognitive-behavior therapy (CBT) components: relaxation training, cognitive restructuring, and exposure tasks.

METHOD

Four hundred eighty-eight youths ages 7-17 years (50% female; 74% ≤ 12 years) were randomly assigned to receive either CBT, sertraline (SRT), their combination (COMB), or pill placebo (PBO) as part of their participation in the Child/Adolescent Anxiety Multimodal Study (CAMS). Youths in the CBT conditions were evaluated weekly by therapists using the Clinical Global Impression Scale-Severity (CGI-S; Guy, 1976) and the Children's Global Assessment Scale (CGAS; Shaffer et al., 1983) and every 4 weeks by blind independent evaluators (IEs) using the Pediatric Anxiety Ratings Scale (PARS; RUPP Anxiety Study Group, 2002). Youths in SRT and PBO were included as controls.

RESULTS

Longitudinal discontinuity analyses indicated that the introduction of both cognitive restructuring (e.g., changing self-talk) and exposure tasks significantly accelerated the rate of progress on measures of symptom severity and global functioning moving forward in treatment; the introduction of relaxation training had limited impact. Counter to expectations, no strategy altered the rate of progress in the specific domain of anxiety that it was intended to target (i.e., somatic symptoms, anxious self-talk, avoidance behavior).

CONCLUSIONS

Findings support CBT theory and suggest that cognitive restructuring and exposure tasks each make substantial contributions to improvement in youth anxiety. Implications for future research are discussed. (PsycINFO Database Record

Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program

BACKGROUND

Despite the availability of psychosocial evidence-based practices (EBPs), treatment and outcomes for persons with mental disorders remain suboptimal. Replicating Effective Programs (REP), an effective implementation strategy, still resulted in less than half of sites using an EBP. The primary aim of this cluster randomized trial is to determine, among sites not initially responding to REP, the effect of adaptive implementation strategies that begin with an External Facilitator (EF) or with an External Facilitator plus an Internal Facilitator (IF) on improved EBP use and patient outcomes in 12 months.

METHODS/DESIGN

This study employs a sequential multiple assignment randomized trial (SMART) design to build an adaptive implementation strategy. The EBP to be implemented is life goals (LG) for patients with mood disorders across 80 community-based outpatient clinics (N = 1,600 patients) from different U.S. regions. Sites not initially responding to REP (defined as < 50% patients receiving ≥ 3 EBP sessions) will be randomized to receive additional support from an EF or both EF/IF. Additionally, sites randomized to EF and still not responsive will be randomized to continue with EF alone or to receive EF/IF. The EF provides technical expertise in adapting LG in routine practice, whereas the on-site IF has direct reporting relationships to site leadership to support LG use in routine practice. The primary outcome is mental health-related quality of life; secondary outcomes include receipt of LG sessions, mood symptoms, implementation costs, and organizational change.

DISCUSSION

This study design will determine whether an off-site EF alone versus the addition of an on-site IF improves EBP uptake and patient outcomes among sites that do not respond initially to REP. It will also examine the value of delaying the provision of EF/IF for sites that continue to not respond despite EF.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT02151331.

Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research

The management of many health disorders often entails a sequential, individualized approach whereby treatment is adapted and readapted over time in response to the specific needs and evolving status of the individual. Adaptive interventions provide one way to operationalize the strategies (e.g., continue, augment, switch, step-down) leading to individualized sequences of treatment. Often, a wide variety of critical questions must be answered when developing a high-quality adaptive intervention. Yet, there is often insufficient empirical evidence or theoretical basis to address these questions. The Sequential Multiple Assignment Randomized Trial (SMART)-a type of research design-was developed explicitly for the purpose of building optimal adaptive interventions by providing answers to such questions. Despite increasing popularity, SMARTs remain relatively new to intervention scientists. This manuscript provides an introduction to adaptive interventions and SMARTs. We discuss SMART design considerations, including common primary and secondary aims. For illustration, we discuss the development of an adaptive intervention for optimizing weight loss among adult individuals who are overweight.