Enhanced fidelity to treatment for bipolar disorder: results from a randomized controlled implementation trial

Effect of a needs-based model of care on the characteristics of healthcare services in England: the i-THRIVE National Implementation Programme

AIMS

Developing integrated mental health services focused on the needs of children and young people is a key policy goal in England. The THRIVE Framework and its implementation programme, i-THRIVE, are widely used in England. This study examines experiences of staff using i-THRIVE, estimates its effectiveness, and assesses how local system working relationships influence programme success.

METHODS

This evaluation uses a quasi-experimental design (10 implementation and 10 comparison sites.) Measurements included staff surveys and assessment of 'THRIVE-like' features of each site. Additional site-level characteristics were collected from health system reports. The effect of i-THRIVE was evaluated using a four-group propensity-score-weighted difference-in-differences model; the moderating effect of system working relationships was evaluated with a difference-in-difference-in-differences model.

RESULTS

Implementation site staff were more likely to report using THRIVE and more knowledgeable of THRIVE principles than comparison site staff. The mean improvement of fidelity scores among i-THRIVE sites was 16.7, and 8.8 among comparison sites; the weighted model did not find a statistically significant difference. However, results show that strong working relationships in the local system significantly enhance the effectiveness of i-THRIVE. Sites with highly effective working relationships showed a notable improvement in 'THRIVE-like' features, with an average increase of 16.41 points (95% confidence interval: 1.69-31.13, P-value: 0.031) over comparison sites. Sites with ineffective working relationships did not benefit from i-THRIVE (-2.76, 95% confidence interval: - 18.25-12.73, P-value: 0.708).

CONCLUSIONS

The findings underscore the importance of working relationship effectiveness in the successful adoption and implementation of multi-agency health policies like i-THRIVE.

Collaborative care approaches for people with severe mental illness

BACKGROUND

Collaborative care for severe mental illness (SMI) is a community-based intervention that promotes interdisciplinary working across primary and secondary care. Collaborative care interventions aim to improve the physical and/or mental health care of individuals with SMI. This is an update of a 2013 Cochrane review, based on new searches of the literature, which includes an additional seven studies.

OBJECTIVES

To assess the effectiveness of collaborative care approaches in comparison with standard care (or other non-collaborative care interventions) for people with diagnoses of SMI who are living in the community.

SEARCH METHODS

We searched the Cochrane Schizophrenia Study-Based Register of Trials (10 February 2021). We searched the Cochrane Common Mental Disorders (CCMD) controlled trials register (all available years to 6 June 2016). Subsequent searches on Ovid MEDLINE, Embase and PsycINFO together with the Cochrane Central Register of Controlled Trials (with an overlap) were run on 17 December 2021.

SELECTION CRITERIA

Randomised controlled trials (RCTs) where interventions described as 'collaborative care' were compared with 'standard care' for adults (18+ years) living in the community with a diagnosis of SMI. SMI was defined as schizophrenia, other types of schizophrenia-like psychosis or bipolar affective disorder. The primary outcomes of interest were: quality of life, mental state and psychiatric admissions at 12 months follow-up.

DATA COLLECTION AND ANALYSIS

Pairs of authors independently extracted data. We assessed the quality and certainty of the evidence using RoB 2 (for the primary outcomes) and GRADE. We compared treatment effects between collaborative care and standard care. We divided outcomes into short-term (up to six months), medium-term (seven to 12 months) and long-term (over 12 months). For dichotomous data we calculated the risk ratio (RR) and for continuous data we calculated the standardised mean difference (SMD), with 95% confidence intervals (CIs). We used random-effects meta-analyses due to substantial levels of heterogeneity across trials. We created a summary of findings table using GRADEpro.

MAIN RESULTS

Eight RCTs (1165 participants) are included in this review. Two met the criteria for type A collaborative care (intervention comprised of the four core components). The remaining six met the criteria for type B (described as collaborative care by the trialists, but not comprised of the four core components). The composition and purpose of the interventions varied across studies. For most outcomes there was low- or very low-certainty evidence. We found three studies that assessed the quality of life of participants at 12 months. Quality of life was measured using the SF-12 and the WHOQOL-BREF and the mean endpoint mental health component scores were reported at 12 months. Very low-certainty evidence did not show a difference in quality of life (mental health domain) between collaborative care and standard care in the medium term (at 12 months) (SMD 0.03, 95% CI -0.26 to 0.32; 3 RCTs, 227 participants). Very low-certainty evidence did not show a difference in quality of life (physical health domain) between collaborative care and standard care in the medium term (at 12 months) (SMD 0.08, 95% CI -0.18 to 0.33; 3 RCTs, 237 participants). Furthermore, in the medium term (at 12 months) low-certainty evidence did not show a difference between collaborative care and standard care in mental state (binary) (RR 0.99, 95% CI 0.77 to 1.28; 1 RCT, 253 participants) or in the risk of being admitted to a psychiatric hospital at 12 months (RR 5.15, 95% CI 0.67 to 39.57; 1 RCT, 253 participants). One study indicated an improvement in disability (proxy for social functioning) at 12 months in the collaborative care arm compared to usual care (RR 1.38, 95% CI 0.97 to 1.95; 1 RCT, 253 participants); we deemed this low-certainty evidence. Personal recovery and satisfaction/experience of care outcomes were not reported in any of the included studies. The data from one study indicated that the collaborative care treatment was more expensive than standard care (mean difference (MD) international dollars (Int$) 493.00, 95% CI 345.41 to 640.59) in the short term. Another study found the collaborative care intervention to be slightly less expensive at three years.

AUTHORS' CONCLUSIONS

This review does not provide evidence to indicate that collaborative care is more effective than standard care in the medium term (at 12 months) in relation to our primary outcomes (quality of life, mental state and psychiatric admissions). The evidence would be improved by better reporting, higher-quality RCTs and the assessment of underlying mechanisms of collaborative care. We advise caution in utilising the information in this review to assess the effectiveness of collaborative care.

Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication

The promise of adaptation and adaptive designs in implementation science has been hindered by the lack of clarity and precision in defining what it means to adapt, especially regarding the distinction between adaptive study designs and adaptive implementation strategies. To ensure a common language for science and practice, authors reviewed the implementation science literature and found that the term adaptive was used to describe interventions, implementation strategies, and trial designs. To provide clarity and offer recommendations for reporting and strengthening study design, we propose a taxonomy that describes fixed versus adaptive implementation strategies and implementation trial designs. To improve impact, (a) futureimplementation studies should prespecify implementation strategy core functions that in turn can be taught to and replicated by health system/community partners, (b) funders should support exploratory studies that refine and specify implementation strategies, and (c) investigators should systematically address design requirements and ethical considerations (e.g., randomization, blinding/masking) with health system/community partners.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 3: study protocol to evaluate sustainment in a hybrid type 2 effectiveness-implementation cluster-randomized trial

BACKGROUND

Although research on the implementation of evidence-based psychological treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase-the Sustainment Phase-of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial-the Implementation Phase and Train-the-Trainer Phase-TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes.

METHODS

Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data.

DISCUSSION

The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT05956678 . Registered on July 21, 2023.

Calculating power for multilevel implementation trials in mental health: Meaningful effect sizes, intraclass correlation coefficients, and proportions of variance explained by covariates

Background

Despite the ubiquity of multilevel sampling, design, and analysis in mental health implementation trials, few resources are available that provide reference values of design parameters (e.g., effect size, intraclass correlation coefficient [ICC], and proportion of variance explained by covariates [covariate R 2]) needed to accurately determine sample size. The aim of this study was to provide empirical reference values for these parameters by aggregating data on implementation and clinical outcomes from multilevel implementation trials, including cluster randomized trials and individually randomized repeated measures trials, in mental health. The compendium of design parameters presented here represents plausible values that implementation scientists can use to guide sample size calculations for future trials.

Method

We searched NIH RePORTER for all federally funded, multilevel implementation trials addressing mental health populations and settings from 2010 to 2020. For all continuous and binary implementation and clinical outcomes included in eligible trials, we generated values of effect size, ICC, and covariate R2 at each level via secondary analysis of trial data or via extraction of estimates from analyses in published research reports. Effect sizes were calculated as Cohen d; ICCs were generated via one-way random effects ANOVAs; covariate R2 estimates were calculated using the reduction in variance approach.

Results

Seventeen trials were eligible, reporting on 53 implementation and clinical outcomes and 81 contrasts between implementation conditions. Tables of effect size, ICC, and covariate R2 are provided to guide implementation researchers in power analyses for designing multilevel implementation trials in mental health settings, including two- and three-level cluster randomized designs and unit-randomized repeated-measures designs.

Conclusions

Researchers can use the empirical reference values reported in this study to develop meaningful sample size determinations for multilevel implementation trials in mental health. Discussion focuses on the application of the reference values reported in this study.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 3: Study protocol to evaluate sustainment in a hybrid type 2 effectiveness-implementation cluster-randomized trial

treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase - the Sustainment Phase - of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial - the Implementation Phase and Train-the-Trainer Phase - TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes.

Methods

Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data.

Discussion

The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change.

Trial Registration

ClinicalTrials.gov identifier: NCT05956678. Registered on July 21, 2023. https://classic.clinicaltrials.gov/ct2/show/NCT05956678?term=NCT05956678&draw=2&rank=1.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 1: study protocol for a hybrid type 2 effectiveness-implementation cluster-randomized trial

BACKGROUND

Serious mental illness (SMI) can have devastating consequences. Unfortunately, many patients with SMI do not receive evidence-based psychological treatment (EBPTs) in routine practice settings. One barrier is poor "fit" between EBPTs and contexts in which they are implemented. The present study will evaluate implementation and effectiveness outcomes of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) implemented in community mental health centers (CMHCs). TranS-C was designed to target a range of SMI diagnoses by addressing a probable mechanism and predictor of SMI: sleep and circadian problems. We will investigate whether adapting TranS-C to fit CMHC contexts improves providers' perceptions of fit and patient outcomes.

METHODS

TranS-C will be implemented in at least ten counties in California, USA (N = 96 providers; N = 576 clients), via facilitation. CMHC sites are cluster-randomized by county to Adapted TranS-C or Standard TranS-C. Within each county, patients are randomized to immediate TranS-C or usual care followed by delayed treatment with TranS-C (UC-DT). Aim 1 will compare TranS-C (combined Adapted and Standard) with UC-DT on improvements in sleep and circadian problems, functional impairment, and psychiatric symptoms. Sleep and circadian problems will also be tested as a mediator between treatment condition (combined TranS-C versus UC-DT) and functional impairment/psychiatric symptoms. Aim 2 will evaluate whether Adapted TranS-C is superior to Standard TranS-C with respect to provider perceptions of fit. Aim 3 will evaluate whether the relation between TranS-C treatment condition (Adapted versus Standard) and patient outcomes is mediated by better provider perceptions of fit in the Adapted condition. Exploratory analyses will (1) compare Adapted versus Standard TranS-C on patient perceptions of credibility/improvement and select PhenX Toolkit outcomes and (2) evaluate possible moderators.

DISCUSSION

This trial has the potential to (a) expand support for TranS-C, a promising transdiagnostic treatment delivered to patients with SMI in CMHCs; (b) take steps toward addressing challenges faced by providers in delivering EBPTs (i.e., high caseloads, complex patients, poor fit); and (c) advance evidence on causal strategies (i.e., adapting treatments to fit context) in implementation science.

TRIAL REGISTRATION

Clinicaltrials.gov NCT04154631. Registered on 6 November 2019. https://clinicaltrials.gov/ct2/show/NCT04154631.

Primary aim results of a clustered SMART for developing a school-level, adaptive implementation strategy to support CBT delivery at high schools in Michigan

BACKGROUND

Schools increasingly provide mental health services to students, but often lack access to implementation strategies to support school-based (and school professional [SP]) delivery of evidence-based practices. Given substantial heterogeneity in implementation barriers across schools, development of adaptive implementation strategies that guide which implementation strategies to provide to which schools and when may be necessary to support scale-up.

METHODS

A clustered, sequential, multiple-assignment randomized trial (SMART) of high schools across Michigan was used to inform the development of a school-level adaptive implementation strategy for supporting SP-delivered cognitive behavioral therapy (CBT). All schools were first provided with implementation support informed by Replicating Effective Programs (REP) and then were randomized to add in-person Coaching or not (phase 1). After 8 weeks, schools were assessed for response based on SP-reported frequency of CBT delivered to students and/or barriers reported. Responder schools continued with phase 1 implementation strategies. Slower-responder schools (not providing ≥ 3 CBT components to ≥10 students or >2 organizational barriers identified) were re-randomized to add Facilitation to current support or not (phase 2). The primary aim hypothesis was that SPs at schools receiving the REP + Coaching + Facilitation adaptive implementation strategy would deliver more CBT sessions than SPs at schools receiving REP alone. Secondary aims compared four implementation strategies (Coaching vs no Coaching × Facilitation vs no Facilitation) on CBT sessions delivered, including by type (group, brief and full individual). Analyses used a marginal, weighted least squares approach developed for clustered SMARTs.

RESULTS

SPs (n = 169) at 94 high schools entered the study. N = 83 schools (88%) were slower-responders after phase 1. Contrary to the primary aim hypothesis, there was no evidence of a significant difference in CBT sessions delivered between REP + Coaching + Facilitation and REP alone (111.4 vs. 121.1 average total CBT sessions; p = 0.63). In secondary analyses, the adaptive strategy that offered REP + Facilitation resulted in the highest average CBT delivery (154.1 sessions) and the non-adaptive strategy offering REP + Coaching the lowest (94.5 sessions).

CONCLUSIONS

The most effective strategy in terms of average SP-reported CBT delivery is the adaptive implementation strategy that (i) begins with REP, (ii) augments with Facilitation for slower-responder schools (schools where SPs identified organizational barriers or struggled to deliver CBT), and (iii) stays the course with REP for responder schools.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03541317 , May 30, 2018.

Development of an Advance Care Planning Policy within an Evidenced-Based Evaluation Framework

Background: As the population is aging and medical advancements enable people to live longer, advance care planning (ACP) becomes increasingly important in guiding future care decisions; however, they are often incomplete or absent from the patient chart. This study describes the development and implementation of an ACP policy in a post-acute care and long-term care setting using a systematic implementation framework. Methods: A process evaluation that parallels the Replicating Effective Programs (REP) framework was used to understand stakeholder experiences with ACP and identify gaps in practice. Physicians, multidisciplinary staff, patients, and substitute decision makers engaged in focus groups and interviews, and completed surveys. A retrospective chart review determined Plan for Life Sustaining Treatment (PLST) form completion rates. Results: Stakeholder feedback identified barriers and facilitators to ACP including a need for staff training, user-friendly resources, and standardization of ACP practice. The PLST form was developed and embedded in the electronic medical record, and had a 92% and an 87% PLST completion rate on 2 pilot units. Conclusion: The study showed the usefulness of the REP model in guiding the evaluation as an effective tool to enhance implementation practices and inform ACP policy development that can be replicated in other organizations.

Care team and practice-level implementation strategies to optimize pediatric collaborative care: study protocol for a cluster-randomized hybrid type III trial

BACKGROUND

Implementation facilitation is an effective strategy to support the implementation of evidence-based practices (EBPs), but our understanding of multilevel strategies and the mechanisms of change within the "black box" of implementation facilitation is limited. This implementation trial seeks to disentangle and evaluate the effects of facilitation strategies that separately target the care team and leadership levels on implementation of a collaborative care model in pediatric primary care. Strategies targeting the provider care team (TEAM) should engage team-level mechanisms, and strategies targeting leaders (LEAD) should engage organizational mechanisms.

METHODS

We will conduct a hybrid type 3 effectiveness-implementation trial in a 2 × 2 factorial design to evaluate the main and interactive effects of TEAM and LEAD and test for mediation and moderation of effects. Twenty-four pediatric primary care practices will receive standard REP training to implement Doctor-Office Collaborative Care (DOCC) and then be randomized to (1) Standard REP only, (2) TEAM, (3) LEAD, or (4) TEAM + LEAD. Implementation outcomes are DOCC service delivery and change in practice-level care management competencies. Clinical outcomes are child symptom severity and quality of life.

DISCUSSION

This statewide trial is one of the first to test the unique and synergistic effects of implementation strategies targeting care teams and practice leadership. It will advance our knowledge of effective care team and practice-level implementation strategies and mechanisms of change. Findings will support efforts to improve common child behavioral health conditions by optimizing scale-up and sustainment of CCMs in a pediatric patient-centered medical home.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT04946253 . Registered June 30, 2021.

A randomized controlled trial comparing a manual and computer version of CALM in VA community-based outpatient clinics

Background:

This study compared a computer and manual version of a tailored Coordinated Anxiety Learning and Management (VA CALM) protocol on provider fidelity to CBT and patient outcomes.

Methods:

This study was a cluster randomized controlled trial. Providers (N = 32) were randomized to deliver VA CALM by computer or manual. Veteran patients (N = 135), treated by study providers, were recruited. The primary outcome was CBT fidelity, measured by rating audiotaped sessions. Secondary outcomes were Veterans’ general (BSI-18 GSI, SF-12) and disorder-specific (GAD-7, PCL-5, PHQ-9) outcomes assessed at baseline, three and six month follow-up.

Results:

We found a large (d = 0.88) but not statistically significant difference in mean fidelity rating scores between conditions. Compared with the manual, participants with generalized anxiety disorder receiving VA CALM by computer reported lower GAD-7 scores at three (−5.88; 95% CI=−11.37, −0.39) and six month (−5.25; 95% CI=−10.29, −0.22) follow-ups (d = 0.37 to 0.55). Participants in the computer and manual conditions reported lower PHQ-9 (−3.11; 95% CI=−5.51, −0.71; −4.06; 95% CI=−7.22, −0.90, respectively) and BSI-18 GSI (0.78; 95% CI=0.68,0.90; 0.71; 95% CI=0.58, 0.87, respectively) scores from baseline to six month follow-up. We did not find statistically significant differences over time or between conditions on SF-12 or PCL-5 scores.

Limitations:

This study was underpowered to test the primary outcome. Small samples sizes in the disorder-specific subgroup analysis may limit the generalizability of findings.

Conclusions:

Neither modality proved to be superior on VA CALM fidelity. The computer version of VA CALM, compared to the manual, may provide modest benefit to Veterans with GAD.

Assessing Implementation Strategy Reporting in the Mental Health Literature: A Narrative Review

Inadequate implementation strategy reporting restricts research synthesis and replicability. We explored the implementation strategy reporting quality of a sample of mental health articles using Proctor et al.'s (Implement Sci 8:139, 2013) reporting recommendations. We conducted a narrative review to generate the sample of articles and assigned a reporting quality score to each article. The mean article reporting score was 54% (range 17-100%). The most reported domains were: name (100%), action (82%), target (80%), and actor (67%). The least reported domains included definition (6%), temporality (26%), justification (34%), and outcome (37%). We discuss limitations and provide recommendations to improve reporting.

Comparative effectiveness of external vs blended facilitation on collaborative care model implementation in slow-implementer community practices

OBJECTIVE

To evaluate the comparative effectiveness of external facilitation (EF) vs external + internal facilitation (EF/IF), on uptake of a collaborative chronic care model (CCM) in community practices that were slower to implement under low-level implementation support.

STUDY SETTING

Primary data were collected from 43 community practices in Michigan and Colorado at baseline and for 12 months following randomization.

STUDY DESIGN

Sites that failed to meet a pre-established implementation benchmark after six months of low-level implementation support were randomized to add either EF or EF/IF support for up to 12 months. Key outcomes were change in number of patients receiving the CCM and number of patients receiving a clinically significant dose of the CCM. Moderators' analyses further examined whether comparative effectiveness was dependent on prerandomization adoption, number of providers trained or practice size. Facilitation log data were used for exploratory follow-up analyses.

DATA COLLECTION

Sites reported monthly on number of patients that had received the CCM. Facilitation logs were completed by study EF and site IFs and shared with the study team.

PRINCIPAL FINDINGS

N = 21 sites were randomized to EF and 22 to EF/IF. Overall, EF/IF practices saw more uptake than EF sites after 12 months (Δ_EF/IF-EF  = 4.4 patients, 95% CI = 1.87-6.87). Moderators' analyses, however, revealed that it was only sites with no prerandomization uptake of the CCM (nonadopter sites) that saw significantly more benefit from EF/IF (Δ_EF/IF-EF  = 9.2 patients, 95% CI: 5.72, 12.63). For sites with prerandomization uptake (adopter sites), EF/IF offered no additional benefit (Δ_EF/IF-EF  = -0.9; 95% CI: -4.40, 2.60). Number of providers trained and practice size were not significant moderators.

CONCLUSIONS

Although stepping up to the more intensive EF/IF did outperform EF overall, its benefit was limited to sites that failed to deliver any CCM under the low-level strategy. Once one or more providers were delivering the CCM, additional on-site personnel did not appear to add value to the implementation effort.

Implementation of a stepped wedge cluster randomized trial to evaluate a hospital mobility program

Background

Stepped wedge cluster randomized trials (SW-CRT) are increasingly used to evaluate new clinical programs, yet there is limited guidance on practical aspects of applying this design. We report our early experiences conducting a SW-CRT to examine an inpatient mobility program (STRIDE) in the Veterans Health Administration (VHA). We provide recommendations for future research using this design to evaluate clinical programs.

Methods

Based on data from study records and reflections from the investigator team, we describe and assess the design and initial stages of a SW-CRT, from site recruitment to program launch in 8 VHA hospitals.

Results

Site recruitment consisted of thirty 1-h conference calls with representatives from 22 individual VAs who expressed interest in implementing STRIDE. Of these, 8 hospitals were enrolled and randomly assigned in two stratified blocks (4 hospitals per block) to a STRIDE launch date. Block 1 randomization occurred in July 2017 with first STRIDE launch in December 2017; block 2 randomization occurred in April 2018 with first STRIDE launch in January 2019. The primary study outcome of discharge destination will be assessed using routinely collected data in the electronic health record (EHR). Within randomized blocks, two hospitals per sequence launched STRIDE approximately every 3 months with primary outcome assessment paused during the 3-month time period of program launch. All sites received 6–8 implementation support calls, according to a pre-specified schedule, from the time of recruitment to program launch, and all 8 sites successfully launched within their assigned 3-month window. Seven of the eight sites initially started with a limited roll out (for example on one ward) or modified version of STRIDE (for example, using existing staff to conduct walks until new positions were filled).

Conclusions

Future studies should incorporate sufficient time for site recruitment and carefully consider the following to inform design of SW-CRTs to evaluate rollout of a new clinical program: (1) whether a blocked randomization fits study needs, (2) the amount of time and implementation support sites will need to start their programs, and (3) whether clinical programs are likely to include a “ramp-up” period. Successful execution of SW-CRT designs requires both adherence to rigorous design principles and also careful consideration of logistical requirements for timing of program roll out.

Trial registration

ClinicalsTrials.gov NCT03300336. Prospectively registered on 3 October 2017.

An Economic Analysis of the Implementation of Team-based Collaborative Care in Outpatient General Mental Health Clinics

BACKGROUND

Collaborative Chronic Care Models represent an evidence-based way to structure care for chronic conditions, including mental health conditions. Few studies, however, have examined the cost implications of collaborative care for mental health.

OBJECTIVE

We aimed to conduct an economic analysis of implementing collaborative care in 9 outpatient general mental health clinics.

RESEARCH DESIGN

Analyses were derived from a stepped wedge hybrid implementation-effectiveness trial. We conducted cost-minimization analyses from the health system perspective, incorporating implementation costs, outpatient costs, and inpatient costs for the year before collaborative care implementation and the implementation year. We used a difference-in-differences approach and conducted 1-way sensitivity analyses to determine the robustness of results to variations ±15% in model parameters, along with probabilistic sensitivity analysis using Monte Carlo simulation.

SUBJECTS

Our treatment group included 5507 patients who were initially engaged in care within 9 outpatient general mental health teams that underwent collaborative care implementation. We compared costs for this group to 45,981 control patients who received mental health treatment as usual at the same medical centers.

RESULTS

Collaborative care implementation cost about $40 per patient and was associated with a significant decrease in inpatient costs and a nonsignificant increase in outpatient mental health costs. This implementation was associated with $78 in cost savings per patient. Monte Carlo simulation suggested that implementation was cost saving in 78% of iterations.

CONCLUSIONS

Collaborative care implementation for mental health teams was associated with significant reductions in mental health hospitalizations, leading to substantial cost savings of about $1.70 for every dollar spent for implementation.

Practicing in a person-centred environment - self-help groups in psycho-social rehabilitation

AIM

The increasing prevalence of chronic conditions and impairments in the population is putting new demands on health and rehabilitation services. Research on self-help groups suggest that participation in these groups might have a positive impact on people who are struggling with chronic illnesses or disabilities. In this study, we explore person-centred support in which participants in self-help groups are undergoing rehabilitation to develop their knowledge, skills and confidence necessary to handle life's challenges.

METHOD

The design is exploratory, analysing data from informant interviews and focus groups (a total of 32 participants) using a Grounded Theory inspired approach to analyse. The participants were rehabilitation clients aged between 20 and 60 years; eight were men and twenty-six were women.

RESULTS

Three main categories emerged as being important self-help processes that were likely to promote positive rehabilitation outcomes: (1) Learning and practicing safely, (2) A refuge from expectations, (3) Internal processes that accentuate the positives.

CONCLUSION

Peer support delivered through the structured self-help environment can facilitate the development of new self-awareness, promote acceptance and adjustment, facilitate the establishment of new skills and enable transfer of learning to new environments, including the workplace.IMPLICATIONS FOR REHABILITATIONSelf-help groups may support the process of rehabilitation.Participating in self-help groups provides an enabling context for individuals to address challenges and limitations.Peer support delivered through the structured self-help environment can facilitate the development of new self-awareness, promote adjustment, and facilitate the establishment of new skills.Participating in peer led self-help groups can assist with the transfer of learning to new environments, including development of potential work capacity.

Implementing a State-Adopted High School Health Curriculum: A Case Study

BACKGROUND

The Michigan Model for Health™ (MMH) is the official health curriculum for the State of Michigan and prevailing policy and practice has encouraged its adoption. Delivering evidence-based programs such as MMH with fidelity is essential to program effectiveness. Yet, most schools do meet state-designated fidelity requirements for implementation (delivering 80% or more of the curriculum).

METHODS

We collected online survey (N = 20) and in-person interview (N = 5) data investigating fidelity and factors related to implementation of the MMH curriculum from high school health teachers across high schools in one socioeconomically challenged Michigan county and key stakeholders.

RESULTS

We found that 68% of teachers did not meet state-identified standards of fidelity for curriculum delivery. Our results indicate that factors related to the context and implementation processes (eg, trainings) may be associated with fidelity. Teachers reported barriers to program delivery, including challenges with adapting the curriculum to suit their context, competing priorities, and meeting students' needs on key issues such as substance use and mental health issues.

CONCLUSIONS

Multiple factors influence the fidelity of health curriculum delivery in schools serving low-income students. Investigating these factors guided by implementation science frameworks can inform use of implementation strategies to support and enhance curriculum delivery.

The Collaborative Chronic Care Model for Mental Health Conditions: From Evidence Synthesis to Policy Impact to Scale-up and Spread

BACKGROUND

Extensive evidence indicates that Collaborative Chronic Care Models (CCMs) improve outcome in chronic medical conditions and depression treated in primary care. Beginning with an evidence synthesis which indicated that CCMs are also effective for multiple mental health conditions, we describe a multistage process that translated this knowledge into evidence-based health system change in the US Department of Veterans Affairs (VA).

EVIDENCE SYNTHESIS

In 2010, recognizing that there had been numerous CCM trials for a wide variety of mental health conditions, we conducted an evidence synthesis compiling randomized controlled trials of CCMs for any mental health condition. The systematic review demonstrated CCM effectiveness across mental health conditions and treatment venues. Cumulative meta-analysis and meta-regression further informed our approach to subsequent CCM implementation.

POLICY IMPACT

In 2015, based on the evidence synthesis, VA Office of Mental Health and Suicide Prevention (OMHSP) adopted the CCM as the model for their outpatient mental health teams.

RANDOMIZED IMPLEMENTATION TRIAL

In 2015-2018 we partnered with OMHSP to conduct a 9-site stepped wedge implementation trial, guided by insights from the evidence synthesis.

SCALE-UP AND SPREAD

In 2017 OMHSP launched an effort to scale-up and spread the CCM to additional VA medical centers. Seventeen facilitators were trained and 28 facilities engaged in facilitation.

DISCUSSION

Evidence synthesis provided leverage for evidence-based policy change. This formed the foundation for a health care leadership/researcher partnership, which conducted an implementation trial and subsequent scale-up and spread effort to enhance adoption of the CCM, as informed by the evidence synthesis.

[A review of the update of the German S3-guideline on diagnostics and therapy of bipolar disorders 2019]

Since the first publication of the guideline in 2012, which included critically reviewed evidence up to 2010, several hundred articles with new evidence were published and some topics of the clinical consensus needed to be reconsidered. Therefore, it was urgently necessary to revise the guideline to bring them up to date. In this article important revisions and updates are presented and the chances and limitations of the development of the guidelines and their implementation are discussed.

An Evidence-Based Model for Disseminating-Implementing Coordinated Anxiety Learning and Management in Department of Veterans Affairs' Community-Based Outpatient Clinics

PURPOSE

To explore the feasibility and utility of using a workshop, and supervision-consultation plus external facilitation to disseminate and implement cognitive-behavioral therapy in Veterans Affairs (VA) community-based outpatient clinics (CBOCs).

METHODS

This study occurred in the context of a randomized controlled trial aimed at comparing 2 methods for implementing Coordinated Anxiety Learning Management (CALM) in VA CBOCs. A 3-phase (workshop, supervision-consultation, external facilitation) model was used to support 32 VA CBOC mental health providers in learning and adopting CALM in their clinical practice. Qualitative data describe training activities and the feasibility and utility of each training phase in addressing challenges to adopting CALM.

FINDINGS

All 3 phases of the model were feasible to use with our sample of CBOC mental health providers. Providers reported challenges learning CALM during the workshop and concerns about not having enough training post-workshop to use CALM in practice. Providers primarily utilized supervision-consultation to tailor CALM to their practice, including learning how to prioritize a target disorder, "switch" the focus of treatment to a different disorder when comorbidities were present, and modify CALM sessions to fit shorter treatment visits. Providers primarily utilized external facilitation to further tailor CALM to their practice through implementation (eg, concrete help) and support-oriented help. Key lessons for implementing CALM in CBOCs are presented and discussed.

CONCLUSIONS

Findings provide initial evidence for the feasibility and utility of using each component of a facilitation-enhanced training model to promote CBOC VA providers' implementation of a computer and manual version of CALM in their practice.

Assessing Implementation Strategy Reporting in the Mental Health Literature: A Narrative Review

Inadequate implementation strategy reporting restricts research synthesis and replicability. We explored the implementation strategy reporting quality of a sample of mental health articles using Proctor et al.'s (Implement Sci 8:139, 2013) reporting recommendations. We conducted a narrative review to generate the sample of articles and assigned a reporting quality score to each article. The mean article reporting score was 54% (range 17-100%). The most reported domains were: name (100%), action (82%), target (80%), and actor (67%). The least reported domains included definition (6%), temporality (26%), justification (34%), and outcome (37%). We discuss limitations and provide recommendations to improve reporting.

Effectiveness of Implementing a Collaborative Chronic Care Model for Clinician Teams on Patient Outcomes and Health Status in Mental Health: A Randomized Clinical Trial

IMPORTANCE

Collaborative chronic care models (CCMs) have extensive randomized clinical trial evidence for effectiveness in serious mental illnesses, but little evidence exists regarding their feasibility or effect in typical practice conditions.

OBJECTIVE

To determine the effectiveness of implementation facilitation in establishing the CCM in mental health teams and the impact on health outcomes of team-treated individuals.

DESIGN, SETTING, AND PARTICIPANTS

This quasi-experimental, randomized stepped-wedge implementation trial was conducted from February 2016 through February 2018, in partnership with the US Department of Veterans Affairs (VA) Office of Mental Health and Suicide Prevention. Nine facilities were enrolled from all VA facilities in the United States to receive CCM implementation support. All veterans (n = 5596) treated by designated outpatient general mental health teams were included for hospitalization analyses, and a randomly selected sample (n = 1050) was identified for health status interviews. Individuals with dementia were excluded. Clinicians (n = 62) at the facilities were surveyed, and site process summaries were rated for concordance with the CCM process. The CCM implementation start time was randomly assigned across 3 waves. Data analysis of this evaluable population was performed from June to September 2018.

INTERVENTIONS

Internal-external facilitation, combining a study-funded external facilitator and a facility-funded internal facilitator working with a designated team for 1 year.

MAIN OUTCOMES AND MEASURES

Facilitation was hypothesized to be associated with improvements in both implementation and intervention outcomes (hybrid type II trial). Implementation outcomes included the clinician Team Development Measure (TDM) and proportion of CCM-concordant team care processes. The study was powered for the primary health outcome, mental component score (MCS). Hospitalization rate was derived from administrative data.

RESULTS

The veteran population (n = 5596) included 881 women (15.7%), and the mean (SD) age was 52.2 (14.5) years. The interviewed sample (n = 1050) was similar but was oversampled for women (n = 210 [20.0%]). Facilitation was associated with improvements in TDM subscales for role clarity (53.4%-68.6%; δ = 15.3; 95% CI, 4.4-26.2; P = .01) and team primacy (50.0%-68.6%; δ = 18.6; 95% CI, 8.3-28.9; P = .001). The percentage of CCM-concordant processes achieved varied, ranging from 44% to 89%. No improvement was seen in veteran self-ratings, including the primary outcome. In post hoc analyses, MCS improved in veterans with 3 or more treated mental health diagnoses compared with others (β = 5.03; 95% CI, 2.24-7.82; P < .001). Mental health hospitalizations demonstrated a robust decrease during facilitation (β = -0.12; 95% CI, -0.16 to -0.07; P < .001); this finding withstood 4 internal validity tests.

CONCLUSIONS AND RELEVANCE

Implementation facilitation that engages clinicians under typical practice conditions can enhance evidence-based team processes; its effect on self-reported overall population health status was negligible, although health status improved for individuals with complex conditions and hospitalization rate declined.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02543840.

Refining a model of collaborative care for people with a diagnosis of bipolar, schizophrenia or other psychoses in England: a qualitative formative evaluation

BACKGROUND

Many people diagnosed with schizophrenia, bipolar or other psychoses in England receive the majority of their healthcare from primary care. Primary care practitioners may not be well equipped to meet their needs and there is often poor communication with secondary care. Collaborative care is a promising alternative model but has not been trialled specifically with this service user group in England. Collaborative care for other mental health conditions has not been widely implemented despite evidence of its effectiveness. We carried out a formative evaluation of the PARTNERS model of collaborative care, with the aim of establishing barriers and facilitators to delivery, identifying implementation support requirements and testing the initial programme theory.

METHODS

The PARTNERS intervention was delivered on a small scale in three sites. Qualitative data was collected from primary and secondary care practitioners, service users and family carers, using semi-structured interviews, session recordings and tape-assisted recall. Deductive and inductive thematic analysis was carried out; themes were compared to the programme theory and used to inform an implementation support strategy.

RESULTS

Key components of the intervention that were not consistently delivered as intended were: interaction with primary care teams, the use of coaching, and supervision. Barriers and facilitators identified were related to service commitment, care partner skills, supervisor understanding and service user motivation. An implementation support strategy was developed, with researcher facilitation of communication and supervision and additional training for practitioners. Some components of the intervention were not experienced as intended; this appeared to reflect difficulties with operationalising the intervention. Analysis of data relating to the intended outcomes of the intervention indicated that the mechanisms proposed in the programme theory had operated as expected.

CONCLUSIONS

Additional implementation support is likely to be required for the PARTNERS model to be delivered; the effectiveness of such support may be affected by practitioner and service user readiness to change. There is also a need to test the programme theory more fully. These issues will be addressed in the process evaluation of our full trial.

TRIAL REGISTRATION

ISRCTN95702682 , 26 October 2017.

Early Mobility in the Hospital: Lessons Learned from the STRIDE Program

Immobility during hospitalization is widely recognized as a contributor to deconditioning, functional loss, and increased need for institutional post-acute care. Several studies have demonstrated that inpatient walking programs can mitigate some of these negative outcomes, yet hospital mobility programs are not widely available in U.S. hospitals. STRIDE (assiSTed eaRly mobIlity for hospitalizeD older vEterans) is a supervised walking program for hospitalized older adults that fills this important gap in clinical care. This paper describes how STRIDE works and how it is being disseminated to other hospitals using the Replicating Effective Programs (REP) framework. Guided by REP, we define core components of the program and areas where the program can be tailored to better fit the needs and local conditions of its new context (hospital). We describe key adaptations made by four hospitals who have implemented the STRIDE program and discuss lessons learned for successful implementation of hospital mobility programs.

Adaptive School-based Implementation of CBT (ASIC): clustered-SMART for building an optimized adaptive implementation intervention to improve uptake of mental health interventions in schools

Background

Depressive and anxiety disorders affect 20–30% of school-age youth, most of whom do not receive adequate services, contributing to poor developmental and academic outcomes. Evidence-based practices (EBPs) such as cognitive behavioral therapy (CBT) can improve outcomes, but numerous barriers limit access among affected youth. Many youth try to access mental health services in schools, but school professionals (SPs: counselors, psychologists, social workers) are rarely trained adequately in CBT methods. Further, SPs face organizational barriers to providing CBT, such as lack of administrative support. Three promising implementation strategies to address barriers to school-based CBT delivery include (1) Replicating Effective Programs (REP), which deploys customized CBT packaging, didactic training in CBT, and technical assistance; (2) coaching, which extends training via live supervision to improve SP competence in CBT delivery; and (3) facilitation, which employs an organizational expert who mentors SPs in strategic thinking to promote self-efficacy in garnering administrative support. REP is a relatively low-intensity/low-cost strategy, whereas coaching and facilitation require additional resources. However, not all schools will require all three strategies. The primary aim of this study is to compare the effectiveness of a school-level adaptive implementation intervention involving REP, coaching, and facilitation versus REP alone on the frequency of CBT delivered to students by SPs and student mental health outcomes. Secondary and exploratory aims examine cost-effectiveness, moderators, and mechanisms of implementation strategies.

Methods

Using a clustered, sequential multiple-assignment, randomized trial (SMART) design, ≥ 200 SPs from 100 schools across Michigan will be randomized initially to receive REP vs. REP+coaching. After 8 weeks, schools that do not meet a pre-specified implementation benchmark are re-randomized to continue with the initial strategy or to augment with facilitation.

Discussion

EBPs need to be implemented successfully and efficiently in settings where individuals are most likely to seek care in order to gain large-scale impact on public health. Adaptive implementation interventions hold the promise of providing cost-effective implementation support. This is the first study to test an adaptive implementation of CBT for school-age youth, at a statewide level, delivered by school staff, taking an EBP to large populations with limited mental health care access.

Trial registration

NCT03541317—Registered on 29 May 2018 on ClinicalTrials.gov PRS

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0808-8) contains supplementary material, which is available to authorized users.

Organizational culture and climate as moderators of enhanced outreach for persons with serious mental illness: results from a cluster-randomized trial of adaptive implementation strategies

Background

Organizational culture and climate are considered key factors in implementation efforts but have not been examined as moderators of implementation strategy comparative effectiveness. We investigated organizational culture and climate as moderators of comparative effectiveness of two sequences of implementation strategies (Immediate vs. Delayed Enhanced Replicating Effective Programs [REP]) combining Standard REP and REP enhanced with facilitation on implementation of an outreach program for Veterans with serious mental illness lost to care at Veterans Health Administration (VA) facilities nationwide.

Methods

This study is a secondary analysis of the cluster-randomized Re-Engage implementation trial that assigned 3075 patients at 89 VA facilities to either the Immediate or Delayed Enhanced REP sequences. We hypothesized that sites with stronger entrepreneurial culture, task, or relational climate would benefit more from Enhanced REP than Standard REP. Veteran- and site-level data from the Re-Engage trial were combined with site-aggregated measures of entrepreneurial culture and task and relational climate from the 2012 VA All Employee Survey. Longitudinal mixed-effects logistic models examined whether the comparative effectiveness of the Immediate vs. Delayed Enhanced REP sequences were moderated by culture or climate measures at 6 and 12 months post-randomization. Three Veteran-level outcomes related to the engagement with the VA system were assessed: updated documentation, attempted contact by coordinator, and completed contact.

Results

For updated documentation and attempted contact, Veterans at sites with higher entrepreneurial culture and task climate scores benefitted more from Enhanced REP compared to Standard REP than Veterans at sites with lower scores. Few culture or climate moderation effects were detected for the comparative effectiveness of the full sequences of implementation strategies.

Conclusions

Implementation strategy effectiveness is highly intertwined with contextual factors, and implementation practitioners may use knowledge of contextual moderation to tailor strategy deployment. We found that facilitation strategies provided with Enhanced REP were more effective at improving uptake of a mental health outreach program at sites with stronger entrepreneurial culture and task climate; Veterans at sites with lower levels of these measures saw more similar improvement under Standard and Enhanced REP. Within resource-constrained systems, practitioners may choose to target more intensive implementation strategies to sites that will most benefit from them.

Trial registration

ISRCTN: ISRCTN21059161. Date registered: April 11, 2013.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0787-9) contains supplementary material, which is available to authorized users.

Methods to Improve the Selection and Tailoring of Implementation Strategies

Implementing behavioral health interventions is a complicated process. It has been suggested that implementation strategies should be selected and tailored to address the contextual needs of a given change effort; however, there is limited guidance as to how to do this. This article proposes four methods (concept mapping, group model building, conjoint analysis, and intervention mapping) that could be used to match implementation strategies to identified barriers and facilitators for a particular evidence-based practice or process change being implemented in a given setting. Each method is reviewed, examples of their use are provided, and their strengths and weaknesses are discussed. The discussion includes suggestions for future research pertaining to implementation strategies and highlights these methods' relevance to behavioral health services and research.

Development and validation of the ASPIRE-VA coaching fidelity checklist (ACFC): a tool to help ensure delivery of high-quality weight management interventions

Practical and valid instruments are needed to assess fidelity of coaching for weight loss. The purpose of this study was to develop and validate the ASPIRE Coaching Fidelity Checklist (ACFC). Classical test theory guided ACFC development. Principal component analyses were used to determine item groupings. Psychometric properties, internal consistency, and inter-rater reliability were evaluated for each subscale. Criterion validity was tested by predicting weight loss as a function of coaching fidelity. The final 19-item ACFC consists of two domains (session process and session structure) and five subscales (sets goals and monitor progress, assess and personalize self-regulatory content, manages the session, creates a supportive and empathetic climate, and stays on track). Four of five subscales showed high internal consistency (Cronbach alphas > 0.70) for group-based coaching; only two of five subscales had high internal reliability for phone-based coaching. All five sub-scales were positively and significantly associated with weight loss for group- but not for phone-based coaching. The ACFC is a reliable and valid instrument that can be used to assess fidelity and guide skill-building for weight management interventionists.

Improving Physical Health in Patients With Chronic Mental Disorders: Twelve-Month Results From a Randomized Controlled Collaborative Care Trial

OBJECTIVE

Persons with chronic mental disorders are disproportionately burdened with physical health conditions. We determined whether Life Goals Collaborative Care compared to usual care improves physical health in patients with mental disorders within 12 months.

METHODS

This single-blind randomized controlled effectiveness study of a collaborative care model was conducted at a midwestern Veterans Affairs urban outpatient mental health clinic. Patients (N = 293 out of 474 eligible approached) with an ICD-9-CM diagnosis of schizophrenia, bipolar disorder, or major depressive disorder and at least 1 cardiovascular disease risk factor provided informed consent and were randomized (February 24, 2010, to April 29, 2015) to Life Goals (n = 146) or usual care (n = 147). A total of 287 completed baseline assessments, and 245 completed 12-month follow-up assessments. Life Goals included 5 weekly sessions that provided semistructured guidance on managing physical and mental health symptoms through healthy behavior changes, augmented by ongoing care coordination. The primary outcome was change in physical health-related quality of life score (Veterans RAND 12-item Short Form Health Survey [VR-12] physical health component score). Secondary outcomes included control of cardiovascular risk factors from baseline to 12 months (blood pressure, lipids, weight), mental health-related quality of life, and mental health symptoms.

RESULTS

Among patients completing baseline and 12-month outcomes assessments (N = 245), the mean age was 55.3 years (SD = 10.8; range, 25-78 years), and 15.4% were female. Intent-to-treat analysis revealed that compared to those in usual care, patients randomized to Life Goals had slightly increased VR-12 physical health scores (coefficient = 3.21; P = .01).

CONCLUSIONS

Patients with chronic mental disorders and cardiovascular disease risk who received Life Goals had improved physical health-related quality of life.

TRIAL REGISTRATION

ClinicalTrials.gov identifiers: NCT01487668 and NCT01244854.

Methods used to address fidelity of receipt in health intervention research: a citation analysis and systematic review

BACKGROUND

The American Behaviour Change Consortium (BCC) framework acknowledges patients as active participants and supports the need to investigate the fidelity with which they receive interventions, i.e. receipt. According to this framework, addressing receipt consists in using strategies to assess or enhance participants' understanding and/or performance of intervention skills. This systematic review aims to establish the frequency with which receipt is addressed as defined in the BCC framework in health research, and to describe the methods used in papers informed by the BCC framework and in the wider literature.

METHODS

A forward citation search on papers presenting the BCC framework was performed to determine the frequency with which receipt as defined in this framework was addressed. A second electronic database search, including search terms pertaining to fidelity, receipt, health and process evaluations was performed to identify papers reporting on receipt in the wider literature and irrespective of the framework used. These results were combined with forward citation search results to review methods to assess receipt. Eligibility criteria and data extraction forms were developed and applied to papers. Results are described in a narrative synthesis.

RESULTS

19.6% of 33 studies identified from the forward citation search to report on fidelity were found to address receipt. In 60.6% of these, receipt was assessed in relation to understanding and in 42.4% in relation to performance of skill. Strategies to enhance these were present in 12.1% and 21.1% of studies, respectively. Fifty-five studies were included in the review of the wider literature. Several frameworks and operationalisations of receipt were reported, but the latter were not always consistent with the guiding framework. Receipt was most frequently operationalised in relation to intervention content (16.4%), satisfaction (14.5%), engagement (14.5%), and attendance (14.5%). The majority of studies (90.0%) included subjective assessments of receipt. These relied on quantitative (76.0%) rather than qualitative (42.0%) methods and studies collected data on intervention recipients (50.0%), intervention deliverers (28.0%), or both (22.0%). Few studies (26.0%) reported on the reliability or validity of methods used.

CONCLUSIONS

Receipt is infrequently addressed in health research and improvements to methods of assessment and reporting are required.

Correlates of Emergency Department Use by Individuals With Bipolar Disorder Enrolled in a Collaborative Care Implementation Study

OBJECTIVE

The study assessed correlates of emergency department use among participants in a collaborative care program for bipolar disorder.

METHODS

Community-based clinics from two states implemented Life Goals-Collaborative Care (LG-CC), an evidence-based model that includes self-management sessions and care management contacts. Logistic regression determined participant factors associated with emergency department use between six and 12 months after LG-CC implementation.

RESULTS

Of 219 participants with baseline and 12-month data, 24% reported at least one emergency department visit. Participants with a recent homelessness history (odds ratio [OR]=3.76, p=.01) or five or more care management contacts (OR=2.62, p=.05) had a higher probability of visiting an emergency department, after the analyses were adjusted for demographic and clinical factors, including physical health score and hospitalization history.

CONCLUSIONS

Participants in a collaborative care program who had a history of homelessness were more likely to use the emergency department, suggesting a greater need for more intensive care coordination.

Assessing fidelity of cognitive behavioral therapy in rural VA clinics: design of a randomized implementation effectiveness (hybrid type III) trial

BACKGROUND

Broadly disseminating and implementing evidence-based psychotherapies with high fidelity, particularly cognitive behavioral therapy (CBT), has proved challenging for many health-care systems, including the Department of Veterans Affairs, especially in primary care settings such as small or remote clinics. A computer-based tool (based on the coordinated anxiety learning and management (CALM) program) was designed to support primary care-based mental health providers in delivering CBT. The objectives of this study are to modify the CALM tool to meet the needs of mental health clinicians in veterans affairs (VA) community-based outpatient clinics (CBOCs) and rural "veterans", use external facilitation to implement CBT and determine the effect of the CALM tool versus a manualized version of CALM to improve fidelity to the CBT treatment model, and conduct a needs assessment to understand how best to support future implementation of the CALM tool in routine care.

METHODS/DESIGN

Focus groups will inform the redesign of the CALM tool. Mental health providers at regional VA CBOCs; CBT experts; VA experts in implementation of evidence-based mental health practices; and veterans with generalized anxiety disorder, panic disorder, social anxiety disorder, posttraumatic stress disorder, "with or without" depression will be recruited. A hybrid type III design will be used to examine the effect of receiving CBT training plus either the CALM tool or a manual version of CALM on treatment fidelity. External facilitation will be used as the overarching strategy to implement both CBT delivery methods. Data will also be collected on symptoms of the targeted disorders. To help prepare for the future implementation of the CALM tool in VA CBOCs, we will perform an implementation need assessment with mental health providers participating in the clinical trial and their CBOC directors.

DISCUSSION

This project will help inform strategies for delivering CBT with high fidelity in VA CBOCs to veterans with anxiety disorders and PTSD with or without depression. If successful, results of this study could be used to inform a national rollout of the CALM tool in VA CBOCs including providing recommendations for optimizing the adoption and sustained use of the computerized CALM tool among mental health providers in this setting.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02488551.

Longitudinal impact of a collaborative care model on employment outcomes in bipolar disorder

BACKGROUND

Few treatments are available to directly address employment or work functioning among individuals with bipolar disorder (BD) and currently available treatment models have not been evaluated to examine their impact employment outcomes. We examined impact of affective symptoms and health-related quality of life (HRQoL) on longitudinal employment outcomes in a community-based sample of individuals with bipolar disorder who completed the Life Goals-Collaborative Care (LG-CC) intervention.

METHODS

Participants (N=178) were assessed based on HRQoL, employment status, affective symptoms (depressive/manic), and work hours at baseline, 6-, 12- and 24-months after initiation of LG-CC. Frequency of LG-CC sessions and number of care-manager contacts also were ascertained.

RESULTS

At baseline, 21% were employed, 29.5% were unemployed, and 49.6% were on disability. Improvement in affective symptoms was seen over the 24-month period, but not in HRQoL. Lower depression symptoms, but not mania, at baseline predicted greater likelihood of employment status in 24-months. Degree of LG-CC participation was associated with a reduced likelihood of becoming disabled/unemployed and increased number of hours worked in 24-months.

LIMITATIONS

The study was originally designed to compare implementation strategies and not the effectiveness of LG-CC on employment outcomes. Further, it was unclear whether improvement in work functioning were personal goals of the participants of this study.

CONCLUSIONS

Fewer depressive symptoms were associated with positive employment outcomes over time. Collaborative Care Models that are already implemented by existing providers that focus on management of affective symptoms show promise in positively impacting employment outcomes.

An introduction to implementation science for the non-specialist

BACKGROUND

The movement of evidence-based practices (EBPs) into routine clinical usage is not spontaneous, but requires focused efforts. The field of implementation science has developed to facilitate the spread of EBPs, including both psychosocial and medical interventions for mental and physical health concerns.

DISCUSSION

The authors aim to introduce implementation science principles to non-specialist investigators, administrators, and policymakers seeking to become familiar with this emerging field. This introduction is based on published literature and the authors' experience as researchers in the field, as well as extensive service as implementation science grant reviewers. Implementation science is "the scientific study of methods to promote the systematic uptake of research findings and other EBPs into routine practice, and, hence, to improve the quality and effectiveness of health services." Implementation science is distinct from, but shares characteristics with, both quality improvement and dissemination methods. Implementation studies can be either assess naturalistic variability or measure change in response to planned intervention. Implementation studies typically employ mixed quantitative-qualitative designs, identifying factors that impact uptake across multiple levels, including patient, provider, clinic, facility, organization, and often the broader community and policy environment. Accordingly, implementation science requires a solid grounding in theory and the involvement of trans-disciplinary research teams. The business case for implementation science is clear: As healthcare systems work under increasingly dynamic and resource-constrained conditions, evidence-based strategies are essential in order to ensure that research investments maximize healthcare value and improve public health. Implementation science plays a critical role in supporting these efforts.

Long-Term Clinical Outcomes from a Randomized Controlled Trial of Two Implementation Strategies to Promote Collaborative Care Attendance in Community Practices

This randomized controlled implementation study compared the effectiveness of a standard versus enhanced version of the replicating effective programs (REP) implementation strategy to improve the uptake of the life goals-collaborative care model (LG-CC) for bipolar disorder. Seven community-based practices (384 patient participants) were randomized to standard (manual/training) or enhanced REP (customized manual/training/facilitation) to promote LG-CC implementation. Participants from enhanced REP sites had no significant changes in primary outcomes (improved quality of life, reduced functioning or mood symptoms) by 24 months. Further research is needed to determine whether implementation strategies can lead to sustained, improved participant outcomes in addition to program uptake.

Methods to Improve the Selection and Tailoring of Implementation Strategies

Implementing behavioral health interventions is a complicated process. It has been suggested that implementation strategies should be selected and tailored to address the contextual needs of a given change effort; however, there is limited guidance as to how to do this. This article proposes four methods (concept mapping, group model building, conjoint analysis, and intervention mapping) that could be used to match implementation strategies to identified barriers and facilitators for a particular evidence-based practice or process change being implemented in a given setting. Each method is reviewed, examples of their use are provided, and their strengths and weaknesses are discussed. The discussion includes suggestions for future research pertaining to implementation strategies and highlights these methods' relevance to behavioral health services and research.

Enhancing outreach for persons with serious mental illness: 12-month results from a cluster randomized trial of an adaptive implementation strategy

Background

Few implementation strategies have been empirically tested for their effectiveness in improving uptake of evidence-based treatments or programs. This study compared the effectiveness of an immediate versus delayed enhanced implementation strategy (Enhanced Replicating Effective Programs (REP)) for providers at Veterans Health Administration (VA) outpatient facilities (sites) on improved uptake of an outreach program (Re-Engage) among sites not initially responding to a standard implementation strategy.

Methods

One mental health provider from each U.S. VA site (N = 158) was initially given a REP-based package and training program in Re-Engage. The Re-Engage program involved giving each site provider a list of patients with serious mental illness who had not been seen at their facility for at least a year, requesting that providers contact these patients, assessing patient clinical status, and where appropriate, facilitating appointments to VA health services. At month 6, sites considered non-responsive (N = 89, total of 3,075 patients), defined as providers updating documentation for less than <80% of patients on their list, were randomized to two adaptive implementation interventions: Enhanced REP (provider coaching; N = 40 sites) for 6 months followed by Standard REP for 6 months; versus continued Standard REP (N = 49 sites) for 6 months followed by 6 months of Enhanced REP for sites still not responding. Outcomes included patient-level Re-Engage implementation and utilization.

Results

Patients from sites that were randomized to receive Enhanced REP immediately compared to Standard REP were more likely to have a completed contact (adjusted OR = 2.13; 95% CI: 1.09–4.19, P = 0.02). There were no differences in patient-level utilization between Enhanced and Standard REP sites.

Conclusions

Enhanced REP was associated with greater Re-Engage program uptake (completed contacts) among sites not responding to a standard implementation strategy. Further research is needed to determine whether national implementation of Facilitation results in tangible changes in patient-level outcomes.

Trial registration

ISRCTN: ISRCTN21059161

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-014-0163-3) contains supplementary material, which is available to authorized users.

Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program

BACKGROUND

Despite the availability of psychosocial evidence-based practices (EBPs), treatment and outcomes for persons with mental disorders remain suboptimal. Replicating Effective Programs (REP), an effective implementation strategy, still resulted in less than half of sites using an EBP. The primary aim of this cluster randomized trial is to determine, among sites not initially responding to REP, the effect of adaptive implementation strategies that begin with an External Facilitator (EF) or with an External Facilitator plus an Internal Facilitator (IF) on improved EBP use and patient outcomes in 12 months.

METHODS/DESIGN

This study employs a sequential multiple assignment randomized trial (SMART) design to build an adaptive implementation strategy. The EBP to be implemented is life goals (LG) for patients with mood disorders across 80 community-based outpatient clinics (N = 1,600 patients) from different U.S. regions. Sites not initially responding to REP (defined as < 50% patients receiving ≥ 3 EBP sessions) will be randomized to receive additional support from an EF or both EF/IF. Additionally, sites randomized to EF and still not responsive will be randomized to continue with EF alone or to receive EF/IF. The EF provides technical expertise in adapting LG in routine practice, whereas the on-site IF has direct reporting relationships to site leadership to support LG use in routine practice. The primary outcome is mental health-related quality of life; secondary outcomes include receipt of LG sessions, mood symptoms, implementation costs, and organizational change.

DISCUSSION

This study design will determine whether an off-site EF alone versus the addition of an on-site IF improves EBP uptake and patient outcomes among sites that do not respond initially to REP. It will also examine the value of delaying the provision of EF/IF for sites that continue to not respond despite EF.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT02151331.

Collaborative care outcomes for pediatric behavioral health problems: a cluster randomized trial

OBJECTIVE

To assess the efficacy of collaborative care for behavior problems, attention-deficit/hyperactivity disorder (ADHD), and anxiety in pediatric primary care (Doctor Office Collaborative Care; DOCC).

METHODS

Children and their caregivers participated from 8 pediatric practices that were cluster randomized to DOCC (n = 160) or enhanced usual care (EUC; n = 161). In DOCC, a care manager delivered a personalized, evidence-based intervention. EUC patients received psychoeducation and a facilitated specialty care referral. Care processes measures were collected after the 6-month intervention period. Family outcome measures included the Vanderbilt ADHD Diagnostic Parent Rating Scale, Parenting Stress Index-Short Form, Individualized Goal Attainment Ratings, and Clinical Global Impression-Improvement Scale. Most measures were collected at baseline, and 6-, 12-, and 18-month assessments. Provider outcome measures examined perceived treatment change, efficacy, and obstacles, and practice climate.

RESULTS

DOCC (versus EUC) was associated with higher rates of treatment initiation (99.4% vs 54.2%; P < .001) and completion (76.6% vs 11.6%, P < .001), improvement in behavior problems, hyperactivity, and internalizing problems (P < .05 to .01), and parental stress (P < .05-.001), remission in behavior and internalizing problems (P < .01, .05), goal improvement (P < .05 to .001), treatment response (P < .05), and consumer satisfaction (P < .05). DOCC pediatricians reported greater perceived practice change, efficacy, and skill use to treat ADHD (P < .05 to .01).

CONCLUSIONS

Implementing a collaborative care intervention for behavior problems in community pediatric practices is feasible and broadly effective, supporting the utility of integrated behavioral health care services.

Cluster randomized adaptive implementation trial comparing a standard versus enhanced implementation intervention to improve uptake of an effective re-engagement program for patients with serious mental illness

Background

Persons with serious mental illness (SMI) are disproportionately burdened by premature mortality. This disparity is exacerbated by poor continuity of care with the health system. The Veterans Health Administration (VA) developed Re-Engage, an effective population-based outreach program to identify veterans with SMI lost to care and to reconnect them with VA services. However, such programs often encounter barriers getting implemented into routine care. Adaptive designs are needed when the implementation intervention requires augmentation within sites that do not initially respond to an initial implementation intervention. This protocol describes the methods used in an adaptive implementation design study that aims to compare the effectiveness of a standard implementation strategy (Replicating Effective Programs, or REP) with REP enhanced with External Facilitation (enhanced REP) to promote the uptake of Re-Engage.

Methods/Design

This study employs a four-phase, two-arm, longitudinal, clustered randomized trial design. VA sites (n = 158) across the United States with a designated Re-Engage provider, at least one Veteran with SMI lost to care, and who received standard REP during a six-month run-in phase. Subsequently, 88 sites with inadequate uptake were stratified at the cluster level by geographic region (n = 4) and VA regional service network (n = 20) and randomized to REP (n = 49) vs. enhanced REP (n = 39) in phase two. The primary outcome was the percentage of veterans on each facility outreach list documented on an electronic web registry. The intervention was at the site and network level and consisted of standard REP versus REP enhanced by external phone facilitation consults. At 12 months, enhanced REP sites returned to standard REP and 36 sites with inadequate participation received enhanced REP for six months in phase three. Secondary implementation outcomes included the percentage of veterans contacted directly by site providers and the percentage re-engaged in VA health services.

Discussion

Adaptive implementation designs consisting of a sequence of decision rules that are tailored based on a site’s uptake of an effective program may produce more relevant, rapid, and generalizable results by more quickly validating or rejecting new implementation strategies, thus enhancing the efficiency and sustainability of implementation research and potentially leading to the rollout of more cost-efficient implementation strategies.

Trial registration

Current Controlled Trials ISRCTN21059161.