Consultation as an implementation strategy for evidence-based practices across multiple contexts: unpacking the black box

Consultation Content and Techniques for measurement-Based Care Implementation in Youth Community Mental Health Settings

Measurement-based care (MBC) is an evidence-based practice (EBP) focused on regularly administering outcome measures to clients to inform clinical decision making. While MBC shows promise for improving youth treatment outcomes, therapist adoption remains low. Clinical consultation is one strategy that improves MBC implementation, but our limited understanding of consultation hinders the ability to optimize its impact. This research explored the content of, and techniques used during MBC consultation calls. Therapists (N = 55) in a randomized controlled trial treating adolescents with anxiety and/or depression were trained to utilize MBC with usual treatment using the Youth Outcome Questionnaire (YOQ) through an online measurement feedback system (MFS). Weekly ongoing consultation followed an initial workshop training in MBC. Case discussions (N = 294) during consultation calls were coded using a developed codebook, including 12 content and 10 consultant techniques. Results indicated that content focused predominantly on interpretation of client symptom and alliance report, planning for YOQ administration, and discussion of data with clients in session. Common consultant techniques included modeling and eliciting report viewing and interpretation, making clinical suggestions, and didactics about clinical and technical issues. Notably, role-play/behavioral rehearsal was not used. The prevalence of passive consultation techniques (suggestions, didactics) suggests a focus on teaching rather than active techniques (behavioral rehearsal, modeling), potentially influenced by the novelty of MBC and MFS. Technical aspects of MBC, such as measure administration and system usage, emerged as key consultation content, highlighting an unanticipated emphasis on logistics over clinical implementation. These findings underscore the evolving role of consultation in supporting MBC implementation and suggest that addressing technical challenges early in training might enhance adoption.

Clinical Consultation During a Trauma-Focused Cognitive Behavioral Therapy Community-Based Learning Collaborative: Examination of Consultation Content, Consultative Strategies, and Provider Engagement

Consultation following evidence-based practice (EBP) training enhances the uptake of EBPs. Yet, little is known about what occurs during consultation, and it is often difficult for providers to engage in consultation. This study examined provider engagement in consultation and the content and strategies used during consultation following training in Trauma-focused Cognitive Behavioral Therapy (TF-CBT) as part of a community-based learning collaborative (CBLC). Minute-to-minute live coding of consultation calls revealed most content was clinically-oriented and the most common strategies used by consultants were didactic in nature. Providers with more years of professional experience and those with greater TF-CBT knowledge attended significantly more consultation calls. Providers with a greater average weekly caseload and providers who were supervisors presented significantly more cases on calls. Providers with greater TF-CBT knowledge spoke significantly more minutes on calls. Consistent with previous work, findings highlight difficulties with provider engagement in consultation and that providers with more baseline expertise are most likely to be engaged. Findings suggest tailoring EBP training efforts to better engage providers at greatest risk for low engagement.

Building trusting relationships in teams to support evidence use and implementation in human services: feasibility and acceptability of a training and coaching approach

Background

Professionals who provide implementation support in human service systems describe relationships as being critical to support evidence use; however, developing trusting relationships are not strongly featured in implementation science literature. The aims of this study were to (a) assess the feasibility and acceptability of a theory-driven training and coaching approach for building trusting relationships among members of an implementation team who were supporting the implementation of an evidence-informed program in a public child welfare system in the United States and (b) gauge the initial efficacy of the approach in terms of the development of trusting relationships and subsequent implementation outcomes.

Methods

Consistent with a convergent mixed-methods approach, we collected both quantitative and qualitative data to address our research questions. Quantitative methods included an adapted version of the Trusting Relationship Questionnaire, a seven-item measure of psychological safety, and items designed to measure acceptability of the training. Qualitative data were collected through semi-structured interviews with participants.

Results

Sixteen individuals participated in the program, consisting of a kick-off training event, five monthly training modules, and five monthly coaching sessions with implementation team leads. Session attendance rates and self-reported satisfaction highlight the general feasibility and acceptability of the training and coaching approach. On average, participants also reported significant increases over time in their perceptions that they were trusted by their team. Results from in-depth interviews further indicated the efficacy of the program in terms of cultivating trust among team members and promoting several elements that were theorized to link trusting relationships to implementation outcomes.

Discussion

Findings suggest the training and coaching approach for trust building was acceptable and feasible. Additionally, results indicate the value of the approach in building trust among implementation partners to increase commitment to implementation efforts, promote a culture of learning and psychological safety, and increase participants' sense of capability and motivation for supporting implementation.

Evaluating the Implementation of the Individualized Positive Psychosocial Interaction (IPPI) as a Quality Improvement Project in Nursing Homes

OBJECTIVES

The Individualized Positive Psychosocial Interaction (IPPI) is a non-pharmacological, person-centered, intervention for nursing home (NH) residents living with moderate to severe dementia. The purpose of this study was to assess the pragmatic implementation of the IPPI by leveraging Ohio's Nursing Home Quality Improvement Program (QIP).

METHODS

Implementation teams collected resident mood ratings pre- and post-IPPI and completed virtual interviews to assess their Organizational Readiness for Implementing Change as well as the acceptability, feasibility, and appropriateness of the IPPI. Implementation strategies included: providing ongoing consultation; identifying and preparing champions; assessing for readiness and identifying barriers; and developing and distributing effective educational materials.

RESULTS

Fifteen NHs completed the QIP (65% completion rate) and reported high organizational commitment to change and high change efficacy. NHs engaged n = 65 residents in n = 638 IPPIs. Residents experienced a positive mood change after 47% of IPPIs. NHs found the IPPI program to be highly acceptable, feasible, and appropriate.

CONCLUSIONS

Overall, 65% of NHs successfully implemented the IPPI QIP with people living with moderate to severe dementia.

CLINICAL IMPLICATIONS

Given the positive mood changes and high staff satisfaction, results suggest that these brief, individualized activities can be effective strategies to address the communication of distress among PLWD.

Partnering with Educators to Iteratively Co-create Tools to Support Teachers’ Use of Equity-Focused Positive Behavioral Supports

In the Maximize Project, we are engaging in a research-practice partnership to co-create implementation strategies to facilitate elementary school teachers’ use of equity-focused positive behavior supports (EF-PBS). In this paper, we describe the processes used to build an interactive, technology-based platform to enhance teachers’ use of EF-PBS via self-reflection, self-assessment, goal setting, and goal review. We describe how we established a multi-disciplinary, multi-state community advisory board to collaborate on Version 1 of platform (Phase 1). We explain how we obtained quantitative and qualitative feedback about the platform from educators in three partnering schools, and how we used those data to produce Version 2 of the platform (Phase 2). Platform use data suggested high utilization in Quarter 1 (August–October) of the school year, when there was protected time to complete activities. However, platform use was moderate in Quarter 2 (October–December) and low in Quarters 3 and 4 (January–May). Educator feedback revealed moderate acceptability, feasibility, and appropriateness of the platform and highlighted ways to improve the user experience (e.g., streamlining steps in goal setting, making resources about strategy implementation easier to find). We discuss lessons learned to inform school mental health co-creation endeavors, including strategies for supporting diverse perspectives, for enhancing advisory board members’ voices and confidence, and for creating practical and feasible methods for teachers to benefit from co-created technology-based implementation strategies. Our processes offer guidance for others engaging in research-practice partnerships, developing education technologies and/or supporting teachers’ use of equity-focused practices to improve daily school experiences for all students.

The Secret Sauce? Taking the Mystery Out of Scaling-Up School-Based Physical Activity Interventions

Over the last 4 decades, physical activity researchers have invested heavily in determining "what works" to promote healthy behaviors in schools. Single and multicomponent school-based interventions that target physical education, active transportation, and/or classroom activity breaks effectively increased physical activity among children and youth. Yet, few of these interventions are ever scaled-up and implemented under real-world conditions and in diverse populations. To achieve population-level health benefits, there is a need to design school-based health-promoting interventions for scalability and to consider key aspects of the scale-up process. In this opinion piece, we aim to identify challenges and advance knowledge and action toward scaling-up school-based physical activity interventions. We highlight the key roles of planning for scale-up at the outset, scale-up pathways, trust among partners and program support, program adaptation, evaluation of scale-up, and barriers and facilitators to scaling-up. We draw upon our experience scaling-up effective school-based interventions and provide a solid foundation from which others can work toward bridging the implementation-to-scale-up gap.

Implementing evidence-based practices to improve primary care for high-risk patients: study protocol for the VA high-RIsk VETerans (RIVET) type III effectiveness-implementation trial

BACKGROUND

Patients with significant multimorbidity and other factors that make healthcare challenging to access and coordinate are at high risk for poor health outcomes. Although most (93%) of Veterans' Health Administration (VHA) patients at high risk for hospitalization or death ("high-risk Veterans") are primarily managed by primary care teams, few of these teams have implemented evidence-based practices (EBPs) known to improve outcomes for the high-risk patient population's complex healthcare issues. Effective implementation strategies could increase adoption of these EBPs in primary care; however, the most effective implementation strategies to increase evidence-based care for high-risk patients are unknown. The high-RIsk VETerans (RIVET) Quality Enhancement Research Initiative (QUERI) will compare two variants of Evidence-Based Quality Improvement (EBQI) strategies to implement two distinct EBPs for high-risk Veterans: individual coaching (EBQI-IC; tailored training with individual implementation sites to meet site-specific needs) versus learning collaborative (EBQI-LC; implementation sites trained in groups to encourage collaboration among sites). One EBP, Comprehensive Assessment and Care Planning (CACP), guides teams in addressing patients' cognitive, functional, and social needs through a comprehensive care plan. The other EBP, Medication Adherence Assessment (MAA), addresses common challenges to medication adherence using a patient-centered approach.

METHODS

We will recruit and randomize 16 sites to either EBQI-IC or EBQI-LC to implement one of the EBPs, chosen by the site. Each site will have a site champion (front-line staff) who will participate in 18 months of EBQI facilitation.

ANALYSIS

We will use a mixed-methods type 3 hybrid Effectiveness-Implementation trial to test EBQI-IC versus EBQI-LC versus usual care using a Concurrent Stepped Wedge design. We will use the Practical, Robust Implementation and Sustainability Model (PRISM) framework to compare and evaluate Reach, Effectiveness, Adoption, Implementation, and costs. We will then assess the maintenance/sustainment and spread of both EBPs in primary care after the 18-month implementation period. Our primary outcome will be Reach, measured by the percentage of eligible high-risk patients who received the EBP.

DISCUSSION

Our study will identify which implementation strategy is most effective overall, and under various contexts, accounting for unique barriers, facilitators, EBP characteristics, and adaptations. Ultimately this study will identify ways for primary care clinics and teams to choose implementation strategies that can improve care and outcomes for patients with complex healthcare needs.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT05050643. Registered September 9th, 2021, https://clinicaltrials.gov/study/NCT05050643 PROTOCOL VERSION: This protocol is Version 1.0 which was created on 6/3/2020.

Unpacking Technical Assistance (TA) Strategies Within a State-Level Prevention Support System: A Mixed-Method Study in Determining Types and Amount of TA

The research-practice gap between evidence-based intervention efficacy and its uptake in real-world contexts remains a central challenge for prevention and implementation science. Providing technical assistance (TA) is considered a crucial support mechanism that can help narrow the gap. However, empirical measurement of TA strategies and their variation is often lacking. The current study unpacks the black box of TA, highlighting different TA strategies, amounts, and their relation to intervention characteristics. First, we qualitatively categorized interactions between TA providers and implementers. Second, we explored how characteristics of implementing organizations and the intervention related to variations in the amount of TA delivered. Using data spanning six years, we analyzed over 10,000 encounters between TA providers and implementers. Content analysis yielded four distinct strategies: Consultation (27.2%), Coordination Logistics (24.5%), Monitoring (16.5%), and Resource Delivery (28.2%). Organizations with prior experience required less monitoring and resource delivery. Additionally, characteristics of the intervention were significantly associated with the amount of consultation, monitoring, coordination logistics, and resource delivery provided. The specific features of the intervention showed significant variation in their relation to TA strategies. These findings provide initial insights into the implications of intervention characteristics in determining how much of which TA strategies are needed to support implementations in real-world settings.

Communication Training within Partners in School: Feasibility, Acceptability, and Usability

Partners in School is a consultation model adapted from Conjoint Behavioral Consultation where parents and teachers identify a mutual concern for children with Autism and then implement the same evidence-based practices (EBPs) across home and school. Adding parent-teacher communication training (School Talk) may bolster the effects of this consultation approach. The purpose of this study was to explore parents' and teachers' experiences with School Talk, as well as examine the clinical outcomes of Partners in School plus School Talk. Participants were 21 parents and 21 teachers (n=21 dyads, N=42 participants) of preschool to first-grade children with Autism. Results indicated that parents and teachers both rated School Talk as feasible and acceptable, but parents rated it as more usable. There was preliminary evidence that Partners in School with School Talk may be associated with improvements in parent-teacher communication, as well as reductions in the frequency of child concerns.

Exploring the Impact of External Facilitation Using Evidence-Based Implementation Strategies for Increasing Motivational Interviewing Capacity Among Outpatient Substance Use Disorder (SUD) Treatment Providers

The large majority of individuals who access substance use disorders (SUD) treatment do not receive evidence-based care. Little attention has been paid to the notion that the scale-up of evidence-based practices (EBPs) has been limited in large part due to a weakness in the "distribution system" for bringing new innovations to the attention of practitioners and into practice settings. This study explores the impact of the Training and Practice Implementation Institute (TPII; funded by the New York City Department of Health and Mental Hygiene), an intensive technical assistance initiative that offers external facilitation to outpatient SUD treatment providers via the incorporation of multiple evidence-based implementation strategies to enhance the practice of motivational interviewing (MI). Findings from this study show that staff completed a large majority (86%) of required training/technical assistance (TTA) activities across the 9-month implementation period, demonstrating a high level of engagement among staff and the feasibility of externally facilitated intensive TTA delivered to community-based organizations for the purpose of enhancing implementation of MI for SUDs. Results also show statistically significant improvements in the delivery of MI's technical components among staff, though did not reveal corresponding improvements in the delivery of MI's relational components. An understanding of the potential return on investment associated with externally facilitated TA, offers organizations an opportunity to inform the allocation of limited resources to areas where they will have the greatest impact and ultimately improve the quality and efficacy of services.

Supporting the implementation of written exposure therapy for posttraumatic stress disorder in an obstetrics-substance use disorder clinic in the Northeastern United States

Pregnant people with comorbid posttraumatic stress disorder (PTSD) and substance use disorder (SUD) constitute a highly vulnerable population. PTSD and SUD confer risks to both the pregnant person and the fetus, including a host of physical and mental health consequences. When PTSD and SUD co-occur, potential negative impacts are amplified, and the symptoms of each may exacerbate and maintain the other. Pregnancy often increases engagement in the healthcare system, presenting a unique and critical opportunity to provide PTSD and SUD treatment to birthing people motivated to mitigate risks of losing custody of their children. This paper presents implementation process outcomes of Written Exposure Therapy (WET), a brief, scalable, and sustainable evidence-based PTSD treatment delivered to pregnant persons receiving care in an integrated obstetrical-addiction recovery program at Boston Medical Center. Trial participants (N = 18) were mostly White, non-Hispanic (61.1%), not currently working (77.8%), had a high school or lower level of education (55.5%), had an annual household income less than $35,000 (94.4%), and were living in a substance use residential program (55.6%). We examined intervention feasibility, acceptability, appropriateness, adoption; barriers and facilitators to implementation; and feedback on supporting uptake and sustainability of the intervention using coded qualitative sources (consultation field notes [N = 47] and semi-structured interviews [N = 5]) from providers involved in trial planning and treatment delivery. Results reflected high acceptability, appropriateness, and adoption of WET. Participants described system-, provider-, and patient-level barriers to implementation, offered suggestions to enhance uptake, but did not raise concerns about core components of the intervention. Findings suggest that WET is an appropriate and acceptable PTSD treatment for this difficult-to-reach, complex population, and has the potential to positively impact pregnant persons and their children.

Consultation as an Implementation Strategy to Increase Fidelity of Measurement-Based Care Delivery in Community Mental Health Settings: An Observational Study

Regularly administering outcome measures to clients to inform clinical decision making, referred to as measurement-based care (MBC), has the potential to improve mental health treatment due to its applicability across disorders and treatment settings. Given the utility of MBC, understanding predictors of high-fidelity MBC implementation is important. Training and consultation are widely used implementation strategies to increase the fidelity of evidence-based practice delivery but have rarely been studied with MBC. The current study will examine the relationship between time clinicians spent discussing a case in consultation ("dosage") and MBC fidelity. Thirty clinicians and 56 youth were in the MBC condition of a randomized controlled trial and completed baseline questionnaires. Consultation dosage was extracted from call notes. MBC fidelity was measured using the implementation index, which combines rates of administering and viewing questionnaires, using objective data from the online MBC system. Multi-level modeling was used. Greater consultation dosage significantly predicted a higher implementation index (ß = 0.27, SE = 0.06, p < .001). For every 30 min spent discussing a case in consultation, the case's MBC fidelity increased by 8.1%. Greater consultation dosage significantly predicted higher rates of administration (ß = 0.07, SE = 0.03, p = 0.033) and feedback report viewing (ß = 0.24, SE = 0.06, p < .001). More consultation at the case level predicts greater MBC fidelity using objective measures. The greatest impact was increasing the clinician feedback report viewing rate. The results of this study can help inform future efforts to increase the fidelity with which MBC is delivered and to make consultation as efficient and effective as possible.

Supporting Emotionally Exhausted Community Mental Health Therapists in Appropriately Adapting EBPs for Children and Adolescents

Evidence-based practices (EBPs) are often adapted during community implementation to improve EBP fit for clients and the service context. Augmenting EBPs with additional dosing and content may improve fit. However, reducing EBP content can reduce EBP effectiveness. Using multilevel regression models, this study examined whether supportive program climate and program-furnished EBP-specific implementation strategies (e.g., materials, ongoing training, in-house experts) are associated with augmenting and reducing adaptations, and whether therapist emotional exhaustion moderated these associations. Data were collected from surveys completed by 439 therapists from 102 programs 9 years after a system-driven EBP implementation initiative. Supportive program climate was associated with more augmenting adaptations. Emotional exhaustion was a significant moderator. When organizations used more EBP-specific implementation strategies, more emotionally exhausted therapists reduced EBPs less and less emotionally exhausted therapists augmented EBPs more. Findings provide guidance on how organizations can support appropriate EBP adaptations in spite of therapist emotional exhaustion.

Study protocol for type II hybrid implementation-effectiveness trial of strategies for depression care task-sharing in community health stations in Vietnam: DEP Project

BACKGROUND

It is not clear what the most effective implementation strategies are for supporting the enactment and sustainment of depression care services in primary care settings. This type-II Hybrid Implementation-Effectiveness study will compare the effectiveness of three system-level strategies for implementing depression care programs at 36 community health stations (CHSs) across 2 provinces in Vietnam.

METHODS

In this cluster-randomized controlled trial, CHSs will be randomly assigned to one of three implementation conditions: (1) Usual Implementation (UI), which consists of training workshops and toolkits; (2) Enhanced Supervision (ES), which includes UI combined with bi-weekly/monthly supervision; and (3) Community-Engaged Learning Collaborative (CELC), which includes all components of ES, combined with bi-monthly province-wide learning collaborative meetings, during which cross-site learning and continuous quality improvement (QI) strategies are implemented to achieve better implementation outcomes. The primary outcome will be measured based on the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation quality, and Maintenance) using indicators on implementation, provider, and client factors. The secondary outcome examines factors associated with barriers and facilitators of quality implementation, while the tertiary outcome evaluates the incremental cost-effectiveness ratio of services provided in the ES and CELC conditions, relative to UI condition for depression care. A total of 1,296 clients receiving depression care at CHSs will be surveyed at baseline and 6-month follow-up to assess mental health and psychosocial outcomes (e.g., depression and anxiety severity, health function, quality of life). Additionally, 180 CHS staff and 180 non-CHS staff will complete pre- and post-training evaluation and surveys at baseline, 6, 12, and 24 months.

DISCUSSION

We hypothesize that the additional implementation supports will make mental health service implementation superior in the ES and CELC arms compared to the UI arm. The findings of this project could identify effective implementation models and assess the added value of specific QI strategies for implementing depression care in primary care settings in Vietnam, with implications and recommendations for other low- and middle-income settings. More importantly, this study will provide evidence for key stakeholders and policymakers to consider policies that disseminate, scale up, and advance quality mental health care in Vietnam.

TRIAL REGISTRATION

NCT04491045 on Clinicaltrials.gov. Registered July 29, 2020.

A thematic analysis of broker consultation in a TF-CBT community based learning collaborative

Background

Consultation is an important implementation strategy to increase competence, knowledge, and fidelity to evidence-based practice in community settings. However, the literature has primarily focused on consultation for clinical providers, and less is known about consultation for "broker" professionals, or those who identify and refer children to mental health services. Given their critical role in connecting youth to evidence-based treatment, investigating broker knowledge and use of evidence-based screening and referral are needed.

Objective

To address this gap, the current study examines the content of consultation provided to broker professionals.

Taking a Magnifying Glass to Measurement-Based Care Consultation Sessions: with What Issues Do Mental Health Clinicians Struggle?

PURPOSE

Ongoing consultation following initial training is one of the most commonly deployed implementation strategies to facilitate uptake of evidence-based practices, such as measurement-based care (MBC). Group consultation provides an interactive experience with an expert and colleagues to get feedback on actual issues faced, yet there is little research that unpacks the questions raised in consultation and what types of issues are important to address.

METHODS

The current study characterized the questions and concerns raised by community mental health clinicians (N = 38 across six clinics) during group consultation sessions completed as part of an MBC implementation trial. We conducted a qualitative content analysis of consultation forms completed by clinicians before each MBC consultation session.

RESULTS

Clinicians sought MBC consultation for clients across a range of ages and levels of depression severity. Qualitative results revealed five main questions and concerns in consultation sessions: (1) how to administer the PHQ-9, (2) how to review PHQ-9 scores, (3) how to respond to PHQ-9 score, (4) the types of clients for whom MBC would be appropriate, and (5) how MBC could impact a clinician's usual care.

CONCLUSION

Findings highlight the need for ongoing consultation and limitations of workshop training alone. Practical recommendations for addressing the common questions and concerns identified are presented to support MBC use.

Common Elements Approaches to Implementation Research and Practice: Methods and Integration with Intervention Science

We propose that common elements approaches can advance implementation research and practice and facilitate pragmatic use of intervention and implementation evidence. Common elements are practices or processes frequently shared by interventions or implementations. Traditional common elements methodologies use synthesis, distillation, and statistics to describe and evaluate the merit of common ingredients in effective interventions. Recent developments include identifying and testing common configurations of elements, processes, and context variables across the literature of effective interventions and implementations. While common elements thinking has grown popular in intervention science, it has rarely been utilized in implementation science, and specifically, combined with the intervention literature. The goals of this conceptual methodology paper are to (1) provide an overview of the common elements concept and how it may advance implementation research and usability for practice, (2) give a step-by-step guide to systematic common elements reviews that synthesizes and distills the intervention and implementation literature together, and (3) offer recommendations for advancing element-level evidence in implementation science. A narrative review of the common elements literature was conducted with attention to applications to implementation research. A six-step guide to using an advanced common elements methodology was provided. Examples of potential results are presented, along with a review of the implications for implementation research and practice. Finally, we reviewed methodological limitations in current common elements approaches, and identified steps towards realizing their potential. Common elements methodologies can (a) synthesize and distill the implementation science literature into practical applications, (b) generate evidence-informed hypotheses about key elements and determinants in implementation and intervention processes and mechanisms, and (c) promote evidence-informed precision tailoring of intervention and implementation to context. To realize this potential, common elements approaches need improved reporting of details from both successful and unsuccessful intervention and implementation research, more data availability, and more testing and investigation of causal processes and mechanisms of change from diverse theories.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-023-00077-4.

Intervening for sustainable change: Tailoring strategies to align with values and principles of communities

This paper presents a rationale for tailoring implementation strategies within a values-driven implementation approach. Values-driven implementation seeks to organize implementers around clarifying statements of their shared values in ways that harmonize implementation dynamics related to individual and group mental models, relationships among implementers, and the implementation climate. The proposed approach to tailoring strategies is informed by systems theory and emphasizes the need to focus on both tangible events and behaviors, as well deeper patterns, structures, relationships, and mental models, in order to increase the likelihood of sustaining implementation efforts and improving outcomes for people and communities. We offer for consideration three specific sets of context determinants that are under-represented in the implementation literature and that emerge as especially relevant within a systems approach to identifying and successfully tailoring implementation strategies in the implementation setting including relationships, mental models, and implementation climate.

Mechanisms of change in external implementation support: A conceptual model and case examples to guide research and practice

Background

External implementation support (EIS) is a well-recognized feature of implementation science and practice, often under related terms such as technical assistance and implementation facilitation. Existing models of EIS have gaps related to addressing practice outcomes at both individual and organizational levels, connecting practice activities to intended outcomes, or grounding in well-established theories of behavior and organization change. Moreover, there have been calls to clarify the mechanisms of change through which EIS influences related outcomes.

Method

In this article, we theorize about mechanisms of change within EIS. Our theorizing process aligns with the approach advocated by Kislov et al. We aim to consolidate prior EIS literature, combining related constructs from previous empirical and conceptual work while drawing on our extensive EIS experience to develop a higher-order, midrange theory of change.

Results

Our theory of change is empirically and practically informed, conceptually situated within an established grand theory of change, and guided by eight practice principles and social cognitive theory. The theory of change proposes 10 core practice components as mechanisms of change within EIS. When used according to underlying theory and principles, they are believed to contribute to favorable practice outcomes at individual, team, organizational, and system levels. The model offers flexibility by recognizing the need for sequential support processes and the demand to practice in dynamic and responsive ways. Case examples are presented to illustrate major themes and patterns of the model in action.

Conclusions

The proposed model is intended to support prospective EIS studies by conceptualizing discernable practice components with hypothesized relationships to proximal and distal practice outcomes. The model can be behaviorally operationalized to compliment and extend competency-based approaches to implementation support practitioner (ISP) training and coaching. Over time, the model should be refined based on new empirical findings and contributions from ISPs across the field.

Therapist and supervisor perspectives about two train-the-trainer implementation strategies in schools: A qualitative study

Background

Train-the-trainer (TT) implementation strategies (in which designated clinicians are trained to then train others in an intervention) are promising approaches to support mental health clinician use of evidence-based interventions in school contexts. However, there is little evidence to date examining clinicians' perceptions of the acceptability and feasibility of TT strategies, or comparing clinicians' perceptions of different types of TT strategies.

Methods

The current study was conducted as part of a larger hybrid effectiveness-implementation trial, in which school-based therapists and supervisors received one of two different types of implementation support to implement cognitive behavioral therapy (CBT) groups for anxiety: TT (i.e., initial training for therapists and supervisors) or enhanced TT (TT+; i.e., initial training for therapists and supervisors, and ongoing external consultation for supervisors). We used applied thematic analysis to compare qualitative interview transcripts from 28 therapist interviews and 33 supervisor interviews from therapists and supervisors who received TT or TT+ support and report themes that were similar and different across the two groups.

Results

Most themes were similar across the TT and TT+ conditions: therapists and supervisors in both conditions perceived the group anxiety intervention as acceptable and viewed supervision as acceptable, helpful, and feasible. Therapists and supervisors in both conditions had mixed impressions of the contextual appropriateness of the group anxiety intervention, and some reported logistical challenges with weekly supervision. Some unique themes were identified among the TT+ condition, including supervisors experiencing professional growth, and therapists and supervisors perceiving supervision as critically important and enjoyable.

Conclusions

These results suggest that TT implementation support, using a model in which an internal supervisor receives initial training and then provides ongoing supervision, is acceptable and feasible to support a group CBT intervention in schools. The results also highlight additional benefits that therapists and supervisors perceived when supervisors received ongoing consultation.

Clinical Trial Registration Information

The clinical trial from which these data were derived was registered at ClinicalTrials.gov (https://clinicaltrials.gov/) prior to the time of first patient enrollment. The registration number is: NCT02651402.

Individual and organizational factors as predictors of early evidence-based practice adoption in Michigan high schools: Baseline data from an implementation trial

Background

Adolescents increasingly access mental health services in schools. School mental health professionals (SPs; school counselors, social workers, etc.) can offer evidence-based mental health practices (EBPs) in schools, which may address access gaps and improve clinical outcomes. Although some studies have assessed factors associated with EBP adoption in schools, additional research focusing on SP- and school-level factors is warranted to support EBP implementation as SPs' mental health delivery grows.

Methods

Baseline data were collected from SPs at Michigan high schools participating in a statewide trial to implement SP-delivered cognitive behavioral therapy (CBT) to students. Models examined factors associated with attitudes about EBPs, implementation climate, and implementation leadership, and their associations with CBT knowledge, training attendance, and pre-training CBT delivery.

Results

One hundred ninety-eight SPs at 107 schools (87%) completed a baseline survey. The mean Evidence-Based Practice Attitude Scale (EBPAS) total score was 2.9, and school-aggregated mean scores of the Implementation Climate Scale (ICS) and Implementation Leadership Scale (ILS) were 1.83 and 1.77, respectively, all on a scale ranging from 0 (low) to 4 (high). ICS and ILS scores were lower than typically reported in clinical settings, while EBPAS scores were higher. School characteristics were not significantly associated with EBPAS, ICS, or ILS scores, but scores did differ by SP role. Higher EBPAS scores were associated with more CBT knowledge (average marginal effect for 1 SD change [AME] = 0.15 points) and a higher probability of training completion (AME = 8 percentage points). Higher ICS scores were associated with a higher probability of pre-training CBT delivery (AME = 6 percentage points), and higher ILS scores were associated with higher probability of training completion (AME = 10 percentage points).

Conclusions

Our findings suggest that SPs' attitudes toward EBPs and organizational support were positively associated with early signs of implementation success. As schools increasingly fill the adolescent mental healthcare access gap, efforts to strengthen both provider attitudes toward EBP and strategic organizational factors supporting EBP delivery will be key to encouraging EBP uptake in schools.

Plain Language Summary

Schools are an important setting in which adolescents receive mental healthcare. We need to better understand how to implement evidence-based practices (EBPs) in this setting to improve student mental health. This study examined the attitudes and perceptions of school professionals (SPs) as key contributors to the implementation of a particular EBP, the delivery of cognitive behavioral therapy (CBT) in schools. The study found that implementation climate and leadership scores in participating schools were lower than scores typically reported in clinical settings, while scores for SP attitudes about EBP adoption were higher than typical scores in clinical settings. Results further suggest that SPs with more positive attitudes toward EBPs are more knowledgeable of CBT and more likely to complete a 1-day CBT training. We also found that higher implementation climate scores were associated with SPs reporting pre-training CBT delivery (although this association was not statistically significant), and more implementation leadership was associated with SPs completing the CBT training. These findings suggest that SP attitudes toward EBPs and organizational support in schools are positively associated with early signs of implementation success. Early, low-intensity efforts to (1) improve SP attitudes about mental health EBPs, and (2) increase schools' support for implementation may scaffold more intensive implementation efforts in schools down the road.

Building trusting relationships to support implementation: A proposed theoretical model

This paper presents a theory of change that articulates (a) proposed strategies for building trust among implementation stakeholders and (b) the theoretical linkages between trusting relationships and implementation outcomes. The theory of change describes how trusting relationships cultivate increases in motivation, capability, and opportunity for supporting implementation among implementation stakeholders, with implications for commitment and resilience for sustained implementation, and ultimately, positive implementation outcomes. Recommendations related to the measurement of key constructs in the theory of change are provided. The paper highlights how the development of a testable causal model on trusting relationships and implementation outcomes can provide a bridge between implementation research and implementation practice.

Evaluating the implementation of the active life improving health behavior change program "BCP-VAMOS" in primary health care: Protocol of a pragmatic randomized controlled trial using the RE-AIM and CFIR frameworks

Introduction

The effective translation of evidence-based interventions has contributed to implementing actions that impact public policies and the population's health. However, there is a gap in the literature regarding the factors associated with the successful implementation of these interventions. The Active Life Improving Health Behavior Change Program (BCP-VAMOS) uses behavioral strategies to promote an active and healthy lifestyle in the community. Characterized as a health innovation, it also provides health professionals with online training to implement the program in Primary Health Care (PHC). Our study describes a pragmatic trial that aims to evaluate the implementation of BCP-VAMOS, version 3.0, in PHC in southern Brazil.

Methods and analysis

A pragmatic randomized controlled trial (PRCT) of two arms comparing a group of PHC professionals who will participate in a traditional didactic approach (control group) vs. a group that will receive ongoing support (intervention group) for the implementation of BCP-VAMOS. The intervention will be available to adults (≥18 years old) registered at PHC. Program recipient's will be assessed at baseline and post-intervention (9 months after) to measure markers of physical activity and eating behavior (primary outcomes). Program's implementation process will be monitored for 12 months and will be evaluated using the RE-AIM and Consolidated Framework for Implementation Research (CFIR) frameworks.

Discussions

The survey findings can be used widely throughout Brazil, guiding the work of health professionals, service planners and policy-makers. Also, the results may help to inform the national health promotion policy to plan interventions and improve the implementation of programs in PHC. This research results will provide practical guidance for researchers to develop similar protocols to implement and adapt public health interventions.

Ethics and dissemination

Ethics approval has been granted by the Human Research Ethics Committee of the Federal University of Santa Catarina (UFSC), Brazil, under no. 1394492. Results will be published in full as open access in the UFSC library and main trial results and associated papers in high-impact peer-reviewed journals.

Trial registration number

RBR-2vw77q-Brazilian Registry of Clinical Trials - ReBEC (http://www.ensaiosclinicos.gov.br).

Consultation and professional development within the prolonged exposure initiative

Previous studies have identified that ongoing consultation and organizational support are key factors in sustaining evidence-based implementation initiatives in community mental health. Clinician-level factors likely influence implementation but are not well delineated. This study assessed clinician-reported professional development incentives for participating in consultation in prolonged exposure (PE) therapy to guide future evidence-based practice (EBP) initiatives. Providers in the PE initiative (N = 22) completed a survey asking about their motivations to participate, the impact of consultation on the implementation of PE and their professional development goals, and the impact of organizational-level support on their goals and on organizational-level clinician turnover. Of the respondents,   10 had decided to pursue additional training to become PE consultants at their agencies. Providers (response rate = 22/35) reported joining the PE Initiative to learn an evidence-based practice in general (not PE specifically) and increase their skill in treating posttraumatic stress disorder. Providers largely found ongoing consultation helpful and reported that consultation provided the feedback and support necessary to achieve their professional goals. Providers who decided to pursue PE consultant training reported that expanding upon supervision and clinical skills were strong motivators. Providers largely attributed clinician turnover within the PE Initiative to organizational-level factors. Ongoing consultation facilitated providers' professional development goals. The survey identified several areas for further exploration, including how consultation can reduce clinician turnover, and how taking on additional responsibility within implementation initiatives can facilitate provider engagement in the EBP.

Developing the Workforce of the Digital Future: Leveraging Technology to Train Community-Based Mobile Mental Health Specialists

Challenges in training, dissemination, and implementation have impeded the ability of providers to integrate promising digital health tools in real-world services. There is a need for generalizable strategies to rapidly train real-world providers at scale to support the adoption of digital health. This study describes the development of principles guiding rapid training of community-based clinicians in the support of digital health. This training approach was developed in the context of an ongoing trial examining implementation strategies for FOCUS, a mobile mental health intervention designed for people with serious mental illness. The SAIL (Simple, Accessible, Inverted, Live) model introduces how digital tools can be leveraged to facilitate rapid training of community agency-based personnel to serve as digital mental health champions, promoters, and providers. This model emphasizes simple and flexible principles of intervention delivery, accessible materials in a virtual learning environment, inverted or "flipped" live training structure, and live consultation calls for ongoing support. These initial insights lay the groundwork for future work to test and replicate generalizable training strategies focused on real-world delivery of digital mental health services. These strategies have the potential to remove key obstacles to the implementation and dissemination of digital health interventions for mental health.

Leveraging academic initiatives to advance implementation practice: a scoping review of capacity building interventions

BACKGROUND

Academic institutions building capacity for implementation scholarship are also well positioned to build capacity in real world health and human service settings. How practitioners and policy makers are included and trained in implementation capacity-building initiatives, and their impact on building implementation practice capacity is unclear. This scoping review identified and examined features of interventions that build implementation practice capacity across researchers and practitioners or practitioners-in-training.

METHODS

Five bibliographic databases were searched. Eligible studies (a) described an implementation capacity building intervention with a connection to an academic institution, (b) targeted researchers and practitioners (including practitioners-in-training, students, or educators), and (c) reported intervention or participant outcomes. Articles that only described capacity building interventions without reporting outcomes were excluded. Consistent with Arksey and O'Malley's framework, key study characteristics were extracted (target participants, core components, and outcomes) and analyzed using open coding and numerical analysis.

RESULTS

Of 1349 studies identified, 64 met eligibility for full-text review, and 14 were included in the final analysis. Half of the studies described implementation capacity building interventions that targeted health or behavioral health researchers, practitioners, and practitioners-in-training together, and half targeted practitioners or practitioners-in-training only. The most common components included structured didactic activities offered in person or online, mentorship and expert consultation to support implementation, and practical application activities (e.g., field placements, case studies). Knowledge sharing activities and technical assistance were less common. All studies reported favorable outcomes related to knowledge attainment, increased ability to implement evidence, productivity, and satisfaction.

CONCLUSIONS

Building implementation capacity among practitioners is critical for integrating insights from implementation science into the field and preventing the "secondary" implementation research-to-practice gap. This scoping review identified several promising implementation practice capacity building interventions that tend to build practitioner capacity via expert led activities which may be relevant for academic institutions seeking to build implementation practice capacity. To avoid widening the implementation research-to-practice gap, implementation capacity building interventions are needed that target policy makers, expand beyond multiple practice settings, and leverage university/community partnerships or on-site academic medical centers. Future studies will also be needed to test the impact on service quality and public health outcomes.

How low can you go? Examining the effects of brief online training and post-training consultation dose on implementation mechanisms and outcomes for measurement-based care

BACKGROUND

Initial training and ongoing post-training consultation (i.e., ongoing support following training, provided by an expert) are among the most common implementation strategies used to change clinician practice. However, extant research has not experimentally investigated the optimal dosages of consultation necessary to produce desired outcomes. Moreover, the degree to which training and consultation engage theoretical implementation mechanisms-such as provider knowledge, skills, and attitudes-is not well understood. This study examined the effects of a brief online training and varying dosages of post-training consultation (BOLT+PTC) on implementation mechanisms and outcomes for measurement-based care (MBC) practices delivered in the context of education sector mental health services.

METHODS

A national sample of 75 clinicians who provide mental health interventions to children and adolescents in schools were randomly assigned to BOLT+PTC or control (services as usual). Those in BOLT+PTC were further randomized to 2-, 4-, or 8-week consultation conditions. Self-reported MBC knowledge, skills, attitudes, and use (including standardized assessment, individualized assessment, and assessment-informed treatment modification) were collected for 32 weeks. Multilevel models were used to examine main effects of BOLT+PTC versus control on MBC use at the end of consultation and over time, as well as comparisons among PTC dosage conditions and theorized mechanisms (skills, attitudes, knowledge).

RESULTS

There was a significant linear effect of BOLT+PTC over time on standardized assessment use (b = .02, p < .01), and a significant quadratic effect of BOLT+PTC over time on individualized assessment use (b = .04, p < .001), but no significant effect on treatment modification. BOLT + any level of PTC resulted in higher MBC knowledge and larger growth in MBC skill over the intervention period as compared to control. PTC dosage levels were inconsistently predictive of outcomes, providing no clear evidence for added benefit of higher PTC dosage.

CONCLUSIONS

Online training and consultation in MBC had effects on standardized and individualized assessment use among clinicians as compared to services as usual with no consistent benefit detected for increased consultation dosage. Continued research investigating optimal dosages and mechanisms of these established implementation strategies is needed to ensure training and consultation resources are deployed efficiently to impact clinician practices.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05041517 . Retrospectively registered on 10 September 2021.

Primary aim results of a clustered SMART for developing a school-level, adaptive implementation strategy to support CBT delivery at high schools in Michigan

BACKGROUND

Schools increasingly provide mental health services to students, but often lack access to implementation strategies to support school-based (and school professional [SP]) delivery of evidence-based practices. Given substantial heterogeneity in implementation barriers across schools, development of adaptive implementation strategies that guide which implementation strategies to provide to which schools and when may be necessary to support scale-up.

METHODS

A clustered, sequential, multiple-assignment randomized trial (SMART) of high schools across Michigan was used to inform the development of a school-level adaptive implementation strategy for supporting SP-delivered cognitive behavioral therapy (CBT). All schools were first provided with implementation support informed by Replicating Effective Programs (REP) and then were randomized to add in-person Coaching or not (phase 1). After 8 weeks, schools were assessed for response based on SP-reported frequency of CBT delivered to students and/or barriers reported. Responder schools continued with phase 1 implementation strategies. Slower-responder schools (not providing ≥ 3 CBT components to ≥10 students or >2 organizational barriers identified) were re-randomized to add Facilitation to current support or not (phase 2). The primary aim hypothesis was that SPs at schools receiving the REP + Coaching + Facilitation adaptive implementation strategy would deliver more CBT sessions than SPs at schools receiving REP alone. Secondary aims compared four implementation strategies (Coaching vs no Coaching × Facilitation vs no Facilitation) on CBT sessions delivered, including by type (group, brief and full individual). Analyses used a marginal, weighted least squares approach developed for clustered SMARTs.

RESULTS

SPs (n = 169) at 94 high schools entered the study. N = 83 schools (88%) were slower-responders after phase 1. Contrary to the primary aim hypothesis, there was no evidence of a significant difference in CBT sessions delivered between REP + Coaching + Facilitation and REP alone (111.4 vs. 121.1 average total CBT sessions; p = 0.63). In secondary analyses, the adaptive strategy that offered REP + Facilitation resulted in the highest average CBT delivery (154.1 sessions) and the non-adaptive strategy offering REP + Coaching the lowest (94.5 sessions).

CONCLUSIONS

The most effective strategy in terms of average SP-reported CBT delivery is the adaptive implementation strategy that (i) begins with REP, (ii) augments with Facilitation for slower-responder schools (schools where SPs identified organizational barriers or struggled to deliver CBT), and (iii) stays the course with REP for responder schools.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03541317 , May 30, 2018.

A Behavior-based Coding Tool for Assessing Supervisors' Adherence and Competence: Findings From a Motivational Interviewing Implementation Study

Supervision seems to be an essential part of therapist training, and thus also of implementing evidence-based practices. However, there is a shortage of valid and reliable instruments for objective assessment of supervision competence that include both global measures and frequency counts of behavior - two essential aspects of supervisory competence. This study tests the internal consistency and inter-rater reliability of an assessment tool that includes both these measures. Additionally, strategies and techniques used by ten supervisors in 35 Motivational interviewing supervision sessions are described. Codings were conducted after two separate coding training sessions. The internal consistency across the global measures was acceptable (α = 0.70; 0.71). After the second training, the inter-rater reliabilities for all frequency counts were in the moderate to good range, except for two that were in the poor range; inter-rater reliability for one of the four global measures was in the moderate range, and three were in the poor range. A prerequisite for identifying specific supervisor skills central to the development of therapist skills, teaching these skills to supervisors, and performing quality assurance of supervision, is to create instruments that can measure these behaviors. This study is a step in that direction.

Care team and practice-level implementation strategies to optimize pediatric collaborative care: study protocol for a cluster-randomized hybrid type III trial

BACKGROUND

Implementation facilitation is an effective strategy to support the implementation of evidence-based practices (EBPs), but our understanding of multilevel strategies and the mechanisms of change within the "black box" of implementation facilitation is limited. This implementation trial seeks to disentangle and evaluate the effects of facilitation strategies that separately target the care team and leadership levels on implementation of a collaborative care model in pediatric primary care. Strategies targeting the provider care team (TEAM) should engage team-level mechanisms, and strategies targeting leaders (LEAD) should engage organizational mechanisms.

METHODS

We will conduct a hybrid type 3 effectiveness-implementation trial in a 2 × 2 factorial design to evaluate the main and interactive effects of TEAM and LEAD and test for mediation and moderation of effects. Twenty-four pediatric primary care practices will receive standard REP training to implement Doctor-Office Collaborative Care (DOCC) and then be randomized to (1) Standard REP only, (2) TEAM, (3) LEAD, or (4) TEAM + LEAD. Implementation outcomes are DOCC service delivery and change in practice-level care management competencies. Clinical outcomes are child symptom severity and quality of life.

DISCUSSION

This statewide trial is one of the first to test the unique and synergistic effects of implementation strategies targeting care teams and practice leadership. It will advance our knowledge of effective care team and practice-level implementation strategies and mechanisms of change. Findings will support efforts to improve common child behavioral health conditions by optimizing scale-up and sustainment of CCMs in a pediatric patient-centered medical home.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT04946253 . Registered June 30, 2021.

The "case" for case studies: why we need high-quality examples of global implementation research

Rigorous and systematic documented examples of implementation research in global contexts can be a valuable resource and help build research capacity. In the context of low- and middle-income countries (LMICs), there is a need for practical examples of how to conduct implementation studies. To address this gap, Fogarty’s Center for Global Health Studies in collaboration with the Cincinnati Children's Hospital Medical Center and the National Cancer Institute is commissioning a collection of implementation science case studies in LMICs that describe key components of conducting implementation research, including how to select, adapt, and apply implementation science models, theories, and frameworks to these settings; develop and test implementation strategies; and evaluate implementation processes and outcomes. The case studies describe implementation research in various disease areas in LMICs around the world. This commentary highlights the value of case study methods commonly used in law and business schools as a source of “thick” (i.e., context-rich) description and a teaching tool for global implementation researchers. It addresses the independent merit of case studies as an evaluation approach for disseminating high-quality research in a format that is useful to a broad range of stakeholders. This commentary finally describes an approach for developing high-quality case studies of global implementation research, in order to be of value to a broad audience of researchers and practitioners.

A mixed-methods study to understand the impact of practitioner and organizational factors on fidelity of a child maltreatment prevention intervention in community-based settings

It has been well-documented that the degree to which interventions are implemented with fidelity in typical service settings has varied. Frequently, interventions are developed and tested in highly controlled or early adopter settings. Less attention has been given to what implementation looks like in usual care, and which factors promote practitioners' ability to implement with fidelity. Individuals and organizations implementing interventions in the real world receive varying levels of external supports and may apply a new intervention unaided. The purpose of this mixed-methods study was to explore factors that support implementation as intended in local community agencies. In the quantitative phase of this study, 32 case planners implementing Family Connections (FC), a child maltreatment preventive intervention, completed a survey about their perceptions of practitioner and organizational factors related to fidelity. The survey data were connected to case-level fidelity scores to understand the relationship between perceptions and fidelity. The qualitative phase of this study involved further exploration with nine case planner interviews and two separate focus groups with supervisors and agency leadership. The results of this study suggest that supervision is a key contributor to a practitioner's ability to implement an intervention in usual care. The quantitative and qualitative results suggest supervision, including supervisors’ perseverance, proactiveness, knowledge, availability, and skill reinforcement are important components of enhancing a practitioners' ability to learn and use FC. The quantitative results suggest that the level of education was positively associated with fidelity and perceptions of the intervention's limitations may be negatively related to implementation. Additional components that influence implementation for future research emerged from the qualitative phase related to system expectations and policies, individual practitioner attributes, and characteristics of the intervention.

Plain Language Abstract

This mixed-methods study sought to understand the impact of practitioner and organizational factors on fidelity of a child maltreatment prevention intervention in community-based settings. The study first asked case planners about their perceptions of practitioner and organizational factors related to fidelity through an online survey. This survey was connected to case-level fidelity scores to understand the relationship between perceptions and fidelity. The qualitative phase of this study involved further exploration with nine case planner interviews and two separate focus groups with supervisors and agency leadership. The results of this study suggest that supervision is a key contributor to a practitioner’s ability to implement a maltreatment prevention intervention. Both methods of the study suggest that various aspects of supervision, including supervisors’ perseverance, proactiveness, knowledge, availability, and skill reinforcement are important components of enhancing a practitioner's ability to learn and use the intervention. Additional components that influence the implementation for future research emerged from the qualitative phase related to system expectations and policies, individual practitioner attributes, and characteristics of the intervention.

Using Total Worker Health® Implementation Guidelines to Design an Organizational Intervention for Low-Wage Food Service Workers: The Workplace Organizational Health Study

Total Worker Health® (TWH) interventions that utilize integrated approaches to advance worker safety, health, and well-being can be challenging to design and implement in practice. This may be especially true for the food service industry, characterized by high levels of injury and turnover. This paper illustrates how we used TWH Implementation Guidelines to develop and implement an organizational intervention to improve pain, injury, and well-being among low-wage food service workers. We used the Guidelines to develop the intervention in two main ways: first, we used the six key characteristics of an integrated approach (leadership commitment; participation; positive working conditions; collaborative strategies; adherence; data-driven change) to create the foundation of the intervention; second, we used the four stages to guide integrated intervention planning. For each stage (engaging collaborators; planning; implementing; evaluating for improvement), the Guidelines provided a flexible and iterative process to plan the intervention to improve safety and ergonomics, work intensity, and job enrichment. This paper provides a real-world example of how the Guidelines can be used to develop a complex TWH intervention for food service workers that is responsive to organizational context and addresses targeted working conditions. Application of the Guidelines is likely transferable to other industries.

Advancing implementation frameworks with a mixed methods case study in child behavioral health

Despite a growing policy push for the provision of services based on evidence, evidence-based treatments for children and youth with mental health challenges have poor uptake, yielding limited benefit. With a view to improving implementation in child behavioral health, we investigated a complementary implementation approach informed by three implementation frameworks in the context of implementing motivational interviewing in four child and youth behavioral health agencies: the Active Implementation Frameworks (AIF) (process), the Consolidated Framework for Implementation Research (factors), and the Implementation Outcomes Framework (evaluation). The study design was mixed methods with embedded interrupted time series and motivational interviewing (MI) fidelity was the primary outcome. Focus groups and field notes informed perspectives on the implementation approach, and a questionnaire explored the salience of Consolidated Framework for Implementation Research (CFIR) factors. Findings validate the process guidance provided by the AIF and highlight CIFR factors related to implementation success. Novel CFIR factors, not elsewhere reported in the literature, are identified that could potentially extend the framework if validated in future research. Introducing fidelity measurement in practice proved challenging and was not sustained beyond the study. A complementary implementation approach was successful in implementing MI in child behavioral health agencies. In contrast with the typical train and hope approach to implementation, practice change did not occur immediately post-training but emerged over a 7 month period of consultation and practice following a discrete interactive training period. The saliency of CFIR constructs aligned with findings from studies conducted in other contexts, demonstrating external validity and highlighting common factors that can focus planning and measurement.

The Cognitive Walkthrough for Implementation Strategies (CWIS): a pragmatic method for assessing implementation strategy usability

BACKGROUND

Implementation strategies have flourished in an effort to increase integration of research evidence into clinical practice. Most strategies are complex, socially mediated processes. Many are complicated, expensive, and ultimately impractical to deliver in real-world settings. The field lacks methods to assess the extent to which strategies are usable and aligned with the needs and constraints of the individuals and contexts who will deliver or receive them. Drawn from the field of human-centered design, cognitive walkthroughs are an efficient assessment method with potential to identify aspects of strategies that may inhibit their usability and, ultimately, effectiveness. This article presents a novel walkthrough methodology for evaluating strategy usability as well as an example application to a post-training consultation strategy to support school mental health clinicians to adopt measurement-based care.

METHOD

The Cognitive Walkthrough for Implementation Strategies (CWIS) is a pragmatic, mixed-methods approach for evaluating complex, socially mediated implementation strategies. CWIS includes six steps: (1) determine preconditions; (2) hierarchical task analysis; (3) task prioritization; (4) convert tasks to scenarios; (5) pragmatic group testing; and (6) usability issue identification, classification, and prioritization. A facilitator conducted two group testing sessions with clinician users (N = 10), guiding participants through 6 scenarios and 11 associated subtasks. Clinicians reported their anticipated likelihood of completing each subtask and provided qualitative justifications during group discussion. Following the walkthrough sessions, users completed an adapted quantitative assessment of strategy usability.

RESULTS

Average anticipated success ratings indicated substantial variability across participants and subtasks. Usability ratings (scale 0-100) of the consultation protocol averaged 71.3 (SD = 10.6). Twenty-one usability problems were identified via qualitative content analysis with consensus coding, and classified by severity and problem type. High-severity problems included potential misalignment between consultation and clinical service timelines as well as digressions during consultation processes.

CONCLUSIONS

CWIS quantitative usability ratings indicated that the consultation protocol was at the low end of the "acceptable" range (based on norms from the unadapted scale). Collectively, the 21 resulting usability issues explained the quantitative usability data and provided specific direction for usability enhancements. The current study provides preliminary evidence for the utility of CWIS to assess strategy usability and generate a blueprint for redesign.

Therapy Dogs in Educational Settings: Guidelines and Recommendations for Implementation

Therapy dogs in educational settings have gained increase traction in recent years. Despite its potential benefits and popularity, there remain concerns of perceived risks such as sanitation, allergies, and safety issues, as well as a lack of guidelines, regulations and support resources available to school staff. Research is further lacking into the implementation process of therapy dog programs in educational settings. To construct a set of recommendations for school staff to achieve successful implementation of a therapy dog program, the present study investigated the perceived facilitative and impeding factors when implementing a therapy dog program. A total of 13 school staff and 2 coordinators from therapy dog organisations took part in an open-ended online survey and/or a semi-structured interview over the phone, with the aim of gathering their perspectives of implementing a therapy dog program in schools. The thematic analysis of the data indicated facilitative factors such as program flexibility, whole-school support, the need for communication and training for all staff, as well as dog welfare. Successful implementation of therapy dogs in an education setting appear to revolve around (1) flexibility of the dog therapy program to target school's needs, (2) qualities of program instigator, (3) whole-school support, (4) communication, training and education, (5) considerations for dog's welfare. Key barriers identified included a high workload, lack of guidelines on processes, lack of support from the school community, as well as the need for better understanding of the role of a therapy dog. The results highlighted the importance of a whole-school effort when implementing a therapy dog program, as well as the need for guidelines for assessing school readiness, key factors for consideration, and strategies to overcome challenges associated with program implementation.

Implementation of clinical pharmacy services using problematization with Maguerez Arc: A quasi-experimental before-after study

RATIONALE, AIMS, AND OBJECTIVES

The growth of clinical pharmacy services (CPS) has positively impacted clinical, economic, and humanistic health outcomes. However, detailed studies on the process of implementing CPS in healthcare systems are incipient. Thus, the present study aimed to evaluate the CPS implementation in certain public health units in a metropolis in northeast Brazil.

METHODS

A quasi-experimental before-and-after study was carried out in Recife City, from July 2015 to March 2016. The study was carried out using the Methodology of Problematization with Maguerez Arc and was divided into: initial evaluation (before), planning, interventions, and preliminary evaluation (after). The participants were pharmacists, patients, health professionals, and local health managers. Descriptive statistics were used to report data. The statistical significance of the comparison between variables was evaluated using the Wilcoxon test (95% CI; P ≤ .05).

RESULTS

Initial evaluation: Identified incipient CPS, a lack of structure and work process of pharmacists. Planning: Sixteen brainstorming meetings were held with the different actors resulting in a strategic plan.

INTERVENTION

Twenty-two political-administrative meetings were held with managers and health teams and 768 hours of theoretical and practical training with mentoring for pharmacists. Preliminary evaluation: Structure indicators presented a statistically significant difference, differently from the process indicators. Pharmacists attended 842 patients and performed 1465 pharmaceutical consultations in 6 months. Regarding the outcome indicators, it was possible to identify changes in the clinical status of the most prevalent diseases among those patients who attended three pharmaceutical consultations.

CONCLUSIONS

It was possible to evaluate the CPS implementation in certain public health units in a metropolis in Brazil, through the Methodology of Problematization with Maguerez Arc. This methodology may be part of models for future implementations of CPS in health systems.

Clinical Supervision in Community Mental Health: Characterizing Supervision as Usual and Exploring Predictors of Supervision Content and Process

Clinical supervision can be leveraged to support implementation of evidence-based practices in community mental health settings, though it has been understudied. This study focuses on 32 supervisors at 23 mental health organizations in Philadelphia. We describe characteristics of supervisors and organizations and explore predictors of supervision content and process. Results highlight a low focus on evidence-based content and low use of active supervision processes. They underscore the need for further attention to the community mental health context when designing supervision-targeted implementation strategies. Future work should assess whether supervision models specific to community mental health are needed.

Making Implementation Last: The Impact of Training Design on the Sustainability of an Evidence-Based Treatment in a Randomized Controlled Trial

Although advances have been made in facilitating the implementation of evidence-based treatments, little is known about the most effective way to sustain their use over time. The current study examined the sustainability of one evidence-based treatment, Parent-Child Interaction Therapy (PCIT), following a statewide implementation trial testing three training methods: Cascading Model, Learning Collaborative, and Distance Education. Participants included 100 clinicians and 50 administrators from 50 organizations across Pennsylvania. Clinicians and administrators reported on sustainability at 24-months, as measured by the number of clients receiving PCIT and the continued use of the PCIT protocol. Multi-level path analysis was utilized to examine the role of training on sustainability. Clinicians and administrators reported high levels of sustainability at 24-months. Clinicians in the Cascading Model reported greater average PCIT caseloads at 24-months, whereas clinicians in the Learning Collaborative reported greater full use of the PCIT protocol at 24-months. Attending consultation calls was associated with delivering PCIT to fewer families. Implications for the sustainable delivery of PCIT beyond the training year as well as for the broader field of implementation science are discussed.

Implementing Intergenerational Best Practices in Community-Based Settings: A Preliminary Study

Evidence-based intergenerational (IG) programs bring youth and older adults together in shared activities and promote socioemotional health across age-groups. The core components of these IG programs include 14 IG "best practices" that should be implemented during IG program sessions to optimize program effectiveness for both youth and older adult participants. Despite the proliferation of IG programs across the United States, it is unclear the extent to which these IG best practices have been implemented in the community. This preliminary study assesses the implementation of IG best practices at two community-based sites by program leaders who participated in a multifaceted professional education intervention for IG best practice use. Implementation of best practices was measured through the Best Practices Checklist completed by program leaders and trained coders as well as through narrative written comments. Program leaders indicated that they were able to consistently implement six out of the 14 IG best practices in 46 IG sessions, whereas the best practice named "Adaptations to equipment were made" was least likely to be implemented. Analysis of narrative comments indicated that (a) the group arrangement of participants and (b) program leaders' familiarity with activities also influenced implementation. While many IG best practices can be implemented in the community, some best practices can be implemented with greater ease and consistency. Training resources can support IG best practice implementation; however, our multifaceted professional education intervention may benefit from the addition of case examples or vignettes to depict potential strategies for optimizing evidence-based IG practices.

Strategies that Promote Therapist Engagement in Active and Experiential Learning: Micro-Level Sequential Analysis

Therapists' active learning increases treatment fidelity, but research is needed on supervisory strategies to engage therapists in active learning. This study used sequential analysis to examine consultant behaviors associated with increased and decreased probability of eliciting therapists' active learning. The study included 162 consultation sessions from 27 community therapists implementing Attachment and Biobehavioral Catch-up. Consultants' client discussion, information provision, and modeling were associated with reduced likelihood of active learning. Consultants' questions, engagement in active learning strategies, use of video, and silence were associated with greater likelihood of therapist active learning. These findings inform supervisors' attempts to encourage active learning.

A Glimpse into the "Black Box": Which Elements of Consultation in an EBP are Associated with Client Symptom Change and Therapist Fidelity?

Background: Consultation is an implementation strategy that improves delivery and clinical outcomes for Cognitive Processing Therapy (CPT), an evidence-based practice (EBP) for posttraumatic stress disorder (PTSD). However, little is known about the specific components of consultation that influence the fidelity of treatment delivery or clinical outcomes. Methods: The current study examined whether specific activities performed during CPT consultation meetings were associated with better fidelity to the CPT protocol among 60 newly trained therapists or improved clinical outcomes among 135 clients treated by these therapists. Consultation activities that fall under three broad categories (discussion of the application of CPT to individual cases, review/feedback on fidelity, and technical difficulties) were measured by consultant checklists for each consultation session. Treatment fidelity (adherence to the protocol and competence of delivery) was rated by trained observers for a random sample of therapists' CPT sessions following consultation. The self-reported PTSD Checklist-IV assessed PTSD symptom change. Results: Multilevel regression analyses indicated that higher therapist consultation attendance predicted a greater decrease in their clients' PTSD symptoms and that attendance was not associated with observer-rated treatment fidelity. Discussion of the application of specific CPT strategies was the only consultation activity that was significantly associated with greater improvement in PTSD symptoms. Lastly, no consultation activities were significantly associated with treatment fidelity. Conclusions: Our findings suggest that specific consultation strategies such as emphasizing the discussion of the application of specific CPT strategies to individual cases during consultation meetings may be effective in improving the clinical outcomes of CPT.

Parent Training for Youth with Autism Served in Community Settings: A Mixed-Methods Investigation Within a Community Mental Health System

Parent training programs focus on parent knowledge and/or skill development regarding strategies to improve child outcomes. Parent training programs are considered evidenced-based treatments for autism spectrum disorder (ASD). Yet little is known about parent training use for youth with ASD served in community settings. This mixed methods project examined parent training for Medicaid-enrolled youth with ASD under age 21. Data were obtained from Medicaid claims for 879 youth and surveys from 97 applied behavior analysis (ABA) providers. Open-ended survey items were analyzed with content analysis. Results demonstrated that the frequency of parent training was low and providers' conceptualization of parent training was inconsistent with evidence-based models. Providers are largely unaware of evidence-based components (i.e., modeling, caregiver practice with feedback) and use them infrequently. Implications for increasing parent training in community settings are discussed.

Telepsychiatric Consultation as a Training and Workforce Development Strategy for Rural Primary Care

PURPOSE

There is a shortage of rural primary care personnel with expertise in team care for patients with common mental disorders. Building the workforce for this population is a national priority. We investigated the feasibility of regular systematic case reviews through telepsychiatric consultation, within collaborative care for depression, as a continuous training and workforce development strategy in rural clinics.

METHODS

We developed and pilot-tested a qualitative interview guide based on a conceptual model of training and learning. We conducted individual semistructured interviews in 2018 with diverse clinical and nonclinical staff at 3 rural primary care sites in Washington state that used ongoing collaborative care and telepsychiatric consultation. Two qualitative researchers independently analyzed transcripts with iterative input from other research team members.

RESULTS

A total of 17 clinical, support, and administrative staff completed interviews. Participants' feedback supported the view that telepsychiatric case review-based consultation enhanced skills of diverse clinical team members over time, even those who had not directly participated in case reviews. All interviewees identified specific ways in which the consultations improved their capacity to identify and treat psychiatric disorders. Perceived benefits in implementation and sustainability included fidelity of the care process, team resilience despite member turnover, and enhanced capacity to use quality improvement methods.

CONCLUSIONS

Weekly systematic case reviews using telepsychiatric consultation served both as a model for patient care and as a training and workforce development strategy in rural primary care sites delivering collaborative care. These are important benefits to consider in implementing the collaborative care model of behavioral health integration.

Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes

A lack of effective therapist training is a major barrier to evidence-based intervention (EBI) delivery in the community. Systematic reviews published nearly a decade ago suggested that traditional EBI training leads to higher knowledge but not more EBI use, indicating that more work is needed to optimize EBI training and implementation. This systematic review synthesizes the training literature published since 2010 to evaluate how different training models (workshop, workshop with consultation, online training, train-the-trainer, and intensive training) affect therapists' knowledge, beliefs, and behaviors. Results and limitations for each approach are discussed. Findings show that training has advanced beyond provision of manuals and brief workshops; more intensive training models show promise for changing therapist behavior. However, methodological issues persist, limiting conclusions and pointing to important areas for future research.

How can mobile applications support suicide prevention gatekeepers in Australian Indigenous communities?

RATIONALE

Suicide prevention training in Aboriginal and Torres Strait Islander communities is a national health priority in Australia.

OBJECTIVE

This paper describes a qualitative study to increase understanding of how a mobile application (app) could be used to support suicide prevention gatekeepers in Indigenous communities. We respectfully use the term Indigenous to refer to Australian peoples of Aboriginal and/or Torres Strait Islander descent.

METHOD

Two participatory design workshops were held with 12 participants who were either Indigenous health workers or community members. The workshops first explored what knowledge, skills, and support suicide prevention gatekeepers in Indigenous communities may require, as well as how technology, specifically mobile apps, could be used to support these needs.

RESULTS

Qualitative analysis identified four themes related to perceptions of who gatekeepers are, their role requirements, technology and supporting resources, as well as broader community issues. Participants thought training programs should target key, accessible, and respected people from diverse, designated, and emergent groups in Indigenous communities to act as gatekeepers, but requested an alternative, more culturally appropriate term to 'gatekeeper' (e.g., responder). Training should prepare gatekeepers for multifaceted suicide prevention roles, including the identification and management of at-risk Indigenous persons, the provision of psychoeducation and ongoing support, as well as facilitate integrated care in collaboration with community services. A combination of multiple support resources was recommended, including multi-platform options in the technology (e.g., mobile applications, social media) and physical domains (e.g., wallet cards, regular meetings). Recommended app features included culturally appropriate refresher content on suicide intervention, training recall, integrated care, how to access gatekeeper peer support, and debriefing. Broader community concerns on gatekeeper support needs were also considered.

Implementation of targeted mental health interventions in urban schools: Preliminary findings for impact of training strategy on program fidelity

School-based mental health programs are increasingly recognized as methods by which to improve children's access to evidence-based practices (EBPs), particularly in urban under resourced communities. School-wide positive behavior interventions and supports (PBIS) is one approach to integrating mental health services into school-based programming; however, school providers require training and support to implement programs as intended. We have conducted a randomized controlled trial to compare two models for training school-based personnel to deliver group EBPs to children at high risk of developing internalizing or externalizing problems. School personnel (N = 24) from 6 schools in a large urban school district were trained with either a basic training and consultation strategy, or an enhanced training and consultation strategy. Preliminary findings show that the enhanced strategy resulted in 9% higher content fidelity than the basic strategy. School personnel who were switched to the basic strategy had slightly lower content fidelity for the last two years of the trial and school personnel who continued to receive basic consultation during the step-down phase saw their fidelity decline. The two conditions did not differ with regard to process fidelity.

‘We struggle and muddle.’ A qualitative study exploring community ABI therapists’ experiences of using, training in and implementing behaviour interventions

Objectives:

Individuals with acquired brain injury (ABI) may present with challenging behaviours (CB) that place themselves and others at risk of harm and impact their community integration. It is crucial for community ABI therapists to successfully train in and implement behaviour interventions. The current study aimed to investigate community ABI therapists’ experiences of using, training in and implementing behaviour interventions. An additional aim was to determine these therapists’ understanding of Positive Behaviour Support (PBS), one approach to addressing CB with a focus on improving quality of life.

Methods:

Semi-structured interviews were conducted with 24 Australian community ABI therapists about their experiences of using, training in and implementing behaviour interventions and understanding of PBS. Inductive thematic analysis and content analysis were performed on interview transcripts.

Findings:

The thematic analysis resulted in the generation of six themes which described the difficulties participants faced in training in and delivering behaviour interventions and identified their training and implementation needs. The content analysis resulted in 10 categories that characterised participants’ understanding of PBS, which centred around the absence of consequences, a focus on antecedents, person-centred practice and encouraging prosocial alternatives to CB.

Conclusions:

The findings highlight a need and desire for more practical and interactive clinician training in behaviour interventions for individuals with ABI. Moreover, the findings suggest a limited understanding of PBS amongst community ABI therapists. Important considerations for the development of clinician training in ABI behaviour interventions and subsequent implementation into community practice are discussed.