A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy

BACKGROUND

Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers' implementation of evidence-based psychosocial interventions-(1) audit and feedback and (2) active learning.

METHODS

Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples.

RESULTS

In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ2=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ2=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples.

CONCLUSIONS

Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04096274 . Registered on 19 September 2019.

Systematic review of training in cognitive-behavioral therapy: Summarizing effects, costs and techniques

With the steadily growing importance of psychotherapeutic care, there is also an increasing need for high-quality training. We analyze the literature published between 2009 and 2022 on the effectiveness of training in cognitive behavioral therapy. The review addresses current gaps in the literature by focusing on the description of specific training components and their associated costs, as well as examining therapist-level predictors of training effectiveness. Our findings confirm the effect of additional supervision on both therapist competence and patient outcomes. Instructor-led training and self-guided web-based training seem to moderately increase competence, especially when targeting specific and highly structured treatments or skills. The level of prior training and experience of a therapist appears to predict the strength of training-related gains in competence. Few studies analyzed the differential effect of certain elements of training (e.g., the amount of active learning strategies) and training costs were generally not reported. Future studies should replicate or expand the existing evidence on active ingredients and therapist-level predictors of training effectiveness. Costs should be systematically reported to enhance the comparability of different training strategies.

Training home visitors in mentalization-based practice: A qualitative case study of clinical supervision in mothering from the inside out

Attachment-based interventions are important for improving parent-child outcomes. These interventions must be scaled and made available to under-resourced communities. An important part of scaling these interventions is delineating and reproducing high-quality training, including clinical training which often requires the completion of a supervised case. However, descriptions and guidelines for clinical training are frequently broad or not available in the literature. A detailed description of clinical training could lead to further research to improve the effectiveness and dissemination of evidence-based interventions. Mothering from the Inside Out (MIO) is an attachment-based parenting intervention effective at reducing substance use and depression, improving caregiving, and enhancing child attachment. It is now being brought from research to community settings. This paper outlines the didactic and clinical training components of MIO. We then present a qualitative case study of one community-based counselor participating in the clinical training of MIO and employ qualitative methods to describe the main themes that arose during the training. We aim to illustrate how the trainer assisted the counselor in implementing the core components of MIO, which included (a) refining the language used in MIO sessions, (b) making space to explore mental states, and (c) addressing trauma. We conclude by presenting the implications of these findings.

Concordance of multiple informant assessment of school-based social skills intervention and association with child outcomes: Results from a randomized trial

Background

Fidelity, or the degree to which an intervention is implemented as designed, is essential for effective implementation. There has been a growing emphasis on assessing fidelity of evidence-based practices for autistic children in schools. Fidelity measurement should be multidimensional and focus on core intervention components and assess their link with program outcomes. This study evaluated the relation between intervention fidelity ratings from multiple sources, tested the relation between fidelity ratings and child outcomes, and determined the relations between core intervention components and child outcomes in a study of an evidence-based psychosocial intervention designed to promote inclusion of autistic children at school, Remaking Recess.

Method

This study extends from a larger randomized controlled trial examining the effect of implementation support on Remaking Recess fidelity and child outcomes. Schools were randomized to receive the intervention or the intervention plus implementation support. Observers, intervention coaches, and school personnel completed fidelity measures to rate completion and quality of intervention delivery. A measure of peer engagement served as the child outcome. Pearson correlation coefficients were calculated to determine concordance between raters. Two sets of hierarchical linear models were conducted using fidelity indices as predictors of peer engagement.

Results

Coach- and self-rated completion and quality scores, observer- and self-rated quality scores, and observer- and coach-rated quality fidelity scores were significantly correlated. Higher observer-rated completion and quality fidelity scores were predictors of higher peer engagement scores. No single intervention component emerged as a significant predictor of peer engagement.

Conclusions

This study demonstrates the importance of using a multidimensional approach for measuring fidelity, testing the link between fidelity and child outcomes, and examining how core intervention components may be associated with child outcomes. Future research should clarify how to improve multi-informant reports to provide "good enough" ratings of fidelity that provide meaningful information about outcomes in community settings.

Plain Language Summary

Fidelity is defined as how closely an intervention is administered in the way the creators intended. Fidelity is important because it allows researchers to determine what exactly is leading to changes. In recent years, there has been an interest in examining fidelity of interventions for autistic children who receive services in school. This study looked at the relationship between fidelity ratings from multiple individuals, the relationship between fidelity and child outcomes, and the relationship between individual intervention component and child changes in a study of Remaking Recess, an intervention for autistic children at school. Schools were randomly selected to receive the intervention only or the intervention plus implementation support from the research team. Observers, intervention coaches, and individuals delivering the intervention themselves completed fidelity measures. Child engagement with peers was measured before and after the intervention. Several measures of self-, coach-, and observer-report fidelity were associated with each other. Higher observer-reported fidelity was associated with higher child peer engagement scores. No single intervention step was linked to child peer engagement and both treatment groups had similar outcomes in terms of fidelity. This study shows the importance of having multiple raters assess different parts of intervention fidelity, looking at the link between fidelity and child outcomes, and seeing how individual intervention steps may be related to outcomes. Future research should aim to find out which types of fidelity ratings are "good enough" to lead to positive changes following treatment so that those aspects can be used and targeted in the future.

Competence in Delivering Cognitive Processing Therapy and the Therapeutic Alliance Both Predict PTSD Symptom Outcomes

Cognitive Processing Therapy (CPT) is efficacious in treating PTSD, but there remains a need to improve outcomes for individuals who do not fully respond to treatment. Differences between patient-therapist dyads in the fidelity (i.e., adherence and competence) of CPT delivery and the quality of the therapeutic relationship may partly explain differential levels of symptom improvement. Sessions were sampled from a randomized trial comparing different consultation conditions in training therapists new to CPT. Among 69 patients, one session from Sessions 1-3 and one session from Sessions 4-7 were reliably rated for adherence and competence using the CPT Therapist Adherence and Competence Scale, and for therapeutic alliance using the Working Alliance Inventory-Observer scale. Mixed models, including detrending using a fixed effect of session, predicted self-reported Posttraumatic Stress Disorder Checklist (PCL-IV) scores in one session using process scores from the previous session. The statistical interaction between fidelity and alliance scores to predict outcome was also examined. Alliance had significant, positive correlations (rs = 0.18-0.21) with same-session adherence and competence. Higher competence scores and higher therapeutic alliance scores in one session were independently associated with lower PCL-IV scores in the subsequent session. Adherence scores, which tended to be very high with relatively less variability, did not significantly relate to subsequent-session PCL-IV scores. Competence significantly interacted with alliance, such that sessions high in both competence and alliance predicted especially lower subsequent-session PCL-IV scores. A strong therapeutic alliance may have a synergistic, salutary effect with the competent delivery of CPT.

Implementing Evidence-Based Mental-Health Treatments: Attending to Training, Fidelity, Adaptation, and Context

This article provides a very high-level overview of some key considerations in the field of implementation science as it relates to clinical psychology. The article reviews recent findings regarding treatment fidelity, adaptation of treatments, and clinical outcomes. It then details some recent findings on training therapists to provide evidence-based therapies as they were designed to be delivered, with sufficient skill and adaptations if needed to ensure better fit for clients who receive treatments in different treatment settings. Finally, the article considers implications for implementing and sustaining new treatments and supporting their fidelity and adaptation, as well as key directions for future research.

Knowledge and attitudes of Implementation Support Practitioners-Findings from a systematic integrative review

BACKGROUND

It requires thoughtful planning and work to successfully apply and sustain research-supported interventions like healthcare treatments, social support, or preventive programs in practice. Implementation support practitioners (ISPs) such as facilitators, technical assistance providers, knowledge brokers, coaches or consultants may be involved to actively support the implementation process. This article presents knowledge and attitudes ISPs bring to their work.

METHODS

Building on a previously developed program logic, a systematic integrative review was conducted. Literature was sourced by searching nine electronic data bases, organizational websites, and by launching a call for publications among selected experts and social media. Article screening was performed independently by two researchers, and data from included studies were extracted by members of the research team and quality-assured by the lead researcher. The quality of included RCTs was assessed based on a framework by Hodder and colleagues. Thematic Analysis was used to capture information on knowledge and attitudes of ISPs across the included studies. Euler diagrams and heatmaps were used to present the results.

RESULTS

Results are based on 79 included studies. ISPs reportedly displayed knowledge about the clinical practice they work with, implementation / improvement practice, the local context, supporting change processes, and facilitating evidence-based practice in general. In particular, knowledge about the intervention to be implemented and its target population, specific improvement / implementation methods and approaches, organizational structures and sensitivities, training, and characteristics of (good) research was described in the literature. Seven themes describing ISPs' attitudes were identified: 1) professional, 2) motivated / motivating / encouraging / empowering, 3) empathetic / respectful / sensitive, 4) collaborative / inclusive, 5) authentic, 6) creative / flexible / innovative / adaptive, and 7) frank / direct / honest. Pertaining to a professional attitude, being responsive and focused were the most prevalent indicators across included publications.

CONCLUSION

The wide range and complexity of knowledge and attitudes found in the literature calls for a comprehensive and systematic approach to collaboratively develop a professional role for ISPs across disciplines. Embedding the ISP role in different health and social welfare settings will enhance implementation capacities considerably.

A Novel Approach to the Assessment of Fidelity to a Cognitive Behavioral Therapy for PTSD Using Clinical Worksheets: A Proof of Concept With Cognitive Processing Therapy

Fidelity monitoring is a critical indicator of psychotherapy quality and is central to successful implementation. A major barrier to fidelity in routine care is the lack of feasible, scalable, and valid measurement strategies. A reliable, low-burden fidelity assessment would promote sustained implementation of cognitive behavioral therapies (CBTs). The current study examined fidelity measurement for cognitive processing therapy (CPT) for posttraumatic stress disorder (PTSD) using clinical worksheets. External raters evaluated patient worksheets done as a part of treatment, both guided by the therapist and completed independently as homework. Results demonstrated that fidelity ratings from CPT session worksheets were feasible and efficient. Notably, they were strongly correlated with observer ratings of the fidelity of CPT strategies that were present on the worksheets. Agreement among ratings conducted by individuals with a range of experience with CPT was acceptable to high. There was not a main effect of therapist-guided, in-session worksheet ratings on PTSD symptom change. However, patient competence in completing worksheets independently was associated with greater PTSD symptom decline and in-session, therapist-guided worksheet completion was associated with larger symptom decreases among patients with high levels of competence. With further research and refinement, rating of worksheets may be an efficient way to examine therapist and patient skill in key CPT elements, and their interactions, compared to the gold standard of observer ratings of therapy video-recordings. Additional research is needed to determine if worksheets are an accurate and scalable alternative to gold standard observer ratings in settings in which time and resources are limited.

Clinical Supervision in Community Mental Health: Characterizing Supervision as Usual and Exploring Predictors of Supervision Content and Process

Clinical supervision can be leveraged to support implementation of evidence-based practices in community mental health settings, though it has been understudied. This study focuses on 32 supervisors at 23 mental health organizations in Philadelphia. We describe characteristics of supervisors and organizations and explore predictors of supervision content and process. Results highlight a low focus on evidence-based content and low use of active supervision processes. They underscore the need for further attention to the community mental health context when designing supervision-targeted implementation strategies. Future work should assess whether supervision models specific to community mental health are needed.

Making Psychotherapy Available in the United States: Implementation Challenges and Solutions

About 46.6 million Americans have mental illnesses, yet profound treatment gaps exist in this country. Among those receiving treatment, most are treated solely with medication, even though >50% prefer to include psychotherapy in treatment. Other countries have changed policies to enhance access to evidence-based psychotherapy, resulting in availability and utilization reflective of patient preferences. To improve access to these treatments, the U.S. health care system would need to seek agreement among stakeholders regarding what constitutes effective treatment; enhance training, credentialing, and continuing education requirements; implement quality measures; integrate services into nontraditional venues; and incentivize best practices.

Prospective predictors of sustainment of multiple EBPs in a system-driven implementation context: Examining sustained delivery based on administrative claims

Background

System-driven scale-up of multiple evidence-based practices (EBPs) is an increasingly common method used in public mental health to improve care. However, there are little data on the long-term sustained delivery of EBPs within these efforts, and previous studies have relied on retrospective self-report within cross-sectional studies. This study identified prospective predictors of sustained EBP delivery at the EBP-, therapist-, and organizational-levels using survey and administrative claims data within a large-scale system-driven implementation effort.

Methods

777 therapists and 162 program leaders delivering at least one of six EBPs of interest completed surveys assessing perceptions of EBPs and organizational context. These surveys were linked to administrative data to examine prospective predictors of therapists' EBP delivery over 33 months.

Results

Five of the six EBPs implemented showed sustained delivery in the system, with volume varying by EBP. Although total EBP claim volume per therapist decreased over time, the volume ratio (ratio of EBP-specific claims to total EBP and non-EBP claims) stayed relatively stable. Multilevel models revealed that EBPs that required consultation, had unstructured content, higher therapist self-efficacy with the EBP, and more positive program leader perceptions of the EBP were associated with greater sustained volume and volume ratio of the EBP. Therapists who were trained in fewer EBPs, who were unlicensed, and who worked in agencies rated by program leaders as lower on organizational staff autonomy and stress showed greater sustained EBP volume and volume ratio. Finally, more direct service hours per week provided by therapist predicted greater sustained EBP volume, but lower volume ratio.

Conclusions

The results point to the importance of EBP, therapist, and organizational factors that may be targeted in implementation strategies to promote the sustainment of EBPs.

A Glimpse into the "Black Box": Which Elements of Consultation in an EBP are Associated with Client Symptom Change and Therapist Fidelity?

Background: Consultation is an implementation strategy that improves delivery and clinical outcomes for Cognitive Processing Therapy (CPT), an evidence-based practice (EBP) for posttraumatic stress disorder (PTSD). However, little is known about the specific components of consultation that influence the fidelity of treatment delivery or clinical outcomes. Methods: The current study examined whether specific activities performed during CPT consultation meetings were associated with better fidelity to the CPT protocol among 60 newly trained therapists or improved clinical outcomes among 135 clients treated by these therapists. Consultation activities that fall under three broad categories (discussion of the application of CPT to individual cases, review/feedback on fidelity, and technical difficulties) were measured by consultant checklists for each consultation session. Treatment fidelity (adherence to the protocol and competence of delivery) was rated by trained observers for a random sample of therapists' CPT sessions following consultation. The self-reported PTSD Checklist-IV assessed PTSD symptom change. Results: Multilevel regression analyses indicated that higher therapist consultation attendance predicted a greater decrease in their clients' PTSD symptoms and that attendance was not associated with observer-rated treatment fidelity. Discussion of the application of specific CPT strategies was the only consultation activity that was significantly associated with greater improvement in PTSD symptoms. Lastly, no consultation activities were significantly associated with treatment fidelity. Conclusions: Our findings suggest that specific consultation strategies such as emphasizing the discussion of the application of specific CPT strategies to individual cases during consultation meetings may be effective in improving the clinical outcomes of CPT.

Systematic review to examine the methods used to adapt evidence-based psychological treatments for adults diagnosed with a mental illness

QUESTION

The context for the implementation of evidence-based psychological treatments (EBPTs) often differs from the context in which the treatment was developed, which necessitates adaptations. In this systematic review we build on, and add to, prior approaches by examining the method used to guide such adaptations. In particular, we sought to elucidate the extent to which an empirical process is used.

STUDY SELECTION AND ANALYSIS

We focused on publications describing adaptations made to EBPTs for adults diagnosed with a mental illness. We searched PubMed, PsycINFO, Embase and Web of Science from database inception to July 2018. Two raters independently coded the articles for the method used to conduct the adaptation, the reason for and nature of the adaptation, and who made the adaptation.

FINDINGS

The search produced 20 194 citations, which yielded 152 articles after screening. The most commonly used methods for planned adaptations were literature review (57.7%), clinical intuition (47.0%) and theory (38.9%). The use of data from stakeholder interviews ranked fourth (21.5%) and the use of other types of data (eg, pilot study, experiment, survey, interview) ranked last at fifth (12.1%). Few publications reporting ad hoc adaptations were identified (n=3).

CONCLUSIONS

This review highlights a need to (a) educate providers and researchers to carefully consider the methods used for the treatment adaptation process, and to use empirical methods where possible and where appropriate, (b) improve the quality of reporting of stakeholder interviews and (c) develop reporting standards that articulate optimal methods for conducting treatment adaptations.

Priority domains, aims, and testable hypotheses for implementation research: Protocol for a scoping review and evidence map

BACKGROUND

The challenge of implementing evidence-based innovations within practice settings is a significant public health issue that the field of implementation research (IR) is focused on addressing. Significant amounts of funding, time, and effort have been invested in IR to date, yet there remains significant room for advancement, especially regarding IR's development of scientific theories as defined by the National Academy of Sciences (i.e., a comprehensive explanation of the relationship between variables that is supported by a vast body of evidence). Research priority setting (i.e., promoting consensus about areas where research effort will have wide benefits to society) is a key approach to helping accelerate research advancements. Thus, building upon existing IR, general principles of data reduction, and a general framework for moderated mediation, this article identifies four priority domains, three priority aims, and four testable hypotheses for IR, which we organize in the priority aims and testable hypotheses (PATH) diagram.

METHODS

The objective of this scoping review is to map the extent to which IR has examined the identified PATH priorities to date. Our sample will include IR published in leading implementation-focused journals (i.e., Implementation Science, Implementation Science Communications, and Implementation Research and Practice) between their inception and December 2020. The protocol for the current scoping review and evidence map has been developed in accordance with the approach developed by Arksey and O'Malley and advanced by Levac, Colquhoun, and O'Brien. Because scoping reviews seek to provide an overview of the identified evidence base rather than synthesize findings from across studies, we plan to use our data-charting form to provide a descriptive overview of implementation research to date and summarize the research via one or more summary tables. We will use the PATH diagram to organize a map of the evidence to date.

DISCUSSION

This scoping review and evidence map is intended to help accelerate IR focused on suggested priority aims and testable hypotheses, which in turn will accelerate IR's development of National Academy of Sciences-defined scientific theories and, subsequently, improvements in public health.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework https://osf.io/3vhuj/.

Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes

A lack of effective therapist training is a major barrier to evidence-based intervention (EBI) delivery in the community. Systematic reviews published nearly a decade ago suggested that traditional EBI training leads to higher knowledge but not more EBI use, indicating that more work is needed to optimize EBI training and implementation. This systematic review synthesizes the training literature published since 2010 to evaluate how different training models (workshop, workshop with consultation, online training, train-the-trainer, and intensive training) affect therapists' knowledge, beliefs, and behaviors. Results and limitations for each approach are discussed. Findings show that training has advanced beyond provision of manuals and brief workshops; more intensive training models show promise for changing therapist behavior. However, methodological issues persist, limiting conclusions and pointing to important areas for future research.

What is important for the sustained implementation of evidence-based brief psychotherapy interventions in psychiatric care? A quantitative evaluation of a real-world programme

PURPOSE

Behavioural activation and motivational interviewing, both evidence-based treatments (EBTs), were implemented in secondary psychiatric care. This longitudinal evaluation of a real-world programme focused on the penetration of EBT adoption and its associations with therapist-related and perceived intervention-related variables. The implementation plan was also compared to sub-processes of Normalization Process Theory.

MATERIAL AND METHODS

Six participating units employed 72 therapists regularly and they comprise the target group. Due to staff turnover, a total of 84 therapists were trained stepwise. Three survey points (q1, q2, q3) were set for a four-year cycle beginning a year after the initial training and completed 4-5 months after closing patient recruitment. The implementation plan included two workshop days, one for each EBT, and subsequent case consultation groups and other more general strategies.

RESULTS

Fifty-seven (68%) of programme-trained therapists responded to one or more of three questionnaires. The self-reported penetration covers about a third of the target group a few months after the completion of the programme. Therapists' favourable perceptions of the EBTs regarding relative advantage, compatibility and complexity were associated with their sustained adoption. Therapists' background factors (e.g. work experience) and positive adoption intention at q1 did not predict the actual adoption of the EBTs at q3. No specific sustainment strategies were included in the implementation plan.

CONCLUSION

Brief but multi-faceted training with subsequent case consultations promoted the adoption of EBTs in a real-world setting. Adding specific sustainment strategies to the implementation plan is proposed to ensure the long-term survival of the implementation outcomes.

Dissemination and Implementation of Cognitive Behavioral Therapy for Depression in the Kaiser Permanente Health Care System: Evaluation of Initial Training and Clinical Outcomes

Notwithstanding its empirical status and strong recommendation in clinical practice guidelines, cognitive behavioral therapy (CBT) continues to be delivered infrequently and with low fidelity on the clinical front lines. Recently, organized efforts and policies within the public sector to disseminate and implement CBT and other evidence-based psychotherapies have yielded encouraging results and provided optimism for bridging the research-to-practice-gap. Following from these efforts, the current article examines the initial impact and experience of the implementation of an individualized approach to CBT training and treatment within the Kaiser Permanente health care system. Initial training outcomes, including changes in general and specific competencies, were assessed using divergent assessment methods within the initial cohort of therapists undergoing training. Initial patient outcomes, including changes in depression and anxiety, were assessed among patients receiving treatment from therapists in training. Results revealed training in and implementation of CBT-D was associated with overall large improvements in therapist competencies and in clinically significant improvements in both depression and anxiety among patients. Findings from the initial phase of dissemination and implementation within a large private system provide support for, and extend recent findings related to, the feasibility and effectiveness of training in and implementation of CBT-D in a real-world context.

A randomized controlled effectiveness trial of training strategies in cognitive processing therapy for posttraumatic stress disorder: Impact on patient outcomes

This randomized controlled hybrid implementation/effectiveness trial aimed to compare the impact of three different models of training and consultation by examining the PTSD treatment outcomes achieved by therapists who were learning a front-line recommended psychotherapy for posttraumatic stress disorder (PTSD), Cognitive Processing Therapy (CPT; Resick, Monson, & Chard, 2017). Therapists (N = 134) were randomized into one of three conditions after attending a standard CPT training workshop: No Consultation with delayed feedback on CPT fidelity, Standard Consultation involving discussion and conceptualization of cases without session audio review, and Consultation Including Audio Review, which included a review of segments of audiorecorded CPT sessions. Across all training conditions, the patients treated by these therapists (N = 188) evidenced statistically significant reductions in PTSD symptoms, (d = -0.95 to -1.78), comorbid symptoms and functioning (d = -0.27 to -0.51). However, patients of therapists in the Standard Consultation condition (ΔPTSD = - 19.64, d = -1.78) experienced significantly greater improvement than those in the No Consultation condition (ΔPTSD = - 10.54, d = -0.95, ΔDEV = 6.30, ΔParms = 2, p = .043). This study demonstrates that patients who receive evidence-based psychotherapy for PTSD in routine care settings can experience significant symptom improvement. Our findings also suggest that to maximize patient benefit, therapist training should include consultation, but that audio review of sessions during consultation may not be necessary, at least for structured protocols. Implications for implementation, including the reduction of burden and cost of post-workshop support, are discussed.

The Juggling Act of Supervision in Community Mental Health: Implications for Supporting Evidence-Based Treatment

Supervisors are an underutilized resource for supporting evidence-based treatments (EBTs) in community mental health. Little is known about how EBT-trained supervisors use supervision time. Primary aims were to describe supervision (e.g., modality, frequency), examine functions of individual supervision, and examine factors associated with time allocation to supervision functions. Results from 56 supervisors and 207 clinicians from 25 organizations indicate high prevalence of individual supervision, often alongside group and informal supervision. Individual supervision serves a wide range of functions, with substantial variation at the supervisor-level. Implementation climate was the strongest predictor of time allocation to clinical and EBT-relevant functions.

The effectiveness of consultation for clinicians learning to deliver motivational interviewing with fidelity

Despite the emerging literature documenting gains in clinician competence following consultation, little empirical work has examined consultation as an implementation strategy. To this end, the present study examined consultation in the context of implementing motivational interviewing in four community child and youth mental health organizations. We used qualitative methods with a dual goal: to describe the consultation process and to explore trainees' perspectives on consultation. Participants included 22 clinicians and 9 supervisors who received monthly, group, phone-based consultation for seven months following training in motivational interviewing. Analyses showed that consultation was perceived as effective because it helped to "keep motivational interviewing alive," fulfilled a profound learning function through collaboration and connection with others, and served as protected time for reflection on practice change. Our findings contribute to a body of knowledge about consultation elements that appear to be effective when implementing research-supported interventions in child and youth mental health.

Leveraging routine clinical materials and mobile technology to assess CBT fidelity: the Innovative Methods to Assess Psychotherapy Practices (imAPP) study

BACKGROUND

Identifying scalable strategies for assessing fidelity is a key challenge in implementation science. However, for psychosocial interventions, the existing, reliable ways to test treatment fidelity quality are often labor intensive, and less burdensome strategies may not reflect actual clinical practice. Cognitive behavioral therapies (CBTs) provide clinicians with a set of effective core elements to help treat a multitude of disorders, which, evidence suggests, need to be delivered with fidelity to maximize potential client impact. The current "gold standard" for rating CBTs is rating recordings of therapy sessions, which is extremely time-consuming and requires a substantial amount of initial training. Although CBTs can vary based on the target disorder, one common element employed in most CBTs is the use of worksheets to identify specific behaviors and thoughts that affect a client's ability to recover. The present study will develop and evaluate an innovative new approach to rate CBT fidelity, by developing a universal CBT scoring system based on worksheets completed in therapy sessions.

METHODS

To develop a scoring system for CBT worksheets, we will compile common CBT elements from a variety of CBT worksheets for a range of psychiatric disorders and create adherence and competence measures. We will collect archival worksheets from past studies to test the scoring system and assess test-retest reliability. To evaluate whether CBT worksheet scoring accurately reflects clinician fidelity, we will recruit clinicians who are engaged in a CBT for depression, anxiety, and/or posttraumatic stress disorder. Clinicians and clients will transmit routine therapy materials produced in session (e.g., worksheets, clinical notes, session recordings) to the study team after each session. We will compare observer-rated fidelity, clinical notes, and fidelity-rated worksheets to identify the most effective and efficient method to assess clinician fidelity. Clients will also be randomly assigned to either complete the CBT worksheets on paper forms or on a mobile application (app) to learn if worksheet format influences clinician and client experience or differs in terms of reflecting fidelity.

DISCUSSION

Scoring fidelity using CBT worksheets may allow clinics to test fidelity in a short and effective manner, enhancing continuous quality improvement in the workplace. Clinicians and clinics can use such data to improve clinician fidelity in real time, leading to improved patient outcomes.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03479398 . Retrospectively registered March 20, 2018.

Objective coding of content and techniques in workplace-based supervision of an EBT in public mental health

BACKGROUND

Workplace-based clinical supervision as an implementation strategy to support evidence-based treatment (EBT) in public mental health has received limited research attention. A commonly provided infrastructure support, it may offer a relatively cost-neutral implementation strategy for organizations. However, research has not objectively examined workplace-based supervision of EBT and specifically how it might differ from EBT supervision provided in efficacy and effectiveness trials.

METHODS

Data come from a descriptive study of supervision in the context of a state-funded EBT implementation effort. Verbal interactions from audio recordings of 438 supervision sessions between 28 supervisors and 70 clinicians from 17 public mental health organizations (in 23 offices) were objectively coded for presence and intensity coverage of 29 supervision strategies (16 content and 13 technique items), duration, and temporal focus. Random effects mixed models estimated proportion of variance in content and techniques attributable to the supervisor and clinician levels.

RESULTS

Interrater reliability among coders was excellent. EBT cases averaged 12.4 min of supervision per session. Intensity of coverage for EBT content varied, with some discussed frequently at medium or high intensity (exposure) and others infrequently discussed or discussed only at low intensity (behavior management; assigning/reviewing client homework). Other than fidelity assessment, supervision techniques common in treatment trials (e.g., reviewing actual practice, behavioral rehearsal) were used rarely or primarily at low intensity. In general, EBT content clustered more at the clinician level; different techniques clustered at either the clinician or supervisor level.

CONCLUSIONS

Workplace-based clinical supervision may be a feasible implementation strategy for supporting EBT implementation, yet it differs from supervision in treatment trials. Time allotted per case is limited, compressing time for EBT coverage. Techniques that involve observation of clinician skills are rarely used. Workplace-based supervision content appears to be tailored to individual clinicians and driven to some degree by the individual supervisor. Our findings point to areas for intervention to enhance the potential of workplace-based supervision for implementation effectiveness.

TRIAL REGISTRATION

NCT01800266 , Clinical Trials, Retrospectively Registered (for this descriptive study; registration prior to any intervention [part of phase II RCT, this manuscript is only phase I descriptive results]).

Training Community-Based Clinicians in Parent-Child Interaction Therapy: The Interaction Between Expert Consultation and Caseload

Professional psychologists are increasingly encouraged to utilize evidence-based treatments (EBTs), and therefore have a need to participate and provide the most efficient training methods for these treatments. Multicomponent trainings, which commonly include ongoing support, are more effective than brief methods such as 1-day workshops or reading treatment manuals. The present study examined the effectiveness of 1 form of ongoing support, consultation, as part of a multicomponent training protocol. Thirty-two community-based clinicians were trained in Parent-Child Interaction Therapy (PCIT) as part of a statewide implementation effort, and data were collected on clinician and implementation outcomes at pre-, mid-, and posttraining. Simple and multiple linear regression analyses were conducted to predict posttraining knowledge, skill, acceptability, and feasibility, as well as to examine clinician variables that might moderate these relations. Greater consultation call attendance significantly predicted higher posttraining skill; however, this association was qualified by a significant interaction with PCIT caseload. Implications for training guidelines are discussed.

Implementation of transdiagnostic cognitive therapy in community behavioral health: The Beck Community Initiative

OBJECTIVE

Progress bringing evidence-based practice (EBP) to community behavioral health (CBH) has been slow. This study investigated feasibility, acceptability, and fidelity outcomes of a program to implement transdiagnostic cognitive therapy (CT) across diverse CBH settings, in response to a policy shift toward EBP.

METHOD

Clinicians (n = 348) from 30 CBH programs participated in workshops and 6 months of consultation. Clinician retention was examined to assess feasibility, and clinician feedback and attitudes were evaluated to assess implementation acceptability. Experts rated clinicians' work samples at baseline, mid-, and end-of-consultation with the Cognitive Therapy Rating Scale (CTRS) to assess fidelity.

RESULTS

Feasibility was demonstrated through high program retention (i.e., only 4.9% of clinicians withdrew). Turnover of clinicians who participated was low (13.5%) compared to typical CBH turnover rates, even during the high-demand training period. Clinicians reported high acceptability of EBP and CT, and self-reported comfort using CT improved significantly over time. Most clinicians (79.6%) reached established benchmarks of CT competency by the final assessment point. Mixed-effects hierarchical linear models indicated that CTRS scores increased significantly from baseline to the competency assessment (p < .001), on average by 18.65 points. Outcomes did not vary significantly between settings (i.e., outpatient vs. other).

CONCLUSIONS

Even clinicians motivated by policy-change rather than self-nomination may feasibly be trained to deliver a case-conceptualization driven EBP with high levels of competency and acceptability. (PsycINFO Database Record