National trainers' perspectives on challenges to implementation of an empirically-supported mental health treatment

Therapists' Perceived Competence in Delivering Trauma-Focused Cognitive Behavioral Therapy During Statewide Learning Collaboratives

The learning collaborative (LC), a multi-component training and implementation model, is one promising approach to address the need for increased availability of trauma-focused evidence-based practices. The current study used data from four cohorts of a statewide LC on Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) to 1) evaluate pre- to post-LC changes in therapists' perceived competence in delivering TF-CBT and 2) explore therapist and contextual factors related to therapists' perceived TF-CBT competence. Therapists (N = 237) completed pre- and post-LC measures of practice information, interprofessional collaboration, organizational climate, and TF-CBT knowledge, perceived competence, and use. Findings indicated therapists' perceived TF-CBT competence significantly increased, pre- to post-LC (d = 1.31), with greater use of trauma-focused practices at pre-training and more TF-CBT training cases completed predicting greater pre- to post-LC gains in perceived TF-CBT competence. These findings highlight the need to assist therapists in identifying and completing training cases to promote competence and implementation.

A machine learning approach to improve implementation monitoring of family-based preventive interventions in primary care

Background

Evidence-based parenting programs effectively prevent the onset and escalation of child and adolescent behavioral health problems. When programs have been taken to scale, declines in the quality of implementation diminish intervention effects. Gold-standard methods of implementation monitoring are cost-prohibitive and impractical in resource-scarce delivery systems. Technological developments using computational linguistics and machine learning offer an opportunity to assess fidelity in a low burden, timely, and comprehensive manner.

Methods

In this study, we test two natural language processing (NLP) methods [i.e., Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers (BERT)] to assess the delivery of the Family Check-Up 4 Health (FCU4Health) program in a type 2 hybrid effectiveness-implementation trial conducted in primary care settings that serve primarily Latino families. We trained and evaluated models using 116 English and 81 Spanish-language transcripts from the 113 families who initiated FCU4Health services. We evaluated the concurrent validity of the TF-IDF and BERT models using observer ratings of program sessions using the COACH measure of competent adherence. Following the Implementation Cascade model, we assessed predictive validity using multiple indicators of parent engagement, which have been demonstrated to predict improvements in parenting and child outcomes.

Results

Both TF-IDF and BERT ratings were significantly associated with observer ratings and engagement outcomes. Using mean squared error, results demonstrated improvement over baseline for observer ratings from a range of 0.83-1.02 to 0.62-0.76, resulting in an average improvement of 24%. Similarly, results demonstrated improvement over baseline for parent engagement indicators from a range of 0.81-27.3 to 0.62-19.50, resulting in an approximate average improvement of 18%.

Conclusions

These results demonstrate the potential for NLP methods to assess implementation in evidence-based parenting programs delivered at scale. Future directions are presented.

Trial registration

NCT03013309 ClinicalTrials.gov.

Measuring clinician stuck points about trauma-focused cognitive behavior therapy: The TF-CBT Stuck Points Questionnaire

Although evidence-based treatments (EBTs) for youth trauma have been developed, trauma-informed EBTs are rarely used in community settings. Clinician concerns about evidence-based trauma treatment may be a barrier to adoption and delivery. However, few instruments to assess clinician beliefs about specific EBTs, such as trauma-focused cognitive behavior therapy (TF-CBT) are available. This study evaluated an instrument of clinician concerns about TF-CBT, the TF-CBT Therapist Stuck Points questionnaire, in a sample of community mental health clinicians training in a year-long TF-CBT community-based learning collaborative. The 26 items in the instruments, which aim to assess clinician views on child trauma treatment and TF-CBT, indicate preliminary psychometric support (i.e., item-total correlations, internal consistency, negative correlations with measures of attitudes towards evidence-based practice). Scores on the TF-CBT Therapist Stuck Points questionnaire revealed that, on average, clinicians expressed concerns about having children talk about their trauma in session, the effectiveness of certain TF-CBT components, and whether to involve caregivers in treatment. Clinician doubts could be targeted during a TF-CBT implementation effort and clinical supervision to facilitate treatment delivery. Implications for assessing TF-CBT specific beliefs during implementation are discussed.

Therapists' Perceived Competence in Trauma-Focused Cognitive Behavioral Therapy and Client Outcomes: Findings From a Community-Based Learning Collaborative

This study examined therapists' perceived competence in Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) and its association with youth treatment outcomes (posttraumatic stress and depression). Participants included 99 community therapists enrolled in a TF-CBT-focused Learning Collaborative (LC), along with one of their randomly selected TF-CBT training cases. Analyzed data included: 1) caregiver/youth-reported posttraumatic stress and depressive symptoms, pre- and post-treatment, and 2) therapist-perceived competence with TF-CBT components across treatment delivery. Youth- and caregiver-reports indicated large, significant pre- to post-treatment decreases in youth posttraumatic stress (ds = 1.10-1.30, ps < .001) and depressive symptoms (d = 1.01, p < .001). Higher therapist-perceived competence with TF-CBT predicted positive treatment responses for posttraumatic stress (ds = 0.38-0.39, ps = .03) and depression (d = 0.25), though only the former association was significant (ps = .03 vs. p = .15). Findings highlight the need to monitor and improve therapists' competencies to enhance clinical outcomes for trauma-exposed youth and suggest that LCs may be an effective training/implementation model to help achieve those critical goals.

Predictors of Clinician-Reported Self-Efficacy in Treating Trauma-Exposed Youth

Clinicians' self-efficacy with regard to delivering evidence-based interventions (EBIs) to youth is an important target for both improving EBI use in the community and mitigating the risk of clinician burnout and turnover. Examining predictors of clinician self-efficacy to treat trauma-exposed youth is, therefore, an important step for informing the design of implementation strategies to enhance the mental health workforce's capacity to deliver EBIs in this population. We examined predictors of clinician self-efficacy in working with trauma-exposed youth in a sample of practicing mental health clinicians (N = 258, M age = 34.4 years, 85.0% female). Clinicians were recruited and surveyed as part of a larger study examining how clients' exposure to potentially traumatic events influences clinician decision-making. Results of regression models indicated that training in any trauma treatment model, being trained via a variety of formats (e.g., in-person training, online, supervision), and training in a variety of treatment models were all associated with higher perceived self-efficacy regarding effectively treating trauma-exposed youth. Of the treatment models and training formats examined, receiving in-person training, R2 = .10, and training in trauma-focused cognitive behavioral therapy, R2 = .10, were the strongest predictors of higher self-efficacy ratings. Clinician discipline, R2 = .04, and clinical practice factors, R2 = .20, were also related to self-efficacy. Collectively, the R2 indicated a large effect, with the predictors explaining 25.4% of the variance in self-efficacy ratings. Implications for designing implementation strategies targeting clinician self-efficacy and future research are discussed.

Trauma-Focused Cognitive Behavioral Therapy in 13 Urban Public Schools: Mixed Methods Results of Barriers, Facilitators, and Implementation Outcomes

Trauma-focused cognitive behavioral therapy (TF-CBT) is an evidence-based intervention to treat Post-traumatic Stress Disorder among youth ages 3-18 years. In this pilot study, 31 clinicians in 13 urban public schools received TF-CBT training to improve access to a high-quality trauma treatment for youth in need. A mixed methods design was used to assess implementation barriers, facilitators and outcomes to examine initial feasibility, acceptability, and appropriateness of TF-CBT delivered in school settings. Although 70% of clinicians reported confidence in the evidence-base of TF-CBT and its potential to support their students who have a very high prevalence of lifetime trauma exposure, implementation practice outcomes suggest a wide range of TF-CBT use (Range = 0 to 11 students enrolled per clinician, Mean = 1.4 students) with 23 clinicians implementing the TF-CBT model with at least one case. Results point to the potential value of training school mental health clinicians in TF-CBT based on its compatibility with student needs. Yet, by connecting focus group results to top-rated barriers and facilitators, we identify several adjustments that should be made to improve intervention-setting fit in future studies. Specifically, school setting-specific intervention adaptations, school-tailored implementation support and thoughtful consideration of school-based clinicians' roles and responsibilities are needed to enhance future implementation success.

Determinants of Implementing Evidence-Based Trauma-Focused Interventions for Children and Youth: A Systematic Review

A systematic review was conducted to identify determinants (barriers and facilitators) of implementing evidence-based psychosocial interventions for children and youth who experience emotional or behavioral difficulties due to trauma exposure. Determinants were coded, abstracted, and synthesized using the Exploration, Preparation, Implementation, and Sustainment framework. Twenty-three articles were included, all of which examined implementation of Trauma-Focused Cognitive Behavioral Therapy or Cognitive-Behavioral Intervention for Trauma in Schools. This review identified multilevel and multiphase determinants that can be addressed by implementation strategies to improve implementation and clinical outcomes, and suggests how future studies might address gaps in the evidence base.

Barriers Associated with the Implementation of Homework in Youth Mental Health Treatment and Potential Mobile Health Solutions

BACKGROUND

Homework, or between-session practice of skills learned during therapy, is integral to effective youth mental health TREATMENTS. However, homework is often under-utilized by providers and patients due to many barriers, which might be mitigated via mHealth solutions.

METHODS

Semi-structured qualitative interviews were conducted with nationally certified trainers in Trauma Focused Cognitive Behavioral Therapy (TF-CBT; n=21) and youth TF-CBT patients ages 8-17 (n=15) and their caregivers (n=12) to examine barriers to the successful implementation of homework in youth mental health treatment and potential mHealth solutions to those barriers.

RESULTS

The results indicated that many providers struggle to consistently develop, assign, and assess homework exercises with their patients. Patients are often difficult to engage and either avoid or have difficulty remembering to practice exercises, especially given their busy/chaotic home lives. Trainers and families had positive views and useful suggestions for mHealth solutions to these barriers in terms of functionality (e.g., reminders, tracking, pre-made homework exercises, rewards) and user interface (e.g., easy navigation, clear instructions, engaging activities).

CONCLUSIONS

This study adds to the literature on homework barriers and potential mHealth solutions to those barriers, which is largely based on recommendations from experts in the field. The results aligned well with this literature, providing additional support for existing recommendations, particularly as they relate to treatment with youth and caregivers.

Using behavioral insights to design implementation strategies in public mental health settings: a qualitative study of clinical decision-making

BACKGROUND

Trauma-focused cognitive behavioral therapy (TF-CBT) is an evidence-based intervention for youth with posttraumatic stress disorder. An important component of TF-CBT is the trauma narrative (TN), a phase in the intervention in which youth are guided to process the memories, thoughts, and feelings associated with their traumatic experience(s). Previous work has shown that TF-CBT clinicians complete TNs with only half of their clients, yet little is known about what determines TF-CBT clinicians' use of TNs. The behavioral insights literature-an interdisciplinary field studying judgment and decision-making-offers theoretical and empirical tools to conceptualize what drives complex human behaviors and decisions. Drawing from the behavioral insights literature, the present study seeks to understand what determines clinician use of TNs and to generate strategies that target these determinants.

METHODS

Through semi-structured qualitative interviews, we sought the perspectives of trained TF-CBT clinicians working in public mental health settings across the city of Philadelphia (N = 17) to understand their decisions to use TNs with clients. We analyzed the qualitative data using a coding approach informed by the behavioral insights literature. We used an iterative process of structured hypothesis generation, aided by a behavioral insights guide, and rapid validation informed by behavioral insights to uncover the determinants of TN use. We then generated implementation strategies that targeted these determinants using the "Easy Attractive Social Timely" framework, a behavioral insights design approach.

RESULTS

We generated and validated three broad themes about what determines clinician implementation of TNs: decision complexity, clinician affective experience, and agency norms. We hypothesized behavioral insights that underlie these implementation determinants and designed a list of nine corresponding behavioral insights strategies that may facilitate TN implementation.

CONCLUSIONS

Our study investigated why an effective component of an evidence-based intervention is difficult to implement. We leveraged robust scientific theories and empirical regularities from the behavioral insights literature to understand clinician perspectives on TN implementation. These factors were theoretically linked to implementation strategies. Our work revealed the potential for using behavioral insights in the diagnosis of evidence-based intervention determinants and the design of implementation strategies.

Protocol for a hybrid type 2 cluster randomized trial of trauma-focused cognitive behavioral therapy and a pragmatic individual-level implementation strategy

BACKGROUND

More than two-thirds of youth experience trauma during childhood, and up to 1 in 5 of these youth develops posttraumatic stress symptoms that significantly impair their functioning. Although trauma-focused cognitive behavior therapy (TF-CBT) has a strong evidence base, it is rarely adopted, delivered with adequate fidelity, or evaluated in the most common setting where youth access mental health services-schools. Given that individual behavior change is ultimately required for successful implementation, even when organizational factors are firmly in place, focusing on individual-level processes represents a potentially parsimonious approach. Beliefs and Attitudes for Successful Implementation in Schools (BASIS) is a pragmatic, motivationally focused multifaceted strategy that augments training and consultation and is designed to target precise mechanisms of behavior change to produce enhanced implementation and youth clinical outcomes. This study protocol describes a hybrid type 2 effectiveness-implementation trial designed to concurrently evaluate the main effects, mediators, and moderators of both the BASIS implementation strategy on implementation outcomes and TF-CBT on youth mental health outcomes.

METHODS

Using a cluster randomized controlled design, this trial will assign school-based mental health (SMH) clinicians and schools to one of three study arms: (a) enhanced treatment-as-usual (TAU), (b) attention control plus TF-CBT, or (c) BASIS+TF-CBT. With a proposed sample of 120 SMH clinicians who will each recruit 4-6 youth with a history of trauma (480 children), this project will gather data across 12 different time points to address two project aims. Aim 1 will evaluate, relative to an enhanced TAU condition, the effects of TF-CBT on identified mechanisms of change, youth mental health outcomes, and intervention costs and cost-effectiveness. Aim 2 will compare the effects of BASIS against an attention control plus TF-CBT condition on theoretical mechanisms of clinician behavior change and implementation outcomes, as well as examine costs and cost-effectiveness.

DISCUSSION

This study will generate critical knowledge about the effectiveness and cost-effectiveness of BASIS-a pragmatic, theory-driven, and generalizable implementation strategy designed to enhance motivation-to increase the yield of evidence-based practice training and consultation, as well as the effectiveness of TF-CBT in a novel service setting.

TRIAL REGISTRATION

ClinicalTrials.gov registration number NCT04451161 . Registered on June 30, 2020.

Disseminating TF-CBT: A Mixed Methods Investigation of Clinician Perspectives and the Impact of Training Format and Formalized Problem-Solving Approaches on Implementation Outcomes

BACKGROUND

Trauma Focused-Cognitive Behavioural Therapy (TF-CBT) has been established as an evidence-based treatment for youth with traumatic stress symptoms. The versatility of TF-CBT in conjunction with its established effectiveness has led to its widespread dissemination. However, dissemination efforts have not always translated into sustainability, which has prompted a more thorough investigation into those factors that impact implementation and encouraged the development of strategies that promote sustainability. Toward this end, the aims of this study were to: (1) determine which components of TF-CBT clinicians found the most difficult to implement; (2) explore clinicians' perceptions as to why these components were challenging; and (3) examine whether the use of formalized problem-solving approaches (FPSAs) or training format was associated with implementation outcomes.

METHODS

A mixed methods design was utilized to analyse survey data from mental health professionals (N = 85) who received TF-CBT training. Thematic analysis explored responses to questions concerning those TF-CBT components respondents found most difficult to implement, while bivariate analyses helped determine whether training format or the use of FPSAs was associated with training engagement, confidence in delivering TF-CBT, fidelity or sustainability.

RESULTS

Clinicians perceived the same three components of TF-CBT as most difficult regardless of the type of training they received, and provider, youth, caregiver, and organizational-related themes emerged from thematic analysis. Bivariate analyses indicated more extended training formats and the use of FPSAs were associated with greater implementation success.

CONCLUSION

Findings suggest that perceived difficulty of TF-CBT components did not vary by training format, but more extended formats and the use of FPSAs were associated with more favourable implementation outcomes. Implementers should consider ways to utilize FPSAs within training programs, as well as focus on content identified by clinicians as difficult, as this may assist clinicians in developing skills and managing implementation barriers.

Hybrid type 1 randomized controlled trial of a tablet-based application to improve quality of care in child mental health treatment

The quality of child mental health care is highly variable in community practice settings. Innovative technology-based solutions may be leveraged to improve quality of care and, in turn, treatment outcomes. This is a protocol paper that describes an innovative study design in which we rigorously evaluate the effectiveness of a tablet-assisted intervention, Supporting Providers and Reaching Kids (SPARK). SPARK consists of a collection of interactive games and activities that are designed to improve provider fidelity and child engagement in evidence-based psychotherapies. The methodology also allows us to explore the implementation and sustainability of a technology-enhanced intervention in more than two dozen community practice settings. This paper includes a description and justification for sample selection and recruitment procedures, selection of assessment measures and methods, design of the intervention, and statistical evaluation of critical outcomes. Novel features of the design include the tablet-based toolkit approach that has strong applicability to a range of child mental health interventions and the use of a hybrid type 1 effectiveness-implementation trial that allows for the simultaneous investigation of the effectiveness of the intervention and the implementation context. Challenges related to the implementation of a technology-enhanced intervention in existing mental health clinics are discussed, as well as implications for future research and practice.

Usability Testing of a Mobile Health Intervention to Address Acute Care Needs after Sexual Assault

Sexual assault is associated with a range of poor mental health outcomes. To enhance access to care by this population, technology-based mental health interventions have been implemented in the emergency room; however, more accessible and easily disseminated interventions are needed. The aim of the present study was to test the usability of a mobile health intervention targeting alcohol and drug misuse, suicide prevention, posttraumatic stress symptoms, coping skills, and referral to formal assistance for individuals who have experienced sexual assault. Feedback on the usability of the intervention was collected from individuals who received a sexual assault medical forensic examination (n = 13), and feedback on the usability and likelihood of recommending the application was collected from community providers (n = 25). Thematic analysis was used to describe qualitative data. Content themes related to aesthetics, usability, barriers to resources, and likes/dislikes about the intervention arose from interviews following the intervention. Participants found the intervention to be user friendly and endorsed more likes than dislikes. Providers rated the intervention as being helpful and would recommend it to survivors of sexual assault. Findings suggest that the intervention is usable and fit for future effectiveness testing, filling an important gap in treatment for individuals who experience sexual assault.

Enhancing the Delivery of an Empirically-Supported Trauma-Focused Treatment for Adolescents: Providers' Views of the Role of Technology and Web-Based Resources

This mixed-methods study assessed providers' views of the use of technology in the delivery of an empirically supported mental health treatment for adolescents (Trauma-Focused Cognitive Behavioral Therapy; TF-CBT). Thematic qualitative interviews were conducted with nine experienced providers. Emerging themes served as the basis for the creation of a quantitative web-based survey, completed by 56 TF-CBT experts, to assess the perceived helpfulness of the recommendations. Technology was perceived as a useful, appealing, and familiar tool that could greatly enhance the delivery of this treatment modality with adolescents. Main recommendations included the creation of a mobile application targeting all of the treatment components and a website with developmentally appropriate resources for providers, caregivers, and teens. Technology may be a useful tool for enhancing service delivery and promoting engagement among youth receiving trauma-focused mental health treatment.

Pilot Evaluation of a Tablet-Based Application to Improve Quality of Care in Child Mental Health Treatment

Mental health systems need scalable solutions that can reduce the efficacy-effectiveness gap and improve mental health outcomes in community mental health service settings. Two major challenges to delivery of high-quality care are providers' fidelity to evidence-based treatment models and children's and caregivers' engagement in the treatment process. We developed a novel, tablet-based application designed to enhance via technology the quality of delivery of trauma-focused cognitive-behavioral therapy (TF-CBT). We piloted its use in four community mental health service organizations using a blocked randomized controlled trial to examine the feasibility of implementing tablet-facilitated TF-CBT versus standard TF-CBT with 13 providers and 27 families. Provider fidelity and child engagement in treatment were observationally measured via session audio recording. Parent and child perceptions of the tablet application were assessed using structured interviews and mixed-method analyses. Providers actively and appropriately used tablet TF-CBT to facilitate treatment activities. Providers and families expressed high satisfaction with its use, demonstrating acceptability of this approach. Youth and caregivers in both conditions reported high alliance with their providers. Overall, we found that tablet-facilitated treatment is accepted by providers and families and may be integrated into mental health treatment with minimal training. Further study is needed to examine the extent to which technology-based applications may enhance the reach, quality, and clinical outcomes of mental health treatment delivered to children and families.

Effectiveness and Efficiency of Observationally Assessing Fidelity to a Family-Centered Child Intervention: A Quasi-Experimental Study

Assessment of fidelity that is effective, efficient, and differentiates from usual practices is critical for effectively implementing evidence-based programs for families. This quasi-experiemntal study sought to determine whether observational ratings of fidelity to the Family Check-Up (FCU) could differentiate between levels of clinician training in the model, and from services as usual, and whether rating segments of sessions could be equivalent to rating complete sessions. Coders rated 75 videotaped sessions-complete and 20-min segments-for fidelity, using a valid and reliable rating system across three groups: (a) highly trained in FCU with universal, routine monitoring; (b) minimally trained in FCU with optional, variable monitoring; and (c) services as usual with no training in the FCU. We hypothesized that certain dimensions of fidelity would differ by training, whereas others would not. The results indicated that, as expected, one dimension of fidelity to the FCU, Conceptually accurate to the FCU, was reliably different between the groups (χ² = 44.63, p < .001). The differences observed were in the expected direction, showing higher scores for therapists with more training. The rating magnitude of session segments largely did not differ from those of complete session ratings; however, interrater reliabilities were low for the segments. Although observational ratings were shown to be sensitive to the degree of training in the FCU on a unique and theoretically critical dimension, observational coding of complete sessions is resource intensive and limits scalability. Additional work is needed to reduce the burden of assessing fidelity to family-centered programs.

Pilot to policy: statewide dissemination and implementation of evidence-based treatment for traumatized youth

BACKGROUND

A model for statewide dissemination of evidence-based treatment (EBT) for traumatized youth was piloted and taken to scale across North Carolina (NC). This article describes the implementation platform developed, piloted, and evaluated by the NC Child Treatment Program to train agency providers in Trauma-Focused Cognitive Behavioral Therapy using the National Center for Child Traumatic Stress Learning Collaborative (LC) Model on Adoption & Implementation of EBTs. This type of LC incorporates adult learning principles to enhance clinical skills development as part of training and many key implementation science strategies while working with agencies and clinicians to implement and sustain the new practice.

METHODS

Clinicians (n = 124) from northeastern NC were enrolled in one of two TF-CBT LCs that lasted 12 months each. During the LC clinicians were expected to take at least two clients through TF-CBT treatment with fidelity and outcomes monitoring by trainers who offered consultation by phone and during trainings. Participating clinicians initiated treatment with 281 clients. The relationship of clinician and client characteristics to treatment fidelity and outcomes was examined using hierarchical linear regression.

RESULTS

One hundred eleven clinicians completed general training on trauma assessment batteries and TF-CBT. Sixty-five clinicians met all mastery and fidelity requirements to meet roster criteria. One hundred fifty-six (55%) clients had fidelity-monitored assessment and TF-CBT. Child externalizing, internalizing, and post-traumatic stress symptoms, as well as parent distress levels, decreased significantly with treatment fidelity moderating child PTSD outcomes. Since this pilot, 11 additional cohorts of TF-CBT providers have been trained to these roster criteria.

CONCLUSION

Scaling up or outcomes-oriented implementation appears best accomplished when training incorporates: 1) practice-based learning, 2) fidelity coaching, 3) clinical assessment and outcomes-oriented treatment, 4) organizational skill-building to address barriers for agencies, and 5) linking clients to trained clinicians via an online provider roster. Demonstrating clinician performance and client outcomes in this pilot and subsequent cohorts led to legislative support for dissemination of a service array of EBTs by the NC Child Treatment Program.

Qualitative Reports of How and When Therapists Adapt Children's Evidence-Based Practices during Community Implementation

This study analyzed qualitative therapist reports of adaptations to the delivery of multiple evidence-based practices (EBPs) within the context of a system-driven reform of children's community mental health services to understand how therapists adapt EBPs as well as contexts of these adaptations to identify when these adaptations are made. The study sought to complement and expand upon previous quantitative survey findings of two categories of Augmenting and Reducing/Reordering adaptations to EBPs. Data included interviews from 60 therapists (88.3% female, 61.7% Latina/o, 80.0% unlicensed) across 20 program sites in 11 mental health agencies that served racial/ethnically diverse children. Interviews were coded to identify themes surrounding the types of adaptations made and the contexts for these adaptations. The majority of therapists' qualitative descriptions of adaptations converged with the 2 broad categories in the Augmenting and Reducing/Reordering Framework, with therapists describing augmenting (e.g., modifying presentation, lengthening or extending pacing) most often, and reducing/reordering adaptations were discussed less frequently. Child and family characteristics were most frequently cited as indications prompting adaptations; however, the specific characteristics motivating adaptations differed by type. Therapists reporting using augmenting adaptations in the context of a wide range of client characteristics, whereas reducing/reordering adaptations occurred more specifically as a function of clinical presentation, family and caregiver functioning, and emergent life events. Therapists described making adaptations to improve the fit of multiple EBPs for the clients they served. Findings could have implications for implementation efforts with diverse clients served in community settings.

Objective coding of content and techniques in workplace-based supervision of an EBT in public mental health

BACKGROUND

Workplace-based clinical supervision as an implementation strategy to support evidence-based treatment (EBT) in public mental health has received limited research attention. A commonly provided infrastructure support, it may offer a relatively cost-neutral implementation strategy for organizations. However, research has not objectively examined workplace-based supervision of EBT and specifically how it might differ from EBT supervision provided in efficacy and effectiveness trials.

METHODS

Data come from a descriptive study of supervision in the context of a state-funded EBT implementation effort. Verbal interactions from audio recordings of 438 supervision sessions between 28 supervisors and 70 clinicians from 17 public mental health organizations (in 23 offices) were objectively coded for presence and intensity coverage of 29 supervision strategies (16 content and 13 technique items), duration, and temporal focus. Random effects mixed models estimated proportion of variance in content and techniques attributable to the supervisor and clinician levels.

RESULTS

Interrater reliability among coders was excellent. EBT cases averaged 12.4 min of supervision per session. Intensity of coverage for EBT content varied, with some discussed frequently at medium or high intensity (exposure) and others infrequently discussed or discussed only at low intensity (behavior management; assigning/reviewing client homework). Other than fidelity assessment, supervision techniques common in treatment trials (e.g., reviewing actual practice, behavioral rehearsal) were used rarely or primarily at low intensity. In general, EBT content clustered more at the clinician level; different techniques clustered at either the clinician or supervisor level.

CONCLUSIONS

Workplace-based clinical supervision may be a feasible implementation strategy for supporting EBT implementation, yet it differs from supervision in treatment trials. Time allotted per case is limited, compressing time for EBT coverage. Techniques that involve observation of clinician skills are rarely used. Workplace-based supervision content appears to be tailored to individual clinicians and driven to some degree by the individual supervisor. Our findings point to areas for intervention to enhance the potential of workplace-based supervision for implementation effectiveness.

TRIAL REGISTRATION

NCT01800266 , Clinical Trials, Retrospectively Registered (for this descriptive study; registration prior to any intervention [part of phase II RCT, this manuscript is only phase I descriptive results]).

A mixed-methods study of system-level sustainability of evidence-based practices in 12 large-scale implementation initiatives

Background

In recent decades, evidence-based practices (EBPs) have been broadly promoted in community behavioural health systems in the United States of America, yet reported EBP penetration rates remain low. Determining how to systematically sustain EBPs in complex, multi-level service systems has important implications for public health. This study examined factors impacting the sustainability of parent-child interaction therapy (PCIT) in large-scale initiatives in order to identify potential predictors of sustainment.

Methods

A mixed-methods approach to data collection was used. Qualitative interviews and quantitative surveys examining sustainability processes and outcomes were completed by participants from 12 large-scale initiatives.

Results

Sustainment strategies fell into nine categories, including infrastructure, training, marketing, integration and building partnerships. Strategies involving integration of PCIT into existing practices and quality monitoring predicted sustainment, while financing also emerged as a key factor.

Conclusions

The reported factors and strategies impacting sustainability varied across initiatives; however, integration into existing practices, monitoring quality and financing appear central to high levels of sustainability of PCIT in community-based systems. More detailed examination of the progression of specific activities related to these strategies may aide in identifying priorities to include in strategic planning of future large-scale initiatives.

Trial registration

ClinicalTrials.gov ID NCT02543359; Protocol number PRO12060529.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-017-0230-8) contains supplementary material, which is available to authorized users.

Supporting Homework Compliance in Cognitive Behavioural Therapy: Essential Features of Mobile Apps

Cognitive behavioral therapy (CBT) is one of the most effective psychotherapy modalities used to treat depression and anxiety disorders. Homework is an integral component of CBT, but homework compliance in CBT remains problematic in real-life practice. The popularization of the mobile phone with app capabilities (smartphone) presents a unique opportunity to enhance CBT homework compliance; however, there are no guidelines for designing mobile phone apps created for this purpose. Existing literature suggests 6 essential features of an optimal mobile app for maximizing CBT homework compliance: (1) therapy congruency, (2) fostering learning, (3) guiding therapy, (4) connection building, (5) emphasis on completion, and (6) population specificity. We expect that a well-designed mobile app incorporating these features should result in improved homework compliance and better outcomes for its users.

Structured Trauma-Focused CBT and Unstructured Play/Experiential Techniques in the Treatment of Sexually Abused Children: A Field Study With Practicing Clinicians

Structured, trauma-focused cognitive-behavioral therapy (CBT) techniques are widely considered an effective intervention for children who experienced sexual abuse. However, unstructured (i.e., nondirective) play/experiential techniques have a longer history of widespread promotion and are preferred by many practicing clinicians. No evidence is available, however, to determine how the integration of these techniques impacts treatment outcome. In this study, community-based clinicians who received training in a structured, trauma-focused cognitive-behavioral intervention administered pretreatment and posttreatment evaluations to 260 sexually abused children presenting with elevated posttraumatic stress. In addition, they completed a questionnaire describing the treatment techniques implemented with each child. Overall, significant improvement was observed for each of the six clinical outcomes. Regression analyses indicated that technique selection was a significant factor in posttreatment outcome for posttraumatic stress, dissociation, anxiety, and anger/aggression. In general, a greater utilization of the structured CBT techniques was related to lower posttreatment scores, whereas a higher frequency of play/experiential techniques was associated with higher posttreatment scores. However, no interaction effects were observed. The implication of these findings for clinical practice and future research are examined.

Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention

Careful fidelity monitoring and feedback are critical to implementing effective interventions. A wide range of procedures exist to assess fidelity; most are derived from observational assessments (Schoenwald and Garland, Psycholog Assess 25:146-156, 2013). However, these fidelity measures are resource intensive for research teams in efficacy/effectiveness trials, and are often unattainable or unmanageable for the host organization to rate when the program is implemented on a large scale. We present a first step towards automated processing of linguistic patterns in fidelity monitoring of a behavioral intervention using an innovative mixed methods approach to fidelity assessment that uses rule-based, computational linguistics to overcome major resource burdens. Data come from an effectiveness trial of the Familias Unidas intervention, an evidence-based, family-centered preventive intervention found to be efficacious in reducing conduct problems, substance use and HIV sexual risk behaviors among Hispanic youth. This computational approach focuses on "joining," which measures the quality of the working alliance of the facilitator with the family. Quantitative assessments of reliability are provided. Kappa scores between a human rater and a machine rater for the new method for measuring joining reached 0.83. Early findings suggest that this approach can reduce the high cost of fidelity measurement and the time delay between fidelity assessment and feedback to facilitators; it also has the potential for improving the quality of intervention fidelity ratings.

Lessons learned while building a trauma-informed public behavioral health system in the City of Philadelphia

Exposure to traumatic experiences among youth is a serious public health concern. A trauma-informed public behavioral health system that emphasizes core principles such as understanding trauma, promoting safety, supporting consumer autonomy, sharing power, and ensuring cultural competence, is needed to support traumatized youth and the providers who work with them. This article describes a case study of the creation and evaluation of a trauma-informed publicly funded behavioral health system for children and adolescents in the City of Philadelphia (the Philadelphia Alliance for Child Trauma Services; PACTS) using the Exploration, Preparation, Implementation, and Sustainment (EPIS) as a guiding framework. We describe our evaluation of this effort with an emphasis on implementation determinants and outcomes. Implementation determinants include inner context factors, specifically therapist knowledge and attitudes (N=114) towards evidence-based practices. Implementation outcomes include rate of PTSD diagnoses in agencies over time, number of youth receiving TF-CBT over time, and penetration (i.e., number of youth receiving TF-CBT divided by the number of youth screening positive on trauma screening). We describe lessons learned from our experiences building a trauma-informed public behavioral health system in the hopes that this case study can guide other similar efforts.

Toward Treatment Integrity: Developing an Approach to Measure the Treatment Integrity of a Dialectical Behavior Therapy Intervention With Homeless Youth in the Community

The current paper discusses an approach to measuring treatment integrity of dialectical behavioral therapy (DBT) when implemented within two programs providing services to street-involved youth in the community. Measuring treatment integrity is a critical component of effective implementation of evidence-based interventions in clinical practice, since sound treatment integrity increases confidence in client outcomes and intervention replicability. Despite being an essential part of implementation science, few studies report on treatment integrity, with limited research addressing either measurement tools or maintenance of treatment integrity. To address the lack of available treatment integrity measures, researchers in the current study developed and piloted a treatment integrity measure which pertain to the individual and group components of DBT. A total of 20 recordings were assessed using the treatment integrity measure. Results indicate that the community agency staff (e.g. youth workers, social workers & nurses) implemented the intervention as intended; increasing confidence in the outcome variables, the staffs' training and the replicability of the intervention. This article offers one approach to addressing treatment integrity when implementing evidence-based interventions, such as DBT in a community setting, and discusses the need for effective and feasible integrity measures that can be adopted in order to strengthen mental health practice in community settings.

In-session caregiver behavior predicts symptom change in youth receiving trauma-focused cognitive behavioral therapy (TF-CBT)

OBJECTIVE

Involving caregivers in trauma-focused treatments for youth has been shown to result in better outcomes, but it is not clear which in-session caregiver behaviors enhance or inhibit this effect. The current study examined the associations between caregiver behaviors during Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) and youth cognitive processes and symptoms.

METHOD

Participants were a racially diverse sample of Medicaid-eligible youth (ages 7-17) and their nonoffending caregivers (N = 71 pairs) who received TF-CBT through an effectiveness study in a community setting. Caregiver and youth processes were coded from audio-recorded sessions, and outcomes were measured using the Child Behavior Checklist (CBCL) and UCLA PTSD Reaction Index for Diagnostic and Statistical Manual for Mental Disorders-Fourth Edition (DSM-IV; UPID) at 3, 6, 9, and 12 months postintake.

RESULTS

Piecewise linear growth curve modeling revealed that during the trauma narrative phase of TF-CBT, caregivers' cognitive-emotional processing of their own and their child's trauma-related reactions predicted decreases in youth internalizing and externalizing symptoms over treatment. Caregiver support predicted lower internalizing symptoms over follow-up. In contrast, caregiver avoidance and blame of the child predicted worsening of youth internalizing and externalizing symptoms over follow-up. Caregiver avoidance early in treatment also predicted worsening of externalizing symptoms over follow-up. During the narrative phase, caregiver blame and avoidance were correlated with more child overgeneralization of trauma beliefs, and blame was also associated with less child accommodation of balanced beliefs.

CONCLUSIONS

The association between in-session caregiver behaviors and youth symptomatology during and after TF-CBT highlights the importance of assessing and targeting these behaviors to improve clinical outcomes. (PsycINFO Database Record

[Trauma-focused Cognitive-behavioral Therapy with children and adolescents: Practice, evidence base, and future directions]

Trauma-focused cognitive behavioral therapy (Tf-CBT) is an evidence-based trauma therapy for children and adolescents aged 3–17 years with posttraumatic stress disorder (PTSD). One supportive caregiver is regularly included in the therapeutic process. Tf-CBT consists of eight components and can be summarized by the acronym P.R.A.C.T.I.C.E: Psychoeducation and Parenting skills, Relaxation skills, Affective modulation skills, Cognitive coping skills, Trauma narrative and cognitive processing of the traumatic event(s), In vivo mastery of trauma reminders, Conjoint child-parent sessions, and Enhancing safety and future developmental trajectory. Based on a total of 13 randomized controlled trials, several meta-analyses and systematic reviews confirm the efficacy of Tf-CBT, which received evidence level Ia in several international guidelines and is therefore recommended as first-line treatment for children and adolescents with PTSD.

Effects of dialectical behavior therapy skills training on outcomes for mental health staff in a child and adolescent residential setting

Training in Dialectical Behavior Therapy (DBT) skills coaching is desirable for staff in psychiatric settings, due to the efficacy of DBT in treating difficult patient populations. In such settings, training resources are typically limited, and staff turnover is high, necessitating brief training. This study evaluated the effects of a brief training in DBT skills coaching for nursing staff working in a child and adolescent psychiatric residential program. Nursing staff (n = 22) completed assessments of DBT skill knowledge, burnout, and stigma towards patients with borderline personality disorder (BPD) before and after a six-week DBT skills coaching training. Repeated measure ANOVAs were conducted to examine changes on all measures from pre- to post- treatment and hierarchical linear regressions to examine relationships between pre- training DBT knowledge, burnout, and BPD stigma and these same measures post-training. The brief DBT skill coaching training significantly increased DBT knowledge (p = .007) and decreased staff personal (p = .02) and work (p = .03) burnout and stigma towards BPD patients (p = .02). Burnout indices and BPD stigma were highly correlated at both time points (p < .001); however, while pre-training BPD stigma significantly predicted post-training client burnout (p = .04), pre-training burnout did not predict post-training BPD stigma. These findings suggest that brief training of psychiatric nursing staff in DBT skills and coaching techniques can result in significant benefits, including reduced staff burnout and stigma toward patients with BPD-related problems, and that reducing BPD stigma may particularly promote lower burnout.

Development and Pilot Evaluation of a Tablet-Based Application to Improve Quality of Care in Child Mental Health Treatment

Background

Children need access to high quality mental health care. Effective treatments now exist for a wide range of mental health conditions. However, these interventions are delivered with variable effectiveness in traditional mental health service settings. Innovative solutions are needed to improve treatment delivery quality and effectiveness.

Objective

The aim of this study was to develop a scalable, sustainable technology-based approach to improve the quality of care in child mental health treatment.

Methods

A tablet-based resource was developed with input from mental health training experts, mental health providers, and patients. A series of qualitative data collection phases (ie, expert interviews, patient and provider focus groups, usability testing) guided the initial concept and design of the resource, and then its refinement. The result was an iPad-based “e-workbook” designed to improve child engagement and provider fidelity in implementation of a best-practice treatment. We are currently conducting a small scale randomized controlled trial to evaluate the feasibility of e-workbook facilitated child mental health treatment with 10 providers and 20 families recruited from 4 local community-based mental health clinics.

Results

Usability and focus group testing yielded a number of strong, favorable reactions from providers and families. Recommendations for refining the e-workbook also were provided, and these guided several improvements to the resource prior to initiating the feasibility trial, which is currently underway.

Conclusions

This study aimed to develop and preliminarily evaluate a tablet-based application to improve provider fidelity and child engagement in child mental health treatment. If successful, this approach may serve as a key step toward making best-practice treatment more accessible to children and families. As various technologies continue to increase in popularity worldwide and within the health care field more specifically, it is essential to rigorously test the usability, feasibility, acceptability, and effectiveness of novel health technology solutions. It is also essential to ensure that patients and providers drive decision making that supports the development of these resources to ensure that they can be seamlessly integrated into practice.

Trial Registration

Clinicaltrials.gov NCT01915160; https://clinicaltrials.gov/ct2/show/NCT01915160 (Archived by WebCite at http://www.webcitation.org/6cPIiQDpu)

Building an evidence-base for the training of evidence-based treatments in community settings: Use of an expert-informed approach

In order to make EBTs available to a large number of children and families, developers and expert therapists have used their experience and expertise to train community-based therapists in EBTs. Understanding current training practices of treatment experts may be one method for establishing best practices for training community-based therapists prior to comprehensive empirical examinations of training practices. A qualitative study was conducted using surveys and phone interviews to identify the specific procedures used by treatment experts to train and implement an evidence-based treatment in community settings. Twenty-three doctoral-level, clinical psychologists were identified to participate because of their expertise in conducting and training Parent-Child Interaction Therapy. Semi-structured qualitative interviews were completed by phone, later transcribed verbatim, and analyzed using thematic coding. The de-identified data were coded by two independent qualitative data researchers and then compared for consistency of interpretation. The themes that emerged following the final coding were used to construct a training protocol to be empirically tested. The goal of this paper is to not only understand the current state of training practices for training therapists in a particular EBT, Parent-Child Interaction Therapy, but to illustrate the use of expert opinion as the best available evidence in preparation for empirical evaluation.

Providers' perspectives regarding the development of a web-based depression intervention for Latina/o youth

Latina/o youth appear to be at significant risk for depression and, of concern, is the high underutilization of mental health services observed in this population. There is a tremendous need for novel intervention methods to better serve the unique needs of this population. This article describes the development of Rise Above (Siempre Sale el Sol), a Web-based, self-help, depression intervention for Latina/o adolescents funded by the National Institute of Mental Health. We applied a cultural adaptation model to an evidence-based depression treatment to reduce potential service barriers and increase the relevance and potential efficacy of the intervention for Latina/o youth. We conducted thematic interviews with 32 national experts to obtain feedback that would inform our application of the cultural adaptation model, the potential efficacy of the intervention, and the feasibility of implementation. Future directions for the evaluation of Rise Above (Siempre Sale el Sol) are described.

Guidelines for establishing a telemental health program to provide evidence-based therapy for trauma-exposed children and families

Although similar rates of traumatic experiences exist in both rural and urban settings, mental health resources available to those living in rural areas are often scarce. Limited resources pose a problem for children and families living in rural areas, and several barriers to service access and utilization exist including reduced anonymity, few "after hours" services, decreased availability of evidence-based treatments, few specialty clinics, and expenses associated with travel, taking time off work, and provision of childcare. As a solution, the authors discuss the utility, use, and set-up of a telemental health program within an existing community outreach program. Suggestions for establishing a telemental health clinic are presented along with guidelines for the delivery of trauma-focused, cognitive-behavioral therapy (TF-CBT) via telemental health videoconferencing technology. Specific guidelines discussed include (1) establishing and using community partnerships, (2) Memoranda of Understanding (MOU), (3) equipment setup and technological resources, (4) videoconferencing software, (5) physical setup, (6) clinic administration, (7) service reimbursement and start-up costs, (8) therapy delivery modifications, and (9) delivering culturally competent services to rural and remote areas.

Development and preliminary evaluation of a behavioural HIV-prevention programme for teenage girls of Latino descent in the USA

National data suggests that teenage girls of Latino descent in the USA are disproportionately affected by HIV, with the US Centers for Disease Control and Prevention reporting the rate of new infections being approximately four times higher compared to White women of comparable age . This paper highlights the need for an effective single-sex HIV-prevention programme for teenage girls of Latino descent and describes the development and preliminary evaluation of Chicas Healing, Informing, Living and Empowering (CHILE), a culturally-tailored, HIV-prevention programme exclusively for teenage girls of Latino descent that was adapted from Sisters Informing, Healing, Living and Empowering (SiHLE), an evidence-based HIV- prevention program that is culturally tailored for African American young women. Theatre testing, a pre-testing methodology to assess consumer response to a demonstration of a product, was utilised to evaluate the relevance and utility of the HIV programme as well as opportunities for the integration of cultural constructs. Future directions for the evaluation of CHILE are discussed.

Guidelines for establishing a telemental health program to provide evidence-based therapy for trauma-exposed children and families

Although similar rates of traumatic experiences exist in both rural and urban settings, mental health resources available to those living in rural areas are often scarce. Limited resources pose a problem for children and families living in rural areas, and several barriers to service access and utilization exist including reduced anonymity, few "after hours" services, decreased availability of evidence-based treatments, few specialty clinics, and expenses associated with travel, taking time off work, and provision of childcare. As a solution, the authors discuss the utility, use, and set-up of a telemental health program within an existing community outreach program. Suggestions for establishing a telemental health clinic are presented along with guidelines for the delivery of trauma-focused, cognitive-behavioral therapy (TF-CBT) via telemental health videoconferencing technology. Specific guidelines discussed include (1) establishing and using community partnerships, (2) Memoranda of Understanding (MOU), (3) equipment setup and technological resources, (4) videoconferencing software, (5) physical setup, (6) clinic administration, (7) service reimbursement and start-up costs, (8) therapy delivery modifications, and (9) delivering culturally competent services to rural and remote areas.