Group CBT for Externalizing Disorders in Urban Schools: Effect of Training Strategy on Treatment Fidelity and Child Outcomes

Utilizing the CFIR framework for mapping the facilitators and barriers of implementing teachers led school mental health programs - a scoping review

BACKGROUND

Ample evidence has been generated regarding the effectiveness of school-based mental health interventions as part of the continuum of care for children and capacity building of teachers to deal with the emotional and behavioral challenges of students. The increasing trend of utilization of teachers as the natural support system of children and the huge financial impact of public health interventions highlights the need to review all available evidence regarding multilevel factors that facilitate or pose a challenge to the provision of School Mental Health Programs (SMHP) using teachers as providers. The current review aims to map extracted evidence under the Consolidated Framework for Implementation Research (CFIR) domains to support future implementation research on school-based mental health services.

METHODS

The scoping review included experimental, qualitative studies, and systematic reviews involving teacher-led mental health programs conducted in the school setting to improve the socio-emotional well-being of children and adolescents irrespective of time and geographical limitations. All (published and unpublished) evidence in English from Pubmed, Cochrane database of systematic reviews and clinical trials, Scopus, and Science Direct was searched using keywords and Boolean combinations and extracted using study designs, place of study, year of publication, sample size, and target population. A qualitative analysis of implementation facilitators and barriers cited by the studies was carried out and mapped on CFIR.

RESULTS

This review identified barriers and facilitators of implementation across school-based mental health Programs in 29 studies. The major emerging themes were those related to the inner setting of the organization, the process of implementation, and the characteristics of individuals involved in implementation. These included the availability of structural characteristics, positive school culture, organizational readiness, committed leadership, and beliefs of the providers.

CONCLUSION

Findings highlight the need for early assessment of contextual factors acting as barriers and facilitators and careful execution following realistic planning and stakeholders' engagement to ensure the success of SMHP.

Clinical Consultation During a Trauma-Focused Cognitive Behavioral Therapy Community-Based Learning Collaborative: Examination of Consultation Content, Consultative Strategies, and Provider Engagement

Consultation following evidence-based practice (EBP) training enhances the uptake of EBPs. Yet, little is known about what occurs during consultation, and it is often difficult for providers to engage in consultation. This study examined provider engagement in consultation and the content and strategies used during consultation following training in Trauma-focused Cognitive Behavioral Therapy (TF-CBT) as part of a community-based learning collaborative (CBLC). Minute-to-minute live coding of consultation calls revealed most content was clinically-oriented and the most common strategies used by consultants were didactic in nature. Providers with more years of professional experience and those with greater TF-CBT knowledge attended significantly more consultation calls. Providers with a greater average weekly caseload and providers who were supervisors presented significantly more cases on calls. Providers with greater TF-CBT knowledge spoke significantly more minutes on calls. Consistent with previous work, findings highlight difficulties with provider engagement in consultation and that providers with more baseline expertise are most likely to be engaged. Findings suggest tailoring EBP training efforts to better engage providers at greatest risk for low engagement.

Effects of Training in Cognitive Behavioural Therapy and Motivational Interviewing on Mental Health Practitioner Behaviour: A Systematic Review and Meta-Analysis

Effective training of mental health professionals is crucial for bridging the gap between research and practice when delivering cognitive behavioural therapy (CBT) and motivational interviewing (MI) within community settings. However, previous research has provided inconclusive evidence regarding the impact of training efforts. The current study aimed to systematically search, review and synthesize the literature on CBT and MI training to assess its effect on practitioner behavioural outcomes. Following prospective registration, a literature search was conducted for studies where mental health practitioners were exposed to training in face-to-face CBT or MI, reporting on at least one quantitative practitioner behavioural outcome. A total of 116 studies were eligible for the systematic review, and 20 studies were included in four meta-analyses. The systematic review highlights the need to establish psychometrically valid outcome measures for practitioner behaviour. Results of the meta-analyses suggest that training has a greater effect on practitioner behaviour change compared to receiving no training or reading a treatment manual. Training combined with consultation/supervision was found to be more effective than training alone, and no differences were found between face-to-face and online training. Results should be interpreted with caution due to methodological limitations in the primary studies, large heterogeneity, and small samples in the meta-analyses. Future directions are discussed.

Implementation fidelity, student outcomes, and cost-effectiveness of train-the-trainer strategies for Masters-level therapists in urban schools: results from a cluster randomized trial

BACKGROUND

Little is known about the effectiveness and cost-effectiveness of train-the-trainer implementation strategies in supporting mental health evidence-based practices in schools, and about the optimal level of support needed for TT strategies.

METHODS

The current study is part of a larger type 2 hybrid cluster randomized controlled trial. It compares two train-the-trainer strategies, Train-the-Trainer (TT) and Train-the-Trainer plus ongoing consultation for trainers (TT +) on the delivery of a group cognitive behavioral treatment protocol for anxiety disorders. Participants were 33 therapists, 29 supervisors, and 125 students who were at risk for anxiety disorders from 22 urban schools. Implementation outcomes were implementation fidelity and treatment dosage. Student outcomes were child- and parent-reported symptoms of anxiety, child-reported symptoms of depression, and teacher-reported academic engagement. We estimated the cost of implementing the intervention in each condition and examined the probability that a support strategy for supervisors (TT vs TT +) is a good value for varying values of willingness to pay.

RESULTS

Therapists in the TT and TT + conditions obtained similarly high implementation fidelity and students in the conditions received similar treatment dosages. A mixed effects modeling approach for student outcomes revealed time effects for symptoms of anxiety and depression reported by students, and emotional disaffection reported by teachers. There were no condition or condition × times effects. For both conditions, the time effects indicated an improvement from pre-treatment to post-treatment in symptoms of anxiety and depression and academic emotional engagement. The average cost of therapist, supervisor, and consultant time required to implement the intervention in each condition was $1002 for TT and $1431 for TT + (p = 0.01). There was a greater than 80% chance that TT was a good value compared to TT + for all values of willingness to pay per one-point improvement in anxiety scores.

CONCLUSIONS

A TT implementation approach consisting of a thorough initial training workshop for therapists and supervisors as well as ongoing supervision for therapists resulted in adequate levels of fidelity and student outcomes but at a lower cost, compared to the TT + condition that also included ongoing external expert consultation for supervisors.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT02651402.

Preparation for implementation of evidence-based practices in urban schools: A shared process with implementing partners

Background

Shifting organizational priorities can negatively affect the sustainment of innovations in community settings. Shifting priorities can present barriers to conducting clinical research in schools if a misalignment occurs between school district priorities and the aims of the study. Often this misalignment occurs due to a shift during the period between when the study is submitted for funding and when research activities begin. Participatory research approaches can be employed to restore alignment between study processes and school district priorities. The purpose of the study is to describe data from a shared process with district partners. The shared process resulted in modifications to the main study's implementation processes and strategies in order to restore alignment with evolving school priorities while remaining faithful to the aims of the study.

Method

Data originated from qualitative interviews conducted with 20 school district and school personnel in a large urban school district. Qualitative themes were organized into categories based on a social-ecological school implementation framework. Data from team meetings, meetings with school district administrators, and emails served to supplement and verify findings from interview analyses.

Results

Themes included barriers and facilitators at the macro-, school-, individual-, team-, and implementation quality levels. Adaptations were made to address barriers and facilitators and restore alignment with school district priorities. Most adaptations to study processes and implementation strategies focused on re-training and providing more information to school district coaches and school-based staff. New procedures were created, and resources were re-allocated for the larger study.

Conclusions

Findings were discussed in relation to the implementation literature in schools. Recommendations for sustaining strong collaboration among researchers and school partners are provided.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) in community mental health: Comorbidity and use of modules under the microscope

Sleep and circadian problems are intertwined with serious mental illness (SMI). Thus, optimizing treatments that target comorbid sleep and circadian problems and SMI is critical. Among adults with sleep and circadian problems and SMI, the present study conducted a secondary data analysis of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C). TranS-C targets a range of sleep and circadian problems and SMI with 15 modules, seven of which are optional. In a 'real world' sample recruited from a community setting (N=121, 52.07% female, 42.97% African American or Black), we aimed to (1) elucidate patterns of sleep and circadian problems that met criteria for full diagnoses and subdiagnostic symptoms across (a) the full sample and (b) SMI diagnoses, and (2) determine whether TranS-C optional modules were delivered as intended based on participants' sleep and circadian problems. Results indicated that most participants (> 85.0%) had full diagnoses or subdiagnostic symptoms of two or more sleep and circadian problems. Further, participants exhibited heterogenous comorbidities between sleep and circadian problems and SMI diagnoses. Specifically, participants with a schizophrenia spectrum disorder (n=50), bipolar disorder (n=35), and major depressive disorder (n=26) exhibited 25, 24, and 21 patterns of sleep and circadian comorbidity, respectively. Notably, most participants with insomnia, hypersomnia, and an advanced or delayed circadian rhythm phase disorder did not receive the intended TranS-C optional modules designed to target these problems. Results underscore sleep and circadian and SMI diagnostic complexity in the community. Additionally, findings reveal discrepancies between intended and 'real-world' use of treatment modules. Future research investigating clinician decision-making-particularly when treating patients with comorbidities or using modularized treatments-is needed.

Development of an Online Training Platform and Implementation Strategy for School-Based Mental Health Professionals in Rural Elementary Schools: A Mixed-Methods Study

Children in rural settings are less likely to receive mental health services than their urban and suburban counterparts and even less likely to receive evidence-based care. Rural schools could address the need for mental health interventions by using evidence-based practices within a tiered system of supports such as positive behavioral interventions and supports. However, very few school professionals, with or without mental health training, have received training on evidence-based practices. Rural schools need implementation strategies focused on training to prepare school personnel for the implementation of interventions with fidelity. Little is known about training strategies that are feasible and appropriate for the rural school context. User-centered design is an appropriate framework for the development of training strategies for professionals in rural schools because of its participatory approach and the development of products that fit the context where they are going to be used. The purpose of the study was to develop and assess components of an online training platform and implementation strategy based on the user-centered design. Quantitative and qualitative data from 25 participants from an equal number of schools in rural areas of Pennsylvania were used in the study. A mixed-methods design utilizing complementary descriptive statistics and theme analyses indicated that the training platform and implementation strategy were perceived as highly acceptable, appropriate, feasible and usable by school professionals. The resulting training platform and implementation strategy will fill a void in the training literature in rural schools.

A Behavior-based Coding Tool for Assessing Supervisors' Adherence and Competence: Findings From a Motivational Interviewing Implementation Study

Supervision seems to be an essential part of therapist training, and thus also of implementing evidence-based practices. However, there is a shortage of valid and reliable instruments for objective assessment of supervision competence that include both global measures and frequency counts of behavior - two essential aspects of supervisory competence. This study tests the internal consistency and inter-rater reliability of an assessment tool that includes both these measures. Additionally, strategies and techniques used by ten supervisors in 35 Motivational interviewing supervision sessions are described. Codings were conducted after two separate coding training sessions. The internal consistency across the global measures was acceptable (α = 0.70; 0.71). After the second training, the inter-rater reliabilities for all frequency counts were in the moderate to good range, except for two that were in the poor range; inter-rater reliability for one of the four global measures was in the moderate range, and three were in the poor range. A prerequisite for identifying specific supervisor skills central to the development of therapist skills, teaching these skills to supervisors, and performing quality assurance of supervision, is to create instruments that can measure these behaviors. This study is a step in that direction.

Development and evaluation of a remote training strategy for the implementation of mental health evidence-based practices in rural schools: pilot study protocol

Background

An increasing number of schools in rural settings are implementing multi-tier positive behavioral interventions and supports (PBIS) to address school-climate problems. PBIS can be used to provide the framework for the implementation of evidence-based practices (EBPs) to address children’s mental health concerns. Given the large service disparities for children in rural areas, offering EBPs through PBIS can improve access and lead to better long-term outcomes. A key challenge is that school personnel need technical assistance in order to implement EBPs with fidelity and clinical effectiveness. Providing ongoing on-site support is not feasible or sustainable in the majority of rural schools, due to their remote physical location. For this reason, remote training technology has been recommended for providing technical assistance to behavioral health staff (BHS) in under-served rural communities.

Objectives

The purpose of this study is to use the user-centered design, guided by an iterative process (rapid prototyping), to develop and evaluate the appropriateness, feasibility, acceptability, usability, and preliminary student outcomes of two online training strategies for the implementation of EBPs at PBIS Tier 2.

Methods

The study will employ a pragmatic design comprised of a mixed-methods approach for the development of the training platform, and a hybrid type 2, pilot randomized controlled trial to examine the implementation and student outcomes of two training strategies: Remote Video vs. Remote Video plus Coaching.

Discussion

There is a clear need for well-designed remote training studies focused on training in non-traditional settings. Given the lack of well-trained mental health professionals in rural settings and the stark disparities in access to services, the development and pilot-testing of a remote training strategy for BHS in under-served rural schools could have a significant public health impact.

Ethics and dissemination

The project was reviewed and approved by the institutional review board. Results will be submitted to ClinicalTrials.gov and disseminated to community partners and participants, peer-reviewed journals, and academic conferences.

Trial registration

ClinicialTrials.gov, NCT05034198 and NCT05039164

Knowledge and attitudes of Implementation Support Practitioners-Findings from a systematic integrative review

BACKGROUND

It requires thoughtful planning and work to successfully apply and sustain research-supported interventions like healthcare treatments, social support, or preventive programs in practice. Implementation support practitioners (ISPs) such as facilitators, technical assistance providers, knowledge brokers, coaches or consultants may be involved to actively support the implementation process. This article presents knowledge and attitudes ISPs bring to their work.

METHODS

Building on a previously developed program logic, a systematic integrative review was conducted. Literature was sourced by searching nine electronic data bases, organizational websites, and by launching a call for publications among selected experts and social media. Article screening was performed independently by two researchers, and data from included studies were extracted by members of the research team and quality-assured by the lead researcher. The quality of included RCTs was assessed based on a framework by Hodder and colleagues. Thematic Analysis was used to capture information on knowledge and attitudes of ISPs across the included studies. Euler diagrams and heatmaps were used to present the results.

RESULTS

Results are based on 79 included studies. ISPs reportedly displayed knowledge about the clinical practice they work with, implementation / improvement practice, the local context, supporting change processes, and facilitating evidence-based practice in general. In particular, knowledge about the intervention to be implemented and its target population, specific improvement / implementation methods and approaches, organizational structures and sensitivities, training, and characteristics of (good) research was described in the literature. Seven themes describing ISPs' attitudes were identified: 1) professional, 2) motivated / motivating / encouraging / empowering, 3) empathetic / respectful / sensitive, 4) collaborative / inclusive, 5) authentic, 6) creative / flexible / innovative / adaptive, and 7) frank / direct / honest. Pertaining to a professional attitude, being responsive and focused were the most prevalent indicators across included publications.

CONCLUSION

The wide range and complexity of knowledge and attitudes found in the literature calls for a comprehensive and systematic approach to collaboratively develop a professional role for ISPs across disciplines. Embedding the ISP role in different health and social welfare settings will enhance implementation capacities considerably.

Self-Coding of Fidelity as a Potential Active Ingredient of Consultation to Improve Clinicians' Fidelity

A key goal for implementation science is the identification of evidence-based consultation protocols and the active ingredients within these protocols that drive clinician behavior change. The current study examined clinicians' self-coding of fidelity as a potential active ingredient of consultation for the Attachment and Biobehavioral Catch-up (ABC) intervention. It also examined two other potential predictors of clinician fidelity in response to consultation: dosage of consultation and working alliance. Twenty-nine clinicians (97% female, 62% White, M age = 34 years) participated in a year of weekly fidelity-focused ABC consultation sessions, for which clinicians self-coded fidelity and received consultant feedback on both their coding and their fidelity. Data from the ABC fidelity measure were available for 1067 sessions coded by consultants, and clinicians' self-coding accuracy was calculated from 1044 sessions coded by both clinicians and consultants. Alliance was measured with the Working Alliance Inventory-Trainee and Supervisor Versions. The study was observational, and fidelity and self-coding accuracy were modeled across time using hierarchical linear modeling. Clinicians' ABC fidelity, as well as their self-coding accuracy, increased over the course of consultation. Clinicians' self-coding accuracy predicted their initial fidelity and growth in fidelity. Working alliance was also linked to fidelity and self-coding accuracy. These results suggest that clinician self-coding should be further examined as an active ingredient of consultation. The study has important implications for the design of consultation procedures and fidelity assessments.

Study protocol: cluster randomized trial of consultation strategies for the sustainment of mental health interventions in under-resourced urban schools: rationale, design, and methods

Background

The school is a key setting for the provision of mental health services to children, particularly those underserved through traditional service delivery systems. School-wide Positive Behavioral Interventions and Supports (PBIS) is a tiered approach to service delivery based on the public health model that schools use to implement universal (Tier 1) supports to improve school climate and safety. As our prior research has demonstrated, PBIS is a useful vehicle for implementing mental and behavioral health evidence-based practices (EBPs) at Tier 2 for children with, or at risk for, mental health disorders. Very little research has been conducted regarding the use of mental health EBPs at Tier 2 or how to sustain implementation in schools.

Methods/design

The main aim of the study is to compare fidelity, penetration, cost-effectiveness, and student outcomes of Tier 2 mental health interventions across 2 sustainment approaches for school implementers in 12 K-8 schools. The study uses a 2-arm, cluster randomized controlled trial design. The two arms are: (a) Preparing for Sustainment (PS)—a consultation strategy implemented by school district coaches who receive support from external consultants, and (b) Sustainment as Usual (SAU)—a consultation strategy implemented by school district coaches alone. Participants will be 60 implementers and 360 students at risk for externalizing and anxiety disorders. The interventions implemented by school personnel are: Coping Power Program (CPP) for externalizing disorders, CBT for Anxiety Treatment in Schools (CATS) for anxiety disorders, and Check-in/Check-out (CICO) for externalizing and internalizing disorders. The Interactive Systems Framework (ISF) for Dissemination and Implementation guides the training and support procedures for implementers.

Discussion

We expect that this study will result in a feasible, effective, and cost-effective strategy for sustaining mental health EBPs that is embedded within a multi-tiered system of support. Results from this study conducted in a large urban school district would likely generalize to other large, urban districts and have an impact on population-level child mental health.

Trial registration ClinicalTrials.gov identifier number NCT04869657. Registered May 3, 2021.

Substance Use Outcomes from Two Formats of a Cognitive-Behavioral Intervention for Aggressive Children: Moderating Roles of Inhibitory Control and Intervention Engagement

Although cognitive-behavioral interventions have reduced the risk of substance use, little is known about moderating factors in children with disruptive behaviors. This study examined whether aggressive preadolescents’ inhibitory control and intervention engagement moderates the effect of group versus individual delivery on their substance use. Following screening for aggression in 4th grade, 360 children were randomly assigned to receive the Coping Power intervention in either group or individual formats. The sample was primarily African American (78%) and male (65%). Assessments were made of children’s self-reported substance use from preintervention through a six-year follow-up after intervention, parent-reported inhibitory control at preintervention, and observed behavioral engagement in the group intervention. Multilevel growth modeling found lower increases in substance use slopes for children with low inhibitory control receiving individual intervention, and for children with higher inhibitory control receiving group intervention. Children with low inhibitory control but who displayed more positive behavioral engagement in the group sessions had slower increases in their substance use than did similar children without positive engagement. Aggressive children’s level of inhibitory control can lead to tailoring of group versus individual delivery of intervention. Children’s positive behavioral engagement in group sessions is a protective factor for children with low inhibitory control.

Strategies that Promote Therapist Engagement in Active and Experiential Learning: Micro-Level Sequential Analysis

Therapists' active learning increases treatment fidelity, but research is needed on supervisory strategies to engage therapists in active learning. This study used sequential analysis to examine consultant behaviors associated with increased and decreased probability of eliciting therapists' active learning. The study included 162 consultation sessions from 27 community therapists implementing Attachment and Biobehavioral Catch-up. Consultants' client discussion, information provision, and modeling were associated with reduced likelihood of active learning. Consultants' questions, engagement in active learning strategies, use of video, and silence were associated with greater likelihood of therapist active learning. These findings inform supervisors' attempts to encourage active learning.

Effects of Fidelity-Focused Consultation on Clinicians' Implementation: An Exploratory Multiple Baseline Design

Identification of effective consultation models could inform implementation efforts. This study examined the effects of a fidelity-focused consultation model among community-based clinicians implementing Attachment and Biobehavioral Catch-up. Fidelity data from 1217 sessions from 7 clinicians were examined in a multiple baseline design. In fidelity-focused consultation, clinicians received feedback from consultants' fidelity coding, and also coded their own fidelity. Clinicians' fidelity increased after fidelity-focused consultation began, but did not increase during other training periods. Fidelity was sustained for 30 months after consultation ended. Findings suggest that consultation procedures involving fidelity coding feedback and self-monitoring of fidelity may promote implementation outcomes.

Pilot Study for the Fidelity, Acceptability and Effectiveness of a PBIS Program plus Mental Health Supports in Under-resourced Urban Schools

This paper describes implementation (fidelity, perceived acceptability) and tier 1 and tier 2 outcomes of a school-wide positive behavior interventions and supports approach (PBIS) including mental health supports at tier 2 in two K-8 urban schools. Interventions for tier 2 consisted of three manualized group cognitive behavioral therapy (GCBT) protocols for externalizing behavior problems, depression and anxiety. tier 1 and tier 2 interventions were implemented with fidelity but program feasibility for tier 2 was in question because school personnel needed a great deal of external support in order to implement the interventions. tier 1 interventions were associated with a decrease in office discipline referrals. Students participating in GCBT showed a significant decrease in mental health diagnostic severity at post-treatment. A discussion of perceived and actual implementation barriers and how they were addressed is provided. Implications for practice in low-income urban schools are discussed.