Behavioral Rehearsal for Analogue Fidelity: Feasibility in a State-Funded Children's Mental Health Initiative

Developing the Workforce of the Digital Future: mHealth Competency and Fidelity Measurement in Community-Based Care

Integrating mobile health (mHealth) interventions into settings that serve diverse patient populations requires that prerequisite professional competencies are delineated and that standards for clinical quality assurance can be pragmatically assessed. Heretofore, proposed mHealth competencies have been broad and have lacked a framework to support specific applications. We outline the meta-competencies identified in the literature relevant to mHealth interventions and demonstrate how these meta-competencies can be integrated with population- and intervention-related competencies to help guide a pragmatic approach to competency assessment. We present a use case based on FOCUS-an evidence-based mHealth intervention designed for individuals with serious mental illness and currently being implemented in geographically and demographically diverse community behavioral health settings. Subsequent to identifying the cross-cutting competencies relevant to the target population (outpatients experiencing psychotic symptoms), substratal intervention (Cognitive Behavioral Therapy for psychosis), and treatment modality (mHealth), we detail the development process of an mHealth fidelity monitoring system (mHealth-FMS). We adhered to a published sequential 5-step process to design a fidelity monitoring system that aligns with our integrated mHealth competency framework and that was guided by best practices prescribed by the Treatment Fidelity Workgroup of the National Institutes of Health Behavior Change Consortium. The mHealth-FMS is intended to enhance both clinical and implementation outcomes by grounding the mHealth interventionist and the system of care in which they operate in the core functions, tasks, knowledge, and competencies associated with system-integrated mHealth delivery. Future research will explore acceptability and feasibility of the mHealth-FMS.

Identifying Core Competencies for Remote Delivery of Psychological Interventions: A Rapid Review

OBJECTIVE

The COVID-19 pandemic led to a rapid shift toward remote delivery of psychological interventions and transition to voice-only and video communication platforms. However, agreement is lacking on key competencies that are aligned with equitable approaches for standardized training and supervision of remote psychological intervention delivery. A rapid review was conducted to identify and describe competencies that could inform best practices of remote services delivery during and after the COVID-19 pandemic.

METHODS

Scopus, MEDLINE, and PsycINFO were searched for literature published in English (2015-2021) on competencies for synchronous, remote psychological interventions that can be measured through observation.

RESULTS

Of 135 articles identified, 12 met inclusion criteria. Studies targeted populations in high-income countries (11 in the United States and Canada, one in Saudi Arabia) and focused on specialist practitioners, professionals, or trainees in professional or prelicensure programs working with adult populations. Ten skill categories were identified: emergency and safety protocols for remote services, facilitating communication over remote platforms, remote consent procedures, technological literacy, practitioner-client identification for remote services, confidentiality during remote services, communication skills during remote services, engagement and interpersonal skills for remote services, establishing professional boundaries during remote services, and encouraging continuity of care during remote services.

CONCLUSIONS

These 10 skills domains can offer a foundation for refinement of discrete, individual-level competencies that can be aligned with global initiatives promoting use of observational competency assessment during training and supervision programs for psychological interventions. More research is needed on identification of and agreement on remote competencies and on their evaluation.

What is the Status of Multi-Informant Treatment Fidelity Research?

OBJECTIVE

The precise measurement of treatment fidelity (quantity and quality in the delivery of treatment strategies in an intervention) is essential for intervention development, evaluation, and implementation. Various informants are used in fidelity assessment (e.g., observers, practitioners [clinicians, teachers], clients), but these informants often do not agree on ratings. This scoping review aims to ascertain the state of science around multi-informant assessment of treatment fidelity.

METHOD

A literature search of articles published through December 2021 identified 673 articles. Screening reduced the number of articles to 44, and the final study set included 35 articles.

RESULTS

There was substantial variability across studies regarding study design, how fidelity was operationalized, and how reliability was defined and assessed. Most studies evaluated the agreement between independent observers and practitioner-report, though several other informant pairs were assessed. Overall, findings suggest that concordance across fidelity informants was low to moderate, with a few key exceptions.

CONCLUSIONS

It is difficult to draw clear conclusions about the degree to which single versus multiple informant assessment is needed to produce an accurate and complete picture of treatment fidelity. The field needs to take steps to determine how to leverage multi-informant assessment to accurately assess treatment fidelity.

The Cognitive Behavioral Therapy Competence Scale (CCS): initial development and validation

Background:

Nearly one-third of youth are affected by a mental health disorder, and the majority do not receive adequate care. To improve clinical outcomes among youth, efforts have been made to train providers in evidence-based mental health practices, such as cognitive behavioral therapy (CBT). Such efforts call for valid assessment measures that can inform and evaluate training activities.

Aims:

This study presents the development and validation of the CBT Competence Scale (CCS), a brief self-report measure to assess provider competence for CBT delivery.

Method:

Participants were 387 school mental health professionals (SMHPs) working with students in Michigan, USA. Initial items (n=59) were developed to evaluate competence in delivering common elements of CBT, with competence conceptualized as covering domains of knowledge, perception, and use of CBT techniques. CCS validation proceeded in three steps: using item response theory to select the most important items for assessing knowledge, evaluating the factor structure using exploratory and then confirmatory factor analyses, and examining reliability and validity of the resultant measure.

Results:

The validated CCS measure consists of four dimensions of CBT competence across 33 items: Non-behavioral skills, Behavioral skills, Perceptions, and Knowledge. The CCS demonstrated excellent internal consistency and good construct-based validity.

Conclusions:

The CCS holds promise as a valid, informative measure of CBT competence appropriate for the school setting, with potential for application in other environments such as mental health clinics.

Key learning aims

(1)

Development of a Brief Group CBT Intervention to Reduce COVID-19 Related Distress Among School-Age Youth

School-aged youth have been significantly impacted by the COVID-19 pandemic. The effects of the pandemic will likely have long-standing effects on the well-being of youth, and access to mental health care is even more critical during this time. For the past 5 years, TRAILS (Transforming Research into Action to Improve the Lives of Students) has been working throughout the state to increase utilization of evidence-based mental health practices among K-12 school mental health professionals (SMHPs). By leveraging SMHPs who are widely accessible to students, TRAILS seeks to improve youth access to effective mental health care and reduce current mental health inequities. In March 2020, TRAILS responded to the COVID-19 pandemic by developing a group manual designed to be delivered virtually by SMHPs to help students develop effective coping skills to mitigate the impact of COVID-19. TRAILS focuses on promoting use of CBT and mindfulness, as these skills are ideally suited for school-based delivery, and thus the new manual, Coping with COVID-19 (CC-19), was grounded in these modalities. This article will describe the design, development, and deployment of the CC-19 program to address the mental health needs of students in the context of the pandemic. Early acceptability and penetration data will also be discussed.

Role-Play Assessment of Therapist Adherence and Skill in Implementation of Trauma-Focused Cognitive-Behavioral Therapy

Numerous efforts are underway to train clinicians in evidence-based practices. Unfortunately, the field has few practical measures of therapist adherence and skill with which to judge the success of these training and implementation efforts. One possible assessment method is using behavioral rehearsal, or role-play, as an analogue for therapist in-session behavior. The current study describes aspects of reliability, validity and utility of a behavioral role-play assessment developed to evaluate therapist adherence and skill in implementing Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT). TF-CBT role-play assessments were conducted with a sample of 43 therapists as part of a larger training study. The TF-CBT role-play assessments were independently coded for TF-CBT adherence and skill by a certified TF-CBT trainer and three clinical psychology doctoral students. Findings indicated good interrater reliability for the individual items (ICC: M = .71, SD = .15). Regarding utility, 67.19% (n = 43/64) of contacted therapists completed the role-play assessment, which took an average of 30 min (M = 31.42, SD = 5.65) to complete and 60 min (M = 62.84, SD = 11.31) to code. Therapists with a master's degree were more likely to complete the role-play assessment than those with other degrees but no other differences in demographic variables, practice characteristics, or TF-CBT knowledge or training were found between participants and nonparticipants. Role-play assessments may offer an alternative to observational coding for assessing therapist adherence and skill, particularly in contexts where session recordings are not feasible.

Project Khanya: results from a pilot randomized type 1 hybrid effectiveness-implementation trial of a peer-delivered behavioural intervention for ART adherence and substance use in HIV care in South Africa

INTRODUCTION

South Africa (SA) has the highest number of people living with HIV (PLWH) globally, and a significant burden of alcohol and other drug use (AOD). Although integrating AOD treatment into HIV care may improve antiretroviral therapy (ART) adherence, this is not typically routine practice in SA or other low-resource settings. Identifying interventions that are feasible and acceptable for implementation is critical to improve HIV and AOD outcomes.

METHODS

A pilot randomized hybrid type 1 effectiveness-implementation trial (N = 61) was conducted to evaluate the feasibility and acceptability of Khanya, a task-shared, peer-delivered behavioral intervention to improve ART adherence and reduce AOD in HIV care in SA. Khanya was compared to enhanced treatment as usual (ETAU), a facilitated referral to on-site AOD treatment. Implementation outcomes, defined by Proctor's model, included feasibility, acceptability, appropriateness and fidelity. Primary pilot effectiveness outcomes were ART adherence at post-treatment (three months) measured via real-time electronic adherence monitoring, and AOD measured using biomarker and self-report assessments over six months. Data collection was conducted from August 2018 to April 2020.

RESULTS AND DISCUSSION

Ninety-one percent of participants (n = 56) were retained at six months. The intervention was highly feasible, acceptable, appropriate and delivered with fidelity (>90% of components delivered as intended by the peer). There was a significant treatment-by-time interaction for ART adherence (estimate = -0.287 [95% CI = -0.507, -0.066]), revealing a 6.4 percentage point increase in ART adherence in Khanya, and a 22.3 percentage point decline in ETAU. Both groups evidenced significant reductions in alcohol use measured using phosphatidylethanol (PEth) (F(2,101) = 4.16, p = 0.01), significantly decreased likelihood of self-reported moderate or severe AOD (F(2,104) = 7.02, p = 0.001), and significant declines in alcohol use quantity on the timeline follow-back (F(2,102) = 21.53, p < 0.001). Among individuals using drugs and alcohol, there was a greater reduction in alcohol use quantity in Khanya compared to ETAU over six months (F(2,31) = 3.28, p = 0.05).

CONCLUSIONS

Results of this pilot trial provide initial evidence of the feasibility and acceptability of the Khanya intervention for improving adherence in an underserved group at high risk for ongoing ART non-adherence and HIV transmission. Implementation results suggest that peers may be a potential strategy to extend task-sharing models for behavioral health in resource-limited, global settings.

Implementing Transdiagnostic Cognitive Behavioral Psychotherapy in Adult Public Behavioral Health: A Pilot Evaluation of the Feasibility of the Common Elements Treatment Approach (CETA)

Few evidence-based psychotherapies are provided in adult public behavioral health (PBH), despite the need for such treatments. The common elements treatment approach (CETA) was developed for use by lay providers in low- and middle-income countries and may have relevance in PBH given its unique application with individuals with multiple diagnoses including PTSD, depression, and anxiety. This study utilized data collected as part of the implementation of CETA in 9 PBH agencies in Washington State with 58 providers, including a 2-day workshop and 6 months of consultation. Outcomes included provider-perceived skill in CETA delivery, training and consultation completion rates, and perceived appropriateness of CETA for clients. Thirty-nine (67%) providers completed requirements for training and consultation, and delivered CETA to a total of 56 clients. Perceived competence in delivering CETA improved over time, as well as client symptom scores. CETA shows promise for feasible and effective implementation within US-based PBH systems.

Pilot to policy: statewide dissemination and implementation of evidence-based treatment for traumatized youth

BACKGROUND

A model for statewide dissemination of evidence-based treatment (EBT) for traumatized youth was piloted and taken to scale across North Carolina (NC). This article describes the implementation platform developed, piloted, and evaluated by the NC Child Treatment Program to train agency providers in Trauma-Focused Cognitive Behavioral Therapy using the National Center for Child Traumatic Stress Learning Collaborative (LC) Model on Adoption & Implementation of EBTs. This type of LC incorporates adult learning principles to enhance clinical skills development as part of training and many key implementation science strategies while working with agencies and clinicians to implement and sustain the new practice.

METHODS

Clinicians (n = 124) from northeastern NC were enrolled in one of two TF-CBT LCs that lasted 12 months each. During the LC clinicians were expected to take at least two clients through TF-CBT treatment with fidelity and outcomes monitoring by trainers who offered consultation by phone and during trainings. Participating clinicians initiated treatment with 281 clients. The relationship of clinician and client characteristics to treatment fidelity and outcomes was examined using hierarchical linear regression.

RESULTS

One hundred eleven clinicians completed general training on trauma assessment batteries and TF-CBT. Sixty-five clinicians met all mastery and fidelity requirements to meet roster criteria. One hundred fifty-six (55%) clients had fidelity-monitored assessment and TF-CBT. Child externalizing, internalizing, and post-traumatic stress symptoms, as well as parent distress levels, decreased significantly with treatment fidelity moderating child PTSD outcomes. Since this pilot, 11 additional cohorts of TF-CBT providers have been trained to these roster criteria.

CONCLUSION

Scaling up or outcomes-oriented implementation appears best accomplished when training incorporates: 1) practice-based learning, 2) fidelity coaching, 3) clinical assessment and outcomes-oriented treatment, 4) organizational skill-building to address barriers for agencies, and 5) linking clients to trained clinicians via an online provider roster. Demonstrating clinician performance and client outcomes in this pilot and subsequent cohorts led to legislative support for dissemination of a service array of EBTs by the NC Child Treatment Program.

Best practices in supervising cognitive behavioral therapy with youth

Clinical supervision of cognitive behavioral therapy (CBT) with youth ensures better patient care and fosters trainees' professional development. However, often insufficient attention is directed toward disseminating best practices in supervision of CBT with youth. This Therapeutic Advances contribution aims to communicate the core content of supervision. Additionally, the key supervisory practices associated with CBT with youth are described. Supervisory outcomes are summarized and recommendations for supervisory practices are made.

The Role of Context in the Implementation of Trauma-Focused Treatments: Effectiveness Research and Implementation in Higher and Lower Income Settings

In recent years, the implementation of trauma-focused treatments has expanded across settings that vary widely in the availability of resources, infrastructure, and personnel. The present review aims to inform researchers, policy makers, trainers, and administrators about this diverse range of research. Taking a global health perspective, this review of effectiveness trials and implementation studies compares strategies used in high-income countries to those in low- and medium-income countries. A primary difference between studies in high-income and low- and medium-income countries is the relative emphasis placed on fidelity or adaptation. Adaptations used in low- and medium-income countries might offer useful ideas for increasing the portability, impact, and accessibility of evidence-based interventions in high-income countries.