Integrity in mental health systems: An expanded framework for managing uncertainty in clinical care

Provider Adherence to Modular Cognitive Behavioral Therapy for Children and Adolescents

This study explored predictors of community-based providers' adherence to MATCH, a modular cognitive behavioral therapy for children and adolescents. Provider-reported adherence to MATCH was measured using three increasingly strict criteria: (1) session content (whether the session covered MATCH content consistent with the client's target problem), (2) session content and sequencing (whether the session covered MATCH content in the expected sequence for the client's target problem), and (3) session content, sequencing, and participant (whether the session covered MATCH content in the expected sequence and with the expected participant(s) for the client's target problem). Session, client, provider, and organizational predictors of adherence to MATCH were assessed using multilevel modeling. Results revealed that nearly all providers delivered MATCH content that corresponded to the target problem, but only one-third of providers delivered MATCH content in the expected sequence and with the expected participant for the client's target problem. This difference underscores the need for nuanced adherence measurement to capture important implementation information that broad operationalizations of adherence miss. Regardless of the criteria used providers were most adherent to MATCH during sessions when clients presented with interfering comorbid mental health symptoms. This suggests that the design of MATCH, which offers flexibility and structured guidance to address comorbid mental health problems, may allow providers to personalize treatment to address interfering comorbidity symptoms while remaining adherent to evidence-based practices. Additional guidance for providers on managing other types of session interference (e.g., unexpected events) may improve treatment integrity in community settings.

The role of implementation organizations in scaling evidence-based psychosocial interventions

BACKGROUND

To bring evidence-based interventions (EBIs) to individuals with behavioral health needs, psychosocial interventions must be delivered at scale. Despite an increasing effort to implement effective treatments in communities, most individuals with mental health and behavioral problems do not receive EBIs. We posit that organizations that commercialize EBIs play an important role in disseminating EBIs, particularly in the USA. The behavioral health and implementation industry is growing, bringing the implementation field to an important inflection point: how to scale interventions to improve access while maintaining EBI effectiveness and minimizing inequities in access to psychosocial intervention.

MAIN BODY

We offer a first-hand examination of five illustrative organizations specializing in EBI implementation: Beck Institute for Cognitive Behavioral Therapy; Incredible Years, Inc.; the PAXIS Institute; PracticeWise, LLC; and Triple P International. We use the Five Stages of Small Business Growth framework to organize themes. We discuss practical structures (e.g., corporate structures, intellectual property agreements, and business models) and considerations that arise when trying to scale EBIs including balancing fidelity and reach of the intervention. Business models consider who will pay for EBI implementation and allow organizations to scale EBIs.

CONCLUSION

We propose research questions to guide scaling: understanding the level of fidelity needed to maintain efficacy, optimizing training outcomes, and researching business models to enable organizations to scale EBIs.

Therapist Self-Report of Fidelity to Core Elements of Family Therapy for Adolescent Behavior Problems: Psychometrics of a Pragmatic Quality Indicator Tool

Therapist-report measures of evidence-based interventions have enormous potential utility as quality indicators in routine care; yet, few such tools have shown strong psychometric properties. This study describes reliability and validity characteristics of a therapist-report measure of family therapy techniques for treating adolescent conduct and substance use problems: Inventory of Therapy Techniques for Core Elements of Family Therapy (ITT-CEFT). Study participants included 31 staff therapists treating 68 adolescent clients in eight community-based mental health and substance use clinics. Therapists submitted ITT-CEFT checklists and companion audio recordings for 189 sessions. The ITT-CEFT contains 13 techniques identified as core elements of three manualized family therapy models that are empirically supported for the target group. Therapists also reported on their use of three motivational interventions, and independent observers coded the submitted recordings. ITT-CEFT factor validity was shown via confirmatory factor analyses of the tool's theoretical structure. Derived modules were: Family Engagement (four items; Cronbach's α = .72); Relational Orientation (five items; α = .74); and Interactional Change (four items; α = .66). Concurrent validity analyses showed fair-to-excellent therapist reliability compared to observer ratings (ICCs range .64-.75); they showed moderate therapist accuracy compared to observer mean scores, reflecting a tendency to overestimate delivery of the techniques. Discriminant validity analyses showed tool differentiation from motivational interventions. Results offer provisional evidence for the feasibility of using the therapist-report ITT-CEFT to anchor quality procedures for family therapy interventions in real-world settings.Trial Registration: The parent clinical trial is registered at www.ClinicalTrials.gov , ID: NCT03342872 (registration date: 11.10.17).

A qualitative analysis of collaborative efforts to build a school-based intervention for multiple common adolescent mental health difficulties in India

INTRODUCTION

In low- and middle-income countries (LMICs), the gap between need for mental health (MH) treatment and access to services is stark, particularly among children and adolescents. In service of addressing this treatment gap, the current study provides an in-depth illustration of later-stage collaborative design of a school-based, transdiagnostic MH intervention in New Delhi and Goa, India, using a combination of contextual insights from local stakeholders and knowledge derived from the global evidence base.

METHODS

Using an inductive-deductive approach to qualitative thematic analysis, we examined coded data from qualitative sources related to experiences of developing and implementing an intervention prototype. These sources included notes from meetings attended by treatment development team members and providers, written feedback on protocol materials (e.g., provider manual, student handouts), field notes reflecting researcher observations, and minutes from weekly clinical supervision meetings.

RESULTS

Results revealed that codes involving cultural/contextual considerations, protocol material and content, and intervention complexity arose consistently throughout treatment development and across document types, illustrating their central role in finalizing protocol design.

DISCUSSION

These findings have implications for the future of mental health treatment development and implementation globally.

Implementing Evidence Based Practices for Children's Mental Health: A Case Study in Implementing Modular Treatments in Community Mental Health

There is strong enthusiasm for utilizing implementation science in the implementation of evidence-based programs in children's community mental health, but there remains work to be done to improve the process. Despite the proliferation of implementation frameworks, there is limited literature providing case examples of overcoming implementation barriers. This article examines whether the use of three implementations strategies, a structured training and coaching program, the use of professional development portfolios for coaching, and a progress monitoring data system, help to overcome barriers to implementation by facilitating four implementation drivers at a community mental health agency. Results suggest that implementation is a process of recognizing and adapting to both predictable and unpredictable barriers. Furthermore, the use of these implementation strategies is important in improving implementation outcomes.

Choices in Treatment Integrity:Considering the Protocol and Consultant Recommendations in Child and Adolescent Therapy

Codifying a complex treatment intervention into a manual necessarily involves simplifying procedures, possibly resulting in developer and expert adaptations that align with but are not fully consistent with the manualized content. This study evaluated the integrity of an evidence-based treatment to two information sources (i.e., an expert consultant and the treatment manual) within youth community mental health settings. Consultant recommendations were compared with manual-prescribed therapy procedures, which identified instances of agreement between the sources as well as consultant adaptations. To determine clinicians' integrity to each source, clinician-reported therapy procedures were compared with consultant recommendations and manual prescriptions. Participants consisted of 30 clinicians, five consultants, and 78 youths (53% male; M age = 9.73 years, SD = 2.84; 81% Latino). Findings indicated that consultants generally recommended content from the manual but also made frequent adaptations that involved changing the sequence or problem focus of practices and adding additional content. Clinicians followed both information sources in a majority of their sessions when these sources agreed but were more likely to follow only one or neither source when they disagreed. Findings suggest that consultants often make adaptations to the manual and that clinicians are more likely to employ recommendations from the manual and/or consultant when the information sources are in agreement. Although it is as yet unclear whether consultant recommendations are effective when they differ from the prescriptions of the manual, results from this study indicate that information sources that supplement treatment manuals may benefit clinical decision making.

Relationships between clinician-level attributes and fidelity-consistent and fidelity-inconsistent modifications to an evidence-based psychotherapy

BACKGROUND

Clinicians often modify evidence-based psychotherapies (EBPs) when delivering them in routine care settings. There has been little study of factors associated with or implications of modifications to EBP protocols. This paper differentiates between fidelity-consistent and fidelity-inconsistent modifications and it examines the potential influence of two clinician characteristics, training outcomes, and attitudes toward EBPs on fidelity-consistent and fidelity-inconsistent modifications of cognitive behavioral therapy in a sample of clinicians who had been trained to deliver these treatments for children or adults.

METHODS

Survey and coded interview data collected 2 years after completion of training programs in cognitive behavioral therapy were used to examine associations between successful or unsuccessful completion of training, clinician attitudes, and modifications. Modifications endorsed by clinicians were categorized as fidelity-consistent or fidelity-inconsistent and entered as outcomes into separate regression models, with training success and attitudes entered as independent variables.

RESULTS

Successful completion of a training program was associated with subsequent fidelity-inconsistent modifications but not fidelity-consistent modifications. Therapists who reported greater openness to using EBPs prior to training reported more fidelity-consistent modifications at follow-up, and those who reported greater willingness to adopt EBPs if they found them appealing were more likely to make fidelity-inconsistent modifications.

CONCLUSIONS

Implications of these findings for training, implementation, EBP sustainment, and future studies are discussed. Research on contextual and protocol-related factors that may impact decisions to modify EBPs will be an important future direction of study to complement to this research.

An analysis of stakeholder views on children's mental health services

The purpose was to examine the perspectives of mental health stakeholders as a means to guide the adaptation of evidence-based treatments. The Mental Health System Ecological (MHSE) model was used to organize therapist, administrator, and parent perspectives gathered using qualitative methods. The MHSE model posits the influences of client-level, provider-level, intervention-specific, service delivery, organizational, and service system characteristics on implementation. Focus groups and interviews were conducted with community mental health stakeholders and included parents, therapists, and administrators (N = 21). Participants included 11 primarily Caucasian (90.48%) female participants, ranging in ages between 31 and 57 years. Data were analyzed according to the MHSE model. Frequency counts were tabulated for each theme and stakeholder group differences were determined using the Mann-Whitney test. Stakeholder groups mentioned needs at each level of the MHSE model. Stakeholder group differences most notably emerged with child and family themes, which included complexity of mental health issues, parenting differences, and family stressors. Stakeholders identified challenges for optimal mental health services for children across multiple levels of an ecological model. Implications of the findings are discussed, including the continued relevance of adapting mental health interventions by increasing their flexible application across multiple target problems and the promise of partnership approaches.

Advances in Applying Treatment Integrity Research for Dissemination and Implementation Science: Introduction to Special Issue

This special series focuses upon the ways in which research on treatment integrity, a multidimensional construct including assessment of the content and quality of a psychosocial treatment delivered to a client as well as relational elements, can inform dissemination and implementation science. The five articles for this special series illustrate how treatment integrity concepts and methods can be applied across different levels of the mental health service system to advance dissemination and implementation science. In this introductory article, we provide an overview of treatment integrity research and describe three broad conceptual models that are relevant to the articles in the series. We conclude with a brief description of each of the five articles in the series.