Coordinated strategic action: Aspiring to wisdom in mental health service systems

How Do Treatment Protocols Affect the Use of Engagement Practices in Youth Mental Health Services?

PURPOSE

Treatment engagement poses challenges for youth mental health providers. With the expansion of evidence-based treatments (EBTs), providers face complex decisions regarding how to engage youth and families using available information sources. This study investigated how EBT protocols are associated with the selection and delivery of engagement practices.

METHOD

Twenty engagement practices were coded in a sample of digital recordings of early treatment sessions (N = 193) from the Child STEPs in California study, a randomized trial testing modular treatment and community-implemented treatment for youth mental health problems. Data were collected on which protocols mental health providers reportedly used to guide their sessions and the protocols in which they had received training. We examined which information sources (i.e., the guiding protocol, other protocols in training history, unspecified source) were associated with observed engagement practices.

RESULTS

In sessions guided by a protocol, most observed engagement practices were accounted for by the guiding protocol (p < .001), rather than protocols in training history or unspecified sources (p < .001). In sessions not guided by a protocol, most observed practices were accounted for by training history (p < .001). Practice frequency and extensiveness was generally greater when a protocol guided the session.

CONCLUSIONS

Inclusion in protocols is associated with the selection and delivery of engagement practices, but this strategy might be insufficient for supporting the use of the full range of engagement practices supported by evidence. Supports are needed that leverage the engagement evidence base to ensure that selected practices empirically fit the engagement needs of youth and families.

How Flexible are Treatments for Youth Internalizing Disorders? Examining Modification Guidelines Included across Supported Treatments

OBJECTIVE

Treatment protocols for youth-internalizing disorders have been developed, however these protocols have yielded mixed findings in routine care settings. Despite increased recognition of the importance of flexibility when delivering evidence-based treatments (EBTs), little is known about the extent to which protocols offer guidance to providers in flexible EBT implementation. The current study examined the extent to which supported EBTs for youth internalizing disorders explicitly incorporate guidance for treatment modification.

METHODS

Supported treatment protocols for youth internalizing disorders were identified (N = 44), from which 4,021 modification guidelines were extracted and coded using a structured coding system to classify modification strategies (i.e., the forms that recommended modifications take), and associated tailoring factors (i.e., the rationale for which modifications are recommended).

RESULTS

Across all EBTs, modification guidelines were quite common, with the average protocol including almost 91 text passages providing guidance for modification. The majority of modification guidelines functionally increase session or treatment length by recommending the addition or repetition of material, whereas less than 5% of modification guidelines provided strategies for condensing or streamlining care. Strikingly, less than 2% of modification guidelines in EBT protocols address patient cultural factors, and rarely address provider or setting issues that can challenge standard implementation.

CONCLUSIONS

Findings highlight critical gaps in the available guidance to modify EBTs for youth internalizing disorders, and suggest EBT protocols may not be optimally poised to flexibly address the broad diversity of children and adolescents across varied settings in need of mental health care.

A pilot study to evaluate feasibility and acceptability of training mental health workers in India to select case-specific intervention procedures within a dynamic modular treatment designed for a low-resource setting

RATIONALE, AIMS AND OBJECTIVES

A key consideration in designing scalable solutions for improving global mental health involves balancing the need for interventions to be uncomplicated for mental health workers (MHWs) and the need for the intervention to be widely applicable to many clients. Often these needs are in competition, since interventions are routinely simplified by removing procedures or reducing their dynamic responsivity, which in turn lowers their overall utility in serving large, clinically diverse populations. The principal aim of this pilot study involved evaluating the feasibility and acceptability of a brief strategy designed to delegate problem classification and practice selection to MHWs operating within a flexible, modular, cognitive behavioural protocol. A secondary aim involved gathering data on which to base a hypothesis regarding the potential effectiveness of this strategy.

METHOD

Within an open trial, an educationally diverse sample of local MHWs in India (N = 18) reviewed fictional case vignettes, classified mental health problems, and then selected practices before and after a two-hour training that included a one-page decision-making resource. Feasibility was measured by assessing the integrity of the study protocol and training, the measurement and administration of questionnaires as well as study recruitment and completion. Acceptability of the intervention was measured by MHW-perceived performance, ease of use, value, importance, and intention for continued use. Decision-making accuracy was assessed by comparing MHWs' clinical decisions with criteria established through consensus among psychologists with expertise in modular protocols.

RESULTS

Results suggested high feasibility and acceptability on all metrics. Secondary analysis revealed that MHW's decision-making accuracy and confidence also significantly improved, providing a basis for the hypothesis that this brief approach is useful for building MHW capacity in low-resource settings.

CONCLUSION

Overall these findings provide initial support for these methods and potential training outcomes to test within a larger, randomized controlled trial.

A qualitative analysis of collaborative efforts to build a school-based intervention for multiple common adolescent mental health difficulties in India

INTRODUCTION

In low- and middle-income countries (LMICs), the gap between need for mental health (MH) treatment and access to services is stark, particularly among children and adolescents. In service of addressing this treatment gap, the current study provides an in-depth illustration of later-stage collaborative design of a school-based, transdiagnostic MH intervention in New Delhi and Goa, India, using a combination of contextual insights from local stakeholders and knowledge derived from the global evidence base.

METHODS

Using an inductive-deductive approach to qualitative thematic analysis, we examined coded data from qualitative sources related to experiences of developing and implementing an intervention prototype. These sources included notes from meetings attended by treatment development team members and providers, written feedback on protocol materials (e.g., provider manual, student handouts), field notes reflecting researcher observations, and minutes from weekly clinical supervision meetings.

RESULTS

Results revealed that codes involving cultural/contextual considerations, protocol material and content, and intervention complexity arose consistently throughout treatment development and across document types, illustrating their central role in finalizing protocol design.

DISCUSSION

These findings have implications for the future of mental health treatment development and implementation globally.

Assessing practical implementation of modular psychotherapy for youth in community-based settings using benchmarking

Background

Achieving high quality outcomes in a community context requires the strategic coordination of many activities in a service system, involving families, clinicians, supervisors, and administrators. In modern implementation trials, the therapy itself is guided by a treatment manual; however, structured supports for other parts of the service system may remain less well-articulated (e.g., supervision, administrative policies for planning and review, information/feedback flow, resource availability). This implementation trial investigated how a psychosocial intervention performed when those non-therapy supports were not structured by a research team, but were instead provided as part of a scalable industrial implementation, testing whether outcomes achieved would meet benchmarks from published research trials.

Method

In this single-arm observational benchmarking study, a total of 59 community clinicians were trained in the Modular Approach to Therapy for Children (MATCH) treatment program. These clinicians delivered MATCH treatment to 166 youth ages 6 to 17 naturally presenting for psychotherapy services. Clinicians received substantially fewer supports from the treatment developers or research team than in the original MATCH trials and instead relied on explicit process management tools to facilitate implementation. Prior RCTs of MATCH were used to benchmark the results of the current initiative. Client improvement was assessed using the Top Problems Assessment and Brief Problem Monitor.

Results

Analysis of client symptom change indicated that youth experienced improvement equal to or better than the experimental condition in published research trials. Similarly, caregiver-reported outcomes were generally comparable to those in published trials.

Conclusions

Although results must be interpreted cautiously, they support the feasibility of using process management tools to facilitate the successful implementation of MATCH outside the context of a formal research or funded implementation trial. Further, these results illustrate the value of benchmarking as a method to evaluation industrial implementation efforts. Plain Language Summary: Randomized effectiveness trials are inclusive of clinicians and cases that are routinely encountered in community-based settings, while continuing to rely on the research team for both clinical and administrative guidance. As a result, the field still struggles to understand what might be needed to support sustainable implementation and how interventions will perform when brought to scale in community settings without those clinical trial supports. Alternative approaches are needed to delineate and provide the clinical and operational support needed for implementation and to efficiently evaluate how evidence-based treatments perform. Benchmarking findings in the community against findings of more rigorous clinical trials is one such approach. This paper offers two main contributions to the literature. First, it provides an example of how benchmarking is used to evaluate how the Modular Approach to Therapy for Children (MATCH) treatment program performed outside the context of a research trial. Second, this study demonstrates that MATCH produced comparable symptom improvements to those seen in the original research trials and describes the implementation strategies associated with this success. In particular, although clinicians in this study had less rigorous expert clinical supervision as compared with the original trials, clinicians were provided with process management tools to support implementation. This study highlights the importance of evaluating the performance of intervention programs when brought to scale in community-based settings. This study also provides support for the use of process management tools to assist providers in effective implementation.

Evidence Based Practice in Systems of Care for Children with Complex Mental Health Needs

Purpose: Community-based social work with families and youth with complex behavioral health needs highlights challenges to incorporating empirical evidence into routine practice. This article presents a framework for integrating evidence in community-based Systems of Care for these children and their families.Method: This article reviews research on various approaches to integrating evidence into children's behavioral health and community-based care and contextualizes it within dominant paradigms of Systems of Care (SoC) and Wraparound principles.Results: Based on this review, this article proposes the Evidence-Based Practice in Systems of Care (EBP in SoC) model. The model describes how to incorporate evidence into every aspect of community-based SoCs for children with mental health concerns.Discussion and Conclusion: Discussion of the model will focus on implications of using the framework for practitioners, mental health organizations, communities, and state and federal administration and policymaking.

Creating a prepared mental health workforce: comparative illustrations of implementation strategies

BACKGROUND

Psychotherapy implementation must contend with the task of preparing a mental health workforce to provide the highest quality services to as much of a service population as possible, in high-income as well as low-to-middle income countries.

OBJECTIVE

We outline general challenges and solutions and investigate how well various implementation strategies would fit a clinical population.

METHODS

Using a data set from a prior cluster randomised trial with a clinically diverse population and 33 intervention practices, we presented multiple illustrations comparing the ability of different implementation strategies to serve youth and families with procedures in which service providers were trained.

FINDINGS

A series of survival functions demonstrated that many common implementation strategies are unlikely to create a prepared workforce, given the large and diverse number of practices needed to be mastered by providers.

CLINICAL IMPLICATIONS

'Benchmark' solutions that afforded superior coverage of the service population could be supported through paced learning approaches (ie, training interventions a little at a time) using extensible, modular intervention designs.

Prospective predictors of sustainment of multiple EBPs in a system-driven implementation context: Examining sustained delivery based on administrative claims

Background

System-driven scale-up of multiple evidence-based practices (EBPs) is an increasingly common method used in public mental health to improve care. However, there are little data on the long-term sustained delivery of EBPs within these efforts, and previous studies have relied on retrospective self-report within cross-sectional studies. This study identified prospective predictors of sustained EBP delivery at the EBP-, therapist-, and organizational-levels using survey and administrative claims data within a large-scale system-driven implementation effort.

Methods

777 therapists and 162 program leaders delivering at least one of six EBPs of interest completed surveys assessing perceptions of EBPs and organizational context. These surveys were linked to administrative data to examine prospective predictors of therapists' EBP delivery over 33 months.

Results

Five of the six EBPs implemented showed sustained delivery in the system, with volume varying by EBP. Although total EBP claim volume per therapist decreased over time, the volume ratio (ratio of EBP-specific claims to total EBP and non-EBP claims) stayed relatively stable. Multilevel models revealed that EBPs that required consultation, had unstructured content, higher therapist self-efficacy with the EBP, and more positive program leader perceptions of the EBP were associated with greater sustained volume and volume ratio of the EBP. Therapists who were trained in fewer EBPs, who were unlicensed, and who worked in agencies rated by program leaders as lower on organizational staff autonomy and stress showed greater sustained EBP volume and volume ratio. Finally, more direct service hours per week provided by therapist predicted greater sustained EBP volume, but lower volume ratio.

Conclusions

The results point to the importance of EBP, therapist, and organizational factors that may be targeted in implementation strategies to promote the sustainment of EBPs.

Service use by youth with autism within a system-driven implementation of evidence-based practices in children's mental health services

Public mental health systems play an important role in caring for youth with autism spectrum disorder. Like other dually diagnosed populations, youth with autism spectrum disorder may receive services in the context of evidence-based practice implementation efforts within public mental health systems. Little is known about service use patterns within the context of system-driven implementations efforts for this population. This case-control study examined mental health service patterns of 2537 youth with autism spectrum disorder compared to 2537 matched peers receiving care in the Los Angeles County Department of Mental Health, the largest public mental health department in the United States, within the context of a system-driven implementation of multiple evidence-based practices. Although not the primary target of this implementation effort, youth with autism spectrum disorder were served when they met criteria for the services based on their presenting mental health symptoms. Comparative analyses using administrative claims data were conducted to examine differences in mental health utilization patterns and clinical characteristics. Findings revealed significant differences in the volume and duration of mental health services as well as differences in the service type and evidence-based practice delivered between youth with and without autism spectrum disorder. Results provide direction targeting implementation efforts for youth with autism spectrum disorder within a public mental health system care reform.

Therapist-Observer Concordance in Ratings of EBP Strategy Delivery: Challenges and Targeted Directions in Pursuing Pragmatic Measurement in Children's Mental Health Services

Pragmatic measures of therapist delivery of evidence-based practice (EBP) are critical to assessing the impact of large-scale, multiple EBP implementation efforts. As an initial step in the development of pragmatic measurement, the current study examined the concordance between therapist and observer ratings of items assessing delivery of EBP strategies considered essential for common child EBP targets. Possible EBP-, session-, and therapist-levels factors associated with concordance were also explored. Therapists and independent observers rated the extensiveness of therapist (n = 103) EBP strategy delivery in 680 community psychotherapy sessions in which six EBPs were used. Concordance between therapist- and observer-report of the extensiveness of therapist EBP strategy use was at least fair (ICC ≥ .40) for approximately half of the items. Greater therapist-observer concordance was observed in sessions where a structured EBP was delivered and in sessions where therapists reported being able to carry out planned activities. Findings highlighted conditions that may improve or hinder therapists' ability to report on their own EBP strategy delivery in a way that is consistent with independent observers. These results can help inform the development of pragmatic therapist-report measures of EBP strategy delivery and implementation efforts more broadly.

A systematic review of prevention programs targeting depression, anxiety, and stress in university students

BACKGROUND

Given the prevalence of anxiety, depression, and stress among university students, it is important to assess the effectiveness of prevention programs for these problems. Beyond examining effect sizes, applying a common elements approach can enhance our understanding of which practice elements are most frequently included in symptom-reducing programs.

METHOD

This review examined effective (i.e., outcome-producing) prevention programs targeting depression, anxiety, and/or stress in university students. Programs could be delivered in a group-based, online/computer-delivered, or self-administered format and at the universal, selective, or indicated prevention level.

RESULTS

The resulting sample of 62 articles covered 68 prevention programs for college, graduate, or professional students across 15 countries. Average effect sizes for programs were moderate (overall g = 0.65), regardless of delivery format or prevention level. The most common practice elements (overall and for programs producing large effects) were: psychoeducation (72%), relaxation (69%), and cognitive monitoring/restructuring (47%). Many programs were limited by: (a) symptom target-outcome mismatches, (b) disproportionately female samples, and (c) inconsistently reported adherence data.

LIMITATIONS

Commonness of practice elements across outcome-producing interventions does not imply their extensiveness nor unique contribution to effectiveness. Coding was based on information in articles rather than manuals, and inter-rater reliability was moderate for some practice elements.

CONCLUSION

The outcome-producing prevention programs in our sample had common practice elements and produced moderate reduction in symptoms overall. Future research of depression, anxiety, and stress prevention programs for university students can investigate practice elements' unique and combined impact on outcomes, further explore under-tested practice elements, and use findings to inform intervention design.

Commentary: Metaknowledge is power: envisioning models to address unmet mental health needs: reflections on Kazdin (2019)

Despite having more knowledge and resources at our disposal than ever before in history, the human suffering due to mental health concerns is extensive and mostly untreated. Kazdin (Journal of Child Psychology and Psychiatry, 2019) calls for a significant departure from prevailing conventions of psychotherapy delivery, noting that the traditional formats, settings, and workforce limit the potential of science to help those in need. This paper supports and extends Kazdin's argument, urging an additional shift in perspective about treatments altogether. A reconceptualization of treatment protocols as knowledge resources is offered, such that the strategy to reduce human suffering becomes one of flexibly moving knowledge of how to address mental health concerns to any and all contexts in which those problems can be addressed, whether or not a treatment is used. A dedicated pursuit of metaknowledge-knowing how to structure, coordinate, and apply knowledge-promises to lift the world's collective therapeutic intelligence and thereby move us closer to the goal of substantively addressing the global mental health burden.