Economic Impact of the Statewide Implementation of an Evidence-Based Treatment: Multisystemic Therapy in New Mexico

Effect of Providing Evidence-Based Mental Health Treatment on Retention in Care Among Medicaid-Enrolled Youths

OBJECTIVE

Youths who start behavioral health treatment often stop before completing a therapeutic course of care. To increase treatment engagement and quality of care, the Evidence-Based Practice and Innovation Center in Philadelphia has incentivized use of evidence-based practices (EBPs) for mental health care of youths. The authors aimed to compare treatment outcomes between youths who received EBP care and those who did not.

METHODS

Using EBP-specific billing codes and propensity score matching, the authors compared treatment retention among youths who received trauma-focused cognitive-behavioral therapy (TF-CBT; N=413) or parent-child interaction therapy (PCIT; N=90) relative to matched samples of youths in standard outpatient therapy (N=503).

RESULTS

Youths with a minimum of one session of TF-CBT or PCIT attended a second session at higher rates than did youths in the matched control group (TF-CBT: 96% vs. 68%, p<0.01; PCIT: 94% vs. 69%, respectively, p<0.01). On average, these returning youths attended more sessions in the EBP groups than in the control group (TF-CBT: 15.9 vs. 11.5 sessions, p<0.01; PCIT: 11.2 vs. 6.9 sessions, p<0.01).

CONCLUSIONS

These findings indicate that, in addition to improving quality of care, EBP implementation helps address the major challenge that most youths who engage with treatment are not retained long enough for care to have therapeutic effects. Future research should examine the mechanisms through which EBPs can improve treatment retention.

Implementation Feasibility and Hidden Costs of Statewide Scaling of Evidence-Based Therapies for Children and Adolescents

OBJECTIVE

State mental health systems are retraining their workforces to deliver services supported by research. Knowledge about evidence-based therapies (EBTs) for child and adolescent disorders is robust, but the feasibility of their statewide scaling has not been examined. The authors reviewed implementation feasibility for 12 commonly used EBTs, defining feasibility for statewide scaling as an EBT having at least one study documenting acceptability, facilitators and barriers, or fidelity; at least one study with a racially and ethnically diverse sample; an entity for training, certification, or licensing; and fiscal data reflecting the costs of implementation.

METHODS

The authors reviewed materials for 12 EBTs being scaled in New York State and conducted a literature review with search terms relevant to their implementation. Costs and certification information were supplemented by discussions with treatment developers and implementers.

RESULTS

All 12 EBTs had been examined for implementation feasibility, but only three had been examined for statewide scaling. Eleven had been studied in populations reflecting racial-ethnic diversity, but few had sufficient power for subgroup analyses to demonstrate effectiveness with these samples. All had certifying or licensing entities. The per-clinician costs of implementation ranged from $500 to $3,500, with overall ongoing costs ranging from $100 to $6,000. A fiscal analysis of three EBTs revealed hidden costs ranging from $5,000 to $24,000 per clinician, potentially limiting sustainability.

CONCLUSIONS

The evidence necessary for embedding EBTs in state systems has notable gaps that may hinder sustainability. Research-funding agencies should prioritize studies that focus on the practical aspects of scaling to assist states as they retrain their workforces.

Revisiting the policy ecology framework for implementation of evidence-based practices in mental health settings

BACKGROUND

Over the past three decades, policy actors and actions have been highly influential in supporting the implementation of evidence-based practices (EBPs) in mental health settings. An early examination of these actions resulted in the Policy Ecology Framework (PEF), which was originally developed as a tactical primer for state and local mental health regulators in the field of child mental health. However, the policy landscape for implementation has evolved significantly since the original PEF was published. An interrogation of the strategies originally proposed in the PEF is necessary to provide an updated menu of strategies to improve our understanding of the mechanisms of policy action and promote system improvement.

OBJECTIVES

This paper builds upon the original PEF to address changes in the policy landscape for the implementation of mental health EBPs between 2009 and 2022. We review the current state of policy strategies that support the implementation of EBPs in mental health care and outline key areas for policy-oriented implementation research. Our review identifies policy strategies at federal, state, agency, and organizational levels, and highlights developments in the social context in which EBPs are implemented. Furthermore, our review is organized around some key changes that occurred across each PEF domain that span organizational, agency, political, and social contexts along with subdomains within each area.

DISCUSSION

We present an updated menu of policy strategies to support the implementation of EBPs in mental health settings. This updated menu of strategies considers the broad range of conceptual developments and changes in the policy landscape. These developments have occurred across the organizational, agency, political, and social contexts and are important for policymakers to consider in the context of supporting the implementation of EBPs. The updated PEF expands and enhances the specification of policy levers currently available, and identifies policy targets that are underdeveloped (e.g., de-implementation and sustainment) but are becoming visible opportunities for policy to support system improvement. The updated PEF clarifies current policy efforts within the field of implementation science in health to conceptualize and better operationalize the role of policy in the implementation of EBPs.

Juvenile confinement exacerbates adversity burden: A neurobiological impetus for decarceration

Every year, about 700,000 youth arrests occur in the United States, creating significant neurodevelopmental strain; this is especially concerning as most of these youth have early life adversity exposures that may alter brain development. Males, Black, and Latinx youth, and individuals from low socioeconomic status households have disproportionate contact with the juvenile justice system (JJS). Youth confined in the JJS are frequently exposed to threat and abuse, in addition to separation from family and other social supports. Youths’ educational and exploratory behaviors and activities are substantially restricted, and youth are confined to sterile environments that often lack sufficient enrichment resources. In addition to their demonstrated ineffectiveness in preventing future delinquent behaviors, high recidivism rates, and costs, juvenile conditions of confinement likely exacerbate youths’ adversity burden and neurodevelopmentally harm youth during the temporally sensitive window of adolescence. Developmentally appropriate methods that capitalize on adolescents’ unique rehabilitative potential should be instated through interventions that minimize confinement. Such changes would require joint advocacy from the pediatric and behavioral health care communities. “The distinct nature of children, their initial dependent, and developmental state, their unique human potential as well as their vulnerability, all demand the need for more, rather than less, legal and other protection from all forms of violence (United Nations Committee on the Rights of the Child, 2007).”

Coordination of sustainable financing for evidence-based youth mental health treatments: protocol for development and evaluation of the fiscal mapping process

BACKGROUND

Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health and economic impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies (hereafter, "service agencies"). Strategic planning tools are needed that can guide these service agencies in their coordination of sustainable funding for EBTs. This protocol describes a mixed-methods research project designed to (1) develop and (2) evaluate our novel fiscal mapping process that guides strategic planning efforts to finance the sustainment of EBTs in youth mental health services.

METHOD

Participants will be 48 expert stakeholder participants, including representatives from ten service agencies and their partners from funding agencies (various public and private sources) and intermediary organizations (which provide guidance and support on the delivery of specific EBTs). Aim 1 is to develop the fiscal mapping process: a multi-step, structured tool that guides service agencies in selecting the optimal combination of strategies for financing their EBT sustainment efforts. We will adapt the fiscal mapping process from an established intervention mapping process and will incorporate an existing compilation of 23 financing strategies. We will then engage participants in a modified Delphi exercise to achieve consensus on the fiscal mapping process steps and gather information that can inform the selection of strategies. Aim 2 is to evaluate preliminary impacts of the fiscal mapping process on service agencies' EBT sustainment capacities (i.e., structures and processes that support sustainment) and outcomes (e.g., intentions to sustain). The ten agencies will pilot test the fiscal mapping process. We will evaluate how the fiscal mapping process impacts EBT sustainment capacities and outcomes using a comparative case study approach, incorporating data from focus groups and document review. After pilot testing, the stakeholder participants will conceptualize the process and outcomes of fiscal mapping in a participatory modeling exercise to help inform future use and evaluation of the tool.

DISCUSSION

This project will generate the fiscal mapping process, which will facilitate the coordination of an array of financing strategies to sustain EBTs in community youth mental health services. This tool will promote the sustainment of youth-focused EBTs.

The Design and Evaluation of Clinical Pathway for Disease Management to Maximize Public Health Benefit

As an implementation strategy, the actual functional scope of clinical pathway (CPW) can be mastered is the basis for evaluating and improving CPW management, which is very important for bridging the evidence-to-practice gap. Here, we propose and discuss a design and evaluation model for CPWs through some theoretical lenses that may help the designer master the actual functional scope of CPWs and accumulate and refine effective key management steps of a specific disease. We can thus determine the best-optimized design with the largest net benefit that allows definition of the core management steps for a specific disease.

Market viability: a neglected concept in implementation science

This debate paper asserts that implementation science needs to incorporate a key concept from entrepreneurship—market demand—and demonstrates how assessing an innovation’s potential market viability might advance the pace and success of innovation adoption and sustainment. We describe key concepts, language distinctions, and questions that entrepreneurs pose to implementation scientists—many of which implementation scientists appear ill-equipped to answer. The paper concludes with recommendations about how concepts from entrepreneurship, notably market viability assessment, can enhance the translation of research discoveries into real-world adoption, sustained use, and population health benefits. The paper further proposes activities that can advance implementation science’s capacity to draw from the field of entrepreneurship, along with the data foundations required to assess and cultivate market demand.

Translating economic evaluations into financing strategies for implementing evidence-based practices

BACKGROUND

Implementation researchers are increasingly using economic evaluation to explore the benefits produced by implementing evidence-based practices (EBPs) in healthcare settings. However, the findings of typical economic evaluations (e.g., based on clinical trials) are rarely sufficient to inform decisions about how health service organizations and policymakers should finance investments in EBPs. This paper describes how economic evaluations can be translated into policy and practice through complementary research on financing strategies that support EBP implementation and sustainment.

MAIN BODY

We provide an overview of EBP implementation financing, which outlines key financing and health service delivery system stakeholders and their points of decision-making. We then illustrate how economic evaluations have informed decisions about EBP implementation and sustainment with three case examples: (1) use of Pay-for-Success financing to implement multisystemic therapy in underserved areas of Colorado, USA, based in part on the strength of evidence from economic evaluations; (2) an alternative payment model to sustain evidence-based oncology care, developed by the US Centers for Medicare and Medicaid Services through simulations of economic impact; and (3) use of a recently developed fiscal mapping process to collaboratively match financing strategies and needs during a pragmatic clinical trial for a newly adapted family support intervention for opioid use disorder.

CONCLUSIONS

EBP financing strategies can help overcome cost-related barriers to implementing and sustaining EBPs by translating economic evaluation results into policy and practice. We present a research agenda to advance understanding of financing strategies in five key areas raised by our case examples: (1) maximize the relevance of economic evaluations for real-world EBP implementation; (2) study ongoing changes in financing systems as part of economic evaluations; (3) identify the conditions under which a given financing strategy is most beneficial; (4) explore the use and impacts of financing strategies across pre-implementation, active implementation, and sustainment phases; and (5) advance research efforts through strong partnerships with stakeholder groups while attending to issues of power imbalance and transparency. Attention to these research areas will develop a robust body of scholarship around EBP financing strategies and, ultimately, enable greater public health impacts of EBPs.

Cost analysis in implementation studies of evidence-based practices for mental health and substance use disorders: a systematic review

Background

This study is a systematic literature review of cost analyses conducted within implementation studies on behavioral health services. Cost analysis of implementing evidence-based practices (EBP) has become important within implementation science and is critical for bridging the research to practice gap to improve access to quality healthcare services. Costing studies in this area are rare but necessary since cost can be a barrier to implementation and sustainment of EBP.

Methods

We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology and applied the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Key search terms included: (1) economics, (2) implementation, (3) EBP, and (4) behavioral health. Terms were searched within article title and abstracts in: EconLit, SocINDEX, Medline, and PsychINFO. A total of 464 abstracts were screened independently by two authors and reduced to 37 articles using inclusion and exclusion criteria. After a full-text review, 18 articles were included.

Results

Findings were used to classify costs into direct implementation, direct services, and indirect implementation. While all studies included phases of implementation as part of their design, only five studies examined resources across multiple phases of an implementation framework. Most studies reported direct service costs associated with adopting a new practice, usually summarized as total EBP cost, cost per client, cost per clinician, and/or cost per agency. For studies with detailed analysis, there were eleven direct cost categories represented. For five studies that reported costs per child served, direct implementation costs varied from $886 to $9470 per child, while indirect implementation costs ranged from $897 to $3805 per child.

Conclusions

This is the first systematic literature review to examine costs of implementing EBP in behavioral healthcare settings. Since 2000, 18 studies were identified that included a cost analysis. Given a wide variation in the study designs and economic methods, comparison across studies was challenging, which is a major limitation in the field, as it becomes difficult to replicate studies or to estimate future costs to inform policy decisions related to budgeting. We recommend future economic implementation studies to consider standard economic costing methods capturing costs across implementation framework phases to support comparisons and replicability.

Important Factors for Evidence-Based Implementation in Child Welfare Settings: A Systematic Review

PURPOSE

This systematic review synthesizes the common barriers and facilitators to implementing research-supported interventions with child welfare agencies using implementation frameworks.

METHODS

For this systematic review, database searches, and quality assessments of relevant literature produced 24 studies that examined the implementation of RSIs in child welfare using an implementation framework.

RESULTS

The studies were coded for themes and the thematic analysis found several common factors to RSI implementation across child welfare agencies including funding, collaboration with external stakeholders, agency culture, staff burden, leadership support, worker's buy-in, client resistance, program fit, intervention clarity, supporting staff competency, and implementation teams.

DISCUSSION

The findings provide important information on the most common barriers and facilitators of implementation in child welfare settings.

The Potential of Pay-for-Success as a Financing Strategy for Evidence-Based Practices: An Illustration with Multisystemic Therapy

There is a critical need to identify strategies for financing the implementation of evidence-based practices. We illustrate the potential of pay-for-success financing (PFS)-a strategy in which private investors fund implementation and receive a return on investment from a government payer-using multisystemic therapy as an example. We argue that standard multisystemic therapy (for serious juvenile offenders) and several of its adaptations (for other complex behavioral problems in youth) would be good candidates for PFS in the right contexts. Despite some challenges for policymakers and administrators, PFS has significant potential as a financing strategy for evidence-based practices.

Economic evaluation in implementation science: Making the business case for implementation strategies

Implementation researchers have made notable progress in developing and testing implementation strategies (i.e., highly-specified methods used to help providers improve uptake of mental health evidence-based practices: EBPs). Yet, implementation strategies are not widely applied in healthcare organizations to improve delivery of EBPs. Economic considerations are a key factor influencing the use of implementation strategies to deliver and sustain mental health evidence-based practices, in part because many health care leaders are reluctant to invest in ongoing implementation strategy support without knowing the return-on-investment. Comparative economic evaluation of implementation strategies provides critical information for payers, policymakers, and providers to make informed decisions if specific strategies are an efficient use of scarce organizational resources. Currently, few implementation studies include implementation cost data and even fewer conduct comparative economic analyses of implementation strategies. This summary will introduce clinicians, researchers and other health professionals to the economic evaluation in implementation science. We provide an overview of different economic evaluation methods, discuss differences between economic evaluation in health services and implementation science. We also highlight approaches and frameworks to guide economic evaluation of implementation, provide an example for a cognitive-behavioral therapy program and discuss recommendations.