Annual Research Review: The state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field

Traveling Without a Map: An Incomplete History of the Road to Implementation Science and Where We May Go from Here

This editorial provides a brief history of mental health services research over the last 30 years and how findings from large-scale studies shocked the field and led to the lines of inquiry culminating in current implementation science research. I review the manuscripts published in this special issue of Administration and Policy in Mental Health in light of that history and usethese studies as a way to assess the state of the field. Finally, I present five takeaways extracted from these articles that may be useful in considering future directions for implementation research.

Editorial Perspective: A call to collective action - improving the implementation of evidence in children and young people's mental health

With growing mental health needs of children and young people and the increasing demand on children and young people's mental health services, narrowing the evidence to practice implementation gap has never been more important. Implementation science and research provides useful theory, identification of barriers and facilitators as well as suggested strategies for improved uptake of evidence-based treatments, but the application of these is often limited. Supporting optimal learning and implementation cultures based on collaborative, relational and pragmatic action planning is likely key. We propose suggested next steps and recommendations to move this agenda forward within the children and young people's mental health field with a 'call to action'. With the need for specific roles and clear accountability, we emphasise that between clinicians, researchers, consumers and policy makers this is everyone's business.

Implementing a complex psychosocial intervention for unstably housed Veterans: A realist-informed evaluation case study

Background

Only 7% of individuals with co-occurring mental health and substance use disorder (COD) receive services for both conditions. We implemented and evaluated maintaining independence and sobriety through systems integration, outreach and networking-Veteran's edition (MISSION-Vet), an evidence-based manualized psychosocial intervention for Veterans with CODs. This paper identifies the generative mechanisms that explain "how, for whom, and under what conditions" MISSION-Vet adoption, implementation, and fidelity work when applied in a complex adaptive system with facilitation support.

Methods

Within two VA healthcare systems (Sites A and B), a hybrid III trial tested facilitation to implement MISSION-Vet. We conducted a two-site case study based on 42 semi-structured consolidated framework for implementation research (CFIR) guided interviews with site leadership, implementers (social workers, peer specialists), and team members (facilitators, site leads). Interviews were coded and CFIR constructs used to generate "Context  +  Mechanism  =  Outcome" configurations to understand the conditions of MISSION-Vet adoption, implementation, and fidelity.

Results

Site A was low, and Site B was high in adoption, implementation, and fidelity. Adoption hesitancy/eagerness (outcome) resulted from the interaction of "external policy" (context) dampening/encouraging a "tension for change" (mechanism). Implementation intensity (outcome) was based on how "peer pressure" or practice culture (context) activated staff "self-efficacy" (mechanism) to engage with MISSION-Vet and appraise its "relative advantage" over current practices (mechanism). Fidelity relied on how "staffing structure and availability" (context) activated/muted "facilitation" (mechanism) to result in strategy and intervention adaptation (outcome).

Conclusions

We delineated how specific contexts activated certain mechanisms to drive the different stages of implementation of a multi-faceted COD treatment intervention.

Trial registration

ClinicalTrials.gov, NCT02942979.

Plain language abstract

Implementation is inherently dynamic and influenced by interdependent factors operating at the individual, organizational, and system levels. This is especially true for complex interventions addressing co-occurring mental health and substance use disorders because such interventions involve multiple treatment modalities delivered simultaneously, in busy practice settings, with challenging populations. This paper pairs consolidated framework for implementation research (CFIR) constructs with a realist evaluation approached to generate configurations important to the adoption, implementation, and adaptation stages of a highly complex intervention addressing the behavioural health and housing needs of a vulnerable population. Each configuration describes how contextual factors trigger mechanisms to generate implementation outcomes and answers "what works for whom, in what circumstances and in what respects, and how?" These findings further our understanding of possible mechanisms of change and push us to be more precise about identifying causal relationships among constructs that contribute to the success of implementing complex interventions. This work also moves us to think theoretically and methodologically in a more dynamic fashion, thereby leading to more responsive implementation practice.

Partnering with School Providers to Co-Design Mental Health Interventions: An Open Trial of Act & Adapt in Urban Public Middle Schools

Schools are well positioned to provide access to youth mental health services, but implementing effective programs that promote emotional and behavioral functioning in school settings is complicated by the poor fit of interventions developed in research settings to complex school contexts. The current study formed a research-practice partnership with two urban public schools and mental health providers employed by those schools (N = 6, 100% female, 50% Black/African American, 50% White/Caucasian) in the adaptation of a depression prevention intervention, Act & Adapt. The intervention was modified by decreasing meeting time and streamlining session content, increasing flexibility, making intervention materials more similar to academic curriculum, and increasing the focus on managing disruptive behavior within group sessions. In an open trial, 6^th grade students (N = 22; 59% boys, 31% Hispanic, 22% Black/African American, 4% Asian, 30% White/Caucasian) at both schools who were identified as clinically "at risk" reported improvements from baseline to post-intervention and at one-year follow-up on measures of emotional and behavioral difficulties and coping strategies, with parallel results by caregiver report. The providers reported satisfaction with the intervention, and qualitative analyses of provider focus groups suggested both barriers and facilitators to research-practice collaborations to implement mental health interventions in schools.

Leveraging human-centered design to implement modern psychological science: Return on an early investment

Although classic models of implementation emphasized the importance of innovation characteristics in their adoption and sustained use, contemporary implementation research and practice have deprioritized these variables. Human-centered design (HCD) is an approach that grounds product development in information collected about the people and settings that will ultimately use those products. HCD has strong roots in psychological theory, but its application is typically limited to the development of digital technologies. HCD is rarely applied to the design of psychosocial innovations-including both service-recipient-facing interventions and implementation strategies-within the applied psychological disciplines. The current article reviews the psychological origins of HCD and details pathways through which HCD theories and methods can be leveraged to advance the "core tasks" of contemporary implementation research and practice in psychology. These include (a) identification of multilevel implementation determinants through specification of user needs and contexts; (b) tailoring of implementation strategies, such as contextually driven intervention redesign; and (c) evaluating implementation mechanisms and outcomes, including disentangling how the core HCD focus on usability relates to closely associated implementation variables such as acceptability, feasibility, and appropriateness. Collectively, these applications provide directions through which to leverage the mature field of HCD, maximize psychology's return on its early theoretical investment, and promote the large-scale impact of findings from across the applied fields of psychology. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Dissemination Strategies to Accelerate the Policy Impact of Children's Mental Health Services Research

The United States is in the midst of a children's mental health crisis, with rates of depression, anxiety, and suicide increasing precipitously. Evidence produced by children's mental health services research can help address this crisis by informing public policy decisions about service delivery, system design, and investments in the social determinants of mental health. Unfortunately, the policy impact of children's mental health services research is limited because evidence often fails to reach policy makers, be responsive to their needs, resonate with their worldview, or reflect the contexts in which they make decisions. Dissemination strategies-defined as the development and targeted distribution of messages and materials about research evidence pertaining to a specific issue or intervention-can help address these challenges. Yet, limited integrated guidance exists to inform the design of such strategies. This article addresses this need by synthesizing the results of empirical studies to provide guidance about how to enhance the dissemination of children's mental health services research to policy makers. The article provides four recommendations about the content of policy maker-focused dissemination materials, discusses how strategic framing and message tailoring can increase the chances that evidence is persuasive to policy makers, and highlights strategies to ensure that evidence reaches policy makers.

"Putting Meat on the Bones": Understanding the Implementation of a Community-Based Early Intervention and Prevention Programme-Contextual, Person, and Programme Influences

The adoption and effective delivery of evidence-based interventions within "real-world" community-based, primary health care service settings are of crucial importance. In this paper, we explore the successes and challenges of implementing a new complex, group-based, early parenting intervention called the Parent and Infant (PIN) programme. This study involved a systematic analysis of the processes and factors that influence the implementation of the PIN programme; the analysis was guided and informed by the Implementation Outcome Framework and the Consolidated Framework for Implementation Research. A documentary review, alongside a series of one-to-one interviews and small group discussions with a range of stakeholders (n = 44), and 7 focus groups (n = 24) were used as data sources. Factors that promoted programme adoption, acceptability, and implementation feasibility included programme characteristics and stakeholder attitudes, as well as organisational and systems factors (e.g. leadership and collaboration). Key challenges to implementation success included engagement and adoption barriers. This research provides a useful and important example of real-world, theory-driven implementation research which helped to identify interrelated processes, factors, and contexts which shape and influence the implementation of early intervention and prevention programmes, removed for blind review.

Implementation Science, Behavior Analysis, and Supporting Evidence-based Practices for Individuals with Autism

Although applied behavior analysis researchers have created efficacious treatment and intervention practices for children and youth with autism spectrum disorder (ASD), there is a gap between research and practice. Implementation Science (IS) and Organizational Behavior Management (OBM), based with Applied Behavior Analysis, are two parallel fields that could close this gap. This paper provides descriptions of both IS and OBM, highlighting their commonalities and unique featuers. The paper concludes with examples of how researchers have used IS and OBM to promote practitioners' use of evidence-based practices and services for children and youth with ASD.

Implementing nudges for suicide prevention in real-world environments: project INSPIRE study protocol

Background

Suicide is a global health issue. There are a number of evidence-based practices for suicide screening, assessment, and intervention that are not routinely deployed in usual care settings. The goal of this study is to develop and test implementation strategies to facilitate evidence-based suicide screening, assessment, and intervention in two settings where individuals at risk for suicide are especially likely to present: primary care and specialty mental health care. We will leverage methods from behavioral economics, which involves understanding the many factors that influence human decision making, to inform strategy development.

Methods

We will identify key mechanisms that limit implementation of evidence-based suicide screening, assessment, and intervention practices in primary care and specialty mental health through contextual inquiry involving behavioral health and primary care clinicians. Second, we will use contextual inquiry results to systematically design a menu of behavioral economics-informed implementation strategies that cut across settings, in collaboration with an advisory board composed of key stakeholders (i.e., behavioral economists, clinicians, implementation scientists, and suicide prevention experts). Finally, we will conduct rapid-cycle trials to test and refine the menu of implementation strategies. Primary outcomes include clinician-reported feasibility and acceptability of the implementation strategies.

Discussion

Findings will elucidate ways to address common and unique barriers to evidence-based suicide screening, assessment, and intervention practices in primary care and specialty mental health care. Results will yield refined, pragmatically tested strategies that can inform larger confirmatory trials to combat the growing public health crisis of suicide.

Determinants of Implementing Evidence-Based Trauma-Focused Interventions for Children and Youth: A Systematic Review

A systematic review was conducted to identify determinants (barriers and facilitators) of implementing evidence-based psychosocial interventions for children and youth who experience emotional or behavioral difficulties due to trauma exposure. Determinants were coded, abstracted, and synthesized using the Exploration, Preparation, Implementation, and Sustainment framework. Twenty-three articles were included, all of which examined implementation of Trauma-Focused Cognitive Behavioral Therapy or Cognitive-Behavioral Intervention for Trauma in Schools. This review identified multilevel and multiphase determinants that can be addressed by implementation strategies to improve implementation and clinical outcomes, and suggests how future studies might address gaps in the evidence base.

Practitioners' Use and Evaluation of Transdiagnostic Youth Psychotherapy Years After Training and Consultation Have Ended

We examined practitioners' use of the transdiagnostic Modular Approach to Therapy for Children (MATCH) 7 years after learning MATCH for a clinical trial. The practitioners (N = 29; M_age = 52.10, SD = 12.29, 86% women, 97% white) reported using MATCH with 55% of their caseload; use of the various MATCH modules ranged from 39 to 70%. Use was positively associated with amount of MATCH experience in the trial, perceived effectiveness, and ease of implementation. Patterns of specific module use did not consistently match strength of prior evidence (e.g., exposure was least used of the anxiety modules), suggesting challenges for implementation science.

Designing the Future of Children's Mental Health Services

Advancements in evidence-based psychosocial interventions, digital technologies, and implementation strategies (i.e., health services research products) for youth mental health services have yet to yield significant improvement in public health outcomes. Achieving such impact will require that these research products are easy to use, useful, and contextually appropriate. This paper describes how human-centered design (HCD), an approach that aligns product development with the needs of the people and settings that use those products, can be leveraged to improve youth mental health services. We articulate how HCD can advance accessibility, effectiveness, and equity, with specific consideration of unique aspects of youth mental health services.

Therapist Financial Strain and Turnover: Interactions with System-Level Implementation of Evidence-Based Practices

Therapist turnover is a major problem in community mental health. Financial strain, which is composed of cognitive, emotional, and behavioral responses to the experience of economic hardship, is an understudied antecedent of therapist turnover given the tumultuous financial environment in community mental health. We prospectively examined the relationship between therapist financial strain and turnover in 247 therapists in 28 community mental health agencies. We expected greater therapist financial strain to predict higher turnover and participation in a system-funded evidence-based practice (EBP) training initiative to alleviate this effect. Controlling for covariates, financial strain predicted therapist turnover (OR 1.12, p = .045), but not for therapists who participated in an EBP training initiative. Reducing financial strain and/or promoting EBP implementation may be levers to reduce turnover.

Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes

A lack of effective therapist training is a major barrier to evidence-based intervention (EBI) delivery in the community. Systematic reviews published nearly a decade ago suggested that traditional EBI training leads to higher knowledge but not more EBI use, indicating that more work is needed to optimize EBI training and implementation. This systematic review synthesizes the training literature published since 2010 to evaluate how different training models (workshop, workshop with consultation, online training, train-the-trainer, and intensive training) affect therapists' knowledge, beliefs, and behaviors. Results and limitations for each approach are discussed. Findings show that training has advanced beyond provision of manuals and brief workshops; more intensive training models show promise for changing therapist behavior. However, methodological issues persist, limiting conclusions and pointing to important areas for future research.

Applying NUDGE to Inform Design of EBP Implementation Strategies in Community Mental Health Settings

We demonstrate the application of NUDGE (Narrow, Understand, Discover, Generate, Evaluate), a behavioral economics approach to systematically identifying behavioral barriers that impede behavior enactment, to the challenge of evidence-based practice (EBP) use in community behavioral health. Drawing on 65 clinician responses to a system-wide crowdsourcing challenge about EBP underutilization, we applied NUDGE to discover, synthesize and validate specific behavioral barriers to EBP utilization that directly inform the design of tailored implementation strategies. To our knowledge, this is the first study to apply behavioral economic insights to clinician-proposed solutions to implementation challenges in order to design implementation strategies. The study demonstrates the successful application of NUDGE to implementation strategy design and provides novel targets for intervention.

Strategy Configurations Directly Linked to Higher Hepatitis C Virus Treatment Starts: An Applied Use of Configurational Comparative Methods

BACKGROUND

The Department of Veterans Affairs (VA) cares for more patients with hepatitis C virus (HCV) than any other US health care system. We tracked the implementation strategies that VA sites used to implement highly effective new treatments for HCV with the aim of uncovering how combinations of implementation strategies influenced the uptake of the HCV treatment innovation. We applied Configurational Comparative Methods (CCMs) to uncover causal dependencies and identify difference-making strategy configurations, and to distinguish higher from lower HCV treating sites.

METHODS

We surveyed providers to assess VA sites' use of 73 implementation strategies to promote HCV treatment in the fiscal year 2015. CCMs were used to identify strategy configurations that uniquely distinguished higher HCV from lower HCV treating sites.

RESULTS

From the 73 possible implementation strategies, CCMs identified 5 distinct strategy configurations, or "solution paths." These were comprised of 10 individual strategies that collectively explained 80% of the sites with higher HCV treatment starts with 100% consistency. Using any one of the following 5 solution paths was sufficient to produce higher treatment starts: (1) technical assistance; (2) engaging in a learning collaborative AND designating leaders; (3) site visits AND outreach to patients to promote uptake and adherence; (4) developing resource sharing agreements AND an implementation blueprint; OR (5) creating new clinical teams AND sharing quality improvement knowledge with other sites AND engaging patients. There was equifinality in that the presence of any one of the 5 solution paths was sufficient for higher treatment starts.

CONCLUSIONS

Five strategy configurations distinguished higher HCV from lower HCV treating sites with 100% consistency. CCMs represent a methodological advancement that can help inform high-yield implementation strategy selection and increase the efficiency and effectiveness of future implementation efforts.

Improving the implementation and sustainment of evidence-based practices in community mental health organizations: a study protocol for a matched-pair cluster randomized pilot study of the Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS)

Background

Implementing and sustaining evidence-based programs with fidelity may require multiple implementation strategies tailored to address multi-level, context-specific barriers and facilitators. Ideally, selecting and tailoring implementation strategies should be guided by theory, evidence, and input from relevant stakeholders; however, methods to guide the selection and tailoring of strategies are not well-developed. There is a need for more rigorous methods for assessing and prioritizing implementation determinants (barriers and facilitators) and linking implementation strategies to determinants. The Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS) is an intervention designed to increase the effectiveness of evidence-based practice implementation and sustainment. COAST-IS will enable organizational leaders and clinicians to use Intervention Mapping to select and tailor implementation strategies to address their site-specific needs. Intervention Mapping is a multi-step process that incorporates theory, evidence, and stakeholder perspectives to ensure that implementation strategies effectively address key determinants of change.

Methods

COAST-IS will be piloted with community mental health organizations that are working to address the needs of children and youth who experience trauma-related emotional or behavioral difficulties by engaging in a learning collaborative to implement an evidence-based psychosocial intervention (trauma-focused cognitive behavioral therapy). Organizations will be matched and then randomized to participate in the learning collaborative only (control) or to receive additional support through COAST-IS. The primary aims of this study are to (1) assess the acceptability, appropriateness, feasibility, and perceived utility of COAST-IS; (2) evaluate the organizational stakeholders' fidelity to the core elements of COAST-IS; and (3) demonstrate the feasibility of testing COAST-IS in a larger effectiveness trial.

Discussion

COAST-IS is a systematic method that integrates theory, evidence, and stakeholder perspectives to improve the effectiveness and precision of implementation strategies. If effective, COAST-IS has the potential to improve the implementation and sustainment of a wide range of evidence-based practices in mental health and other sectors.

Trial registration

This study was registered in ClinicalTrials.gov (NCT03799432) on January 10, 2019 (last updated August 5, 2019).

Testing a theory of strategic implementation leadership, implementation climate, and clinicians' use of evidence-based practice: a 5-year panel analysis

BACKGROUND

Implementation theory suggests that first-level leaders, sometimes referred to as middle managers, can increase clinicians' use of evidence-based practice (EBP) in healthcare settings by enacting specific leadership behaviors (i.e., proactive, knowledgeable, supportive, perseverant with regard to implementation) that develop an EBP implementation climate within the organization; however, longitudinal and quasi-experimental studies are needed to test this hypothesis.

METHODS

Using data collected at three waves over a 5-year period from a panel of 30 outpatient children's mental health clinics employing 496 clinicians, we conducted a quasi-experimental difference-in-differences study to test whether within-organization change in implementation leadership predicted within-organization change in EBP implementation climate, and whether change in EBP implementation climate predicted within-organization change in clinicians' use of EBP. At each wave, clinicians reported on their first-level leaders' implementation leadership, their organization's EBP implementation climate, and their use of both EBP and non-EBP psychotherapy techniques for childhood psychiatric disorders. Hypotheses were tested using econometric two-way fixed effects regression models at the organization level which controlled for all stable organizational characteristics, population trends in the outcomes over time, and time-varying covariates.

RESULTS

Organizations that improved from low to high levels of implementation leadership experienced significantly greater increases in their level of EBP implementation climate (d = .92, p = .017) and within-organization increases in implementation leadership accounted for 11% of the variance in improvement in EBP implementation climate beyond all other covariates. In turn, organizations that improved from low to high levels of EBP implementation climate experienced significantly greater increases in their clinicians' average EBP use (d = .55, p = .007) and within-organization improvement in EBP implementation climate accounted for 14% of the variance in increased clinician EBP use. Mediation analyses indicated that improvement in implementation leadership had a significant indirect effect on clinicians' EBP use via improvement in EBP implementation climate (d = .26, 95% CI [.02 to .59]).

CONCLUSIONS

When first-level leaders increase their frequency of implementation leadership behaviors, organizational EBP implementation climate improves, which in turn contributes to increased EBP use by clinicians. Trials are needed to test strategies that target this implementation leadership-EBP implementation climate mechanism.

Reprint of: An introduction to effectiveness-implementation hybrid designs

The traditional research pipeline that encourages a staged approach to moving an intervention from efficacy trials to the real world can take a long time. To address this issue, hybrid effectiveness-implementation designs were codified to promote examination of both effectiveness and implementation outcomes within a study. There are three types of hybrid designs and they vary based on their primary focus and the amount of emphasis on effectiveness versus implementation outcomes. A type 1 hybrid focuses primarily on the effectiveness outcomes of an intervention while exploring the "implementability" of the intervention. A type 2 hybrid has a dual focus on effectiveness and implementation outcomes; these designs allow for the simultaneous testing or piloting of implementation strategies during an effectiveness trial. A type 3 hybrid focuses primarily on implementation outcomes while also collecting effectiveness outcomes as they relate to uptake or fidelity of the intervention. This paper provides an introduction to these designs and describes each of the three types, design considerations, and examples for each.

Effectiveness of community-based early intervention for children with autism spectrum disorder: a meta-analysis

BACKGROUND

Research trials of early intervention (EI) programs for children with autism spectrum disorder (ASD) generally demonstrate medium-to-large gains, on average, compared with "treatment as usual," in different developmental domains. Almost all children with ASD receive their treatment through community-based services, however, and studies suggest that evidence-based interventions rarely make their way into community practice. Understanding the effectiveness of community-based EI and factors associated with these effects is the first step in developing strategies for wide-scale implementation of effective EI.

METHODS

Studies of community-based EI for children with ASD were identified through a systematic search. Changes in cognitive, communication, social, and adaptive functioning from pre-treatment to post-treatment were assessed using standardized mean gain scores. Effect sizes were estimated using random effects models. Moderators of interest included type of community EI program, year of publication, intervention duration, and sample selection. Moderator effects were assessed using analysis of variance of mixed-effects models and meta-regression analyses.

RESULTS

Forty-six groups from 33 studies met inclusion criteria (1,713 participants, mean age 37.4 months, 81.1% male). There were small but statistically significant gains in each of the four domains. Hedges's g ranged from 0.21 for adaptive behavior to 0.32 for communication outcomes, after removing outliers and correcting for publication bias. EI programs associated with universities and hospitals were superior, on average, to other community EI programs for cognitive and adaptive behavior outcomes. Intervention duration was negatively associated with effect sizes for communication and adaptive behavior outcomes.

CONCLUSIONS

These results indicate that there remains a large gap between outcomes observed in community settings and those reported in efficacy trials.

A group randomized control trial to test the efficacy of the Road to Mental Readiness (R2MR) program among Canadian military recruits

BACKGROUND

Despite increased interest in workplace mental health interventions, the evidence for beneficial effects is mixed. Furthermore, many existing studies lack methodological rigor. We report results from a group randomized control trial to test the efficacy of a vastly popular intervention in Canada, the Road to Mental Readiness (R2MR) program, which has been widely disseminated in military, first responder, and civilian settings.

METHODS

The trial took place among Canadian Armed Forces military recruits completing their basic military qualification (BMQ) training, and randomized 65 platoons (N = 2831) into either (a) an Intervention (R2MR at week 2 of BMQ), or (b) a delayed Intervention Control (R2MR at week 9 of BMQ) condition. The principal investigator, participants, and data collection staff were blinded to platoon condition. Individual-level psychological functioning, resilience, mental health service use attitudes, intentions, and behaviours, and additional covariates were assessed with questionnaires around week 2 (a day or two before Intervention platoons received R2MR), at week 5, and at week 9 (a day or two before the Control platoons received R2MR). Military performance outcomes were obtained from administrative databases.

RESULTS

The full trial results were mixed; for some outcomes (psychological functioning, resilience, and military performance), we saw no evidence of beneficial effects; where we did see benefits (mental health service use attitudes, intentions, behaviours), the effects were very small, or disappeared over time. Analyses among two subsamples (Group 1: Intervention platoons with a Fidelity Check and their Controls, and Group 2: Intervention platoons without Fidelity Check and their Controls) indicated that for some outcomes (attitudes and help-seeking), under high fidelity conditions, the beneficial effects of R2MR were increased and better sustained; Conversely, under poor fidelity conditions, decreased beneficial effects or even iatrogenic effects were observed. Analyses across three training divisions indicated the larger organizational climate further influences efficacy.

CONCLUSIONS

Our findings paint a very complex picture in which it is made evident that sensible, evidence-informed workplace mental health interventions such as R2MR may work under high fidelity conditions, but may yield no discernable benefit or even inadvertent iatrogenic effects if implemented poorly or without sufficient consideration to the larger organizational context.

TRIAL REGISTRATION

ISRCTN 52557050 Registered 13 October 2016.

An introduction to effectiveness-implementation hybrid designs

The traditional research pipeline that encourages a staged approach to moving an intervention from efficacy trials to the real world can take a long time. To address this issue, hybrid effectiveness-implementation designs were codified to promote examination of both effectiveness and implementation outcomes within a study. There are three types of hybrid designs and they vary based on their primary focus and the amount of emphasis on effectiveness versus implementation outcomes. A type 1 hybrid focuses primarily on the effectiveness outcomes of an intervention while exploring the "implementability" of the intervention. A type 2 hybrid has a dual focus on effectiveness and implementation outcomes; these designs allow for the simultaneous testing or piloting of implementation strategies during an effectiveness trial. A type 3 hybrid focuses primarily on implementation outcomes while also collecting effectiveness outcomes as they relate to uptake or fidelity of the intervention. This paper provides an introduction to these designs and describes each of the three types, design considerations, and examples for each.

Mindfulness interventions in schools: Integrity and feasibility of implementation

Research on mindfulness-based interventions (MBIs) has proliferated in recent years and the integration of mindfulness-based programs in school settings has been at the forefront of implementation research. Recent reviews of studies evaluating the efficacy of school-based mindfulness programs have produced mixed findings, which may in part be due to methodological differences across studies to date, as well as adaptations of MBIs in order to implement them within school settings. For example, there are vast differences across studies in the content, delivery, and training requirements of school-based MBIs, which may influence both fidelity and efficacy. This systematic review aimed to synthesize the literature on the implementation of school-based MBIs and determine the degree to which the interventions align to standards for MBIs. A systematic search identified studies evaluating the effects of a school-based MBI on mental health outcomes in schools, utilizing quantitative and qualitative designs. The results draw comparison across the identified MBIs in relation to intervention integrity, and teacher training and competence according to existing standards. The findings indicate a poor alignment to all standards; for example, intervention integrity was indicated in 45% of studies that included core mindfulness practices, and standards of teacher training were met in 26% of studies. In addition, the feasibility design of studies is critiqued against recommended standards. Despite claims from many studies, feasibility has not yet been established for school settings. Feasibility studies have failed to adequately assess organizational factors that influence implementation. The lack of detail and consistency in reporting across studies is a particular limitation of the field of research, which may have disadvantaged the assessment of studies against stringent guidelines. The ongoing challenges to developing an evidence base for school-based MBIs as well as to implementation are discussed. Recommendations for future directions in implementation science of mindfulness interventions within a school context are provided.

A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia

BACKGROUND

Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static.

METHOD

Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia's public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised.

RESULTS

Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048).

CONCLUSIONS

System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.

Editorial Perspective: Reaching beyond the clinic: leveraging implementation science to improve access to child and adolescent mental health services

The mental health treatment gap for children and adolescents worldwide calls for implementation science research to help bring evidence-based interventions into practice. Given the near absence of services specific to child and adolescent mental health in both high-income and low- and middle-income countries, we recommend that implementation research for child and adolescent mental health extend beyond mental health clinics and primary care settings and focus on implementing, adopting, and sustaining integrated interventions from prevention to treatment, delivered by lay health workers, across sectors. We depict through selected examples what this research looks like in practice, highlight gaps in implementation research specific to children and adolescents, and make recommendations for future research. Child and adolescent focused mental health research informed by an implementation framework can generate effective solutions and increase the well-being of children worldwide.

Future Directions for Early Childhood Prevention of Mental Disorders: A Road Map to Mental Health, Earlier

Mental disorders are the predominant chronic diseases of youth, with substantial life span morbidity and mortality. A wealth of evidence demonstrates that the neurodevelopmental roots of common mental health problems are present in early childhood. Unfortunately, this has not been translated to systematic strategies for improving population-level mental health at this most malleable neurodevelopmental period. We lay out a translational Mental Health, Earlier road map as a key future direction for prevention of mental disorder. This paradigm shift aims to reduce population attributable risk of mental disorder emanating from early life, by preventing, attenuating, or delaying onset/course of chronic psychopathology via the promotion of self-regulation in early childhood within large-scale health care delivery systems. The Earlier Pillar rests on a "science of when to worry" that (a) optimizes clinical assessment methods for characterizing probabilistic clinical risk beginning in infancy via deliberate incorporation of neurodevelopmental heterogeneity, and (b) universal primary-care-based screening targeting patterns of dysregulated irritability as a robust transdiagnostic marker of vulnerability to life span mental health problems. The core of the Healthier Pillar is provision of low-intensity selective intervention promoting self-regulation for young children with developmentally atypical patterns of irritability within an implementation science framework in pediatric primary care to ensure highest population impact and sustainability. These Mental Health, Earlier strategies hold much promise for transforming clinical outlooks and ensuring young children's mental health and well-being in a manner that reverberates throughout the life span.

Commentary: A pathway forward for implementation science in the search to accelerate the delivery of effective mental health treatment and services for youth: reflections on Williams and Beidas (2019)

Over the past several decades, implementation science has emerged as a promising new field designed to reduce the gap between research and practice. One population that has not fully benefitted from recent efforts to implement evidence-based mental health treatment and services are children and adolescents. In the review by Williams and Beidas, the authors provide a thoughtful discussion on where the implementation science field has previously been within the area of child psychology and psychiatry, identified the scientific limitations of prior phases or 'waves' in the historical development of implementation science, and recommend where the field needs to go - by underscoring the importance of developing and testing multilevel, integrated causal theories and mechanism-based approaches. This commentary draws attention to the importance of the perspectives and recommendations proposed in this review, which offer the potential to generate new advances in the field of implementation science that will ultimately, improve mental health care and clinical outcomes for youth.

Organizational culture and climate profiles: relationships with fidelity to three evidence-based practices for autism in elementary schools

BACKGROUND

Implementation researchers have typically studied organizational culture and climate by testing whether individual dimensions are linked to the implementation of evidence-based practices (EBPs) rather than examining how the overarching social context influences implementation. This approach may limit implementation theory and strategy development to the extent that individual dimensions of culture and climate interact, mutually reinforce or counteract one another, or exhibit non-linear relationships. This study tests whether empirically identifiable culture and climate profiles emerge in a sample of organizations and examines how these profiles relate to EBP fidelity and work attitudes that support EBP sustainment, focusing on three EBPs for youth with autism delivered in schools as an example.

METHODS

The study included 65 elementary schools in the U.S. that implemented three EBPs-discrete trial training, pivotal response training, and visual schedules-for youth with autism. Organizational culture and climate and work attitudes were assessed using the Organizational Social Context measure at the beginning of the school year. Observations of EBP fidelity occurred mid school-year. We used bias-adjusted stepwise latent profile modeling to (1) identify subpopulations of schools that share similar culture and climate profiles, and (2) test for mean differences across profiles in observed EBP fidelity and teacher and staff work attitudes.

RESULTS

Controlling for region, four profiles best characterized the organizational cultures and climates of schools. Teachers and staff in schools with a comprehensive profile (high proficiency culture, positive climate) exhibited higher fidelity to two of three EBPs (d's = .95 to 1.64) and reported superior work attitudes (d's = .71 to 1.93) than teachers and staff in all other schools. Teachers and staff in supportive schools (low rigidity culture, positive climate) had better work attitudes, but not better fidelity, than those in schools with indifferent (low culture/climate, elevated stress) and constrained (high rigidity and resistance, high stress) profiles.

CONCLUSIONS

Organizational culture and climate profiles are a strong predictor of EBP fidelity and work attitudes that support EBP sustainment, highlighting the importance of an organization's overarching social context when developing implementation theory and strategies. Strategies that foster a comprehensive profile may improve EBP implementation.

Transforming Mental Health Delivery Through Behavioral Economics and Implementation Science: Protocol for Three Exploratory Projects

BACKGROUND

Efficacious psychiatric treatments are not consistently deployed in community practice, and clinical outcomes are attenuated compared with those achieved in clinical trials. A major focus for mental health services research is to develop effective and cost-effective strategies that increase the use of evidence-based assessment, prevention, and treatment approaches in community settings.

OBJECTIVE

The goal of this program of research is to apply insights from behavioral economics and participatory design to advance the science and practice of implementing evidence-based practice (EBP) for individuals with psychiatric disorders across the life span.

METHODS

Project 1 (Assisting Depressed Adults in Primary care Treatment [ADAPT]) is patient-focused and leverages decision-making heuristics to compare ways to incentivize adherence to antidepressant medications in the first 6 weeks of treatment among adults newly diagnosed with depression. Project 2 (App for Strengthening Services In Specialized Therapeutic Support [ASSISTS]) is provider-focused and utilizes normative pressure and social status to increase data collection among community mental health workers treating children with autism. Project 3 (Motivating Outpatient Therapists to Implement: Valuing a Team Effort [MOTIVATE]) explores how participatory design can be used to design organizational-level implementation strategies to increase clinician use of EBPs. The projects are supported by a Methods Core that provides expertise in implementation science, behavioral economics, participatory design, measurement, and associated statistical approaches.

RESULTS

Enrollment for project ADAPT started in 2018; results are expected in 2020. Enrollment for project ASSISTS will begin in 2019; results are expected in 2021. Enrollment for project MOTIVATE started in 2018; results are expected in 2019. Data collection had begun for ADAPT and MOTIVATE when this protocol was submitted.

CONCLUSIONS

This research will advance the science of implementation through efforts to improve implementation strategy design, measurement, and statistical methods. First, we will test and refine approaches to collaboratively design implementation strategies with stakeholders (eg, discrete choice experiments and innovation tournaments). Second, we will refine the measurement of mechanisms related to heuristics used in decision making. Third, we will develop new ways to test mechanisms in multilevel implementation trials. This trifecta, coupled with findings from our 3 exploratory projects, will lead to improvements in our knowledge of what causes successful implementation, what variables moderate and mediate the effects of those causal factors, and how best to leverage this knowledge to increase the quality of care for people with psychiatric disorders.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03441399; https://www.clinicaltrials.gov/ct2/show/NCT03441399 (Archived by WebCite at http://www.webcitation.org/74dRbonBD).

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/12121.

Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda

The field of implementation science was developed to better understand the factors that facilitate or impede implementation and generate evidence for implementation strategies. In this article, we briefly review progress in implementation science, and suggest five priorities for enhancing the impact of implementation strategies. Specifically, we suggest the need to: (1) enhance methods for designing and tailoring implementation strategies; (2) specify and test mechanisms of change; (3) conduct more effectiveness research on discrete, multi-faceted, and tailored implementation strategies; (4) increase economic evaluations of implementation strategies; and (5) improve the tracking and reporting of implementation strategies. We believe that pursuing these priorities will advance implementation science by helping us to understand when, where, why, and how implementation strategies improve implementation effectiveness and subsequent health outcomes.