Transforming Mental Health Delivery Through Behavioral Economics and Implementation Science: Protocol for Three Exploratory Projects

Identifying Methods to Select and Tailor Implementation Strategies to Context-Specific Determinants in Child Mental Health Settings: A Scoping Review

This scoping review describes the state of the literature regarding Implementation Strategy Mapping Methods (ISMMs) within the context of child mental health practice delivery. Goals included (a) identify and describe ISMMs addressing determinants of implementing mental health evidence-based interventions (MH-EBIs) for children and (b) describe the scope of the literature (e.g., outcomes, remaining gaps) related to identified ISMMs. Following PRISMA-ScR guidelines, 197 articles were identified. After removing 54 duplicates, 152 titles and abstracts were screened, yielding 36 articles that were screened during the full-text review. The final sample included four studies and two protocol papers (n = 6). A data charting codebook was developed a priori to capture relevant information (e.g., outcomes) and content analysis was utilized to synthesize findings. Six ISMMs were identified: innovation tournament, concept mapping, modified conjoint analysis, COAST-IS, focus group, and intervention mapping. ISMMs were successful in leading to the identification and selection of implementation strategies at participating organizations, and all ISMMs included stakeholders throughout these processes. Findings revealed the novelty of this research area and highlighted numerous areas for future investigation. Implications related to implementation, service, and client outcomes are discussed, including the possible impact of utilizing ISMMs to increase access to MH-EBIs for children receiving services in community settings. Overall, these findings contribute to our understanding of one of the five priority areas within implementation strategy research-enhancing methods used to design and tailor implementation strategies-by providing an overview of methods that may be utilized to facilitate MH-EBI implementation in child mental health care settings. Trial Registration: Not applicable.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-023-00086-3.

Applying concepts from "rapid" and "agile" implementation to advance implementation research

Background

The translation of research findings into practice can be improved to maximize benefits more quickly and with greater flexibility. To expedite translation, researchers have developed innovative approaches to implementation branded as “rapid” and “agile” implementation. Rapid implementation has roots in precision medicine and agile implementation has roots in systems engineering and software design. Research has shown that innovation often derives from learning and applying ideas that have impacted other fields.

Implications for implementation researchers

This commentary examines “rapid” and “agile” approaches to implementation and provides recommendations to implementation researchers stemming from these approaches. Four key ideas are synthesized that may be broadly applicable to implementation research, including (1) adopting a problem orientation, (2) applying lessons from behavioral economics, (3) using adaptive study designs and adaptive interventions, and (4) using multi-level models to guide implementation. Examples are highlighted from the field where researchers are applying these key ideas to illustrate their potential impact.

Conclusions

“Rapid” and “agile” implementation approaches to implementation stem from diverse fields. Elements of these approaches show potential for advancing implementation research, although adopting them may entail shifting scientific norms in the field.

Leveraging behavioral economics and implementation science to engage patients at risk for suicide in mental health treatment: a pilot study protocol

Background

Primary care is an ideal setting to connect individuals at risk for suicide to follow-up care; however, only half of the patients referred from the primary care attend an initial mental health visit. We aim to develop acceptable, feasible, low-cost, and effective new strategies to increase treatment initiation among at-risk individuals identified in primary care.

Methods

We will conduct a multi-phase, mixed-methods study. First, we will conduct a chart review study by using administrative data, including medical records, to identify characteristics of primary care patients at risk for suicide who do or do not attend an initial mental health visit following a referral. Second, we will conduct a mixed methods study by using direct observations and qualitative interviews with key stakeholders (N = 65) to understand barriers and facilitators to mental health service initiation among at-risk individuals. Stakeholders will include patients with suicidal ideation referred from primary care who do and do not attend a first mental health visit, primary care and behavioral health providers, and individuals involved in the referral process. We also will collect preliminary self-report and behavioral data regarding potential mechanisms of behavior change (i.e., self-regulation and social support) from patients. Third, we will leverage these findings, relevant frameworks, and the extant literature to conduct a multi-arm, non-randomized feasibility trial. During this trial, we will rapidly prototype and test strategies to support attendance at initial mental health visits. Strategies will be developed with subject matter experts (N = 10) and iteratively pilot tested (~5 patients per strategy) and refined. Research will be completed in the Penn Integrated Care Program (PIC), which includes fourteen primary care clinics in Philadelphia that provide infrastructure for electronic referrals, patient communication, and data access.

Discussion

We will leverage frameworks and methods from behavioral economics and implementation science to develop strategies to increase mental health treatment initiation among individuals at risk for suicide identified in primary care. This project will lead to an evaluation of these strategies in a fully powered randomized trial and contribute to improvements in access to and engagement in mental health services for individuals at risk for suicide.

Trial registration

ClinicalTrials.gov Identifier: NCT05021224

Comparing output from two methods of participatory design for developing implementation strategies: traditional contextual inquiry vs. rapid crowd sourcing

Background

Participatory design methods are a key component of designing tailored implementation strategies. These methods vary in the resources required to execute and analyze their outputs. No work to date has examined the extent to which the output obtained from different approaches to participatory design varies.

Methods

We concurrently used two separate participatory design methods: (1) field observations and qualitative interviews (i.e., traditional contextual inquiry) and (2) rapid crowd sourcing (an innovation tournament). Our goal was to generate and compare information to tailor implementation strategies to increase the use of evidence-based data collection practices among one-to-one aides working with children with autism. Each method was executed and analyzed by study team members blinded to the output of the other method. We estimated the personnel time and monetary costs associated with each method to further facilitate comparison.

Results

Observations and interviews generated nearly double the number of implementation strategies (n = 26) than did the innovation tournament (n = 14). When strategies were classified into implementation strategies from the Expert Recommendations for Implementing Change (ERIC) taxonomy, there was considerable overlap in the content of identified strategies. However, strategies derived from observations and interviews were more specific than those from the innovation tournament. Five strategies (13%) reflected content unique to observations and interviews and 3 (8%) strategies were unique to the innovation tournament. Only observations and interviews identified implementation strategies related to adapting and tailoring to context; only the innovation tournament identified implementation strategies that used incentives. Observations and interviews required more than three times the personnel hours than the innovation tournament, but the innovation tournament was more costly overall due to the technological platform used.

Conclusions

There was substantial overlap in content derived from observations and interviews and the innovation tournament, although there was greater specificity in the findings from observations and interviews. However, the innovation tournament yielded unique information. To select the best participatory design approach to inform implementation strategy design for a particular context, researchers should carefully consider unique advantages of each method and weigh the resources available to invest in the process.

Making Progress Monitoring Easier and More Motivating: Developing a Client Data Collection App Incorporating User-Centered Design and Behavioral Economics Insights

Data collection is an important component of evidence-based behavioral interventions for children with autism, but many one-to-one aides (i.e., behavioral support staff) do not systemically collect quantitative data that are necessary for best-practice client progress monitoring. Data collection of clients' behaviors often involves labor-intensive pen-and-paper practices. In addition, the solitary nature of one-to-one work limits opportunities for timely supervisor feedback, potentially reducing motivation to collect data. We incorporated principles from behavioral economics and user-centered design to develop a phone-based application, Footsteps, to address these challenges. We interviewed nine one-to-one aides working with children with autism and seven supervisors to ask for their app development ideas. We then developed the Footsteps app prototype and tested the prototype with 10 one-to-one aides and supervisors through three testing cycles. At each cycle, one-to-one aides rated app usability. Participants provided 76 discrete suggestions for improvement, including 29 new app features (e.g., behavior timer), 20 feature modifications (e.g., numeric type-in option for behavior frequency), four flow modifications (e.g., deleting a redundant form), and 23 out-of-scope suggestions. Of the participants that tested the app, 90% rated usability as good or excellent. Results support continuing to develop Footsteps and testing its impact in a clinical trial.

Study protocol for a type III hybrid effectiveness-implementation trial of strategies to implement firearm safety promotion as a universal suicide prevention strategy in pediatric primary care

Background

Insights from behavioral economics, or how individuals’ decisions and behaviors are shaped by finite cognitive resources (e.g., time, attention) and mental heuristics, have been underutilized in efforts to increase the use of evidence-based practices in implementation science. Using the example of firearm safety promotion in pediatric primary care, which addresses an evidence-to-practice gap in universal suicide prevention, we aim to determine: is a less costly and more scalable behavioral economic-informed implementation strategy (i.e., “Nudge”) powerful enough to change clinician behavior or is a more intensive and expensive facilitation strategy needed to overcome implementation barriers?

Methods

The Adolescent and child Suicide Prevention in Routine clinical Encounters (ASPIRE) hybrid type III effectiveness-implementation trial uses a longitudinal cluster randomized design. We will test the comparative effectiveness of two implementation strategies to support clinicians’ use of an evidence-based firearm safety practice, S.A.F.E. Firearm, in 32 pediatric practices across two health systems. All pediatric practices in the two health systems will receive S.A.F.E. Firearm materials, including training and cable locks. Half of the practices (k = 16) will be randomized to receive Nudge; the other half (k = 16) will be randomized to receive Nudge plus 1 year of facilitation to target additional practice and clinician implementation barriers (Nudge+). The primary implementation outcome is parent-reported clinician fidelity to the S.A.F.E Firearm program. Secondary implementation outcomes include reach and cost. To understand how the implementation strategies work, the primary mechanism to be tested is practice adaptive reserve, a self-report practice-level measure that includes relationship infrastructure, facilitative leadership, sense-making, teamwork, work environment, and culture of learning.

Discussion

The ASPIRE trial will integrate implementation science and behavioral economic approaches to advance our understanding of methods for implementing evidence-based firearm safety promotion practices in pediatric primary care. The study answers a question at the heart of many practice change efforts: which strategies are sufficient to support change, and why? Results of the trial will offer valuable insights into how best to implement evidence-based practices that address sensitive health matters in pediatric primary care.

Trial registration

ClinicalTrials.gov, NCT04844021. Registered 14 April 2021.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01154-8.

Understanding the value of adhering to or adapting evidence-based interventions: a study protocol of a discrete choice experiment

Background

Whereas the value of an evidence-based intervention (EBI) is often determined by its effect on clinical outcomes, the value of implementing and using EBIs in practice is broader, reflecting qualities such as appropriateness, equity, costs, and impact. Reconciling these value conflicts involves a complicated decision process that has received very limited scholarly attention. Inspired by studies on decision-making, the objective of this project is to explore how practitioners appraise the values of different outcomes and to test how this appraisal influences their decisions surrounding the so-called fidelity–adaptation dilemma. This dilemma is related to the balance between using an EBI as it was designed (to ensure its effectiveness) and making appropriate adaptations (to ensure alignment with constraints and possibilities in the local context).

Methods

This project consists of three sub-studies. The participants will be professionals leading evidence-based parental programs in Sweden and, in Sub-study 1, parents and decision-makers. Sub-study 1 will use sequential focus groups and individual interviews to explore parameters that influence fidelity and adaptation decisions—the dilemmas encountered, available options, how outcomes are valued by practitioners as well as other stakeholders, and value trade-offs. Sub-study 2 is a discrete choice experiment that will test how value appraisals influence decision-making using data from Sub-study 1 as input. Sub-study 3 uses a mixed-method design, with findings from the two preceding sub-studies as input in focus group interviews to investigate how practitioners make sense of findings from optimal decision situations (experiment) and constrained, real-world decision situations.

Discussion

The project will offer unique insights into decision-making processes that influence how EBIs are used in practice. Such knowledge is needed for a more granular understanding of how practitioners manage the fidelity–adaptation dilemma and thus, ultimately, how the value of EBI implementation can be optimized. This study contributes to our knowledge of what happens once EBIs are adopted—that is, the gap between the way in which EBIs are intended to be used and the way in which they are used in practice.

Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement

Meta-analyses have shown that digital mental health apps can be efficacious in reducing symptoms of depression and anxiety. However, real-world usage of apps is typically not sustained over time, and no studies systematically examine which features increase sustained engagement with apps or the relationship between engagement features and clinical efficacy. We conducted a systematic search of the literature to identify empirical studies that (1) investigate standalone apps for depression and/or anxiety in symptomatic participants and (2) report at least one measure of engagement. Features intended to increase engagement were categorized using the persuasive system design (PSD) framework and principles of behavioral economics. Twenty-five studies with 4159 participants were included in the analysis. PSD features were commonly used, whereas behavioral economics techniques were not. Smartphone apps were efficacious in treating symptoms of anxiety and depression in randomized controlled trials, with overall small-to-medium effects (g = 0.2888, SE = 0.0999, z(15) = 2.89, p = 0.0119, Q(df = 14) = 41.93, p < 0.0001, I2 = 66.6%), and apps that employed a greater number of engagement features as compared to the control condition had larger effect sizes (β = 0.0450, SE = 0.0164, t(15) = 2.7344, p = 0.0161). We observed an unexpected negative association between PSD features and engagement, as measured by completion rate (β = -0.0293, SE = 0.0121, t(17) = 02.4142, p = 0.0281). Overall, PSD features show promise for augmenting app efficacy, though engagement, as reflected in study completion, may not be the primary factor driving this association. The results suggest that expanding the use of PSD features in mental health apps may increase clinical benefits and that other techniques, such as those informed by behavioral economics, are employed infrequently.

Implementing nudges for suicide prevention in real-world environments: project INSPIRE study protocol

Background

Suicide is a global health issue. There are a number of evidence-based practices for suicide screening, assessment, and intervention that are not routinely deployed in usual care settings. The goal of this study is to develop and test implementation strategies to facilitate evidence-based suicide screening, assessment, and intervention in two settings where individuals at risk for suicide are especially likely to present: primary care and specialty mental health care. We will leverage methods from behavioral economics, which involves understanding the many factors that influence human decision making, to inform strategy development.

Methods

We will identify key mechanisms that limit implementation of evidence-based suicide screening, assessment, and intervention practices in primary care and specialty mental health through contextual inquiry involving behavioral health and primary care clinicians. Second, we will use contextual inquiry results to systematically design a menu of behavioral economics-informed implementation strategies that cut across settings, in collaboration with an advisory board composed of key stakeholders (i.e., behavioral economists, clinicians, implementation scientists, and suicide prevention experts). Finally, we will conduct rapid-cycle trials to test and refine the menu of implementation strategies. Primary outcomes include clinician-reported feasibility and acceptability of the implementation strategies.

Discussion

Findings will elucidate ways to address common and unique barriers to evidence-based suicide screening, assessment, and intervention practices in primary care and specialty mental health care. Results will yield refined, pragmatically tested strategies that can inform larger confirmatory trials to combat the growing public health crisis of suicide.

Estimating Downstream Budget Impacts in Implementation Research

Health care decision makers often request information showing how a new treatment or intervention will affect their budget (i.e., a budget impact analysis; BIA). In this article, we present key topics for considering how to measure downstream health care costs, a key component of the BIA, when implementing an evidence-based program designed to reduce a quality gap. Tracking health care utilization can be done with administrative or self-reported data, but estimating costs for these utilization data raises 2 issues that are often overlooked in implementation science. The first issue has to do with applicability: are the cost estimates applicable to the health care system that is implementing the quality improvement program? We often use national cost estimates or average payments, without considering whether these cost estimates are appropriate. Second, we need to determine the decision maker's time horizon to identify the costs that vary in that time horizon. If the BIA takes a short-term time horizon, then we should focus on costs that vary in the short run and exclude costs that are fixed over this time. BIA is an increasingly popular tool for health care decision makers interested in understanding the financial effect of implementing an evidence-based program. Without careful consideration of some key conceptual issues, we run the risk of misleading decision makers when presenting results from implementation studies.

Applying NUDGE to Inform Design of EBP Implementation Strategies in Community Mental Health Settings

We demonstrate the application of NUDGE (Narrow, Understand, Discover, Generate, Evaluate), a behavioral economics approach to systematically identifying behavioral barriers that impede behavior enactment, to the challenge of evidence-based practice (EBP) use in community behavioral health. Drawing on 65 clinician responses to a system-wide crowdsourcing challenge about EBP underutilization, we applied NUDGE to discover, synthesize and validate specific behavioral barriers to EBP utilization that directly inform the design of tailored implementation strategies. To our knowledge, this is the first study to apply behavioral economic insights to clinician-proposed solutions to implementation challenges in order to design implementation strategies. The study demonstrates the successful application of NUDGE to implementation strategy design and provides novel targets for intervention.