Mediators of measurement-based care implementation in community mental health settings: results from a mixed-methods evaluation

Mechanisms of change of a multifaceted implementation strategy on fidelity to a guideline for the prevention of mental health problems at the workplace: A mechanism analysis within a cluster-randomized trial

Background

Occupational guidelines exist to support workplaces with the prevention of mental health problems (MHP) among their staff. However, knowledge of effective implementation strategies to support their implementation is limited. This study experimentally tested whether a multifaceted implementation strategy - comprising an educational meeting, five workshops, implementation teams, small cyclical tests of change, and facilitation - improves fidelity to a guideline for preventing MHP in a school setting through the pathway of change of the Capability Opportunity Motivation-Behavior (COM-B)-model. To gain a more granular understanding of the mechanisms of change, the Theoretical Domains Framework (TDF) was used to specify mediators related to capability, opportunity, and motivation. This study tested whether the multifaceted strategy versus a discrete strategy (1) improves fidelity, (2) enhances capability, opportunity, and motivation over time, and (3) if the strategy's effect on fidelity is mediated by capability, opportunity, and motivation.

Methods

55 schools were randomly assigned to a multifaceted strategy or a discrete strategy. Fidelity was measured by questionnaires at baseline and 12 months, while capability, opportunity, and motivation were assessed three times within this period (directly after the educational meeting and at three and nine months). The Determinants of Implementation Behavior Questionnaire was used to assess TDF hypothesized mediators corresponding to the COM-B components. Separate pathways were analyzed for each mediator. Linear Mixed Modeling was employed to test the strategy's effect on fidelity, and mediation analyses were conducted using the PROCESS Macro.

Results

The multifaceted strategy led to improved fidelity at 12 months (B= 2.81, p<.001). Multifaceted schools reported higher scores for all mediators after nine months compared to schools receiving the discrete strategy. The effect of the multifaceted strategy on fidelity was partially mediated by all TDF mediators (p=<.05) except for beliefs about consequences. Capability-related mediators, including skills (Proportion-mediated= 41%, p=<.01) and behavioral regulation (Proportion-mediated= 35%, p=<.001), accounted for the largest proportion of the effect, followed by the motivation-related mediator goals (Proportion-mediated = 34%, p=<.01).

Conclusions

The multifaceted strategy improved guideline fidelity by enhancing capability, opportunity, and motivation confirming the proposed function of COM-B. This study addresses calls for experimental evidence on how multifaceted implementation strategies achieve implementation outcomes.

Trial registration

ClinicalTrials.org dr.nr 2020-01214.

Pre-implementation patient, provider, and administrator perspectives of remote measurement-based care in a safety net outpatient psychiatry department

Introduction

Psychiatric measurement-based care (MBC) can be more effective than usual care, but health systems face implementation challenges. Achieving attitudinal alignment before implementing MBC is critical, yet few studies incorporate perspectives from multiple stakeholders this early in planning. This analysis identifies alignment and themes in pre-implementation feedback from patients, providers, and administrators regarding a planned MBC implementation in a safety net psychiatry clinic.

Methods

We used interview guides informed by Conceptual Model of Implementation Research to gather qualitative pre-implementation attitudes about perceived Appropriateness, Acceptability, and Feasibility of an MBC measure (Computerized Adaptive Test-Mental Health; CAT-MH) from five patients, two providers, and six administrators. We applied rapid qualitative analysis methods to generate actionable feedback for department leadership still planning implementation. [Correction added on 22 January 2025, after first online publication: In the previous sentence, the word 'general' was replaced with the word 'generate'.] We used a multistep process to generate thematic findings with potential relevance for other similar mental health settings.

Results

There was more attitudinal alignment across stakeholder groups regarding MBC's Acceptability and Feasibility than its Appropriateness. All three groups agreed that it was important to contextualize MBC for patients and providers, anticipate MBC's impact on patient-provider relationships, and consider the system's capacity to respond to patient needs unearthed by CAT-MH before implementation began. Our thematic analysis suggests: (1) Introducing MBC may complicate patient-provider relationships by adding a new and potentially conflicting input for decision making, that is, MBC data, to the more typical inputs of patient report and provider expertise; [Correction added on 22 January 2025, after first online publication: In the previous sentence, the word 'complicated' was replaced with the word 'complicate'.] (2) MBC poses theoretical risks to health equity for safety net patients because of limitations in access to MBC tools themselves and the resources needed to respond to MBC data; and (3) Tension exists between individual- and system-level applications of MBC.

Conclusions

Our analysis highlights shifting treatment dynamics, equity considerations, and tension between individual- and population-level needs that our participants anticipated when planning for MBC implementation in a safety net psychiatry clinic.

A research agenda to advance the study of implementation mechanisms

BACKGROUND

Implementation science scholars have made significant progress identifying factors that enable or obstruct the implementation of evidence-based interventions, and testing strategies that may modify those factors. However, little research sheds light on how or why strategies work, in what contexts, and for whom. Studying implementation mechanisms-the processes responsible for change-is crucial for advancing the field of implementation science and enhancing its value in facilitating equitable policy and practice change. The Agency for Healthcare Research and Quality funded a conference series to achieve two aims: (1) develop a research agenda on implementation mechanisms, and (2) actively disseminate the research agenda to research, policy, and practice audiences. This article presents the resulting research agenda, including priorities and actions to encourage its execution.

METHOD

Building on prior concept mapping work, in a semi-structured, 3-day, in-person working meeting, 23 US-based researchers used a modified nominal group process to generate priorities and actions for addressing challenges to studying implementation mechanisms. During each of the three 120-min sessions, small groups responded to the prompt: "What actions need to be taken to move this research forward?" The groups brainstormed actions, which were then shared with the full group and discussed with the support of facilitators trained in structured group processes. Facilitators grouped critical and novel ideas into themes. Attendees voted on six themes they prioritized to discuss in a fourth, 120-min session, during which small groups operationalized prioritized actions. Subsequently, all ideas were collated, combined, and revised for clarity by a subset of the authorship team.

RESULTS

From this multistep process, 150 actions emerged across 10 priority areas, which together constitute the research agenda. Actions included discrete activities, projects, or products, and ways to shift how research is conducted to strengthen the study of implementation mechanisms.

CONCLUSIONS

This research agenda elevates actions to guide the selection, design, and evaluation of implementation mechanisms. By delineating recommended actions to address the challenges of studying implementation mechanisms, this research agenda facilitates expanding the field of implementation science, beyond studying what works to how and why strategies work, in what contexts, for whom, and with which interventions.

Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial

BACKGROUND

Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity.

METHODS

Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses.

RESULTS

LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045).

CONCLUSIONS

LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

Study protocol: Novel Methods for Implementing Measurement-Based Care with youth in Low-Resource Environments (NIMBLE)

BACKGROUND

For youth receiving care in community mental health centers, comorbidities are the rule rather than the exception. Using measurement-based care (MBC), or the routine evaluation of symptoms to inform care decisions, as the foundation of treatment for youth with comorbid problems significantly improves the impact of psychotherapy by focusing care and building engagement and alliance. MBC increases the rate of symptom improvement, detects clients who would otherwise deteriorate, and alerts clinicians to non-responders. Despite its demonstrated utility, MBC is rarely implemented with fidelity; less than 15% of providers report using MBC per recommendations. Previous efforts to support MBC implementation have yielded suboptimal outcomes, in part, due to organizations' challenges with identifying and prioritizing barriers and selecting and developing strategies to overcome them. New methods are needed for identifying and prioritizing barriers, and matching strategies to barriers to optimize MBC implementation and treatment quality to improve youth mental health outcomes in community settings.

METHODS

Pragmatic implementation methods will be piloted in four diverse community mental health centers. Methods include (a) rapid evidence synthesis; (b) rapid ethnography; (c) design kits (e.g., kits with disposable cameras, journals, maps); (d) barrier prioritization, and (e) causal pathway diagramming. These activities will generate actionable barriers; subsequently, we will use facilitated group processes to prioritize barriers and develop causal pathway diagrams to match strategies to barriers to create implementation plans that optimize MBC fidelity (Aim 1). We will track strategy deployment for 6 months, then compare MBC fidelity for another 6 months post-implementation with data from 2 years of historical controls (Aim 2). Finally, we will co-design a toolkit for design kit methods with youth and the practice and scientific communities (Aim 3).

DISCUSSION

Optimizing MBC implementation in community mental health centers could transform youth mental health care by ensuring the most pressing symptoms are targeted early in treatment. The discussion section highlights expected challenges and limits to using the five methods, including recruitment and engagement given the high pressure on community mental health settings.

TRIAL REGISTRATION

Clinicaltrials.gov. NCT05644756 . Registered on 18 November 2022. This trial was retrospectively registered.

A Clinical Leadership Lens on Implementing Progress Feedback in Three Countries: Development of a Multidimensional Qualitative Coding Scheme

BACKGROUND

Progress feedback, also known as measurement-based care (MBC), is the routine collection of patient-reported measures to monitor treatment progress and inform clinical decision-making. Although a key ingredient to improving mental health care, sustained use of progress feedback is poor. Integration into everyday workflow is challenging, impacted by a complex interrelated set of factors across patient, clinician, organizational, and health system levels. This study describes the development of a qualitative coding scheme for progress feedback implementation that accounts for the dynamic nature of barriers and facilitators across multiple levels of use in mental health settings. Such a coding scheme may help promote a common language for researchers and implementers to better identify barriers that need to be addressed, as well as facilitators that could be supported in different settings and contexts.

METHODS

Clinical staff, managers, and leaders from two Dutch, three Norwegian, and four mental health organizations in the USA participated in semi-structured interviews on how intra- and extra-organizational characteristics interact to influence the use of progress feedback in clinical practice, supervision, and program improvement. Interviews were conducted in the local language, then translated to English prior to qualitative coding.

RESULTS

A team-based consensus coding approach was used to refine an a priori expert-informed and literature-based qualitative scheme to incorporate new understandings and constructs as they emerged. First, this hermeneutic approach resulted in a multi-level coding scheme with nine superordinate categories and 30 subcategories. Second-order axial coding established contextually sensitive categories for barriers and facilitators.

CONCLUSIONS

The primary outcome is an empirically derived multi-level qualitative coding scheme that can be used in progress feedback implementation research and development. It can be applied across contexts and settings, with expectations for ongoing refinement. Suggestions for future research and application in practice settings are provided. Supplementary materials include the coding scheme and a detailed playbook.

A study protocol for a cluster randomized pragmatic trial for comparing strategies for implementing primary HPV testing for routine cervical cancer screening in a large health care system

BACKGROUND

Limited guidance exists regarding implementation strategies that best facilitate cancer screening practice substitution and achieve optimal stakeholder-centered outcomes. Here we describe the protocol for a randomized pragmatic trial comparing two implementation strategies to facilitate substitution of primary HPV screening for Pap and HPV co-testing to perform routine cervical cancer screening of women aged 30-65 years at Kaiser Permanente Southern California (KPSC).

METHODS

Twelve service areas within KPSC will be randomized to a "centrally-administered system-wide implementation + local-tailored implementation" strategy or a "centrally-administered system-wide implementation only" strategy. The centrally-administered strategy comprises clinician and staff educational activities. Sites in the local-tailored arm will then conduct a structured local needs assessment followed by site-specific selection and deployment of implementation interventions. Surveys and interviews will be conducted among women and providers from the primary care and ob/gyn departments prior to the system-wide transition, shortly after the transition, and after the completion of local-tailored interventions. A stakeholder advisory committee will assist with study design, defining stakeholder-centered outcomes, and developing data collection tools.

RESULTS

The primary outcome of interest is uptake of primary HPV screening. Secondary provider-centered outcomes include provider knowledge, delivery of patient education, satisfaction with the practice substitution process, and resistance to primary HPV screening. Secondary patient-centered outcomes include patient knowledge, stigma, and satisfaction with the screening process. Intervention fidelity will also be measured via surveys.

CONCLUSIONS

Findings from this study will help inform future use of a local-tailored implementation strategy for adopting primary HPV screening at large health care systems. Findings may also be applicable to other types of practice substitution.