Advancing Implementation Research and Practice in Behavioral Health Systems

Identifying implementation barriers and facilitators in an integrated behavioral health training program to improve workforce development

Integrated behavioral health (IBH) in pediatric primary care settings can improve access to needed care and outcomes. Behavioral Health Workforce Education and Training (BHWET) programs can better prepare the workforce and support improved care access. This case study identified factors that impede and facilitate the implementation of practice-based training in a BHWET program using the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Surveys were administered to BHWET trainees and primary care partners to assess stakeholder perspectives on field-based training processes. Qualitative data were extracted from surveys and trainee clinical logs. Providers reported high levels of satisfaction with IBH care training experiences. Data analysis identified several implementation facilitators and barriers. Implementation facilitators were associated with two EPIS outer context factors (leadership and collaboration) and one inner context factor (partner staffing). Identified barriers were associated with two inner context factors (organizational characteristics and data collection). Although the facilitators and barriers identified in this study were program specific, they have relevance for similar programs. Barriers identified at the organizational level reflect those identified in the research literature and provide insights for university programs regarding factors that must be considered when integrating IBH training components in primary care settings.

Bridges and Mechanisms: Integrating Systems Science Thinking into Implementation Research

We present a detailed argument for how to integrate, or bridge, systems science thinking and methods with implementation science. We start by showing how fundamental systems science principles of structure, dynamics, information, and utility are relevant for implementation science. Then we examine the need for implementation science to develop and apply richer theories of complex systems. This can be accomplished by emphasizing a causal mechanisms approach. Identifying causal mechanisms focuses on the "cogs and gears" of public health, clinical, and organizational interventions. A mechanisms approach focuses on how a specific strategy will produce the implementation outcome. We show how connecting systems science to implementation science opens new opportunities for examining and addressing social determinants of health and conducting equitable and ethical implementation research. Finally, we present case studies illustrating successful applications of systems science within implementation science in community health policy, tobacco control, health care access, and breast cancer screening.

What do you think it means? Using cognitive interviewing to improve measurement in implementation science: description and case example

Pragmatic measures are essential to evaluate the implementation of evidence-based interventions. Cognitive interviewing, a qualitative method that collects partner feedback throughout measure development, is particularly useful for developing pragmatic implementation measures. Measure developers can use cognitive interviewing to increase a measure's fit within a particular implementation context. However, cognitive interviewing is underused in implementation research, where most measures remain "homegrown" and used for single studies. We provide a rationale for using cognitive interviewing in implementation science studies and illustrate its use through a case example employing cognitive interviewing to inform development of a measurement-based care protocol for implementation in opioid treatment programs. Applications of cognitive interviewing, including developing a common language with partners and collecting multi-level feedback on assessment procedures, to improve measurement in implementation science are discussed.

Studying the implementation of Zero Suicide in a large health system: Challenges, adaptations, and lessons learned

Background

Suicide remains the 10th leading cause of death in the United States. Many patients presenting to healthcare settings with suicide risk are not identified and their risk mitigated during routine care. Our aim is to describe the planned methodology for studying the implementation of the Zero Suicide framework, a systems-based model designed to improve suicide risk detection and treatment, within a large healthcare system.

Methods

We planned to use a stepped wedge design to roll-out the Zero Suicide framework over 4 years with a total of 39 clinical units, spanning emergency department, inpatient, and outpatient settings, involving ∼310,000 patients. We used Lean, a widely adopted a continuous quality improvement (CQI) model, to implement improvements using a centralize "hub" working with smaller "spoke" teams comprising CQI personnel, unit managers, and frontline staff.

Results

Over the course of the study, five major disruptions impacted our research methods, including a change in The Joint Commission's safety standards for suicide risk mitigation yielding massive system-wide changes and the COVID-19 pandemic. What had been an ambitious program at onset became increasingly challenging because of the disruptions, requiring significant adaptations to our implementation approach and our study methods.

Conclusions

Real-life obstacles interfered markedly with our plans. While we were ultimately successful in implementing Zero Suicide, these obstacles led to adaptations to our approach and timeline and required substantial changes in our study methodology. Future studies of quality improvement efforts that cut across multiple units and settings within a given health system should avoid using a stepped-wedge design with randomization at the unit level if there is the potential for sentinel, system-wide events.

Moving from Implementation Science to Implementation Practice: The Need to Solve Practical Problems to Improve Behavioral Health Services

It is well recognized that the use of evidence-based practices (EBPs) is critical to improving service outcomes for those receiving behavioral health services. However, EBPs are not easily implemented in behavioral health settings, and there are many challenges to supporting these services over time. Recently, research efforts in implementation science (IS) have greatly expanded our understanding of issues that influence the successful implementation of EBPs. Unfortunately, less effort has been devoted to translating this research theory on a practical level to help individual service entities solve the specific problems of putting programs into place. A process is needed where service organizations and practitioners can build their capacity, informed by IS research, to improve service outcomes. The purpose of this commentary is to describe the IS research base, provide an introduction to implementation practice, describe challenges confronting practitioners, and propose necessary steps in building organizational capacity that enables practitioners to implement the most effective services available.

Implementation stages in practice: A review of behavioral health innovation within hospitals

Behavioral health influences patient health outcomes and healthcare utilization rates. Hospitals are promising settings for appropriate identification, treatment, and referral of behavioral health issues and may affect hospital admission rates and healthcare costs. Implementation frameworks are designed to aid successful adoption and scaling of health innovations. One type - process models - present staged frameworks for rolling out an innovation into routine practice. Process models are appealing for their pragmatism but are criticized for oversimplifying the complexity of implementation. This review investigates the empirical evidence for process models' utility in hospitals, chosen for their uniquely complex structures, by determining whether their use impacts implementation outcomes. Using systematic search and selection criteria across six databases, ten peer-reviewed studies were identified. Each applied a process model for implementing behavioral health innovations within hospital systems. Studies were coded by type of stage framework and reported implementation outcomes. Studies reported mostly favorable or mixed outcomes. No one framework prevailed in use nor evidence. Due to the paucity of published literature and reported data, there is limited evidence that process model application propels implementation outcomes in hospital settings. Furthering the science requires creating and utilizing systematic guidelines to employ process models, measure and report implementation stage transition, and measure and report implementation outcomes. Management and practitioners can include such data collection in standard process evaluations of hospital implementation and scale-up activities, or adopt complexity-informed approaches that lack the simplicity of process models but may be more realistic for complex settings.

Roles dissemination and implementation scientists can play in supporting research teams

The field of dissemination and implementation (D&I) science is rapidly growing, with many scientists seeking to apply D&I science to enhance and expand the impact of their work. As the D&I field grows and collaborations of implementation scientists with other fields flourish, a description for the roles for D&I scientists as they collaborate with researchers from other fields could be beneficial. This paper exemplifies how the D&I scientist/researcher collaborative process might work and important elements to consider in doing so, as well as provide an outline on how collaborations might progress for different project needs. This is discussed through example scenarios to consider an implementation scientists' engagement in a research project and describe potential roles for implementation scientists in supporting research teams. We then discuss characteristics to consider when incorporating a D&I expert into a team and considerations in navigating the scenarios.

A Case Study of Implementing Grant-Funded Integrated Care in a Community Mental Health Center

The US government funds integrated care demonstration projects to decrease health disparities for individuals with serious mental illness. Drawing on the Exploration Preparation Implementation Sustainability (EPIS) implementation framework, this case study of a community mental health clinic describes implementation barriers and sustainability challenges with grant-funded integrated care. Findings demonstrate that integrated care practices evolve during implementation and the following factors influenced sustainability: workforce rigidity, intervention clarity, policy and funding congruence between the agency and state/federal regulations, on-going support and training in practice application, and professional institutions. Implementation strategies for primary care integration within CMHCs include creating a flexible workforce, shared definition of integrated care, policy and funding congruence, and on-going support and training.

HIV/STI Service Delivery Within Juvenile Community Supervision Agencies: A National Survey of Practices and Approaches to Moving High-Risk Youth Through the HIV Care Cascade

Justice-involved youth (JIY) are at considerable risk for human immunodeficiency virus (HIV), but are disconnected from treatment and prevention. Juvenile justice agencies providing community supervision (CS) are well positioned to provide HIV prevention, testing, and prompt referral to treatment for JIY. However, we lack an understanding of juvenile CS agency responses to HIV/sexually transmitted infection (STI) needs among JIY. We conducted a nationwide systematic assessment of how juvenile CS agencies identify, refer, and move youth through the HIV care cascade using a nationally representative sample of 195 juvenile CS agencies across 20 states. Two-thirds of CS agencies did not offer any HIV-/STI-related services, and 82% reported no collaboration with health agencies. Screening or referral for HIV risk behaviors was reported by 32% of the CS agencies and 12% for any intervention or prevention for HIV/STI risk behaviors. Between 21% and 30% of agencies were unaware of the location of local HIV/STI services. HIV/STI prevention training was not a priority for directors and was ranked second to last out of 16 training topics. Agencies where staff expressed need for HIV risk training and where specific court programming was available were more likely to provide or refer for HIV/STI screening and/or testing. Agencies were more likely to provide or refer for services if they provided pre-trial/pre-adjudication supervision, parole, or court programming. Considering the low provision of HIV/STI-related services and limited collaboration between health and justice agencies, interventions that promote cross-system collaboration designed to minimize barriers and facilitate identification, referral, and linkage to HIV services for JIY are necessary.

Using implementation science to build intimate partner violence screening and referral capacity in a fracture clinic

RATIONALE, AIMS, AND OBJECTIVES

Over the past two decades, research informing good clinical practices related to intimate partner violence (IPV) has been plentiful, yet partner violence screening remains challenging to translate into action. In spite of the documented efficacy of routine screening for women of reproductive age and the availability of validated screening instruments, many IPV screening programmes lack the components necessary for success. In Toronto, a multidisciplinary team of researchers and clinicians is using the tools of implementation science to scale up an evidence-based IPV screening and response programme in an urban orthopaedic clinic where prior screening attempts have been ineffective.

METHODS

Using the Active Implementation Framework as a guide, researchers collected data across multiple sources to inform the first stage of implementation. Analysis focused on identifying the characteristics of the clinic that support or hinder implementation of new processes, evidence-based screening practices that fit with the clinic, and characteristics of a strong implementation team.

RESULTS

Through this process, researchers and clinicians uncovered organizational strengths and weaknesses related to IPV screening that may not have been identified previously. The need to incorporate technology into our screening processes became apparent, as did the importance of shared communication and colearning between clinicians and researchers.

CONCLUSIONS

The benefits of investing in the preparatory phases of implementation are discussed. Without undertaking the process of gathering and analysing data, examining the factors that support effective and sustainable implementation, and investing in the creation of a strong implementation team, it is likely that decisions about our screening approaches would have resulted in a less-effective and sustainable process.

Measuring implementation in global mental health: validation of a pragmatic implementation science measure in eastern Ukraine using an experimental vignette design

BACKGROUND

There is mounting evidence supporting the effectiveness of task-shifted mental health interventions in low- and middle-income countries (LMIC). However, there has been limited systematic scale-up or sustainability of these programs, indicating a need to study implementation. One barrier to progress is a lack of locally relevant and valid implementation measures. We adapted an existing brief dissemination and implementation (D&I) measure which includes scales for acceptability, appropriateness, feasibility and accessibility for local use and studied its validity and reliability among a sample of consumers in Ukraine.

METHODS

Local qualitative data informed adaptation of the measure and development of vignettes to test the reliability and validity. Participants were veterans and internally displaced persons (IDPs) recruited as part of a separate validity study of adapted mental health instruments. We examined internal consistency reliability, test-retest reliability, and construct and criterion validity for each scale on the measure. We randomly assigned half the participants to respond to a vignette depicting existing local psychiatric services which we knew were not well regarded, while the other half was randomized to a vignette describing a potentially more well-implemented mental health service. Criterion validity was assessed by comparing scores on each scale by vignette and by overall summary ratings of the programs described in the vignettes.

RESULTS

N = 169 participated in the qualitative study and N = 153 participated in the validity study. Qualitative findings suggested the addition of several items to the measure and indicated the importance of addressing professionalism/competency of providers in both the scales and the vignettes. Internal consistency reliabilities ranged from α = 0.85 for feasibility to α = 0.91 for appropriateness. Test-rest reliabilities were acceptable to good for all scales (rho: 0.61-0.79). All scales demonstrated substantial and significant differences in average scores by vignette assignment (ORs: 2.21-5.6) and overall ratings (ORs: 5.1-14.47), supporting criterion validity.

CONCLUSIONS

This study represents an innovative mixed-methods approach to testing an implementation science measure in contexts outside the United States. Results support the reliability and validity of most scales for consumers in Ukraine. Challenges included large amounts of missing data due to participants' difficulties responding to questions about a hypothetical program.

Advancing the state-level tracking of evidence-based practices: a case study

Background

Despite a sustained focus by policymakers and researchers on improving the standard of clinical care in public mental health services, the use of evidence-based practice remains low. Among other challenges, this reflects the difficulty of translating clinical research into useable policy that can be feasibly funded and monitored by state or large healthcare systems.

Case presentation

In this paper we present a case study of Washington State’s strategy for monitoring the use of clinical elements at the session level for all Medicaid-funded children’s mental health services. The implementation of this strategy reflects policy actions to promote effective practice while also actively influencing multiple other levels of the implementation ecology. The approach is informed by the Policy Ecology Framework, the Consolidated Framework for Implementation Research, the evidence-based policymaking literature, and common ontology and clinical elements models.

Conclusions

We found the strategy developed in Washington State to be a feasible method of collecting session level information about the use of effective clinical mental health practices. In addition, the approach appears to be having influence on multiple layers of the implementation ecology that could be explored through further study.

Challenges and opportunities for implementation of substance misuse interventions in conflict-affected populations

Alcohol and other drug misuse are significant but neglected public health issues in conflict-affected populations. In this article, we review the literature on the challenges and strategies for implementing substance misuse treatment and prevention services in conflict and post-conflict settings in low- and middle-income countries. We identified nine studies describing interventions in conflict-affected populations residing in Afghanistan, Croatia, India, Kenya, Kosovo, Pakistan, and Thailand. Six of these nine studies focused on refugee populations. Reports revealed challenges to intervention implementation, as well as promising practices and recommendations for future implementation that we characterized as existing in the inner and outer contexts of an implementing organization. Challenges existing in the outer context included low political prioritization, lack of coordination and integration, and limited advocacy for access to substance misuse services. Challenges within the inner context related to competing priorities and a shortage of providers. Resource limitations existed in both the inner and outer contexts. Stigma was a challenge that threatened implementation and utilization of substance use services in situations when substance use interventions were not congruent with the roles, structure, values, and authority of the system or implementing organization. Future research should focus on developing, applying, and evaluating strategies for overcoming these challenges in order to make progress toward meeting the need for substance misuse services in conflict-affected populations.

Researcher readiness for participating in community-engaged dissemination and implementation research: a conceptual framework of core competencies

Participating in community-engaged dissemination and implementation (CEDI) research is challenging for a variety of reasons. Currently, there is not specific guidance or a tool available for researchers to assess their readiness to conduct CEDI research. We propose a conceptual framework that identifies detailed competencies for researchers participating in CEDI and maps these competencies to domains. The framework is a necessary step toward developing a CEDI research readiness survey that measures a researcher's attitudes, willingness, and self-reported ability for acquiring the knowledge and performing the behaviors necessary for effective community engagement. The conceptual framework for CEDI competencies was developed by a team of eight faculty and staff affiliated with a university's Clinical and Translational Science Award (CTSA). The authors developed CEDI competencies by identifying the attitudes, knowledge, and behaviors necessary for carrying out commonly accepted CE principles. After collectively developing an initial list of competencies, team members individually mapped each competency to a single domain that provided the best fit. Following the individual mapping, the group held two sessions in which the sorting preferences were shared and discrepancies were discussed until consensus was reached. During this discussion, modifications to wording of competencies and domains were made as needed. The team then engaged five community stakeholders to review and modify the competencies and domains. The CEDI framework consists of 40 competencies organized into nine domains: perceived value of CE in D&I research, introspection and openness, knowledge of community characteristics, appreciation for stakeholder's experience with and attitudes toward research, preparing the partnership for collaborative decision-making, collaborative planning for the research design and goals, communication effectiveness, equitable distribution of resources and credit, and sustaining the partnership. Delineation of CEDI competencies advances the broader CE principles and D&I research goals found in the literature and facilitates development of readiness assessments tied to specific training resources for researchers interested in conducting CEDI research.

Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study

BACKGROUND

Examining the role of modifiable barriers and facilitators is a necessary step toward developing effective implementation strategies. This study examines whether both general (organizational culture, organizational climate, and transformational leadership) and strategic (implementation climate and implementation leadership) organizational-level factors predict therapist-level determinants of implementation (knowledge of and attitudes toward evidence-based practices).

METHODS

Within the context of a system-wide effort to increase the use of evidence-based practices (EBPs) and recovery-oriented care, we conducted an observational, cross-sectional study of 19 child-serving agencies in the City of Philadelphia, including 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators. Organizational variables included characteristics such as EBP initiative participation, program size, and proportion of independent contractor therapists; general factors such as organizational culture and climate (Organizational Social Context Measurement System) and transformational leadership (Multifactor Leadership Questionnaire); and strategic factors such as implementation climate (Implementation Climate Scale) and implementation leadership (Implementation Leadership Scale). Therapist-level variables included demographics, attitudes toward EBPs (Evidence-Based Practice Attitudes Scale), and knowledge of EBPs (Knowledge of Evidence-Based Services Questionnaire). We used linear mixed-effects regression models to estimate the associations between the predictor (organizational characteristics, general and strategic factors) and dependent (knowledge of and attitudes toward EBPs) variables.

RESULTS

Several variables were associated with therapists' knowledge of EBPs. Clinicians in organizations with more proficient cultures or higher levels of transformational leadership (idealized influence) had greater knowledge of EBPs; conversely, clinicians in organizations with more resistant cultures, more functional organizational climates, and implementation climates characterized by higher levels of financial reward for EBPs had less knowledge of EBPs. A number of organizational factors were associated with the therapists' attitudes toward EBPs. For example, more engaged organizational cultures, implementation climates characterized by higher levels of educational support, and more proactive implementation leadership were all associated with more positive attitudes toward EBPs.

CONCLUSIONS

This study provides evidence for the importance of both general and strategic organizational determinants as predictors of knowledge of and attitudes toward EBPs. The findings highlight the need for longitudinal and mixed-methods studies that examine the influence of organizational factors on implementation.