Enhancing outreach for persons with serious mental illness: 12-month results from a cluster randomized trial of an adaptive implementation strategy

Effect of a needs-based model of care on the characteristics of healthcare services in England: the i-THRIVE National Implementation Programme

AIMS

Developing integrated mental health services focused on the needs of children and young people is a key policy goal in England. The THRIVE Framework and its implementation programme, i-THRIVE, are widely used in England. This study examines experiences of staff using i-THRIVE, estimates its effectiveness, and assesses how local system working relationships influence programme success.

METHODS

This evaluation uses a quasi-experimental design (10 implementation and 10 comparison sites.) Measurements included staff surveys and assessment of 'THRIVE-like' features of each site. Additional site-level characteristics were collected from health system reports. The effect of i-THRIVE was evaluated using a four-group propensity-score-weighted difference-in-differences model; the moderating effect of system working relationships was evaluated with a difference-in-difference-in-differences model.

RESULTS

Implementation site staff were more likely to report using THRIVE and more knowledgeable of THRIVE principles than comparison site staff. The mean improvement of fidelity scores among i-THRIVE sites was 16.7, and 8.8 among comparison sites; the weighted model did not find a statistically significant difference. However, results show that strong working relationships in the local system significantly enhance the effectiveness of i-THRIVE. Sites with highly effective working relationships showed a notable improvement in 'THRIVE-like' features, with an average increase of 16.41 points (95% confidence interval: 1.69-31.13, P-value: 0.031) over comparison sites. Sites with ineffective working relationships did not benefit from i-THRIVE (-2.76, 95% confidence interval: - 18.25-12.73, P-value: 0.708).

CONCLUSIONS

The findings underscore the importance of working relationship effectiveness in the successful adoption and implementation of multi-agency health policies like i-THRIVE.

Study protocol: type II hybrid effectiveness-implementation study of routine functional status screening in VA primary care

BACKGROUND

Maintaining functional status, defined as the ability to perform daily activities such as bathing, dressing, and preparing meals, is central to older adults' quality of life, health, and ability to remain independent. Identifying functional impairments - defined as having difficulty or needing help performing these activities - is essential for clinicians to provide optimal care to older adults, and on a population level, understanding function can help anticipate service needs. Yet uptake of standardized measurement of functional status into routine patient care has been slow and inconsistent due to the burden posed by current tools. The goal of the Patient-Aligned Care Team (PACT) Functional Status Screening Initiative is to implement and evaluate a patient-centered, low-burden intervention to improve identification and management of functional impairment among older veterans in Veterans Health Administration (VHA) primary care settings.

METHODS

We will conduct a hybrid type 2 implementation-effectiveness cluster-randomized adaptive trial at 8 VHA sites using the Practical, Robust Implementation and Sustainability Model (PRISM) to guide implementation and evaluation. During a Pre-Implementation phase, we will engage clinical partners and develop local adaptations to maximize intervention-setting fit. During an Implementation phase, we will launch a standard bundle of implementation strategies (coalition building, champions, technical assistance) and system-level audit and feedback, identify sites with low uptake, and randomize those sites to receive continued standard vs. enhanced strategies (standard strategies plus clinician-level audit and feedback). The primary implementation outcome is reach (proportion of eligible patients at each site who receive screening/assessment) and the primary effectiveness outcome is appropriate management of impairment (proportion of patients with identified impairments who receive related referrals).

DISCUSSION

Implementing routine measurement of functional status in primary care has the potential to improve identification and management of functional impairment for older veterans. Improved management includes increasing access to services and supports for veterans and family caregivers, reducing potentially preventable acute care utilization, and allowing veterans to live in the least restrictive setting for as long as possible. Implementation will also provide data to inform the delivery of proactive interventions to prevent and delay development of functional impairment and improve quality of life, health, and independence.

TRIAL REGISTRATION

Registered at ClinicalTrials.gov on May 7, 2024, at NCT06404970 ( https://clinicaltrials.gov/ ).

REPORTING GUIDELINES

Standards for Reporting Implementation Studies (Additional file 1).

Similarities and Differences Between Pragmatic Trials and Hybrid Effectiveness-Implementation Trials

Pragmatism in clinical trials is focused on increasing the generalizability of research findings for routine clinical care settings. Hybridism in clinical trials (i.e., assessing both clinical effectiveness and implementation success) is focused on speeding up the process by which evidence-based practices are developed and adopted into routine clinical care. Even though pragmatic trial methodologies and implementation science evolved from very different disciplines, Pragmatic Trials and Hybrid Effectiveness-Implementation Trials share many similar design features. In fact, these types of trials can easily be conflated, creating the potential for investigators to mislabel their trial type or mistakenly use the wrong trial type to answer their research question. Blurred boundaries between trial types can hamper the evaluation of grant applications, the scientific interpretation of findings, and policy-making. Acknowledging that most trials are not pure Pragmatic Trials nor pure Hybrid Effectiveness-Implementation Trials, there are key differences in these trial types and they answer very different research questions. The purpose of this paper is to clarify the similarities and differences of these trial types for funders, researchers, and policy-makers. In addition, recommendations are offered to help investigators choose, label, and operationalize the most appropriate trial type to answer their research question. These recommendations complement existing reporting guidelines for clinical effectiveness trials (TIDieR) and implementation trials (StaRI).

A trial of implementation facilitation to increase timely admission to methadone treatment

BACKGROUND

During the ongoing opioid epidemic, some Opioid Treatment Programs (OTPs) are unable to admit program applicants in a timely fashion. Interim methadone (IM) treatment (without routine counseling) is an effective approach to overcome this challenge when counseling capacity is inadequate to permit admissions within 14 days of request. It requires both federal and state approval and has been rarely utilized since its incorporation into the federal OTP regulations in 1993.

METHODS

We evaluated the impact of Implementation Facilitation (IF) on OTPs providing timely admission to methadone treatment (i.e., within 14 days of request), adopting IM, and changing admissions procedures. IF included data collection on admission processes and an external facilitator who engaged OTP leadership, Local Champions through site visits, remote academic detailing, and feedback. Local Champions and State Opioid Treatment Authorities (SOTAs) participated in learning collaboratives. Using a modified stepped wedge design, six OTPs in four US states on the east and west coasts were randomly assigned to one of two clusters that staggered the timing of IF receipt. Study Phases included: Pre-Implementation, IF, and Sustainability. OTPs submitted data on treatment requests and admissions for 28 months (N = 3108 requests for treatment).

RESULTS

Although none of the OTPs adopted IM, all six developed policies and procedures to enable its use. Some OTPs streamlined admissions processes prior to study launch and during the IF intervention. OTPs reduced admission delays over time, although there was substantial site heterogeneity. The IF Phase for the early cluster coincided with the onset of COVID-19, complicating the study. Rates of timely admission within 14 days of request were 56.2 % (Pre-Implementation), 55.8 % (IF), and 78.8 % (Sustainability). Compared to the Pre-Implementation Phase, the odds of timely admission were not significantly different during the IF Phase but significantly higher during the Sustainability Phase (OR = 2.35 [95 % CI = 1.34, 4.12]; p = 0.003).

CONCLUSIONS

Committing to study participation and IF activities may have prompted some OTPs to change practices that improved timely admission. Attributing changes to IF should be done with caution considering study limitations. Data collection for the study spanned the COVID-19 pandemic, which complicates interpretation.

TRIAL REGISTRATION

Clinicaltrials.gov registration # NCT04188977.

Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication

The promise of adaptation and adaptive designs in implementation science has been hindered by the lack of clarity and precision in defining what it means to adapt, especially regarding the distinction between adaptive study designs and adaptive implementation strategies. To ensure a common language for science and practice, authors reviewed the implementation science literature and found that the term adaptive was used to describe interventions, implementation strategies, and trial designs. To provide clarity and offer recommendations for reporting and strengthening study design, we propose a taxonomy that describes fixed versus adaptive implementation strategies and implementation trial designs. To improve impact, (a) futureimplementation studies should prespecify implementation strategy core functions that in turn can be taught to and replicated by health system/community partners, (b) funders should support exploratory studies that refine and specify implementation strategies, and (c) investigators should systematically address design requirements and ethical considerations (e.g., randomization, blinding/masking) with health system/community partners.

Group physical therapy for knee osteoarthritis: protocol for a hybrid type III effectiveness-implementation trial

BACKGROUND

Knee osteoarthritis (OA) is a leading cause of chronic pain and disability and one of the most common conditions treated in outpatient physical therapy (PT). Because of the high and growing prevalence of knee OA, there is a need for efficient approaches for delivering exercise-based PT to patients with knee OA. A prior randomized controlled trial (RCT) showed that a 6-session Group Physical Therapy Program for Knee OA (Group PT) yields equivalent or greater improvements in pain and functional outcomes compared with traditional individual PT, while requiring fewer clinician hours per patient to deliver. This manuscript describes the protocol for a hybrid type III effectiveness-implementation trial comparing two implementation packages to support delivery of Group PT.

METHODS

In this 12-month embedded trial, a minimum of 16 Veterans Affairs Medical Centers (VAMCs) will be randomized to receive one of two implementation support packages for their Group PT programs: a standard, low-touch support based on Replicating Effective Programs (REP) versus enhanced REP (enREP), which adds tailored, high-touch support if sites do not meet Group PT adoption and sustainment benchmarks at 6 and 9 months following launch. Implementation outcomes, including penetration (primary), adoption, and fidelity, will be assessed at 6 and 12 months (primary assessment time point). Additional analyses will include patient-level effectiveness outcomes (pain, function, satisfaction) and staffing and labor costs. A robust qualitative evaluation of site implementation context and experience, as well as site-led adaptations to the Group PT program, will be conducted.

DISCUSSION

To our knowledge, this study is the first to evaluate the impact of tailored, high-touch implementation support on implementation outcomes when compared to standardized, low-touch support for delivering a PT-based intervention. The Group PT program has strong potential to become a standard offering for PT, improving function and pain-related outcomes for patients with knee OA. Results will provide information regarding the effectiveness and value of this implementation approach and a deeper understanding of how healthcare systems can support wide-scale adoption of Group PT.

TRIAL REGISTRATION

This study was registered on March 7, 2022 at ClinicalTrials.gov (identifier NCT05282927 ).

What is the role of randomised trials in implementation science?

BACKGROUND

There is a consistent demand for implementation science to inform global efforts to close the gap between evidence and practice. Key evaluation questions for any given implementation strategy concern the assessment and understanding of effects. Randomised trials are generally accepted as offering the most trustworthy design for establishing effectiveness but may be underused in implementation science.

MAIN BODY

There is a continuing debate about the primacy of the place of randomised trials in evaluating implementation strategies, especially given the evolution of more rigorous quasi-experimental designs. Further critiques of trials for implementation science highlight that they cannot provide 'real world' evidence, address urgent and important questions, explain complex interventions nor understand contextual influences. We respond to these critiques of trials and highlight opportunities to enhance their timeliness and relevance through innovative designs, embedding within large-scale improvement programmes and harnessing routine data. Our suggestions for optimising the conditions for randomised trials of implementation strategies include strengthening partnerships with policy-makers and clinical leaders to realise the long-term value of rigorous evaluation and accelerating ethical approvals and decluttering governance procedures for lower risk studies.

CONCLUSION

Policy-makers and researchers should avoid prematurely discarding trial designs when evaluating implementation strategies and work to enhance the conditions for their conduct.

Connecting the science and practice of implementation - applying the lens of context to inform study design in implementation research

The saying "horses for courses" refers to the idea that different people and things possess different skills or qualities that are appropriate in different situations. In this paper, we apply the analogy of "horses for courses" to stimulate a debate about how and why we need to get better at selecting appropriate implementation research methods that take account of the context in which implementation occurs. To ensure that implementation research achieves its intended purpose of enhancing the uptake of research-informed evidence in policy and practice, we start from a position that implementation research should be explicitly connected to implementation practice. Building on our collective experience as implementation researchers, implementation practitioners (users of implementation research), implementation facilitators and implementation educators and subsequent deliberations with an international, inter-disciplinary group involved in practising and studying implementation, we present a discussion paper with practical suggestions that aim to inform more practice-relevant implementation research.

How does facilitation in healthcare work? Using mechanism mapping to illuminate the black box of a meta-implementation strategy

BACKGROUND

Healthcare facilitation, an implementation strategy designed to improve the uptake of effective clinical innovations in routine practice, has produced promising yet mixed results in randomized implementation trials and has not been fully researched across different contexts.

OBJECTIVE

Using mechanism mapping, which applies directed acyclic graphs that decompose an effect of interest into hypothesized causal steps and mechanisms, we propose a more concrete description of how healthcare facilitation works to inform its further study as a meta-implementation strategy.

METHODS

Using a modified Delphi consensus process, co-authors developed the mechanistic map based on a three-step process. First, they developed an initial logic model by collectively reviewing the literature and identifying the most relevant studies of healthcare facilitation components and mechanisms to date. Second, they applied the logic model to write vignettes describing how facilitation worked (or did not) based on recent empirical trials that were selected via consensus for inclusion and diversity in contextual settings (US, international sites). Finally, the mechanistic map was created based on the collective findings from the vignettes.

FINDINGS

Theory-based healthcare facilitation components informing the mechanistic map included staff engagement, role clarification, coalition-building through peer experiences and identifying champions, capacity-building through problem solving barriers, and organizational ownership of the implementation process. Across the vignettes, engagement of leaders and practitioners led to increased socialization of the facilitator's role in the organization. This in turn led to clarifying of roles and responsibilities among practitioners and identifying peer experiences led to increased coherence and sense-making of the value of adopting effective innovations. Increased trust develops across leadership and practitioners through expanded capacity in adoption of the effective innovation by identifying opportunities that mitigated barriers to practice change. Finally, these mechanisms led to eventual normalization and ownership of the effective innovation and healthcare facilitation process.

IMPACT

Mapping methodology provides a novel perspective of mechanisms of healthcare facilitation, notably how sensemaking, trust, and normalization contribute to quality improvement. This method may also enable more efficient and impactful hypothesis-testing and application of complex implementation strategies, with high relevance for lower-resourced settings, to inform effective innovation uptake.

Enhancing Use of Medications for Opioid Use Disorder Through External Coaching

OBJECTIVE

This randomized controlled trial tested whether external coaching influences addiction treatment providers' utilization of medications to treat opioid use disorder (MOUDs).

METHODS

This study recruited 75 unique clinical sites in Florida, Ohio, and Wisconsin, including 61 sites in specialty treatment agencies and 14 behavioral health sites within health systems. The trial used external coaching to increase use of MOUDs in the context of a learning collaborative and compared it with no coaching and no learning collaborative (control condition). Outcome measures of MOUD capacity and utilization were monthly tabulations of licensed buprenorphine slots (i.e., the number of patients who could be treated based on the buprenorphine waiver limits of the site's providers), buprenorphine use, and injectable naltrexone administration.

RESULTS

The coaching and control arms showed no significant difference at baseline. Although buprenorphine slots increased in both arms during the 30-month trial, growth increased twice as fast at the coaching sites, compared with the control sites (average monthly rate of 6.1% vs. 3.0%, respectively, p<0.001). Buprenorphine use showed a similar pattern; the monthly growth rate in the coaching arm was more than twice the rate in the control arm (5.3% vs. 2.4%, p<0.001). Coaching did not have an impact on injectable naltrexone, which grew less than 1% in both arms over the trial period.

CONCLUSIONS

External coaching can increase organizational capacity for and growth of buprenorphine use. Future research should explore the dimensions of coaching practice, dose, and delivery modality to better understand and enhance the coaching function.

Implementation Methods and Research for a Post-truth World with Growing Inequities

The purpose of this article is to consider the changing context for implementation research and practice and new approaches which might now be more relevant for some implementation objectives. Factors that hindered implementation of evidence-based practices before the COVID-19 pandemic was an anti-science culture, strengthened by different media and appeals to emotion and identity. The article questions how effective are the rational-cognitive and individual models of change that frequency informs our research and practice. It describes challenges we face and considers methods we could use that might be more effective, including research-informed narrative methods, participatory research and practice, especially with culturally and linguistically diverse peoples, and adaptive implementation.

Engaging stakeholders to inform national implementation of critical time intervention in a program serving homeless-experienced Veterans

The Veterans Affairs (VA) Grant and Per Diem Case Management "Aftercare" program provides 6 months of case management for homeless-experienced Veterans (HEVs) transitioning to permanent housing, with the aim of decreasing returns to homelessness. Implementing Critical Time Intervention (CTI)-an evidence-based case management practice-would standardize care across the 128 community-based agencies that provide Aftercare services. To prepare for national CTI implementation in Aftercare, guided by Replicating Effective Programs (REP), we conducted a four-site pilot in which we adapted a CTI implementation package (training, technical assistance, and external facilitation); characterized stakeholder perspectives regarding the acceptability and appropriateness of this package; and identified contextual factors that affected CTI implementation. We engaged a stakeholder workgroup to tailor existing CTI training and technical assistance materials for Aftercare. To provide tailored support for providers and leaders to adopt and incorporate evidence-based practices (EBPs) into routine care, we also developed external facilitation materials and processes. Over 9 months, we implemented this package at four sites. We conducted semi-structured interviews at pre-implementation, mid-implementation, and 6 months post-implementation, with HEVs (n = 37), case managers (n = 16), supervisors (n = 10), and VA leaders (n = 4); these data were integrated with templated reflection notes from the project facilitator. We used rapid qualitative analysis and targeted coding to assess the acceptability and appropriateness of CTI and our implementation package and identify factors influencing CTI implementation. Stakeholders generally found CTI acceptable and appropriate; there was consensus that components of CTI were useful and compatible for this setting. To adapt our implementation package for scale-up, this pilot highlighted the value of robust and tangible CTI training and technical assistance-grounded in real-world cases-that highlights the congruence of CTI with relevant performance metrics. Variations in agency-level contextual factors may necessitate more intense and tailored supports to implement and sustain complex EBPs like CTI. Processes used in this pilot are relevant for implementing other EBPs in organizations that serve vulnerable populations. EBP scale-up and sustainment can be enhanced by engaging stakeholders to tailor EBPs for specific contexts; pilot testing and refining implementation packages for scale-up; and using qualitative methods to characterize contextual factors that affect EBP implementation.

Study protocol for a randomized trial of bridge: Person-centered collaborative care for serious mental illness and cancer

BACKGROUND

Individuals with serious mental illness (SMI) experience inequities in cancer care that contribute to increased cancer mortality. Involving mental health at the time of cancer diagnosis may improve cancer care delivery for patients with SMI yet access to care remains challenging. Collaborative care is a promising approach to integrate mental health and cancer care that has not yet been studied in this marginalized population.

METHODS/DESIGN

We describe a 24-week, two-arm, single-site randomized trial of person-centered collaborative care (Bridge) for patients with SMI (schizophrenia, bipolar disorder, or major depression with psychiatric hospitalization) and their caregivers. 120 patients are randomized 1:1 to Bridge or Enhanced Usual Care (EUC) along with their caregivers. Researchers proactively identify individuals with SMI and a new breast, lung, gastrointestinal, or head and neck cancer that can be treated with curative intent. EUC includes informing oncologists about the patient's psychiatric diagnosis, notifying patients about available psychosocial services, and tracking patient and caregiver outcomes. Bridge includes a proactive assessment by psychiatry and social work, a person-centered, team approach including collaboration between mental health and oncology, and increased access to evidence-based psycho-oncology care. The primary outcome is cancer care disruptions evaluated by a blinded panel of oncologists. Secondary outcomes include patient and caregiver-reported outcomes and healthcare utilization. Barriers to Bridge implementation and dissemination are assessed using mixed methods.

DISCUSSION

This trial will inform efforts to systematically identify individuals with SMI and cancer and generate the first experimental evidence for the impact of person-centered collaborative care on cancer care for this underserved population.

Comparing two implementation strategies for implementing and sustaining a case management practice serving homeless-experienced veterans: a protocol for a type 3 hybrid cluster-randomized trial

Background

The Veterans Health Administration (VA) Grant and Per Diem case management “aftercare” program provides 6 months of case management for homeless-experienced veterans (HEVs) undergoing housing transitions. To standardize and improve aftercare services, we will implement critical time intervention (CTI), an evidence-based, structured, and time-limited case management practice. We will use two strategies to support the implementation and sustainment of CTI at 32 aftercare sites, conduct a mixed-methods evaluation of this implementation initiative, and generate a business case analysis and implementation playbook to support the continued spread and sustainment of CTI in aftercare.

Methods

We will use the Replicating Effective Programs (REP) implementation strategy to support CTI implementation at 32 sites selected by our partners. Half (n=16) of these sites will also receive 9 months of external facilitation (EF, enhanced REP). We will conduct a type 3 hybrid cluster-randomized trial to compare the impacts of REP versus enhanced REP. We will cluster potential sites into three implementation cohorts staggered in 9-month intervals. Within each cohort, we will use permuted block randomization to balance key site characteristics among sites receiving REP versus enhanced REP; sites will not be blinded to their assigned strategy. We will use mixed methods to assess the impacts of the implementation strategies. As fidelity to CTI influences its effectiveness, fidelity to CTI is our primary outcome, followed by sustainment, quality metrics, and costs. We hypothesize that enhanced REP will have higher costs than REP alone, but will result in stronger CTI fidelity, sustainment, and quality metrics, leading to a business case for enhanced REP. This work will lead to products that will support our partners in spreading and sustaining CTI in aftercare.

Discussion

Implementing CTI within aftercare holds the potential to enhance HEVs’ housing and health outcomes. Understanding effective strategies to support CTI implementation could assist with a larger CTI roll-out within aftercare and support the implementation of other case management practices within and outside VA.

Trial registration

This project was registered with ClinicalTrials.gov as “Implementing and sustaining Critical Time Intervention in case management programs for homeless-experienced Veterans.” Trial registration NCT05312229, registered April 4, 2022.

Michigan Model for HealthTM Learning to Enhance and Adapt for Prevention (Mi-LEAP): protocol of a pilot randomized trial comparing Enhanced Replicating Effective Programs versus standard implementation to deliver an evidence-based drug use prevention curriculum

BACKGROUND

School-based drug use prevention programs have demonstrated notable potential to reduce the onset and escalation of drug use, including among youth at risk of poor outcomes such as those exposed to trauma. Researchers have found a robust relationship between intervention fidelity and participant (i.e., student) outcomes. Effective implementation of evidence-based interventions, such as the Michigan Model for HealthTM (MMH), is critical to achieving desired public health objectives. Yet, a persistent gap remains in what we know works and how to effectively translate these findings into routine practice. The objective of this study is to design and test a multi-component implementation strategy to tailor MMH to meet population needs (i.e., students exposed to trauma), and improve the population-context fit to enhance fidelity and effectiveness.

METHODS

Using a 2-group, mixed-method randomized controlled trial design, this study will compare standard implementation versus Enhanced Replicating Effective Programs (REP) to deliver MMH. REP is a theoretically based implementation strategy that promotes evidence-based intervention (EBI) fidelity through a combination of EBI curriculum packaging, training, and as-needed technical assistance and is consistent with standard MMH implementation. Enhanced REP will tailor the intervention and training to integrate trauma-informed approaches and deploy customized implementation support (i.e., facilitation). The research will address the following specific aims: (1) design and test an implementation strategy (Enhanced REP) to deliver the MMH versus standard implementation and evaluate feasibility, acceptability, and appropriateness using mixed methods, (2) estimate the costs and cost-effectiveness of Enhanced REP to deliver MMH versus standard implementation.

DISCUSSION

This research will design and test a multi-component implementation strategy focused on enhancing the fit between the intervention and population needs while maintaining fidelity to MMH core functions. We focus on the feasibility of deploying the implementation strategy bundle and costing methods and preliminary information on cost input distributions. The substantive focus on youth at heightened risk of drug use and its consequences due to trauma exposure is significant because of the public health impact of prevention. Pilot studies of implementation strategies are underutilized and can provide vital information on designing and testing effective strategies by addressing potential design and methods uncertainties and the effects of the implementation strategy on implementation and student outcomes.

TRIAL REGISTRATION

NCT04752189-registered on 8 February 2021 on ClinicalTrials.gov PRS.

Primary aim results of a clustered SMART for developing a school-level, adaptive implementation strategy to support CBT delivery at high schools in Michigan

BACKGROUND

Schools increasingly provide mental health services to students, but often lack access to implementation strategies to support school-based (and school professional [SP]) delivery of evidence-based practices. Given substantial heterogeneity in implementation barriers across schools, development of adaptive implementation strategies that guide which implementation strategies to provide to which schools and when may be necessary to support scale-up.

METHODS

A clustered, sequential, multiple-assignment randomized trial (SMART) of high schools across Michigan was used to inform the development of a school-level adaptive implementation strategy for supporting SP-delivered cognitive behavioral therapy (CBT). All schools were first provided with implementation support informed by Replicating Effective Programs (REP) and then were randomized to add in-person Coaching or not (phase 1). After 8 weeks, schools were assessed for response based on SP-reported frequency of CBT delivered to students and/or barriers reported. Responder schools continued with phase 1 implementation strategies. Slower-responder schools (not providing ≥ 3 CBT components to ≥10 students or >2 organizational barriers identified) were re-randomized to add Facilitation to current support or not (phase 2). The primary aim hypothesis was that SPs at schools receiving the REP + Coaching + Facilitation adaptive implementation strategy would deliver more CBT sessions than SPs at schools receiving REP alone. Secondary aims compared four implementation strategies (Coaching vs no Coaching × Facilitation vs no Facilitation) on CBT sessions delivered, including by type (group, brief and full individual). Analyses used a marginal, weighted least squares approach developed for clustered SMARTs.

RESULTS

SPs (n = 169) at 94 high schools entered the study. N = 83 schools (88%) were slower-responders after phase 1. Contrary to the primary aim hypothesis, there was no evidence of a significant difference in CBT sessions delivered between REP + Coaching + Facilitation and REP alone (111.4 vs. 121.1 average total CBT sessions; p = 0.63). In secondary analyses, the adaptive strategy that offered REP + Facilitation resulted in the highest average CBT delivery (154.1 sessions) and the non-adaptive strategy offering REP + Coaching the lowest (94.5 sessions).

CONCLUSIONS

The most effective strategy in terms of average SP-reported CBT delivery is the adaptive implementation strategy that (i) begins with REP, (ii) augments with Facilitation for slower-responder schools (schools where SPs identified organizational barriers or struggled to deliver CBT), and (iii) stays the course with REP for responder schools.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03541317 , May 30, 2018.

Determining feasibility of incorporating consumer engagement into implementation activities: study protocol of a hybrid effectiveness-implementation type II pilot

INTRODUCTION

Implementation researchers could draw from participatory research to engage patients (consumers of healthcare) in implementation processes and possibly reduce healthcare disparities. There is a little consumer involvement in healthcare implementation, partially because no formal guidance exists. We will create and pilot a toolkit of methods to engage consumers from the US' Veterans Health Administration (VHA) in selecting and tailoring implementation strategies. This toolkit, Consumer Voice, will provide guidance on what, when, where, how and why an implementer might engage consumers in implementing treatments. We will pilot the toolkit by implementing Safety Planning Intervention for suicide prevention with rural veterans, a population with suicide disparities. Safety Planning Intervention is effective for reducing suicidal behaviours.

METHODS AND ANALYSIS

In Aim 1, we will use participatory approaches and user-centred design to develop Consumer Voice and its methods. In Aim 2, we will pilot Consumer Voice by implementing the Safety Planning Intervention in two clinics serving rural VHA patients. One site will receive a current implementation strategy (Implementation Facilitation) only; the second will receive Implementation Facilitation plus Consumer Voice. We will use mixed methods to assess feasibility and acceptability of Consumer Voice. We will compare sites on preliminary implementation (reach, adoption, fidelity) and clinical outcomes (depression severity, suicidal ideation, suicidal behaviour). In Aim 3, we will evaluate Aim 2 outcomes at 20 months to assess sustained impact. We will gather qualitative data on sustainability of the Safety Planning Intervention.

ETHICS AND DISSEMINATION

These studies are overseen by the Institutional Review Board at the Central Arkansas Veterans Healthcare System. We plan to use traditional academic modalities of dissemination (eg, conferences, publications). We plan to disseminate findings through meetings with other trainers in implementation practice so they may adopt Consumer Voice. We plan to share results with local community boards.

Combining Improvement and Implementation Sciences and Practices for the Post COVID-19 Era

Health services made many changes quickly in response to the SARS-CoV-2 pandemic. Many more are being made. Some changes were already evaluated, and there are rigorous research methods and frameworks for evaluating their local implementation and effectiveness. But how useful are these methods for evaluating changes where evidence of effectiveness is uncertain, or which need adaptation in a rapidly changing situation? Has implementation science provided implementers with tools for effective implementation of changes that need to be made quickly in response to the demands of the pandemic? This perspectives article describes how parts of the research and practitioner communities can use and develop a combination of implementation and improvement to enable faster and more effective change in the future, especially where evidence of local effectiveness is limited. We draw on previous reviews about the advantages and disadvantages of combining these two domains of knowledge and practice. We describe a generic digitally assisted rapid cycle testing (DA-RCT) approach that combines elements of each in order to better describe a change, monitor outcomes, and make adjustments to the change when implemented in a dynamic environment.

Implementing eScreening for suicide prevention in VA post-9/11 transition programs using a stepped-wedge, mixed-method, hybrid type-II implementation trial: a study protocol

BACKGROUND

Post-9/11 veterans who enroll in VA health care frequently present with suicidal ideation and/or recent suicidal behavior. Most of these veterans are not screened on their day of enrollment and their risk goes undetected. Screening for suicide risk, and associated mental health factors, can lead to early detection and referral to effective treatment, thereby decreasing suicide risk. eScreening is an innovative Gold Standard Practice with evidence to support its effectiveness and implementation potential in transition and care management (TCM) programs. We will evaluate the impact of eScreening to improve the rate and speed of suicide risk screening and referral to mental health care compared to current screening methods used by transition care managers. We will also evaluate the impact of an innovative, multicomponent implementation strategy (MCIS) on the reach, adoption, implementation, and sustained use of eScreening.

METHODS

This is an eight-site 4-year, stepped-wedge, mixed-method, hybrid type-II implementation trial comparing eScreening to screening as usual while also evaluating the potential impact of the MCIS focusing on external facilitation and Lean/SixSigma rapid process improvement workshops in TCM. The aims will address: 1) whether using eScreening compared to oral and/or paper-based methods in TCM programs is associated with improved rates and speed of PTSD, depression, alcohol, and suicide screening & evaluation, and increased referral to mental health treatment; 2) whether and to what degree our MCIS is feasible, acceptable, and has the potential to impact adoption, implementation, and maintenance of eScreening; and 3) how contextual factors influence the implementation of eScreening between high- and low-eScreening adopting sites. We will use a mixed methods approach guided by the RE-AIM outcomes of the Practical Robust Implementation and Sustainability Model (PRISM). Data to address Aim 1 will be collected via medical record query while data for Aims 2 and 3 will be collected from TCM staff questionnaires and qualitative interviews.

DISCUSSION

The results of this study will help identify best practices for screening in suicide prevention for Post-9/11 veterans enrolling in VA health care and will provide information on how best to implement technology-based screening into real-world clinical care programs.

TRIAL REGISTRATION

ClinicalTrials.gov : NCT04506164; date registered: August 20, 2020; retrospectively registered.

Adaptation of a quality improvement approach to implement eScreening in VHA healthcare settings: innovative use of the Lean Six Sigma Rapid Process Improvement Workshop

BACKGROUND

The Veterans Health Administration (VHA) developed a comprehensive mobile screening technology (eScreening) that provides customized and automated self-report health screening via mobile tablet for veterans seen in VHA settings. There is agreement about the value of health technology, but limited knowledge of how best to broadly implement and scale up health technologies. Quality improvement (QI) methods may offer solutions to overcome barriers related to broad scale implementation of technology in health systems. We aimed to develop a process guide for eScreening implementation in VHA clinics to automate self-report screening of mental health symptoms and psychosocial challenges.

METHODS

This was a two-phase, mixed methods implementation project building on an adapted quality improvement method. In phase one, we adapted and conducted an RPIW to develop a generalizable process guide for eScreening implementation (eScreening Playbook). In phase two, we integrated the eScreening Playbook and RPIW with additional strategies of training and facilitation to create a multicomponent implementation strategy (MCIS) for eScreening. We then piloted the MCIS in two VHA sites. Quantitative eScreening pre-implementation survey data and qualitative implementation process "mini interviews" were collected from individuals at each of the two sites who participated in the implementation process. Survey data were characterized using descriptive statistics, and interview data were independently coded using a rapid qualitative analytic approach.

RESULTS

Pilot data showed overall satisfaction and usefulness of our MCIS approach and identified some challenges, solutions, and potential adaptations across sites. Both sites used the components of the MCIS, but site 2 elected not to include the RPIW. Survey data revealed positive responses related to eScreening from staff at both sites. Interview data exposed implementation challenges related to the technology, support, and education at both sites. Workflow and staffing resource challenges were only reported by site 2.

CONCLUSIONS

Our use of RPIW and other QI methods to both develop a playbook and an implementation strategy for eScreening has created a testable implementation process to employ automated, patient-facing assessment. The efficient collection and communication of patient information have the potential to greatly improve access to and quality of healthcare.

Lagniappes: "A Little Something Extra" or Unintended Positive Consequences of Implementation Facilitation

OBJECTIVE

Implementation facilitation is an effective strategy that increases uptake of behavioral health interventions. Facilitation is grounded in partnerships with leadership and clinical stakeholders. Researchers have documented some negative consequences of facilitation-time, financial, and opportunity costs. Clinical leaders often agree to facilitation with the promise of increased implementation of an intervention. This study examined whether unintended positive consequences of facilitation might offset known costs.

METHODS

This study was part of a stepped-wedge, hybrid type 2, pragmatic trial that used implementation facilitation to implement primary care mental health integration (PCMHI) via telehealth technology in six satellite Veterans Health Administration (VHA) clinics. Two facilitators provided facilitation for at least 6 months. This study included a focused analysis of an emerging phenomenon captured through weekly debriefing interviews with facilitators: unintended positive consequences of implementation facilitation, termed "lagniappes" here. A rapid content analysis was conducted to identify and categorize these consequences.

RESULTS

The authors documented unintended positive consequences of the facilitation across the six VHA sites and categorized them into three clinically relevant domains: supporting PCMHI outreach at other clinics not in the original catchment area (e.g., providing tools to other sites), strengthening patient access (e.g., resolving unnecessary patient travel), and improving or modifying work processes (e.g., clarifying suicide assessment protocols). The positive consequences benefited sites and strengthened ongoing partnerships.

CONCLUSIONS

Documenting unintended positive consequences of implementation facilitation may increase leadership engagement. Facilitators should consider leveraging unintended positive consequences as advantages for the site that may add efficiency to facility processes and workflows.

Adaptation of an exercise intervention for pregnant women to community-based delivery: a study protocol

INTRODUCTION

Despite well-established guidelines and benefits to exercise, the majority of pregnant women in the USA fail to meet recommended activity levels. Studies need to determine feasible ways to translate clinical interventions to community settings by engaging pregnant women in widely accessible locations to ensure benefits to more women. The aim of this study is to adapt and determine feasibility, acceptability and fidelity of the research clinic-based Expecting intervention (NCT02125149) with pregnant women with obesity in community settings.

METHODS AND ANALYSIS

We will use the Replicating Effective Programs (REP) to guide the adaptation and implementation of the research clinic-based intervention into the community. REP provides a four-phase process for implementing evidence-based interventions including collection of feedback from community stakeholders, iterative piloting of the intervention in the community and a process for standardising the intervention across community settings. Following adaptation, the updated intervention will be piloted. The pilot study will include 60 expecting women. We will randomise half to receive the community-adapted Expecting intervention (intervention, N=30) and half to receive standard of care (control, N=30). Feasibility and Acceptability of Intervention Measures are primary outcomes as key indicators of feasibility. Secondary outcomes will include the number of intervention sessions completed, the change in the number of minutes of physical activity as measured by accelerometer, as well as change in health indicators from enrolment to time of delivery and 6 months post-delivery (ie, body mass index, blood pressure and total cholesterol).

ETHICS AND DISSEMINATION

This study has been approved by the Institutional Review Board (#260132). Findings will be shared with study participants and stakeholder advisors through written summaries and in-person presentations; results will also be shared through presentations at scientific conferences and publications in peer-reviewed journals.

TRIAL REGISTRATION NUMBER

NCT04298125; Pre-results.

Three Flavorings for a Soup to Cure what Ails Mental Health Services

With new tools from artificial intelligence and new perspectives on personalizing interventions, we could revolutionize the way mental health services are delivered and achieve major gains in improving the public's mental health. We examine Dr. Bickman's vision around these technological and paradigm changes that would usher in major scientific, workforce training, and societal cultural changes. We argue that additional efforts in research evaluations in implementation have the potential to scale up and adapt existing interventions and scale them out to diverse populations and service systems. The next stage of this work involves testing the effectiveness of personalized interventions that are preferred by the public and integrating these choices into sustainable service systems. We note cautions on the delivery of these programs as automated algorithmic recommendations are heretofore foreign to humans.

The Balanced Opioid Initiative: protocol for a clustered, sequential, multiple-assignment randomized trial to construct an adaptive implementation strategy to improve guideline-concordant opioid prescribing in primary care

BACKGROUND

Rates of opioid prescribing tripled in the USA between 1999 and 2015 and were associated with significant increases in opioid misuse and overdose death. Roughly half of all opioids are prescribed in primary care. Although clinical guidelines describe recommended opioid prescribing practices, implementing these guidelines in a way that balances safety and effectiveness vs. risk remains a challenge. The literature offers little help about which implementation strategies work best in different clinical settings or how strategies could be tailored to optimize their effectiveness in different contexts. Systems consultation consists of (1) educational/engagement meetings with audit and feedback reports, (2) practice facilitation, and (3) prescriber peer consulting. The study is designed to discover the most cost-effective sequence and combination of strategies for improving opioid prescribing practices in diverse primary care clinics.

METHODS/DESIGN

The study is a hybrid type 3 clustered, sequential, multiple-assignment randomized trial (SMART) that randomizes clinics from two health systems at two points, months 3 and 9, of a 21-month intervention. Clinics are provided one of four sequences of implementation strategies: a condition consisting of educational/engagement meetings and audit and feedback alone (EM/AF), EM/AF plus practice facilitation (PF), EM/AF + prescriber peer consulting (PPC), and EM/AF + PF + PPC. The study's primary outcome is morphine-milligram equivalent (MME) dose by prescribing clinicians within clinics. The study's primary aim is the comparison of EM/AF + PF + PPC versus EM/AF alone on change in MME from month 3 to month 21. The secondary aim is to derive cost estimates for each of the four sequences and compare them. The exploratory aim is to examine four tailoring variables that can be used to construct an adaptive implementation strategy to meet the needs of different primary care clinics.

DISCUSSION

Systems consultation is a practical blend of implementation strategies used in this case to improve opioid prescribing practices in primary care. The blend offers a range of strategies in sequences from minimally to substantially intensive. The results of this study promise to help us understand how to cost effectively improve the implementation of evidence-based practices.

TRIAL REGISTRATION

NCT04044521 (ClinicalTrials.gov). Registered 05 August 2019.

Experimental and quasi-experimental designs in implementation research

Implementation science is focused on maximizing the adoption, appropriate use, and sustainability of effective clinical practices in real world clinical settings. Many implementation science questions can be feasibly answered by fully experimental designs, typically in the form of randomized controlled trials (RCTs). Implementation-focused RCTs, however, usually differ from traditional efficacy- or effectiveness-oriented RCTs on key parameters. Other implementation science questions are more suited to quasi-experimental designs, which are intended to estimate the effect of an intervention in the absence of randomization. These designs include pre-post designs with a non-equivalent control group, interrupted time series (ITS), and stepped wedges, the last of which require all participants to receive the intervention, but in a staggered fashion. In this article we review the use of experimental designs in implementation science, including recent methodological advances for implementation studies. We also review the use of quasi-experimental designs in implementation science, and discuss the strengths and weaknesses of these approaches. This article is therefore meant to be a practical guide for researchers who are interested in selecting the most appropriate study design to answer relevant implementation science questions, and thereby increase the rate at which effective clinical practices are adopted, spread, and sustained.

Change in Patient Outcomes After Augmenting a Low-level Implementation Strategy in Community Practices That Are Slow to Adopt a Collaborative Chronic Care Model: A Cluster Randomized Implementation Trial

BACKGROUND

Implementation strategies are essential for promoting the uptake of evidence-based practices and for patients to receive optimal care. Yet strategies differ substantially in their intensity and feasibility. Lower-intensity strategies (eg, training and technical support) are commonly used but may be insufficient for all clinics. Limited research has examined the comparative effectiveness of augmentations to low-level implementation strategies for nonresponding clinics.

OBJECTIVES

To compare 2 augmentation strategies for improving uptake of an evidence-based collaborative chronic care model (CCM) on 18-month outcomes for patients with depression at community-based clinics nonresponsive to lower-level implementation support.

RESEARCH DESIGN

Providers initially received support using a low-level implementation strategy, Replicating Effective Programs (REP). After 6 months, nonresponsive clinics were randomized to add either external facilitation (REP+EF) or external and internal facilitation (REP+EF/IF).

MEASURES

The primary outcome was patient 12-item short form survey (SF-12) mental health score at month 18. Secondary outcomes were patient health questionnaire (PHQ-9) depression score at month 18 and receipt of the CCM during months 6 through 18.

RESULTS

Twenty-seven clinics were nonresponsive after 6 months of REP. Thirteen clinics (N=77 patients) were randomized to REP+EF and 14 (N=92) to REP+EF/IF. At 18 months, patients in the REP+EF/IF arm had worse SF-12 [diff, 8.38; 95% confidence interval (CI), 3.59-13.18] and PHQ-9 scores (diff, 1.82; 95% CI, -0.14 to 3.79), and lower odds of CCM receipt (odds ratio, 0.67; 95% CI, 0.30-1.49) than REP+EF patients.

CONCLUSIONS

Patients at sites receiving the more intensive REP+EF/IF saw less improvement in mood symptoms at 18 months than those receiving REP+EF and were no more likely to receive the CCM. For community-based clinics, EF augmentation may be more feasible than EF/IF for implementing CCMs.

Bridge: Person-Centered Collaborative Care for Patients with Serious Mental Illness and Cancer

BACKGROUND

Individuals with serious mental illness (SMI) experience increased cancer mortality due to inequities in cancer treatment. Psychiatric care at cancer diagnosis may improve care delivery, yet models for integrating psychiatry and cancer care are lacking. We assessed the feasibility and acceptability of a person-centered collaborative care trial for SMI and cancer.

SUBJECTS, MATERIALS, AND METHODS

We developed the Bridge intervention for patients with SMI (schizophrenia, bipolar disorder, and severe major depression) and cancer. Bridge includes proactive identification of SMI, person-centered care from a psychiatrist and case manager, and collaboration with oncology. We conducted a 12-week, single-group trial in patients with SMI and a new breast, gastrointestinal, lung, or head/neck cancer. We assessed the feasibility of patient identification, enrollment and study completion; evaluated acceptability and perceived benefit with exit interviews with patients, caregivers, and oncology clinicians; and examined change in psychiatric symptoms with the Brief Psychiatric Rating Scale (BPRS).

RESULTS

From November 2015 to April 2016, 30/33 eligible patients (90.9%) enrolled, and 25/29 (86.2%) completed assessments at all timepoints, meeting feasibility criteria. Of 24 patients, 23 (95.8%) found meeting with the psychiatrist helpful; 16/19 caregivers (84.2%) shared that Bridge addressed key caregiving challenges. Oncology clinicians evaluated Bridge as "very" or "most" useful for 94.3% of patients. Exit interviews with all participant groups suggested that Bridge fostered patient-clinician trust, increased access to psychiatric treatment, and enabled patients to initiate and complete cancer treatment. Psychiatric symptoms on the BPRS improved from baseline to 12 weeks.

CONCLUSION

Bridge is a feasible and acceptable care delivery model for patients with SMI, their caregivers, and oncology clinicians. Randomized trials are warranted to assess the efficacy of improving cancer outcomes in this underserved population.

IMPLICATIONS FOR PRACTICE

Serious mental illness affects 13 million U.S. adults who experience increased cancer mortality. To improve outcomes, new models of integrated oncology and mental health care are urgently needed. This study found that it was feasible to identify, enroll, and retain patients with serious mental illness and a new cancer in a trial of integrated mental health and cancer care (Bridge). Patients, caregivers, and oncologists reported that Bridge facilitated the initiation and completion of cancer care. Randomized trials are warranted to investigate the impact on cancer outcomes. Trial procedures may inform consent, engagement, and trial retention for patients with mental illness.

Developing a Comprehensive Evidence-Based Service Package for Toddlers with Autism in a Low Resource Setting: Early Detection, Early Intervention, and Care Coordination

Objective: The number of children with autism, who have many unmet needs, is increasing dramatically. However, the existing evidence shows that early identification and intervention are effective in reducing the later costs and burdens of autism spectrum disorder (ASD). Thus, the present study aimed to develop evidence-based services for children with autism in Iran to reduce its impacts on the affected children and their families and to decrease its burden on the society. Method : A 3-step study was conducted based on a modification of the Replicating Effective Programs (REP) framework (step 1: need assessment and situation analysis; step 2: identifying current evidence-based services; step 3: designing the first draft of the package and its core elements). Each step was conducted by a specific methodology. Results: By considering the obtained data, it was found that a package of services with 4 core components to respond to the perceived needs in Iran was needed: (1) early detection of at-risk children; (2) care coordination and facilitation of access to current services; (3) implementation of an evidence-based early intervention program; and (4) training interventionists using an effective educational framework based on evidence-based material. Conclusion: REP framework was used in the present study, which has been shown to be effective in adapting and implementing health care services. By considering the preconditions of REP, a comprehensive package of services, with 4 components was designed for toddlers with autism in Iran. The next step will be to study this package using a multicenter hybrid effectiveness-implementation randomized control trial.

Adaptive School-based Implementation of CBT (ASIC): clustered-SMART for building an optimized adaptive implementation intervention to improve uptake of mental health interventions in schools

Background

Depressive and anxiety disorders affect 20–30% of school-age youth, most of whom do not receive adequate services, contributing to poor developmental and academic outcomes. Evidence-based practices (EBPs) such as cognitive behavioral therapy (CBT) can improve outcomes, but numerous barriers limit access among affected youth. Many youth try to access mental health services in schools, but school professionals (SPs: counselors, psychologists, social workers) are rarely trained adequately in CBT methods. Further, SPs face organizational barriers to providing CBT, such as lack of administrative support. Three promising implementation strategies to address barriers to school-based CBT delivery include (1) Replicating Effective Programs (REP), which deploys customized CBT packaging, didactic training in CBT, and technical assistance; (2) coaching, which extends training via live supervision to improve SP competence in CBT delivery; and (3) facilitation, which employs an organizational expert who mentors SPs in strategic thinking to promote self-efficacy in garnering administrative support. REP is a relatively low-intensity/low-cost strategy, whereas coaching and facilitation require additional resources. However, not all schools will require all three strategies. The primary aim of this study is to compare the effectiveness of a school-level adaptive implementation intervention involving REP, coaching, and facilitation versus REP alone on the frequency of CBT delivered to students by SPs and student mental health outcomes. Secondary and exploratory aims examine cost-effectiveness, moderators, and mechanisms of implementation strategies.

Methods

Using a clustered, sequential multiple-assignment, randomized trial (SMART) design, ≥ 200 SPs from 100 schools across Michigan will be randomized initially to receive REP vs. REP+coaching. After 8 weeks, schools that do not meet a pre-specified implementation benchmark are re-randomized to continue with the initial strategy or to augment with facilitation.

Discussion

EBPs need to be implemented successfully and efficiently in settings where individuals are most likely to seek care in order to gain large-scale impact on public health. Adaptive implementation interventions hold the promise of providing cost-effective implementation support. This is the first study to test an adaptive implementation of CBT for school-age youth, at a statewide level, delivered by school staff, taking an EBP to large populations with limited mental health care access.

Trial registration

NCT03541317—Registered on 29 May 2018 on ClinicalTrials.gov PRS

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0808-8) contains supplementary material, which is available to authorized users.

Organizational culture and climate as moderators of enhanced outreach for persons with serious mental illness: results from a cluster-randomized trial of adaptive implementation strategies

Background

Organizational culture and climate are considered key factors in implementation efforts but have not been examined as moderators of implementation strategy comparative effectiveness. We investigated organizational culture and climate as moderators of comparative effectiveness of two sequences of implementation strategies (Immediate vs. Delayed Enhanced Replicating Effective Programs [REP]) combining Standard REP and REP enhanced with facilitation on implementation of an outreach program for Veterans with serious mental illness lost to care at Veterans Health Administration (VA) facilities nationwide.

Methods

This study is a secondary analysis of the cluster-randomized Re-Engage implementation trial that assigned 3075 patients at 89 VA facilities to either the Immediate or Delayed Enhanced REP sequences. We hypothesized that sites with stronger entrepreneurial culture, task, or relational climate would benefit more from Enhanced REP than Standard REP. Veteran- and site-level data from the Re-Engage trial were combined with site-aggregated measures of entrepreneurial culture and task and relational climate from the 2012 VA All Employee Survey. Longitudinal mixed-effects logistic models examined whether the comparative effectiveness of the Immediate vs. Delayed Enhanced REP sequences were moderated by culture or climate measures at 6 and 12 months post-randomization. Three Veteran-level outcomes related to the engagement with the VA system were assessed: updated documentation, attempted contact by coordinator, and completed contact.

Results

For updated documentation and attempted contact, Veterans at sites with higher entrepreneurial culture and task climate scores benefitted more from Enhanced REP compared to Standard REP than Veterans at sites with lower scores. Few culture or climate moderation effects were detected for the comparative effectiveness of the full sequences of implementation strategies.

Conclusions

Implementation strategy effectiveness is highly intertwined with contextual factors, and implementation practitioners may use knowledge of contextual moderation to tailor strategy deployment. We found that facilitation strategies provided with Enhanced REP were more effective at improving uptake of a mental health outreach program at sites with stronger entrepreneurial culture and task climate; Veterans at sites with lower levels of these measures saw more similar improvement under Standard and Enhanced REP. Within resource-constrained systems, practitioners may choose to target more intensive implementation strategies to sites that will most benefit from them.

Trial registration

ISRCTN: ISRCTN21059161. Date registered: April 11, 2013.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0787-9) contains supplementary material, which is available to authorized users.

Enhancing Mental and Physical Health of Women through Engagement and Retention (EMPOWER): a protocol for a program of research

BACKGROUND

The Enhancing Mental and Physical health of Women through Engagement and Retention or EMPOWER program represents a partnership with the US Department of Veterans Health Administration (VA) Health Service Research and Development investigators and the VA Office of Women's Health, National Center for Disease Prevention and Health Promotion, Primary Care-Mental Health Integration Program Office, Women's Mental Health Services, and the Office of Patient Centered Care and Cultural Transformation. EMPOWER includes three projects designed to improve women Veterans' engagement and retention in evidence-based care for high-priority health conditions, i.e., prediabetes, cardiovascular, and mental health.

METHODS/DESIGN

The three proposed projects will be conducted in VA primary care clinics that serve women Veterans including general primary care and women's health clinics. The first project is a 1-year quality improvement project targeting diabetes prevention. Two multi-site research implementation studies will focus on cardiovascular risk prevention and collaborative care to address women Veterans' mental health treatment needs respectively. All projects will use the evidence-based Replicating Effective Programs (REP) implementation strategy, enhanced with multi-stakeholder engagement and complexity theory. Mixed methods implementation evaluations will focus on investigating primary implementation outcomes of adoption, acceptability, feasibility, and reach. Program-wide organizational-, provider-, and patient-level measures and tools will be utilized to enhance synergy, productivity, and impact. Both implementation research studies will use a non-randomized stepped wedge design.

DISCUSSION

EMPOWER represents a coherent program of women's health implementation research and quality improvement that utilizes cross-project implementation strategies and evaluation methodology. The EMPOWER Quality Enhancement Research Initiative (QUERI) will constitute a major milestone for realizing women Veterans' engagement and empowerment in the VA system. EMPOWER QUERI will be conducted in close partnership with key VA operations partners, such as the VA Office of Women's Health, to disseminate and spread the programs nationally.

TRIAL REGISTRATION

The two implementation research studies described in this protocol have been registered as required: Facilitating Cardiovascular Risk Screening and Risk Reduction in Women Veterans: Trial registration NCT02991534 , registered 9 December 2016. Implementation of Tailored Collaborative Care for Women Veterans: Trial registration NCT02950961 , registered 21 October 2016.

Moving From Discovery to System-Wide Change: The Role of Research in a Learning Health Care System: Experience from Three Decades of Health Systems Research in the Veterans Health Administration

The Veterans Health Administration is unique, functioning as an integrated health care system that provides care to more than six million veterans annually and as a home to an established scientific enterprise that conducts more than $1 billion of research each year. The presence of research, spanning the continuum from basic health services to translational research, has helped the Department of Veterans Affairs (VA) realize the potential of a learning health care system and has contributed to significant improvements in clinical quality over the past two decades. It has also illustrated distinct pathways by which research influences clinical care and policy and has provided lessons on challenges in translating research into practice on a national scale. These lessons are increasingly relevant to other health care systems, as the issues confronting the VA—the need to provide timely access, coordination of care, and consistent high quality across a diverse system—mirror those of the larger US health care system.

Implementing Effective Policy in a National Mental Health Reengagement Program for Veterans

Policy is a powerful motivator of clinical change, but implementation success can depend on organizational characteristics. This article used validated measures of organizational resources, culture, and climate to predict uptake of a nationwide Veteran's Health Administration (VA) policy aimed at implementing Re-Engage, a brief care management program that reestablishes contact with veterans with serious mental illness lost to care. Patient care databases were used to identify 2738 veterans lost to care. Local recovery coordinators (LRCs) were to update disposition for 2738 veterans at 158 VA facilities and, as appropriate, facilitate a return to care. Multivariable regression was used to assess organizational culture and climate as predictors of early policy compliance (via LRC presence) and uptake at 6 months. Higher composite climate and culture scores were associated with higher odds of having a designated LRC but were not predictive of higher uptake. Sites with LRCs had significantly higher rates of updated documentation than sites without LRCs.

Intensive case management for severe mental illness

BACKGROUND

Intensive Case Management (ICM) is a community-based package of care aiming to provide long-term care for severely mentally ill people who do not require immediate admission. Intensive Case Management evolved from two original community models of care, Assertive Community Treatment (ACT) and Case Management (CM), where ICM emphasises the importance of small caseload (fewer than 20) and high-intensity input.

OBJECTIVES

To assess the effects of ICM as a means of caring for severely mentally ill people in the community in comparison with non-ICM (caseload greater than 20) and with standard community care. We did not distinguish between models of ICM. In addition, to assess whether the effect of ICM on hospitalisation (mean number of days per month in hospital) is influenced by the intervention's fidelity to the ACT model and by the rate of hospital use in the setting where the trial was conducted (baseline level of hospital use).

SEARCH METHODS

We searched the Cochrane Schizophrenia Group's Trials Register (last update search 10 April 2015).

SELECTION CRITERIA

All relevant randomised clinical trials focusing on people with severe mental illness, aged 18 to 65 years and treated in the community care setting, where ICM is compared to non-ICM or standard care.

DATA COLLECTION AND ANALYSIS

At least two review authors independently selected trials, assessed quality, and extracted data. For binary outcomes, we calculated risk ratio (RR) and its 95% confidence interval (CI), on an intention-to-treat basis. For continuous data, we estimated mean difference (MD) between groups and its 95% CI. We employed a random-effects model for analyses.We performed a random-effects meta-regression analysis to examine the association of the intervention's fidelity to the ACT model and the rate of hospital use in the setting where the trial was conducted with the treatment effect. We assessed overall quality for clinically important outcomes using the GRADE approach and investigated possible risk of bias within included trials.

MAIN RESULTS

The 2016 update included two more studies (n = 196) and more publications with additional data for four already included studies. The updated review therefore includes 7524 participants from 40 randomised controlled trials (RCTs). We found data relevant to two comparisons: ICM versus standard care, and ICM versus non-ICM. The majority of studies had a high risk of selective reporting. No studies provided data for relapse or important improvement in mental state.1. ICM versus standard careWhen ICM was compared with standard care for the outcome service use, ICM slightly reduced the number of days in hospital per month (n = 3595, 24 RCTs, MD -0.86, 95% CI -1.37 to -0.34,low-quality evidence). Similarly, for the outcome global state, ICM reduced the number of people leaving the trial early (n = 1798, 13 RCTs, RR 0.68, 95% CI 0.58 to 0.79, low-quality evidence). For the outcome adverse events, the evidence showed that ICM may make little or no difference in reducing death by suicide (n = 1456, 9 RCTs, RR 0.68, 95% CI 0.31 to 1.51, low-quality evidence). In addition, for the outcome social functioning, there was uncertainty about the effect of ICM on unemployment due to very low-quality evidence (n = 1129, 4 RCTs, RR 0.70, 95% CI 0.49 to 1.0, very low-quality evidence).2. ICM versus non-ICMWhen ICM was compared with non-ICM for the outcome service use, there was moderate-quality evidence that ICM probably makes little or no difference in the average number of days in hospital per month (n = 2220, 21 RCTs, MD -0.08, 95% CI -0.37 to 0.21, moderate-quality evidence) or in the average number of admissions (n = 678, 1 RCT, MD -0.18, 95% CI -0.41 to 0.05, moderate-quality evidence) compared to non-ICM. Similarly, the results showed that ICM may reduce the number of participants leaving the intervention early (n = 1970, 7 RCTs, RR 0.70, 95% CI 0.52 to 0.95,low-quality evidence) and that ICM may make little or no difference in reducing death by suicide (n = 1152, 3 RCTs, RR 0.88, 95% CI 0.27 to 2.84, low-quality evidence). Finally, for the outcome social functioning, there was uncertainty about the effect of ICM on unemployment as compared to non-ICM (n = 73, 1 RCT, RR 1.46, 95% CI 0.45 to 4.74, very low-quality evidence).3. Fidelity to ACTWithin the meta-regression we found that i.) the more ICM is adherent to the ACT model, the better it is at decreasing time in hospital ('organisation fidelity' variable coefficient -0.36, 95% CI -0.66 to -0.07); and ii.) the higher the baseline hospital use in the population, the better ICM is at decreasing time in hospital ('baseline hospital use' variable coefficient -0.20, 95% CI -0.32 to -0.10). Combining both these variables within the model, 'organisation fidelity' is no longer significant, but the 'baseline hospital use' result still significantly influences time in hospital (regression coefficient -0.18, 95% CI -0.29 to -0.07, P = 0.0027).

AUTHORS' CONCLUSIONS

Based on very low- to moderate-quality evidence, ICM is effective in ameliorating many outcomes relevant to people with severe mental illness. Compared to standard care, ICM may reduce hospitalisation and increase retention in care. It also globally improved social functioning, although ICM's effect on mental state and quality of life remains unclear. Intensive Case Management is at least valuable to people with severe mental illnesses in the subgroup of those with a high level of hospitalisation (about four days per month in past two years). Intensive Case Management models with high fidelity to the original team organisation of ACT model were more effective at reducing time in hospital.However, it is unclear what overall gain ICM provides on top of a less formal non-ICM approach.We do not think that more trials comparing current ICM with standard care or non-ICM are justified, however we currently know of no review comparing non-ICM with standard care, and this should be undertaken.