Provider REport of Sustainment Scale (PRESS): development and validation of a brief measure of inner context sustainment

Using the implementation research logic model to examine high-intensity resistance rehabilitation implementation in skilled nursing facilities: a mixed methods multi-site case study

BACKGROUND

Implementing evidence-based rehabilitation in skilled nursing facilities (SNFs) is essential for enhancing physical function outcomes and mitigating risk of adverse events. Best implementation approaches in this complex setting are unknown. This study uses the Implementation Research Logic Model (IRLM) to retrospectively examine the implementation of high-intensity resistance rehabilitation (HIR) in SNFs, aiming to elucidate contextual factors and pathways that could enhance future HIR implementation endeavors.

METHODS

We conducted a convergent, mixed-methods multi-site case study (n = 8 sites). A standardized implementation strategy was employed, allowing sites to adapt this approach. HIR use was measured using the Provider Report of Sustainment Scale (PRESS). Contextual factors were identified using the Practical Robust Implementation and Sustainability Model (PRISM) through study-specific questionnaires and validated measures (Inner Setting Scale, Provider Perspective of Team Effectiveness, Evidence Based Practice Attitudes Scale, Perceived Characteristics of Intervention Scale, Self-Defined Burnout Measure, and Utrecht Engagement Scale), and analyzed descriptively. Interviews and focus groups with leadership and clinicians revealed contextual factors and strategies influencing implementation. Heat maps visualized site patterns, while an IRLM proposed provisional implementation pathways.

RESULTS

PRESS scores ranged from 3.75 (0.17) to 2.33 (0.67), indicating all sites implemented HIR to at least a "moderate extent". Higher-implementing sites demonstrated full-team ability to adapt HIR to diverse patients. Differentiating contextual factors between higher and lower implementing sites included clinician perspectives, site infrastructure, and satisfaction with leadership. Higher-implementing sites employed a higher volume of site-initiated implementation strategies, notably having a champion and patient engagement. Pathways that appeared to contribute to higher implementation extent included: 1) overcoming inertia of current practice through HIR salience, 2) overcoming clinician concerns of patient compatibility through affirmative experiences, 3) addressing clinician perspective of complexity with session planning, and 4) optimizing patient rehabilitation mindset through encouraging environments.

CONCLUSION

Improving physical function in older adults necessitates adoption of evidence-based rehabilitation like HIR. Implementation strategies that target infrastructure, including leadership support and communication channels, inertia of current practice, and clinician perspectives of HIR complexity and patient compatibility may facilitate implementation. Identifying a champion and providing guidance for effective patient engagement appear to be key.

The role of organizational characteristics in intervention sustainment: findings from a quantitative analysis in 42 HIV testing clinics in Vietnam

BACKGROUND

Evidence-based intervention (EBI) sustainment is one of public health's largest translational research problems. Fewer than half of public health EBIs are sustained long-term, and sustainment challenges are even more pressing in low and middle-income countries (LMICs). Organizational characteristics, including organizations' inner structures, culture, and climate, may play a key role in EBI sustainment. However, little quantitative research has examined these relationships, particularly in LMICs.

METHODS

In this observational study, we assessed the association between baseline organizational characteristics and EBI sustainment within a cluster randomized implementation trial in Vietnam testing strategies to scale-up Systems Navigation and Psychosocial Counseling (SNaP) for people who inject drugs (PWID) living with HIV across 42 HIV testing clinics. From the Exploration, Preparation, Implementation, and Sustainment (EPIS) Framework, five baseline organizational characteristics were selected for investigation: 1) organizational readiness for implementing change; 2) implementation leadership; 3) implementation climate; 4) percent PWID; and 5) staff workload. Six to ten months post-study completion, clinic staff and leadership completed a survey that included the Provider Report of Sustainment Scale (PRESS), a measure of EBI sustainment across a clinic. We conducted clinic-level simple and multiple linear regression analyses to evaluate the association between organizational characteristics and sustainment.

RESULTS

218 participants (94% completion rate) completed the PRESS survey. All implementation scales had good individual-level internal consistency reliability. Clinics with high organizational readiness to change at baseline had significantly greater SNaP sustainment than clinics with low organizational readiness to change (ß = 1.91, p = 0.015). None of the other organizational characteristics were associated with sustainment, controlling for study arm.

CONCLUSIONS

We identified the importance of organizational readiness for SNaP sustainment in Vietnam. This study adds to the evidence base around the relationship between organizational characteristics and HIV intervention sustainment and could inform the development of future sustainment strategies. We also identified several areas for organizational characteristic and sustainment measure advancement, including the need for pragmatic sustainment measures that also capture EBI adaptation. This research demonstrates that assessing clinics' organizational readiness pre-implementation and providing tailored support to those with low readiness scores could improve HIV intervention sustainment for key populations.

Using state opioid response grant funding to disseminate contingency management for substance use disorder treatment in Indiana

INTRODUCTION

Contingency management (CM) is an efficacious psychosocial intervention for substance use disorders with over 25-years of empirical support, yet CM adoption in SUD treatment settings is limited. In 2020, SAMHSA's State Opioid Response Grant (SOR) initiative included CM as an allowable activity to "treat stimulant use disorder and concurrent substance misuse, and to improve retention in care." This policy-driven funding mechanism has significant potential to expand CM implementation nationally. This study describes an SOR-funded program to disseminate CM in Indiana.

METHODS

Indiana government and university partners developed a multi-component, statewide CM dissemination and implementation plan, including 1) statewide promotion, 2) detailed application process for interested SUD treatment agencies, 3) live, expert-led CM workshop, 4) technical assistance (TA) sessions for participating agencies, and 5) agency-level start-up funds to offset CM-related expenses. The study collected data on provider/staff characteristics, CM knowledge and attitudes, readiness, perceived barriers, and CM implementation at pre- and post-training workshop and at 3- and 6-month follow-up. In Year 2, the study collected client-reported quality assurance data.

RESULTS

Staff (N = 72) from 12 selected agencies (13 sites) attended the CM workshop. About half (57 %) had some familiarity with CM, but only 14 % had any prior CM experience or training. Post workshop, participants reported increased CM knowledge and increased confidence in ability to implement CM. Sites completed 3-7 CM TA sessions and developed a tailored CM program. By 6 months, 9 sites had begun CM implementation. These sites averaged 57 days of implementation (range = 25-122), engagement of 23 clients (range = 4-77), delivery of 208 CM reinforcers (gift card codes; range = 8-366), and per-client payouts of $33.77 (range = $11.25-$49.48). Identified barriers to CM implementation included lack of time, client referrals, and resources (administrative, economic). Client-level quality assurance data indicated provider adherence to CM.

CONCLUSIONS

A multi-component training model funded by SOR yielded several new CM programs that operated successfully within SAMHSA's guidelines. Organizational barriers related to readiness/capacity, turnover, and buy-in remained for some sites and warrant attention in future CM dissemination and implementation efforts. This work illustrates the promise of applying SAMHSA SOR awards to disseminate CM in community agencies.

Study protocol for a hybrid type 3 effectiveness-implementation trial of a team-based implementation strategy to support educators' use of a social engagement intervention

BACKGROUND

Remaking Recess (RR) is a school-based evidence-based peer social engagement intervention for autistic students. RR involves direct training and coaching with educators; however, educators face several barriers to implementation at both the individual- and organizational-levels. This protocol paper describes a multi-site study that will test whether an educator-level implementation strategy, coaching, with or without a school-level implementation strategy, school-based teams, will maximize educators' use (fidelity and sustainment) of RR for autistic students and their peers who are socially-isolated, rejected, or peripheral and may need additional support during recess.

METHODS

This study will employ a hybrid type-3 effectiveness-implementation trial. Fifty-five elementary schools will be recruited as well as 121 educators (e.g., classroom assistants, aides), 55 general and special educator teachers, and 83-138 other school personnel (e.g., administrators). Additionally, at least 118 autistic students and allistic or non-autistic classmates will be recruited as RR recipients. Participants will complete baseline assessments at the beginning of the year, and all schools will be provided RR training. Schools will be randomized to coaching with or without school-based teams. This study will measure RR fidelity (primary outcome), RR sustainment, as well as peer engagement, social network inclusion, and social skills (secondary outcomes). It is expected that coaching with school-based teams will improve both RR fidelity and social network inclusion, while coaching with and without school-based teams will result in improved peer engagement and social skills.

DISCUSSION

Previous research has documented barriers to RR implementation at both the individual- (provider) and organization-level (school). Using multi-level implementation strategies such as coaching with school-based teams may address these barriers and support RR implementation in schools. Findings from this study may guide future efforts to scale up tailored implementation strategies for use in public school districts, with the ultimate goal of increasing intervention access and improving student outcomes.

TRIAL REGISTRATION

Name of the Registry: clinicaltrials.gov.

TRIAL REGISTRATION

Clinical Trials ID: NCT06559267 . Date of Registration: August 15, 2024. Prospectively registered.

Sustainment and adaptation of systems navigation and psychosocial counseling across HIV testing clinics in Vietnam: A qualitative assessment

Background

Few evidence-based interventions have been successfully scaled up and sustained long-term. Within an implementation trial testing strategies for scale-up of the Systems Navigation and Psychosocial Counseling (SNaP) intervention for people who inject drugs (PWID) with HIV across HIV testing clinics in Vietnam, we sought to assess if the implementation of SNaP was sustained after study support ended and to identify factors, including adaptations, that affected SNaP sustainment.

Method

Across all 42 SNaP clinics, we surveyed clinic staff at 6-10 months post-study completion to assess SNaP sustainment. We purposively selected six high and six low-sustaining clinics and conducted 31 in-depth interviews with clinic staff (n = 23) and clinic directors (n = 8). Interviews were coded and analyzed using thematic analysis informed by the Integrated Sustainability Framework. Matrices were used to compare themes across high and low-sustaining clinics.

Results

1/12 clinics sustained all of SNaP's core components, 2/12 would continue to sustain SNaP if they had new PWID patients, and the remainder did not fully sustain SNaP but continued conducting a modified version, including shorter or fewer SNaP sessions, tailoring SNaP to participants' specific needs, and conducting SNaP-style counseling for all clients. Facilitators of sustainment included leadership directives to clinic staff around SNaP sustainment, clinicians' belief in SNaP's effectiveness, and SNaP's perceived fit with clinic activities and mission. Major barriers to SNaP sustainment included lack of funding for PWID outreach activities, time, staff, training continuity, and systemic challenges with getting PWID into care, such as poverty and lack of transportation.

Conclusions

We identified the challenge of sustaining the SNaP intervention long-term, the ubiquity of intervention adaptations, and multi-level barriers and facilitators to intervention sustainment. These findings demonstrate the need for sustainment strategies and could inform trials of strategies to improve the longevity of effective HIV interventions for populations that are disproportionately affected by this epidemic.

Development of the psychopathological vulnerability index for screening at-risk youths: a Rasch model approach

Accumulating research on mental health emphasizes the general factor of psychopathology (p-factor) that unites various mental health issues. This study develops a psychopathological vulnerability assessment for youths, evaluating its psychometric properties and clinical utility. An umbrella review conceptualized multifactor psychopathological vulnerability, leading to a 57-item pool. A total of 11,224 individuals participated in this study. The resulting 22-item psychopathological vulnerability index (PVI) fitted the unidimensional Rasch model, demonstrating a person separation reliability of 0.78 and a Cronbach's alpha of 0.84. Cut-off points of 11 and 5, derived from latent class analysis, were used to distinguish vulnerable and high-protection populations. The PVI's concurrent and predictive hit rates ranged from 36.00% to 53.57% in clinical samples. The PVI concretized the vulnerability-stress model for identifying at-risk youths and may facilitate universal interventions by integrating the theoretical foundations of bifactor S-1 models with key symptoms from network models for theoretically grounded approaches.

Multi-level alignment processes in the sustainment of a youth substance use treatment model following a federal implementation initiative: A mixed method study

INTRODUCTION

Government agencies have identified evidence-based practice (EBP) dissemination as a pathway to high-quality behavioral health care for youth. However, gaps remain about how to best sustain EBPs in treatment organizations in the U.S., especially in resource-constrained settings like publicly-funded youth substance use services. One important, but understudied, determinant of EBP sustainment is alignment: the extent to which multi-level factors that influence sustainment processes and outcomes are congruent, consistent, and/or coordinated. This study examined the role of alignment in U.S. states' efforts to sustain the Adolescent Community Reinforcement Approach (A-CRA), an EBP for youth substance use disorders, during the COVID-19 pandemic.

METHODS

In this mixed methods study, the qualitative investigation preceded and informed the quantitative investigation. We interviewed state administrators and providers (i.e., supervisors and clinicians) from 15 states that had completed a federal A-CRA implementation grant; providers also completed surveys. The sample included 50 providers from 35 treatment organizations that reported sustaining A-CRA when the COVID-19 pandemic began, and 20 state administrators. In qualitative thematic analyses, we applied the EPIS (Exploration, Preparation, Implementation, Sustainment) framework to characterize alignment processes that interviewees described as influential on sustainment. We then used survey items to quantitatively explore the associations described in qualitative themes, using bivariate linear regressions.

RESULTS

At the time of interview, staff from 80 % of the treatment organizations (n = 28), reported sustaining A-CRA. Providers from both sustainer and non-sustainer organizations, as well as state administrators, described major sources of misalignment when state agencies ceased technical assistance post-grant, and because limited staff capacity conflicted with A-CRA's training model, which was perceived as time-intensive. Participants described the pandemic as exacerbating preexisting challenges, including capacity issues. Sustainer organizations reported seeking new funding to help sustain A-CRA. Quantitative associations between self-rated extent of sustainment and other survey items mostly followed the pattern predicted from the qualitative findings.

CONCLUSIONS

The COVID-19 pandemic amplified longstanding A-CRA sustainment challenges, but treatment organizations already successfully sustaining A-CRA pre-pandemic largely continued. There are missed opportunities for state-level actors to coordinate with providers on the shared goal of EBP sustainment. A greater focus on alignment processes in research and practice could help states and providers strengthen sustainability planning.

Randomized trial of the leadership and organizational change for implementation (LOCI) strategy in substance use treatment clinics

INTRODUCTION

Evidence-based practice (EBP) implementation represents a strategic change that requires alignment of leadership and support throughout organizations. Leadership and Organizational Change for Implementation (LOCI) is a multifaceted implementation strategy that aims to improve implementation leadership and climate within organizations through iterative cycles of leadership and climate assessment and feedback, leadership training and coaching, and strategic planning with upper-level leaders. This study tested the effects of LOCI on transformational and implementation leadership, implementation climate, implementation citizenship behavior, and EBP reach.

METHODS

A multiple cohort, cluster randomized trial tests the effect of LOCI in 60 clinics across nine behavioral health organizations in California and Arizona, USA. The study randomized clinics within organizations to either LOCI or a leadership training webinar control condition in three consecutive cohorts. Repeated web-based surveys of direct service providers (nLOCI = 201, nControl = 179) assessed leadership, implementation climate, and implementation citizenship over time. Multilevel autoregressive modeling was the primary statistical analysis such that providers (level-1) were nested within clinics (level-2). The study predicted between-condition differences at 4-, 8-, and 12-month follow-up assessments. Provider engagement in a fidelity monitoring process assessed reach of motivational interviewing (i.e., number of sessions recorded/submitted for fidelity coding). An independent sample t-test explored between condition differences in motivational interviewing reach.

RESULTS

Results indicated between condition differences at 4 months for implementation leadership, implementation climate, and implementation citizenship behavior such that greater improvements were evidenced in the LOCI condition compared to the control condition. Reach of MI was significantly greater in the LOCI vs control condition such that LOCI providers were significantly more likely to engage in the fidelity monitoring process (chi-square (1, n = 370) = 5.59, p = .018).

CONCLUSIONS

LOCI was developed based on organizational theories of strategic leadership and climate to affect organizational change processes that communicate that innovation implementation is expected, supported, and recognized as a value of the organization. The LOCI implementation strategy resulted in more positive hypothesized outcomes compared to the control condition. Organizational change strategies have utility for implementing health innovations in complex, multilevel contexts and for greater sustainment of facilitative leader behaviors, strategic implementation climate, and improved implementation outcomes.

TRIAL REGISTRATION

This study is registered with Clinicaltrials.gov gov (NCT03042832, 2 February 2017; retrospectively registered).

Study Protocol for a Cluster, Randomized, Controlled Community Effectiveness Trial of the Early Start Denver Model (ESDM) Compared to Community Early Behavioral Intervention (EBI) in Community Programs serving Young Autistic Children: Partnering for Autism: Learning more to improve Services (PALMS)

BACKGROUND

The rising number of children identified with autism has led to exponential growth in for-profit applied behavior analysis (ABA) agencies and the use of highly structured approaches that may not be developmentally appropriate for young children. Multiple clinical trials support naturalistic developmental behavior interventions (NDBIs) that integrate ABA and developmental science and are considered best practices for young autistic children. The Early Start Denver Model (ESDM) is a comprehensive NDBI shown to improve social communication outcomes for young autistic children in several controlled efficacy studies. However, effectiveness data regarding NDBI use in community-based agencies are limited.

METHODS

This study uses a community-partnered approach to test the effectiveness of ESDM compared to usual early behavioral intervention (EBI) for improving social communication and language in autistic children served by community agencies. This is a hybrid type 1 cluster-randomized controlled trial with 2 conditions: ESDM and EBI. In the intervention group, supervising providers will receive training in ESDM; in the control group, they will continue EBI as usual. We will enroll and randomize 100 supervisors (50 ESDM, 50 EBI) by region. Each supervisor enrolls 3 families of autistic children under age 5 (n = 300) and accompanying behavior technicians (n = 200). The primary outcome is child language and social communication at 6 and 12 months. Secondary outcomes include child adaptive behavior, caregiver use of ESDM strategies, and provider intervention fidelity. Child social motivation and caregiver fidelity will be tested as mediating variables. ESDM implementation determinants will be explored using mixed methods.

DISCUSSION

This study will contribute novel knowledge on ESDM effectiveness, the variables that mediate and moderate child outcomes, and engagement of its mechanisms in community use. We expect results from this trial to increase community availability of this model and access to high-quality intervention for young autistic children, especially those who depend on publicly funded intervention services. Understanding implementation determinants will aid scale-up of effective models within communities.

TRAIL REGISTRATION

Clinicaltrials.gov identifier number NCT06005285. Registered on August 21, 2023.

PROTOCOL VERSION

Issue date 6 August 2024; Protocol amendment number: 02.

Integrated Measure of PRogram Element SuStainability in Childcare Settings (IMPRESS-C): development and psychometric evaluation of a measure of sustainability determinants in the early childhood education and care setting

BACKGROUND

There is a need for valid and reliable measures of determinants of sustainability of public health interventions in early childhood education and care (ECEC) settings. This study aimed to develop and evaluate the psychometric and pragmatic properties of such a measure - the Integrated Measure of PRogram Element SuStainability in Childcare Settings (IMPRESS-C).

METHODS

We undertook a two-phase process guided by the COnsensus-based Standards for the selection of health status Measurement INstruments checklist (COSMIN) and Psychometric and Pragmatic Evidence Rating Scale (PAPERS). Phase 1 involved measure development; i.e., determining items and scales through an iterative process and assessment of face and content validity. Phase 2 involved the evaluation of psychometric and pragmatic properties. The 29-item measure completed by service executives (directors and nominated supervisors) was embedded in a larger survey from a national sample of Australian ECEC services assessing their implementation of nutrition and physical activity programs. Structural validity, concurrent validity, known groups validity, internal consistency, floor and ceiling effects, norms, and pragmatic qualities of the measure were assessed according to the PAPERS criteria.

RESULTS

The final measure contained 26 items, with respondents reporting how strongly they agreed or disagreed on a five-point Likert scale. Phase 1 assessments confirmed the relevance, and face and content validity of the scale. In Phase 2, we obtained 482 completed surveys, of which 84% (n = 405) completed the entire measure across 405 ECEC settings (one executive per service). Three of the four fit indices for the confirmatory factor analysis met the pre-specified criteria (SRMR = 0.056, CFI = 0.993, RMSEA = 0.067) indicating 'good' structural validity. The IMPRESS-C illustrated: 'good' internal consistency, with Cronbach's alpha values from 0.53 to 0.92; 'emerging' concurrent validity; 'poor' known groups validity; 'good' norms; and 'good' overall pragmatic qualities (cost, readability, length, and assessor burden).

CONCLUSIONS

The IMPRESS-C possesses strong psychometric and pragmatic qualities for assessing service executive-level perceptions of determinants influencing sustainment of public health interventions within ECEC settings. To achieve a full range of perspectives in this setting, future work should be directed to also develop and test measures of sustainability determinants at the implementer level (e.g., among individual educators and staff).

Scaling up a brief alcohol intervention to prevent HIV infection in Vietnam: a cluster randomized, implementation trial

BACKGROUND

Evidence-based interventions (EBIs) often address normative behaviors. If a behavior is also common among clinicians, they may be skeptical about the necessity or effectiveness of an EBI. Alternatively, clinicians' attitudes and behaviors may be misaligned, or they may lack the knowledge and self-efficacy to deliver the EBI. Several EBIs address unhealthy alcohol use, a common and often culturally acceptable behavior. But unhealthy alcohol use may be particularly harmful to people with HIV (PWH). Here, we present an implementation trial using an experiential implementation strategy to address clinicians' knowledge, attitudes, and behaviors. Clinicians receive the experiential intervention before they begin delivering an evidence-based brief alcohol intervention (BAI) to PWH with unhealthy alcohol use.

METHODS

Design: In this hybrid type 3 implementation-effectiveness cluster randomized controlled trial, ART clinics (n = 30) will be randomized 1:1 to facilitation, a flexible strategy to address implementation barriers, or facilitation plus the experiential brief alcohol intervention (EBAI). In the EBAI arm, clinicians, irrespective of their alcohol use, will be offered the BAI as experiential learning. EBAI will address clinicians' alcohol-related attitudes and behaviors and increase their knowledge and confidence to deliver the BAI.

PARTICIPANTS

ART clinic staff will be enrolled and assessed at pre-BAI training, post-BAI training, 3, 12, and 24 months. All PWH at the ART clinics who screen positive for unhealthy alcohol use will be offered the BAI. A subset of PWH (n = 810) will be enrolled and assessed at baseline, 3, and 12 months.

OUTCOMES

We will compare implementation outcomes (acceptability, fidelity, penetration, costs, and sustainability) and effectiveness outcomes (viral suppression and alcohol use) between the two arms. We will assess the impact of site-level characteristics on scaling-up the BAI. We will also evaluate how experiencing the BAI affected clinical staff's alcohol use and clinic-level alcohol expectations in the EBAI arm.

DISCUSSION

This trial contributes to implementation science by testing a novel strategy to implement a behavior change intervention in a setting in which clinicians themselves may engage in the behavior. Experiential learning may be useful to address normative and difficult to change lifestyle behaviors that contribute to chronic diseases.

TRIAL REGISTRATION

NCT06358885 (04/10/2024), https://clinicaltrials.gov/study/NCT06358885 .

Understanding pathways from implementation to sustainment: a longitudinal, mixed methods analysis of promising practices implemented in the Veterans Health Administration

BACKGROUND

The Veterans Health Administration (VHA) is the United States largest learning health system. The Diffusion of Excellence (DoE) program is a large-scale model of diffusion that identifies and diffuses evidence-informed practices across VHA. During the period of 2016-2021, 57 evidence-informed practices were implemented across 82 VHA facilities. This setting provides a unique opportunity to understand sustainment determinants and pathways. Our objective was to characterize the longitudinal pathways of practices as they transition from initial implementation to long-term sustainment at each facility.

METHODS

A longitudinal, mixed-methods evaluation of 82 VHA facilities. Eighty-two facility representatives, chosen by leadership as points-of-contact for 57 DoE practices, were eligible for post-implementation interviews and annual sustainment surveys. Primary outcomes (implementation, sustainment), and secondary outcomes (institutionalization, effectiveness, anticipated sustainment) at four time-points were collected. We performed descriptive statistics and directed content analysis using Hailemariam et al.'s factors influencing sustainment.

RESULTS

After approximately five years post-implementation (e.g., 2021 sustainment outcomes), of the 82 facilities, about one-third fully sustained their practice compared to one-third that did not fully sustain their practice because it was in a "liminal" stage (neither sustained nor discontinued) or permanently discontinued. The remaining one-third of facilities had missing 2021 sustainment outcomes. A higher percentage of facilities (70%) had inconsistent primary outcomes (changing over time) compared to facilities (30%) with consistent primary outcomes (same over time). Thirty-four percent of facilities with sustained practices reported resilience since they overcame implementation and sustainment barriers. Facilities with sustained practices reported more positive secondary outcomes compared to those that did not sustain their practice. Key factors facilitating practice sustainment included: demonstrating practice effectiveness/benefit, sufficient organizational leadership, sufficient workforce, and adaptation/alignment with local context. Key factors hindering practice sustainment included: insufficient workforce, not able to maintain practice fidelity/integrity, critical incidents related to the COVID-19 pandemic, organizational leadership did not support sustainment of practice, and no ongoing support.

CONCLUSIONS

We identified diverse pathways from implementation to sustainment, and our data underscore that initial implementation outcomes may not determine long-term sustainment outcomes. This longitudinal evaluation contributes to understanding impacts of the DoE program, including return on investment, achieving learning health system goals, and insights into achieving high-quality healthcare in VHA.

Effect of implementation strategies on the routine provision of antenatal care addressing smoking in pregnancy: study protocol for a non-randomised stepped-wedge cluster controlled trial

INTRODUCTION

Globally, guideline-recommended antenatal care for smoking cessation is not routinely delivered by antenatal care providers. Implementation strategies have been shown to improve the delivery of clinical practices across a variety of clinical services but there is an absence of evidence in applying such strategies to support improvements to antenatal care for smoking cessation in pregnancy. This study aims to determine the effectiveness and cost effectiveness of implementation strategies in increasing the routine provision of recommended antenatal care for smoking cessation in public maternity services.

METHODS AND ANALYSIS

A non-randomised stepped-wedge cluster-controlled trial will be conducted in maternity services across three health sectors in New South Wales, Australia. Implementation strategies including guidelines and procedures, reminders and prompts, leadership support, champions, training and monitoring and feedback will be delivered sequentially to each sector over 4 months. Primary outcome measures will be the proportion of: (1) pregnant women who report receiving a carbon monoxide breath test; (2) smokers or recent quitters who report receiving quit/relapse advice; and (3) smokers who report offer of help to quit smoking (Quitline referral or nicotine replacement therapy). Outcomes will be measured via cross-sectional telephone surveys with a random sample of women who attend antenatal appointments each week. Economic analyses will be undertaken to assess the cost effectiveness of the implementation intervention. Process measures including acceptability, adoption, fidelity and reach will be reported.

ETHICS AND DISSEMINATION

Ethics approval was obtained through the Hunter New England Human Research Ethics Committee (16/11/16/4.07; 16/10/19/5.15) and the Aboriginal Health and Medical Research Council (1236/16). Trial findings will be disseminated to health policy-makers and health services to inform best practice processes for effective guideline implementation. Findings will also be disseminated at scientific conferences and in peer-reviewed journals.

TRIAL REGISTRATION NUMBER

Australian New Zealand Clinical Trials Registry-ACTRN12622001010785.

Implementation science for cancer control: One center's experience addressing context, adaptation, equity, and sustainment

Implementation science (IS) has great potential to enhance the frequency, speed, and quality of the translation of evidence-based programs, policies, products, and guidelines into practice. Progress has been made, but with some notable exceptions, this promise has not been achieved for cancer prevention and control. We discuss five interrelated but conceptually distinct, crosscutting issues important to accelerate IS for cancer prevention and control and how our Colorado Implementation Science Center in Cancer Control (COISC3) addressed these issues. These needs and opportunities include more fully addressing changing, multi-level context; guiding rapid, iterative adaptations; evaluating innovative approaches to engagement and health equity; greater attention to costs and economic issues; and sustainability. We summarize conceptual issues; evaluation needs and capacity building activities and then provide examples of how our IS center addressed these five needs for cancer prevention and control. We discuss changes made to address priorities of (i) guiding adaptations of implementation strategies to address changing context and (ii) working on issues identified and prioritized by our primary care partners rather than the research team. We conclude with discussion of lessons learned, limitations, and directions for future research and practice in IS to enhance cancer prevention and control as well as translational behavioral medicine more generally.

Protocol for the process evaluation of an intervention to improve antenatal smoking cessation support (MOHMQuit) in maternity services in New South Wales, Australia

INTRODUCTION

Smoking cessation in pregnancy remains a public health priority. Our team used the Behaviour Change Wheel to develop the Midwives and Obstetricians Helping Mothers to Quit smoking (MOHMQuit) intervention with health system, leader (including managers and educators) and clinician components. MOHMQuit addresses a critical evidence to practice gap in the provision of smoking cessation support in antenatal care. It involves nine maternity services in New South Wales in a cluster randomised stepped-wedge controlled trial of effectiveness. This paper describes the design and rationale for the process evaluation of MOHMQuit. The process evaluation aims to assess to what extent and how MOHMQuit is being implemented (acceptability; adoption/uptake; appropriateness; feasibility; fidelity; penetration and sustainability), and the context in which it is implemented, in order to support further refinement of MOHMQuit throughout the trial, and aid understanding and interpretation of the results of the trial.

METHODS AND ANALYSIS

The process evaluation is an integral part of the stepped-wedge trial. Its design is underpinned by implementation science frameworks and adopts a mixed methods approach. Quantitative evidence from participating leaders and clinicians in our study will be used to produce individual and site-level descriptive statistics. Qualitative evidence of leaders' perceptions about the implementation will be collected using semistructured interviews and will be analysed descriptively within-site and thematically across the dataset. The process evaluation will also use publicly available data and observations from the research team implementing MOHMQuit, for example, training logs. These data will be synthesised to provide site-level as well as individual-level implementation outcomes.

ETHICS AND DISSEMINATION

The study received ethical approval from the Population Health Services Research Ethics Committee for NSW, Australia (Reference 2021/ETH00887). Results will be communicated via the study's steering committee and will also be published in peer-reviewed journals and presented at conferences.

TRIAL REGISTRATION NUMBER

Australian New Zealand Trials Registry ACTRN12622000167763. https://www.australianclinicaltrials.gov.au/anzctr/trial/ACTRN12622000167763.

Comparative effectiveness of implementation strategies for Accelerating Cervical Cancer Elimination through the integration of Screen-and-treat Services (ACCESS study): protocol for a cluster randomized hybrid type III trial in Nigeria

BACKGROUND

Despite the increased risk of cervical cancer (CC) among women living with HIV (WLHIV), CC screening and treatment (CCST) rates remain low in Africa. The integration of CCST services into established HIV programs in Africa can improve CC prevention and control. However, the paucity of evidence on effective implementation strategies (IS) has limited the success of integration in many countries. In this study, we seek to identify effective IS to enhance the integration of CCST services into existing HIV programs in Nigeria.

METHODS

Our proposed study has formative and experimental activities across the four phases of the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework. Through an implementation mapping conducted with stakeholders in the exploration phase, we identified a core package of IS (Core) and an enhanced package of IS (Core+) mostly selected from the Expert Recommendations for Implementing Change. In the preparation phase, we refined and tailored the Core and Core+ IS with the implementation resource teams for local appropriateness. In the implementation phase, we will conduct a cluster-randomized hybrid type III trial to assess the comparative effectiveness of Core versus Core+. HIV comprehensive treatment sites (k = 12) will be matched by region and randomized to Core or Core+ in the ratio of 1:1 stratified by region. In the sustainment phase, we will assess the sustainment of CCST at each site. The study outcomes will be assessed using RE-AIM: reach (screening rate), adoption (uptake of IS by study sites), IS fidelity (degree to which the IS occurred according to protocol), clinical intervention fidelity (delivery of CC screening, onsite treatment, and referral according to protocol), clinical effectiveness (posttreatment screen negative), and sustainment (continued integrated CCST service delivery). Additionally, we will descriptively explore potential mechanisms, including organizational readiness, implementation climate, CCST self-efficacy, and implementation intentions.

DISCUSSION

The assessment of IS to increase CCST rates is consistent with the global plan of eliminating CC as a public health threat by 2030. Our study will identify a set of evidence-based IS for low-income settings to integrate evidence-based CCST interventions into routine HIV care in order to improve the health and life expectancy of WLHIV.

TRIAL REGISTRATION

Prospectively registered on November 7, 2023, at ClinicalTrials.gov no. NCT06128304. https://classic.

CLINICALTRIALS

gov/ct2/show/study/NCT06128304.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 3: study protocol to evaluate sustainment in a hybrid type 2 effectiveness-implementation cluster-randomized trial

BACKGROUND

Although research on the implementation of evidence-based psychological treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase-the Sustainment Phase-of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial-the Implementation Phase and Train-the-Trainer Phase-TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes.

METHODS

Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data.

DISCUSSION

The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT05956678 . Registered on July 21, 2023.

Pilot evaluation of the Fiscal Mapping Process for sustainable financing of evidence-based youth mental health treatments: A comparative case study analysis

Background

Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies. Supports are needed that can guide service agencies in accessing sustainable funding for EBTs. We conducted a pilot evaluation of the Fiscal Mapping Process, an Excel-based strategic planning tool that helps service agency leaders identify and coordinate financing strategies for their EBT programs.

Method

Pilot testing of the Fiscal Mapping Process was completed with 10 youth mental health service agencies over a 12-month period, using trauma-focused cognitive-behavioral therapy or parent-child interaction therapy programs. Service agency representatives received initial training and monthly coaching in using the tool. We used case study methods to synthesize all available data (surveys, focus groups, coaching notes, document review) and contrast agency experiences to identify key findings through explanation building.

Results

Key evaluation findings related to the process and outcomes of using the Fiscal Mapping Process, as well as contextual influences. Process evaluation findings helped clarify the primary use case for the tool and identified the importance-and challenges-of engaging external collaborators. Outcome evaluation findings documented the impacts of the Fiscal Mapping Process on agency-reported sustainment capacities (strategic planning, funding stability), which fully explained reported improvements in outcomes (extent and likelihood)-although these impacts were incremental. Findings on contextual factors documented the influence of environmental and organizational capacities on engagement with the tool and concerns about equitable impacts, but also the view that the process could usefully generalize to other EBTs.

Conclusions

Our pilot evaluation of the Fiscal Mapping Process was promising. In future work, we plan to integrate the tool into EBT implementation initiatives and test its impact on long-term sustainment outcomes across various EBTs, while increasing attention to equity considerations.

Enhancing the implementation of provider-to-provider telehealth in rural and remote areas: A mixed methods study protocol

Background

Virtual healthcare solutions are proposed as a way to combat the inequity of access to healthcare in rural and remote areas, and to better support the front-line providers who work in these areas. Rural provider-to-provider telehealth (RPPT) connects rural and remote clinicians to a 'hub' of healthcare specialists who can increase access to emergency and specialised healthcare via an integrated model. Reported benefits for the place-based provider include enhanced knowledge, expanded professional development opportunities, improved scope of practice, and increased confidence in treating more complex cases. These reported benefits could have implications for supporting and futureproofing our health workforce in terms of productivity, burnout, recruitment, and retention.

Methods

The research uses an explanatory sequential mixed methods approach across multiple phases to evaluate the current implementation of Western Australia Country Health Service's (WACHS) Command Centre (CC) services and explore factors associated with their differential use. The primary population of interest and participants in this study are the place-based providers in country Western Australia (WA). Patient data constitutes the secondary population, informing the access and reach of CC services into country WA. Data collection will include service data, an online survey, and semi-structured interviews with the primary population. The data will be interpreted to inform evidence-based strategies and recommendations to improve the implementation and sustainment of RPPT.

Discussion

Innovative and sustained workforce models and solutions are needed globally. Virtual healthcare, including provider-to-provider models, demonstrate potential, especially in rural and remote areas, designed to increase access to specialised expertise for patients and to support the local workforce. This research will generate new data around behaviour, perceptions, and value from the WACHS rural and remote workforce about provider-to-provider telehealth, to explore the implementation and investigate strategies for the long-term sustainment of RPPT services.

Supporting adolescents' participation in muscle-strengthening physical activity: protocol for the 'Resistance Training for Teens' (RT4T) hybrid type III implementation-effectiveness trial

INTRODUCTION

In Australia, only 22% of male and 8% of female adolescents meet the muscle-strengthening physical activity guidelines, and few school-based interventions support participation in resistance training (RT). After promising findings from our effectiveness trial, we conducted a state-wide dissemination of the 'Resistance Training for Teens' (RT4T) intervention from 2015 to 2020. Despite high estimated reach, we found considerable variability in programme delivery and teachers reported numerous barriers to implementation. Supporting schools when they first adopt evidence-based programmes may strengthen programme fidelity, sustainability, and by extension, programme impact. However, the most effective implementation support model for RT4T is unclear.

OBJECTIVE

To compare the effects of three implementation support models on the reach (primary outcome), dose delivered, fidelity, sustainability, impact and cost of RT4T.

METHODS AND ANALYSIS

We will conduct a hybrid type III implementation-effectiveness trial involving grade 9 and 10 (aged 14-16 years) students from 90 secondary schools in New South Wales (NSW), Australia. Schools will be recruited across one cohort in 2023, stratified by school type, socioeconomic status and location, and randomised in a 1:1:1 ratio to receive one of the following levels of implementation support: (1) 'low' (training and resources), (2) 'moderate' (training and resources+external support) or 'high' (training and resources+external support+equipment). Training includes a teacher workshop related to RT4T programme content (theory and practical sessions) and the related resources. Additional support will be provided by trained project officers from five local health districts. Equipment will consist of a pack of semiportable RT equipment (ie, weighted bars, dumbbells, resistance bands and inverted pull up bar stands) valued at ~$A1000 per school. Study outcomes will be assessed at baseline (T0), 6 months (T1) and 18 months (T2). A range of quantitative (teacher logs, observations and teacher surveys) and qualitative (semistructured interviews with teachers) methods will be used to assess primary (reach) and secondary outcomes (dose delivered, fidelity, sustainability, impact and cost of RT4T). Quantitative analyses will use logistic mixed models for dichotomous outcomes, and ordinal or linear mixed effects regression models for continuous outcomes, with alpha levels set at p<0.025 for the outcomes and cost comparisons of the moderate and high support arms against the low support arm.

ETHICS AND DISSEMINATION

Ethics approval has been obtained from the University of Newcastle (H-2021-0418), the NSW Department of Education (SERAP:2022215), Hunter New England Human Research Ethics Committee (2023/ETH00052) and the Catholic Schools Office. The design, conduct and reporting will adhere to the Consolidated Standards of Reporting Trials statement, the Standards for Reporting Implementation Studies statement and the Template for Intervention Description and Replication checklist. Findings will be published in open access peer-reviewed journals, key stakeholders will be provided with a detailed report. We will support ongoing dissemination of RT4T in Australian schools via professional learning for teachers.

TRIAL REGISTRATION NUMBER

ACTRN12622000861752.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 3: Study protocol to evaluate sustainment in a hybrid type 2 effectiveness-implementation cluster-randomized trial

treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase - the Sustainment Phase - of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial - the Implementation Phase and Train-the-Trainer Phase - TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes.

Methods

Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data.

Discussion

The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change.

Trial Registration

ClinicalTrials.gov identifier: NCT05956678. Registered on July 21, 2023. https://classic.clinicaltrials.gov/ct2/show/NCT05956678?term=NCT05956678&draw=2&rank=1.

Ten years of implementation outcomes research: a scoping review

BACKGROUND

Proctor and colleagues' 2011 paper proposed a taxonomy of eight implementation outcomes and challenged the field to address a research agenda focused on conceptualization, measurement, and theory building. Ten years later, this paper maps the field's progress in implementation outcomes research. This scoping review describes how each implementation outcome has been studied, research designs and methods used, and the contexts and settings represented in the current literature. We also describe the role of implementation outcomes in relation to implementation strategies and other outcomes.

METHODS

Arksey and O'Malley's framework for conducting scoping reviews guided our methods. Using forward citation tracing, we identified all literature citing the 2011 paper. We conducted our search in the Web of Science (WOS) database and added citation alerts sent to the first author from the publisher for a 6-month period coinciding with the WOS citation search. This produced 1346 titles and abstracts. Initial abstract screening yielded 480 manuscripts, and full-text review yielded 400 manuscripts that met inclusion criteria (empirical assessment of at least one implementation outcome).

RESULTS

Slightly more than half (52.1%) of included manuscripts examined acceptability. Fidelity (39.3%), feasibility (38.6%), adoption (26.5%), and appropriateness (21.8%) were also commonly examined. Penetration (16.0%), sustainability (15.8%), and cost (7.8%) were less frequently examined. Thirty-two manuscripts examined implementation outcomes not included in the original taxonomy. Most studies took place in healthcare (45.8%) or behavioral health (22.5%) organizations. Two-thirds used observational designs. We found little evidence of progress in testing the relationships between implementation strategies and implementation outcomes, leaving us ill-prepared to know how to achieve implementation success. Moreover, few studies tested the impact of implementation outcomes on other important outcome types, such as service systems and improved individual or population health.

CONCLUSIONS

Our review presents a comprehensive snapshot of the research questions being addressed by existing implementation outcomes literature and reveals the need for rigorous, analytic research and tests of strategies for attaining implementation outcomes in the next 10 years of outcomes research.

Sustainment of diverse evidence-informed practices disseminated in the Veterans Health Administration (VHA): initial development and piloting of a pragmatic survey tool

BACKGROUND

There are challenges associated with measuring sustainment of evidence-informed practices (EIPs). First, the terms sustainability and sustainment are often falsely conflated: sustainability assesses the likelihood of an EIP being in use in the future while sustainment assesses the extent to which an EIP is (or is not) in use. Second, grant funding often ends before sustainment can be assessed. The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program is one of few large-scale models of diffusion; it seeks to identify and disseminate practices across the VHA system. The DoE sponsors "Shark Tank" competitions, in which leaders bid on the opportunity to implement a practice with approximately 6 months of implementation support. As part of an ongoing evaluation of the DoE, we sought to develop and pilot a pragmatic survey tool to assess sustainment of DoE practices.

METHODS

In June 2020, surveys were sent to 64 facilities that were part of the DoE evaluation. We began analysis by comparing alignment of quantitative and qualitative responses; some facility representatives reported in the open-text box of the survey that their practice was on a temporary hold due to COVID-19 but answered the primary outcome question differently. As a result, the team reclassified the primary outcome of these facilities to Sustained: Temporary COVID-Hold. Following this reclassification, the number and percent of facilities in each category was calculated. We used directed content analysis, guided by the Consolidated Framework for Implementation Research (CFIR), to analyze open-text box responses.

RESULTS

A representative from forty-one facilities (64%) completed the survey. Among responding facilities, 29/41 sustained their practice, 1/41 partially sustained their practice, 8/41 had not sustained their practice, and 3/41 had never implemented their practice. Sustainment rates increased between Cohorts 1-4.

CONCLUSIONS

The initial development and piloting of our pragmatic survey allowed us to assess sustainment of DoE practices. Planned updates to the survey will enable flexibility in assessing sustainment and its determinants at any phase after adoption. This assessment approach can flex with the longitudinal and dynamic nature of sustainment, including capturing nuances in outcomes when practices are on a temporary hold. If additional piloting illustrates the survey is useful, we plan to assess the reliability and validity of this measure for broader use in the field.

Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review

BACKGROUND

Sustainability is concerned with the long-term delivery and subsequent benefits of evidence-based interventions. To further this field, we require a strong understanding and thus measurement of sustainability and what impacts sustainability (i.e., sustainability determinants). This systematic review aimed to evaluate the quality and empirical application of measures of sustainability and sustainability determinants for use in clinical, public health, and community settings.

METHODS

Seven electronic databases, reference lists of relevant reviews, online repositories of implementation measures, and the grey literature were searched. Publications were included if they reported on the development, psychometric evaluation, or empirical use of a multi-item, quantitative measure of sustainability, or sustainability determinants. Eligibility was not restricted by language or date. Eligibility screening and data extraction were conducted independently by two members of the research team. Content coverage of each measure was assessed by mapping measure items to relevant constructs of sustainability and sustainability determinants. The pragmatic and psychometric properties of included measures was assessed using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The empirical use of each measure was descriptively analyzed.

RESULTS

A total of 32,782 articles were screened from the database search, of which 37 were eligible. An additional 186 publications were identified from the grey literature search. The 223 included articles represented 28 individual measures, of which two assessed sustainability as an outcome, 25 covered sustainability determinants and one explicitly assessed both. The psychometric and pragmatic quality was variable, with PAPERS scores ranging from 14 to 35, out of a possible 56 points. The Provider Report of Sustainment Scale had the highest PAPERS score and measured sustainability as an outcome. The School-wide Universal Behaviour Sustainability Index-School Teams had the highest PAPERS score (score=29) of the measure of sustainability determinants.

CONCLUSIONS

This review can be used to guide selection of the most psychometrically robust, pragmatic, and relevant measure of sustainability and sustainability determinants. It also highlights that future research is needed to improve the psychometric and pragmatic quality of current measures in this field.

TRIAL REGISTRATION

This review was prospectively registered with Research Registry (reviewregistry1097), March 2021.

Understanding How Newly Implemented Rehabilitation Best Practices Are Sustained: A Realist Review

OBJECTIVE

We conducted a realist review to understand how (mechanism) and in what circumstances (context) evidence-based practices are sustained in rehabilitation (outcome).

DATA SOURCES

MEDLINE, Embase, reference lists, and targeted websites.

STUDY SELECTION

Two independent reviewers calibrated study selection; then 1 reviewer screened all titles and abstracts, while the second reviewer screened a random 20%. We repeated this process for full texts. We included 115 documents representing 61 implementation projects (8.9% of identified documents). Included documents described implementation projects in which physical therapists, occupational therapists, and/or speech-language pathologists were the target users of an evidence-based practice.

DATA EXTRACTION

Two reviewers repeated the independent process described in study selection to extract basic study and sustainability characteristics as well as context, mechanism, outcome, and strategy text.

DATA SYNTHESIS

Using basic numerical analyses, we found that only 54% of evidence-based practices in rehabilitation are sustained. Furthermore, while authors who reported sustainability planning sustained the practice 94% of the time, sustainability planning in rehabilitation is rare (only reported 26% of the time). Extracted text was synthesized using the realist technique of inductive and deductive retroduction in which context, mechanism, outcome, and strategy text are combined into narrative explanations of how sustainability works. To inform these explanations, we applied normalization process theory and the theory of planned behavior. Collectively, the 52 identified narratives provide evidence for 3 patterns: (1) implementation and sustainability phases are interconnected, (2) continued use of the evidence-based practice can be interpreted as the ultimate sustainability outcome, and (3) intermediate sustainability outcomes (ie, fit/alignment, financial support, benefits, expertise) can become contextual features influencing other sustainability outcomes.

CONCLUSIONS

Implementation teams can use the narrative explanations generated in this review to optimize sustainability planning. This can sustain practice changes and improve quality of care and patient outcomes. Future research should seek to iteratively refine the proposed narrative explanations.

Iterative delivery of an implementation support package to increase and sustain the routine provision of antenatal care addressing alcohol consumption during pregnancy: study protocol for a stepped-wedge cluster trial

INTRODUCTION

Antenatal care addressing alcohol consumption during pregnancy is not routinely delivered in maternity services. Although a number of implementation trials have reported significant increases in such care, the majority of women still did not receive all recommended care elements, and improvements dissipated over time. This study aims to assess the effectiveness of an iteratively developed and delivered implementation support package in: (1) increasing the proportion of pregnant women who receive antenatal care addressing alcohol consumption and (2) sustaining the rate of care over time.

METHODS AND ANALYSIS

A stepped-wedge cluster trial will be conducted as a second phase of a previous trial. All public maternity services within three sectors of a local health district in Australia will receive an implementation support package that was developed based on an assessment of outcomes and learnings following the initial trial. The package will consist of evidence-based strategies to support increases in care provision (remind clinicians; facilitation; conduct educational meetings) and sustainment (develop a formal implementation blueprint; purposely re-examine the implementation; conduct ongoing training). Measurement of outcomes will occur via surveys with women who attend antenatal appointments each week. Primary outcomes will be the proportion of women who report being asked about alcohol consumption at subsequent antenatal appointments; and receiving complete care (advice and referral) relative to alcohol risk at initial and subsequent antenatal appointments. Economic and process evaluation measures will also be reported.

ETHICS AND DISSEMINATION

Ethical approval was obtained through the Hunter New England (16/11/16/4.07, 16/10/19/5.15) and University of Newcastle Human Research Ethics Committees (H-2017-0032, H-2016-0422) and the Aboriginal Health and Medical Research Council (1236/16). Trial findings will be disseminated to health service decision makers to inform the feasibility of conducting additional cycles to further improve antenatal care addressing alcohol consumption as well as at scientific conferences and in peer-reviewed journals.

TRIAL REGISTRATION NUMBER

Australian and New Zealand Clinical Trials Registry (ACTRN12622000295741).

Comparing two federal financing strategies on penetration and sustainment of the adolescent community reinforcement approach for substance use disorders: protocol for a mixed-method study

BACKGROUND

Sustained, widespread availability of evidence-based practices (EBPs) is essential to address the public health and societal impacts of adolescent substance use disorders (SUD). There remains a particularly significant need to identify effective financing strategies, which secure and direct financial resources to support the costs associated with EBP implementation and sustainment. This protocol describes a new project comparing two types of U.S. federal grant mechanisms (i.e., a type of financing strategy), which supported the implementation of the Adolescent Community Reinforcement Approach (A-CRA) EBP for SUD, through either organization-focused or state-focused granting of funds. The Exploration-Preparation-Implementation-Sustainment (EPIS) framework will guide our study aims, hypotheses, and selection of measures.

METHOD

We will employ a longitudinal, mixed-method (i.e., web surveys, semi-structured interviews, document review, focus groups, administrative data), quasi-experimental design to compare the grant types' outcomes and examine theoretically informed mediators and moderators. Aim 1 will examine the proportion of eligible clinicians certified in A-CRA with adequate fidelity levels (i.e., penetration outcomes) at the end of grant funding. Aim 2 will examine the sustainment of A-CRA up to 5 years post-funding, using a 10-element composite measure of treatment delivery and supervision activities. We will integrate the new data collected from state-focused grant recipients (~85 organizations in 19 states) with previously collected data from organization-focused grant recipients (Hunter et al., Implement Sci 9:104, 2014) (82 organizations in 26 states) for analysis. We will also use sensitivity analyses to characterize the effects of observed and unobserved secular trends in our quasi-experimental design. Finally, aim 3 will use comparative case study methods (integrating diverse quantitative and qualitative measures) to identify and disseminate policy implications about the roles of state- and organization-focused federal grants in efforts to promote adolescent SUD EBP implementation and sustainment.

DISCUSSION

The proposed research will have direct, practical implications for behavioral health administrators, policymakers, implementation experts, and the public. It will offer new knowledge that can directly inform financing strategies to support large-scale, sustained EBP delivery in behavioral health-while advancing implementation science through the use of novel methods to study financing strategies and sustainment.

Coordination of sustainable financing for evidence-based youth mental health treatments: protocol for development and evaluation of the fiscal mapping process

BACKGROUND

Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health and economic impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies (hereafter, "service agencies"). Strategic planning tools are needed that can guide these service agencies in their coordination of sustainable funding for EBTs. This protocol describes a mixed-methods research project designed to (1) develop and (2) evaluate our novel fiscal mapping process that guides strategic planning efforts to finance the sustainment of EBTs in youth mental health services.

METHOD

Participants will be 48 expert stakeholder participants, including representatives from ten service agencies and their partners from funding agencies (various public and private sources) and intermediary organizations (which provide guidance and support on the delivery of specific EBTs). Aim 1 is to develop the fiscal mapping process: a multi-step, structured tool that guides service agencies in selecting the optimal combination of strategies for financing their EBT sustainment efforts. We will adapt the fiscal mapping process from an established intervention mapping process and will incorporate an existing compilation of 23 financing strategies. We will then engage participants in a modified Delphi exercise to achieve consensus on the fiscal mapping process steps and gather information that can inform the selection of strategies. Aim 2 is to evaluate preliminary impacts of the fiscal mapping process on service agencies' EBT sustainment capacities (i.e., structures and processes that support sustainment) and outcomes (e.g., intentions to sustain). The ten agencies will pilot test the fiscal mapping process. We will evaluate how the fiscal mapping process impacts EBT sustainment capacities and outcomes using a comparative case study approach, incorporating data from focus groups and document review. After pilot testing, the stakeholder participants will conceptualize the process and outcomes of fiscal mapping in a participatory modeling exercise to help inform future use and evaluation of the tool.

DISCUSSION

This project will generate the fiscal mapping process, which will facilitate the coordination of an array of financing strategies to sustain EBTs in community youth mental health services. This tool will promote the sustainment of youth-focused EBTs.