Measuring Implementation Strategy Fidelity in HealthyHearts NYC: A Complex Intervention Using Practice Facilitation in Primary Care

Fidelity to the ACT SMART Toolkit: an instrumental case study of implementation strategy fidelity

BACKGROUND

Evidence-based practices (EBPs) are shown to improve a variety of outcomes for autistic children. However, EBPs often are mis-implemented or not implemented in community-based settings where many autistic children receive usual care services. A blended implementation process and capacity-building implementation strategy, developed to facilitate the adoption and implementation of EBPs for autism spectrum disorder (ASD) in community-based settings, is the Autism Community Toolkit: Systems to Measure and Adopt Research-based Treatments (ACT SMART Toolkit). Based on an adapted Exploration, Adoption decision, Preparation, Implementation, Sustainment (EPIS) Framework, the multi-phased ACT SMART Toolkit is comprised of (a) implementation facilitation, (b) agency-based implementation teams, and (c) a web-based interface. In this instrumental case study, we developed and utilized a method to evaluate fidelity to the ACT SMART Toolkit. This study responds to the need for implementation strategy fidelity evaluation methods and may provide evidence supporting the use of the ACT SMART Toolkit.

METHODS

We used an instrumental case study approach to assess fidelity to the ACT SMART Toolkit during its pilot study with six ASD community agencies located in southern California. We assessed adherence, dose, and implementation team responsiveness for each phase and activity of the toolkit at both an aggregate and individual agency level.

RESULTS

Overall, we found that adherence, dose, and implementation team responsiveness to the ACT SMART Toolkit were high, with some variability by EPIS phase and specific activity as well as by ASD community agency. At the aggregate level, adherence and dose were rated notably lowest during the preparation phase of the toolkit, which is a more activity-intensive phase of the toolkit.

CONCLUSIONS

This evaluation of fidelity to the ACT SMART Toolkit, utilizing an instrumental case study design, demonstrated the potential for the strategy to be used with fidelity in ASD community-based agencies. Findings related to the variability of implementation strategy fidelity in the present study may also inform future adaptations to the toolkit and point to broader trends of how implementation strategy fidelity may vary by content and context.

Study design and protocol of a stepped wedge cluster randomized trial using a practical implementation strategy as a model for hypertension-HIV integration - the MAP-IT trial

BACKGROUND

As people living with HIV (PLWH) experience earlier and more pronounced onset of noncommunicable diseases (NCDs), advancing integrated care networks and models in low-resource-high-need settings is critical. Leveraging current health system initiatives and addressing gaps in treatment for PLWH, we report our approach using a late-stage (T4) implementation research study to test the adoption and sustainability of a proven-effective implementation strategy which has been minimally applied in low-resource settings for the integration of hypertension control into HIV treatment. We detail our protocol for the Managing Hypertension Among People Living with HIV: an Integrated Model (MAP-IT) trial, which uses a stepped wedge cluster randomized trial (SW-CRT) design to evaluate the effectiveness of practice facilitation on the adoption of a hypertension treatment program for PLWH receiving care at primary healthcare centers (PHCs) in Akwa Ibom State, Nigeria.

DESIGN

In partnership with the Nigerian Federal Ministry of Health (FMOH) and community organizations, the MAP-IT trial takes place in 30 PHCs. The i-PARiHS framework guided pre-implementation needs assessment. The RE-AIM framework will guide post-implementation activities to evaluate the effect of practice facilitation on the adoption, implementation fidelity, and sustainability of a hypertension program, as well as blood pressure (BP) control. Using a SW-CRT design, PHCs sequentially crossover from the hypertension program only (usual care) to hypertension plus practice facilitation (experimental condition). PHCs will recruit and enroll an average of 28-32 patients to reach a maximum of 960 PLWH participants with uncontrolled hypertension who will be followed longitudinally for BP outcomes.

DISCUSSION

Given the need for integrated NCD-HIV care platforms in low-resource settings, MAP-IT will underscore the challenges and opportunities for integrating hypertension treatment into HIV care, particularly concerning adoption and sustainability. The evaluation of our integration approach will also highlight the potential impact of a health systems strengthening approach on BP control among PLWH.

TRIAL REGISTRATION

Clinicaltrials.gov ( NCT05031819 ). Registered on 2nd September 2021.

Leveraging Implementation Science for Cardiovascular Health Equity: A Scientific Statement From the American Heart Association

Reducing cardiovascular disease disparities will require a concerted, focused effort to better adopt evidence-based interventions, in particular, those that address social determinants of health, in historically marginalized populations (ie, communities excluded on the basis of social identifiers like race, ethnicity, and social class and subject to inequitable distribution of social, economic, physical, and psychological resources). Implementation science is centered around stakeholder engagement and, by virtue of its reliance on theoretical frameworks, is custom built for addressing research-to-practice gaps. However, little guidance exists for how best to leverage implementation science to promote cardiovascular health equity. This American Heart Association scientific statement was commissioned to define implementation science with a cardiovascular health equity lens and to evaluate implementation research that targets cardiovascular inequities. We provide a 4-step roadmap and checklist with critical equity considerations for selecting/adapting evidence-based practices, assessing barriers and facilitators to implementation, selecting/using/adapting implementation strategies, and evaluating implementation success. Informed by our roadmap, we examine several organizational, community, policy, and multisetting interventions and implementation strategies developed to reduce cardiovascular disparities. We highlight gaps in implementation science research to date aimed at achieving cardiovascular health equity, including lack of stakeholder engagement, rigorous mixed methods, and equity-informed theoretical frameworks. We provide several key suggestions, including the need for improved conceptualization and inclusion of social and structural determinants of health in implementation science, and the use of adaptive, hybrid effectiveness designs. In addition, we call for more rigorous examination of multilevel interventions and implementation strategies with the greatest potential for reducing both primary and secondary cardiovascular disparities.

The case for prioritizing implementation strategy fidelity measurement: benefits and challenges

Implementation strategies are systematic approaches to improve the uptake and sustainability of evidence-based interventions. They frequently focus on changing provider behavior through the provision of interventions such as training, coaching, and audit-and-feedback. Implementation strategies often impact intermediate behavioral outcomes like provider guideline adherence, in turn improving patient outcomes. Fidelity of implementation strategy delivery is defined as the extent to which an implementation strategy is carried out as it was designed. Implementation strategy fidelity measurement is under-developed and under-reported, with the quality of reporting decreasing over time. Benefits of fidelity measurement include the exploration of the extent to which observed effects are moderated by fidelity, and critical information about Type-III research errors, or the likelihood that null findings result from implementation strategy fidelity failure. Reviews of implementation strategy efficacy often report wide variation across studies, commonly calling for increased implementation strategy fidelity measurement to help explain variations. Despite the methodological benefits of rigorous fidelity measurement, implementation researchers face multi-level challenges and complexities. Challenges include the measurement of a complex variable, multiple data collection modalities with varying precision and costs, and the need for fidelity measurement to change in-step with adaptations. In this position paper, we weigh these costs and benefits and ultimately contend that implementation strategy fidelity measurement and reporting should be improved in trials of implementation strategies. We offer pragmatic solutions for researchers to make immediate improvements like the use of mixed methods or innovative data collection and analysis techniques, the inclusion of implementation strategy fidelity assessment in reporting guidelines, and the staged development of fidelity tools across the evolution of an implementation strategy. We also call for additional research into the barriers and facilitators of implementation strategy fidelity measurement to further clarify the best path forward.

Measuring implementation fidelity in a cluster-randomized pragmatic trial: development and use of a quantitative multi-component approach

BACKGROUND

In pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music & MemorySM (M&M), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs).

METHODS

Intervention NHs (N = 27) implemented M&M using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core M&M components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to M&M at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity).

RESULTS

The 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was τb = 0.11 (p = 0.59) and with Duration was τb = - 0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer's unit, suggesting face validity.

CONCLUSIONS

Guided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach.

TRIAL REGISTRATION

Clinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://clinicaltrials.gov/ct2/show/NCT03821844 .

A practice facilitation-guided intervention in primary care settings to reduce cardiovascular disease risk: a cost analysis

Background

A stepped-wedge, cluster randomized controlled trial assessed the effectiveness of practice facilitation (PF) for adoption of guidelines for prevention and treatment of cardiovascular disease risk factors. This study estimated the associated cost of PF for guideline adoption in small, private primary care practices.

Methods

The cost analysis included categories for start-up costs, intervention costs, and practice staff costs for the implemented PF-guided intervention. We estimated the total 1-year costs to operate the program and calculated the mean and range of the cost-per-practice by quarter of the intervention. We estimated the lower and upper bounds for all salary expenses, rounding to the nearest $100.

Results

Total 1-year intervention costs for all 261 practices ranged from $7,900,000 to $10,200,000, with program and practice salaries comprising $6,600,000–$8,400,000 of the total. Start-up costs were a small proportion (3%) of the total 1-year costs. Excluding start-up costs, quarter 1 cost-per-practice was the most expensive at $20,400–$26,700, and quarter 4 was the least expensive at about $10,000. Practice staff time (compared with program staff time) was the majority of the staffing costs at 75–84%.

Conclusions

The PF strategy costs approximately $10,000 per practice per quarter for program and practice costs, once implemented and running at highest efficiency. Whether this program is “worth it” to the decision-maker depends on the relative costs and effectiveness of their other options for improving cardiovascular risk reduction.

Trial registration

This study is retrospectively registered on January 5, 2016, at www.clinicaltrials.gov as NCT02646488.