Use of the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework to guide iterative adaptations: Applications, lessons learned, and future directions

Evaluation of the Implementation of the FLOW Program for Increasing Access to Mental Health Care

The FLOW program was designed to facilitate appropriate and safe transitions of patients from specialty mental health (SMH) to primary care (PC) as a method of improving access and reducing appointment burden on veterans who have improved or remitted. In this study, the team evaluated the implementation of FLOW across nine Veterans Affairs (VA) sites using a mixed-methods evaluation in a cluster-randomized stepped wedge trial design. Outcome assessments used data from VA databases, dashboards, and semi-structured interviews and were guided by the Reach, Adoption, Effectiveness, Implementation, and Maintenance (RE-AIM) framework. Across the sites, mean level reach was 1.36% of all specialty mental health patients transitioned to primary care (standard deviation [SD] = 1.05). Mean provider adoption was 10.2% (SD = 8.3%). Approximately 75% of veterans were fully satisfied with their transition and reported shared decision-making in the decision to transition. Rates of transitions did not decrease over the 6-month maintenance period following implementation. These data suggest that FLOW can be successfully implemented and maintained, although there was wide variation in implementation across sites. Future research should examine how to support sites that struggle with implementation.

Perspectives of healthcare professionals on training for quantitative G6PD testing during implementation of tafenoquine in Brazil (QualiTRuST Study)

Effective radical cure of Plasmodium vivax malaria is essential for malaria elimination in Brazil. P. vivax radical cure requires administration of a schizonticide, such as chloroquine, plus an 8-aminoquinoline. However, 8-aminoquinolines cause hemolysis in individuals with glucose-6-phosphate dehydrogenase (G6PD) deficiency, requiring prior screening to exclude those at risk. Brazil is pioneering the implementation of tafenoquine, a single-dose 8-aminoquinoline indicated for P. vivax patients with >70% of normal G6PD activity. Tafenoquine implementation in Manaus and Porto Velho, two municipalities located in the western Brazilian Amazon, included comprehensive training of healthcare professionals (HCPs) on point-of-care quantitative G6PD testing and a new treatment algorithm for P. vivax radical cure incorporating tafenoquine. Training was initially provided to higher-level facilities (phase one) and later adapted for primary care units (phase two). This study analyzed HCP experiences during training and implementation and identified barriers and facilitators. In-depth interviews and focus discussion groups were conducted 30 days after each training for a purposive random sample of 115 HCPs. Thematic analysis was employed using MAXQDA software, analyzing data through inductive and deductive coding. Analysis showed that following the initial training for higher-level facilities, some HCPs did not feel confident performing quantitative G6PD testing and prescribing the tafenoquine regimen. Modifications to the training in phase two resulted in an improvement in understanding the implementation process of the G6PD test and tafenoquine, as well as in the knowledge acquired by HCPs. Additionally, knowledge gaps were addressed through in situ training, peer communication via a messaging app, and educational materials. Training supported effective deployment of the new tools in Manaus and Porto Velho and increased awareness of the need for pharmacovigilance. A training approach for nationwide implementation of these tools was devised. Implementing quantitative G6PD testing and tafenoquine represents a significant shift in P. vivax malaria case management. Consistent engagement with HCPs is needed to overcome challenges in fully integrating these tools within the Brazilian health system.

Leveraging systems science and design thinking to advance implementation science: moving toward a solution-oriented paradigm

Many public health challenges are characterized by complexity that reflects the dynamic systems in which they occur. Such systems involve multiple interdependent factors, actors, and sectors that influence health, and are a primary driver of challenges of insufficient implementation, sustainment, and scale of evidence-based public health interventions. Implementation science frameworks have been developed to help embed evidence-based interventions in diverse settings and identify key factors that facilitate or hinder implementation. These frameworks are largely static in that they do not explain the nature and dynamics of interrelationships among the identified determinants, nor how those determinants might change over time. Furthermore, most implementation science frameworks are top-down, deterministic, and linear, leaving critical gaps in understanding of both how to intervene on determinants of successful implementation and how to scale evidence-based solutions. Design thinking and systems science offer methods for transforming this problem-oriented paradigm into one that is solution-oriented. This article describes these two approaches and how they can be integrated into implementation science strategies to promote implementation, sustainment, and scaling of public health innovation, ultimately resulting in transformative systems changes that improve population health.

Aligning the planning, development, and implementation of complex interventions to local contexts with an equity focus: application of the PRISM/RE-AIM Framework

For the fields of implementation science and health equity, understanding and being responsive to local contexts is of utmost importance to better inform the development, implementation, and evaluation of healthcare and public health interventions to increase their uptake and sustainment. Contexts are multi-level and include political, historical, economic, and social factors that influence health, as well as organizational characteristics, reflecting the richness of members' views, resources, values, and needs. Poor alignment between solutions and those contextual characteristics could have an impact on inequities. The PRISM (Practical Robust Implementation and Sustainability Model) is a context-based implementation science framework that incorporates RE-AIM outcomes (Reach, Effectiveness, Adoption, Implementation, Maintenance) and offers guidance to researchers, practitioners, and their patient and community partners on how to conceptualize, assess, and address contextual domains with a focus on health equity. Drawing from systems thinking, participatory engagement, and health equity principles, this commentary expands on previous work to 1) offer a novel perspective on how to align an intervention's core functions and forms with the PRISM's contextual domains, and 2) foster an ongoing and iterative engagement process with diverse partners throughout the research and practice process using a co-creation approach. We recommend intervention-to-context alignment through iterative cycles. To that end, we present the RE-AIM Framework's 'outcomes cascade' to illustrate touch points of opportunity and gaps within and across each of the five RE-AIM outcomes to illustrate 'where things go wrong'. We present a case study to illustrate and offer recommendations for research and practice efforts to increase contextual responsiveness, and enhance alignment with context before, during, and after implementation efforts and to ensure equity is being addressed. We strive to make a conceptual contribution to advance the field of pragmatic research and implementation of evidence-based practices through the application of the contextually-based PRISM framework with a focus on health equity.

Evaluation of the Effectiveness of a Bilingual Nutrition Education Program in Partnership with a Mobile Health Unit

There are limited reports of community-based nutrition education with culinary instruction that measure biomarkers, particularly in low-income and underrepresented minority populations. Teaching kitchens have been proposed as a strategy to address social determinants of health, combining nutrition education, culinary demonstration, and skill building. The purpose of this paper is to report on the development, implementation, and evaluation of Journey to Health, a program designed for community implementation using the RE-AIM planning and evaluation framework. Reach and effectiveness were the primary outcomes. Regarding reach, 507 individuals registered for the program, 310 participants attended at least one nutrition class, 110 participants completed at least two biometric screens, and 96 participants attended at least two health coaching appointments. Participants who engaged in Journey to Health realized significant improvements in body mass index, blood pressure, and triglycerides. For higher risk participants, we additionally saw significant improvements in total and LDL cholesterol. Regarding dietary intake, we observed a significant increase in cups of fruit and a decrease in sugar sweetened beverages consumed per day. Our findings suggest that Journey to Health may improve selected biometrics and health behaviors in low-income and underrepresented minority participants.

Adaptation of the brainwriting premortem technique to inform the co-creation of COVID-19 testing strategies in underserved communities in South San Diego

INTRODUCTION

Meaningful engagement of partners in co-creating and refining health-related programs can increase the initial uptake, sustained implementation, broad reach, and effectiveness of these programs. This is especially important for underserved communities where resources are limited and need to be prioritized. Brainwriting premortem is a novel qualitative approach to partner engagement that combines the strengths of individual idea generation with the concept of premortem exercise that addresses failure points prior to the implementation of new programs.

METHODS

An adapted form of brainwriting premortem was used to inform iterative refinements to a COVID-19 testing program at a Federally Qualified Health Center (FQHC) in San Diego. Patients and providers from the FQHC participated in interviews at two time points (early- and mid-implementation of the program). Interview data were transcribed, translated, and analyzed using a rapid qualitative approach. Key themes and sub-themes were identified and used to inform refinements to the program.

RESULTS

A total of 11 patients (7 Spanish- and 4 English-speaking) and 8 providers participated in the brainwriting premortem interviews. Key themes related to possible reasons for COVID-19 testing program failure: advertising/sharing information; access to testing; handling of test results; staff and patient safety; patient beliefs and views regarding the SARS-CoV-2 virus; and COVID-19 testing options offered. Proposed solutions were offered for the key failures except for patient beliefs and views regarding the SARS-CoV-2 virus. Additional solutions offered were related to education, physical operations, and recruitment strategies. Real-time changes to the program flow and components were made in response to 7 suggestions from patients and 11 from providers. Changes related to the process of returning results were the most common, and included sending results via email with distinct workflows based on the test result.

CONCLUSION

The implementation of the adapted brainwriting premortem technique allowed us to incorporate the perspective of key partners in the delivery and iterative refinement of the COVID-19 testing program. This was an effective tool in the context of an FQHC and can be a promising and approach to incorporate iterative input from patients and providers to ensure successful program implementation. Future studies, particularly those requiring rapid response to public health emergencies, should consider the use of this technique.

Systematic symptom management in the IMPACT Consortium: rationale and design for 3 effectiveness-implementation trials

Cancer and its treatment produce deleterious symptoms across the phases of care. Poorly controlled symptoms negatively affect quality of life and result in increased health-care needs and hospitalization. The Improving the Management of symPtoms during And following Cancer Treatment (IMPACT) Consortium was created to develop 3 large-scale, systematic symptom management systems, deployed through electronic health record platforms, and to test them in pragmatic, randomized, hybrid effectiveness and implementation trials. Here, we describe the IMPACT Consortium's conceptual framework, its organizational components, and plans for evaluation. The study designs and lessons learned are highlighted in the context of disruptions related to the COVID-19 pandemic.

Utilizing PRISM and RE-AIM to implement and evaluate the Rural Telementoring Training Center (RTTC) for health care workforce development in rural communities

Introduction

Amid rural health worker shortages and hospital closures, it is imperative to build and maintain the local workforce. Telementoring (TM) or technology-enabled mentoring, is a tool for improving health care quality and access by increasing workforce capacity and support. The national Rural Telementoring Training Center (RTTC) was developed to compile and disseminate TM best practices by delivering free training, tools, and technical assistance to support the implementation, sustainability, and evaluation of new and current TM programs for rural health workers. This paper details how the Practical, Robust Implementation and Sustainability Model (PRISM) was used to understand the context that shaped implementation as well as how Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) was concurrently applied to frame outcomes.

Methods

The RTTC has three implementation strategies: outreach, training and technical assistance (TTA), and a Quality Measure Toolkit. Ongoing periodic reflections with the RTTC team, informed by PRISM, were collected, as were RE-AIM outcomes. Central to this design was the continuous review of incoming data in team meetings to inform programmatic changes by identifying challenges and applying modifications to strategies in real time.

Results

Major implementation changes discussed during reflections included providing timely and relevant messaging through various platforms, streamlining and customizing a TTA approach, and offering different options for accessing the Toolkit. The outreach strategy resulted in high Reach across the US, with over 300 organizations contacted. The effectiveness of the RTTC was demonstrated by counts of people engaging with outreach (ex. over 8,300 impressions on LinkedIn), the website (over 6,400 views), and e-bursts (33% open rate). Moreover, there were 32 TTA requests and 70 people accessing the Toolkit. Adoption was demonstrated by 27 people participating in TTA and 14 individuals utilizing the Toolkit.

Discussion

The integration of PRISM and RE-AIM frameworks promoted a holistic implementation and evaluation plan. Using PRISM, the RTTC team was able to reflect on the implementation strategies through the lens of contextual factors and make rapid programmatic changes within team meetings. That process resulted in outcomes framed by RE-AIM. The integration of two frameworks in tandem provided an adaptive and comprehensive approach to implementing a large-scale, national program.

The development and evaluation of an intervention to promote the uptake of preventive tasks by occupational physicians targeting work-related mental health problems: protocol for the IM-PROmPt-study

OBJECTIVE

Work-related mental health problems are a major and growing public and occupational health issue. Although prevention of work-related disease is a central task in the work of occupational physicians, implementation of preventive tasks can still improve. The aim of this paper is to present the development of an intervention to support occupational physicians in the execution of preventive tasks and a protocol for its evaluation.

METHODS

An intervention to support occupational physicians has been developed making use of the implementation mapping protocol. The intervention was based on barriers and facilitators for the execution of preventive tasks, input from stakeholders, and evidence-based strategies from literature.

RESULTS

The intervention consists of three peer group supervision meetings directed to preventive tasks. During these meetings, occupational physicians will receive materials and will use goal-setting to formulate their own action plans. The IM-PROmPt-study (Implementation of PReventive tasks by Occupational Physicians) is a two-armed cluster randomized controlled trial, comparing peer group supervision directed to the implementation of preventive tasks for occupational physicians with usual peer group supervision. The evaluation will include an effect and process evaluation to examine if the intervention is successful in supporting OPs to implement preventive activities, specifically aimed to prevent work-related mental health problems.

DISCUSSION

The intervention is expected to lead to more knowledge and awareness of the value of prevention among OPs, anticipated to lead to both organizational and individual gains.

TRIAL REGISTRATION

ISRCTN registry; ISRCTN15394765. Registered on 27 June 2023.

Advancing Rehabilitation Paradigms for Older Adults in Skilled Nursing Facilities: An Effectiveness-Implementation Hybrid Type 1 Clinical Trial Protocol

OBJECTIVE

Skilled nursing facility rehabilitation is commonly required to address hospital-associated deconditioning among older adults with medical complexity. In skilled nursing facilities, standard-of-care rehabilitation focuses on low-intensity interventions, which are not designed to sufficiently challenge skeletal muscle and impart functional improvements. In contrast, a high-intensity resistance training approach (IntenSive Therapeutic Rehabilitation for Older NursinG homE Residents; i-STRONGER) in a single-site pilot study resulted in better physical function among patients in skilled nursing facilities. To extend this work, an effectiveness-implementation hybrid type 1 design, cluster-randomized trial will be conducted to compare patient outcomes between 16 skilled nursing facilities utilizing i-STRONGER principles and 16 Usual Care sites.

METHODS

Clinicians at i-STRONGER sites will be trained to deliver i-STRONGER as a standard of care using an implementation package that includes a clinician training program. Clinicians at Usual Care sites will continue to provide usual care. Posttraining, changes in physical performance (eg, gait speed, Short Physical Performance Battery scores) from patients' admission to discharge will be collected over a period of 12 months. The Reach, Effectiveness, Adoption, Implementation, and Maintenance framework will be used to evaluate i-STRONGER effectiveness and factors underlying successful i-STRONGER implementation. Effectiveness will be evaluated by comparing changes in physical function between study arms. Reach (proportion of patients treated with i-STRONGER), adoption (proportion of clinicians utilizing i-STRONGER), implementation (i-STRONGER fidelity), and maintenance (i-STRONGER sustainment) will be concurrently quantified and informed by clinician surveys and focus groups.

IMPACT

This effectiveness-implementation hybrid type 1 cluster-randomized trial has the potential to shift rehabilitation care paradigms in a nationwide network of skilled nursing facilities, resulting in improved patient outcomes and functional independence. Furthermore, evaluation of the facilitators of, and barriers to, implementation of i-STRONGER in real-world clinical settings will critically inform future work evaluating and implementing best rehabilitation practices in skilled nursing facilities.

Applying an equity lens to assess context and implementation in public health and health services research and practice using the PRISM framework

Dissemination and implementation science seeks to enhance the uptake, successful implementation, and sustainment of evidence-based programs and policies. While a focus on health equity is implicit in many efforts to increase access to and coverage of evidence-based programs and policies, most implementation frameworks and models do not explicitly address it. Disparities may in fact be increased by emphasizing high intensity interventions or ease of delivery over meeting need within the population, addressing deep-rooted structural inequities, and adapting to local context and priorities. PRISM (Practical, Robust Implementation and Sustainability Model), the contextual expansion of the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework has several elements that address health equity, but these have not been explicated, integrated, or illustrated in one place. We present guidance for applying PRISM with an equity lens across its four context domains (external environment; multi-level perspectives on the intervention; characteristics of implementers and intended audience; and the implementation and sustainability infrastructure-as well as the five RE-AIM outcome dimensions. We then present an example with health equity considerations and discuss issues of representation and participation, representativeness and the importance of ongoing, iterative assessment of dynamic context and structural drivers of inequity. We also elaborate on the importance of a continuous process that requires addressing community priorities and responding to capacity and infrastructure needs and changes. We conclude with research and practice recommendations for applying PRISM with an increased emphasis on equity.