Evaluating the Application of the RE-AIM Planning and Evaluation Framework: An Updated Systematic Review and Exploration of Pragmatic Application

Planning, implementation, evaluation, and sustainment of digital health interventions for adolescent substance use prevention: a systematic review of influencing factors based on the RE-AIM framework

School-based health interventions can help prevent adolescent substance use, but their success depends on various factors affecting planning and evaluation. This study examines the factors that facilitate or hinder the success of school-based digital health interventions. A systematic review was conducted in September-October 2024, following PRISMA guidelines, using PubMed, Scopus, Web of Science, CINAHL, and PsycINFO databases. From a total of 2530 studies, after removing duplicates and screening titles and abstracts, 51 full-text articles were assessed and 12 met the inclusion criteria. Data were extracted using an adapted tool for systematic reviews applying the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework. The methodological quality was assessed with Joanna Briggs Institute tools. The studies included cluster-randomized controlled trials, quasi-experimental designs, and one cross-sectional study, primarily targeting adolescents aged 11.9-17.4 years, with sample sizes ranging from 90 to nearly 5000 participants. Most interventions were implemented in schools in the USA, Europe, and Taiwan, with balanced gender distribution. Many studies reported on reach and effectiveness in reducing substance use, particularly alcohol. However, adoption and implementation factors influencing school participation were less frequently addressed, and maintenance was not mentioned. To maximize impact, future efforts should focus on enhancing sustainment and participation by leveraging emerging technologies for personalization.

Methadone Patient Access to Collaborative Treatment: Protocol for a Pilot and a Randomized Controlled Trial to Establish Feasibility of Adoption and Impact on Methadone Treatment Delivery and Patient Outcomes

BACKGROUND

Access to methadone treatment can reduce opioid overdose death by up to 60%, but US patient outcomes are suboptimal. Federally allowed methadone treatment accommodations during the COVID-19 public health emergency were not widely adopted. It is likely that staff-level characteristics such as trauma symptoms influence the adoption of treatment innovation.

OBJECTIVE

Methadone Patient Access to Collaborative Treatment (MPACT) is a 2-phased project (pilot and field trial) to develop and test a staff-level, multimodal intervention to increase staff adoption of low-barrier, patient-centered methadone treatment practices and ultimately improve treatment retention and patient outcomes.

METHODS

A pilot and national trial will measure implementation feasibility, acceptability, and effects of the MPACT intervention on treatment practice change, clinic culture, patient retention, and patient posttraumatic stress symptoms (PTSS). The pilot will be a single-arm 5.5-month pilot study of MPACT conducted in 2 Arizona methadone treatment clinics (rural and urban) among 100 patients and 22 staff. The national trial will be a 20-month cluster randomized trial conducted among 30 clinics, 600 patients (20 per clinic), and 480 staff (18 per clinic). Data will be gathered by staff and patient surveys and patient chart review. The primary study outcome is increased patient methadone treatment retention measured as (1) time to first treatment interruption from study enrollment; (2) active in treatment at enrollment, day 30, 60, 90, and 120; and (3) continuous days in treatment during the study period. Secondary study outcomes include reductions in vicarious trauma and PTSS among enrolled opioid treatment program staff and PTSS among enrolled patients.

RESULTS

The pilot study was funded by the National Institute on Drug Abuse (award R61DA059889, funded September 2023), and the field trial will be funded under the associated R33 mechanism in September 2025. The pilot study was completed in March 2025. The randomized controlled trial will begin in December 2025. Both the pilot and trial have been approved by the University of Arizona Human Subjects Protection Program and have been registered with the clinical trials network.

CONCLUSIONS

The MPACT study will provide a foundation for an evidence-based, staff-level intervention aimed at improving patient retention in methadone treatment. Future studies should examine the individual components of MPACT to determine their differential contributions to the primary outcome of patient methadone treatment retention and to secondary outcomes of staff and patient reduction in stress symptoms.

TRIAL REGISTRATION

ClinicalTrials.gov NCT06513728; https://clinicaltrials.gov/study/NCT06513728 and ClinicalTrials.gov NCT06556602; https://clinicaltrials.gov/study/NCT06556602.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/69829.

Applying an Implementation Framework to the Dissemination of a School-Based Child Sexual Abuse Prevention Program

Since the 1980s, school-based child sexual abuse (CSA) prevention programs have been the prevailing prevention strategy in the United States. Despite demonstrated effectiveness, there is a lack of infrastructure and educational policy ensuring all students receive these programs. A pragmatic application of the RE-AIM implementation framework, this study provides an overview of a multi-county implementation effort of the school-based CSA prevention program, Safe Touches. Implementation efforts across five counties in a Mid-Atlantic state are described at three levels: organizational (school districts), child, and program facilitator. Children's CSA-related knowledge was measured at four time points: pre-workshop, immediately post-workshop, and then 6 and 12 months post-workshop. Facilitators completed an anonymous survey post-implementation. Over the course of one and a half academic years, Safe Touches was implemented in 718 public school districts, reaching in total 14,235 second-grade students. Students' significantly increased knowledge from pre- to post-workshop and gains were maintained at 6 and 12 months (ps <.001). A total of 29 disclosures of maltreatment were made by students to facilitators during or after the workshop. Facilitators generally adopted Safe Touches and attested to the feasibility and benefits of its large-scale implementation as well as the negligible negative impacts for children. When implemented systematically, school-based CSA prevention is able to reach a high number of students, effectively increase CSA-related knowledge, and facilitates disclosures. To maximize the potential public health impact, it is suggested that state funds be allocated to support the implementation of such programs as part of standard education costs.

Consumer and community involvement: implementation research for impact (CCIRI) - implementing evidence-based patient and public involvement across health and medical research in Australia - a mixed methods protocol

BACKGROUND

Within Australia, there is increasing recognition of the importance and value of patient and public involvement, or consumer and community involvement (CCI), in health and medical research and healthcare improvement. Despite this and policy mandates, there has been little behavioural and systems change to embed and support CCI. Often, this is relegated to tokenistic gestures rather than authentic partnerships. The aim of this national project is to use evidence-generated knowledge co-led by consumers, community members, researchers and clinicians to embed CCI in health and medical research and healthcare improvement.

METHODS

The Consolidated Framework for Implementation Research and the Learning Health System framework underpin the project to facilitate an iterative process to change systems and individual behaviour towards adoption of CCI in health and medical research and healthcare improvement. Key stakeholder groups include research translation centres, funding bodies, clinicians, professional staff involved in healthcare improvement, researchers and consumers and community members. To understand the attitudes, knowledge, beliefs, behaviours, system barriers and facilitators around CCI in health and medical research and healthcare improvement, semi-structured interviews and surveys will be conducted across key stakeholder groups. Template analysis and descriptive statistics will be used to report data from the national survey respectively prior to triangulation of data. Findings will be reported through traditional scientific outputs such as conference presentations and peer-reviewed publications. Other anticipated outputs include policy briefs, organizational implementation toolkits and resources and a co-designed digital knowledge hub to support individuals with implementation and scale up across stakeholders.

DISCUSSION

This study will build on considerable stakeholder engagement and prior priority-setting and includes broad and detailed consideration of perspectives from diverse stakeholders at a national level. Robust methodological frameworks, co-design and partnership with stakeholders will be used to inform resources to support systems change to facilitate CCI in health and medical research and healthcare improvement. Ethics approval was obtained from Monash Health (RES-23-0000-275Q).

Using RE-AIM Framework to Evaluate Recovery Opioid Overdose Team Plus: A Peer-Led Post-overdose Quick Response Team

Peer recovery coaches utilize their lived experiences to support overdose survivors, a role gaining prominence across communities. A convergent mixed methods design, informed by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework, was used to evaluate the Recovery Opioid Overdose Team Plus (ROOT +), through an iterative evaluation using web-based surveys and qualitative interviews. Reach: Over 27 months, ROOT + responded to 83% of suspected overdose referrals (n = 607) and engaged with 41% of survivors (n = 217) and 7% of survivors' family/friends (n = 38). Effectiveness: Among those initially engaged with ROOT +, 36% of survivors remained engaged, entered treatment, or were in recovery at 90 days post-overdose (n = 77). Adoption: First responders completed 77% of ROOT + referrals (n = 468). Implementation: Barriers included lack of awareness of ROOT + , working phones, and access to treatment from community partner interviews (n = 15). Maintenance: Adaptations to ROOT + were made to facilitate implementation. Peer-led teams are promising models to engage with overdose survivors.

Using RE-AIM to Assess Infant Early Childhood Mental Health Practices in Classrooms Serving Children with and Without Disabilities

Background/Objectives: High-quality inclusive education is important for promoting the positive development of children with disabilities in early childhood care and education (ECCE) settings. However, ECCE teachers may not have the knowledge and skills to manage challenging behaviors in young children, especially those with disabilities. Infant and Early childhood mental health consultation (IECMHC) is one mechanism to support the professional development of teachers. This study explored the impact of an evidence-based IECMHC program, Jump Start Plus COVID Support (JS+CS), on outcomes for teachers in classrooms including children with disabilities. Methods: Utilizing a RE-AIM framework, we examined the extent that JS+CS impacted teacher outcomes related to classroom practice and teacher attitudes after the initial intervention period. In addition, we examined the extent that the classroom children with disability ratio moderated the impact of the intervention on teacher outcomes. Using a cluster randomized controlled trial in a sample of 138 racially and ethnically diverse teachers in 31 ECCE centers, we examined the reach, effectiveness, adoption, and implementation of JS+CS. Results: The results indicate that the JS+CS program shows promise as an intervention to support ECCE teachers working in classrooms with children with disabilities, particularly in improving teacher safety practices, behavior management skills, and resiliency coping. In addition, the program was adopted equally in classrooms that served children with and without disabilities. Conclusions: This is a unique contribution to the literature given that no previous IECMHC programs have examined adoption in classrooms serving children with disabilities. Further investigation is needed to determine the specific factors that impact program implementation considering that this study was conducted during various phases of the COVID-19 pandemic.

Evaluating the use of systems thinking methods in healthcare: a RE-AIM analysis of AcciMap and Net-HARMS

There are increasing calls for the application of systems ergonomics methods in healthcare, although evidence for their utility and uptake is limited. In this study, 67 Australian healthcare workers participated in a six-month longitudinal study where they were trained to apply the AcciMap adverse event analysis and Net-HARMS risk assessment methods. Data were gathered in line with the RE-AIM (Reach, Efficacy, Adoption, Implementation, and Maintenance) evaluation framework, including rates of organisational uptake and method validity, perceived workload, usability, and barriers and facilitators to use in practice. Overall RE-AIM ratings for AcciMap were relatively high, and more moderate for Net-HARMS. Time constraints was the most frequently identified barrier to the use of both methods in practice, while there was more organisational resistance to Net-HARMS uptake. Facilitators for the use of both methods include providing quality training and mentorship, additional time and software resources, and dedicated job roles.

Bridging the Gap: The Need to Implement Dietary Guidance to Address Cardiovascular Health

Cardiovascular disease (CVD) is the leading cause of death in the U.S. and globally. Research demonstrates that diet is a leading contributor to the development of CVD, its prevention and management, and the overall promotion of cardiovascular health. This article describes the current state of the evidence, including research on the DASH and Mediterranean diets to promote cardiovascular health and prevent CVD. The article suggests approaches to implement evidence-based diets and federal dietary guidance to promote the adoption and integration of these interventions in both community and clinical settings. It highlights the current U.S. federal interest in "Food is Medicine" and its importance in addressing diet-related chronic diseases and promoting cardiovascular health.

Designing multimedia patient education materials for adolescent idiopathic scoliosis: A protocol for a feasibility randomized controlled trial of patient education videos

Triple-masked three-armed feasibility parallel randomized controlled trial. Multimedia patient education materials are increasingly used in healthcare. While much research focuses on optimising their scientific content, research is equally needed to optimise design and implementation. This study aims to determine the feasibility of a study examining how the implementation of scientific advice on design affects patient outcomes. Participants aged 10-18 with radiographically confirmed adolescent idiopathic scoliosis will be recruited from community settings in Ireland and randomized into usual care or receiving multimedia educational videos with or without evidence-informed design principles. Participants will be masked in the two video intervention arms, as will the therapist sending the educational videos. Outcomes will include the number of participants recruited and randomized, the number analysed post-intervention and at week eight, and the outcomes for baseline, post-intervention, and week 8. Adverse events will also be reported. This feasibility randomized controlled trial will offer insight into the feasibility of implementing advice from the literature in designing a trial of multimedia patient education materials for a population with adolescent idiopathic scoliosis. Trial registration: Clinical Trail: Trial is registered on ClinicalTrials.gov as NCT06090344.

Building a sharable literature collection to advance the science and practice of implementation facilitation

Background

Implementation science seeks to produce generalizable knowledge on strategies that promote the adoption and sustained use of evidence-based innovations. Literature reviews on specific implementation strategies can help us understand how they are conceptualized and applied, synthesize findings, and identify knowledge gaps. Although rigorous literature reviews can advance scientific knowledge and facilitate theory development, they are time-consuming and costly to produce. Improving the efficiency of literature review processes and reducing redundancy of effort is especially important for this rapidly developing field. We sought to amass relevant literature on one increasingly used evidence-based strategy, implementation facilitation (IF), as a publicly available resource.

Methods

We conducted a rigorous systematic search of PubMed, CINAHL, and Web of Science citation databases for peer-reviewed, English-language articles with "facilitation" and a combination of other terms published from January 1996 to December 2021. We searched bibliographies of articles published from 1996 to 2015 and identified articles during the full text review that reported on the same study. Two authors screened 3,168 abstracts. After establishing inter-rater reliability, they individually conducted full-text review of 786 relevant articles. A multidisciplinary team of investigators provided recommendations for preparing and disseminating the literature collection.

Findings

The literature collection is comprised of 510 articles. It includes 277 empirical studies of IF and 77 other articles, including conceptual/theoretical articles, literature reviews, debate papers and descriptions of large-scale clinical initiatives. Over half of the articles were published between 2017 and 2021. The collection is publicly available as an Excel file and as an xml file that can be imported into reference management software.

Conclusion

We created a publicly accessible collection of literature about the application of IF to implement evidence-based innovations in healthcare. The comprehensiveness of this collection has the potential to maximize efficiency and minimize redundancy in scientific inquiry about this strategy. Scientists and practitioners can use the collection to more rapidly identify developments in the application of IF and to investigate a wide range of compelling questions on its use within and across different healthcare disciplines/settings, countries, and payer systems. We offer several examples of how this collection has already been used.

Bridges and Mechanisms: Integrating Systems Science Thinking into Implementation Research

We present a detailed argument for how to integrate, or bridge, systems science thinking and methods with implementation science. We start by showing how fundamental systems science principles of structure, dynamics, information, and utility are relevant for implementation science. Then we examine the need for implementation science to develop and apply richer theories of complex systems. This can be accomplished by emphasizing a causal mechanisms approach. Identifying causal mechanisms focuses on the "cogs and gears" of public health, clinical, and organizational interventions. A mechanisms approach focuses on how a specific strategy will produce the implementation outcome. We show how connecting systems science to implementation science opens new opportunities for examining and addressing social determinants of health and conducting equitable and ethical implementation research. Finally, we present case studies illustrating successful applications of systems science within implementation science in community health policy, tobacco control, health care access, and breast cancer screening.

RE-AIM for rural health innovations: perceptions of (mis) alignment between the RE-AIM framework and evaluation reporting in the Department of Veterans Affairs Enterprise-Wide Initiatives program

Background

The Department of Veterans Affairs (VA) Office of Rural Health (ORH) supports national VA program offices' efforts to expand health care to rural Veterans through its Enterprise-Wide Initiatives (EWIs) program. In 2017, ORH selected Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM), an implementation science framework, to structure the EWI evaluation and reporting process. As part of its mandate to improve EWI program evaluation, the Center for the Evaluation of Enterprise-Wide Initiatives conducted a qualitative evaluation to better understand EWI team' perceptions of, and barriers and facilitators to, the EWI evaluation process.

Methods

We conducted 43 semi-structured interviews with 48 team members (e.g., evaluators, program office leads, and field-based leads) representing 21 EWIs from April-December 2020. Questions focused on participants' experiences using strategies targeting each RE-AIM dimension. Interviews were inductively analyzed in MAXQDA. We also systematically reviewed 51 FY19-FY20 EWI annual reports to identify trends in misapplications of RE-AIM.

Results

Participants had differing levels of experience with RE-AIM. While participants understood ORH's rationale for selecting a common framework to structure evaluations, the perceived misalignment between RE-AIM and EWIs' work emerged as an important theme. Concerns centered around 3 sub-themes: (1) (Mis)Alignment with RE-AIM Dimensions, (2) (Mis)Alignment between RE-AIM and the EWI, and (3) (Mis)Alignment with RE-AIM vs. other Theories, Models, or Frameworks. Participants described challenges differentiating between and operationalizing dimensions in unique contexts. Participants also had misconceptions about RE-AIM and its relevance to their work, e.g., that it was meant for established programs and did not capture aspects of initiative planning, adaptations, or sustainability. Less commonly, participants shared alternative models or frameworks to RE-AIM. Despite criticisms, many participants found RE-AIM useful, cited training as important to understanding its application, and identified additional training as a future need.

Discussion

The selection of a shared implementation science framework can be beneficial, but also challenging when applied to diverse initiatives or contexts. Our findings suggest that establishing a common understanding, operationalizing framework dimensions for specific programs, and assessing training needs may better equip partners to integrate a shared framework into their evaluations.

Artificial intelligence predictive analytics in heart failure: results of the pilot phase of a pragmatic randomized clinical trial

OBJECTIVES

We conducted an implementation planning process during the pilot phase of a pragmatic trial, which tests an intervention guided by artificial intelligence (AI) analytics sourced from noninvasive monitoring data in heart failure patients (LINK-HF2).

MATERIALS AND METHODS

A mixed-method analysis was conducted at 2 pilot sites. Interviews were conducted with 12 of 27 enrolled patients and with 13 participating clinicians. iPARIHS constructs were used for interview construction to identify workflow, communication patterns, and clinician's beliefs. Interviews were transcribed and analyzed using inductive coding protocols to identify key themes. Behavioral response data from the AI-generated notifications were collected.

RESULTS

Clinicians responded to notifications within 24 hours in 95% of instances, with 26.7% resulting in clinical action. Four implementation themes emerged: (1) High anticipatory expectations for reliable patient communications, reduced patient burden, and less proactive provider monitoring. (2) The AI notifications required a differential and tailored balance of trust and action advice related to role. (3) Clinic experience with other home-based programs influenced utilization. (4) Responding to notifications involved significant effort, including electronic health record (EHR) review, patient contact, and consultation with other clinicians.

DISCUSSION

Clinician's use of AI data is a function of beliefs regarding the trustworthiness and usefulness of the data, the degree of autonomy in professional roles, and the cognitive effort involved.

CONCLUSION

The implementation planning analysis guided development of strategies that addressed communication technology, patient education, and EHR integration to reduce clinician and patient burden in the subsequent main randomized phase of the trial. Our results provide important insights into the unique implications of implementing AI analytics into clinical workflow.

An integrative review of parent education approaches in sport: Considerations for program planning and evaluation

In recent years, there has been an increase in the delivery and evaluation of parent education programs within youth sport. Subsequently, some recent reviews of these programs have been conducted. However, one consistent issue across many of the programs and associated review papers is the lack of an appropriate evaluation framework to guide the planning or associated reporting of the outcomes of the interventions. This has limited understanding of the overall impact of sport parenting interventions. Thus, the purposes of the current study were as follows: (a) to identify commonalities in the reporting and evaluation of parent education programs; (b) to identify gaps in the reporting and evaluation of parent education programs; (c) to draw these insights together to provide suggestions regarding how the RE-AIM could be used to enhance planning and evaluation of evidence-based programs for parent education in sport. Specifically, utilizing the RE-AIM framework to provide insights into pertinent evaluation metrics, this integrative review aimed to identify commonalities and gaps in the reporting of parent education programs. The RE-AIM framework considers the essential elements to assess the external and internal validity of interventions through five dimensions: Reach, Effectiveness, Adoption, Implementation, and Maintenance (Am J Public Health. 1999;89(9):1322-1327). Subsequently, the review aimed to provide suggestions regarding strategies to enhance the planning and evaluation of evidence-based programs for parent education in sport. Overall, the analysis demonstrated that most studies presented some pertinent evaluation information related to the RE-AIM framework, such as the number of participants and contacts made, the measures used, and the program level. However, the studies also lacked information on participant exclusion criteria, the method used to select the delivery agent (e.g., parents engaged in the program), and cost measures. Overall, the current study identified various areas where programs could be enhanced, specifically related to reporting procedural elements (e.g., program design, target population, and costs) pertaining to the implementation of parent education programs.

Implementation of the Baby-Friendly Hospital Initiative in Mexico: a systematic literature review using the RE-AIM framework

The Baby-Friendly Hospital Initiative (BFHI) is a global strategy to encourage health facilities to promote, support, and protect breastfeeding by implementing a package of policies and practices known as the Ten Steps to Successful Breastfeeding. Prior studies have found that implementing the Ten Steps has a positive impact on breastfeeding outcomes. Yet, little is known about the implementation of the Ten Steps in Mexico. The objective of this study was to conduct a systematic review to evaluate the reach, efficacy/effectiveness, adoption, implementation, and maintenance of the Ten Steps in Mexico, using the RE-AIM framework. The systematic literature review included studies published in English or Spanish without date restrictions. Two of the authors coded each of the articles through a harmonized data extraction tool, and group meetings were used to discuss any discrepancies. The reviewed data were managed in the Rayyan platform. The risk of study bias was assessed through the Johanna Briggs Institute critical appraisal checklists. Of the 1,123 articles initially identified, 6 met the review inclusion criteria. None of the articles evaluated the reach and maintenance of the Ten Steps. The articles identified major gaps in the implementation of the Ten Steps. Most of the articles had important limitations in terms of their quality. In Mexico, it is necessary to rethink the BFHI and employ multiple strategies to improve implementation of the Ten Steps, including developing transparent BFHI monitoring mechanisms that produce data on implementation and that are publicly available, as well as investing in implementation research and evaluation to generate strong evidence to support the adoption and efficient maintenance of the Ten Steps in health facilities in Mexico. When properly implemented, BFHI becomes central to promote, protect, and support breastfeeding. Therefore, it is essential for Mexico to position BFHI as a top priority of the country's public policy agenda.

Systematic Review Registration

identifier: CRD42021248118.

Benefits and challenges in the use of RE-AIM for evaluation of a national social work staffing program in the veterans health administration

Background

In the Department of Veterans Affairs (VA) Veterans Health Administration (VHA), social workers embedded in primary care teams address social and emotional needs that are associated with health outcomes. The mission of the National Social Work PACT Staffing Program is to improve access to social work services for rural Veterans by supporting additional social work staffing in VA medical centers serving rural areas.

Methods

We obtained data from the VA corporate data warehouse on Veterans' characteristics and health care use from 2016 to 2022 for all Veterans who received primary care at a Veterans Affairs Medical Center (VAMC) or associated clinic that received funding from the program. We evaluated the program according to RE-AIM constructs as follows: Reach [total number of Veterans who engaged with PACT social work and representativeness with regard to race, rural residence, chronic conditions and health behaviors, and hospital and emergency department (ED) use in the previous 12 months]; Effectiveness (impact of the program on key health care use outcomes which include hospitalizations, emergency department visits, and palliative care); Adoption (number of VA medical centers and outpatient clinics serving rural Veterans that have participated in the program, and number and representativeness of sites eligible for program participation that have not yet received funding); Implementation (adherence to standardized note templates), and Maintenance (permanent social work positions created by the program and continued technical support).

Results

In 2022, the program engaged with 30,982 Veterans, 65% of whom lived in rural areas. The program increased social work encounters, reduce hospital and emergency department use, and increase use of palliative care services among Veterans. Key elements of implementation include proactive outreach to Veterans with high-risk indicators and assessment for social risk factors using standardized, national note templates. In terms of maintenance, the program continues to provide data and technical assistance to 23 sites and has created 171 permanent social work positions.

Conclusions and implications

The Social Work PACT Staffing Program demonstrates positive outcomes and program sustainment. The RE-AIM framework was a useful tool to evaluate the program, but additional adaption was needed to fit the program's needs.

RE-AIM applied to a primary care workforce training for rural providers and nurses: the Department of Veterans Affairs' Rural Women's Health Mini-Residency

Introduction

Application of the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to evaluate workforce education and training programs targeting clinical health care staff has received relatively little attention. This paper aims to contribute to this area with RE-AIM findings from a women's health-focused workforce training program implemented by the U.S. Department of Veterans Affairs (VA). Over the past two decades, the rapid expansion of the women Veteran population in VA has necessitated a quick response to meet clinical demand. To address this health care need, the VA Offices of Rural Health (ORH) and Women's Health (OWH) partnered to deploy a primary care workforce development initiative for Rural Providers and Nurses-the Rural Women's Health Mini-Residency (Rural WH-MR)-to train VA clinicians in rural locations in skills for the care of women Veterans. Here we assess the applicability of RE-AIM as an evaluation framework in this context.

Methods

We evaluated the Rural WH-MR, relying on a primarily quantitative approach, rooted in RE-AIM. It included longitudinal and cross-sectional measurements from multiple quantitative and qualitative data sources to develop selected metrics. Data collection instruments consisted of pre-, post-, and follow-up training surveys, course evaluations, existing VA databases, and implementation reports. We developed metrics for and assessed each RE-AIM component by combining data from multiple instruments and then triangulating findings.

Results

Results from the Rural WH-MR program for fiscal years 2018-2020 indicate that RE-AIM provides an instructive evaluation framework for a rural workforce training program, particularly in eliciting clarity between measures of Reach vs. Adoption and focusing attention on both provider- and patient-level outcomes.

Discussion

We describe evaluation metric development and barriers to and facilitators of utilizing RE-AIM as an evaluation framework for a provider- and nurse-facing intervention such as this workforce training program. We also reflect upon RE-AIM benefits for highlighting process and outcomes indicators of a training program's success and lessons learned for evaluating rural workforce development innovations. Several of our observations have implications for training and evaluation approaches in rural areas with more limited access to health care services.

Coming together for something good: recommendations from a scoping review for dissemination and implementation science to improve indigenous substance use disorder treatment

Introduction

Dissemination and Implementation (D&I) science is growing among Indigenous communities. Indigenous communities are adapting and implementing evidence-based treatments for substance use disorders (SUD) to fit the needs of their communities. D&I science offers frameworks, models, and theories to increase implementation success, but research is needed to center Indigenous knowledge, enhancing D&I so that it is more applicable within Indigenous contexts. In this scoping review, we examined the current state of D&I science for SUD interventions among Indigenous communities and identified best-practice SUD implementation approaches.

Methods

PubMed and PsycINFO databases were queried for articles written in English, published in the United States, Canada, Australia, and New Zealand. We included key search terms for Indigenous populations and 35 content keywords. We categorized the data using the adapted and extended Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework that emphasizes equity and sustainability. RE-AIM has also been used as a primary model to consistently identify implementation outcomes.

Results

Twenty articles were identified from the original unduplicated count of over 24,000. Over half the articles discussed processes related to Reach, Adoption, and Implementation. Effectiveness was discussed by 50% of the studies (n = 10), with 25% of the articles discussing Maintenance/sustainability (n = 4). Findings also highlighted the importance of the application of each RE-AIM domain for meaningful, well-defined community-engaged approaches.

Conclusion

Finding indicated a need to prioritize Indigenous methods to culturally center, re-align and adapt Western treatments and frameworks to increase health equity and improve SUD treatment outcomes. Utility in the use of the modified RE-AIM and the continued modification for Indigenous communities was also noted.

A Mixed Methods Evaluation of a Pilot Multidisciplinary Breathlessness Support Service

Breathlessness support services have demonstrated benefits for breathlessness mastery, quality of life and psychosocial outcomes for people living with breathlessness. However, these services have predominantly been implemented in hospital and home care contexts. This study aims to evaluate the adaptation and implementation of a hospice-based outpatient Multidisciplinary Breathlessness Support Service (MBSS) in Ireland. A sequential explanatory mixed methods design guided this study. People with chronic breathlessness participated in longitudinal questionnaires (n = 10), medical record audit (n = 14) and a post-discharge interview (n = 8). Caregivers (n = 1) and healthcare professionals involved in referral to (n = 2) and delivery of (n = 3) the MBSS participated in a cross-sectional interview. Quantitative and qualitative data were integrated deductively via the pillar integration process, guided by the RE-AIM framework. Integration of mixed methods data enhanced understanding of factors influencing the reach, adoption, implementation and maintenance of the MBSS, and the potential outcomes that were most meaningful for service users. Potential threats to the sustainability of the MBSS related to potential preconceptions of hospice care, the lack of standardized discharge pathways from the service and access to primary care services to sustain pharmacological interventions. This study suggests that an adapted multidisciplinary breathlessness support intervention is feasible and acceptable in a hospice context. However, to ensure optimal reach and maintenance of the intervention, activities are required to ensure that misconceptions about the setting do not influence willingness to accept referral to MBSS services and integration of services is needed to enable consistency in referral and discharge processes.

Implementing barbershop-based health-promotion interventions for Black men: A systematic scoping review

OBJECTIVE

Health-promotion efforts among Black men in the United States have been limited in their ability to recruit, retain, and produce meaningful health-related changes. These difficulties have led to Black men being referred to as a "hard-to-reach" population-a designation that places undue blame on these men as opposed to the dissemination and implementation strategies being used by health-promotion specialists. Gender- and race-based strategies that align with the lived experiences of these men are likely to circumvent these challenges. Barbershops are cultural institutions that are uniquely positioned to promote health among Black men. There is little guidance on how to develop, implement, and evaluate barbershop-based efforts. This scoping review seeks to provide this guidance by applying the RE-AIM framework to analyze existing interventions.

METHOD

Information was identified by searching the following bibliographic databases: PubMed, EMBASE PsycINFO, CINAHL, and Web of Science. A grey literature search was conducted using Web of Science and ClinicalTrials.gov. Results were uploaded to Rayyan. Each article was independently and blindly assessed by two reviewers. A third reviewer blindly resolved any discrepancies. Data were then independently extracted by the two reviewers. Discrepancies were flagged and resolved collaboratively.

RESULTS

Results indicate that barbershop-based health-promotion efforts that prioritize community engagement and intentional alignment to the gender- and race-based lived experiences of Black men are likely to result in satisfactory recruitment, retention, and health-related changes among these men.

CONCLUSIONS

More intervention efforts are needed that target young Black adults, rural Black men, mental health outcomes, and which implement peer-to-peer models. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Long-term Mammography Utilization after an Initial Randomized Intervention Period by all Underserved Chilean Women in the Clinics

Chile has one of the highest rates of breast cancer in Latin America. Mammography rates among women, especially those of low socioeconomic status (SES), are thought to contribute to high breast cancer morbidity and mortality. A successful randomized controlled trial among women aged 50 to 70 in a low-SES primary care clinic in Chile led to a significant increase in mammography screening rates in a two-year intervention trial. This study assesses the sustainability of the intervention after ten years and identifies factors that might have been associated with a long-term effect using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework. The mammography rates among women aged 50 to 70 in the low-SES intervention clinic were compared to two populations of women aged 50 to 70 from middle-SES clinics and to national data. Qualitative data were used to answer questions of adoption, implementation, and maintenance, while quantitative data assessed the reach and effectiveness. After ten years, low-SES women at the intervention clinic maintained significantly higher mammography screening rates vs. middle-SES women at the comparison clinics (36.2% vs. 30.1% and 19.4% p < 0.0001). Women of a low SES at the intervention clinic also had significantly higher screening rates compared to women of a low SES at a national level (44.2% vs. 34.2% p < 0.0001). RE-AIM factors contributed to understanding the long-term difference in rates. Mailed contact, outreach interventions, and the integration of health promoters as part of the Community Advisory Board were important factors associated with the effects observed. This study provides information on factors that could contribute to reducing the social gap on breast cancer screening.

Diabetes and Cardiovascular Diseases Risk Assessment in Community Pharmacies: An Implementation Study

The implementation of a new service is often challenging when translating research findings into routine clinical practices. This paper presents the results of the implementation study of a pilot project for a diabetes and cardiovascular diseases risk-assessment service in Belgian community pharmacies. To evaluate the implementation of the service, a mixed method was used that follows the RE-AIM framework. During the testing stage, 37 pharmacies participated, including five that dropped out due to a lack of time or COVID-19-related temporary obligations. Overall, 502 patients participated, of which 376 (74.9%) were eligible for according-to-protocol analysis. Of these, 80 patients (21.3%) were identified as being at high risk for the targeted diseases, and 100 (26.6%) were referred to general practice for further investigation. We presented the limited effectiveness and the key elements influencing optimal implementation. Additional strategies, such as interprofessional workshops, a data-sharing platform, and communication campaigns, should be considered to spread awareness of the new role of pharmacists. Such strategies could also promote collaboration with general practitioners to ensure the follow-up of patients at high risk. Overall, this service was considered easy to perform and feasible in practice but would require financial and external support to ensure its effectiveness, sustainability, and larger-scale implementation.