RE-AIM for rural health innovations: perceptions of (mis) alignment between the RE-AIM framework and evaluation reporting in the Department of Veterans Affairs Enterprise-Wide Initiatives program

Background

The Department of Veterans Affairs (VA) Office of Rural Health (ORH) supports national VA program offices' efforts to expand health care to rural Veterans through its Enterprise-Wide Initiatives (EWIs) program. In 2017, ORH selected Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM), an implementation science framework, to structure the EWI evaluation and reporting process. As part of its mandate to improve EWI program evaluation, the Center for the Evaluation of Enterprise-Wide Initiatives conducted a qualitative evaluation to better understand EWI team' perceptions of, and barriers and facilitators to, the EWI evaluation process.

Methods

We conducted 43 semi-structured interviews with 48 team members (e.g., evaluators, program office leads, and field-based leads) representing 21 EWIs from April-December 2020. Questions focused on participants' experiences using strategies targeting each RE-AIM dimension. Interviews were inductively analyzed in MAXQDA. We also systematically reviewed 51 FY19-FY20 EWI annual reports to identify trends in misapplications of RE-AIM.

Results

Participants had differing levels of experience with RE-AIM. While participants understood ORH's rationale for selecting a common framework to structure evaluations, the perceived misalignment between RE-AIM and EWIs' work emerged as an important theme. Concerns centered around 3 sub-themes: (1) (Mis)Alignment with RE-AIM Dimensions, (2) (Mis)Alignment between RE-AIM and the EWI, and (3) (Mis)Alignment with RE-AIM vs. other Theories, Models, or Frameworks. Participants described challenges differentiating between and operationalizing dimensions in unique contexts. Participants also had misconceptions about RE-AIM and its relevance to their work, e.g., that it was meant for established programs and did not capture aspects of initiative planning, adaptations, or sustainability. Less commonly, participants shared alternative models or frameworks to RE-AIM. Despite criticisms, many participants found RE-AIM useful, cited training as important to understanding its application, and identified additional training as a future need.

Discussion

The selection of a shared implementation science framework can be beneficial, but also challenging when applied to diverse initiatives or contexts. Our findings suggest that establishing a common understanding, operationalizing framework dimensions for specific programs, and assessing training needs may better equip partners to integrate a shared framework into their evaluations.

Adopting the RE-AIM analytic framework for rural program evaluation: experiences from the Advance Care Planning via Group Visits (ACP-GV) national evaluation

Introduction

To support rigorous evaluation across a national portfolio of grants, the United States Department of Veterans Affairs (VA) Office of Rural Health (ORH) adopted an analytic framework to guide their grantees' evaluation of initiatives that reach rural veterans and to standardize the reporting of outcomes and impacts. Advance Care Planning via Group Visits (ACP-GV), one of ORH's Enterprise-Wide Initiatives, also followed the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. ACP-GV is a national patient-centered intervention delivered in a large, veterans integrated healthcare system. This manuscript describes how RE-AIM was used to evaluate this national program and lessons learned from ORH's annual reporting feedback to ACP-GV on their use of the framework to describe evaluation impacts.

Methods

We used patient, provider, and site-level administrative health care data from the VA Corporate Data Warehouse and national program management databases for federal fiscal years (FY) spanning October 1, 2018-September 30, 2023. Measures included cumulative and past FY metrics developed to assess program impacts.

Results

RE-AIM constructs included the following cumulative and annual program evaluation results. ACP-GV reached 54,167 unique veterans, including 19,032 unique rural veterans between FY 2018 to FY 2023. During FY 2023, implementation adherence to the ACP-GV model was noted in 91.7% of program completers, with 55% of these completers reporting a knowledge increase and 14% reporting a substantial knowledge increase (effectiveness). As of FY 2023, 66 ACP-GV sites were active, and 1,556 VA staff were trained in the intervention (adoption). Of the 66 active sites in FY 2023, 27 were sites previously funded by ORH and continued to offer ACP-GV after the conclusion of three years of seed funding (maintenance).

Discussion

Lessons learned developing RE-AIM metrics collaboratively with program developers, implementers, and evaluators allowed for a balance of clinical and scientific input in decision-making, while the ORH annual reporting feedback provided specificity and emphasis for including both cumulative, annual, and rural specific metrics. ACP-GV's use of RE-AIM metrics is a key step towards improving rural veteran health outcomes and describing real world program impacts.

The influence of organizational models on the implementation of internet-based cognitive behavior therapy in primary care: A mixed methods study using the RE-AIM framework

Background

Internet-Based Cognitive Behavioral Therapy (iCBT) holds great potential in addressing mental health issues, yet its real-world implementation poses significant challenges. While prior research has predominantly focused on centralized care models, this study explores the implementation of iCBT in the context of decentralized organizational structures within the Swedish primary care setting, where all interventions traditionally are delivered at local Primary Care Centers (PCCs).

Aim

This study aims to enhance our understanding of iCBT implementation in primary care and assess the impact of organizational models on the implementation's outcome using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework.

Method

A mixed-methods research design was employed to identify the factors influencing iCBT implementation across different levels, involving patients, therapists and managers. Data spanning two years was collected and analyzed through thematic analysis and statistical tests. The study encompassed 104 primary care centers, with patient data (n = 1979) sourced from the Swedish National Quality Register for Internet-Based Psychological Treatment (SibeR). Additionally, 53 iCBT therapists and 50 PCC managers completed the Normalization Measure Development Questionnaire, and 15 leaders participated in interviews.

Results

Our investigation identified two implementation approaches, one concentrated and one decentralized. Implementation effectiveness was evident through adherence rates suggesting that iCBT is a promising approach for treating mental ill-health in primary care, although challenges were observed concerning patient assessment and therapist drift towards unstructured treatment. Mandatory implementation, along with managerial and organizational support, positively impacted adoption. Results vary in terms of adherence to established protocols, with therapists working in concentrated model showing a significantly higher percentage of registration in the quality register SibeR (X2 (1, N = 2973) = 430.5774, p = 0.001). They also showed significantly higher means in cognitive participation (Z = -2.179, p = 0.029) and in reflective monitoring (Z = -2.548, p = 0.011).

Discussion

Overall, the study results demonstrate that iCBT, as a complex and qualitatively different intervention from traditional psychological treatment, can be widely implemented in primary care settings. The study's key finding highlights the substantial advantages of the concentrated organizational model. This model has strengths in sustainability, encourages reflective monitoring among therapists, the use of quality registers, and enforces established protocols.

Conclusion

In conclusion, this study significantly contributes to the understanding of the practical aspects associated with the implementation of complex internet interventions, particularly in the context of internet-based cognitive-behavioral therapy (iCBT). The study highlights that effective iCBT integration into primary care requires a multifaceted approach, taking into account organizational models, robust support structures, and a commitment to maintaining quality standards. By emphasizing these factors, our research aims to provide actionable insights that can enhance the practicability and real-world applicability of implementing iCBT in primary care settings.

Evaluation of the Effectiveness of a Bilingual Nutrition Education Program in Partnership with a Mobile Health Unit

There are limited reports of community-based nutrition education with culinary instruction that measure biomarkers, particularly in low-income and underrepresented minority populations. Teaching kitchens have been proposed as a strategy to address social determinants of health, combining nutrition education, culinary demonstration, and skill building. The purpose of this paper is to report on the development, implementation, and evaluation of Journey to Health, a program designed for community implementation using the RE-AIM planning and evaluation framework. Reach and effectiveness were the primary outcomes. Regarding reach, 507 individuals registered for the program, 310 participants attended at least one nutrition class, 110 participants completed at least two biometric screens, and 96 participants attended at least two health coaching appointments. Participants who engaged in Journey to Health realized significant improvements in body mass index, blood pressure, and triglycerides. For higher risk participants, we additionally saw significant improvements in total and LDL cholesterol. Regarding dietary intake, we observed a significant increase in cups of fruit and a decrease in sugar sweetened beverages consumed per day. Our findings suggest that Journey to Health may improve selected biometrics and health behaviors in low-income and underrepresented minority participants.

Population health interventions for cardiometabolic diseases in primary care: a scoping review and RE-AIM evaluation of current practices

Introduction

Cardiometabolic diseases (CMD) are the leading cause of death in high-income countries and are largely attributable to modifiable risk factors. Population health management (PHM) can effectively identify patient subgroups at high risk of CMD and address missed opportunities for preventive disease management. Guided by the Reach, Efficacy, Adoption, Implementation and Maintenance (RE-AIM) framework, this scoping review of PHM interventions targeting patients in primary care at increased risk of CMD aims to describe the reported aspects for successful implementation.

Methods

A comprehensive search was conducted across 14 databases to identify papers published between 2000 and 2023, using Arksey and O'Malley's framework for conducting scoping reviews. The RE-AIM framework was used to assess the implementation, documentation, and the population health impact score of the PHM interventions.

Results

A total of 26 out of 1,100 studies were included, representing 21 unique PHM interventions. This review found insufficient reporting of most RE-AIM components. The RE-AIM evaluation showed that the included interventions could potentially reach a large audience and achieve their intended goals, but information on adoption and maintenance was often lacking. A population health impact score was calculated for six interventions ranging from 28 to 62%.

Discussion

This review showed the promise of PHM interventions that could reaching a substantial number of participants and reducing CMD risk factors. However, to better assess the generalizability and scalability of these interventions there is a need for an improved assessment of adoption, implementation processes, and sustainability.

Increasing the delivery of upper limb constraint-induced movement therapy programs for stroke and brain injury survivors: evaluation of the ACTIveARM project

PURPOSE

To increase the number of constraint-induced movement therapy (CIMT) programs provided by rehabilitation services.

METHODS

A before-and-after implementation study involving nine rehabilitation services. The implementation package to help change practice included file audit-feedback cycles, 2-day workshops, poster reminders, a community-of-practice and drop-in support. File audits were conducted at baseline, every three months for 1.5 years, and once after support ceased to evaluate maintenance of change. CIMT participant outcomes were collected to evaluate CIMT effectiveness and maintenance (Action Research Arm Test and Motor Activity Log). Staff focus groups explored factors influencing CIMT delivery.

RESULTS

CIMT adoption improved from baseline where only 2% of eligible people were offered and/or received CIMT (n = 408 files) to more than 50% over 1.5 years post-implementation (n = 792 files, 52% to 73% offered CIMT, 27%-46% received CIMT). Changes were maintained at 6-month follow-up (n = 172 files, 56% offered CIMT, 40% received CIMT). CIMT participants (n = 74) demonstrated clinically significant improvements in arm function and occupational performance. Factors influencing adoption included interdisciplinary collaboration, patient support needs, intervention adaptations, a need for continued training, and clinician support.

CONCLUSIONS

The implementation package helped therapists overcome an evidence-practice gap and deliver CIMT more routinely.

Community-engaged optimization of COVID-19 rapid evaluation and testing experiences: roll-out implementation optimization trial

BACKGROUND

There continues to be a need for COVID-19 testing that is pragmatic, community-centered, and sustainable. This study will refine and test implementation strategies prioritized by community partners: (1) walk-up no-cost testing, (2) community health worker (promotores)-facilitated testing and preventive care counseling, (3) vending machines that dispense no-cost, self-testing kits.

METHODS

A co-designed Theory of Change from an earlier study phase and the Practical, Robust Implementation and Sustainment Model (PRISM) will guide the study design, measures selection, and evaluation. The first aim is to refine and operationalize a multi-component implementation strategy bundle and outcome measures for COVID-19 testing. A Community and Scientific Advisory Board (CSAB) will be established and include community members, clinical providers/staff from the partnering Federally Qualified Health Center (FQHC), public health researchers, policymakers, and a county health department ambassador. Engagement of CSAB members will be assessed through structured ethnography and a survey about the quality and quantity of engagement practices. The second aim is to implement and evaluate the impact of the implementation strategy bundle to optimize COVID-19 testing in communities using a roll-out implementation optimization (ROIO) design. Seven thousand and five hundred community members will be enrolled across four FQHC clinics over 18 months. Participants will be invited to complete an electronic survey about their demographics, health, and COVID-19 testing results and experiences. CSAB members and clinic partners will participate in PRISM fit and determinant assessments prior to each clinic rollout and post-trial. Interviews will be conducted with 60 community participants and 12 providers/staff following a 3-month rollout period at each clinic, inquiring about their experiences with the implementation strategies. Quantitative data will be analyzed using hierarchical multilevel models to determine the impact of implementation strategies. Qualitative data will be analyzed using rapid qualitative approaches to summarize implementation experiences and identify necessary changes prior to subsequent rollouts. A matrix approach will be used to triangulate data from quantitative and qualitative sources based on PRISM domains.

DISCUSSION

This is one of the first pragmatic implementation trials to use a ROIO design and aims to co-create a sustainable and equitable COVID-19 testing program. Findings are likely to generalize to other public health prevention efforts.

TRIAL REGISTRATION

NCT05894655 March 2, 2023.

The VA National TeleNeurology Program implementation: a mixed-methods evaluation guided by RE-AIM framework

Introduction

The Veteran Affairs (VA) Office of Rural Health (ORH) funded the Veterans Health Administration (VHA) National TeleNeurology Program (NTNP) as an Enterprise-Wide Initiative (EWI). NTNP is an innovative healthcare delivery model designed to fill the patient access gap for outpatient neurological care especially for Veterans residing in rural communities. The specific aim was to apply the RE-AIM framework in a pragmatic evaluation of NTNP services.

Materials and methods

We conducted a prospective implementation evaluation. Guided by the pragmatic application of the RE-AIM framework, we conceptualized a mixed-methods evaluation for key metrics: (1) reach into the Veteran patient population assessed as total NTNP new patient consult volume and total NTNP clinical encounters (new and return); (2) effectiveness through configurational analysis of conditions leading to high Veteran satisfaction and referring providers perceived effectiveness; (3) adoption and implementation by VA sites through site staff and NTNP interviews; (4) implementation success through perceived management, implementation barriers, facilitators, and adaptations and through rapid qualitative analysis of multiple stakeholders' assessments; and (5) maintenance of NTNP through monitoring quarterly TeleNeurology consultation volume.

Results

NTNP was successfully implemented in 13 VA Medical Centers over 2 years. The total NTNP new patient consult volume in fiscal year 2021 (FY21) was 836 (58% rurally residing); this increased to 1,706 in fiscal year 2022 (FY22) (55% rurally residing). Total (new and follow-up) NTNP clinical encounters were 1,306 in FY21 and 3,730 in FY22. Overall, the sites reported positive experiences with program implementation and perceived that the program was serving Veterans with little access to neurological care. Veterans also reported high satisfaction with the NTNP program. We identified the patient level of perceived excellent teleneurologist-patient communications, reduced need to drive to get care, and that NTNP provided care that the Veteran otherwise could not access as key factors related to high Veteran satisfaction.

Conclusions

The VA NTNP demonstrated substantial reach, adoption, effectiveness, implementation success, and maintenance over the first 2 years of the program. The NTNP was highly acceptable to both the clinical providers making the referrals and the Veterans receiving the referred video care. The pragmatic application of the RE-AIM framework to guide implementation evaluations is appropriate, comprehensive, and recommended for future applications.

Using the RE-AIM Framework to Evaluate Implementation of Male Involvement Strategies to Optimize the PMTCT Program in Malawi: A Mixed-Methods Study

Involvement of male partners has been shown to be key for the prevention of mother-to-child HIV transmission (PMTCT). Despite the recorded success, uptake and implementation of strategies to involve men in PMTCT continues to be low in Malawi. In this study, we used the Reach Effectiveness Adoption Implementation and Maintenance (RE-AIM) implementation science framework to explore the implementation of male involvement (MI) strategies in Lilongwe, Malawi. We used a cross-sectional mixed-methods complementary-concurrent design from September to October 2020 in two health facilities. Qualitatively, we used a phenomenological approach and conducted seven focus group discussions (FGDs), three with women and four with men. We further conducted four key informant interviews (KIIs) among health care workers. Quantitatively, we conducted a cross-sectional study comprising 138 men presenting at an antenatal clinic (ANC). We used univariate analysis in Stata for the quantitative data, whereas a manual thematic analysis was applied to the qualitative data. Implementation and adoption of the strategies was high among health providers and there were indications of maintenance of the strategies. Provider's attitude, coordinated service provision, integrated training and service provision, information provision, and baby's HIV outcomes were driving factors in implementing the MI strategies. These factors have contributed to the sustained implementation of the strategies over time. In contrast, financial and time constraints, inadequate human resources, and male-friendly spaces impede the implementation of MI strategies. Improving MI will require a systems approach considering health system and individual-level factors for both providers and consumers.

Protocol for the implementation of a statewide mobile addiction program

With overdose deaths increasing, improving access to harm reduction and low barrier substance use disorder treatment is more important than ever. The Community Care in Reach® model uses a mobile unit to bring both harm reduction and clinical care for addiction to people experiencing barriers to office-based care. These mobile units provide many resources and services to people who use drugs, including safer consumption supplies, naloxone, medication for substance use disorder treatment, and a wide range of primary and preventative care. This protocol outlines the evaluation plan for the Community in Care® model in MA, USA. Using the RE-AIM framework, this evaluation will assess how mobile services engage new and underserved communities in addiction services and primary and preventative care.

Implementation and maintenance of an enhanced pulmonary rehabilitation program in a single centre: An implementation study

BACKGROUND

Pulmonary rehabilitation (PR) has major benefits for patients with chronic obstructive pulmonary disease (COPD). An enhanced PR program was developed with a self-management education intervention. The objective of our study was to evaluate the implementation of the enhanced PR program into a single centre.

METHODS

Pre-post implementation study consisted of two evaluation periods: immediately after implementation and 18 months later. Guided by the RE-AIM framework, outcomes included: Reach, Effectiveness, Adoption, Implementation and Maintenance.

RESULTS

Reach: 70-75% of referred patients agreed to a PR program (n = 26). Effectiveness: Clinically important improvements occurred in some patients in functional exercise capacity (64% of the patients achieved clinical important difference in 6-min walk test in the first evaluation period and 44% in the second evaluation period), knowledge, functional status, and self-efficacy in both evaluation periods. Adoption: All healthcare professionals (HCPs) involved in PR (n = 8) participated. Implementation: Fidelity for the group education sessions ranged from 76 to 95% (first evaluation) and from 82 to 88% (second evaluation). Maintenance: The program was sustained over 18 months with minor changes. Patients and HCPs were highly satisfied with the program.

CONCLUSIONS

The enhanced PR program was accepted by patients and HCPs and was implemented and maintained at a single expert center with good implementation fidelity.

Implementation of an evidence-based tobacco control intervention for school teachers in India: Evaluating the effects of a capacity-building strategy

Background

Tobacco-Free Teachers, Tobacco-Free Society (TFT-TFS) is an evidence-based intervention that promotes tobacco use cessation among teachers and tobacco control policies among schools in India. This study tested an implementation model to build Bihar Department of Education (DOE) capacity to support and deliver TFT-TFS within schools, leveraging DOE training infrastructure.

Method

We used a training-of-trainers (TOT) "cascade" implementation strategy to embed the TFT-TFS program into the Bihar DOE infrastructure. We trained 46 Cluster Coordinators to train and support Headmasters to implement TFT-TFS in their schools over one academic year. We selected three school districts, representing approximately 46 clusters and 219 schools. We used the RE-AIM framework to assess program adoption (Headmaster participation in at least one of six TFT-TFS trainings), implementation (of four core program components), and reach (teachers' participation in three or more group discussions). Using a non-inferiority design, we hypothesized that program adoption, implementation, and reach would not be inferior to the high standards demonstrated when TFT-TFS was originally tested in the Bihar School Teachers Study. We used self-reported checklists to measure outcomes and SPSS Version 25 to analyze data.

Results

For adoption, 94% of Headmasters attended the first training, although participation declined by the sixth training. Among the 112 schools out of 219 with complete Headmaster checklist data, all met our minimum criteria for implementing TFT-TFS. Over 99% of schools posted a school tobacco control policy and distributed quit booklets. However, only 69% of schools met our criteria for program reach.

Conclusions

This study outlines the processes for taking a tobacco control intervention to scale and implementing it through the Bihar DOE infrastructure. These findings provide a foundation for other Indian states and low- and middle-income countries to implement tobacco control and other health programs for schoolteachers.

Trial registration

NCT05346991.

Plain Language Summary

Each year in India, more than 1.2 million people die from tobacco-related causes, and India has the world's highest oral cancer burden. The world needs more evidence on how to bring cost-effective tobacco control interventions to scale, especially in low- and middle-income countries (LMICs). To address this gap, from 2017 to 2021, we examined the process of scaling up Tobacco-Free Teachers, Tobacco-Free Society (TFT-TFS), an evidence-based intervention promoting tobacco use cessation among teachers and tobacco control policies in schools. Our study tested an implementation model aimed at building the Bihar State Department of Education (DOE) capacity to support and deliver TFT-TFS. We used a training-of-trainers model to embed TFT-TFS into Bihar DOE infrastructure, training 46 Cluster Coordinators to in turn train and support Headmasters to implement TFT-TFS over one academic year. We hypothesized that program adoption, implementation, and reach would not be inferior to the high standards demonstrated when we originally tested TFT-TFS through the Bihar School Teachers Study (2013-2017). For adoption, 94% of Headmasters attended the first training, although participation declined by the sixth training. Of 112 schools (out of 219 with complete Headmaster checklist data), all met our minimum criteria for implementing TFT-TFS. Over 99% of schools posted a school tobacco control policy and distributed quit booklets. However, only 69% of schools met our criteria for program reach. Study findings offer other Indian states and LMICs lessons to implement tobacco control and other health programs for schoolteachers within educational systems.

Effective engagement of a stakeholder advisory board in severe mental illness (SMI) research: A case study of a clinical trial to improve adherence among people with SMI and hypertension

Objective

Poor adherence to antihypertensive medication occurs in 50-80% of patients. An ongoing randomized controlled trial (RCT) is evaluating a personalized mobile-health intervention in poorly adherent hypertensive persons with bipolar disorder. To enhance efficacy, the ongoing trial elicited guidance from a Stakeholder Advisory Board (SAB) comprised of patients, family members, clinicians, and health system administrators. Our goal is to describe the formation, role, decision-making process, and key contributions of the SAB as a means of demonstrating meaningful community engagement in mental health research.

Methods

Using models and measures from the field of implementation science, eleven SAB members convened across three meetings followed by quantitative surveys that assessed SAB member satisfaction and engagement during the meeting.

Results

Significant suggestions from the SAB included 1) expanding inclusion/exclusion criteria, and 2) operationalizing remote implementation of the RCT. Primary study implementation challenges identified by the SAB were 1) participant difficulty engaging in the mHealth intervention, and 2) identification of procedures for monitoring participant adherence to the RCT protocol and contacting under-engaged participants. Quantitative surveys indicated that all SAB members believed that the objectives of the meetings were clear, perceived that they were able to participate in the discussions, and that they were heard.

Conclusions

Increasing evidence demonstrates the feasibility of engaging with SABs in clinical research and that this process improves intervention design, increases participant engagement, reduces mental health-related stigma, and produces more effective implementation strategies. We encourage future investigators to use an implementation science framework in partnership with SABs to refine their proposed interventions and improve clinical outcomes.

Status quo and problem analysis of cervical cancer screening program in China: Based on RE-AIM framework

Background

An organized cervical cancer screening program is an effective method to prevent and control cervical cancer. This study aims to find barriers and facilitators in the implementation process of National Cervical Cancer Screening Program in Rural Areas (NACCSPRA) in China through program evaluation, and thus propose suggestions for optimization of the program.

Methods

Through stratified sampling, 8 provinces (autonomous cities/districts) in eastern, southern, western, northern, and central China were selected for evaluation of NACCSPRA based on the RE-AIM framework. We obtained 15 program providers' experience and perspectives through semi-structured interviews. The data was analyzed using a combination of deductive and inductive analysis methods.

Results

The study found that NACCSPRA mainly serves women with rural household registration or urban minimum living guarantee. Population mobility and certain demographic characteristics such as low education and poor health awareness are common participation barriers, while program publicity acts as a facilitator. A screening program's direct benefit is to promote early detection and treatment of cervical cancer, and its perceived indirect effect is to raise people's health awareness. The proportion of regions adopting the project is relatively high, and factors affecting employees' participation are screening workload, working environment, welfare benefits, degree of preference for grassroots work, and whether the project is included in the performance appraisal; In terms of implementation, there are disparities in screening methods, network informatization levels, and capital investment in various regions. Poor development of screening information system and insufficient screening funds are significant barriers to improvement of project implementation. In contrast, the overall implementation of follow-up is better; related policies issued by the local government and financial subsidies for poor women ensure the maintenance of the project.

Conclusion

Shortage of funds is an important problem faced by current screening project, which negatively influences upgrade of cervical cancer prevention strategy, implementors' working environment, and impedes improvement of information network. In addition, defects in population coverage, especially in mobile population also deserves attention. The study found barriers and facilitators of NACCSPRA perceived by project providers and provided a theoretical foundation for project optimization.

Municipal community centers as healthy settings: evaluation of a real-world health promotion intervention in Jerusalem

Background

This study presents an intervention designed to foster the implementation of health promotion programs within District Municipality Community Centers (DMCCs) in Jerusalem, and the creation of a peer network of healthy settings with a shared aspiration of collaborating and implementing health-promoting policies at the community level. We also present the evaluation strategy, based on the EQUIHP and RE-AIM frameworks.

Methods

Twenty DMCCs completed our program. This evaluation research involved a comprehensive seminar during the first year for DMCCs coordinators, teaching them the principles of health promotion. An educational kit was distributed during the second year. The evaluation strategy included a process evaluation and annual evaluations based on the EQUIHP and RE-AIM frameworks. The EQUIHP tool was divided into four dimensions of evaluation: 1) Framework of health promotion principles, 2) Project development and implementation, 3) Project management, and 4) Sustainability; while the RE-AIM domains included: 1)Reach, 2)Effectiveness, 3)Adoption, 4)Implementation and 5)Maintenance.

Results

The program led to high responsiveness among DMCCs and to the implementation of diverse health promotion initiatives, with a participation of approximately 29,191 residents. The EQUIHP evaluation showed an improvement in program quality in Year 2. The final RE-AIM evaluation presented a total median score of 0.61 for all domains, where 0 was non-performance and 1.0 was full performance. The ‘Framework of health promotion principles’ and ‘Reach’ components received the highest median score (0.83, 1.0 and 0.87), while the ‘Sustainability and ‘Maintenance’ components received the lowest (0.5).

Conclusions

This innovative program adapts the Healthy Cities approach (initiated by the World Health Organization in 1986) to the development of community center health-promoting settings within the larger municipal framework, training local community center staff members to assess and address local health concerns and build community capacity. The local focus and efforts may help community actors to create health promotion programs more likely to be adopted, feasible in the ‘real-world’ and able to produce public health impact in the communities where people live. Moreover, collaboration and cooperation among DMCCs may lead to a broader community health vision, forging coalitions that can advocate more powerfully for health promotion.

Trial registration

NIH trial registration number: NCT04470960.

Retrospectively registered on: 14/07/2020.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-14220-7.

Using a longitudinal multi-method approach to document, assess, and understand adaptations in the Veterans Health Administration Advanced Care Coordination program

Background

Understanding adaptations supports iterative refinement of the implementation process and informs scale out of programs. Systematic documentation of adaptations across the life course of programs is not routinely done, and efficient capture of adaptations in real world studies is not well understood.

Methods

We used a multi-method longitudinal approach to systematically document adaptations during pre-implementation, implementation, and sustainment for the Veteran Health Administration (VA) Advanced Care Coordination program. This approach included documenting adaptations through a real-time tracking instrument, process maps, Implementation and Evaluation (I&E) team meeting minutes, and adaptation interviews. Data collection was guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) enhanced framework for reporting adaptations and modifications to evidence-based interventions (FRAME) model. Adaptations were evaluated across 9 categories, and analytic team consensus and member-checking were used to validate the results.

Results

A total of 144 individual adaptations were identified across two implementation sites and the four data sources; analytic team consensus and member-checking processes resulted in 50 unique adaptations. Most adaptations took place during the early implementation and mid-implementation phases and were: 1) planned; 2) made to address changes in program delivery; 3) made to extend a component; 4) related to the core component of the intervention concerning notification of the community emergency department visit; 5) initiated by the entire or most of the I&E team; 6) made on the basis of: pragmatic/practical considerations; 7) made with an intent to improve implementation domain (to make the intervention delivered more consistently; to better fit the local practice, patient flow or Electronic Health Record (EHR) and/or for practical reasons); 8) a result of internal influences; 9) perceived to impact the RE-AIM implementation dimension (consistent delivery of quality care or costs). I&E team meeting minutes and process maps captured the highest numbers of unique adaptations (n = 19 and n = 13, respectively).

Conclusion

Our longitudinal, multi-method approach provided a feasible way to collect adaptations data through engagement of multiple I&E team members, allowing and a broader understanding of adaptations that took place. Recommendations for future research include pragmatic assessment of the impact of adaptations and meaningful data collection without overburdening the implementing teams and front-line staff.

The RE-AIM framework-based evaluation of the implementation of the Maternal and Child Health Handbook program in Angola: a mixed methods study

Background

The World Health Organization recommends the Maternal and Child Health Handbook (MCH-HB) to promote health service utilization from pregnancy to early childhood. Although many countries have adopted it as a national health policy, there is a paucity of research in MCH-HB’s implementation. Thus, this study aimed to evaluate the MCH-HB’s implementation status based on the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, Maintenance), and identify facilitators of, and barriers to its implementation in Angola to understand effective implementation strategies.

Methods

A cross-sectional survey was conducted targeting all health facilities which implemented MCH-HB, subsamples of health workers, and officers responsible for the MCH-HB at the municipality health office. Using the 14 indicators based on the RE-AIM framework, health facilities’ overall implementation statuses were assessed. This categorized health facilities into optimal-implementation and suboptimal-implementation groups. To identify barriers to and facilitators of MCH-HB implementation, semi-structured interviews were conducted among health workers and municipality health officers responsible for MCH-HB. The data were analyzed via content analysis.

Results

A total of 88 health facilities and 216 health workers were surveyed to evaluate the implementation status, and 155 interviews were conducted among health workers to assess the barriers to and facilitators of the implementation. The overall implementation target was achieved in 50 health facilities (56.8%). The target was achieved by more health facilities in urban than rural areas (urban 68.4%, rural 53.6%) and by more health facilities of higher facility types (hospital 83.3%, health center 59.3%, health post 52.7%). Through the interview data’s analysis, facilitators of and barriers to MCH-HB were comprehensively demonstrated. MCH-HB’s content advantage was the most widely recognized facilitator and inadequate training for health workers was the most widely recognized barrier.

Conclusions

Strengthening education for health workers, supervision by municipality health officers, and community sensitization were potential implementation strategies. These strategies must be intensified in rural and lower-level health facilities.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08454-9.

Long-term Mammography Utilization after an Initial Randomized Intervention Period by all Underserved Chilean Women in the Clinics

Chile has one of the highest rates of breast cancer in Latin America. Mammography rates among women, especially those of low socioeconomic status (SES), are thought to contribute to high breast cancer morbidity and mortality. A successful randomized controlled trial among women aged 50 to 70 in a low-SES primary care clinic in Chile led to a significant increase in mammography screening rates in a two-year intervention trial. This study assesses the sustainability of the intervention after ten years and identifies factors that might have been associated with a long-term effect using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework. The mammography rates among women aged 50 to 70 in the low-SES intervention clinic were compared to two populations of women aged 50 to 70 from middle-SES clinics and to national data. Qualitative data were used to answer questions of adoption, implementation, and maintenance, while quantitative data assessed the reach and effectiveness. After ten years, low-SES women at the intervention clinic maintained significantly higher mammography screening rates vs. middle-SES women at the comparison clinics (36.2% vs. 30.1% and 19.4% p < 0.0001). Women of a low SES at the intervention clinic also had significantly higher screening rates compared to women of a low SES at a national level (44.2% vs. 34.2% p < 0.0001). RE-AIM factors contributed to understanding the long-term difference in rates. Mailed contact, outreach interventions, and the integration of health promoters as part of the Community Advisory Board were important factors associated with the effects observed. This study provides information on factors that could contribute to reducing the social gap on breast cancer screening.

Reaching Older People With a Digital Fall Prevention Intervention in a Swedish Municipality Context-an Observational Study

Background

There is robust evidence that falls in old age can be prevented by exercise programs that include balance training, functional exercises, and strength training. For the interventions to have a population health impact, outreach to the population of focus with suitable interventions is needed. While digital interventions are promising there is limited knowledge on the characteristics of who is reached. The aim of this study was to describe the recruitment process, estimate reach rate at the population level and to describe participants characteristics and representativeness in a digital fall prevention intervention study.

Methods

In a municipality-based observational study, reach of a digital fall prevention intervention was evaluated. The intervention included a digital exercise programme (Safe Step) and optional supportive strategies, complemented with a range of recruitment strategies to optimize reach. Recruitment during a period of 6 months was open to people 70 years or older who had experienced a fall or a decline in balance the past year. Reach was based on data from the baseline questionnaire including health and demographic characteristics of participants. Representativeness was estimated by comparing participants to a sample of older people from the Swedish National Public Health Survey.

Results

The recruitment rate was 4.7% (n = 173) in relation to the estimated population of focus (n = 3,706). Most participants signed up within the first month of the intervention (n = 131). The intervention attracted primarily women, older people with high education, individuals who used the internet or digital applications almost every day and those perceiving their balance as fair or poor. Safe step participants lived more commonly alone and had higher education and better walking ability in comparison to the Swedish National Public Health Survey.

Conclusions

With a range of recruitment strategies most participants were recruited to a digital fall intervention during the first month. The intervention attracted primarily highly educated women who frequently used the internet or smart technologies. In addition to digital fall prevention interventions, a higher diversity of intervention types (digital and non-digital) is more likely to reach a larger group of older people with different needs.

Clinical Trial Registration

ClinicalTrials.gov, NCT04161625 (Retrospectively registered), https://clinicaltrials.gov/ct2/show/NCT04161625.

Process evaluation of a medical assistant health coaching intervention for type 2 diabetes in diverse primary care settings

Team-based models that use medical assistants (MAs) to provide self-management support for adults with type 2 diabetes (T2D) have not been pragmatically tested in diverse samples. This cluster-randomized controlled trial compares MA health coaching with usual care in adults with T2D and poor clinical control ("MAC Trial"). The purpose was to conduct a multi-method process evaluation of the MAC Trial using the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework. Reach was assessed by calculating the proportion of enrolled participants out of the eligible pool and examining representativeness of those enrolled. Key informant interviews documented adoption by MA Health Coaches. We examined implementation from the research and patient perspectives by evaluating protocol adherence and the Patient Perceptions of Chronic Illness Care (PACIC-SF) measure, respectively. Findings indicate that the MAC Trial was efficient and effective in reaching patients who were representative of the target population. The acceptance rate among those approached for health coaching was high (87%). Both MA Health Coaches reported high satisfaction with the program and high levels of confidence in their role. The intervention was well-implemented, as evidenced by the protocol adherence rate of 79%; however, statistically significant changes in PACIC-SF scores were not observed. Overall, if found to be effective in improving clinical and patient-reported outcomes, the MAC model holds potential for wider-scale implementation given its successful adoption and implementation and demonstrated ability to reach patients with poorly controlled T2D who are at-risk for diabetes complications in diverse primary care settings.

Scaling a Community-Wide Campaign Intervention to Manage Hypertension and Weight Loss

Public health impacts can be achieved when evidence-based interventions are implemented to those most in need. Too often implementation never or slowly occurs. The community-wide campaign intervention Tu Salud ¡Si Cuenta! has evidence of improving health outcomes related to chronic disease among low-income, Latinos. Using the RE-AIM Framework, this study examined if the scaled-up version of the intervention is associated with improvements in hypertension and obesity in 12 locations. Each element of the RE-AIM framework was examined. For "Effectiveness," we examined outcomes overall and by implementing location. We used linear and logistic regression to assess if exposure in the intervention was associated with improvement in hypertension and weight loss. Participants were stratified into "low exposure" (2-3 outreach visits) vs. "high exposure" (4-5 outreach visits). Based on the RE-AIM Framework, the intervention "reached" its intended population of low-income Latinos, demonstrated "effectiveness" in improving hypertension and obesity, was "adopted" at a high level in all but one site, was "implemented" with fidelity to the intervention model with moderate success across locations, and showed high "maintenance" over time. For effectiveness specifically, we found that out of 5,019 participants, 2,508 (50%) had a baseline hypertensive blood pressure (BP) reading. Of the 2,508, 1,245 (49.9%) recovered to normal blood pressure or pre-hypertension stage by last follow-up. After adjusting for baseline BP and potential confounders in multivariable linear regression models, the high exposure group had significantly more reduction in systolic BP (adjusted mean difference in % change = -0.96; p = 0.002) and diastolic BP (adjusted mean difference in % change = -1.61; p < 0.0001) compared to the low exposure group. After controlling for baseline weight and other confounders, the high exposure group had significantly greater decrease in weight compared to the low exposure group (adjusted mean difference in % change = -1.28; p < 0.0001). Results from the multivariable logistic regression models indicated that compared to the low exposure group the high exposure group was more likely to achieve a clinically significant minimum 5% weight loss [adjusted odds ratio (OR) = 2.97; p < 0.0001). This study contributes evidence that a Community-Wide Campaign model holds promise for addressing hypertension and obesity among low-income Latinos.

Evaluation of Healthy South Texas Asthma Program on improving health outcomes and reducing health disparities among the underserved Hispanic population: using the RE-AIM model

Background

In the United States, childhood asthma prevalence is higher among low-income communities and Hispanic populations. Previous studies found that asthma education could improve health and quality of life, especially in vulnerable populations lacking healthcare access. This study aims to describe Healthy South Texas Asthma Program (HSTAP), an evidence-based asthma education and environmental modification program in South Texas, and evaluate its associations with health-related outcomes among Hispanic children with asthma and their families.

Methods

The RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) planning and evaluation framework was used as an overarching tool to evaluate the impact of the HSTAP. This educational program included 451 children with asthma and their families living in South Texas, an impoverished area at the Texas-Mexico border. The program consisted of (a) the asthma education (2-h) for children with asthma provided by Respiratory Therapy students at the children’s schools and (b) the home visit Asthma and Healthy Homes education and walk-through sessions (at baseline and 3 months) for parents and two follow-up visits (6 and 9–12 months later) led by community health workers. The education was provided in either English or Spanish between September 2015 and August 2020 as part of the Healthy South Texas Initiative. A pre-and post-test design was implemented to assess the differences in health outcomes, knowledge, and behaviors using standardized self-reported surveys as reported by parents. Analyses included primary descriptive analyses, generalized estimating equation models, the Wilcoxon signed-rank test, and the McNemar test.

Results

The HSTAP was significantly associated with improved individual-level outcomes on the frequency of asthma-related respiratory symptoms, including shortness of breath, chest tightness, coughing, and sleep difficulty, among children with asthma, as well as an enhanced asthma knowledge in their family. This study also showed significant associations with children’s school attendance and participation in physical activities and family social events and decreased families’ worry about their asthma management.

Conclusions

The RE-AIM model was a helpful framework to assess the HSTAP on all its components. The results suggest that participation in an asthma education and environmental modification program was associated with improved individual-level health conditions and reduced health disparities among children with asthma in low-income communities.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12887-021-02991-8.

Health ambassadors in the workplace: a health promotion intervention mobilizing middle managers and RE-AIM evaluation of outcomes

Background

The workplace provides an ideal setting for health promotion, as adults spend most of their day at work. Middle managers hold a strategic position to lead workplace health promotion. This study evaluates the outcomes of an intensive intervention training middle managers to promote health in the workplace.

Methods

A workshop was designed and conducted to train female middle management employees to construct, implement, and evaluate a health promotion program in their workplace. Semi-structured interviews were carried out post-intervention to assess workplace health promotion outcomes according to the RE-AIM framework, and identify variables contributing to success. Additionally, questionnaires were distributed pre and post-program assessing personal health and self-efficacy changes.

Results

Eighteen participants from 13 government offices, who serve 19,560 employees, completed the training course. Nine workplaces had workplace health promotion programs in progress 12 months after the course had ended, of which 8 made health promotion changes in organizational policy. Workplace RE-AIM scores showed that 8 workplaces were high or partial performers, and 5 were low or non-performers. Factors that increased the likelihood of successful interventions included management support, steering committee, comprehensive programming, conducting a needs assessment and flexibility in program implementation in the presence of challenges. Post course, participants reported increased health knowledge related to workplace health promotion (p < 0.001), and increased health promotion self-efficacy (p < 0.05).

Conclusions

Training and continued guidance of middle managers resulted in the design and successful implementation of workplace health promotion interventions. A RE-AIM based assessment was found to be an effective method for evaluating multi-content workplace health promotion programs.

Registered at ClinicalTrials.gov, https://www.clinicaltrials.gov, registration number:

NCT03295136, registration date: 24/09/2017.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-021-11609-8.

'Shape-Up', a Modified Cognitive-Behavioural Community Programme for Weight Management: Real-World Evaluation as an Approach for Delivering Public Health Goals

Obesity is widespread, with serious health consequences; addressing it requires considerable effort at a public health level, incorporating prevention and management along with policies to support implementation. Behavioural weight-management programmes are widely used by public health bodies to address overweight and obesity. Shape-Up is an evidence-based programme combining a structured behavioural intervention (targeting nutrition and physical activity behaviours) within a peer-learning framework. This study was a service-evaluation of Shape-Up, as delivered in Rotherham by a local leisure provider, and included a secondary analysis of data collected in the community by service providers. The RE-AIM (Reach Effectiveness Adoption Implementation Maintenance) framework was used to explore programme effectiveness, implementation, and whom it reached. A total of 141 participants were included. Compared to local demographics, participants were older, at 48.9 (SD 14.47) years, with a lower employment rate (41% employed) and greater proportion female (67% female). Mean BMI was 38.0 (SD 7.54) kg/m2. Mean weight-change between baseline and endpoint (12 weeks, 10 group sessions) was -4.4 (SD 3.38) kg, and degree of weight change was associated with session attendance (F (9, 131) = 6.356, p < 0.0005). There were positive effects on participants' weight, health-related behaviours, and quality of life. The intervention content (including the focus of nutritional recommendations) and structure were adapted during implementation to better suit national guidelines and local population needs. RE-AIM was found to be a useful framework for evaluating and adapting an existing evidence-based weight management programme in line with local population needs. This could be a more cost-effective approach, compared to developing new programmes, for delivering public health goals relating to obesity, nutrition, and physical activity.

Understanding adaptations in the Veteran Health Administration's Transitions Nurse Program: refining methodology and pragmatic implications for scale-up

BACKGROUND

When complex health services interventions are implemented in real-world settings, adaptations are inevitable. Adaptations are changes made to an intervention, implementation strategy, or context prior to, during, and after implementation to improve uptake and fit. There is a growing interest in systematically documenting and understanding adaptations including what is changed, why, when, by whom, and with what impact. The rural Transitions Nurse Program (TNP) is a program in the Veterans Health Administration (VHA), designed to safely transition a rural veteran from a tertiary hospital back home. TNP has been implemented in multiple cohorts across 11 sites nationwide over 4 years. In this paper, we describe adaptations in five TNP sites from the first cohort of sites and implications for the scale-up of TNP and discuss lessons learned for the systematic documentation and analysis of adaptations.

METHODS

We used the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) expanded version of the original Stirman framework to guide the rapid qualitative matrix analysis of adaptations. Adaptations were documented using multiple approaches: real-time database, semi-structured midpoint and exit interviews with implementors, and member checking with the implementation team. Interviews were recorded and transcribed. To combine multiple sources of adaptations, we used key domains from our framework and organized adaptations by time when the adaptation occurred (pre-, early, mid-, late implementation; sustainment) and categorized them as proactive or reactive.

RESULTS

Forty-one unique adaptations were reported during the study period. The most common type of adaptation was changes in target populations (patient enrollment criteria) followed by personnel changes (staff turnover). Most adaptations occurred during the mid-implementation time period and varied in number and type of adaptation. The reasons for this are discussed, and suggestions for future adaptation protocols are included.

CONCLUSIONS

This study demonstrates the feasibility of systematically documenting adaptations using multiple methods across time points. Implementors were able to track adaptations in real time across the course of an intervention, which provided timely and actionable feedback to the implementation team overseeing the national roll-out of the program. Longitudinal semi-structured interviews can complement the real-time database and elicit reflective adaptations.

Process Evaluation of Implementing a Pharmacist-Led Intervention to Improve Adherence to Antihypertensive Drugs Among Patients with Type 2 Diabetes in Indonesian Community Health Centers

Introduction: A pharmacist-led intervention in Community Health Centers (CHCs) in Indonesia targeted at patients with type 2 diabetes non-adherent to antihypertensive drugs resulted in a significant improvement in adherence to these drugs. The aim of this study was to evaluate the process of implementation this intervention intended to improve adherence to antihypertensive drugs from both the pharmacist and the patient perspective. Methods: Using the RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) framework, we conducted a focus group among pharmacists (N = 5) and a survey among patients with complete follow-up (N = 44) participating in the intervention group. Results: All pharmacists adopted the provided training and found support tools useful. The pharmacists implemented the intervention as intended (adequate intervention fidelity >69%). Factors relevant for implementation included having sufficient time and confidence, home visits for specific patients, multidisciplinary collaboration, and availability of a personal counseling room. To maintain the intervention, the need for practical guidance and support from health care authorities was mentioned. Most patients (96%) were satisfied with the information provided by the pharmacists and they believed the tailored counselling was helpful. Most patients (84%) reported that the duration of counselling was sufficient. The large majority of patients would like to receive the counselling regularly. Conclusion: Positive effects of the pharmacist-led intervention can be explained by adequate levels of reach, adoption and implementation in the participating CHCs. For successful implementation and maintenance in Indonesia or other low-and middle-income countries, sufficient training, resources, multidisciplinary collaboration, guidance and support from health care authorities are expected to be important.

Barriers and Facilitators to the International Implementation of Standardized Outcome Measures in Clinical Cleft Practice

Objective:

To identify barriers and facilitators to international implementation of a prospective system for standardized outcomes measurement in cleft care.

Design:

Cleft teams that have implemented the International Consortium for Health Outcomes Measurement Standard Set for cleft care were invited to participate in this 2-part qualitative study: (1) an exploratory survey among clinicians, health information technology professionals, and project coordinators, and (2) semistructured interviews of project leads. Thematic content analysis was performed, with organization of themes according to the dimensions of the reach, effectiveness, adoption, implementation and maintenance (RE-AIM) framework: reach, effectiveness, adoption, implementation, and maintenance.

Results:

Four cleft teams in Europe and North America participated in this study. Thirteen participants completed exploratory questionnaires and 5 interviewees participated in follow-up interviews. Survey responses and thematic content analysis revealed common facilitators and barriers to implementation at all sites. Teams reach patients either via email or during the clinic visit to capture patient-reported outcomes. Adopting routine data collection is enhanced by aligning priorities at the organizational and cleft team level. Streamlining workflows and developing an efficient data collection platform are necessary early on, followed by pilot testing or stepwise implementation. Regular meetings and financial resources are crucial for implementing, sustaining, analyzing collected data, and providing feedback to health care professionals and patients. Fostering patient-centered care was articulated as a positive outcome, whereas time presented challenges across all RE-AIM dimensions.

Conclusions:

Identified themes can inform ongoing implementation efforts. Intentionally investing time to lay a sound foundation early on will benefit every phase of implementation and help overcome barriers such as lack of support or motivation.

Implementing a prospective surveillance and early intervention model of care for breast cancer-related lymphedema into clinical practice: application of the RE-AIM framework

PURPOSE

Chronic lymphedema following breast cancer (BC) affects individuals physically, functionally, psychologically, and financially. Despite national guidelines and evidence-based research supporting a prospective surveillance and early intervention model of care (PSM), bridging the gap between research and clinical practice has been difficult.

METHOD

As part of an international randomized controlled trial (RCT), Australian women with BC from four hospitals were recruited, monitored for lymphedema at regular intervals over a 3-year period, and were provided a compression garment if intervention was triggered. The reach, effectiveness, adoption, implementation and maintenance (RE-AIM) evaluation framework was used retrospectively to assess a PSM at the individual and organizational level for those who had completed at least 2-year follow-up (N = 219) in the RCT.

RESULTS

The application of the RE-AIM framework retrospectively demonstrated an extensive reach to patients across public and private settings; the effectiveness of prospective surveillance and early intervention was achieved through low progression rates to clinical lymphedema (1.8%), and all hospital sites initially approached adopted the research study. Key implementation strategies necessary for effectiveness of this model of care included education to health professionals and patients, staff acceptability, and development of a referral and care pathway. Maintenance dimensions were evaluated both at the individual level with 92-100% adherence rates for all nonoptional study appointments over the 2-year period, and at the organizational-level, PSM was sustained after recruitment ceased for the research study.

CONCLUSION

The PSM for lymphedema in BC can be successfully implemented using the RE-AIM framework applied retrospectively. The implementation of the PSM used in the RCT has assisted in changing clinical practices and improving the quality and effectiveness of the health care system.

Improving Patient Prioritization During Hospital-Homecare Transition: Protocol for a Mixed Methods Study of a Clinical Decision Support Tool Implementation

Background

Homecare settings across the United States provide care to more than 5 million patients every year. About one in five homecare patients are rehospitalized during the homecare episode, with up to two-thirds of these rehospitalizations occurring within the first 2 weeks of services. Timely allocation of homecare services might prevent a significant portion of these rehospitalizations. The first homecare nursing visit is one of the most critical steps of the homecare episode. This visit includes an assessment of the patient’s capacity for self-care, medication reconciliation, an examination of the home environment, and a discussion regarding whether a caregiver is present. Hence, appropriate timing of the first visit is crucial, especially for patients with urgent health care needs. However, nurses often have limited and inaccurate information about incoming patients, and patient priority decisions vary significantly between nurses. We developed an innovative decision support tool called Priority for the First Nursing Visit Tool (PREVENT) to assist nurses in prioritizing patients in need of immediate first homecare nursing visits.

Objective

This study aims to evaluate the effectiveness of the PREVENT tool on process and patient outcomes and to examine the reach, adoption, and implementation of PREVENT.

Methods

Employing a pre-post design, survival analysis, and logistic regression with propensity score matching analysis, we will test the following hypotheses: compared with not using the tool in the preintervention phase, when homecare clinicians use the PREVENT tool, high-risk patients in the intervention phase will (1) receive more timely first homecare visits and (2) have decreased incidence of rehospitalization and have decreased emergency department use within 60 days. Reach, adoption, and implementation will be assessed using mixed methods including homecare admission staff interviews, think-aloud observations, and analysis of staffing and other relevant data.

Results

The study research protocol was approved by the institutional review board in October 2019. PREVENT is currently being integrated into the electronic health records at the participating study sites. Data collection is planned to start in early 2021.

Conclusions

Mixed methods will enable us to gain an in-depth understanding of the complex socio-technological aspects of the hospital to homecare transition. The results have the potential to (1) influence the standardization and individualization of nurse decision making through the use of cutting-edge technology and (2) improve patient outcomes in the understudied homecare setting.

Trial Registration

ClinicalTrials.gov NCT04136951; https://clinicaltrials.gov/ct2/show/NCT04136951

International Registered Report Identifier (IRRID)

PRR1-10.2196/20184

A Process Evaluation of A UK Classroom-Based Physical Activity Intervention-'Busy Brain Breaks'

The gap between development of effective physical activity interventions and the wide-scale adoption of these interventions in real-world settings has been reported since the early 2000s. Evaluations have been criticised for failing to report details of context, implementation, adoption and maintenance. 'Busy Brain Breaks' was an intervention designed to improve fundamental movement patterns whilst increasing physical activity within the classroom. This evaluation study used a mixed-methods approach including questionnaires, observations, semi-structured interviews and quantification of class-level dose. Findings suggest that giving teachers flexibility and autonomy over the way in which they implement physical activity interventions may increase the likelihood of adoption. Time was frequently perceived as a significant barrier to the intervention, giving the teachers flexibility to implement the intervention when they thought most suitable allowed teaching staff to retain their autonomy and make the intervention work with their schedule. Children's behaviour appeared to be both a facilitator and barrier to implementing physical activity interventions within the classroom. Whilst misbehaviour can pose as a barrier, children's enjoyment acts as a key facilitator to implementation for teaching practitioners. Teachers interviewed (n = 17) observed that movement ability had developed as a result of the intervention and recognised co-ordination, balance and stability as areas that had noticeably improved. Conducting an in-depth process evaluation has allowed for greater insight and understanding as to how, and to what extent, the intervention was implemented within the school-based setting.

Model and approach for assessing implementation context and fidelity in the HEALing Communities Study

BACKGROUND

In response to the U.S. opioid epidemic, the HEALing (Helping to End Addiction Long-term^SM) Communities Study (HCS) is a multisite, wait-listed, community-level cluster-randomized trial that aims to test the novel Communities That HEAL (CTH) intervention, in 67 communities. CTH will expand an integrated set of evidence-based practices (EBPs) across health care, behavioral health, justice, and other community-based settings to reduce opioid overdose deaths. We present the rationale for and adaptation of the RE-AIM/PRISM framework and methodological approach used to capture the CTH implementation context and to evaluate implementation fidelity.

METHODS

HCS measures key domains of the internal and external CTH implementation context with repeated annual surveys and qualitative interviews with community coalition members and key stakeholders. Core constructs of fidelity include dosage, adherence, quality, and program differentiation-the adaptation of the CTH intervention to fit each community's needs. Fidelity measures include a monthly CTH checklist, collation of artifacts produced during CTH activities, coalition and workgroup attendance, and coalition meeting minutes. Training and technical assistance delivered by the research sites to the communities are tracked monthly.

DISCUSSION

To help attenuate the nation's opioid epidemic, the adoption of EBPs must be increased in communities. The HCS represents one of the largest and most complex implementation research experiments yet conducted. Our systematic examination of implementation context and fidelity will significantly advance understanding of how to best evaluate community-level implementation of EBPs and assess relations among implementation context, fidelity, and intervention impact.

Effectiveness of upgraded maternity waiting homes and local leader training on improving institutional births: a cluster-randomized controlled trial in Jimma, Ethiopia

Background

Maternity waiting homes (MWHs), residential spaces for pregnant women close to obstetric care facilities, are being used to tackle physical barriers to access. However, their effectiveness has not been rigorously assessed. The objective of this cluster randomized trial was to evaluate the effectiveness of functional MWHs combined with community mobilization by trained local leaders in improving institutional births in Jimma Zone, Ethiopia.

Methods

A pragmatic, parallel arm cluster-randomized trial was conducted in three districts. Twenty-four primary health care units (PHCUs) were randomly assigned to either (i) upgraded MWHs combined with local leader training on safe motherhood strategies, (ii) local leader training only, or (iii) usual care. Data were collected using repeat cross-sectional surveys at baseline and 21 months after intervention to assess the effect of intervention on the primary outcome, defined as institutional births, at the individual level. Women who had a pregnancy outcome (livebirth, stillbirth or abortion) 12 months prior to being surveyed were eligible for interview. Random effects logistic regression was used to evaluate the effect of the interventions.

Results

Data from 24 PHCUs and 7593 women were analysed using intention-to-treat. The proportion of institutional births was comparable at baseline between the three arms. At endline, institutional births were slightly higher in the MWH + training (54% [n = 671/1239]) and training only arms (65% [n = 821/1263]) compared to usual care (51% [n = 646/1271]). MWH use at baseline was 6.7% (n = 256/3784) and 5.8% at endline (n = 219/3809). Both intervention groups exhibited a non-statistically significant higher odds of institutional births compared to usual care (MWH⁺ & leader training odds ratio [OR] = 1.09, 97.5% confidence interval [CI] 0.67 to 1.75; leader training OR = 1.37, 97.5% CI 0.85 to 2.22).

Conclusions

Both the combined MWH⁺ & leader training and the leader training alone intervention led to a small but non-significant increase in institutional births when compared to usual care. Implementation challenges and short intervention duration may have hindered intervention effectiveness. Nevertheless, the observed increases suggest the interventions have potential to improve women’s use of maternal healthcare services. Optimal distances at which MWHs are most beneficial to women need to be investigated.

Trial registration

The trial was retrospectively registered on the Clinical Trials website (https://clinicaltrials.gov) on 3rd October 2017. The trial identifier is NCT03299491.

An Evaluation of the Reach and Effectiveness of a Diabetes Prevention Behaviour Change Program Situated in a Community Site

More than 350 million people are living with prediabetes. Preventing type 2 diabetes (T2D) progression can reduce morbidity, mortality and health-care costs. Interventions can support people with diet and physical activity behaviour changes; however, many interventions are university-based, posing barriers (e.g. accessibility, limited reach and maintenance), which highlight the need for community intervention. Limited research has comprehensively evaluated programs in community contexts. The purpose of this study was to pragmatically examine the reach and effectiveness of a diabetes prevention behaviour change program in the community using the RE-AIM framework. Demographic and outcome data were collected through telephone screening and survey data, and analyzed using descriptive and multivariate analyses. Over 2 years, 9,954 individuals were identified by a medical laboratory as living with prediabetes. Information letters were sent by the laboratory to individuals upon physician approval (N=2,241, 22.5%) as a main form of recruitment. From this, 271 individuals and an additional 160 individuals via other recruitment methods contacted the research team (N=431). Two hundred thirteen adults with prediabetes were enrolled (87.4% Caucasian, 69.7% female; 95% program completion). Analyses of 6-month follow-up data revealed significant maintenance of reductions in weight and waist circumference and improvements in physical function, self-reported physical activity and all-food frequency items except fruit intake (N=121, d=0.21 to 0.68, p <0.05 to 0.001). The program demonstrated diabetes risk-reducing benefits for enrolled individuals. Future work is needed to increase physician referral and participant response rates and to explore program expansion through digitization to reach more individuals at risk of developing T2D.

A systematic review of the implementation of obesity prevention interventions in early childcare and education settings using the RE-AIM framework

Previous systematic reviews have examined the efficacy of obesity prevention interventions within early childcare/education settings. Often lacking in these reviews is reporting on external validity, which continues to be underemphasized compared to internal validity. More attention to external validity would help better translate evidence-based interventions to real-world settings. This systematic review aimed to determine the availability of data on both internal and external validity across dimensions of the Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework in studies reporting on obesity prevention interventions in early childcare/education settings. Inclusion criteria included: randomized controlled trials, early childcare/education setting, targeted children 2-6 years old, addressed both diet and physical activity, collected measures of weight status and diet and/or physical activity, and published within the last 10 years. Searches were conducted in ERIC, PsychInfo, and PubMed; 23 studies met inclusion criteria. A validated RE-AIM abstraction tool was used to code studies. Most commonly reported dimensions were Reach (62.3%), Implementation (53.5%), and Efficacy/Effectiveness (48.7%). Adoption (21.7%) and Maintenance (11.6%) were less often reported. All studies reported on primary outcomes, but few reported on RE-AIM indicators of characteristics of participation and adoption, quality of life, methods used to identify staff, staff inclusion/exclusion criteria and adoption rates, implementation fidelity, measures of cost to start-up and deliver the intervention, and indicators of maintenance. This systematic review underscores the need for more focus on external validity to inform replication, dissemination, and implementation so that evidence-based early childcare/education obesity interventions can be generalized to real-world settings.

Implementing a Diabetes Education Program to Reduce Health Disparities in South Texas: Application of the RE-AIM Framework for Planning and Evaluation

Health disparities in diabetes management and control are well-documented. The objective of this study is to describe one diabetes education program delivered in the United States in terms of the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) Planning and Evaluation Framework. Questionnaires, clinical data, and administrative records were analyzed from 8664 adults with diabetes living in South Texas, an area characterized by high health disparities. The Diabetes Education Program delivered was a professionally led 12-month program involving 8 h of in-person workshop education followed by quarterly follow-up sessions. Changes in average blood glucose levels over the past 3 months (e.g., A1c levels) were the primary clinical outcome. Descriptive and multiple generalized linear mixed models were performed. This community-based initiative reached a large and diverse population, and statistically significant reductions in A1c levels (p < 0.01) were observed among participants with Type 2 diabetes at 3 months. These reductions in A1c levels were sustained at 6-, 9-, and 12-month follow-up assessments (p < 0.01). However, considerable attrition over time at follow-up sessions indicate the need for more robust strategies to keep participants engaged. For this diabetes education program, the RE-AIM model was a useful framework to present study processes and outcomes.

Implementing internet- and tele-based interventions to prevent mental health disorders in farmers, foresters and gardeners (ImplementIT): study protocol for the multi-level evaluation of a nationwide project

BACKGROUND

Farmers are a vulnerable population for developing depression or other mental health disorders due to a variety of risk factors in their work context. Beyond face-to-face resources, preventive internet- and tele-based interventions could extend available treatment options to overcome barriers to care. The German Social Insurance Company for Agriculture, Forestry and Horticulture (SVLFG) implements several guided internet- and mobile-based interventions and personalised tele-based coaching for this specific target group provided by external companies within a nation-wide prevention project for their insured members. The current study aims to evaluate the implementation process and to identify determinants of successful implementation on various individual and organisational levels.

METHODS

The current study includes two groups of participants: 1) insured persons with an observable need for prevention services, and 2) staff-participants who are involved in the implementation process. The Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework and the Consolidated Framework for Implementation Research (CFIR) will be used to track and evaluate the implementation process. A mixed-method approach will provide insights on individual and organizational level (e.g. degree of normalization, readiness for change) and helps to identify determinants of successful implementation. In-depth insights on experiences of the participants (e.g. acceptance, satisfaction, barriers and facilitating factors of intervention use) will be yielded through qualitative interviews. Focus groups with field workers provide insights into barriers and facilitators perceived during their consultations. Furthermore, intervention as well as implementation costs will be evaluated. According to the stepwise, national rollout, data collection will occur at baseline and continuously across 24 months.

DISCUSSION

The results will show to what extent the implementation of the internet- and tele-based services as a preventive offer will be accepted by the participants and involved employees and which critical implementation aspects will occur within the process. If the implementation of the internet- and tele-based services succeeds, these services may be feasible in the long-term.

TRIAL REGISTRATION

German Clinical Trial Registration: DRKS00017078 . Registered on 18.04.2019.

Evaluating the implementation of the PACE Steps to Success Programme in long-term care facilities in seven countries according to the RE-AIM framework

Background

The PACE ‘Steps to Success’ programme is a complex educational and development intervention for staff to improve palliative care in long-term care facilities (LTCFs). In a cluster randomized controlled trial, this programme has been implemented in 37 LTCFs in 7 European countries. Alongside an effectiveness study, a process evaluation study was conducted. This paper reports on the results of this process evaluation, of which the aim was to provide a more detailed understanding of the implementation of the PACE Programme across and within countries.

Methods

The process evaluation followed the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework and involved various measures and tools, including diaries for country trainers, evaluation questionnaires for care staff, attendance lists and interviews (online and face-to-face, individual and in groups) with country trainers, managers, PACE coordinators and other staff members. Based on key elements of the PACE Programme, a priori criteria for a high, medium and low level of the RE-AIM components Reach, Adoption, Implementation and intention to Maintenance were defined. Qualitative data on factors affecting each RE-AIM component gathered in the online discussion groups and interviews were analysed according to the principles of thematic analysis.

Results

The performance of the PACE Programme on the RE-AIM components was highly variable within and across countries, with a high or medium score for in total 28 (out of 37) LTCFs on Reach, for 26 LTCFs on Adoption, for 35 LTCFs on Implementation and for 34 LTCFs on intention to Maintenance. The factors affecting performance on the different RE-AIM components could be classified into three major categories: (1) the PACE Programme itself and its way of delivery, (2) people working with the PACE Programme and (3) contextual factors. Several country-specific challenges in implementing the PACE Programme were identified.

Conclusions

The implementation of the PACE Programme was feasible but leaves room for improvement. Our analysis helps to better understand the optimal levels of training and facilitation and provides recommendations to improve implementation in the LTC setting. The results of the process evaluation will be used to further adapt and improve the PACE Programme prior to its further dissemination.

Trial registration

The PACE study was registered at www.isrctn.com—ISRCTN14741671 (FP7-HEALTH-2013-INNOVATION-1 603111) July 30, 2015.

Effectiveness of upgraded maternity waiting homes and local leader training in improving institutional births among women in the Jimma zone, Ethiopia: study protocol for a cluster-randomized controlled trial

Background

Ethiopia is one of the ten countries in the world that together account for almost 60% of all maternal deaths. Recent reductions in maternal mortality have been seen, yet just 26% of women who gave birth in Ethiopia in 2016 reported doing so at a health facility. Maternity waiting homes (MWHs) have been introduced to overcome geographical and financial barriers to institutional births but there is no conclusive evidence as to their effectiveness. We aim to evaluate the effects of upgraded MWHs and local leader training in increasing institutional births in the Jimma zone of Ethiopia.

Methods

A parallel, three-arm, stratified, cluster-randomized controlled trial design is being employed to evaluate intervention effects on institutional births, which is the primary outcome. Trial arms are: (1) upgraded MWH + religious/community leader training; (2) leader training alone; and (3) standard care. Twenty-four primary health care unit catchment areas (clusters) have been randomized and 3840 women of reproductive age who had a pregnancy outcome (livebirth, stillbirth or abortion) are being randomly recruited for each survey round. Outcome assessments will be made using repeat cross-sectional surveys at baseline and 24 months postintervention. An intention to treat approach will be used and the primary outcome analysed using generalized linear mixed models with a random effect for cluster and time. A cost-effectiveness analysis will also be conducted from a societal perspective.

Discussion

This is one of the first trials to evaluate the effectiveness of upgraded MWHs and will provide much needed evidence to policy makers about aspects of functionality and the community engagement required as they scale-up this programme in Ethiopia.

Trial registration

ClinicalTrial.gov, NCT03299491. Retrospectively registered on 3 October 2017.

Scaling Out a 4-H Healthy Meeting Initiative: Challenges in Implementation and Comprehensive Evaluation

The national Cooperative Extension System delivers a number of nutrition and physical activity interventions, but it is unknown how an intervention may translate from 1 state within the system to another. Using the reach, effectiveness, adoption, implementation, maintenance framework for program evaluation can improve intervention scale-out. Adoption is a key dimension of reach, effectiveness, adoption, implementation, maintenance: if delivery personnel do not deliver an intervention, it can have no impacts on health. Here, differences are discussed regarding adoption rates between state Extension systems when scaling out a 4-H healthy meeting intervention. This experience provides suggestions for improved scale-out of Extension programs, including state-specific adaptation and pragmatic data collection.

Implementation of the Symptom Navi © Programme for cancer patients in the Swiss outpatient setting: a study protocol for a cluster randomised pilot study (Symptom Navi© Pilot Study)

INTRODUCTION

Self-management interventions show promising results on symptom outcomes and self-management behaviours. The Symptom Navi© Programme (SN©P) is a nurse-led intervention supporting patients' symptom self-management during anticancer treatment. It consists of written patient information (Symptom Navi© Flyers (SN©Flyers)), semistructured consultations and a training manual for nurses.

METHODS AND ANALYSIS

This pilot study will evaluate the implementation of the SN©P based on the Reach Effectiveness-Adoption Implementation Maintenance framework at Swiss outpatient cancer centres. We will use a cluster-randomised design and randomise the nine participating centres to the intervention or usual care group. We expect to include 140 adult cancer patients receiving first-line systemic anticancer treatment. Trained nurses at the intervention clusters will provide at least two semistructured consultations with the involvement of SN©Flyers. Outcomes include patients' accrual and retention rates, patient-reported interference of symptoms with daily functions, symptom burden, perceived self-efficacy, quality of nursing care, nurse-reported facilitators and barriers of adopting the programme, nurses' fidelity of providing the intervention as intended, and patients' safety (patients timely reporting of severe symptoms). We will use validated questionnaires for patient-reported outcomes, focus group interviews with nurses and individual interviews with oncologists. Linear mixed models will be used to analyse patient-reported outcomes. Focus group and individual interviews will be analysed by thematic analysis.

ETHICS AND DISSEMINATION

The Symptom Navi© Pilot Study has been reviewed and approved by Swiss Ethic Committee Bern (KEK-BE: 2017-00020). Results of the study will be disseminated in peer-reviewed journal and at scientific conferences.

TRIAL REGISTRATION NUMBER

NCT03649984; Pre-results.

Family-Centred Interventions for Elder Abuse: a Narrative Review

Information about elder abuse has not only been sparse due to factors such as underreporting or lack of standardized research, but also in defining elder abuse. However elder abuse is commonly understood to be a single/repeated act, or lack of appropriate action, occurring within any relationship which embodies an expectation of trust, which causes harm or distress to elderly individual. In elder abuse, the relationship of trust usually entails the family, as it family members who are noted to be the most likely perpetrators of elder abuse With the increased prevalence of elder abuse in developed countries and a proclivity towards individualized interventions, this narrative review sought to explore family-centered interventions used to address elder abuse by using a RE-AIM framework.

Walk This Way: Our Perspective on Challenges and Opportunities for Extension Statewide Walking Promotion Programs

Evidence-based walking programs exist and some have been tailored specifically for the national Cooperative Extension System; however, program outcomes and translational challenges and successes are underreported. This has presented a challenge to scaling the best-fit intervention for walking promotion within this national system. Here, we describe existing open-access walking programs as well as implications for improving the fit and sustainability of this intervention type within the system. Our experience provides suggestions for pragmatic data collection, infrastructure to support pragmatic data collection, and novel ways to disseminate best practices, as well as considerations for de-implementing what is not working.

RE-AIMing conferences: evaluating the adoption, implementation and maintenance of the Rick Hansen Institute's Praxis 2016

BACKGROUND

In April 2016, the Rick Hansen Institute (RHI) hosted an innovative, 2-day conference called Praxis 2016. RHI aimed to bring together a diverse group of stakeholders to develop solutions for overcoming the challenges of translating spinal cord injury (SCI) research into practice. To understand the impact of Praxis, RHI funded an independent team to evaluate Praxis at the individual and setting level using the RE-AIM framework. Individual-level findings are published elsewhere. The aim of this evaluation is to report on the impact of Praxis at the setting level in terms of its adoption, implementation and maintenance.

METHODS

Data sources included interviews with attendees (n = 13) and organisers (n = 9), a fidelity assessment conducted at the conference, and observation notes provided by the evaluation team.

RESULTS

Main findings indicated that the Praxis model was adopted by organisers and attendees, implemented by RHI as intended, and has the potential for long-term maintenance. Lessons learned highlighted the importance of including SCI community members throughout the entire process from development to post-conference activities as well as in the research process, the value of facilitation and fostering interactive problem solving, and the need to identify leadership and funds to foster long-term efforts.

CONCLUSIONS

Developing and implementing a solutions-focused conference that brings together a diverse group of SCI stakeholders was challenging and rewarding for attendees and organisers. Other domains could learn from, adopt and build on the Praxis 2016 approach to address research-to-practice gaps.

Assessment of the Cambodian National Biodigester Program

The National Biodigester Program (NBP) was established in 2006 to build and maintain household biodigesters in Cambodia. In its first six years, the program installed almost 20,000 biodigesters and established an in-country network of local financiers, construction companies, skilled masons, bioslurry specialists and after-care technicians accessible to rural users in 14 of 24 Cambodian provinces. Since its initial success, the program's adoption rates have stalled despite increasing government support and high rates of user satisfaction. Building on an initial evaluation of the NBP in 2013, this updated assessment identifies multiple changes in its second implementation phase that have undermined the initial momentum of the program. Abrupt interruptions in institutional support, deteriorating supply side services (access to construction agents, masons, repair services) and reduced access to credit for farmers have eroded the service network that the first implementation phase established. Structural changes in the rural economy may also contribute to declining demand. Government support to another biodigester program which offers a lower investment price, but does not provide after-sales services has also undercut the long-term implementation strategy of the NBP. The paper finds that despite these programmatic changes, the installed biodigesters continue to perform according to expectation and to be maintained and valued by their users, but the future viability of the program remains uncertain.

Ghana's Rural Liquefied Petroleum Gas Program Scale Up: A case study

BACKGROUND

The Government of Ghana launched the Rural LPG (RLP) promotion program in 2013 as part of its efforts to reduce fuelwood consumption. The aim of the RLP is to contribute to Ghana's overarching goal to provide LPG access to 50% of Ghana's population by 2020. The RLP has not announced long-term program objectives. However, in the interim the RLP targeted a cumulative total of 170,000 LPG cookstoves to rural households by the end of 2017. As of November 2017, 149,500 rural households had received the LPG cook stoves. Our case study documents Ghana's experiences to date with LPG scale up.

METHODS

We carried out a desktop review/document analysis of literature on the RLP. Each document was reviewed for information related to the elements of the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework as it pertained to LPG promotion and adoption. In-depth interviews were held among key stakeholders in Ghana. Previously collected data from a field evaluation of the RLP was also assessed.

FINDINGS

Generally, our evaluation suggests that the current form of the RLP is not achieving its stated goal. Our evaluation of the RLP in five rural communities showed that about 58% of households had never refilled their LPG cylinders nine months after the initial delivery of a filled cylinder. Only 8% still used their LPG at 18 months post distribution. Cost and distance to LPG filling stations were the main reasons for low LPG use. Beneficiaries did not exclusively use their LPG even at the initial stages when all of them had LPG in their cylinders. Ghana is currently undergoing transitions in the LPG sector including a change from the current private cylinder ownership model to a cylinder recirculation model for the distribution of LPG. There was no evidence of a well-documented implementation framework for the RLP.

CONCLUSION

Fuel cost, poor LPG access, and an inadequate implementation framework hinder the RLP implementation.

A group-based lifestyle intervention for diabetes prevention in low- and middle-income country: implementation evaluation of the Kerala Diabetes Prevention Program

BACKGROUND

While several efficacy trials have demonstrated diabetes risk reduction through targeting key lifestyle behaviours, there is a significant evidence gap in relation to the successful implementation of such interventions in low- and middle-income countries (LMICs). This paper evaluates the implementation of a cluster randomised controlled trial of a group-based lifestyle intervention among individuals at high-risk of developing type 2 diabetes mellitus (T2DM) in the state of Kerala, India. Our aim is to uncover provider-, participant- and community-level factors salient to successful implementation and transferable to other LMICs.

METHODS

The 12-month intervention program consisted of (1) a group-based peer-support program consisting of 15 sessions over a period of 12 months for high-risk individuals, (2) peer leader (PL) training and ongoing support for intervention delivery, (3) diabetes education resource materials and (4) strategies to stimulate broader community engagement. The evaluation was informed by the RE-AIM and PIPE frameworks.

RESULTS

Provider-level factors: Twenty-nine (29/30, 97%) intervention groups organised all 15 sessions. A 2-day PL training was attended by 51(85%) of 60 PLs. The PL handbook was found to be 'very useful' by 78% of PLs. Participant-level factors: Of 1327 eligible individuals, 1007(76%) participants were enrolled. On average, participants attended eight sessions. Sixty-eight percent rated their interest in group sessions as 'very interested', and 55% found the group sessions 'very useful' in making lifestyle changes. Inconvenient time (43%) and location (21%) were found to be important barriers for participants who did not attend any sessions. Community-level factors: Community-based activities reached to 41% of the participants for walking groups, 40% for kitchen garden training, and 31% for yoga training. PLs were readily available for support outside the sessions, as 75% of participants reported extracurricular contacts with their PLs. The commitment from the local partner institute and political leaders facilitated the high uptake of the program.

CONCLUSION

A comprehensive evaluation of program implementation from the provider-, participant- and community-level perspectives demonstrates that the K-DPP program was feasible and acceptable in changing lifestyle behaviours in high-risk individuals. The findings from this evaluation will guide the future delivery of structured lifestyle modification diabetes programs in LMICs.

TRIAL REGISTRATION

Trial registration: Australia and New Zealand Clinical Trials Registry ACTRN12611000262909 . Registered 10 March 2011.

Impact of the Canadian Diabetes Association guideline dissemination strategy on clinician knowledge and behaviour change outcomes

AIM

Implementation of clinical practice guideline (CPG) into clinical practice remains limited. Using the Knowledge-To-Action framework, a guideline dissemination and implementation strategy for the Canadian Diabetes Association's 2013 CPG was developed and launched to clinicians and people with diabetes.

METHODS

The RE-AIM framework guided evaluation of this strategy clinician; we report here one aspect of the effectiveness dimension using mixed methods. We measured impact of the strategy on clinican knowledge and behaviour change constructs using evaluation forms, national online survey and individual interviews.

RESULTS

After attending a lecture, clinician confidence (n = 915) increased (3.7(SD 0.7) to 4.5 (SD 0.6) on a 5-point scale (p < 0.001)), with 55% (n = 505) intending to make a practice change (e.g. clinical management regarding glycemic control). Ninety-four percent of survey respondents (n = 907) were aware of the guidelines, attributed to communications from professional associations, continuing professional development events, and colleagues. Forty to 98% of respondents (total n 462-485) were correct in their interpretation of CPG messages, and 33-65%(total n 351-651) reported that they had made changes to their practice. Interviews with 28 clinicians revealed that organizational credibility, online access to tools, clarity of tool content, and education sessions facilitated uptake; lack of time, team-based consensus, and seamless integration into care and patient complexity were barriers.

CONCLUSION

The complexity of diabetes care requires systemic adoption of organization of care interventions, including interprofessional collaboration and consensus. Augmenting our strategy to include scalable models for professional development, integration of guidelines into electronic medical records, and expansion of our target audience to include health care teams and patients, may optimize guideline uptake.

Systematic, Multimethod Assessment of Adaptations Across Four Diverse Health Systems Interventions

Background

Many health outcomes and implementation science studies have demonstrated the importance of tailoring evidence-based care interventions to local context to improve fit. By adapting to local culture, history, resources, characteristics, and priorities, interventions are more likely to lead to improved outcomes. However, it is unclear how best to adapt evidence-based programs and promising innovations. There are few guides or examples of how to best categorize or assess health-care adaptations, and even fewer that are brief and practical for use by non-researchers.

Materials and methods

This study describes the importance and potential of assessing adaptations before, during, and after the implementation of health systems interventions. We present a promising multilevel and multimethod approach developed and being applied across four different health systems interventions. Finally, we discuss implications and opportunities for future research.

Results

The four case studies are diverse in the conditions addressed, interventions, and implementation strategies. They include two nurse coordinator-based transition of care interventions, a data and training-driven multimodal pain management project, and a cardiovascular patient-reported outcomes project, all of which are using audit and feedback. We used the same modified adaptation framework to document changes made to the interventions and implementation strategies. To create the modified framework, we started with the adaptation and modification model developed by Stirman and colleagues and expanded it by adding concepts from the RE-AIM framework. Our assessments address the intuitive domains of Who, How, When, What, and Why to classify and organize adaptations. For each case study, we discuss how the modified framework was operationalized, the multiple methods used to collect data, results to date and approaches utilized for data analysis. These methods include a real-time tracking system and structured interviews at key times during the intervention. We provide descriptive data on the types and categories of adaptations made and discuss lessons learned.

Conclusion

The multimethod approaches demonstrate utility across diverse health systems interventions. The modified adaptations model adequately captures adaptations across the various projects and content areas. We recommend systematic documentation of adaptations in future clinical and public health research and have made our assessment materials publicly available.

Peer support to improve diabetes care: an implementation evaluation of the Australasian Peers for Progress Diabetes Program

BACKGROUND

Several studies have now demonstrated the benefits of peer support in promoting diabetes control. The aim of this study is to evaluate the implementation of a cluster randomised controlled trial of a group-based, peer support program to improve diabetes self-management and thereby, diabetes control in people with Type 2 Diabetes in Victoria, Australia.

METHODS

The intervention program was designed to address four key peer support functions i.e. 1) assistance in daily management, 2) social and emotional support, 3) regular linkage to clinical care, and 4) ongoing and sustained support to assist with the lifelong needs of diabetes self-care management. The intervention participants attended monthly group meetings facilitated by a trained peer leader for 12 months. Data was collected on the intervention's reach, participation, implementation fidelity, groups' effectiveness and participants' perceived support and satisfaction with the intervention. The RE-AIM and PIPE frameworks were used to guide this evaluation.

RESULTS

The trial reached a high proportion (79%) of its target population through mailed invitations. Out of a total of 441 eligible individuals, 273 (61.9%) were willing to participate. The intervention fidelity was high (92.7%). The proportion of successful participants who demonstrated a reduction in 5 years cardiovascular disease risk score was 65.1 and 44.8% in the intervention and control arm respectively. Ninety-four percent (94%) of the intervention participants stated that the program helped them manage their diabetes on a day to day basis. Overall, attending monthly group meetings provided 'a lot of support' to 57% and 'moderate' support to 34% of the participants.

CONCLUSION

Peer support programs are feasible, acceptable and can be used to supplement treatment for patients motivated to improve behaviours related to diabetes. However, program planners need to focus on the participation component in designing future programs. The use of two evaluation frameworks allowed a comprehensive evaluation of the trial from the provider-, participant- and public health perspective. The learnings gained from this evaluation will guide and improve future implementation by improving program feasibility for adoption and acceptability among participants, and will ultimately increase the likelihood of program effectiveness for the participants.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry (ANZCTR) ACTRN12609000469213 . Registered 16 June 2009.