Beyond effectiveness: evaluating the public health impact of the WISEWOMAN program

A continuity of care programme for women at risk of preterm birth in the UK: Process evaluation of a hybrid randomised controlled pilot trial

BACKGROUND

The development and evaluation of specific maternity care packages designed to address preterm birth remains a public health priority. We aim to evaluate the implementation, context, and potential mechanisms of action, of a new care pathway that combined midwifery continuity of care with a specialist obstetric clinic for women at risk of preterm birth (POPPIE) in London (UK).

METHODS

We did a multiphase mixed method triangulation evaluation nested within a hybrid type 2, randomised controlled trial in London (United Kingdom). Pregnant women with identified risk factors for preterm birth were eligible for trial participation and randomly assigned (1:1) to either midwifery continuity of care linked to a specialist obstetric clinic (POPPIE group) or standard maternity care. The primary outcome was a composite of appropriate and timely interventions for the prevention and/or management of preterm labour and birth, analysed according to intention to treat. Clinical and process outcome data were abstracted from medical records and electronic data systems, and coded by study team members, who were masked to study group allocation. Implementation data were collected from meeting records and key documents, postnatal surveys (n = 164), semi-structured interviews with women (n = 30), healthcare providers and stakeholders (n = 24) pre-, mid and post implementation. Qualitative and quantitative data from meeting records and key documents were examined narratively. Qualitative data from interviews were analysed using three thematic frameworks: Proctor's (for implementation outcomes: appropriateness, adoption, feasibility, acceptability, fidelity, penetration, sustainability), the Consolidated Framework for Implementation Research (for determinants of implementation), and published program theories of continuity models (for potential mechanisms). Data triangulation followed a convergent parallel and pragmatic approach which brought quantitative and qualitative data together at the interpretation stage. We averaged individual implementation measures across all domains to give a single composite implementation strength score which was compared to the primary outcome.

RESULTS

Between May 9, 2017, and Sep 30, 2018, 553 women were assessed for eligibility and 334 were enrolled with less than 6% of loss to follow up (169 were assigned to the POPPIE group; 165 were to the standard group). There was no difference in the primary outcome (POPPIE group 83·3% versus standard group 84·7%; risk ratio 0·98 [95% CI 0·90 to 1·08]). Appropriateness and adoption: The introduction of the POPPIE model was perceived as a positive fundamental change for local maternity services. Partnership working and additional funding were crucial for adoption. Fidelity: More than 75% of antenatal and postnatal visits were provided by a named or partner midwife, and a POPPIE midwife was present in more than 80% of births. Acceptability: Nearly 98% of women who responded to the postnatal survey were very satisfied with POPPIE model. Quantitative fidelity and acceptability results were supported by the qualitative findings. Penetration and sustainability: Despite delays (likely associated with lack of existing continuity models at the hospital), the model was embedded within established services and a joint decision was made to sustain and adapt the model after the trial (strongly facilitated by national maternal policy on continuity pathways). Potential mechanisms of impact identified included e.g. access to care, advocacy and perceptions of safety and trust. There was no association between implementation measures and the primary outcome.

CONCLUSIONS

The POPPIE model of care was a feasible and acceptable model of care that was implemented with high fidelity and sustained in maternity services. Larger powered trials are feasible and needed in other settings, to evaluate the impact and implementation of continuity programmes in other communities affected by preterm birth and women who experience social disadvantage and vulnerability.

TRIAL REGISTRATION

UKCRN Portfolio Database (prospectively registered, 24 April 2017): 31951. ISRCTN registry (retrospectively registered, 21 August 2017): ISRCTN37733900.

Using the RE-AIM framework to evaluate the implementation of scaling-up the Friendship Bench in Zimbabwe - a quantitative observational study

BACKGROUND

This study aimed to evaluate the real-world implementation of the Friendship Bench (FB) - an evidence-based brief psychological intervention delivered by community health workers (CHWs) - three years after its implementation in three city health departments in Zimbabwe. Implementation sites were evaluated according to their current performance using the RE-AIM framework making this one of the first evaluations of a scaled-up evidence-based psychological intervention in sub-Saharan Africa (SSA).

METHODS

Using the RE-AIM guide ( www.re-aim.org ), the authors designed quantitative indicators based on existing FB implementation data. Thirty-six primary health care clinics (PHC) in Harare (n=28), Chitungwiza (n=4) and Gweru (n=4) were included. Among these clinics 20 were large comprehensive health care centers, 7 medium (mostly maternal and child healthcare) and 9 small clinics (basic medical care and acting as referral clinic). Existing data from these clinics, added to additionally collected data through interviews and field observations were used to investigate and compare the performance of the FB across clinics. The focus was on the RE-AIM domains of Reach, Adoption, and Implementation.

RESULTS

Small clinics achieved 34% reach, compared to large (15%) and medium clinics (9%). Adoption was high in all clinic types, ranging from 59% to 71%. Small clinics led the implementation domain with 53%, followed by medium sized clinics 43% and large clinics 40%. Small clinics performed better in all indicators and differences in performance between small and large clinics were significant. Program activity and data quality depends on ongoing support for delivering agents and buy-in from health authorities.

CONCLUSION

The Friendship Bench program was implemented over three years transitioning from a research-based implementation program to one led locally. The Reach domain showed the largest gap across clinics where larger clinics performed poorly relative to smaller clinics and should be a target for future implementation improvements. Program data needs to be integrated into existing health information systems. Future studies should seek to optimize scale-up and sustainment strategies to maintain effective task-shared psychological interventions in SSA.

Optimising implementation strategies of the first scaleup of a primary care psychological intervention for common mental disorders in Sub-Saharan Africa: a mixed methods study protocol for the optimised Friendship Bench (OptFB)

INTRODUCTION

Common mental disorders (CMDs) are a leading cause of disability globally. CMDs are highly prevalent in Zimbabwe and have been addressed by an evidence-based, task-shifting psychological intervention called the Friendship Bench (FB). The task-shifted FB programme guides clients through problem-solving therapy. It was scaled up across 36 implementation sites in Zimbabwe in 2016.

METHODS AND ANALYSIS

This study will employ a mixed-method framework. It aims to: (1) use quantitative survey methodologies organised around the Reach, Effectiveness, Adoption and Implementation and Maintenance evaluation framework to assess the current scaleup of the FB intervention and classify 36 clinics according to levels of performance; (2) use qualitative focus group discussions and semistructured interviews organised around the Consolidated Framework for Implementation Research to analyse determinants of implementation success, as well as elucidate heterogeneity in implementation strategies through comparing high-performing and low-performing clinics; and (3) use the results from aims 1 and 2 to develop strategies to optimise the Friendship Bench intervention and apply this model in a cluster randomised controlled trial to evaluate potential improvements among low-performing clinics. The trial will be registered with the Pan African Clinical Trial Registry (www.pactr.org). The planned randomised controlled trial for the third research aim will be registered after completing aims one and two because the intervention is dependent on knowledge generated during these phases.

ETHICS AND DISSEMINATION

The research protocol received full authorisation from the Medical Research Council of Zimbabwe (MRCZ A/242). It is anticipated that changes in data collection tools and consent forms will take place at all three phases of the study and approval from MRCZ will be sought. All interview partners will be asked for informed consent. The research team will prioritise open-access publications to disseminate research results.

Process Evaluation of Implementing a Pharmacist-Led Intervention to Improve Adherence to Antihypertensive Drugs Among Patients with Type 2 Diabetes in Indonesian Community Health Centers

Introduction: A pharmacist-led intervention in Community Health Centers (CHCs) in Indonesia targeted at patients with type 2 diabetes non-adherent to antihypertensive drugs resulted in a significant improvement in adherence to these drugs. The aim of this study was to evaluate the process of implementation this intervention intended to improve adherence to antihypertensive drugs from both the pharmacist and the patient perspective. Methods: Using the RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) framework, we conducted a focus group among pharmacists (N = 5) and a survey among patients with complete follow-up (N = 44) participating in the intervention group. Results: All pharmacists adopted the provided training and found support tools useful. The pharmacists implemented the intervention as intended (adequate intervention fidelity >69%). Factors relevant for implementation included having sufficient time and confidence, home visits for specific patients, multidisciplinary collaboration, and availability of a personal counseling room. To maintain the intervention, the need for practical guidance and support from health care authorities was mentioned. Most patients (96%) were satisfied with the information provided by the pharmacists and they believed the tailored counselling was helpful. Most patients (84%) reported that the duration of counselling was sufficient. The large majority of patients would like to receive the counselling regularly. Conclusion: Positive effects of the pharmacist-led intervention can be explained by adequate levels of reach, adoption and implementation in the participating CHCs. For successful implementation and maintenance in Indonesia or other low-and middle-income countries, sufficient training, resources, multidisciplinary collaboration, guidance and support from health care authorities are expected to be important.

Nation-Wide Dissemination of a Digital Checklist to Improve Work Environment in the Eldercare Sector in Denmark

In this study, we evaluated the dissemination of a digital checklist for improving implementation of work environment initiatives in the Danish eldercare sector. We evaluated the impact of the checklist using the RE-AIM framework. Initiated in 2016, researchers and relevant stakeholders were responsible for disseminating the checklist to all workplaces in the eldercare sector in Denmark through a national campaign. The checklist guided the user to define an action plan to implement, and the checklist covered 11 implementation concept points that should be addressed to reach full implementation of the action in focus. One year after the launch of the campaign almost all municipalities in Denmark had visited the website hosting the checklist (96%), 17% of individual workers within the eldercare responding to a union survey was reached, 4% (n = 199) of all eligible eldercare workplaces in Denmark and 8% of all nursing homes had adopted the checklist. Of the workplaces that used the checklist, 46% typed an action in the checklist. There were 13% of the first time users that used the checklist twice and 29% of the actions were revised (maintenance) after working with the implementation. Finally, the workplaces that had used the checklist showed a higher prioritization of work environment compared to workplaces not using the checklist both at baseline and at follow up. In conclusion, this study employing various strategies, including a 1-year national campaign to disseminate a checklist shows potential to impact implementation of work environment initiatives in the Danish eldercare sector. While dissemination is satisfactory and likely to increase further with time, more efforts is needed to ensure maintenance.

Implementation and scale-up of physical activity and behavioural nutrition interventions: an evaluation roadmap

BACKGROUND

Interventions that work must be effectively delivered at scale to achieve population level benefits. Researchers must choose among a vast array of implementation frameworks (> 60) that guide design and evaluation of implementation and scale-up processes. Therefore, we sought to recommend conceptual frameworks that can be used to design, inform, and evaluate implementation of physical activity (PA) and nutrition interventions at different stages of the program life cycle. We also sought to recommend a minimum data set of implementation outcome and determinant variables (indicators) as well as measures and tools deemed most relevant for PA and nutrition researchers.

METHODS

We adopted a five-round modified Delphi methodology. For rounds 1, 2, and 3 we administered online surveys to PA and nutrition implementation scientists to generate a rank order list of most commonly used; i) implementation and scale-up frameworks, ii) implementation indicators, and iii) implementation and scale-up measures and tools. Measures and tools were excluded after round 2 as input from participants was very limited. For rounds 4 and 5, we conducted two in-person meetings with an expert group to create a shortlist of implementation and scale-up frameworks, identify a minimum data set of indicators and to discuss application and relevance of frameworks and indicators to the field of PA and nutrition.

RESULTS

The two most commonly referenced implementation frameworks were the Framework for Effective Implementation and the Consolidated Framework for Implementation Research. We provide the 25 most highly ranked implementation indicators reported by those who participated in rounds 1-3 of the survey. From these, the expert group created a recommended minimum data set of implementation determinants (n = 10) and implementation outcomes (n = 5) and reconciled differences in commonly used terms and definitions.

CONCLUSIONS

Researchers are confronted with myriad options when conducting implementation and scale-up evaluations. Thus, we identified and prioritized a list of frameworks and a minimum data set of indicators that have potential to improve the quality and consistency of evaluating implementation and scale-up of PA and nutrition interventions. Advancing our science is predicated upon increased efforts to develop a common 'language' and adaptable measures and tools.

Exploring the effect of implementation and context on a stepped-wedge randomised controlled trial of a vital sign triage device in routine maternity care in low-resource settings

Background

Interventions aimed at reducing maternal mortality are increasingly complex. Understanding how complex interventions are delivered, to whom, and how they work is key in ensuring their rapid scale-up. We delivered a vital signs triage intervention into routine maternity care in eight low- and middle-income countries with the aim of reducing a composite outcome of morbidity and mortality. This was a pragmatic, hybrid effectiveness-implementation stepped-wedge randomised controlled trial. In this study, we present the results of the mixed-methods process evaluation. The aim was to describe implementation and local context and integrate results to determine whether differences in the effect of the intervention across sites could be explained.

Methods

The duration and content of implementation, uptake of the intervention and its impact on clinical management were recorded. These were integrated with interviews (n = 36) and focus groups (n = 19) at 3 months and 6–9 months after implementation. In order to determine the effect of implementation on effectiveness, measures were ranked and averaged across implementation domains to create a composite implementation strength score and then correlated with the primary outcome.

Results

Overall, 61.1% (n = 2747) of health care providers were trained in the intervention (range 16.5% to 89.2%) over a mean of 10.8 days. Uptake and acceptability of the intervention was good. All clusters demonstrated improved availability of vital signs equipment. There was an increase in the proportion of women having their blood pressure measured in pregnancy following the intervention (79.2% vs. 97.6%; OR 1.30 (1.29–1.31)) and no significant change in referral rates (3.7% vs. 4.4% OR 0.89; (0.39–2.05)). Availability of resources and acceptable, effective referral systems influenced health care provider interaction with the intervention. There was no correlation between process measures within or between domains, or between the composite score and the primary outcome.

Conclusions

This process evaluation has successfully described the quantity and quality of implementation. Variation in implementation and context did not explain differences in the effectiveness of the intervention on maternal mortality and morbidity. We suggest future trials should prioritise in-depth evaluation of local context and clinical pathways.

Trial registration

Trial registration: ISRCTN41244132. Registered on 2 Feb 2016

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0885-3) contains supplementary material, which is available to authorized users.

An integrated community health worker intervention in rural Nepal: a type 2 hybrid effectiveness-implementation study protocol

Background

Evidence-based medicines, technologies, and protocols exist to prevent many of the annual 300,000 maternal, 2.7 million neonatal, and 9 million child deaths, but they are not being effectively implemented and utilized in rural areas. Nepal, one of South Asia’s poorest countries with over 80% of its population living in rural areas, exemplifies this challenge. Community health workers are an important cadre in low-income countries where human resources for health and health care infrastructure are limited. As local women, they are uniquely positioned to understand and successfully navigate barriers to health care access. Recent case studies of large community health worker programs have highlighted the importance of training, both initial and ongoing, and accountability through structured management, salaries, and ongoing monitoring and evaluation. A gap in the evidence regarding whether such community health worker systems can change health outcomes, as well as be sustainably adopted at scale, remains. In this study, we plan to evaluate a community health worker system delivering an evidence-based integrated reproductive, maternal, newborn, and child health intervention as it is scaled up in rural Nepal.

Methods

We will conduct a type 2 hybrid effectiveness-implementation study to test both the effect of an integrated reproductive, maternal, newborn, and child health intervention and the implementation process via a professional community health worker system. The intervention integrates five evidence-based approaches: (1) home-based antenatal care and post-natal care counseling and care coordination; (2) continuous surveillance of all reproductive age women, pregnancies, and children under age 2 years via a mobile application; (3) Community-Based Integrated Management of Newborn and Childhood Illness; (4) group antenatal and postnatal care; and 5) the Balanced Counseling Strategy to post-partum contraception. We will evaluate effectiveness using a pre-post quasi-experimental design with stepped implementation and implementation using the RE-AIM framework.

Discussion

This is the first hybrid effectiveness-implementation study of an integrated reproductive, maternal, newborn, and child health intervention in rural Nepal that we are aware of. As Nepal takes steps towards achieving the Sustainable Development Goals, the data from this three-year study will be useful in the detailed planning of a professionalized community health worker cadre delivering evidence-based reproductive, maternal, newborn, and child health interventions to the country’s rural population.

Trial registration

ClinicalTrials.gov Identifier: NCT03371186, registered 04 December 2017, retrospectively registered.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0741-x) contains supplementary material, which is available to authorized users.

Healthy Eating for Life English as a second language curriculum: applying the RE-AIM framework to evaluate a nutrition education intervention targeting cancer risk reduction

Medically underserved US immigrants are at an increased risk for death from preventable or curable cancers due to economic, cultural, and/or linguistic barriers to medical care. The purpose of this study was to describe the evaluation of the pilot study of the Healthy Eating for Life (HE4L) English as a second language curriculum. The Reach, Effectiveness Adoption, Implementation, Maintenance (RE-AIM) model was used to design a mixed-methods approach to the evaluation of the HE4L curriculum. Successful implementation was dependent upon enthusiastic teacher and manager support of the curriculum, teachers' ability to flexibly apply the curriculum to meet student needs, and researcher provision of curriculum workbooks. HE4L can be implemented successfully in various adult education settings to teach healthy eating behaviors and English language principles. Scale-up of HE4L may depend on the development of an online version of the curriculum to avoid the costs associated with printing and distributing curriculum materials.

Screening and vaccination as determined by the Social Ecological Model and the Theory of Triadic Influence: a systematic review

BACKGROUND

Vaccination and screening are forms of primary and secondary prevention methods. These methods are recommended for controlling the spread of a vast number of diseases and conditions. To determine the most effective preventive methods to be used by a society, multi-level models have shown to be more effective than models that focus solely on individual level characteristics. The Social Ecological Model (SEM) and the Theory of Triadic Influence (TTI) are such models. The purpose of this systematic review was to identify main differences and similarities of SEM and TTI regarding screening and vaccination in order to prepare potentially successful prevention programs for practice.

METHODS

A systematic review was conducted. Separate literature searches were performed during January and February 2015 using Medline, Ovid, Proquest, PubMed, University of Antwerp Discovery Service and Web of Science, for articles that apply the SEM and TTI. A Data Extraction Form with mostly closed-end questions was developed to assist with data extraction. Aggregate descriptive statistics were utilized to summarize the general characteristics of the SEM and TTI as documented in the scientific literature.

RESULTS

A total of 290 potentially relevant articles referencing the SEM were found. As for the TTI, a total of 131 potentially relevant articles were found. After strict evaluation for inclusion and exclusion criteria, 40 SEM studies and 46 TTI studies were included in the systematic review.

CONCLUSIONS

The SEM and TTI are theoretical frameworks that share many theoretical concepts and are relevant for several types of health behaviors. However, they differ in the structure of the model, and in how the variables are thought to interact with each other, the TTI being a matrix while the SEM has a ring structure. The main difference consists of the division of the TTI into levels of causation (ultimate, distal and proximal) which are not considered within the levels of the SEM. It was further found that in the articles studied in this systematic review, both models are often considered effective, while the empirical basis of these (and other) conclusions reached by their authors is in many cases unclear or incompletely specified.

Framework for Selecting Best Practices in Public Health: A Systematic Literature Review

Evidence-based public health has commonly relied on findings from empirical studies, or research-based evidence. However, this paper advocates that practice-based evidence derived from programmes implemented in real-life settings is likely to be a more suitable source of evidence for inspiring and guiding public health programmes. Selection of best practices from the array of implemented programmes is one way of generating such practice-based evidence. Yet the lack of consensus on the definition and criteria for practice-based evidence and best practices has limited their application in public health so far. To address the gap in literature on practice-based evidence, this paper hence proposes measures of success for public health interventions by developing an evaluation framework for selection of best practices. The proposed framework was synthesised from a systematic literature review of peer-reviewed and grey literature on existing evaluation frameworks for public health programmes as well as processes employed by health-related organisations when selecting best practices. A best practice is firstly defined as an intervention that has shown evidence of effectiveness in a particular setting and is likely to be replicable to other situations. Regardless of the area of public health, interventions should be evaluated by their context, process and outcomes. A best practice should hence meet most, if not all, of eight identified evaluation criteria: relevance, community participation, stakeholder collaboration, ethical soundness, replicability, effectiveness, efficiency and sustainability. Ultimately, a standardised framework for selection of best practices will improve the usefulness and credibility of practice-based evidence in informing evidence-based public health interventions. Significance for public healthBest practices are a valuable source of practice-based evidence on effective public health interventions implemented in real-life settings. Yet, despite the frequent branding of interventions as best practices or good practices, there is no consensus on the definition and desirable characteristics of such best practices. Hence, this is likely to be the first systematic review on the topic of best practices in public health. Having a single widely accepted framework for selecting best practices will ensure that the selection processes by different agencies are fair and comparable, as well as enable public health workers to better appreciate and adopt best practices in different settings. Ultimately, standardisation will improve the credibility and usefulness of practice-based evidence to that of research-based evidence.

Applying the RE-AIM Framework to Evaluate the Dissemination and Implementation of Clinical Practice Guidelines for Sexually Transmitted Infections

Clinical practice guidelines (CPG) are one of the most effective ways to translate evidence of medical improvement into everyday practice. This study evaluated the dissemination and implementation of the Sexually Transmitted Infections-Korean Guidelines (STIKG) by applying the reach, effectiveness, adoption, implementation and maintenance (RE-AIM) framework. A survey questionnaire was administered to clinicians via the internet. Among the 332 respondents, 190 (57.2%) stated that they were aware of STIKG and 107 (33.2%) implemented STIKG in their practice. The odds that a physician was exposed to STIKG (dissemination) were 2.61 times greater among physicians with previous training or education for any CPG than those who did not. Clinicians who indicated that STIKG were easy to understand were 4.88 times more likely to implement STIKG in their practice than those who found them not so easy. When a clinician's workplace had a supporting system for CPG use, the odds of implementation was 3.76 times higher. Perceived level of effectiveness of STIKG did not significantly influence their implementation. The findings of this study suggest that, ultimately, knowing how to engage clinicians in CPG implementation is as important as how to disseminate such guidelines; moreover, easy-to-use guidelines and institutional support are key factors.

Network analysis of RE-AIM framework: chronology of the field and the connectivity of its contributors

The reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework has been widely used for translational research. We used social network analysis (SNA) to explore how innovative research frameworks, such as RE-AIM, have diffused over time in academic literature. A structured literature review was conducted on RE-AIM between 1999 and 2012. SNA indices of degree score, betweenness, centrality, and authorship ties were used to examine use of RE-AIM. Use of RE-AIM has grown since its inception and spread from a few research centers to use internationally. Investigation of co-authorship revealed many have published on RE-AIM, but a much smaller core of RE-AIM researchers have published together two or more times. SNA revealed how the RE-AIM framework has been used over time and identified areas to further expand use of the framework. SNA can be useful to understand how research frameworks diffuse over time.

The StrongWomen-Healthy Hearts program in Pennsylvania: RE-AIM analysis

Dissemination of evidence-based programs is needed to reduce CVD risk among midlife and older women. The aim of this study is to examine the public health impact of StrongWomen-Healthy Hearts in Pennsylvania using the RE-AIM framework. Reach, adoption, implementation, and maintenance were assessed using qualitative and quantitative measures; effectiveness was assessed using a pretest-posttest within-participants design. Reach into the target population was 5 in 100,000. Compared to the target population, a greater percentage of participants were white, married, middle-class, and had a graduate degree. Effectiveness was demonstrated (weight loss -2.0 kg, p < 0.001). Adoption among trained leaders was high (83.3 %), as was fidelity in implementation (average score 9.3 of 10). No leaders maintained the program. To increase impact of the StrongWomen-Healthy Hearts Program, it will be important to lower the costs and modify the recruitment and training strategies to better reach low-income and minority women. Such strategies may also improve program maintenance.

Evaluating the impact of equity focused health impact assessment on health service planning: three case studies

BACKGROUND

Health impact assessment has been identified internationally as a mechanism to ensure potential health impacts and health equity impacts of proposals are considered before implementation. This paper looks at the impact of three equity focused health impact assessments (EFHIAs) of health service plans on subsequent decision-making and implementation, and then utilises these findings to test and refine an existing conceptual framework for evaluating the impact and effectiveness of health impact assessments for use in relation to EFHIAs.

METHODS

Case study analysis of three EFHIAs conducted on health sector plans in New South Wales, Australia. Data was drawn from 14 semi-structured interviews and the analysis of seven related documents (draft plans and EFHIA reports).

RESULTS

The case studies showed that the EFHIAs all had some impact on the decision-making about the plans and their implementation, most clearly in relation to participants' understandings of equity and in the development of options for modifying service plans to ensure this was addressed. The timing of the EFHIA and individual responses to the EFHIA process and its recommendations were identified as critical factors influencing the impact of the EFHIAs. Several modifications to the conceptual framework are identified, principally adding factors to recognise the role individuals play in influencing the impact and effectiveness of EFHIAs.

CONCLUSION

EFHIA has the potential to improve the consideration of health equity in health service planning processes, though a number of contextual and individual factors affect this. Current approaches can be strengthened by taking into account personal and organisational responses to the EFHIA process.

Centering pregnancy in Missouri: a system level analysis

Background. Centering Pregnancy (CP) is an effective method of delivering prenatal care, yet providers have been slow to adopt the CP model. Our main hypothesis is that a site's adoption of CP is contingent upon knowledge of the CP, characteristics health care personnel, anticipated patient impact, and system readiness. Methods. Using a matched, pretest-posttest, observational design, 223 people completed pretest and posttest surveys. Our analysis included the effect of the seminar on the groups' knowledge of CP essential elements, barriers to prenatal care, and perceived value of CP to the patients and to the system of care. Results. Before the CP Seminar only 34% of respondents were aware of the model, while knowledge significantly after the Seminar. The three greatest improvements were in understanding that the group is conducted in a circle, the health assessment occurs in the group space, and a facilitative leadership style is used. Child care, transportation, and language issues were the top three barriers. The greatest improvements reported for patients included improvements in timeliness, patient-centeredness and efficiency, although readiness for adoption was influenced by costs, resources, and expertise. Discussion. Readiness to adopt CP will require support for the start-up and sustainability of this model.

The RE-AIM framework: a systematic review of use over time

We provided a synthesis of use, summarized key issues in applying, and highlighted exemplary applications in the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. We articulated key RE-AIM criteria by reviewing the published literature from 1999 to 2010 in several databases to describe the application and reporting on various RE-AIM dimensions. After excluding nonempirical articles, case studies, and commentaries, 71 articles were identified. The most frequent publications were on physical activity, obesity, and disease management. Four articles reported solely on 1 dimension compared with 44 articles that reported on all 5 dimensions of the framework. RE-AIM was broadly applied, but several criteria were not reported consistently.

Brief cognitive behavioral therapy in primary care: a hybrid type 2 patient-randomized effectiveness-implementation design

BACKGROUND

Despite the availability of evidence-based psychotherapies for depression and anxiety, they are underused in non-mental health specialty settings such as primary care. Hybrid effectiveness-implementation designs have the potential to evaluate clinical and implementation outcomes of evidence-based psychotherapies to improve their translation into routine clinical care practices.

METHODS

This protocol article discusses the study methodology and implementation strategies employed in an ongoing, hybrid, type 2 randomized controlled trial with two primary aims: (1) to determine whether a brief, manualized cognitive behavioral therapy administered by Veterans Affairs Primary Care Mental Health Integration program clinicians is effective in treating depression and anxiety in a sample of medically ill (chronic cardiopulmonary diseases) primary care patients and (2) to examine the acceptability, feasibility, and preliminary outcomes of a focused implementation strategy on improving adoption and fidelity of brief cognitive behavioral therapy at two Primary Care-Mental Health Integration clinics. The study uses a hybrid type 2 effectiveness/implementation design to simultaneously test clinical effectiveness and to collect pilot data on a multifaceted implementation strategy that includes an online training program, audit and feedback of session content, and internal and external facilitation. Additionally, the study engages the participation of an advisory council consisting of stakeholders from Primary Care-Mental Health Integration, as well as regional and national mental health leaders within the Veterans Administration. It targets recruitment of 320 participants randomized to brief cognitive behavioral therapy (n = 200) or usual care (n = 120). Both effectiveness and implementation outcomes are being assessed using mixed methods, including quantitative evaluation (e.g., intent-to-treat analyses across multiple time points) and qualitative methods (e.g., focus interviews and surveys from patients and providers). Patient-effectiveness outcomes include measures of depression, anxiety, and physical health functioning using blinded independent evaluators. Implementation outcomes include patient engagement and adherence and clinician brief cognitive behavioral therapy adoption and fidelity.

CONCLUSIONS

Hybrid designs are needed to advance clinical effectiveness and implementation knowledge to improve healthcare practices. The current article describes the rationale and challenges associated with the use of a hybrid design for the study of brief cognitive behavioral therapy in primary care. Although trade-offs exist between scientific control and external validity, hybrid designs are part of an emerging approach that has the potential to rapidly advance both science and practice.

TRIAL REGISTRATION

NCT01149772 at http://www.clinicaltrials.gov/ct2/show/NCT01149772.

A comparison of breast and cervical cancer legislation and screening in Georgia, North Carolina, and South Carolina

We identified legislation (1989-2005) relating to breast and cervical cancer in Georgia, North Carolina, and South Carolina and examined its impact on screening rates for these cancers and on Black-White disparities in screening rates. Legislation was identified using the National Cancer Institute's (NCI) State Cancer Legislative Database (SCLD) Program. Screening rates were identified using the Centers for Disease Control and Prevention's Behavioral Risk Factor Surveillance System. Georgia and North Carolina enacted more laws on breast and cervical cancer than did South Carolina. The laws specifically intended to increase breast and cervical cancer screening were mandates requiring that insurance policies cover such screening; Georgia and North Carolina enacted such laws, but South Carolina did not. However, we were unable to demonstrate an effect of these laws on either screening rates or disparities. This may reinforce the importance of evidence-based health promotion programs to increase screening.

Applying the RE-AIM framework to the Alberta's Caring for Diabetes Project: a protocol for a comprehensive evaluation of primary care quality improvement interventions

INTRODUCTION

Diabetes represents a major public health and health system burden. As part of the Alberta's Caring for Diabetes (ABCD) Project, two quality-improvement interventions are being piloted in four Primary Care Networks in Alberta. Gaps between health research, policy and practice have been documented and the need to evaluate the impact of public health interventions in real-world settings to inform decision-making and clinical practice is paramount. In this article, we describe the application of the RE-AIM framework to evaluate the interventions beyond effectiveness.

METHODS AND ANALYSIS

Two quality-improvement interventions were implemented, based on previously proven effective models of care and are directed at improving the physical and mental health of patients with type-2 diabetes. Our goal is to adapt and apply the RE-AIM framework, using a mixed-methods approach, to understand the impact of the interventions to inform policy and clinical decision-making. We present the proposed measures, data sources and data management and analysis strategies used to evaluate the interventions by RE-AIM dimension.

ETHICS AND DISSEMINATION

Ethics approval for the ABCD Project has been granted from the Health Research Ethics Board (HREB #PRO00012663) at the University of Alberta. The RE-AIM framework will be used to structure our dissemination activities by dimension.

RESULTS

It will be presented at relevant conferences and prepared for publication in peer-reviewed journals. Various products, such as presentations, briefing reports and webinars, will be developed to inform key stakeholders of the findings. Presentation of findings by RE-AIM dimension will facilitate discussion regarding the public health impact of the two interventions within the primary care context of Alberta and lessons learned to be used in programme planning and care delivery for patients with type-2 diabetes. It will also promote the application of evaluation models to better assess the impact of community-based primary healthcare interventions through our dissemination activities.

Beyond reach and effectiveness: evaluating the not-on-tobacco (N-o-T) program in West Virginia from 2000 to 2005

Despite most teenage smokers wanting to quit, their likelihood of success resembles that of flipping a coin. Evidence-based cessation programs, like the American Lung Association's Not-On-Tobacco (N-O-T) program, are effective. Evaluation of program dissemination is critical. This study uses the RE-AIM framework to evaluate the N-O-T program in West Virginia from 2000 to 2005. RE-AIM components consisted of four measures. Regional dissemination was measured using comparative differences between Regional Educational Service Agency regions (RESAs). Significant associations were found between RESAs for numerous characteristics. Among the RE-AIM components, two measures of Implementation were significantly different between RESAs. Variability between RESAs provided valuable descriptive evidence of N-O-T program dissemination in West Virginia. Therefore, geographical tailoring grounded in community-based participatory research could increase the N-O-T program's overall dissemination.

Applying the RE-AIM framework to evaluate two implementation strategies used to introduce a tool for lifestyle intervention in Swedish primary health care

The aim of this study was to evaluate two implementation strategies for the introduction of a lifestyle intervention tool in primary health care (PHC), applying the RE-AIM framework to assess outcome. A computer-based tool for lifestyle intervention was introduced in PHC. A theory-based, explicit, implementation strategy was used at three centers, and an implicit strategy with a minimum of implementation efforts at three others. After 9 months a questionnaire was sent to staff members (n= 159) and data from a test database and county council registers were collected. The RE-AIM framework was applied to evaluate outcome in terms of reach, effectiveness, adoption and implementation. The response rate for the questionnaire was 73%. Significant differences in outcome were found between the strategies regarding reach, effectiveness and adoption, in favor of the explicit implementation strategy. Regarding the dimension implementation, no differences were found according to the implementation strategy. A theory-based implementation strategy including a testing period before using a new tool in daily practice seemed to be more successful than a strategy in which the tool was introduced and immediately used for patients.

Reaiming RE-AIM: using the model to plan, implement, and evaluate the effects of environmental change approaches to enhancing population health

The RE-AIM (reach, effectiveness, adoption, implementation, maintenance) framework, which provides a practical means of evaluating health interventions, has primarily been used in studies focused on changing individual behaviors. Given the importance of the built environment in promoting health, using RE-AIM to evaluate environmental approaches is logical. We discussed the benefits and challenges of applying RE-AIM to evaluate built environment strategies and recommended modest adaptations to the model. We then applied the revised model to 2 prototypical built environment strategies aimed at promoting healthful eating and active living. We offered recommendations for using RE-AIM to plan and implement strategies that maximize reach and sustainability, and provided summary measures that public health professionals, communities, and researchers can use in evaluating built environment interventions.

The role of behavioral science theory in development and implementation of public health interventions

Increasing evidence suggests that public health and health-promotion interventions that are based on social and behavioral science theories are more effective than those lacking a theoretical base. This article provides an overview of the state of the science of theory use for designing and conducting health-promotion interventions. Influential contemporary perspectives stress the multiple determinants and multiple levels of determinants of health and health behavior. We describe key types of theory and selected often-used theories and their key concepts, including the health belief model, the transtheoretical model, social cognitive theory, and the ecological model. This summary is followed by a review of the evidence about patterns and effects of theory use in health behavior intervention research. Examples of applied theories in three large public health programs illustrate the feasibility, utility, and challenges of using theory-based interventions. This review concludes by identifying cross-cutting themes and important future directions for bridging the divides between theory, practice, and research.

Evaluating an insurance-sponsored weight management program with the RE-AIM Model, West Virginia, 2004-2008

INTRODUCTION

Evaluations of weight management programs in real-world settings are lacking. The RE-AIM model (reach, effectiveness, adoption, implementation, maintenance) was developed to address this deficiency. Our primary objective was to evaluate a 12-week insurance-sponsored weight management intervention by using the RE-AIM model, including short-term and long-term individual outcomes and setting-level implementation factors. Our secondary objective was to critique the RE-AIM model and its revised calculation methods.

METHODS

We created operational definitions for components of the 5 RE-AIM indices and used standardized effect size values from various statistical procedures to measure multiple components or outcomes within each index. We used chi(2) analysis to compare categorical variables and repeated-measures analysis of variance to assess the magnitude of outcome changes over time.

RESULTS

On the basis of data for 1,952 participants and surveys completed by administrators at 23 sites, RE-AIM indices ranging from 0 to 100 revealed low program reach and adoption (5.4 and 8.8, respectively), moderate effectiveness (43.8), high implementation (91.4), low to moderate individual maintenance (21.2), and moderate to high site maintenance (77.8). Median (interquartile range) weight loss was 13 lb (6.5-21.4 lb) among participants who completed phase I (12 weeks; 76.5%) and 15 lb (6.1-30.3 lb) among those who completed phase II (1 year; 45.7%).

CONCLUSION

This program had a significant, positive effect on participants and has been sustainable but needs to be expanded for more public health benefit. The RE-AIM model provided a useful framework to determine program strengths and weaknesses and to present them to the insurance agency and public health decision makers.

A lifestyle approach to reducing cardiovascular risk factors in underserved women: design and methods of the Illinois WISEWOMAN Program

BACKGROUND

Few lifestyle intervention programs address the needs of financially disadvantaged, low literacy populations. The overall goal of the Illinois WISEWOMAN Program (IWP) was to design such a program and test its effectiveness in reducing cardiovascular disease (CVD) risk, specifically physical activity and nutrition factors. The purpose of this paper is to describe the IWP study design and methods, development of the evidence-based curriculum appropriate for a low socioeconomic status (SES) population, and baseline characteristics of IWP participants.

METHODS

The Cooper Institute, in collaboration with the Illinois Department of Public Health and the University of Illinois at Chicago, adapted evidence-based interventions for financially disadvantaged, low literacy populations. The study used a randomized, two-group, experimental design. In total, 1021 women were recruited from the Illinois Breast and Cervical Cancer Program, which serves uninsured and underinsured women, aged 40-64, at or below 200% of poverty. The women were randomized to either a minimum intervention (MI) or an enhanced intervention (EI) group. Both groups received CVD risk factor screening and educational materials. Additionally, the EI group received a 12-week lifestyle intervention.

RESULTS

Baseline comparisons show equivalent groups. IWP participants had a higher prevalence of obesity and smoking than similar national samples.

CONCLUSIONS

IWP addressed many of the cultural and implementation barriers in programs that seek to improve the health of financially disadvantaged, low literacy populations. Because of the high burden of disease, the unique study population, and the sound design, we anticipate that our future results will contribute to the translation literature, which has largely ignored significant health disparities.

A statewide intervention reduces BMI in adults: Shape Up Rhode Island results

Given the epidemic of obesity, approaches to weight loss that can be applied on a community, state, or national level are needed. We report results from Shape Up Rhode Island 2007 (SURI), a statewide Internet-based program involving team-based competition to increase physical activity and achieve weight loss. A total of 4,717 adults (84% women; mean BMI = 29.6 kg/m(2)) enrolled in the 16-week weight loss competition of SURI and 3,311 completed at least 12 weeks. Completers reported losing 3.2 +/- 3.4 kg, and 30% achieved a clinically significant weight loss of >or=5%. Although modest, these weight losses shifted the BMI distribution from a mean BMI of 29.4 to a mean of 28.2 kg/m(2) and reduced the population that was obese from 39 to 31%. More conservative intent-to-treat analyses and analysis of 132 participants with objective weights still showed a significant reduction in BMI of -0.8 units. These findings suggest that statewide weight loss campaigns can produce modest weight losses in large numbers of participants. These data provide a benchmark that can be used for comparisons with other statewide campaigns. Research on ways to improve such campaigns is needed.

Health Risk Reduction in Women

This review discusses evidence-based perspectives on health risk reduction in women and the implications for gender-informed research, clinical best practices, and public policy.

Best Practices in Implementing Lifestyle Interventions in the WISEWOMAN Program: Adaptable Strategies for Public Health Programs

Purpose.

Describe best practices for implementing a variety of lifestyle interventions targeting cardiovascular disease risk factors.

Approach.

A mixed-methods approach was used to collect and analyze data. The study was guided by the RE-AIM framework.

Setting.

Selected Well-Integrated Screening and Intervention for Women Across the Nation (WISEWOMAN) projects funded by the Centers for Disease Control and Prevention.

Participants.

Five of the 15 currently operating WISEWOMAN projects were selected for study. Selection was based on availability of quantitative performance data, which were used to identify two high-performing and one low-performing sites within each project.

Method.

Qualitative data collection included a review of program materials; telephone interviews with federal, project, and local staff and site visits. Site visits involved interviews with staff, observations of the lifestyle intervention, and discussions with focus groups of participants. Analysis involved writing site reports, developing theme tables, identifying practices of interest, and applying an algorithm to identify best practices.

Results.

Eighty-seven best practices were identified. We present a subset of 31 practices applicable to other public health programs and for which differences in how high- and low-performing sites used the practices were identified.

Discussion.

Many of the best practices identified are applicable to broader audiences. Practitioners interested in strategies to recruit, engage, and retain participants and to facilitate behavior change can learn from these practices.