Evaluation of health education programs: current assessment and future directions

Developing relevant assessments of community-engaged research partnerships: A community-based participatory approach to evaluating clinical and health research study teams

Background/Objective

In 2017, the Michigan Institute for Clinical and Health Research (MICHR) and community partners in Flint, Michigan collaborated to launch a research funding program and evaluate the dynamics of those research partnerships receiving funding. While validated assessments for community-engaged research (CEnR) partnerships were available, the study team found none sufficiently relevant to conducting CEnR in the context of the work. MICHR faculty and staff along with community partners living and working in Flint used a community-based participatory research (CBPR) approach to develop and administer a locally relevant assessment of CEnR partnerships that were active in Flint in 2019 and 2021.

Methods

Surveys were administered each year to over a dozen partnerships funded by MICHR to evaluate how community and academic partners assessed the dynamics and impact of their study teams over time.

Results

The results suggest that partners believed that their partnerships were engaging and highly impactful. Although many substantive differences between community and academic partners' perceptions over time were identified, the most notable regarded the financial management of the partnerships.

Conclusion

This work contributes to the field of translational science by evaluating how the financial management of community-engaged health research partnerships in a locally relevant context of Flint can be associated with these teams' scientific productivity and impact with national implications for CEnR. This work presents evaluation methods which can be used by clinical and translational research centers that strive to implement and measure their use of CBPR approaches.

The role of communication, building relationships, and adaptability in non-profit organisational capacity for health promotion

While the non-profit sector has an integral role in health promotion, it is unclear whether these organisations have the capacity for health promotion activities. This study aims to explore and describe capacity changes of a non-profit organisation during a 3-year community-based nutrition intervention. The non-profit organisation, with 3800 members throughout the state of Queensland, Australia, implemented a 3-year food literacy community-based intervention. A team of qualified nutritionists delivered the program in partnership with community-based volunteers. A separate aim of the intervention was to build capacity of the non-profit organisation for health promotion. A qualitative study was undertaken, using a social constructivist approach to explore organisational capacity changes longitudinally. All relevant participants including non-profit executive managers and nutritionists were included in the study (100% response rate). Data collection included semi-structured interviews (n = 17) at multiple intervention time points and document analysis of program newsletters (n = 21). Interview transcripts and documents were analysed separately using thematic and content analysis. Codes and categories between the two data sources were then compared and contrasted to build themes. Organisational capacity was predominantly influenced by four themes; ‘communicating’, ‘changing relationships’, ‘limited organisational learning’ and ‘adaptability and resistance to change’. Developing non-profit organisational health promotion capacity appears to require focusing on fostering communication processes and building positive relationships over time. Capacity changes of the non-profit organisation were not linear, fluctuating across various levels over time. Assessing non-profit organisational capacity to implement community interventions by describing adaptive capacity, may help researchers focus on the processes that influence capacity development.

The Stroman Effect: Participants in MEN Count, an HIV/STI Reduction Intervention for Unemployed and Unstably Housed Black Heterosexual Men, Define Its Most Successful Elements

Interventionists often prioritize quantitative evaluation criteria such as design (e.g., randomized controlled trials), delivery fidelity, and outcome effects to assess the success of an intervention. Albeit important, criteria such as these obscure other key metrics of success such as the role of the interactions between participants and intervention deliverers, or contextual factors that shape an intervention’s activities and outcomes. In line with advocacy to expand evaluation criteria for health interventions, we designed this qualitative study to examine how a subsample of Black men in MEN Count, an HIV/STI risk reduction and healthy relationship intervention with employment and housing stability case management for Black men in Washington, DC, defined the intervention’s success. We also examined the contextual factors that shaped participation in the study’s peer counseling sessions. We conducted structured interviews with 38 Black men, ages 18 to 60 years (M = 31.1, SD = 9.33) who completed at least one of three peer counseling sessions. Analyses highlighted three key themes: (a) the favorable impact of Mr. Stroman, the lead peer counselor, on participants’ willingness to participate in MEN Count and disclose their challenges—we dubbed this the “Stroman Effect”; (b) the importance of Black men intervention deliverers with relatable life experiences; and (c) how contextual factors such as the HIV/AIDS epidemic, needs for housing and employment services and safe spaces to talk about challenges, and absentee fathers shaped participation. We discuss the study’s implications for sustainable programs after funding ends and future multilevel health interventions to promote health equity for poor urban Black men.

Interest in Sexually Transmitted Infections: Analysis of Web Search Data Terms in Eleven Large German Cities from 2015 to 2019

Incidence of sexually transmitted infections (STIs) such as chlamydia, gonorrhea, and syphilis has increased in recent years in the US and in European countries. In order to implement effective educational programs, the interests of target populations have to be identified. Since the internet is an important source of information-gathering on health issues, this study investigates web search data in large German cities related to STIs. Google Ads Keyword Planner was used to identify STI-related terms and their search volume in eleven German cities from June 2015 to May 2019. The data obtained were analyzed descriptively with regard to total search volumes, search volumes of specific thematic areas, and search volumes per 100,000 inhabitants. Overall, 741 terms with a total search volume of 5,142,560 queries were identified, with more than 70% of all search queries including a specific disease and “chlamydia” being the overall most often searched term (n = 1,196,160). Time courses of search behavior displayed a continuous interest in STIs with synchronal and national rather than regional peaks. Volumes of search queries lacked periodic patterns. Based on the findings of this study, a more open public discussion about STIs with linkage to increased media coverage and clarification of responsibilities among all STI-treating disciplines concerning management of STIs seem advisable.

Evaluation of a National Competency-Based Assessment System in Emergency Medicine: A CanDREAM Study

BACKGROUND

In 2018, Canadian postgraduate emergency medicine (EM) programs began implementing a competency-based medical education (CBME) assessment program. Studies evaluating these programs have focused on broad outcomes using data from national bodies and lack data to support program-specific improvement.

OBJECTIVE

We evaluated the implementation of a CBME assessment program within and across programs to identify successes and opportunities for improvement at the local and national levels.

METHODS

Program-level data from the 2018 resident cohort were amalgamated and analyzed. The number of entrustable professional activity (EPA) assessments (overall and for each EPA) and the timing of resident promotion through program stages were compared between programs and to the guidelines provided by the national EM specialty committee. Total EPA observations from each program were correlated with the number of EM and pediatric EM rotations.

RESULTS

Data from 15 of 17 (88%) programs containing 9842 EPA observations from 68 of 77 (88%) EM residents in the 2018 cohort were analyzed. Average numbers of EPAs observed per resident in each program varied from 92.5 to 229.6, correlating with the number of blocks spent on EM and pediatric EM (r = 0.83, P < .001). Relative to the specialty committee's guidelines, residents were promoted later than expected (eg, one-third of residents had a 2-month delay to promotion from the first to second stage) and with fewer EPA observations than suggested.

CONCLUSIONS

There was demonstrable variation in EPA-based assessment numbers and promotion timelines between programs and with national guidelines.

A scoping review of evaluation frameworks and their applicability to real-world physical activity and dietary change programme evaluation

BACKGROUND

Physical activity and dietary change programmes play a central role in addressing public health priorities. Programme evaluation contributes to the evidence-base about these programmes; and helps justify and inform policy, programme and funding decisions. A range of evaluation frameworks have been published, but there is uncertainty about their usability and applicability to different programmes and evaluation objectives, and the extent to which they are appropriate for practitioner-led or researcher-led evaluation. This review appraises the frameworks that may be applicable to evaluation of physical activity and/or dietary change programmes, and develops a typology of the frameworks to help guide decision making by practitioners, commissioners and evaluators.

METHODS

A scoping review approach was used. This included a systematic search and consultation with evaluation experts to identify evaluation frameworks and to develop a set of evaluation components to appraise them. Data related to each framework's general characteristics and components were extracted. This was used to construct a typology of the frameworks based on their intended programme type, evaluation objective and format. Each framework was then mapped against the evaluation components to generate an overview of the guidance included within each framework.

RESULTS

The review identified 71 frameworks. These were described variously in terms of purpose, content, or applicability to different programme contexts. The mapping of frameworks highlighted areas of overlap and strengths and limitations in the available guidance. Gaps within the frameworks which may warrant further development included guidance on participatory approaches, non-health and unanticipated outcomes, wider contextual and implementation factors, and sustainability.

CONCLUSIONS

Our typology and mapping signpost to frameworks where guidance on specific components can be found, where there is overlap, and where there are gaps in the guidance. Practitioners and evaluators can use these to identify, agree upon and apply appropriate frameworks. Researchers can use them to identify evaluation components where there is already guidance available and where further development may be useful. This should help focus research efforts where it is most needed and promote the uptake and use of evaluation frameworks in practice to improve the quality of evaluation and reporting.

Perspectives of Community Co-Researchers About Group Dynamics and Equitable Partnership Within a Community-Academic Research Team

Equitable partnership processes and group dynamics, including individual, relational, and structural factors, have been identified as key ingredients to successful community-based participatory research partnerships. The purpose of this qualitative study was to investigate the key aspects of group dynamics and partnership from the perspectives of community members serving as co-researchers. Semistructured, in-depth interviews were conducted with 15 Latino immigrant co-researchers from an intervention project with Latinos Unidos por la Salud (LU-Salud), a community research team composed of Latino immigrant community members and academic investigators working in a health research partnership. A deductive framework approach guided the interview process and qualitative data analysis. The LU-Salud co-researchers described relationships, personal growth, beliefs/identity motivation (individual dynamics), coexistence (relational dynamics), diversity, and power/resource sharing (structural dynamics) as key foundational aspects of the community-academic partnership. Building on existing CBPR and team science frameworks, these findings demonstrate that group dynamics and partnership processes are fundamental drivers of individual-level motivation and meaning making, which ultimately sustain efforts of community partners to engage with the research team and also contribute to the achievement of intended research outcomes.

Strategies for community-based dementia education and creating sustainable change: Lessons learned from the Community Outreach Education Program

The Community Outreach Education Program (COEP) was established to disseminate information about the diagnosis, assessment, management, and treatment of dementia to health care professionals, service providers, staff of community organizations and voluntary agencies, and family caregivers in communities throughout Michigan. This paper will describe the activities implemented as part of outreach educational interventions in Bay City, Saginaw, Grand Rapids, and Kalamazoo, Michigan. Lessons learned about community-based dementia education and strategies for creating sustainable change are discussed.

A Union-Initiated Safety Training Program Leads to Improved Workplace Safety

Workplace safety training programs initiated by unions have gained strength and numbers over the past ten years. Union-initiated peer train ing offers a new twist on joint labor-management health and safety pro grams, which have become an important area of labor-management coop eration. Peer trainers play a major role in the new partnerships, as man agement learns how effectively workers function as trainers who speak directly to worker interests. This article describes a program of coopera tion between a large international union, a university labor center, and companies where members of the union developed and taught workplace safety courses. Results of a survey to quantify workplace impact of the training program are reported.

Process Evaluation of an Integrated Health Promotion/Occupational Health Model in WellWorks-2

Disparities in chronic disease risk by occupation call for newapproaches to health promotion. Well Works-2 was a randomized, controlled study comparing the effectiveness of a health promotion/occupational health program (HP/OHS) with a standard intervention (HP). Interventions in both studies were based on the same theoretical foundations. Results from process evaluation revealed that a similar number of activities were offered in both conditions and that in the HP/OHS condition there were higher levels of worker participation using three measures: mean participation per activity (HP: 14.2% vs. HP/OHS: 21.2%), mean minutes of worker exposure to the intervention/site (HP: 14.9 vs. HP/OHS: 33.3), and overall mean participation per site (HP: 34.4% vs. HP/ OHS: 45.8%). There were a greater number of contacts with management (HP: 8.8 vs. HP/OHS: 24.9) in the HP/ OHS condition. Addressing occupational health may have contributed to higher levels of worker and management participation and smoking cessation among blue-collar workers.

The Interactive Domain Model of Best Practices in Health Promotion: Developing and Implementing a Best Practices Approach to Health Promotion

This paper discusses issues associated with taking a best practices approach to health promotion including determining factors, implementation, and implications for practitioners and policy makers. We suggest that health promotion effectiveness will be increased through adoption of a systematic and critically reflective approach to practice—one which considers all major factors affecting practice and is consciously guided by health promotion values and goals, theories and beliefs, evidence, and understanding of the environment. To help practitioners develop and implement best practices, we outline our Inter-active Domain Model of Best Practices in Health Promotion, the IDM Operational Framework, and a set of best practices criteria. The conceptual model, framework, and criteria are based on three domains (i.e., underpinnings, understanding of the environment, and practice) and related subdomains, all of which interact with each other within the context of the immediate and broader environments.

A Qualitative Study of a Nutrition Working Group

The Behavioral Change Consortium (BCC) Nutrition Workgroup (NWG) is a multidisciplinary collaboration of representatives from BCC sites and federal agencies. Its mission is to improve measurement of dietary variables. This article presents findings from a qualitative study of perceived effectiveness of the workgroup collaboration. Twelve in-depth interviews were conducted and examined for common themes using the constant comparison method. Themes contributing to perceived effectiveness included: funding and additional resources; invested, committed, and collegial members; strong leadership, clearly articulated goals, and regular communication. Influences seen as reducing effectiveness were: distance, disparate nature of the studies, limited time, and problems associated with starting collaboration after the primary studies had begun data collection. NWG members felt that the workgroup would continue to be successful; however, there were concerns about responsibility for writing and authorship of manuscripts and the need for continued funding to ensure full participation and productivity.

Fostering Youth Leadership to Address Workplace and Community Environmental Health Issues: A University-School-Community Partnership

Many communities of color are disproportionately exposed to workplace and community environmental hazards. This article presents the results of a pilot project designed by a university-school-community partnership to develop youth leadership to confront these exposures. Using a popular empowerment education approach, students applied peer education, research, and organizing skills learned in the classroom to community-based internships in a service-learning model. Evaluation results from pretests and posttests, focus groups, and in-depth interviews demonstrated that students shared what they learned about young workers’ rights and environmental justice with family and friends. They developed a critical analysis of environmental inequities, created a citywide youth coalition that advocates around legal, educational, and environmental issues affecting youth, and implemented campaigns to enforce child labor laws and to prevent school construction on contaminated land. This multifaceted model can serve as an important foundation to develop youth leaders to influence environmental policies in a variety of communities.

Increasing Community Capacity to Respond to Dementing Illnesses: Process Evaluation of the Community Outreach Education Program

The goal of the Community Outreach Education Program(COEP) is to disseminate information about the diagnosis, assessment, management, and treatment of dementia to health care professionals, service providers, staff of community organizations and voluntary agencies, and family caregivers in four rural and two urban sites in Michigan. To accomplish this, outreach educational interventions based on a community-development model were developed in the Thumb Area, Upper Peninsula, Northeast Michigan, Mid-Michigan, Detroit, and Flint. The present study describes a process evaluation of the COEP in these six sites, using a multiple case study approach. Shared themes in the successful implementation of the on-site interventions and in creating community capacity across the varied sites are discussed.

Design of an online health-promoting community: negotiating user community needs with public health goals and service capabilities

Background

An online health-promoting community (OHPC) has the potential to promote health and advance new means of dialogue between public health representatives and the general public. The aim of this study was to examine what aspects of an OHPC that are critical for satisfying the needs of the user community and public health goals and service capabilities.

Methods

Community-based participatory research methods were used for data collection and analysis, and participatory design principles to develop a case study OHPC for adolescents. Qualitative data from adolescents on health appraisals and perspectives on health information were collected in a Swedish health service region and classified into categories of user health information exchange needs. A composite design rationale for the OHPC was completed by linking the identified user needs, user-derived requirements, and technical and organizational systems solutions. Conflicts between end-user requirements and organizational goals and resources were identified.

Results

The most prominent health information needs were associated to food, exercise, and well-being. The assessment of the design rationale document and prototype in light of the regional public health goals and service capabilities showed that compromises were needed to resolve conflicts involving the management of organizational resources and responsibilities. The users wanted to discuss health issues with health experts having little time to set aside to the OHPC and it was unclear who should set the norms for the online discussions.

Conclusions

OHPCs can be designed to satisfy both the needs of user communities and public health goals and service capabilities. Compromises are needed to resolve conflicts between users’ needs to discuss health issues with domain experts and the management of resources and responsibilities in public health organizations.

Project Health: evaluation of a project‐based health education program

Purpose

Milwaukee has very high rates of risky sexual behavior and low rates of academic achievement among adolescents. Milwaukee school representatives partnered with researchers to create and implement an innovative project‐based learning (PBL) high school health curriculum to engage students in school. This health education program, Project Health (PH), aimed to engage Milwaukee Public Schools high school students, by PBL, into the urban health classroom. The purpose of this paper is to examine the impact of the curriculum on student engagement and document the process of implementing this program, in order to identify strengths, weaknesses and areas of needed improvement.

Design/methodology/approach

Student engagement was measured with an 18‐item scale at three time points in intervention and control schools. Attendance data were collected by tracking the number of days students were absent from school the semester the curriculum was implemented. Analysis of covariance was used to test whether students in PH classes were more engaged and if they had fewer absences than students in the control classes. The process evaluation included interviews with teachers, focus groups with high school students, and focus groups with teaching assistants. Interviews were transcribed and analyzed using thematic analysis to extract key themes and categories across all data.

Findings

Students in the intervention health class were significantly less absent from school than control students. However, they were not more engaged than their peers in control schools as measured by a student engagement scale. Various components of the PBL teaching approach used in the Project Health high school curriculum were viewed as positive by health educators, students, and teaching assistants. Results indicate that PBL used in health education may increase school attendance.

Originality/value

This study was original in that it documented how a novel health education program that incorporated PBL can positively impact urban students’ school attendance. It also highlighted the process of implementing this program from the vantage point of students, assistants, and health education.

Lessons from a mixed-methods approach to evaluating Active Living by Design

BACKGROUND

Beginning in 2003, Active Living by Design (ALbD) established innovative approaches across 25 communities to increase physical activity through community design, public policies, programming, and communication strategies.

PURPOSE

The complexity of the ALbD projects called for a mixed-methods evaluation to understand implementation as well as perceived and actual impacts of these efforts.

METHODS

Six primary evaluation methods addressed three primary aims: (1) to assess impacts of physical projects and policy changes on community environments; (2) to document intervention strategies implemented, as well as intended and unintended consequences; and (3) to identify strengths and challenges in planning, developing, and implementing interventions. The ALbD evaluation included cross-site comparisons and more in-depth case studies. This article describes the methods used to address the three aims.

RESULTS

Analysis of the strengths and challenges associated with the different methods, including partnership capacity surveys, Concept Mapping, an online progress reporting system, key informant interviews, focus groups, and photos and videos. Additional methods, including environmental audits and direct observation, were explored to specifically assess environmental changes. Several important challenges included the lack of baseline data, difficulty in evaluating natural experiments, the need for ongoing policy surveillance, and the need to capture longer-term endpoints.

CONCLUSIONS

The mixed-methods evaluation of ALbD advances implementation and evaluation science related to community-based efforts for promoting active living through identification of methods and measures to capture multicomponent and complex interventions as well as translation of a range of approaches to create community change across a variety of populations and settings.

Understanding and addressing barriers to implementation of environmental and policy interventions to support physical activity and healthy eating in rural communities

PURPOSE

Rural residents are at greater risk of obesity than urban and suburban residents. Failure to meet physical activity and healthy eating recommendations play a role. Emerging evidence shows the effectiveness of environmental and policy interventions to promote physical activity and healthy eating. Yet most of the evidence comes from urban and suburban communities. The objectives of this study were to (1) identify types of environmental and policy interventions being implemented in rural communities to promote physical activity or healthy eating, (2) identify barriers to the implementation of environmental or policy interventions, and (3) identify strategies rural communities have employed to overcome these barriers.

METHODS

Key informant interviews with public health professionals working in rural areas in the United States were conducted in 2010. A purposive sample included 15 practitioners engaged in planning, implementing, or evaluating environmental or policy interventions to promote physical activity or healthy eating.

FINDINGS

Our findings reveal that barriers in rural communities include cultural differences, population size, limited human capital, and difficulty demonstrating the connection between social and economic policy and health outcomes. Key informants identified a number of strategies to overcome these barriers such as developing broad-based partnerships and building on the existing infrastructure.

CONCLUSION

Recent evidence suggests that environmental and policy interventions have potential to promote physical activity and healthy eating at the population level. To realize positive outcomes, it is important to provide opportunities to implement these types of interventions and document their effectiveness in rural communities.

Multilevel interventions and racial/ethnic health disparities

To examine the impact of multilevel interventions (with three or more levels of influence) designed to reduce health disparities, we conducted a systematic review and meta-analysis of interventions for ethnic/racial minorities (all except non-Hispanic whites) that were published between January 2000 and July 2011. The primary aims were to synthesize the findings of studies evaluating multilevel interventions (three or more levels of influence) targeted at ethnic and racial minorities to reduce disparities in their health care and obtain a quantitative estimate of the effect of multilevel interventions on health outcomes among these subgroups. The electronic database PubMed was searched using Medical Subject Heading terms and key words. After initial review of abstracts, 26 published studies were systematically reviewed by at least two independent coders. Those with sufficient data (n = 12) were assessed by meta-analysis and examined for quality using a modified nine-item Physiotherapy Evidence Database coding scheme. The findings from this descriptive review suggest that multilevel interventions have positive effects on several health behavior outcomes, including cancer prevention and screening, as well improving the quality of health-care system processes. The weighted average effect size across studies for all health behavior outcomes reported at the individual participant level (k = 17) was odds ratio (OR) = 1.27 (95% confidence interval [CI] = 1.11 to 1.44); for the outcomes reported by providers or organizations, the weighted average effect size (k = 3) was OR = 2.53 (95% CI = 0.82 to 7.81). Enhanced application of theories to multiple levels of change, novel design approaches, and use of cultural leveraging in intervention design and implementation are proposed for this nascent field.

Acupuncture for depression: exploring model validity and the related issue of credibility in the context of designing a pragmatic trial

INTRODUCTION

Evaluating care that is not credible to its practitioners or patients will result in a gap between evidence and practice and the potential value, or harm, of the intervention may be underestimated. Our aim was to develop a pragmatic trial that would have better model validity and credibility than trials to date in this clinical area.

METHODS

In-depth interviews; a nominal consensus technique and five arm pilot trial conducted in UK primary care using counseling and usual general practitioner (GP) care as comparisons for acupuncture.

FINDINGS

Patients with long standing, severe illness that had not responded, or partially responded to conventional treatments may be interested in using acupuncture and participating in a trial. Using a database method to recruit, pilot trial patients were mostly severely depressed (87.5%); chronically ill (60% with 3+ previous episodes), with high levels of comorbidity, and medication use. Acupuncture was as credible to pilot trial participants as usual GP care and more credible than counseling: most patients (62.5%) preferred to be allocated to acupuncture, rating it more highly at baseline than counseling or usual GP care as potentially able to benefit their depression (P = 0.002). Disparities were identified in the working models of acupuncturists and counselors that suggest inherent differences between interventions in terms of the process and intended potential outcomes of therapy, as well as the interaction between patients and therapists.

CONCLUSION

The Medical Research Council (MRC) framework with its phased, mixed method approach has helped to develop research that has better model validity than trials to date in this field. The next phase of research will need to involve acupuncture and counseling practitioners to help researchers to develop realistic and credible care packages for a full-scale trial, where patients are likely to be severely and chronically ill.

Process evaluation of HIV prevention peer groups in Malawi: a look inside the black box

This paper reports the process evaluation of a peer group intervention for human immunodeficiency virus (HIV) prevention which had positive outcomes for three target groups in Malawi: rural adults, adolescents and urban hospital workers. The six-session intervention was delivered to small groups of 10-12 participants by 85 trained volunteer peer leaders working in pairs. A descriptive, observational mixed methods design was used with a convenience sample of 294 intervention sessions. Using project records and a conceptually based observation guide, we examined five aspects of the implementation process. The context was favorable, but privacy to discuss sensitive issues was a concern for some groups. In study communities, program reach was 58% of rural adults, 70% of adolescents and nearly all hospital workers. Session records confirmed that all peer groups received the intended six sessions (dose delivered). The dose received was high, as evidenced by high participant engagement in peer group activities. Peer leaders were rated above the median for three indicators of peer group content and process fidelity: session management skills, interpersonal facilitation skills and whether more like a peer group than classroom. Documenting that this HIV prevention peer group intervention was delivered as intended by trained peer volunteers supports widespread dissemination of the intervention.

Evaluating the implementation of peer counseling in a church-based dietary intervention for African Americans

OBJECTIVE

Body & Soul, an evidence-based nutrition program for African Americans churches, is currently being disseminated nationally and free of charge by the National Cancer Institute. For dissemination feasibility, the peer counseling training is done via DVD rather than by live trainers. We describe implementation and process evaluation of the peer counseling component under real world conditions.

METHODS

The study sample included 11 churches (6 early intervention, 5 delayed intervention) in 6 states. Data sources included training observations, post-training debriefing sessions, coordinator interviews, and church participant surveys. Survey data analysis examined associations between exposure to peer counseling and change in dietary intake. Qualitative data were analyzed using the constant comparative method.

RESULTS

Eight of 11 churches initiated the peer counseling program. Recall of talking with a peer counselor was associated with significantly (p<.02) greater fruit and vegetable intake. Data indicate sub-optimal program execution after peer counselor training.

CONCLUSION

Inconsistent implementation of the peer counseling intervention is likely to dilute program effectiveness in changing nutrition behaviors.

PRACTICE IMPLICATIONS

Disseminating evidence-based programs may require added resources, training, quality control, and technical assistance for improving program uptake. Similar to earlier research phases, systematic efforts at the dissemination phase are needed for program success.

Community-based participatory research: a capacity-building approach for policy advocacy aimed at eliminating health disparities

There have been increasing calls for community-academic partnerships to enhance the capacity of partners to engage in policy advocacy aimed at eliminating health disparities. Community-based participatory research (CBPR) is a partnership approach that can facilitate capacity building and policy change through equitable engagement of diverse partners. Toward this end, the Detroit Community-Academic Urban Research Center, a long-standing CBPR partnership, has conducted a policy training project. We describe CBPR and its relevance to health disparities; the interface between CBPR, policy advocacy, and health disparities; the rationale for capacity building to foster policy advocacy; and the process and outcomes of our policy advocacy training. We discuss lessons learned and implications for CBPR and policy advocacy to eliminate health disparities.

Student Participation in Community-Based Participatory Research To Improve Migrant and Seasonal Farmworker Environmental Health: Issues for Success

Involving students in community-based participatory research is a useful mechanism for engaging the community and helping it build future capacity. This article describes student involvement in a series of community-based environmental health research projects with migrant and seasonal farmworkers in North Carolina. High school, undergraduate, graduate, and professional school students have participated in various aspects of these projects, including planning, data collection, analysis, and reporting results. Students were required to invest time in learning about the farmworker population, as well as in learning to conduct community-based environmental health research. Drawing on these experiences, we offer observations regarding successful student integration in this type of research. Community-based projects benefit from student participation while encouraging the development of future community-oriented environmental health researchers.

Community organizing network for environmental health: using a community health development approach to increase community capacity around reduction of environmental triggers

The Community Organizing Network for Environmental Health (CONEH), a project of Community Action Against Asthma, used a community health development approach to improve children's asthma-related health through increasing the community's capacity to reduce physical and social environmental triggers for asthma. Three community organizers were hired to work with community groups and residents in neighborhoods in Detroit on the priority areas of air quality, housing, and citizen involvement in the environmental project and policy decision-making. As part of the evaluation of the CONEH project, 20 one-on-one semi-structured, in-depth interviews were conducted between August and November 2005 involving steering committee members, staff members, and key community organization staff and/or community members. Using data from the evaluation of the CONEH project, this article identifies the dimensions of community capacity that were enhanced as part of a CBPR community health development approach to reducing physical and social environmental triggers associated with childhood asthma and the factors that facilitated or inhibited the enhancement of community capacity.

Elements of External Validity of Tools for Health: An Intervention for Construction Laborers

Purpose.

To examine the external validity of an efficacious tailored smoking cessation and nutrition improvement telephone intervention.

Design.

Comparison of characteristics of participants and nonparticipants (representativeness); examination of the extent of intervention implementation.

Setting.

Cancer center collaboration with a labor union.

Subjects.

Unionized construction laborers.

Intervention.

Tailored feedback report, telephone counseling, and supplementary educational materials focused on smoking cessation and improved nutrition.

Measures.

Background survey identifying socio-demographic and behavioral characteristics; process evaluation data; and final efficacy survey to determine participant satisfaction.

Analysis.

Cross-classification and the X2 test of homogeneity were used with categorically measured variables comparing participants and nonparticipants. We compared the means in the two groups for continuously scaled measures using the Student t-test and investigated the multivariable association of the characteristics of participation with a multiple logistic regression. For process data we present frequencies, percentages, and means.

Results.

Characteristics associated with participation included self-efficacy to change fruit and vegetable consumption (p = .0009) and self-identification with union's problems (p = 0.05). Eighty-six percent of non-smokers and 61% of smokers completed between 1 and 4 counseling sessions. Over one-half of non-smokers (61%) and smokers (53%) completed 4 or more calls and more smokers (34%) than non-smokers (11%) completed the 5+ sessions.

Conclusions.

These results provide a snapshot of characteristics of construction laborers to whom this intervention can be generalized and indicators of feasibility necessary for translating research into practice.

A multi-method process evaluation for a skin cancer prevention diffusion trial

This article describes process evaluation methods for the Pool Cool diffusion trial across 4 years. Pool Cool is a skin cancer prevention program that was found to improve behaviors and environments for sun protection at swimming pools in a randomized efficacy trial, which was followed by a national diffusion trial. The process evaluation focus shifted from measuring program satisfaction to assessing widespread program implementation, barriers and facilitators to implementation, and program maintenance and sustainability. Data collection methods include training surveys, database tracking, field coordinator activity logs, e-mails, surveys of parents, lifeguards and pool managers, and process evaluation interviews and site visits. The data revealed high levels of implementation of major program components when disseminated in the diffusion trial, including sun safety lessons, sun safety signs, and sunscreen use. This article describes program features and participant factors that facilitated local implementation, maintenance and sustainability across dispersed pools such as linkage agents, a packaged program, and adaptations of program elements.

Evaluating nurses' implementation of an infant-feeding counseling protocol for HIV-infected mothers: The Ban Study in Lilongwe, Malawi

A process evaluation of nurses' implementation of an infant-feeding counseling protocol was conducted for the Breastfeeding, Antiretroviral and Nutrition (BAN) Study, a prevention of mother-to-child transmission of HIV clinical trial in Lilongwe, Malawi. Six trained nurses counseled HIV-infected mothers to exclusively breastfeed for 24 weeks postpartum and to stop breastfeeding within an additional four weeks. Implementation data were collected via direct observations of 123 infant feeding counseling sessions (30 antenatal and 93 postnatal) and interviews with each nurse. Analysis included calculating a percent adherence to checklists and conducting a content analysis for the observation and interview data. Nurses were implementing the protocol at an average adherence level of 90% or above. Although not detailed in the protocol, nurses appropriately counseled mothers on their actual or intended formula milk usage after weaning. Results indicate that nurses implemented the protocol as designed. Results will help to interpret the BAN Study's outcomes.

Developing a Cross-Site Evaluation Tool for Diverse Health Interventions

The Prevention Research Center of Michigan provided technical assistance for the evaluation of 10 projects funded by the Michigan Department of Community Health’s (MDCH) Health Disparities Reduction Program. These projects varied considerably in focus, methodology, geographical coverage, and populations served. The authors developed a cross-site evaluation tool to complement the internal evaluations of the projects. The tool contains four sections based on priorities identified by MDCH: evidence-based practice, research-based learning/evaluation (including process, impact, and outcomes indicators), cultural competence, and sustainability. Recognizing the diversity of programmatic efforts and organizational evaluation capacity, the authors sought to enable each project to create the best evaluation possible given the resources and data available. Each section contains a range of components from basic questions to more advanced evaluation techniques. The instrument attempts to use the highest quality of information available for each project. This evaluation tool can be used by programs with diverse goals and methodology.

Understanding the Data-Action Cycle of Surveillance: A Qualitative Study of Federal and State Stakeholders

This qualitative study aims to understand how personnel in state and federal agencies view surveillance systems and the extent to which systematically collected data inform nutrition and physical activity policies and interventions addressing obesity. In-depth interviews were conducted with 17 respondents purposively sampled from state health departments, federal public health agencies, and prevention research centers. All informants acknowledged the importance of surveillance systems and influence of the data-action cycle for monitoring trends and increasing obesity awareness. However, state-level respondents believed surveillance systems should be designed for programmatic purposes, whereas federal participants thought they should be designed for monitoring. Respondents held differing opinions about the flexibility, timeliness, accessibility, and usefulness of existing national surveillance systems, highlighting significant differences in state and federal agencies’ perceptions and utilization of surveillance systems. Such discrepancies call for increased communication surrounding purposes and uses of surveillance data, enabling stronger partnerships between state and federal agencies.

Process evaluation results from a school- and community-linked intervention: the Trial of Activity for Adolescent Girls (TAAG)

Process evaluation is a component of intervention research that evaluates whether interventions are delivered and received as intended. Here, we describe the process evaluation results for the Trial of Activity for Adolescent Girls (TAAG) intervention. The intervention consisted of four synergistic components designed to provide supportive school- and community-linked environments to prevent the decline in physical activity in adolescent girls. Process evaluation results indicate that the intervention components were delivered from intervention staff to teachers with high fidelity (84-97%) to the protocol and with lower fidelity (range: 18-93%) from teachers to students. Physical activity programs for girls, a unique feature of the TAAG intervention, increased from a mean of 10 programs per school to a mean of 16 and 15 in years 1 and 2, respectively, in intervention schools, with no change in control schools. These findings suggest that a multicomponent school- and community-based physical activity intervention can be delivered with fidelity and result in a middle school environment that supports physical activity for girls.

Latina recruitment for cancer prevention education via Community Based Participatory Research strategies

Increasing minority participation in cancer research is an ethical and statistical necessity for gaining population-specific knowledge of cancer prevention, screening, and treatment. Locating and recruiting eligible and willing minority participants presents unique structural and cultural/linguistic challenges. Community Based Participatory Research provides a viable set of principles for facilitating recruitment in hard-to-recruit communities. We focus on the specific challenge of recruiting and engaging low-income and underinsured Latina women in cancer prevention education research, and present community-based strategies used to recruit women into a recently completed study in Arizona, Juntos en la Salud (Together in Health). Community representatives and promotoras' (Latino community health educators) involvement in site identification, individual recruitment, and development of strategies and materials for the interventions built engagement and trust. These strategies resulted in enrollment of an especially low-income, underinsured population. To emphasize the degree to which a particularly underserved population was recruited, we present data comparing demographic and screening profiles of enrollees to the general population of Latinos in Arizona.

Process evaluation of the Pool Cool Diffusion Trial for skin cancer prevention across 2 years

Though process evaluation of health programs has received growing attention, few interventions have reported process evaluation over multiple years. This article describes 2 years of process evaluation (2003-04) for the Pool Cool Diffusion Trial. Pool Cool is a skin cancer prevention program designed to increase sun protection habits among children and improve organizational and environmental supports for sun protection at swimming pools. Each year, 80 telephone interviews and 40 site visits at pools across the United States were completed, to examine how fully the program was implemented and the extent of use of program components between the two study conditions. Major components of the Pool Cool program, including sun safety lessons, sun safety signs and sunscreen use, had high implementation. Between the 2 years, most of the core elements were either maintained or increased in use. There were no significant differences between the basic and enhanced conditions on implementation. Reasons given for successful implementation were the provision of a toolkit, ease of implementing the program, pool staff and children enjoying the program and the field coordinators' support. These data provide information on programmatic factors that contribute to successful program diffusion.

Enhancing schools' capacity to support children in poverty: an ecological model of school-based mental health services

School based mental health services for children in poverty can capitalize on schools' inherent capacity to support development and bridge home and neighborhood ecologies. We propose an ecological model informed by public health and organizational theories to refocus school based services in poor communities on the core function of schools to promote learning. We describe how coalescing mental health resources around school goals includes a focus on universal programming, mobilizing indigenous school and community resources, and supporting core teaching technologies. We suggest an iterative research-practice approach to program adaptation and implementation as a means toward advancing science and developing healthy children.

Determining client need in a multi-state fetal alcohol syndrome consortium: from training to practice

Background

A multi-state consortium was developed in the US to conduct baseline data collection and intervention research on fetal alcohol syndrome. Each state employed support specialists whose job it was to reduce or eliminate alcohol consumption in women who were at high risk for drinking alcohol during their pregnancy. The purpose of this paper is to report how support specialists in three primarily rural/frontier states were trained to assess client need and how client need was actually assessed in the field.

Methods

A qualitative process evaluation was conducted using semi-structured interviews. Interviews were conducted with state staff involved in support specialist training and consortium activities and the support specialists themselves. Inductive analyses were conducted with interview data.

Results

Need determination varied by state and for one state within the state. How support specialists were trained to assess need and how need was assessed in the field was mostly congruent.

Conclusion

Process evaluation is an effective method for providing practical and useful answers to questions that cannot be answered by outcome evaluation alone.

Understanding non-industrialized workers' approaches to safety: how do commercial fishermen "stay safe"?

BACKGROUND

Commercial fishing is carried out worldwide, often in non-industrialized forms, and is associated with high rates of fatal and non-fatal occupational injury.

PROBLEM

Fishermen who work independently in non-industrialized settings do not have access to union or industry sponsored safety services and must make their own decisions about safety practices. Learning the meaning of safety for them and the safety measures they employ is important before developing interventions.

METHODS

Two fieldworkers conducted in-depth ethnographic interviews with 31 commercial fishermen in North Carolina. Interviews and fieldnotes were analyzed using QSR N5.

RESULTS

Fishermen primarily related staying safe to work practices and attitudes. They identified specific safety measures, appropriate gear and boat maintenance, weather decisions, and working cooperatively when ocean fishing.

DISCUSSION

The ethnographic research process can produce information about a group's norms of preventive behavior and safety concerns. Knowledge of workers' concepts and practices will inform researchers' inquiries.

Avaliação de um programa de acompanhamento pré-natal em meio carente

A medicina tradicional e os programas em saúde pública não conseguiraram prevenir as conseqüências desfavoráveis da gravidez de alto risco em mulheres carentes. O objetivo desta pesquisa é descrever e avaliar o processo de emancipação de um grupo de gestantes carentes, de um grupo de mães da mesma comunidade e de profissionais da saúde que participam de um programa de acompanhamento pré-natal. Análises qualitativas de entrevistas individuais e coletivas levaram ao aparecimento de uma proposta de teoria de nível intermediário sobre o acompanhamento pré-natal, na qual são revelados elementos facilitadores e limitadores ao processo de implantação. Se a saúde pública deve concentrar-se nos problemas de justiça social, deveremos rever os modelos de parceria e de complementaridade para pensar, provavelmente, em nos direcionarmos para outros modelos.

Examining the link between program implementation and behavior outcomes in the lifestyle education for activity program (LEAP)

Lifestyle Education for Activity Program (LEAP) was a comprehensive, school-based intervention designed to promote physical activity in high school girls. The intervention focused on changes in instructional practices and the school environment to affect personal, social, and environmental factors related to physical activity. Multiple process evaluation tools and an organizational assessment tool were developed to monitor program implementation from a framework called the LEAP essential elements, which characterized complete and acceptable intervention delivery; secular trends were also monitored. Using process data, LEAP intervention schools were categorized into low- and high-implementing groups and compared with control schools on nine essential elements assessed at the organizational level. The Wilcoxon scores test revealed that low- and high-implementing intervention, and control schools differed significantly on two of nine administrator-reported organizational-level components: having a physical activity team and having a faculty-staff health promotion program. A mixed-model analysis of covariance indicated that, compared to control schools, a greater percentage of girls in high-implementing schools reported engaging in vigorous physical activity. Process evaluation can be used to understand the relationship between level of implementation and successful program outcome.

Worker participation in change processes in a Danish industrial setting

BACKGROUND

Improving the design, management and organization of work may be an important step in improving occupational health. An intervention, guided by the principles of participatory action research (PAR), is directed at traditional work environment problems in the epoxy plastic industry, that is, eczema and accident-related injuries.

METHODS

The study population consisted of employees at two wind turbine- manufacturing plants. A quasi-experimental design was employed with before and after measurements and a comparison group with a 3(1/2) year follow-up period.

RESULTS

The role of employee elected safety representatives was changed from one of controlling and "policing" to that of safety advisors. The attitudes of employees also changed, from an individualistic understanding of safety as the responsibility of the single employee, to a more collective understanding of safety as being everyone's shared responsibility. Structural changes led to a less hierarchical management system. This process led eventually to the establishment of self-governing work groups in which each member had a well-defined area of responsibility. The change process was associated with improvements in the psychosocial work environment and safety climate, a 66% reduction in the incidence of eczema, and a 48.6% reduction in the incidence of occupational accidents. In the comparison population, a twin factory under the same company, similar but delayed and less dramatic changes also occurred.

CONCLUSIONS

Implementation of a comprehensive intervention was followed by improved employee perceptions of the company's safety standards and the psychosocial work environment, as well as by substantial reductions in the incidence of eczema and occupational accidents.

Process evaluation results from the Healthy Directions-Small Business study

The Healthy Directions-Small Business randomized, controlled study aimed to reduce cancer risk among multiethnic workers in small manufacturing businesses by increasing fruit and vegetable consumption, physical activity, and daily multivitamin in take and decreasing consumption of red meat. The intervention incorporated participatory strategies and was built on a social-contextual framework that addressed people with varying cultural backgrounds and literacy levels. In addition, the intervention aimed to reduce worker exposure to occupational hazards. Process evaluation was conducted using quantitative and qualitative research methods. Quantitative results showed high levels of worker awareness of and participation in programs. Qualitative findings suggested that management support, worker input, and a history of social interaction between workers and management may have contributed to high participation rates. Future studies need to examine characteristics associated with participation and nonparticipation of both managers and nonmanagers to increase the likelihood of participation and ultimately improve health behavior.

Cultural relevancy of a diabetes prevention nutrition program for African American women

Diabetes among African American women is a pressing health concern, yet there are few evaluated culturally relevant prevention programs for this population. This article describes a case study of the Eat Well Live Well Nutrition Program, a community-based, culturally specific diabetes prevention nutrition program for African American women. The stages of change theory and principles from community organization guided the development of the program. Health education strategies, including participatory development and program delivery by peer educators, were applied to promote cultural relevance. Results indicated that overall participants (90%) believed the program to be culturally relevant and were very satisfied with the program (82%). Cultural relevancy was significantly associated with greater program satisfaction and changes in dietary patterns when controlling for the number of sessions attended. Conclusions suggest that participatory strategies can be effective in designing culturally specific prevention programs for African American women.

A longitudinal assessment of inter-sectoral participation in a community-based diabetes prevention programme

To understand the development of inter-sectoral participation in the three intervention municipalities of Stockholm Diabetes Prevention Programme (SDPP) case studies with a longitudinal assessment were conducted using the spidergram method, document analysis and group discussions. At three time points, the members of the local steering committees assessed the extent of participation from narrow to wide inter-sectoral participation in five key areas: planning, resources, leadership, network and implementation. Wide participation of various interest groups was recognised in planning and implementing activities whereas local resources, the representation of the leadership and the extent of the network were perceived as more restricted. Expert involvement varied during the programme period but was not regarded as exerting control over the local programmes. Participation within the local steering committees decreased, with a stronger focus on the project co-ordinator and other local partners in latter years. The extent of partner engagement increased due to focusing on activities approaching multi-sector collaboration and institutionalisation. Overall, communication and shared responsibility appeared critical in influencing both the development and perception of participation. In conclusion, to understand the dynamic process of participation at different times, areas and levels, the development and use of evaluation designs combining different methods and information sources throughout the lifespan of a project are recommended.

Small-group hypertension health education programme: a process and outcome evaluation

AIM

This paper reports a study to explore the expectations, attainments and opinions of the participants, the objectives and experiences of nurse facilitators, and the changes in nurse-client relationship after a small-group hypertension health education programme.

BACKGROUND

Effective control of blood pressure among hypertensive patients is essential in the reduction of cerebrovascular-related morbidity and mortality. A 'small-group hypertension education programme' has been implemented in all general outpatient clinics in Hong Kong since 2000. In order to identify what aspects of the programme were beneficial and how it benefited clients, a process evaluation study was conducted.

METHODS

Following implementation of a small-group hypertension health education programme, two focus groups were run to collect data from eight nurse facilitators, and individual semi-structured interviews were conducted with 16 programme participants. The data were collected in 2001.

FINDINGS

The findings indicate that both clients and nurses gained a great deal from and enjoyed the process of this health education programme, and their relationships were improved after the programme. About half of the clients reported having adopted health-enhancing behaviour. The study also revealed discrepancies between the nurses and clients in their perceptions of the objectives of the programme. This indicates that there is a need to clarify the objectives with clients to minimize disappointments from unrealistic expectations.

CONCLUSION

There is a need for the nurse facilitators to clarify the goal of the programme as an education group rather than as a support group. Also, needs assessment is suggested before group member recruitment in order to make the group more homogeneous.

Public health advocacy to change corporate practices: implications for health education practice and research

Corporate practices, such as advertising, public relations, lobbying, litigation, and sponsoring scientific research, have a significant impact on the health of the people in the United States. Recently, health professionals and advocates have created a new scope of practice that aims to modify corporate practices that harm health. This article describes how corporate policies influence health and reviews recent health campaigns aimed at changing corporate behavior in six industries selected for their central role in the U.S. economy and their influence on major causes of mortality and morbidity. These are the alcohol, automobile, food, gun, pharmaceutical, and tobacco industries. The article defines corporate disease promotion and illustrates the range of public health activities that have emerged to counter such corporate behaviors. It analyzes the role of health professionals, government, and advocacy groups in these campaigns and assesses the implications of this domain for health education practice and research.

Feasibility of implementing intervention methods in an adolescent worksite tobacco control study

OBJECTIVE

To present feasibility data on SMART, the first teen worksite behavioural tobacco control intervention.

DESIGN

This phase II study was designed to estimate the efficacy and feasibility of a small scale, randomised, controlled intervention.

SETTING AND SUBJECTS

This study, addressing youths aged 15-18 years, was implemented in four intervention and five control grocery stores that had an average of 44 eligible teens.

INTERVENTIONS

The tobacco use cessation and prevention interventions were based on social influences and peer leader models. Employee break rooms served as centres both for interactive activities including open houses, teen advisory boards, peer leader interviews, games and contests; and non-interactive interventions including bulletin boards and table tents with health messages and home mailings. MAIN PROCESS MEASURES: Project staff collected process data on the extent of implementation of intervention activities, participation rates in activities, and contacts with peer leaders. On the final survey, teens reported on awareness of, participation in, and motivation for participating in project activities.

RESULTS

Indicators of feasibility were identified and discussed, including the number of activities implemented, teen participation, management support, cost, and barriers to and facilitators of implementation. During the 12 month intervention, a mean of 24.1 interactive activities and 55.3 non-interactive activities were implemented, and a mean 14.2% participation rate per activity per site was achieved. Eighty four per cent of teens reported being aware of SMART, and 39% reported participating in interactive and 67% in non-interactive activities.

CONCLUSIONS

Teen smoking cessation rates in worksite programmes might be improved if they are conducted in companies where there is job stability and if teen programmes are part of worksite-wide tobacco control programmes that include both teens and adults.