Process evaluations of the 5-a-day projects

Process evaluation of school-based high-intensity interval training interventions for children and adolescents: a systematic review and meta-analysis of randomized controlled trials

BACKGROUND

Several systematic reviews have been published to investigate the effectiveness of high-intensity interval training (HIIT) in schools. However, there has been limited attention given to understanding the functioning of the intervention processes, which is of paramount importance for interpreting and translating the intervention effectiveness. The aim of this systematic review is to determine the extent to which process evaluation is measured in school-based HIIT interventions and to explore the effects of process evaluation and intervention characteristics on cardiorespiratory fitness (CRF), body composition, muscular strength, and blood pressure.

METHODS

A comprehensive search was conducted in SPORT Discus (EBSCOhost), Web of Science, Scopus, Medline (Ovid) and Cochrane Central Register of Controlled Trials. The extent to which process evaluation is measured was narratively reported, alongside with the guidance of process evaluation of complex interventions by UK Medical Research Council. Meta-analyses and meta-regressions were conducted to determine the effects of process evaluation and intervention characteristics to the intervention outcomes.

RESULTS

The literature search identified 77 studies reporting on 45 school-based HIIT interventions. In total, five interventions reported process evaluation in a section or in a separate study, and only one intervention adopted a process evaluation framework. On average, 6 out of 12 process evaluation measures were reported in all interventions. Subgroup analyses did not indicate any beneficial treatment effects for studies with process evaluation group, whereas all pooled data and studies without process evaluation group showed significant improvement for CRF and body composition.

CONCLUSION

Process evaluation is frequently omitted in the literature of school-based HIIT in children and adolescents. Although reporting of process evaluation measures may not directly associate with better intervention outcomes, it allows accurate interpretation of intervention outcomes, thereby enhancing the generalisability and dissemination of the interventions.

HipHop2SToP a community-led health promotion initiative empowering Aboriginal youth in the Kimberley region of Western Australia: a process evaluation

Introduction

For millennia, Aboriginal people's ways of knowing, doing and being were shared through art, song, and dance. Colonisation silenced these ways, affecting loss of self-determination for Aboriginal people. Over the past decade in Australia, hip-hop projects have become culturally appropriate approaches for health promotion. When community led, and Aboriginal worldviews centralised, hip-hop workshops are more likely to be effective. In 2020, during the COVID-19 pandemic, a community-led health promotion hip-hop music video, 'HipHop2SToP' was produced involving young people in Dampier Peninsula communities address healthy skin and healthy living practices.

Methods

We report here a qualitative process evaluation of the HipHop2SToP project. Participants who had been involved in the planning and production of HipHop2SToP were selected using a purposive approach and invited either by email or face-to-face to participate in semi-structured interviews and share their experiences. Semi-structured interviews ranged from 30 to 60 min in duration and were conducted either face-to-face or virtually over MS Teams. Due to personal time constraints, two participants provided written responses to the semi-structured questions. All interviews were audio-recorded with consent and saved as a digital recording in a de-identified format. All audio recordings were transcribed verbatim and uploaded into QSR NVivo v12 along with written responses.

Results

As a health promotion project, the critical success factors were community-ownership and discovering novel ways to collaborate virtually with remote communities using Microsoft (MS) software. Highlights included observing the young people actively engaged in the project and their catchy lyrics and key messaging for environmental health and skin infections. COVID-19 presented some challenges. Gaps in communication, clarification of stakeholder roles and expectations, and post-production outcomes were also identified as challenges.

Conclusion

HipHop2SToP validates the need for Aboriginal community led health promotion programs. While creating some challenges COVID-19 also strengthened community ownership and created novel ways of maintaining relationships with remote Aboriginal communities. Future hip-hop projects would benefit from clarity of roles and responsibilities. Strengthening post-production outcomes by including a launch and well-planned, targeted communication and dissemination strategy will ensure the wider translation of important health messages and potential strengthen sustainability.

Chronic Headache Education and Self-Management Study (CHESS): a process evaluation

BACKGROUND

The Chronic Headache Education and Self-Management Study (CHESS) multicentre randomised trial evaluated the impact a group education and self-management support intervention with a best usual care plus relaxation control for people living with chronic headache disorders (tension type headaches or chronic migraine, with or without medication overuse headache). Here we report the process evaluation exploring potential explanations for the lack of positive effects from the CHESS intervention.

METHODS

The CHESS trial included 736 (380 intervention: 356 control) people across the Midlands and London UK. We used a mixed methods approach. Our extensive process evaluation looked at context, reach, recruitment, dose delivered, dose received, fidelity and experiences of participating in the trial, and included participants and trial staff. We also looked for evidence in our qualitative data to investigate whether the original causal assumptions underpinning the intervention were realised.

RESULTS

The CHESS trial reached out to a large diverse population and recruited a representative sample. Few people with chronic tension type headaches without migraine were identified and recruited. The expected 'dose'of the intervention was delivered to participants and intervention fidelity was high. Attendance ("dose received") fell below expectation, although 261/380 (69%) received at least at least the pre-identified minimum dose. Intervention participants generally enjoyed being in the groups but there was little evidence to support the causal assumptions underpinning the intervention were realised.

CONCLUSIONS

From a process evaluation perspective despite our extensive data collection and analysis, we do not have a clear understanding of why the trial outcome was negative as the intervention was delivered as planned. However, the lack of evidence that the intervention causal assumptions brought about the planned behaviour change may provide some insight. Our data suggests only modest changes in managing headache behaviours and some disparity in how participants engaged with components of the intervention within the timeframe of the study. Moving forwards, we need a better understanding of how those who live with chronic headache can be helped to manage this disabling condition more effectively over time.

TRIAL REGISTRATION

ISRCTN79708100 .

Position of the Society for Nutrition Education and Behavior: Nutrition Educator Competencies for Promoting Healthy Individuals, Communities, and Food Systems: Rationale and Application

It is the position of the Society for Nutrition Education and Behavior that to improve the health of individuals, communities, and food systems, it is essential that nutrition educators meet each of 6 content competencies (basic food and nutrition knowledge, nutrition across the life cycle, food science, physical activity, food and nutrition policy, and agricultural production and food systems) and 4 process competencies (behavior and education theory; nutrition education program design, implementation and evaluation; written, oral, and social media communication; and nutrition education research methods). These competencies reflect the breadth of the nutrition education field and are grounded in peer-reviewed research. The rationale and evidence base for these competencies are presented. They are designed for educational institutions to plan curricula and programs; public, private, and nonprofit organizations for training; individuals for professional development; and policymakers and advocates to inform strong, comprehensive nutrition education policy.

Promotors and barriers to the implementation and adoption of assistive technology and telecare for people with dementia and their caregivers: a systematic review of the literature

BACKGROUND

One of the most pressing issues in our society is the provision of proper care and treatment for the growing global health challenge of ageing. Assistive Technology and Telecare (ATT) is a key component in facilitation of safer, longer, and independent living for people with dementia (PwD) and has the potential to extend valuable care and support for caregivers globally. The objective of this study was to identify promotors and barriers to implementation and adoption of ATT for PwD and their informal (family and friends) and formal (healthcare professionals) caregivers.

METHODS

Five databases Medline (Ovid), CINAHL, Web of Science, APA PsycINFO and EMBASE were searched. PRISMA guidelines have been used to guide all processes and results. Retrieved studies were qualitative, mixed-method and quantitative, screened using Rayyan and overall quality assessed using Critical Appraisal Skills Programme (CASP) and Mixed Methods Assessment Tool (MMAT). Certainty of evidence was assessed using Grading of Recommendations Assessment, Development and Evaluation (GRADE) criteria and assigned within categories of high, moderate, or low. NVivo was used for synthesis and analysis of article content. A narrative synthesis combines the study findings.

RESULTS

Thirty studies (7 quantitative, 19 qualitative and 4 mixed methods) met the inclusion criteria. Identified primary promotors for the implementation and adoption of ATT were: personalized training and co-designed solutions, safety for the PwD, involvement of all relevant stakeholders, ease of use and support, and cultural relevance. Main barriers for the implementation and adoption of ATT included: unintended adverse consequences, timing and disease progress, technology anxiety, system failures, digital divide, and lack of access to or knowledge of available ATT.

CONCLUSION

The most crucial elements for the adoption of ATT in the future will be a focus on co-design, improved involvement of relevant stakeholders, and the adaptability (tailoring related to context) of ATT solutions over time (disease process).

What do we want to get out of this? a critical interpretive synthesis of the value of process evaluations, with a practical planning framework

BACKGROUND

Process evaluations aim to understand how complex interventions bring about outcomes by examining intervention mechanisms, implementation, and context. While much attention has been paid to the methodology of process evaluations in health research, the value of process evaluations has received less critical attention. We aimed to unpack how value is conceptualised in process evaluations by identifying and critically analysing 1) how process evaluations may create value and 2) what kind of value they may create.

METHODS

We systematically searched for and identified published literature on process evaluation, including guidance, opinion pieces, primary research, reviews, and discussion of methodological and practical issues. We conducted a critical interpretive synthesis and developed a practical planning framework.

RESULTS

We identified and included 147 literature items. From these we determined three ways in which process evaluations may create value or negative consequences: 1) through the socio-technical processes of 'doing' the process evaluation, 2) through the features/qualities of process evaluation knowledge, and 3) through using process evaluation knowledge. We identified 15 value themes. We also found that value varies according to the characteristics of individual process evaluations, and is subjective and context dependent.

CONCLUSION

The concept of value in process evaluations is complex and multi-faceted. Stakeholders in different contexts may have very different expectations of process evaluations and the value that can and should be obtained from them. We propose a planning framework to support an open and transparent process to plan and create value from process evaluations and negotiate trade-offs. This will support the development of joint solutions and, ultimately, generate more value from process evaluations to all.

Starting the SToP trial: Lessons from a collaborative recruitment approach

OBJECTIVE

Recruitment in research can be challenging in Australian Aboriginal contexts. We aimed to evaluate the SToP (See, Treat, Prevent skin infections) trial recruitment approach for Aboriginal families to identify barriers and facilitators and understand the utility of the visual resource used.

METHODS

This qualitative participatory action research used purposive sampling to conduct six semi-structured interviews with staff and five yarning sessions with Aboriginal community members from the nine communities involved in the SToP trial that were audio recorded and transcribed verbatim before thematic analysis.

FINDINGS

Community members valued the employment of local Aboriginal facilitators who used the flipchart to clearly explain the importance of healthy skin and the rationale for the SToP trial while conducting recruitment. A prolonged process, under-developed administrative systems and stigma of the research topic emerged as barriers.

CONCLUSION

Partnering with a local Aboriginal organisation, employing Aboriginal researchers, and utilising flip charts for recruitment was seen by some as successful. Strengthening governance with more planning and support for recordkeeping emerged as future success factors.

IMPLICATIONS FOR PUBLIC HEALTH

Our findings validate the importance of partnership for this critical phase of a research project. Recruitment strategies should be co-designed with Aboriginal research partners. Further, recruitment rates for the SToP trial provide a firm foundation for building partnerships between organisations and ensuring Aboriginal perspectives determine recruitment methods.

Is level of implementation linked with intervention outcomes? Process evaluation of the TransformUs intervention to increase children’s physical activity and reduce sedentary behaviour

Background

TransformUs was a four-arm school-based intervention to increase physical activity and reduce sedentary behaviour among primary school children. Pedagogical and environmental strategies targeted the classroom, school grounds and family setting. The aims of this study were to evaluate program fidelity, dose, appropriateness, satisfaction and sustainability, and associations between implementation level and outcomes among the three intervention arms.

Methods

At baseline, 18-months (mid-intervention) and 30-months (post-intervention), teachers, parents and children completed surveys, and children wore GT3X ActiGraph accelerometers for 8 days at each time point to determine physical activity and sedentary time. Implementation data were pooled across the three intervention groups and teachers were categorised by level of implementation: (i) ‘Low’ (< 33% delivered); (ii) ‘Moderate’ (33–67% delivered); and (iii) ‘High’ (> 67% delivered). Linear and logistic mixed models examined between group differences in implementation, and the association with children’s physical activity and sedentary time outcomes. Qualitative survey data were analysed thematically.

Results

Among intervention recipients, 52% (n = 85) of teachers, 29% (n = 331) of parents and 92% (n = 407) of children completed baseline evaluation surveys. At 18-months, teachers delivered on average 70% of the key messages, 65% set active/standing homework, 30% reported delivering > 1 standing lesson/day, and 56% delivered active breaks per day. The majority of teachers (96%) made activity/sports equipment available during recess and lunch, and also used this equipment in class (81%). Fidelity and dose of key messages and active homework reduced over time, whilst fidelity of standing lessons, active breaks and equipment use increased. TransformUs was deemed appropriate for the school setting and positively received. Implementation level and child behavioural outcomes were not associated. Integration of TransformUs into existing practices, children’s enjoyment, and teachers’ awareness of program benefits all facilitated delivery and sustainability.

Conclusions

This study demonstrated that intervention dose and fidelity increased over time, and that children’s enjoyment, senior school leadership and effective integration of interventions into school practices facilitated improved intervention delivery and sustainability. Teacher implementation level and child behavioural outcomes were unrelated, suggesting intervention efficacy was achieved irrespective of implementation variability. The potential translatability of TransformUs into practice contexts may therefore be increased. Findings have informed scale-up of TransformUs across Victoria, Australia.

Trial registration

International Standard Randomized Controlled Trial Number ISRCTN83725066; Australian New Zealand Clinical Trials Registry Number ACTRN12609000715279. Registered 19 August 2009. Available at: https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=308387&isReview=true

Supplementary Information

The online version contains supplementary material available at 10.1186/s12966-022-01354-5.

Implementing the Dementia Early Stage Cognitive Aids New Trial (DESCANT) intervention: mixed-method process evaluation alongside a pragmatic randomised trial

OBJECTIVES

The DESCANT (Dementia Early Stage Cognitive Aids New Trial) intervention provided a personalised care package designed to improve the cognitive abilities, function and well-being of people with early-stage dementia and their carers, by providing a range of memory aids, together with appropriate training and support. This sub-study aimed to assess implementation and identify contextual factors potentially associated with participant outcomes.

METHOD

A mixed-methods approach was adopted alongside the pragmatic randomised trial. Data were obtained from intervention records and interviews with five dementia support practitioners across seven National Health Service Trusts in England and Wales. A reporting framework was constructed from available literature and data assessed by descriptive statistics and thematic analysis.

RESULTS

Participation and engagement was high with 126 out of 128 participants completing the intervention with packages tailored to individual participants. Misplacing items and poor orientation to date and time were common areas of need. Memory aids frequently supplied included orientation clocks (91%), whiteboards (60%), calendars (43%) and notebooks (32%), plus bespoke items. Intervention duration and timing were broadly consistent with expectations. Variation reflected participants' needs, circumstances and preferences. Qualitative findings suggested a potentially positive impact on the well-being of people with dementia and their carers. Issues associated with successful roll-out of the intervention are explored in the discussion.

CONCLUSION

Successful implementation increased confidence in future findings of the randomised trial. Depending on these, DESCANT may prove a scalable intervention with potential to improve the function and quality of life of people with dementia and their carers.

Evaluation of Circadian Rhythm and Sleep Focused mHealth Intervention for the Prevention of Accelerated Summer Weight Gain among Elementary School-Age Children: Protocol for a Randomized Controlled Feasibility Study (Preprint)

Background

The i♥rhythm project is a mobile health adaptation of interpersonal and social rhythm therapy designed to promote healthy sleep and behavioral rhythms among 5-8-year olds during summer for the prevention of accelerated summer weight gain.

Objective

This pilot study will examine the feasibility, acceptability, and preliminary efficacy of the i♥rhythm intervention. This will ensure that the research protocol and procedures work as desired and are acceptable to families in preparation for the fully powered randomized controlled trial. The proposed study will examine the willingness of participants to participate in the intervention and determine whether modifications to the intervention, procedures, and measures are needed before conducting a fully powered study. We will assess our ability to (1) recruit, consent, and retain participants; (2) deliver the intervention; (3) implement the study and assessment procedures; (4) assess the reliability of the proposed measures; and (5) assess the acceptability of the intervention and assessment protocol.

Methods

This study will employ a single-blinded 2-group randomized control design (treatment and no-treatment control) with randomization occurring after baseline (Time 0) and 3 additional evaluation periods (postintervention [Time 1], and 9 months [Time 2] and 12 months after intervention [Time 3]). A sample of 40 parent-child dyads will be recruited.

Results

This study was approved by the institutional review board of Baylor College of Medicine (H-47369). Recruitment began in March 2021. As of March 2022, data collection and recruitment are ongoing.

Conclusions

This study will address the role of sleep and circadian rhythms in the prevention of accelerated summer weight gain and assess the intervention’s effects on the long-term prevention of child obesity.

Trial Registration

ClinicalTrials.gov NCT04445740; https://clinicaltrials.gov/ct2/show/NCT04445740.

International Registered Report Identifier (IRRID)

DERR1-10.2196/37002

The Triple P Positive Parenting Program for Parents With Psychosis: A Case Series With Qualitative Evaluation

Although many people with psychosis are parents, managing the dual demands of poor mental health and parenting can be stressful and may contribute to poorer outcomes for both parent and child. Parenting interventions have the potential to improve outcomes for the whole family but need evaluation of feasibility in this context. The Triple-P Self-Help Workbook was implemented with guidance and support with 10 parents experiencing psychosis in a multiple baseline case series study. Sessions were weekly and home-based. Outcome measures examined facets of parenting, child behavior, self-efficacy and parental mental health. Follow up interviews explored parents' perspectives of the perceived impact of the intervention and apparent mechanisms of change. The program resulted in clinically significant change (>25% improvement) in mental health, parenting and child behavior measures post-intervention for the 50% who completed all 10 sessions and improvements were maintained at 3 and 6 month follow up. Interviews with those who completed the program revealed it to have been transformative: parents reported positive changes in parenting style; they were empowered with regard to their parenting and had a greater sense of control over their mental health. This study provides preliminary evidence that self-directed Triple P might be able to reduce the symptoms of psychosis by improving family functioning. Findings could inform the future development or adaptation of evidence-based parenting interventions for parents with psychosis in order to improve their mental health, aid recovery, and intervene early in the lives of children at risk of poor long-term outcomes.

Monitoring of the Village Malaria Workers to conduct activities of Malaria Elimination Demonstration Project in Mandla, Madhya Pradesh

Background

The capacity of the field staff to conduct activities related to disease surveillance, case management, and vector control has been one of the key components for successfully achieving malaria elimination. India has committed to eliminate malaria by 2030, and it has placed significance on monitoring and evaluation at the district level as one of the key strategies in its national framework. To support and guide the country’s malaria elimination objectives, the Malaria Elimination Demonstration Project was conducted in the tribal district of Mandla, Madhya Pradesh. Robust monitoring of human resources received special attention to help the national programme formulate a strategy to plug the gaps in its supply chain and monitoring and evaluation systems.

Methods

A monitoring tool was developed to test the capabilities of field workers to conduct activities related to malaria elimination work. Between November 2018 to February 2021, twenty-five Malaria Field Coordinators (MFCs) of the project utilized this tool everyday during the supervisory visits for their respective Village Malaria Workers (VMWs). The data was analysed and the scores were tested for variations against different blocks, educational status, duration of monitoring, and post-training scores.

Results

During the study period, the VMWs were monitored a total of 8974 times using the monitoring tool. Each VMW was supervised an average of 1.8 times each month. The critical monitoring indicators scored well in all seven quarters of the study as monitored by the MFCs. Monitoring by MFCs remained stable at 97.3% in all quarters. Contrary to expectations, the study observed longer diagnosis to treatment initiation time in urban areas of the district.

Conclusion

This study demonstrated the significance of a robust monitoring tool as an instrument to determine the capacity of the field workers in conducting surveillance, case management, and vector control related work for the malaria elimination programme. Similar tools can be replicated not only for malaria elimination, but other public health interventions as well.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12936-021-04040-2.

Process Evaluation in Health Communication Media Campaigns: A Systematic Review

Objective. The objective is to examine the scope of health communication media campaign process evaluation methods, findings, and dissemination practices. Data Source. A systematic review of peer-reviewed literature was conducted using database searches. Study Inclusion and Exclusion Criteria. Published studies on process and implementation evaluation of health campaigns with a media component were included. Exclusion criteria included not health, non-empirical, no media campaign, or a focus on other evaluation types. Data Extraction. Articles were assessed for general campaign information, theory use, and details about process evaluation plan and procedures. Data Synthesis. A coding scheme based on 9 process evaluation best practice elements (e.g., fidelity and context) was applied. Process evaluation methods, measures, and reporting themes were synthesized. Results. Among 691 unique records, 46 articles were included. Process evaluation was the main focus for 71.7% of articles, yet only 39.1% reported how process evaluation informed campaign implementation strategy. Articles reported 4.39 elements on average (SD = 1.99; range 1-9), with reach (87.0%) and recruitment (73.9%) described most frequently, yet reporting was inconsistent. Further, the level of detail in reporting methods, theory, and analysis varied. Conclusions. Process evaluation provides insight about mechanisms and intervening variables that could meaningfully impact interpretations of outcome evaluations; however, process evaluations are less often included in literature. Recommendations for evidence-based process evaluation components to guide evaluation are discussed.

Implementation Science Strategies Promote Fidelity in the Food, Feeding, and Your Family Study

OBJECTIVE

Use of implementation science strategies to promote fidelity in the Food, Feeding, and Your Family study.

DESIGN

Cluster randomized controlled trial with 3 conditions: control, in-class, or online, delivered in English or Spanish. Observations of 20% of classes.

SETTING

Expanded Food and Nutrition Education Program (EFNEP) in 2 states.

PARTICIPANTS

EFNEP peer educators (n = 11).

INTERVENTION

Parental feeding content incorporated into EFNEP lessons (in-class) or through text with links to videos/activities (online). Extensive educator training, scripted curriculum, frequent feedback.

ANALYSIS

Assessment of fidelity compliance. Qualitative analysis of verbatim educator interviews and classroom observer comments.

RESULTS

During 128 class observations (40-45 per condition), peer educators followed scripted lesson plan 78% to 89% of the time. There was no evidence of cross-contamination of parental feeding content in control and only minor sharing in online conditions. Variations with fidelity were primarily tied to the EFNEP curriculum, not the parent feeding content. Educators (n = 7) expressed favorable opinions about the Food, Feeding, and Your Family study, thought it provided valuable information, and appreciated support from EFNEP leadership.

CONCLUSIONS AND IMPLICATIONS

Incorporating implementation science strategies can help ensure successful adherence to research protocols. With proper training and support, EFNEP peer educators can deliver an evidence-based curriculum as part of a complex research study.

Patient Uptake, Experiences, and Process Evaluation of a Randomized Controlled Trial of Internet-Based Cognitive Behavioral Therapy for Tinnitus in the United States

Introduction: An internet-based cognitive behavioral therapy (ICBT) offers a way to increase access to evidence-based tinnitus care. To increase the accessibility of this intervention, the materials were translated into Spanish to reach Spanish as well as English speakers. A clinical trial indicated favorable outcomes of ICBT for tinnitus for the population of the United States. In view of later dissemination, a way to increase the applicability of this intervention is required. Such understanding is best obtained by considering the perspectives and experiences of participants of an intervention. This study aimed to identify the processes that could facilitate or hinder the clinical implementation of ICBT in the United States. Methods: This study evaluated the processes regarding enrolment, allocation, intervention delivery, the outcomes obtained, and the trial implementation. The study sample consisted of 158 participants who were randomly assigned to the experimental and control group. Results: Although the recruitment was sufficient for English speakers, recruiting the Spanish participants and participants belonging to ethnic minority groups was difficult despite using a wide range of recruitment strategies. The allocation processes were effective in successfully randomizing the groups. The intervention was delivered as planned, but not all the participants chose to engage with the materials provided. Compliance for completing the outcome measures was low. The personal and intervention factors were identified as barriers for the implementation whereas the facilitators included the support received, being empowering, the accessibility of the intervention, and its structure. Conclusion: An understanding regarding the factors contributing to the outcomes obtained, the barriers and facilitators of the results, engagement, and compliance were obtained. These insights will be helpful in preparing for the future dissemination of such interventions. Clinical Trial Registration: www.ClinicalTrials.gov, identifier: NCT04004260. Registered on 2 July 2019.

Using Process Evaluation Results to Compare Peer and Adult Leader Delivery of the PAWS (Peer-Education about Weight Steadiness) Club Program

To date, there is limited published literature on process evaluation of adolescent health promotion programs. In this paper, we describe the methods and results of PAWS Club process evaluation over 2 years of implementation to compare the effectiveness of delivery by peer and adult leaders. PAWS (Peer-education About Weight Steadiness) Club was a 12-week healthy lifestyle program, delivered to 6th and 7th graders by peer and adult educators, using cluster randomized controlled design. Peer educators were 8th graders in the program schools and adult educators were staff/teachers in the program schools. Trained university students filled out fidelity logs at each session led by peer and adult educators to assess program delivery. The fidelity logs included questions to collect information about the number of participants, duration of the session, percent of activities completed, and if lessons started on time, lesson objectives were clearly stated, lesson objectives were emphasized, demonstrations were visible to participants, all activities were completed, the leader was familiar with lessons, the leader maintained an appropriate pace, the leader kept participants on track, and the leader asked if participants had any questions. Adult educators had a higher mean performance for all questions compared to peer leaders. Significant differences were observed for emphasizing lesson objectives (p = 0.005), making demonstrations visible to participants (p = 0.031), being familiar with the lesson plan (p = 0.000), maintaining an appropriate pace (p = 0.000), keeping participants on track (p = 0.000), and asking if participants had any questions (p = 0.000). Significance was set at p < 0.05. Findings from the current study have implications for designing and conducting a process evaluation of complex healthy lifestyle programs with adolescents in schools. Additional training of peer educators may be needed to enhance program delivery.

Evaluation of a Sexual Transmitted Infection Prevention Program Among University Students in Beira City Central Mozambique: A Study Protocol

Background: Unhealthy sexual behaviors, such as unprotected sexual intercourse and lack of using screening services increase cyclical transmission of sexually transmitted infections including Human Immunodeficiency Virus (HIV), especially among young adults. Hence health promotion programs can contribute to reduce the consequences, by changing (determinants of) these behaviors. Such interventions need to embrace a comprehensive approach and apply theory-and evidence-based methods. This article describes the protocol for a process and effect evaluation study of a sexually transmitted infection prevention program among university students in Beira city, central Mozambique.

Methods: The on-going program at Universidade Católica de Moçambique is described following the six steps of Intervention Mapping (IM), with a focus on the evaluation plan (i.e., the final step in IM). The details regarding previous steps in the protocol are briefly described as well, as they lay the foundation for the final step. The overall study will apply a hybrid type 1 approach by assessing the effectiveness of the intervention while gettering implementation. The process evaluation will apply qualitative and quantitative methods to gain insight in the context, reach, dose delivered, dose received and recruitment. Interviews with closed and open-ended questions will be conducted with program implementers and users. A quasi-experimental non-equivalent control group design is used to evaluate the effectiveness. A cohort of university students will be followed for 6 months. Self-administrated questionnaires will be used to collect data every 3 months.

Discussion: A combination of process and effect evaluation is proposed. This is a useful and fruitful procedure, since concurrent process evaluation can allow researchers to better interpret findings from the effect evaluation and understand how the intervention might replicate in similar contexts. We decided to follow the IM approach since, it is a theory-and evidence-based, systematic and detailed guide regarding what to do at every steps. A quasi-experimental non-equivalent control group design was chosen to fit the context of the study and generate outcomes with high external validity.

Study Registration: 004/CIBS/2020.

Effectiveness of non-pharmacological interventions for reducing postpartum fatigue: a systematic review protocol

INTRODUCTION

Postpartum fatigue is a common symptom among new mothers after their pregnancy. It has a considerable negative impact on women's functional and mental status as well as the development of babies. Identifying effective interventions to prevent or reduce postpartum fatigue is meaningful to improve the quality of life and avoid adverse outcomes of this vulnerable population. This systematic review aims to synthesise non-pharmacological evidence and evaluate the effectiveness of interventions for reducing postpartum fatigue among puerperas.

METHODS AND ANALYSIS

This review will be conducted in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols. We will systematically search the Cochrane Library, PubMed, Embase, Web of Science, PsycINFO, CINAHL and ProQuest databases to identify clinical trials implementing non-pharmacological interventions conducted during 0-78 weeks postpartum for fatigue reduction. An additional search of OpenGrey will be conducted to identify grey literature. The search will be performed on 30 March 2021 without restrictions on time and language. Two independent reviewers will be responsible for study selection, data extraction and study quality assessment. The Cochrane risk-of-bias tool will be adopted to evaluate the risk biases of the included randomised controlled trials, and the Risk of Bias in Non-randomised Studies of Interventions will be applied to evaluate non-randomised controlled trials. Any disagreements will be referred to a third reviewer to reach a consensus. Findings will be qualitatively synthesised, and a meta-analysis will be conducted for the statistical combination if outcome data are sufficient and available.

ETHICS AND DISSEMINATION

This systematic review will not involve the collection of primary data and will be based on published data. Therefore, ethics approval is not required. The final findings will be disseminated through peer-reviewed journals and academic conferences.

PROSPERO REGISTRATION NUMBER

CRD42021234869.

Theoretical approaches to process evaluations of complex interventions in health care: a systematic scoping review protocol

BACKGROUND

Complex interventions in health care are characterized by multiple interacting components as well as by numerous nonlinear interactions with the social systems within which they are being implemented. The process of developing, evaluating and implementing complex interventions is therefore challenging. Established guidance such as the MRC (Medical Research Council) framework for developing and evaluating complex interventions refers to process evaluations as an integral part of the development of complex evidence-based interventions. Even though the need for process evaluations is recognized, the realization of such approaches is challenging because methodological instruction is sparse, and the phenomenon of interest is complex. A number of theoretical approaches indicating how to conduct process evaluations of complex interventions in health care exist, but a systematic and comprehensive overview of these is missing. Thus, the objective of the systematic scoping review described herein is to provide an overview and analysis of theoretical approaches suitable for the planning and conducting of process evaluations.

METHODS

The design and conduct of this review will follow the procedures of a systematic scoping review. The search strategy will be developed following the BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) template which has been conceptualized for structured reviews of theory. The systematic search of the MEDLINE (via PubMed), CINAHL (via EBSCO) and PsycInfo (via EBSCO) electronic databases will be complemented by "hand searching" techniques. Study selection, data extraction, and data analysis will be performed by tandems of two researchers independently of each other. Divergent decisions and judgements between the two researchers will be discussed by the whole review team.

DISCUSSION

The findings from this scoping review will provide an overview and comparison of theoretical approaches suitable for process evaluations of complex interventions in health care. The review results will support researchers in choosing the theoretical approach that best fits the respective focus of their process evaluation study.

SYSTEMATIC REVIEW REGISTRATION

This study has been registered with PROSPERO (International Prospective Register of Systematic Reviews) under registration number CRD42020211732 .

Implementation of health education interventions at Dutch music schools

A randomized controlled trial was conducted comparing the effects of a biopsychosocial course (PRESTO-Play) vs. physical activity promotion (PRESTO-Fit) to reduce disability related to musculoskeletal disorders in music students. The current study provides an external validation and a formative and process evaluation, allowing for a better interpretation of results. First, a group of experts was asked to complete a structured evaluation of design and content of the trial. Second, quantitative and qualitative data were analysed from different stakeholders (students, therapists and conservatory staff) using questionnaires, logs, field notes and emails to evaluate fidelity, dose delivered, dose received, reach and context. Results are presented descriptively. Two authors independently identified key responses that were merged into themes. Although no difference in disability was found between interventions, closer evaluation revealed that participants in PRESTO-Play reported that they learned about prevention of physical complaints and were more satisfied with course contents compared with PRESTO-Fit. Study design and contents of the interventions were found to be valid, with an appropriate dose delivered. Feedback from students and logs suggested that behavioural change and psychosocial principles in PRESTO-Play might have not been implemented optimally. Only moderate fidelity in both groups and too little contrast between interventions could have influenced results. Low attendance rates and a presumed lack of generalization further decreased possible effectiveness. Context greatly influenced implementation. Implementing a future health course with closer collaboration with the institution could optimize accessibility and communication, encourage attendance and enhance motivation for behavioural change.

Identifying the 'Active Ingredients' of an Effective Psychological Intervention to Reduce Fear of Cancer Recurrence: A Process Evaluation

Purpose: Psychological interventions targeting fear of cancer recurrence (FCR) are effective in reducing fear and distress. Process evaluations are an important, yet scarce adjunct to published intervention trials, despite their utility in guiding the interpretation of study outcomes and optimizing intervention design for broader implementation. Accordingly, this paper reports the findings of a process evaluation conducted alongside a randomized controlled trial of a psychological intervention for melanoma patients. Methods: Men and women with a history of Stage 0-II melanoma at high-risk of developing new primary disease were recruited via High Risk Melanoma Clinics across Sydney, Australia and randomly allocated to receive the psychological intervention (n = 80) or usual care (n = 84). Intervention participants received a tailored psycho-educational resource and three individual psychotherapeutic sessions delivered via telehealth. Qualitative and quantitative data on intervention context, processes, and delivery (reach, dose, and fidelity), and mechanisms of impact (participant responses, moderators of outcome) were collected from a range of sources, including participant surveys, psychotherapeutic session audio-recordings, and clinical records. Results: Almost all participants reported using the psycho-educational resource (97%), received all intended psychotherapy sessions (96%), and reported high satisfaction with both intervention components. Over 80% of participants would recommend the intervention to others, and a small proportion (4%) found discussion of melanoma-related experiences confronting. Perceived benefits included enhanced doctor-patient communication, talking more openly with family members about melanoma, and improved coping. Of potential moderators, only higher FCR severity at baseline (pre-intervention) was associated with greater reductions in FCR severity (primary outcome) at 6-month follow-up (primary endpoint). Conclusions: Findings support the acceptability and feasibility of a psychological intervention to reduce FCR amongst individuals at high risk of developing another melanoma. Implementation into routine melanoma care is an imperative next step, with FCR screening recommended to identify those most likely to derive the greatest psychological benefit.

The 'Eat Well @ IGA' healthy supermarket randomised controlled trial: process evaluation

Background

Successful implementation and long-term maintenance of healthy supermarkets initiatives are crucial to achieving potential population health benefits. Understanding barriers and enablers of implementation of real-world trials will enhance wide-scale implementation. This process evaluation of a healthy supermarket intervention sought to describe (i) customer, retailer and stakeholder perspectives on the intervention; (ii) intervention implementation; and (iii) implementation barriers and enablers.

Methods

Eat Well @ IGA was a 12-month randomised controlled trial conducted in 11 Independent Grocers of Australia (IGA) chain supermarkets in regional Victoria, Australia (5 intervention and 6 wait-listed control stores). Intervention components included trolley and basket signage, local area and in-store promotion, and shelf tags highlighting the healthiest packaged foods. A sequential mixed-methods process evaluation was undertaken. Customer exit surveys investigated demographics, and intervention recall and perceptions. Logistic mixed-models estimated associations between customer responses and demographics, with store as random effect. Supermarket staff surveys investigated staff demographics, interactions with customers, and intervention component feedback. Semi-structured stakeholder interviews with local government, retail and academic partners explored intervention perceptions, and factors which enabled or inhibited implementation, maintenance and scalability. Interviews were inductively coded to identify key themes.

Results

Of 500 customers surveyed, 33%[95%CI:23,44] recalled the Eat Well @ IGA brand and 97%[95%CI:93,99] agreed that IGA should continue its efforts to encourage healthy eating. The 82 staff surveyed demonstrated very favourable intervention perceptions. Themes from 19 interviews included that business models favour sales of unhealthy foods, and that stakeholder collaboration was crucial to intervention design and implementation. Staff surveys and interviews highlighted the need to minimise staff time for project maintenance and to regularly refresh intervention materials to increase and maintain salience among customers.

Conclusions

This process evaluation found that interventions to promote healthy diets in supermarkets can be perceived as beneficial by retailers, customers, and government partners provided that barriers including staff time and intervention salience are addressed. Collaborative partnerships in intervention design and implementation, including retailers, governments, and academics, show potential for encouraging long-term sustainability of interventions.

Trial registration

ISRCTN, ISRCTN37395231 Registered 4 May 2017.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12966-021-01104-z.

Cluster randomised controlled feasibility study of HENRY: a community-based intervention aimed at reducing obesity rates in preschool children

BACKGROUND

Community-based obesity prevention interventions are often commissioned despite the limited evidence base. HENRY (Health, Exercise, Nutrition for the Really Young) is a programme delivered to parents of preschool children across the UK. Early evidence suggests that it may be effective, but a robust evaluation has not been conducted. We initiated a systematic evaluation of HENRY by studying the feasibility of conducting a multi-centre definitive trial to evaluate its effectiveness and cost-effectiveness to prevent obesity. Objectives were to assess the feasibility of recruiting local authorities, centres and parents; test processes and time required to train and certify intervention staff; explore HENRY commissioning processes; identify potential sources (and associated impact) of contamination; and consider the feasibility of trial procedures.

METHODS

We conducted a multi-centre, open labelled, two group, prospective, cluster randomised, controlled, feasibility study, with embedded process evaluation and pre-defined criteria for progression to definitive trial. We sought to recruit 120 parents from 12 children's centres, across two UK local authority (government) areas. Within each local authority, we planned to randomise three centres to HENRY and three to 'standard care' control. Our plan was to collect data in family homes at baseline and 12 months, including parent and child height and weight, and parent-reported questionnaires on self-efficacy, feeding, eating habits, quality of life and resource use. Contamination, implementation and study acceptability were explored using parent interviews.

RESULTS

We recruited two local authorities and 12 children's centres within eight months. One hundred and seventeen parents were recruited (average 3.9 parents per programme) and follow-up data were collected from 85% of participants. Process data from 20 parents and 24 members of staff indicate that both would benefit from more detail about their involvement as participants, but that methods were acceptable. Contamination was likely, though the impact of this on behaviour was unclear.

CONCLUSION

Our findings indicate that a cluster RCT of HENRY to assess its effect on childhood obesity prevention is feasible. This study has allowed us to design a pragmatic definitive trial with minimal bias, taking account of lessons learnt from conducting evaluation research in public health settings.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier NCT03333733 registered 6th November 2017.

Protocol for process evaluation of integration of mental health into primary healthcare in two states in Nigeria: the mhSUN programme

BACKGROUND

Current international recommendations to address the large treatment gap for mental healthcare in low- and middle-income countries are to scale up integration of mental health into primary care. There are good outcome studies to support this, but less robust evidence for effectively carrying out integration and scale-up of such services, or for understanding how to address contextual issues that routinely arise.

AIMS

This protocol is for a process evaluation of a programme called Mental Health Scale Up Nigeria. The study aims are to determine the extent to which the intervention was carried out according to the plans developed (fidelity), to examine the effect of postulated moderating factors and local context, and the perception of the programme by primary care staff and implementers.

METHOD

We use a theoretical framework for process evaluation based on the Medical Research Council's Guidelines on Process Evaluation. A Theory of Change workshop was carried out in programme development, to highlight relevant factors influencing the process, ensure good adaptation of global normative guidelines and gain buy-in from local stakeholders. We will use mixed methods to examine programme implementation and outcomes, and influence of moderating factors.

RESULTS

Data sources will include the routine health information system, facility records (for staff, medication and infrastructure), log books of intervention activities, supervision records, patient questionnaires and qualitative interviews.

CONCLUSIONS

Evidence from this process evaluation will help guide implementers aiming to scale up mental health services in primary care in low- and middle-income countries.

Intersectoral care management for older people with cognitive impairment during and after hospital stays [intersec-CM]: study protocol for a process evaluation within a randomised controlled trial

Background

In the healthcare system in Germany, different institutions and actors play specific roles in the discharge and transition of patients from hospitals into primary care (Sachverständigenrat zur Begutachtung der Entwicklung im Gesundheitswesen, Wettbewerb an der Schnittstelle zwischen ambulanter und stationärer Gesundheitsversorgung, 2012). However, there are shortcomings in these intersectoral transitions. Especially in older people with cognitive impairment (PCI), discharge management often lacks coordination and cooperation between healthcare providers. This frequently results in higher rates of unscheduled readmission. The project intersec-CM is a randomised controlled trial (RCT) that aims to explore up to what extent an intersectoral care management (ICM) can improve this transition. This ICM is delivered by nurses with special training in care management. The objective of this paper is to describe a mixed-methods process evaluation of the intersectoral care management intervention and the factors that facilitate and inhibit its implementation.

Methods

Different study designs for process evaluations from previous literature were collected and analysed according to the dimension implementation fidelity, satisfaction with the intervention, feasible transfer into routine care, optimum point of time, frequency and execution of the intervention, and context factors.

Results

The actor-network theory was chosen as the theoretic framework for the process evaluation. Based on this theory, a mixed-methods design was developed to combine and integrate qualitative and quantitative evaluation methods. The qualitative part includes semi-structured interviews using topic guides (phase 1) and later in-depth interviews with narrative portions (phase 3), which will be analysed by using the qualitative content analysis according to Kuckartz. The quantitative survey (phase 2) is conducted with standardised questionnaires.

Discussion

Challenges in data collection include the development of interview guidelines, which require different terminologies depending on every specific actor targeted in the intervention. Conducting the interviews, there is a risk of misunderstanding the older PCI by the interviewer and vice versa. However, the combination of qualitative and quantitative approaches as different techniques of process evaluation may help to capture, integrate and analyse data on different dimensions of the intervention.

Conclusions

The results of our process evaluation may serve as an implementation guideline for intersectoral care management in the German healthcare system. Furthermore, the approach to evaluate the process of a complex intervention in health care for older PCI may serve as a stimulus to broaden the evidence base also of other complex intervention studies to improve health care for this vulnerable group.

The study was ethically approved by the Ethics Committee of the Ernst-Moritz-Arndt University of Greifswald. The study has been registered at the U.S. National Library of Medicine.

Trial registration

ClinicalTrials.gov NCT03359408. Registered on 2 December 2017. The approximate date when recruitment to the process evaluation of the study will be completed is 31 May 2021.

Process evaluation of fruit and vegetables distribution interventions in school-based settings: A systematic review

•

Assesses how school fruit & vegetable distribution interventions are implemented.

•

Child fruit & vegetable intake is influenced by how intervention is implemented.

•

Recommendations for implementing fruit & vegetable interventions in schools.

•

Studies should also consider sustainability, cost-effectiveness and implementation.

Despite the importance of process evaluation in program evaluations, research has focused primarily on the effectiveness of fruit and vegetables (FVs) distribution interventions on children’s consumption, with little attention given to how these interventions achieve their outcomes. Five bibliographic databases (Embase, PubMed, ProQuest, Scopus, and Web of Science Core Collection) were searched in June 2019 for studies of interventions where the main focus was the implementation of distributed FVs to school-aged children as a snack. The Critical Appraisal Skills Programme (CASP) tool was used to appraise the risk of bias within included studies. Data were extracted based on study characteristics and findings. Results identified 24 studies reporting on 11 interventions and 1 policy. The findings of this systematic review indicate that the majority of the studies included limited references to implementation research. Recurring limitations include an absence of an evaluation theoretical framework and the data collection methods used. Also, several factors were identified as informing the success of snack-based FVs distribution programs, including participation of the school community, school characteristics, background knowledge, and parental engagement. Lack of timely FVs delivery, limited funding, inadequate awareness about the program, insufficient teachers’ time, and food waste were identified as challenges to successful programming. Findings indicate that distributing FVs to school-aged children as a snack can increase their consumption, but only with proper implementation. Further evaluative research is required to better inform future implementation of snack-based FV distribution interventions in school settings.

Fruit and Vegetable Lesson Plan Pilot Intervention for Grade 5 Students from Southwestern Ontario

The purpose was to create and assess the impact of food literacy curriculum alongside a centrally procured school snack program among grade five students in Southwestern Ontario, Canada. Grade five students (N = 287) from five intervention and three controls schools participated in an 8-week food delivery program. In addition to the food delivery program, intervention schools received a resource kit and access to 42 multidisciplinary food literacy lesson plans using the produce delivered as part of the food delivery program. Participants completed matched pre- and post-test online surveys to assess fruit and vegetable intake, knowledge, preferences, and attitudes. Descriptive analyses and changes in scores between the intervention and control schools were assessed using one-way ANOVAs, paired samples t-tests, and McNemar’s tests. In total, there were 220 participants that completed both the pre- and post-test surveys. There was a significant improvement in fruit and vegetable intake (p = 0.038), yet no differences in knowledge of the recommended number of food group servings, knowledge of food groups, or fruit and vegetable preferences or attitudes were observed. Integrating nutrition lesson plans within core curricula classes (e.g., math, science, and literacy) can lead to modest increases in fruit and vegetable intake.

Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?-a systematic review

BACKGROUND

Process evaluations are increasingly conducted within pragmatic randomised controlled trials (RCTs) of health services interventions and provide vital information to enhance understanding of RCT findings. However, issues pertaining to process evaluation in this specific context have been little discussed. We aimed to describe the frequency, characteristics, labelling, value, practical conduct issues, and accessibility of published process evaluations within pragmatic RCTs in health services research.

METHODS

We used a 2-phase systematic search process to (1) identify an index sample of journal articles reporting primary outcome results of pragmatic RCTs published in 2015 and then (2) identify all associated publications. We used an operational definition of process evaluation based on the Medical Research Council's process evaluation framework to identify both process evaluations reported separately and process data reported in the trial results papers. We extracted and analysed quantitative and qualitative data to answer review objectives.

RESULTS

From an index sample of 31 pragmatic RCTs, we identified 17 separate process evaluation studies. These had varied characteristics and only three were labelled 'process evaluation'. Each of the 31 trial results papers also reported process data, with a median of five different process evaluation components per trial. Reported barriers and facilitators related to real-world collection of process data, recruitment of participants to process evaluations, and health services research regulations. We synthesised a wide range of reported benefits of process evaluations to interventions, trials, and wider knowledge. Visibility was often poor, with 13/17 process evaluations not mentioned in the trial results paper and 12/16 process evaluation journal articles not appearing in the trial registry.

CONCLUSIONS

In our sample of reviewed pragmatic RCTs, the meaning of the label 'process evaluation' appears uncertain, and the scope and significance of the term warrant further research and clarification. Although there were many ways in which the process evaluations added value, they often had poor visibility. Our findings suggest approaches that could enhance the planning and utility of process evaluations in the context of pragmatic RCTs.

TRIAL REGISTRATION

Not applicable for PROSPERO registration.

"Putting Meat on the Bones": Understanding the Implementation of a Community-Based Early Intervention and Prevention Programme-Contextual, Person, and Programme Influences

The adoption and effective delivery of evidence-based interventions within "real-world" community-based, primary health care service settings are of crucial importance. In this paper, we explore the successes and challenges of implementing a new complex, group-based, early parenting intervention called the Parent and Infant (PIN) programme. This study involved a systematic analysis of the processes and factors that influence the implementation of the PIN programme; the analysis was guided and informed by the Implementation Outcome Framework and the Consolidated Framework for Implementation Research. A documentary review, alongside a series of one-to-one interviews and small group discussions with a range of stakeholders (n = 44), and 7 focus groups (n = 24) were used as data sources. Factors that promoted programme adoption, acceptability, and implementation feasibility included programme characteristics and stakeholder attitudes, as well as organisational and systems factors (e.g. leadership and collaboration). Key challenges to implementation success included engagement and adoption barriers. This research provides a useful and important example of real-world, theory-driven implementation research which helped to identify interrelated processes, factors, and contexts which shape and influence the implementation of early intervention and prevention programmes, removed for blind review.

A systematic review of the use and reporting of evaluation frameworks within evaluations of physical activity interventions

BACKGROUND

Evaluation of physical activity interventions is vital to inform, and justify, evidence-based policy and practice to support population-wide changes in physical activity. Several evaluation frameworks and guidance documents have been developed to facilitate the evaluation and reporting of evaluation studies in public health. However, there is a lack of evidence about whether frameworks are being used to guide evaluation. There continues to be claims of poor and inconsistent reporting in evaluation studies. The aim of this review was to assess the use of evaluation frameworks and the quality of reporting of how they were applied within evaluation studies of physical activity interventions.

OBJECTIVES

1. To identify whether evaluation frameworks are reported to have been used within evaluation studies of physical activity interventions, and which frameworks have been used. 2. To appraise the quality of reporting with regards to how evaluation frameworks have been used.

METHOD

We developed a checklist of indicators to enable a critical appraisal of the use and reporting of different evaluation frameworks in evaluation studies. We conducted a systematic search and review of evaluation studies published between 2015 and the date of the search to appraise the use and reporting of evaluation frameworks. A narrative synthesis is provided.

RESULTS

The review identified 292 evaluation studies of physical activity interventions, only 69 (23%) of these mentioned using an evaluation framework, and only 16 different frameworks were referred to. There was variation in the quality of reporting of framework use. 51 (74%) studies were identified as being explicitly based on the stated framework, however only 26 (38%) provided detailed descriptions consistently across all the checklist indicators. Details of adaptations and limitations in how frameworks were applied were less frequently reported. The review also highlighted variability in the reporting of intervention components. More consistent and precise reporting of framework and intervention components is needed.

CONCLUSION

Evaluation frameworks can facilitate a more systematic evaluation report and we argue their limited use suggests missed opportunities to apply frameworks to guide evaluation and reporting in evaluation studies. Variability in the quality of reporting of framework use limits the comparability and transferability of evidence. Where a framework has been used, the checklist of indicators can be employed to facilitate the reporting of an evaluation study and to review the quality of an evaluation report.

Process Evaluation of a National Elderly Nutrition-Care Program in Iran: Perspectives of Clients and Providers

OBJECTIVE

As aging populations increase, many countries have begun implementation of policies to improve elder health and nutrition. This study evaluated key process components of nutrition sections of a national elderly integrated-care program in health-care centers in Iran.

METHODS

With stratified three-stage random cluster sampling, a total of 256 elderly subjects (clients) and 76 staff members of health-care centers (providers) participated in the study. Quantitative and qualitative data were collected using two self-administrated questionnaires to evaluate various components of process evaluation for clients and providers. Program reach was measured by the ratio of the number in the target group who underwent the program to the number of eligible individuals. Exposure rate was measured as clients' awareness of program services. Delivery and fidelity were assessed from providers' reports for each service. Satisfaction rates were assessed for the whole program and for each service of the program. All reasons behind strengths and weaknesses in each of the process-evaluation components were examined and are reported in detail.

RESULTS

The clients reported low-reach (20.0%) and moderate-exposure rates of 77.5%, with a program target of 90%. Primary training sessions were delivered to the providers as intended (100%), but most planned services for clients, especially vitamin and mineral supplementation, follow-up, and physical activity sessions, were poorly implemented, as they were delivered correctly to 24.7% (n=63, vitamin and mineral supplementation), 24% (n=62, follow-up), and 40.3% (n=103, physical activity sessions) of the clients, with a set program goal of 60%. An overall low level of implementation fidelity was observed, and 39.3% (n=30) of the providers believed that most clients did not benefit from the nutritional services. Overall, less than half (42.8%, n=101) the clients were highly satisfied with the program.

CONCLUSION

The process evaluation showed insufficient reach, exposure, and fidelity of the program, as well as imprecise delivery of some services, which resulted in low levels of client satisfaction. The findings may have implications for further reinforcement of the program, and indicate the importance of continuous monitoring and evaluation of such programs.

A scoping review of evaluation frameworks and their applicability to real-world physical activity and dietary change programme evaluation

BACKGROUND

Physical activity and dietary change programmes play a central role in addressing public health priorities. Programme evaluation contributes to the evidence-base about these programmes; and helps justify and inform policy, programme and funding decisions. A range of evaluation frameworks have been published, but there is uncertainty about their usability and applicability to different programmes and evaluation objectives, and the extent to which they are appropriate for practitioner-led or researcher-led evaluation. This review appraises the frameworks that may be applicable to evaluation of physical activity and/or dietary change programmes, and develops a typology of the frameworks to help guide decision making by practitioners, commissioners and evaluators.

METHODS

A scoping review approach was used. This included a systematic search and consultation with evaluation experts to identify evaluation frameworks and to develop a set of evaluation components to appraise them. Data related to each framework's general characteristics and components were extracted. This was used to construct a typology of the frameworks based on their intended programme type, evaluation objective and format. Each framework was then mapped against the evaluation components to generate an overview of the guidance included within each framework.

RESULTS

The review identified 71 frameworks. These were described variously in terms of purpose, content, or applicability to different programme contexts. The mapping of frameworks highlighted areas of overlap and strengths and limitations in the available guidance. Gaps within the frameworks which may warrant further development included guidance on participatory approaches, non-health and unanticipated outcomes, wider contextual and implementation factors, and sustainability.

CONCLUSIONS

Our typology and mapping signpost to frameworks where guidance on specific components can be found, where there is overlap, and where there are gaps in the guidance. Practitioners and evaluators can use these to identify, agree upon and apply appropriate frameworks. Researchers can use them to identify evaluation components where there is already guidance available and where further development may be useful. This should help focus research efforts where it is most needed and promote the uptake and use of evaluation frameworks in practice to improve the quality of evaluation and reporting.

Formative Process Evaluation of the "Connect" Physical Activity Feasibility Trial for Adolescents

Most interventions do not reach full implementation in real-world settings. Due to this issue, formative process evaluation during pilot programs can be especially useful to understand implementation strengths and areas for improvement so that full implementation can be reached in future iterations. This study demonstrated how a formative process evaluation of the Connect through Positive Leisure Activities for Youth (Connect) pilot informed course corrections for year 2 implementation. Connect is an intervention to promote a positive social motivational climate for physical activity (PA) in pre-existing after school programs. Connect ran 3 days a week for 8 weeks and had 2 components: a 30-minute “Get-to-Know-You” (GTKY) session and a 60-minute PA session. Formative evaluation was assessed using an observational tool and staff surveys. Changes in youth PA during program hours was assessed as a process outcome using the System for Observing Children’s Activity and Relationships during Play (SOCARP). All Connect essential elements were assessed with the observational tool including (a) social goal-oriented support; (b) collaborative, cooperative play; (c) equal treatment/access; and (d) an inclusive and engaging climate. Adequate dose was achieved on all items in all sessions. Although GTKY and PA sessions both reached high fidelity in promoting equal treatment and access, success in reaching fidelity varied for the 3 remaining essential elements. Post-intervention staff surveys indicated acceptability/adoptability of the Connect program and SOCARP observations indicated significant increases in PA from baseline to post-intervention. Changes for year 2 implementation based on the findings are discussed.

Process evaluation of complex cardiovascular interventions: How to interpret the results of my trial?

Complex interventions of varying degrees of complexity are commonly used and evaluated in cardiovascular nursing and allied professions. Such interventions are increasingly tested using randomized trial designs. However, process evaluations are seldom used to better understand the results of these trials. Process evaluation aims to understand how complex interventions create change by evaluating implementation, mechanisms of impact, and the surrounding context when delivering an intervention. As such, this method can illuminate important mechanisms and clarify variation in results. In this article, process evaluation is described according to the Medical Research Council guidance and its use exemplified through a randomized controlled trial evaluating the effectiveness of a transition program for adolescents with chronic conditions.

Development of the DAGIS intervention study: a preschool-based family-involving study promoting preschoolers' energy balance-related behaviours and self-regulation skills

Background

Preschoolers’ energy balance-related behaviours (EBRBs) and self-regulation skills are important for their later health. Few preschool-based interventions aiming to promote preschoolers’ EBRBs and self-regulation skills, simultaneously reducing differences in EBRBs, due to children’s socio-economic status (SES) background, have been conducted. This study will present the Increased Health and Wellbeing in Preschools (DAGIS) intervention development process applying the Intervention Mapping (IM) framework.

Methods

The development of the DAGIS intervention study, a preschool level clustered randomized controlled trial (RCT), was based on the IM framework. The protocol in IM guides the development process of an intervention through six steps: needs assessment and logic model of the problem, programme outcomes and objectives, design of the programme, production, implementation plan, and evaluation plan.

Results

The needs assessment, part of the step 1 in IM, yielded the base for the DAGIS logic model of change. The model includes objectives related to changes in children’s EBRBs, self-regulation skills, and in psychosocial and physical environment that is determined by parents and early educators. A 22-week programme was developed, and materials for preschools and families were produced. A feasibility study of the recruitment processes, acceptability of the materials and methods, and implementation was conducted. The DAGIS intervention study was conducted September 2017–May 2018 as a clustered RCT including a comprehensive effectiveness and process evaluation. The process evaluation was run throughout the intervention targeting preschools and families.

Conclusion

A preschool-based family-involving programme was developed in the DAGIS intervention study by applying the IM protocol. It was a time- and resource-consuming process. However, the systematic planning, development, and running of the programme have reinforced a comprehensive evaluation, which is a strength in the intervention. The results from the evaluation will enhance the knowledge of how to promote EBRBs and self-regulation skills among preschoolers, and diminish SES differences in them.

Trial registration

ISRCTN57165350 (Prospectively registered January the 8th, 2015).

"Consent is sexy": A poster campaign using sex-positive images and messages to increase dyadic sexual communication

Objective: "Consent is Sexy" (CIS) is a poster campaign incorporating sex-positive messages to promote consent and increase sexual communication among college students. We assess reactions to the campaign and associations between campaign recall and communication attitudes and behaviors. Participants: Male and female undergraduates at a Midwestern university were recruited (N = 284). Methods: A cross-sectional survey was conducted. t-Tests, logistic and multiple linear regressions were used to analyze the data. Results: Over half (56%) of participants recalled the campaign and reactions were positive. Students who recalled CIS had more positive attitudes towards sexual communication (p = .04) and greater perceived behavioral control (PBC; p < .01). Conclusions: Results show many students paid attention and reacted positively to CIS posters and results offer dissemination insights. Consent campaigns should continue to cultivate positive attitudes and PBC in regards to sexual communication.

Investigating business outcomes of healthy food retail strategies: A systematic scoping review

Large changes to food retail settings are required to improve population diet. However, limited research has comprehensively considered the business implications of healthy food retail strategies for food retailers. We performed a systematic scoping review to identify types of business outcomes that have been reported in healthy food retail strategy evaluations. Peer-reviewed and grey literature were searched. We identified qualitative or quantitative real-world food or beverage retail strategies designed to improve the healthiness of the consumer nutrition environment (eg, changes to the "marketing mix" of product, price, promotion, and/or placement). Eligible studies reported store- or chain-level outcomes for measures of commercial viability, retailer perspectives, customer perspectives, and/or community outcomes. 11 682 titles and abstracts were screened with 107 studies included for review from 15 countries. Overall item sales, revenue, store patronage, and customer level of satisfaction with strategy were the most frequently examined outcomes. There was a large heterogeneity in outcome measures reported and in favourability for retailers of outcomes across studies. We recommend more consistent reporting of business outcomes and increased development and use of validated and reliable measurement tools. This may help generate more robust research evidence to aid retailers and policymakers to select feasible and sustainable healthy food retail strategies to benefit population health within and across countries.

A protocol for a systematic review of process evaluations of interventions investigating sedentary behaviour in adults

INTRODUCTION

Sedentary behaviour is defined as any waking behaviour characterised by low energy expenditure ≤1.5 metabolic equivalents while in a sitting, lying or reclining posture. The expanding evidence base suggests that sedentary behaviour may have a detrimental effect on health, well-being and is associated with an increased risk of all-cause mortality. We aim to review process evaluations of randomised controlled trials (RCTs) which included a measure of sedentary behaviour in adults in order to develop an understanding of intervention content, mechanisms of impact, implementation and delivery approaches and contexts, in which interventions were reported to be effective or effective. A secondary aim is to summarise participants, family and staff experiences of such interventions.

METHODS AND ANALYSIS

Ten electronic databases and reference lists from previous similar reviews will be searched. Eligible studies will be process evaluations of RCTs that measure sedentary behaviour as a primary or secondary outcome in adults. As this review will contribute to a programme to develop a community-based intervention to reduce sedentary behaviour in stroke survivors, interventions delivered in schools, colleges, universities or workplaces will be excluded. Two reviewers will perform study selection, data extraction and quality assessment. Disagreements between reviewers will be resolved by a third reviewer. Process evaluation data to be extracted include the aims and methods used in the process evaluation; implementation data; mechanisms of impact; contextual factors; participant, family and staff experiences of the interventions. A narrative approach will be used to synthesise and report qualitative and quantitative data. Reporting of the review will be informed by Preferred Reporting Items for Systematic Review and Meta-Analysis guidance.

ETHICS AND DISSEMINATION

Ethical approval is not required as it is a protocol for a systematic review. Findings will be disseminated through peer-reviewed publications and conference presentations.

PROSPERO REGISTRATION NUMBER

CRD42018087403.

Variation in the operationalisation of dose in implementation of health promotion interventions: insights and recommendations from a scoping review

BACKGROUND

While 'dose' is broadly understood as the 'amount' of an intervention, there is considerable variation in how this concept is defined. How we conceptualise, and subsequently measure, the dose of interventions has important implications for understanding how interventions produce their effects and are subsequently resourced and scaled up. This paper aims to explore the degree to which dose is currently understood as a distinct and well-defined implementation concept outside of clinical settings.

METHODS

We searched four databases (MEDLINE, PsycINFO, EBM Reviews and Global Health) to identify original research articles published between 2000 and 2015 on health promotion interventions that contained the word 'dose' or 'dosage' in the title, abstract or keywords. We identified 130 articles meeting inclusion criteria and extracted data on how dose/dosage was defined and operationalised, which we then synthesised to reveal key themes in the use of this concept across health promotion interventions.

RESULTS

Dose was defined in a variety of ways, including in relation to the amount of intervention delivered and/or received, the level of participation in the intervention and, in some instances, the quality of intervention delivery. We also observed some conflation of concepts that are traditionally kept separate (such as fidelity) either as slippage or as part of composite measures (such as 'intervention dose').

DISCUSSION

Dose is not a well-defined or consistently applied concept in evaluations of health promotion interventions. While current approaches to conceptualisation and measurement of dose are suitable for interventions in organisational settings, they are less well suited to policies delivered at a population level. Dose often accompanies a traditional monotonic linear view of causality (e.g. dose response) which may or may not fully represent the intervention's theory of how change is brought about. Finally, we found dose and dosage to be used interchangeably. We recommend a distinction between these terms, with 'dosage' having the advantage of capturing change to amount 'dispensed' over time (in response to effects achieved). Dosage therefore acknowledges the inevitable dynamic and complexity of implementation.

The CHESS trial: protocol for the process evaluation of a randomised trial of an education and self-management intervention for people with chronic headache

Background

Process evaluation is increasingly common alongside complex randomised controlled trials (RCTs). This evaluation helps in understanding the mechanisms of impact and how the study processes were executed, and it includes any contextual factors which may have implications for the trial results and any future implementation. This process evaluation is for the Chronic Headache Education and Self-management Study (CHESS) RCT, which is evaluating an education and self-management group behavioural intervention for people with chronic headache. Chronic headache is defined as headaches which are present for 15 or more days per month. The most common types are chronic migraine and chronic tension type and medication overuse headaches.

Methods

We will use a mixed methods approach. Quantitative data will be taken from routine trial data which will help us to assess the reach of the study; i.e. did we reach those whom we expected and from where? Intervention attendance (dose received) and attrition and qualitative data will augment our understanding about reasons why people may not wish to take part in or failed to attend sessions. Interviews with intervention facilitators and trial participants will gain different perspectives on taking part in the trial.

Fidelity will be assessed through listening to audio recordings for adherence to course content and competence of the facilitation of a sample of sessions.

Discussion

Our process evaluation will allow us to gain insight into how the trial was delivered, the obstacles and enablers encountered and the possible reasons why the interventions may or may not be effective.

Trial registration

ISRCTN79708100. Registered on 16 December 2015.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3372-x) contains supplementary material, which is available to authorized users.

The West Midlands ActiVe lifestyle and healthy Eating in School children (WAVES) study: a cluster randomised controlled trial testing the clinical effectiveness and cost-effectiveness of a multifaceted obesity prevention intervention programme targeted at children aged 6-7 years

BACKGROUND

Systematic reviews suggest that school-based interventions can be effective in preventing childhood obesity, but better-designed trials are needed that consider costs, process, equity, potential harms and longer-term outcomes.

OBJECTIVE

To assess the clinical effectiveness and cost-effectiveness of the WAVES (West Midlands ActiVe lifestyle and healthy Eating in School children) study intervention, compared with usual practice, in preventing obesity among primary school children.

DESIGN

A cluster randomised controlled trial, split across two groups, which were randomised using a blocked balancing algorithm. Schools/participants could not be blinded to trial arm. Measurement staff were blind to allocation arm as far as possible.

SETTING

Primary schools, West Midlands, UK.

PARTICIPANTS

Schools within a 35-mile radius of the study centre and all year 1 pupils (aged 5-6 years) were eligible. Schools with a higher proportion of pupils from minority ethnic populations were oversampled to enable subgroup analyses.

INTERVENTIONS

The 12-month intervention encouraged healthy eating/physical activity (PA) by (1) helping teachers to provide 30 minutes of additional daily PA, (2) promoting 'Villa Vitality' (interactive healthy lifestyles learning, in an inspirational setting), (3) running school-based healthy cooking skills/education workshops for parents and children and (4) highlighting information to families with regard to local PA opportunities.

MAIN OUTCOME MEASURES

The primary outcomes were the difference in body mass index z-scores (BMI-zs) between arms (adjusted for baseline body mass index) at 3 and 18 months post intervention (clinical outcome), and cost per quality-adjusted life-year (QALY) (cost-effectiveness outcome). The secondary outcomes were further anthropometric, dietary, PA and psychological measurements, and the difference in BMI-z between arms at 27 months post intervention in a subset of schools.

RESULTS

Two groups of schools were randomised: 27 in 2011 (n = 650 pupils) [group 1 (G1)] and another 27 in 2012 (n = 817 pupils) [group 2 (G2)]. Primary outcome data were available at first follow-up (n = 1249 pupils) and second follow-up (n = 1145 pupils) from 53 schools. The mean difference (MD) in BMI-z between the control and intervention arms was -0.075 [95% confidence interval (CI) -0.183 to 0.033] and -0.027 (95% CI -0.137 to 0.083) at 3 and 18 months post intervention, respectively. The main analyses showed no evidence of between-arm differences for any secondary outcomes. Third follow-up included data on 467 pupils from 27 G1 schools, and showed a statistically significant difference in BMI-z (MD -0.20, 95% CI -0.40 to -0.01). The mean cost of the intervention was £266.35 per consented child (£155.53 per child receiving the intervention). The incremental cost-effectiveness ratio associated with the base case was £46,083 per QALY (best case £26,804 per QALY), suggesting that the intervention was not cost-effective.

LIMITATIONS

The presence of baseline primary outcome imbalance between the arms, and interschool variation in fidelity of intervention delivery.

CONCLUSIONS

The primary analyses show no evidence of clinical effectiveness or cost-effectiveness of the WAVES study intervention. A post hoc analysis, driven by findings at third follow-up, suggests a possible intervention effect, which could have been attenuated by baseline imbalances. There was no evidence of an intervention effect on measures of diet or PA and no evidence of harm.

FUTURE WORK

A realist evidence synthesis could provide insights into contextual factors and strategies for future interventions. School-based interventions need to be integrated within a wider societal framework and supported by upstream interventions.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN97000586.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 8. See the NIHR Journals Library website for further project information.

The Incidence of Obesity, Assessed as Adiposity, Is Reduced After 1 Year in Primary Schoolchildren by the POIBA Intervention

BACKGROUND

Childhood obesity is becoming a serious problem, and prevention programs are needed.

OBJECTIVE

The purpose of this study was to evaluate, after 1 y, the effectiveness of a multicomponent, school-based obesity intervention program.

METHODS

This intervention, conducted in Barcelona, Spain, was a quasi-experimental obesity primary prevention intervention targeting schoolchildren aged 9-10 y. Participants were assigned to an intervention group (IG) (1464 students) or to a comparison group (CG) (1609 students). The intervention consisted of a 9-session classroom program, 6 weekly sessions of physical education and out-of-school physical activity, and a workshop for families. It lasted from October 2011 to May 2012. Data obtained at baseline (spring 2011) and follow-up (spring 2012) included information on nutrition and physical activity, through 2 self-reported questionnaires, and measurement of weight, height, triceps skinfold thickness, and waist circumference. The cumulative incidence rate (CIR) of obesity was calculated from triceps skinfold measures. A multilevel logistic regression model was fitted to determine the association between the intervention and the CIR of obesity. The effect size of the program was estimated with Cohen's criteria.

RESULTS

The overall prevalence of obesity at baseline was 12.7%. At the 12-mo follow-up, the incidence of obesity was 7.8% in the IG compared with 11.4% in the CG (P < 0.005), representing 31% fewer new cases of obesity in the IG. The Cohen's d effect size of the program was 0.33. In the multilevel analysis, there was a protective effect of the intervention on the CIR of obesity at 12 mo (OR: 0.7; 95% CI: 0.5, 0.9) (P = 0.009).

CONCLUSIONS

The first Prevención de la Obesidad Infantil en Barcelona (Childhood Obesity Prevention in Barcelona) (POIBA) intervention, targeting children aged 9-10 y, reduced the incidence of obesity as measured by adiposity. The intervention could prevent 1 in 3 new cases of childhood obesity in this age range.

A protocol for designing online training to support the implementation of community-based interventions

Training program implementers through online methods represents a way of facilitating the widespread implementation of community-based interventions that is more financially and logistically feasible than traditional in-person training methods. However, there are few evidence-informed protocols or models that can guide the development of online training content in a way that is consistent with instructional best practices. This paper presents an evidence-informed protocol for developing a training website, or online training platform, to support the implementation of community-based interventions at scale, which was informed by a critical analysis of the instructional design literature and our experiences developing an online training platform for the HealtheStepsTM Lifestyle Prescription Program. The protocol is an operationalization of the ADDIE model of instructional design, and details the analysis, design, development, implementation, and evaluation stages of the process. Examples from the HealtheStepsTM program are used to illustrate the use of the protocol in practice. The protocol emphasizes the need for rigorous analysis of the target audience and a multidisciplinary literature base drawing from instructional design and implementation science. It can be used by researchers to guide the development of online training platforms to support the widespread implementation of evidence-based health interventions, thus increasing their public health impact.

Promoting Health from the Pulpit: A Process Evaluation of HIV Sermons to Reduce HIV Stigma and Promote Testing in African American and Latino Churches

Embedding health messages into sermons is a potentially valuable strategy to address HIV and other health disparities in churches that predominantly serve racial and ethnic minorities. This study explores implementation of an HIV sermon as part of a multi-component intervention in three churches (Latino Catholic, Latino Pentecostal, and African American Baptist) in high HIV prevalence areas of Los Angeles County, California. Clergy were given an HIV sermon guide that included local public health data, stigma reduction cues, HIV testing messages, and a sample sermon. Findings are based on a process evaluation (i.e., reach, dose delivered, fidelity, and implementation) and in-depth content analysis to explore HIV frames and messages used by clergy. Sermons were audio-recorded, transcribed verbatim, and coded using an inductive approach. Complementary data were collected through systematic observation. Overall, five clergy delivered nine HIV sermons to majority African American or Latino audiences. On average, 174 congregants were reached per sermon. We found large variation in fidelity to communicating key HIV messages from the sermon guide. While promoting HIV testing from the pulpit seemed viable and acceptable to all the participating clergy, fewer embedded explicit stigma reduction cues. Most spoke about HIV using compassionate and non-judgmental terms, however, issue framing varied across clergy. Structured training of clergy may be necessary to implement the more theoretically driven stigma reduction cues included in the sermon guide. More research is needed on the viability and acceptability of embedding specific health promotion messages into sermons.

Cluster randomised controlled feasibility study of HENRY: a community-based intervention aimed at reducing obesity rates in preschool children

BACKGROUND

In the UK and beyond, public funding is used to commission interventions delivered in public health early years settings aimed at improving health and well-being and reducing inequalities in order to promote school readiness. This is a key setting for obesity prevention programmes, which are often commissioned despite the limited evidence base. The HENRY (Health, Exercise, Nutrition for the Really Young) programme is an 8-week programme delivered to parents of preschool children, designed to support families to optimise healthy weight behaviours. Early evidence suggests that it may be effective, but a robust evaluation using a randomised controlled design has not been conducted. This study begins this process by evaluating the feasibility of conducting a multi-centre definitive trial to evaluate the effectiveness and cost-effectiveness of HENRY to prevent obesity in the early years.

METHODS

This is a multi-centre, open labelled, two group, prospective, cluster randomised, controlled, feasibility study aiming to recruit 120 parents from 12 children's centres, based in two local authority areas. Within each of the two local authorities, three centres will be randomised to HENRY and three will be randomised to a control arm of standard care (usual provision of services within children's centres). We will explore HENRY commissioning, provision and delivery and assess the feasibility of local authority, centre and parent recruitment, the processes and time required to train and certify staff to deliver the intervention, the potential sources (and associated risk) of contamination and the feasibility of the trial procedures. Research includes a process evaluation, feasibility of cost-effectiveness evaluation, with progression to the definitive trial judged against pre-defined criteria.

DISCUSSION

This feasibility study will support the decision to proceed to, and the design of, a future definitive trial, providing an evidence base of an approach to prevent childhood obesity, which has been deemed attractive to all stakeholders, including parents. Given the widespread adoption of the intervention, this has the potential to impact on public health in the UK and beyond.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier NCT03333733 registered 6th November 2017Protocol date: 25th October 2017Protocol version: 4.0.

Research Project Evaluation-Learnings from the PATHWAYS Project Experience

Background: Every research project faces challenges regarding how to achieve its goals in a timely and effective manner. The purpose of this paper is to present a project evaluation methodology gathered during the implementation of the Participation to Healthy Workplaces and Inclusive Strategies in the Work Sector (the EU PATHWAYS Project). The PATHWAYS project involved multiple countries and multi-cultural aspects of re/integrating chronically ill patients into labor markets in different countries. This paper describes key project’s evaluation issues including: (1) purposes, (2) advisability, (3) tools, (4) implementation, and (5) possible benefits and presents the advantages of a continuous monitoring. Methods: Project evaluation tool to assess structure and resources, process, management and communication, achievements, and outcomes. The project used a mixed evaluation approach and included Strengths (S), Weaknesses (W), Opportunities (O), and Threats (SWOT) analysis. Results: A methodology for longitudinal EU projects’ evaluation is described. The evaluation process allowed to highlight strengths and weaknesses and highlighted good coordination and communication between project partners as well as some key issues such as: the need for a shared glossary covering areas investigated by the project, problematic issues related to the involvement of stakeholders from outside the project, and issues with timing. Numerical SWOT analysis showed improvement in project performance over time. The proportion of participating project partners in the evaluation varied from 100% to 83.3%. Conclusions: There is a need for the implementation of a structured evaluation process in multidisciplinary projects involving different stakeholders in diverse socio-environmental and political conditions. Based on the PATHWAYS experience, a clear monitoring methodology is suggested as essential in every multidisciplinary research projects.

Factors influencing workplace health promotion intervention: a qualitative systematic review

Although workplace health promotion (WHP) has evolved over the last 40 years, systematically collected knowledge on factors influencing the functioning of WHP is scarce. Therefore, a qualitative systematic literature review was carried out to systematically identify and synthesize factors influencing the phases of WHP interventions: needs assessment, planning, implementation and evaluation. Research evidence was identified by searching electronic databases (Scopus, PubMed, Social Sciences Citation Index, ASSIA, ERIC, IBBS and PsycINFO) from 1998 to 2013, as well as by cross-checking reference lists of included peer-reviewed articles. The inclusion criteria were: original empirical research, description of WHP, description of barriers to and/or facilitators of the planning, implementation and/or evaluation of WHP. Finally, 54 full texts were included. From these, influencing factors were extracted and summarized using thematic analysis. The majority of influencing factors referred to the implementation phase, few dealt with planning and/or evaluation and none with needs assessment. The influencing factors were condensed into topics with respect to factors at contextual level (e.g. economic crisis); factors at organizational level (e.g. management support); factors at intervention level (e.g. quality of intervention concept); factors at implementer level (e.g. resources); factors at participant level (e.g. commitment to intervention) and factors referring to methodological and data aspects (e.g. data-collection issues). Factors regarding contextual issues and organizational aspects were identified across three phases. Therefore, future research and practice should consider not only the influencing factors at different levels, but also at different phases of WHP interventions.

Does the Kids Café Program's Nutrition Education Improve Children's Dietary Intake? A Pilot Evaluation Study

OBJECTIVE

To evaluate the Kids Café Program (KCP) nutrition education and assess its impact on children's diet quality and body mass index (BMI) percentile.

DESIGN

An experimental design consisting of pretest-posttest comparison groups using mixed methods to evaluate a 6-session nutrition education intervention.

SETTING

Four Boys and Girls Club sites PARTICIPANTS: A total of 120 9- to 12-year-old children in the KCP (60 intervention and 60 comparison); 89% completed posttest evaluations.

INTERVENTION

Trained KCP site staff taught the nutrition education curriculum at intervention sites.

MAIN OUTCOME MEASURES

Healthy Eating Index-2010 using 24-hour dietary recall data (primary) and BMI percentile (secondary) ANALYSIS: Repeated-measures mixed-effects modeling RESULTS: Mean age of children was 10.2 years; mean BMI percentile was about 79; 95% were from food-insecure households. The total Healthy Eating Index-2010 score for both groups at baseline and posttest ranged from 50 to 60. At posttest, compared with baseline scores, children from both groups scored significantly lower for total vegetables, and greens and beans; the intervention group children had significantly higher sodium scores. Process evaluation indicated that 60-minute lecture-based sessions were too long after children were in school all day.

CONCLUSION

This pilot study suggests that the KCP nutrition education curriculum needs improvement. Further research based on behavioral constructs is needed to refine the curriculum to encourage healthier food choices among children and using the MyPlate and the 2015-2020 Dietary Guidelines for Americans.