Evaluation of the implementation of a whole-workplace walking programme using the RE-AIM framework

Breaking the chain from the chair: a manager's perspective on reducing employees sedentary time in a home-office context

BACKGROUND

Office workers have the highest estimated daily sedentary time (ST). The Covid-19 pandemic resulted in a switch from office-based work to home-office work or a hybrid of both. Home-office work has been shown to increase ST compared to the office, which may have deleterious health consequences.

OBJECTIVE

This study explored managers perspectives on the factors influencing their employee's ability to reduce ST in a home-office context.

DESIGN

A descriptive qualitative study.

METHODS

Semi-structured interviews (n = 20), which were mapped to the COM-B model, were conducted with managers from Ireland, Spain and The Netherlands. Interviews were conducted through Zoom, recorded and transcribed verbatim. A reflexive thematic analysis approach was used.

RESULTS

Organisational support, management engaging in physical activity (PA) during work and a social element were seen as key to increasing engagement in interventions. Creating opportunities for employees to engage in PA during work was seen as another key element. Leveraging infrastructure put in place during the Covid-19 pandemic was identified as a feasible approach to providing education and encouragement to employees. Lastly, managers feared the impact reducing ST may have on the performance of both employees and the organisation, yet, managers felt an increase in employee well-being would benefit their company long-term.

CONCLUSION

These results highlight the need to include organisational support and leadership from management, ensuring a top-down approach. These changes may create opportunities for employees to reduce their ST while working from home, which may benefit both the employees' and organisations.

Applying implementation science to evaluate participatory ergonomics program for continuous improvement: A case study in the construction industry

While participatory ergonomics (PE) presents numerous benefits, its empirical effectiveness remains elusive due to the lack of transparency in implementation contexts and processes. This hinders our ability to discern the reasons behind a program's success or failure and to determine optimization and adaptation strategies. To unravel this "black box," we present a case study using implementation science to evaluate a PE program and illuminate the mechanisms linking process to outcome. The study examines a 4.5-year PE program at a construction company, targeting musculoskeletal disorders (MSDs) from material-cart handling. Using the RE-AIM framework and Implementation Causal Pathway Model, we evaluated implementation process and catalogued contextual factors through worker surveys (n = 106), safety document review (27 training sessions and 7 construction projects), and key informant observations. We assessed the program's impact using a 42-worker survey and an analysis of 8-year injury data, and determined the return-on-investment (ROI) by monetizing the collected data. The program achieved significant impact: Workers' ergonomic knowledge improved from 73% in baseline to 86% in follow-up; 97% of workers reported at least one positive change in their crews; and no cart handling injuries occurred after the first program change, resulting in a ROI rate of 1.99. Implementation process evaluation revealed that seamless integration, tailored intervention, and ongoing adjustments contributed to the success. Five organizational factors necessary for the effective functioning of these three strategies were identified, along with three moderators that amplified their influence. Finally, this case study demonstrates that implementation science offers a coherent structure for evaluating PE programs, uncovering mechanisms of change, and informing future improvements and adaptations. Our research facilitates knowledge transfer from implementation science to ergonomics, eventually leading to more cost-effective PE programs that are faithfully implemented across various industrial settings to prevent MSDs.

Smartphone-based versus traditional face-to-face collaborative care for community-dwelling older adults living with dementia in China: protocol for an implementation science-based sequential multiple assignment randomised trial

INTRODUCTION

The high costs of institutional care and the burdensome demands of home care are challenging for families of adults with dementia. The collaborative care model (CCM) provides a potential solution to these challenges. Leveraging advancements in mobile technologies, smartphone-based management could offer a feasible means of providing collaborative care in a community setting. Therefore, this study aims to establish a CCM for home-cared older adults with dementia to determine the best strategy to deliver collaborative care, including both the channel and frequency of delivery.

METHODS AND ANALYSIS

This study will be conducted in the communities of Chengdu city, Sichuan province, China. It is designed under the framework of implementation science. In the first stage, intervention strategies for community-dwelling older adults with dementia and their caregivers will be developed using Delphi methods and focus group interviews. The second stage will involve designing a sequential multiple assignment randomised trial to compare the effectiveness of face-to-face intervention versus a WeChat mini program-based intervention. This comparison will involve 358 pairs of older adults with dementia and their caregivers, with the frequency of intervention also assessed. Follow-up evaluations will be implemented at the 6th, 12th and 18th months post-intervention initiation. Primary outcomes encompass the proportion of patients demonstrating an improvement in quality of life and the proportion of caregivers exhibiting a reduction in caregiver burden. Analysis will be based on the intention-to-treat principle, and the generalised estimating equation approach will be used. Incremental cost-effectiveness ratios will be used to evaluate the cost-effectiveness of different delivery methods and frequencies.

ETHICS AND DISSEMINATION

This study has received approval from the Ethics Committee of West China Fourth Hospital/School of Public Health, Sichuan University (Gwll2022004). Informed consent will be obtained for all participants. The findings of the study will be disseminated through peer-reviewed scientific journals.

TRIAL REGISTRATION NUMBER

ChiCTR2200057945.

Evaluation of a systems ergonomics-based incident reporting system

INTRODUCTION

Incident reporting and learning systems are a fundamental component of safety management. The purpose of this study was to evaluate a novel incident reporting system specific to the Australian Led Outdoor Activity (LOA) sector. The Understanding and Preventing Led Outdoor Accidents Data System (UPLOADS), is a recently introduced systems thinking-based incident reporting and learning system that utilises contemporary safety theory and methods.

METHOD

The implementation of UPLOADS was evaluated using the RE-AIM framework based on the following dimensions: Reach, Efficacy, Adoption, Implementation, and Maintenance. A pragmatic evaluation approach was used in which evaluation data were collected through the triangulation of multiple sources including different LOA stakeholders from both the individual organisational level (LOA providers) and LOA sector governance level (LOA industry representatives), incident data collected through the UPLOADS National Incident Dataset, and the online and physical presence of UPLOADS.

RESULTS

The findings show that a key strength of UPLOADS is its effectiveness as incident reporting tool for improving safety in the LOA sector. However, a weakness of UPLOADS is that it is not being implemented appropriately by the LOA providers.

CONCLUSION

Overall, the current findings suggest that UPLOADS incident reporting tool is perceived by SMEs as an effective tool for improving safety in LOA. However, further work is required for UPLOADS to have a greater impact on the LOA sector. Specifically, Implementation of the UPLOADS system requires improvement, as well as additional training and education may be required to upskill and empower LOA providers to improve reporting and enhance the value placed on safety by LOA stakeholders. The RE-AIM framework was an appropriate evaluation framework for understanding the effectiveness of UPLOADS as a LOA sector specific incident reporting and learning system.

PRACTICAL APPLICATIONS

The current findings have practical implications for ergonomics researchers applying evaluation frameworks in the real world, and LOA providers for implementing safety interventions. Lastly, contemporary systems-based incident reporting and learning systems have the capability to enhance the safety practices of the LOA sector.

Leveraging an Implementation Science Framework to Measure the Impact of Efforts to Scale Out a Total Worker Health® Intervention to Employers

The role of dissemination and implementation (D&I) science is critical to the translation of Total Worker Health^® into practice and to the success of interventions in addressing current and future implications for worker safety, health, and well-being. D&I frameworks can guide researchers to design Total Worker Health (“TWH”) delivery approaches that use flexible implementation strategies to implement the core components of programs for employers with varying contextual factors, including small/mid/large-sized businesses and different industry types. To date, there have been very few examples of applying implementation frameworks for the translation and delivery of interventions into organizational settings that require adoption and implementation at the business level to benefit the working individuals. We present a TWH case study, Health Links™, to illustrate an approach to applying an existing implementation framework, RE-AIM, to plan, design, build, and then evaluate TWH implementation strategies. Our case study also highlights key concepts for scaling-out TWH evidence-based interventions where they are implemented in new workplace settings, new delivery systems, or both. Our example provides strong support of key implementation planning constructs including early and consistent stakeholder engagement, tailored messaging and marketing, flexibility, and adaptations in implementation strategies to maximize adoption, implementation, and maintenance among participating businesses.

A systematic review of the use and reporting of evaluation frameworks within evaluations of physical activity interventions

BACKGROUND

Evaluation of physical activity interventions is vital to inform, and justify, evidence-based policy and practice to support population-wide changes in physical activity. Several evaluation frameworks and guidance documents have been developed to facilitate the evaluation and reporting of evaluation studies in public health. However, there is a lack of evidence about whether frameworks are being used to guide evaluation. There continues to be claims of poor and inconsistent reporting in evaluation studies. The aim of this review was to assess the use of evaluation frameworks and the quality of reporting of how they were applied within evaluation studies of physical activity interventions.

OBJECTIVES

1. To identify whether evaluation frameworks are reported to have been used within evaluation studies of physical activity interventions, and which frameworks have been used. 2. To appraise the quality of reporting with regards to how evaluation frameworks have been used.

METHOD

We developed a checklist of indicators to enable a critical appraisal of the use and reporting of different evaluation frameworks in evaluation studies. We conducted a systematic search and review of evaluation studies published between 2015 and the date of the search to appraise the use and reporting of evaluation frameworks. A narrative synthesis is provided.

RESULTS

The review identified 292 evaluation studies of physical activity interventions, only 69 (23%) of these mentioned using an evaluation framework, and only 16 different frameworks were referred to. There was variation in the quality of reporting of framework use. 51 (74%) studies were identified as being explicitly based on the stated framework, however only 26 (38%) provided detailed descriptions consistently across all the checklist indicators. Details of adaptations and limitations in how frameworks were applied were less frequently reported. The review also highlighted variability in the reporting of intervention components. More consistent and precise reporting of framework and intervention components is needed.

CONCLUSION

Evaluation frameworks can facilitate a more systematic evaluation report and we argue their limited use suggests missed opportunities to apply frameworks to guide evaluation and reporting in evaluation studies. Variability in the quality of reporting of framework use limits the comparability and transferability of evidence. Where a framework has been used, the checklist of indicators can be employed to facilitate the reporting of an evaluation study and to review the quality of an evaluation report.

General practice referral of 'at risk' populations to community leisure services: applying the RE-AIM framework to evaluate the impact of a community-based physical activity programme for inactive adults with long-term conditions

BACKGROUND

In the UK a high proportion of adults with long-term conditions do not engage in regular physical activity. General practice (GP) referral to community-based physical activity is one strategy that has gained traction in recent years. However, evidence for the real-world effectiveness and translation of such programmes is limited. This study aimed to evaluate the individual and organisational impacts of the 'CLICK into Activity' programme - GP referral of inactive adults living with (or at risk of) long-term conditions to community-based physical activity.

METHODS

A mixed methods evaluation using the RE-AIM framework was conducted with data obtained from a range of sources: follow-up questionnaires, qualitative interviews, and programme-related documentation, including programme cost data. Triangulation methods were used to analyse data, with findings synthesised across each dimension of the RE-AIM framework.

RESULTS

A total of 602 individuals were referred to CLICK into Activity physical activity sessions. Of those referred, 326 individuals participated in at least one session; the programme therefore reached 30.2% of the 1080 recruitment target. A range of individual-, social-, and environmental-level factors contributed to initial physical activity participation. Positive changes over time in physical activity and other outcomes assessed were observed among participants. Programme adoption at GP surgeries was successful, but the GP referral process was not consistently implemented across sites. Physical activity sessions were successfully implemented, with programme deliverers and group-based delivery identified as having an influential effect on programme outcomes. Changes to physical activity session content were made in response to participant feedback. CLICK into Activity cost £175,000 over 3 years, with an average cost per person attending at least one programme session of £535.

CONCLUSIONS

Despite not reaching its recruitment target, CLICK into Activity was successfully adopted. Positive outcomes were associated with participation, although low 6- and 12-month follow-up response rates limit understanding of longer-term programme effects. Contextual and individual factors, which may facilitate successful implementation with the target population, were identified. Findings highlight strategies to be explored in future development and implementation of GP referral to community-based physical activity programmes targeting inactive adults living with (or at risk of) long-term conditions.

RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review

The RE-AIM planning and evaluation framework was conceptualized two decades ago. As one of the most frequently applied implementation frameworks, RE-AIM has now been cited in over 2,800 publications. This paper describes the application and evolution of RE-AIM as well as lessons learned from its use. RE-AIM has been applied most often in public health and health behavior change research, but increasingly in more diverse content areas and within clinical, community, and corporate settings. We discuss challenges of using RE-AIM while encouraging a more pragmatic use of key dimensions rather than comprehensive applications of all elements. Current foci of RE-AIM include increasing the emphasis on cost and adaptations to programs and expanding the use of qualitative methods to understand "how" and "why" results came about. The framework will continue to evolve to focus on contextual and explanatory factors related to RE-AIM outcomes, package RE-AIM for use by non-researchers, and integrate RE-AIM with other pragmatic and reporting frameworks.

An Integrative, Systematic Review Exploring the Research, Effectiveness, Adoption, Implementation, and Maintenance of Interventions to Reduce Sedentary Behaviour in Office Workers

Sedentary behaviour is associated with poor health outcomes, and office-based workers are at significant health risk, as they accumulate large proportions of their overall sitting time at work. The aim of this integrated systematic review was to collate and synthesize published research on sedentary behaviour interventions in the workplace that have reported on at least one an aspect of the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework. Studies were included if they involved adult office workers, were conducted in an office setting, and changes in sedentary behaviour had been measured as a primary outcome. Five electronic databases were searched yielding 7234 articles, with 75 articles (61 individual interventions) meeting the inclusion criteria. Reach indicators were the most frequently reported RE-AIM dimensions, which were reported on average 59% of the time. Efficacy/effectiveness was the second most reported dimension at 49% reporting across all of the indicators. Implementation indicators were reported an average of 44% of the time, with indicators of adoption and maintenance reported as the lowest of all indicators at 13% and 8%, respectively. Recommendations are provided to improve reporting across all RE-AIM dimensions, which is an important first step to enable the effective translation of interventions into real world settings.