Evidence valued and used by health promotion practitioners

Changing the culture: a qualitative study exploring research capacity in local government

Background

Local government has become a key constituent for addressing health inequalities and influencing the health of individuals and communities in England. Lauded as an effective approach to tackle the multiple determinants of health, there are concerns that generating and utilising research evidence to inform decision-making and action is a challenge. This research was conducted in a local authority situated in the north of England and addressed the research question – ‘What is the capacity to collaborate and deliver research?’. The study explored the assets that exist to foster a stronger research culture, identified barriers and opportunities for developing research capacity, and how a sustainable research system could be developed to impact on local residents’ health and reduce health inequalities.

Methods

This was a qualitative study utilising semi-structured interviews and focus groups. The study used an embedded researcher (ER) who was digitally embedded within the local authority for four months to conduct the data collection. Senior Managers were purposively sampled from across the local authority to take part in interviews. Three focus groups included representation from across the local authority. Framework analysis was conducted to develop the themes which were informed by the Research Capacity Development framework.

Results

Tensions between research led decision making and the political and cultural context of local government were identified as a barrier to developing research which addressed health inequalities. Research was not prioritised through an organisational strategy and was led sporadically by research active employees. A recognition across leaders that a culture shift to an organisation which used research evidence to develop policy and commission services was needed. Building relationships and infrastructure across local government, place-based collaborators and academic institutions was required. The embedded researcher approach is one method of developing these relationships. The study identifies the strengths and assets that are embedded in the organisational make-up and the potential areas for development.

Conclusion

Research leadership is required in local government to create a culture of evidence-based principles and policy. The embedded research model has high utility in gaining depth of information and recognising contextual and local factors which would support research capacity development.

Learning from public health practice: the development of a library of community-centered practice examples

BACKGROUND

Valuable learning derived from public health practice can be captured through practice-based case studies, also known as practice examples. Practice examples of participatory interventions supplement the evidence base by providing information on the complexities of implementation in communities. This paper reports on a Public Health England project to build a bank of community-centered practice examples based on robust processes of collection and curation.

METHODS

The multidisciplinary project had three phases: (i) development and piloting a process to collect practice examples, (ii) refining review processes and gathering further examples via national and regional teams (iii) maintenance of an accessible collection on the library platform.

RESULTS

The project resulted in a searchable collection of 55 practice examples illustrating participatory approaches in public health practice. The collection shows diversity in terms of settings, population, focus and type of approach used to work with communities. A secondary outcome was the development of generic guidance and templates for further collections on public health topics.

CONCLUSIONS

This project illustrates how information on the implementation of community-centered approaches in real-life contexts can be gathered and disseminated through a transferable process. Having collections of practice examples supports knowledge exchange in public health as learning is shared.

How practitioner, organisational and system-level factors act to influence health promotion evaluation capacity: Validation of a conceptual framework

The need to improve the practice and quality of evaluation in the health promotion and disease prevention field is widely recognised. In order to plan, implement and evaluate health promotion evaluation capacity building efforts, there is a need to better understand the practitioner, organisational and system-level determinants of evaluation capacity and practice. This study aimed to assess the validity Evaluation Practice Analysis Survey (EPAS) constructs using confirmatory factor analysis and validate a conceptual framework of health promotion evaluation capacity using path analysis. Experienced Australian health promotion practitioners completed the survey (n = 219). Twenty-one of the original 23 EPAS scales were assessed as reliable and valid. The final model was found to have good fit (χ²₁₄ = 18.72, p = 0.18, root mean square error of approximation = 0.04, 90% CI 0.00-0.82, Comparative Fit Index = 1.00, standardised root mean square residual = 0.04). This model supports the role of the organisation in facilitating evaluation practice through leadership, culture, systems, support and resources. It builds on existing frameworks from other fields to incorporate political, funding and administrative factors. This study provides an evidence-based model of evaluation capacity that organisations, funders and policy makers can use to plan and implement more effective evaluation capacity building strategies within organisations and the wider prevention field.

A systems approach to the exploration of research activity and relationships within a local authority

Background

Organizations with responsibilities for public health are increasingly required to use evidence-based practice to inform programme delivery, requiring research to generate relevant evidence, and dissemination and use of evidence to inform decisions and practices. Understanding how relationships between organizational structures, systems and processes influence evidence-based practices is critical to improving practice at both an institutional and system level, yet how these relationships should best operate is not well understood. Understanding how to better support research within local authorities, the elected administrative bodies responsible for services including public health at a regional level in the United Kingdom, is a priority for the National Institute for Health Research (NIHR) Public Health Research. This study is based on Norfolk County Council, a local authority in the east of England. We aimed to apply a systems perspective to develop a better understanding of the structures, systems and processes that support a local authority in becoming research-active, identifying gaps in understanding and recommendations for action to address them.

Methods

Taking a participatory action research approach, we applied qualitative methods to explore research activity and relationships in Norfolk County Council. We surveyed employees and used network analysis to map individuals, departments and external partners involved in research activities and the connections between them. We then applied participatory approaches to conduct a series of focus groups and semi-structured interviews to explore stakeholders’ experiences and perceptions of being involved in research at, or with, the authority, and their ideas for recommendations for future actions.

Results

A range of research activity is undertaken at the local authority, with an emphasis on applied work to improve service delivery. We identified several examples of effective practice and models of research collaboration in some departments. Challenges such as limitations in resources, capacity and knowledge exchange were evident, yet there was a readiness amongst key stakeholders to develop and implement actions that may better support the authority in becoming more research-active.

Conclusion

In large complex organizations, a key challenge is how to share learning across teams and implement good practice at an organizational and system level. Our findings highlight the potential for developing improved collaborative partnership models and systems to support sustainable processes and practices for research and knowledge exchange at an institutional and interorganizational level. The insights gained and shared will support other local authorities and similar large, multilevel organizations with responsibilities for evidence-based public health to explore their own setting and implement change where needed, and provide stimulus for further research into system-level change.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-021-00792-0.

Exploring influences on evaluation practice: a case study of a national physical activity programme

BACKGROUND

Interventions to improve physical activity behaviour are a core part of public health policy and practice. It is essential that we evaluate these interventions and use the evidence to inform decisions to improve population health. Evaluation of 'real-world' interventions provide an opportunity to generate practice-relevant evidence, however these interventions are difficult to evaluate. Various guidelines have been developed to facilitate evaluation, but evidence about their effectiveness in practice is limited. To explore influences on evaluation practice in an applied context, we conducted a case study of Sport England's 'Get Healthy Get Active' (GHGA) programme. This was a national programme that funded 33 projects that were delivered and evaluated across England. The programme was chosen as it was designed to generate evidence on the role of sport in increasing physical activity and improving health. The study aimed to explore and appraise whether strategies intended to facilitate project evaluation, including funder requirements to use a standardised evaluation framework and specific data collection methods, were effective in generating evidence that enabled the programme to meet its aims.

METHODS

We applied a collective case study design involving 35 semi-structured interviews, and documentary analysis of multiple sources of evidence from 23 physical activity projects funded by GHGA. We applied thematic and framework analysis. We developed a logic model and mapped actual outcomes against intended outcomes. A narrative synthesis is provided. We discuss implications for the effective commissioning and evaluation of public health interventions.

RESULTS

We identified five main themes of influences on evaluation practices that can act as barriers and facilitators to good practice: programme and project design; evaluation design; partnerships; resources; and organisational structures and systems. These influences are context-specific and operate through a complex set of interactions.

CONCLUSION

Developing a better understanding of how influences on evaluation practice can act as facilitators or barriers is vital to help close current gaps in the evidence-based practice cycle. Critically, organisational structures and systems are needed to facilitate collaborative decision making; integration of projects and evaluation across partners organisations; transfer of knowldege and insights between stakeholders; and more rapid feedback and dissemination.

A scoping review of evaluation frameworks and their applicability to real-world physical activity and dietary change programme evaluation

BACKGROUND

Physical activity and dietary change programmes play a central role in addressing public health priorities. Programme evaluation contributes to the evidence-base about these programmes; and helps justify and inform policy, programme and funding decisions. A range of evaluation frameworks have been published, but there is uncertainty about their usability and applicability to different programmes and evaluation objectives, and the extent to which they are appropriate for practitioner-led or researcher-led evaluation. This review appraises the frameworks that may be applicable to evaluation of physical activity and/or dietary change programmes, and develops a typology of the frameworks to help guide decision making by practitioners, commissioners and evaluators.

METHODS

A scoping review approach was used. This included a systematic search and consultation with evaluation experts to identify evaluation frameworks and to develop a set of evaluation components to appraise them. Data related to each framework's general characteristics and components were extracted. This was used to construct a typology of the frameworks based on their intended programme type, evaluation objective and format. Each framework was then mapped against the evaluation components to generate an overview of the guidance included within each framework.

RESULTS

The review identified 71 frameworks. These were described variously in terms of purpose, content, or applicability to different programme contexts. The mapping of frameworks highlighted areas of overlap and strengths and limitations in the available guidance. Gaps within the frameworks which may warrant further development included guidance on participatory approaches, non-health and unanticipated outcomes, wider contextual and implementation factors, and sustainability.

CONCLUSIONS

Our typology and mapping signpost to frameworks where guidance on specific components can be found, where there is overlap, and where there are gaps in the guidance. Practitioners and evaluators can use these to identify, agree upon and apply appropriate frameworks. Researchers can use them to identify evaluation components where there is already guidance available and where further development may be useful. This should help focus research efforts where it is most needed and promote the uptake and use of evaluation frameworks in practice to improve the quality of evaluation and reporting.

Organizational Supports for Research Evidence Use in State Public Health Agencies: A Latent Class Analysis

OBJECTIVE

Use of research evidence in public health decision making can be affected by organizational supports. Study objectives are to identify patterns of organizational supports and explore associations with research evidence use for job tasks among public health practitioners.

DESIGN

In this longitudinal study, we used latent class analysis to identify organizational support patterns, followed by mixed logistic regression analysis to quantify associations with research evidence use.

SETTING

The setting included 12 state public health department chronic disease prevention units and their external partnering organizations involved in chronic disease prevention.

PARTICIPANTS

Chronic disease prevention staff from 12 US state public health departments and partnering organizations completed self-report surveys at 2 time points, in 2014 and 2016 (N = 872).

MAIN OUTCOME MEASURES

Latent class analysis was employed to identify subgroups of survey participants with distinct patterns of perceived organizational supports. Two classify-analyze approaches (maximum probability assignment and multiple pseudo-class draws) were used in 2017 to investigate the association between latent class membership and research evidence use.

RESULTS

The optimal model identified 4 latent classes, labeled as "unsupportive workplace," "low agency leadership support," "high agency leadership support," and "supportive workplace." With maximum probability assignment, participants in "high agency leadership support" (odds ratio = 2.08; 95% CI, 1.35-3.23) and "supportive workplace" (odds ratio = 1.74; 95% CI, 1.10-2.74) were more likely to use research evidence in job tasks than "unsupportive workplace." The multiple pseudo-class draws produced comparable results with odds ratio = 2.09 (95% CI, 1.31-3.30) for "high agency leadership support" and odds ratio = 1.74 (95% CI, 1.07-2.82) for "supportive workplace."

CONCLUSIONS

Findings suggest that leadership support may be a crucial element of organizational supports to encourage research evidence use. Organizational supports such as supervisory expectations, access to evidence, and participatory decision making may need leadership support as well to improve research evidence use in public health job tasks.

Barriers and opportunities in the translation of mobile phone and social media interventions between research and health promotion practice in Australia: a qualitative study of expert perspectives

Background

Newer technologies, such as smartphones and social networking sites, offer new opportunities for health promotion interventions. There is evidence to show that these technologies can be effectively and acceptably used for health promotion activities. However, most interventions produced in research do not end up benefitting non-research populations, while the majority of technology-facilitated interventions which are available outside of research settings are either undocumented or have limited or no evidence to support any benefit. We therefore aimed to explore the perspectives of researchers and health promotion experts on efforts to translate technology-facilitated prevention initiatives into practice, and the barriers to achieving translation.

Methods

We utilised a qualitative study design, involving in-depth interviews with researchers experienced with technology-facilitated prevention interventions and prominent health promotion experts.

Results

Some barriers mirror the findings of other studies into health promotion practice, which have found that competing priorities, resource limitations and organisational capacity are important in determining use of evidence in programme planning, engagement in translation and evaluation practice. We add to this literature by describing barriers that are more specifically related to technology-facilitated prevention, such as the pace of developments in technology, and how this clashes with the time taken to develop and ready evidence for translation.

Conclusions

In order to maximise the vast potential of technology-facilitated prevention interventions to promote population health, it is essential that translation is at the forefront of consideration for both researchers and practitioners. We suggest actions that can be taken by both researchers and practitioners to improve translation of technology-facilitated prevention interventions, and also highlight how funding schemes can be modified to facilitate translation.

Organizational determinants of evaluation practice in Australian prevention agencies

Program evaluation is essential to inform decision making, contribute to the evidence base for strategies, and facilitate learning in health promotion and disease prevention organizations. Theoretical frameworks of organizational learning, and studies of evaluation capacity building describe the organization as central to evaluation capacity. Australian prevention organizations recognize limitations to current evaluation effectiveness and are seeking guidance to build evaluation capacity. This qualitative study identifies organizational facilitators and barriers to evaluation practice, and explores their interactions in Australian prevention organizations. We conducted semi-structured interviews with 40 experienced practitioners from government and non-government organizations. Using thematic analysis, we identified seven key themes that influence evaluation practice: leadership, organizational culture, organizational systems and structures, partnerships, resources, workforce development and training and recruitment and skills mix. We found organizational determinants of evaluation to have multi-level interactions. Leadership and organizational culture influenced organizational systems, resource allocation and support of staff. Partnerships were important to overcome resource deficits, and systems were critical to embed evaluation within the organization. Organizational factors also influenced the opportunities for staff to develop skills and confidence. We argue that investment to improve these factors would allow organizations to address evaluation capacity at multiple levels, and ultimately facilitate effective evaluation practice.

"Layers of translation" - evidence literacy in public health practice: a qualitative secondary analysis

BACKGROUND

Strengthening public health systems has been a concern in Canada in the wake of public health emergencies. In one Canadian province, British Columbia, a high priority has been placed on the role of evidence to guide decision making; however, there are numerous challenges to using evidence in practice. The National Collaborating Centre for Methods and Tools therefore developed the Evidence Informed Public Health Framework (EIPH), a seven step guide to assist public health practitioners to use evidence in practice. We used this framework to examine the evidence literacy of public health practitioners in BC.

METHODS

We conducted a secondary analysis of two separate qualitative studies on the public health renewal process in which the use and understanding of evidence were key interview questions. Using constant comparative analysis, we analyzed the evidence-related data, mapping it to the categories of the EIPH framework.

RESULTS

Participants require both data and evidence for multiple purposes in their daily work; data may be more important to them than research evidence. They are keen to provide evidence-based programs in which research evidence is balanced with community knowledge and local data. Practitioners recognise appraisal as an important step in using evidence, but the type of evidence most often used in daily practice does not easily lend itself to established methods for appraising research evidence. In the synthesis stage of the EIPH process, synthesized evidence in the form of systematic reviews and practice guidelines is emphasized. Participants, however, need to synthesize across the multiple forms of evidence they use and see the need for more skill and resources to help them develop skill in this type of synthesis.

CONCLUSIONS

Public health practitioners demonstrated a good level of evidence literacy, particularly at the collective level in the organization. The EIPH framework provides helpful guidance in how to use research evidence in practice, but it lacks support on appraising and synthesizing across the various types of evidence that practitioners consider essential in their practice. We can better support practitioners by appreciating the range of evidence they use and value and by creating tools that help them to do this.

Process for massage therapy practice and essential assessment

BACKGROUND

Little evidence exists about processes in massage therapy practice. Investigating current frameworks is warranted. This qualitative study is a secondary data analysis using grounded theory to understand how massage therapy experts describe massage therapy practice.

METHODS

31 massage therapy experts were invited to a 2-day symposium to discuss best practices for the profession. Through qualitative analysis, memoing, and discussion, the data were summarized into themes.

RESULTS

Three themes were identified around massage therapy practice: 1) client centered, 2) structure for practice, and 3) influencing factors. Each theme is clarified and expanded.

DISCUSSION

Conceptual models were developed for research and clinical practice and a definition for massage therapy practice was identified. Challenges and limitations are discussed.

CONCLUSION

The goal of providing these models is to give massage therapists tools to deliver the best possible care. The models need testing to see if they help advance the profession.