Building evaluation capacity of local substance abuse prevention programs serving LGBQ populations

Laying a Solid Foundation for the Next Generation of Evaluation Capacity Building: Findings from an Integrative Review

Evaluation capacity building (ECB) continues to attract the attention and interest of scholars and practitioners. Over the years, models, frameworks, strategies, and practices related to ECB have been developed and implemented. Although ECB is highly contextual, the evolution of knowledge in this area depends on learning from past efforts in a structured approach. The purpose of the present article is to integrate the ECB literature in evaluation journals. More specifically, the article aims to answer three questions: What types of articles and themes comprise the current literature on ECB? How are current practices of ECB described in the literature? And what is the current status of research on ECB? Informed by the findings of the review, the article concludes with suggestions for future ECB practice and scholarship.

How practitioner, organisational and system-level factors act to influence health promotion evaluation capacity: Validation of a conceptual framework

The need to improve the practice and quality of evaluation in the health promotion and disease prevention field is widely recognised. In order to plan, implement and evaluate health promotion evaluation capacity building efforts, there is a need to better understand the practitioner, organisational and system-level determinants of evaluation capacity and practice. This study aimed to assess the validity Evaluation Practice Analysis Survey (EPAS) constructs using confirmatory factor analysis and validate a conceptual framework of health promotion evaluation capacity using path analysis. Experienced Australian health promotion practitioners completed the survey (n = 219). Twenty-one of the original 23 EPAS scales were assessed as reliable and valid. The final model was found to have good fit (χ²₁₄ = 18.72, p = 0.18, root mean square error of approximation = 0.04, 90% CI 0.00-0.82, Comparative Fit Index = 1.00, standardised root mean square residual = 0.04). This model supports the role of the organisation in facilitating evaluation practice through leadership, culture, systems, support and resources. It builds on existing frameworks from other fields to incorporate political, funding and administrative factors. This study provides an evidence-based model of evaluation capacity that organisations, funders and policy makers can use to plan and implement more effective evaluation capacity building strategies within organisations and the wider prevention field.

Understanding the factors that influence health promotion evaluation: The development and validation of the evaluation practice analysis survey

The demand for improved quality of health promotion evaluation and greater capacity to undertake evaluation is growing, yet evidence of the challenges and facilitators to evaluation practice within the health promotion field is lacking. A limited number of evaluation capacity measurement instruments have been validated in government or non-government organisations (NGO), however there is no instrument designed for health promotion organisations. This study aimed to develop and validate an Evaluation Practice Analysis Survey (EPAS) to examine evaluation practices in health promotion organisations. Qualitative interviews, existing frameworks and instruments informed the survey development. Health promotion practitioners from government agencies and NGOs completed the survey (n = 169). Principal components analysis was used to determine scale structure and Cronbach's α used to estimate internal reliability. Logistic regression was conducted to assess predictive validity of selected EPAS scale. The final survey instrument included 25 scales (125 items). The EPAS demonstrated good internal reliability (α > 0.7) for 23 scales. Dedicated resources and time for evaluation, leadership, organisational culture and internal support for evaluation showed promising predictive validity. The EPAS can be used to describe elements of evaluation capacity at the individual, organisational and system levels and to guide initiatives to improve evaluation practice in health promotion organisations.

Developing harm reduction in the context of youth substance use: insights from a multi-site qualitative analysis of young people's harm minimization strategies

BACKGROUND

Youth substance use programming and educational strategies are frequently informed by prevention approaches that emphasize abstinence goals, which often do not resonate with youth in their lack of acknowledgment of young people's social context and how young people perceive positive effects of substance use. Further, approaches to drug prevention have been critiqued as adopting a one-size-fits-all approach and therefore inadequate in addressing substance use in the context of population variation and inequities. In response to the limitations of current approaches to prevention, programming informed by harm reduction principles that aims to minimize harms without requiring abstinence is emergent in school settings. However, youth perspectives informing harm reduction are limited in both research and program development.

METHODS

This paper draws on data from the Researching Adolescent Distress and Resilience (RADAR) study, which utilized an ethnographic approach to bring youth voice to the literature on mental health and substance use. Qualitative data collection included individual interviews (n = 86) with young people aged 13-18 across three communities-representing urban, suburban, and rural geographies-in British Columbia, Canada. A multi-site qualitative analysis of interview data was conducted to identify themes across and within each research site.

RESULTS

Across all three sites, young people's individual experiences of substance use were shaped by geographic, socio-cultural, and political contexts, with youth describing their use in relation to the nature of substance use in peer groups and in the broader community. To manage their own substance use and reduce related harms, youth employed a variety of ad hoc harm minimization strategies that were reflective of their respective contexts.

CONCLUSIONS

The findings from this study suggest the importance of harm reduction approaches that are contextually relevant and responsive to the lived experiences of youth. Youth perspectives in the development of harm reduction programming are needed to ensure that approaches are relatable and meaningful to young people, and effective for promoting the minimization of substance-related harms.