Developing guidelines to enhance the evaluation of overseas development projects

Culturally responsive evaluation: A scoping review of the evaluation literature

Evaluators have become increasingly aware of the influence of culture in evaluation, leading to new evaluation approaches that account for the cultural considerations in which evaluations are situated. This scoping review sought to explore how evaluators understand culturally responsive evaluation and identify promising practices. A search of nine evaluation journals yielded 52 articles that were included in this review. Nearly two-thirds of the articles stated that community involvement was essential to culturally responsive evaluation. Power differentials were discussed in almost half of the articles, and the majority used participatory or collaborative approaches to community engagement. Findings from this review suggest that in culturally responsive evaluation, evaluators prioritize community involvement and have an awareness and attentiveness to power differentials. Yet, gaps exist in how culture and evaluation are defined and interpreted, and consequently, inconsistency in how culturally responsive evaluation is practiced.

The last one heard: the importance of an early-stage participatory evaluation for programme implementation

BACKGROUND

The systematic involvement of project beneficiaries in community maternal and child health programmes remains low and limited, especially during the formative stages of the project cycle. Understanding how positive and negative feedbacks obtained from communities can subsequently be used to inform and iterate existing programmes is an important step towards ensuring the success of community health workers for maternal and child health programming and, ultimately, for improving health outcomes.

METHODS

The study took place over a period of 4 weeks in North Rukiga, Kabale District of southwestern Uganda. Using a cross-sectional qualitative study that employed an epistemological approach of phenomenology, nine focus group discussions and eight in-depth interviews were conducted with a total of 76 female participants across six different sites. Women were identified as either users or non-users of the maternal and child health programme. Purposeful sampling was employed to recruit women from six different locations within the programme catchment area. Translated and transcribed transcripts were subjected to a bottom-up thematic analysis using NVivo 10 Software, whereby themes were arrived at inductively.

RESULTS

Predominant themes emerging from the focus groups and key informant interviews identified early trends in programme strengths. Beneficiaries reported confidence in both the programme and the relationships they had forged with community health workers, exhibited pride in the knowledge they had received, and described improved spousal involvement. Beneficiaries also identified a number of programme challenges including barriers to adopting the behaviours promoted by the programme, and highlighted issues with programme dependency and perceived ownership. It also emerged that community health workers were not reaching the entire population of intended programme beneficiaries.

CONCLUSIONS

This research provides support for the importance of an early-stage participatory evaluation of beneficiaries' perceptions of newly initiated health programmes. Our results support how evaluations conducted in the early phases of programme implementation can provide valuable, timely feedback as well as yield recommendations for programme adjustment or re-alignment, and in turn, better meet end-user expectations. Potential reasons for the observed lack of community participation in early stages of programme implementation are considered.

Participatory evaluation of reproductive health care quality in developing countries

Participatory approaches are becoming increasingly important in the field of health, and many organizations, governments and donors have recognized the need to increase stakeholder involvement to ensure sustainable and real change. However, commitment to participation is often lacking and participatory processes, if applied, tend to be short-term and discrete, especially in institutional settings. Rarely, for example, are stakeholders involved in long-term monitoring and evaluation activities, due the time-consuming nature of participation, and to perceptions on the part of donors and other decision-makers that participation lacks the rigor and objectivity of external evaluation. This paper describes the strategies used by an international reproductive health organization to collaborate with local stakeholders in a long-term participatory approach to quality improvement, focusing on defining quality of care, identifying problems in health facilities, setting goals and seeking solutions to those problems, tracking changes in quality over time, and feeding this monitoring and evaluation information back into the quality improvement process. The paper also illustrates how greater participation was achieved over time as local stakeholders moved away from traditional models and relationships and started working together to meet their quality improvement goals. The paper argues that participatory techniques are essential if the real needs of clients are to be met through sustained change and continuous quality improvement at the site level.