A framework for developing evaluation capacity in health care settings

Narrative review of strategies by organizations for building evaluation capacity

Program evaluation is an important source of information to assist organizations to make "evidence-informed" decisions about program planning and development. The objectives of this study were to identify evaluated strategies used by organizations and program developers to build the program evaluation capacity of their workforce, and to describe success factors and lessons learned. Common elements for successful evaluation capacity building (ECB) include: a tailored strategy based on needs assessment, an organizational commitment to evaluation and ECB, experiential learning, training with a practical element, and some form of ongoing technical support within the workplace. ECB is a relatively new field of endeavor, and, while existing studies in ECB are characterized by lower levels of evidence, they suggest the most successful approaches to ECB are likely to be multifaceted. To build the level of evidence in this field, more rigorous study designs need to be implemented in the future.

Using evaluability assessment and evaluation capacity-building to strengthen community-based prevention initiatives

OBJECTIVES

To examine the process of community-campus engagement in an initiative developed to build evaluation capacities of community-based organizations (CBOs).

METHODS

Evaluability assessment, capacity-building, self administered surveys and semi-structured interviews were conducted from 2004 to 2007 and analyzed through transcript assessment and SPSS to identify trends, relationships and capacity changes over time.

RESULTS

Evaluability assessment identified CBO strengths in program planning and implementation and challenges in measurable objective development, systematic use of mixed methods, data management and analysis. Evaluability assessment informed evaluation capacity-building (ECB) trainings, teleconferences and webinars that resulted in statistically significant improvements in evaluation knowledge, skills, and abilities. Post-initiative interviews indicated CBO preferences for face-to-face training in logic model development, mixed method data collection and analysis.

CONCLUSION

This report illustrates the use of mixed methods to plan, implement and evaluate a model to catalyze CBOs systematic assessment of prevention initiatives and considerations in evaluation capacity-building.

Community-based organizations' capacity to plan, implement, and evaluate success

Community-based organizations (CBOs) have the potential to promote and sustain health, prevent disease, and address health disparities, but many lack the capacity to do so. An assessment of the 20 CBOs receiving supplemental grant funding from the Pfizer Foundation Southern HIV/AIDS Prevention Initiative indicated a high level of knowledge for developing goals and objectives (mean score=3.08 on a scale of 0 (none) to 4 (extensive)) and high self-assessed abilities to conduct six of 20 specific intervention activities, including the development of community relationships and coalitions. Lower knowledge and skill levels were observed for intervention evaluation. While CBOs of this Initiative have established prerequisite abilities, they have self-acknowledged needs for technical assistance to maximize HIV/AIDS prevention capacity.