Evaluation of the performance and achievements of the WHO Evidence-informed Policy Network (EVIPNet) Europe

Protocol: What works to increase the use of evidence for policy decision-making: A systematic review

This is the protocol for a Campbell systematic review. The objectives are as follows: Our aim is to collect, assess, and synthesise all the available empirical evidence on what works to support evidence-informed decision-making by policymakers. In doing so, we will aim to answer the following research questions: What are the impacts of interventions to support evidence-informed decision-making by policymakers? What are the factors which have influenced the impact of these interventions, and their design and implementation in low- and middle-income countries? In answering these questions, our goal is to estimate the overall impact and relative effectiveness of different interventions, identify factors or configurations of factors that support or hinder their effectiveness in low- and middle-income countries and to identify gaps and areas for future primary research.

Building bridges: evaluating policymakers' research capacities, engagement, and utilization in health policymaking within the Kuwaiti context: a cross-sectional study

BACKGROUND

Health policymaking is a critical aspect of governmental decision-making that shapes the well-being of populations. In the Middle East and North Africa, particularly in Kuwait, limited attention has been given to exploring the research capacities, engagement, and utilization among health policymakers. This study aims to bridge this gap by investigating how Kuwaiti health policymakers incorporate evidence-based research into the formulation of health-related policies.

METHODS

This cross-sectional study targeted health policymakers in leadership positions within the Kuwait Ministry of Health (MOH). Using the Seeking, Engaging with and Evaluating Research (SEER) questionnaire, participants' capacities, engagement, and use of research were assessed. The targeted sample was all health policymakers in leadership positions, starting from the head of departments and above. The questionnaire comprises four domains, 14 sections, and 50 questions and utilizes Likert and binary scales, with aggregate scores predicting engagement actions and research use. The data were collected between March and July 2023. All the statistical analyses were performed using SPSS v27, and the numerical and categorical variables were analyzed using appropriate statistical tests, including t-tests, ANOVA, and Pearson's correlation.

RESULTS

Out of 205 policymakers, 88 participated (42.9% response rate): predominantly male (51.1%) and married (78.4%). The mean age was 49.84 ± 7.28 years, with a mean MOH tenure of 24.39 ± 6.80 years. Participants demonstrated high value for research (mean score 4.29 ± 0.55) and expressed confidence in the research utilization. Organizational emphasis on research use exhibited nuanced perceptions, identifying areas where MOH support may be lacking. Access to research resources and processes for policy development guidance were highlighted as challenges.

CONCLUSIONS

This study provides crucial insights into the research capacities and engagement of Kuwaiti health policymakers. It emphasizes the need for targeted interventions to align individual perceptions with organizational expectations, address confidence disparities, and enhance collaborative efforts. Organizational investments are crucial for fostering a dynamic research ecosystem to improve evidence-based policy development in Kuwait's healthcare landscape.

Enabling cross-country learning and exchange to support universal health coverage implementation

As countries transition from external assistance while pursuing ambitious plans to achieve universal health coverage (UHC), there is increasing need to facilitate knowledge sharing and learning among them. Country-led and country-owned knowledge management is foundational to sustainable, more equitable external assistance for health and is a useful complement to more conventional capacity-building modalities provided under external assistance. In the context of external assistance, few initiatives use country-to-country sharing of practitioner experiences, and link learning to receiving guidance on how to adapt, apply and sustain policy changes. Dominant knowledge exchange processes are didactic, implicitly assuming static technical needs, and that practitioners in low- and middle-income countries require problem-specific, time-bound solutions. In reality, the technical challenges of achieving UHC and the group of policymakers involved continuously evolve. This paper aims to explore factors which are supportive of experience-based knowledge exchange between practitioners from diverse settings, drawing from the experience of the Joint Learning Network (JLN) for UHC-a global network of practitioners and policymakers sharing experiences about common challenges to develop and implement knowledge products supporting reforms for UHC-as an illustration of a peer-to-peer learning approach. This paper considers: (1) an analysis of JLN monitoring and evaluation data between 2020 and 2023 and (2) a qualitative inquiry to explore policymakers' engagement with the JLN using semi-structured interviews (n = 14) with stakeholders from 10 countries. The JLN's experience provides insights to factors that contribute to successful peer-to-peer learning approaches. JLN relies on engaging a network of practitioners with diverse experiences who organically identify and pursue a common learning agenda. Meaningful peer-to-peer learning requires dynamic, structured interactions, and alignment with windows of opportunity for implementation that enable rapid response to emerging and timely issues. Peer-to-peer learning can facilitate in-country knowledge sharing, learning and catalyse action at the institutional and health system levels.

Knowledge management tools and mechanisms for evidence-informed decision-making in the WHO European Region: a scoping review

BACKGROUND

Knowledge management (KM) emerged as a strategy to promote evidence-informed decision-making. This scoping review aims to map existing KM tools and mechanisms used to promote evidence-informed health decision-making in the WHO European Region and identify knowledge gaps.

METHODS

Following the Joanna Briggs Institute (JBI) guidance for conducting scoping reviews, we searched Medline, PubMed, EMBASE, the Cochrane library, and Open Grey. We conducted a descriptive analysis of the general characteristics of the included papers and conducted narrative analysis of the included studies and categorized studies according to KM type and phase.

RESULTS

Out of 9541 citations identified, we included 141 studies. The KM tools mostly assessed are evidence networks, surveillance tools, observatories, data platforms and registries, with most examining KM tools in high-income countries of the WHO European region. Findings suggest that KM tools can identify health problems, inform health planning and resource allocation, increase the use of evidence by policymakers and stimulate policy discussion.

CONCLUSION

Policymakers and funding agencies are called to support capacity-building activities, and future studies to strengthen KM in the WHO European region particularly in Eastern Europe and Central Asia. An updated over-arching strategy to coordinate KM activities in the WHO European region will be useful in these efforts.

Housing and Support for People with Disability: Perspectives of Motor Accident, Disability and Injury Insurers across Australia and New Zealand

Housing is a critical enabler of a dignified life, civic participation and the achievement of human rights. Providing appropriate housing for people who experience neurotrauma as a result of road or workplace injury, with both the assistive technology and human support required, continues however to be a policy and practice challenge. Australian and New Zealand motor accident, disability and injury insurers have high and enduring liability in this area, and their under-researched perspectives are needed to strengthen the evidence base for policy and practice development. This qualitative study utilised focus group methodology with representatives from government injury and disability insurers across Australia and New Zealand (n = 8). The study aimed to identify (a) issues and trends; (b) factors for decision making; and (c) service impacts relating to housing and support for people with disability and high daily support needs. Thematic analysis generated results across four key areas: influences on the decision to fund housing and/or support; identifying 'good' housing solutions; evaluating cost-benefit of housing and support investments; and developing future investment in housing and support. Findings such as those regarding decision-making, and investment, attest to the value of capturing the perspectives of this key group of stakeholders to assist to envision better housing and support for people with disability.

[Cross-cultural adaptation of "Is research working for you? A self-assessment tool" in Brazil]

The scope of this article is to translate and adapt the tool "Is research working for you? A self-assessment tool and discussion guide for health services management and policy organization," which self-assesses the institutional capacity of Health organizations to Acquire, Evaluate, Adapt and Apply scientific evidence in policy decision-making. It is a methodological, analytical study with a quantitative and qualitative approach. A sample of managers was used (n=99), representing 24 Brazilian health organizations, between October 2018 and November 2020. Proposed adaptations were made in a consensual way after adjustments of the disagreements of the expert committee - Kappa coefficient [0.41; 0.43 and 0.74]. Tool 4A, Brazilian version, had semantic, idiomatic, experimental, and conceptual equivalence with the original. The value of Cronbach's Alpha was 0.961; the value of the Intraclass Correlation Coefficient was 0.960. Tool 4A is valid and reliable in Brazil in terms of internal consistency and content. Its application is a catalyst for actions to structure institutional capacities to make the use of scientific evidence systematic in the decision-making processes of health organizations.

Mapping Chilean clinical research: a protocol for a scoping review and multiple evidence gap maps

INTRODUCTION

Clinical research broadly aims to influence decision-making in order to promote appropriate healthcare. Funding agencies should prioritise research projects according to needed research topics, methodological and cost-effectiveness considerations, and expected social value. In Chile, there is no local diagnosis regarding recent clinical research that might inform prioritisation for future research funding. This research aims to comprehensively identify and classify Chilean health research studies, elaborating evidence gap maps for the most burdensome local conditions.

METHODS AND ANALYSIS

We will search in electronic databases (MEDLINE, Embase, PsycINFO, CINAHL, LILACS and WoS) and perform hand searches to retrieve, identify and classify health research studies conducted in Chile or by authors whose affiliations are based in Chile, from 2000 onwards. We will elaborate evidence matrices for the 20 conditions with the highest burden in Chile (according to the Global Burden of Disease 2019) selected from those defined under the General Regime of the Health Guarantees Act. To elaborate the evidence gap maps, we will consider prioritised interventions and core outcome sets. To identify knowledge gaps and estimate redundant research, we will contrast these gap maps with the available international evidence of high or moderate certainty of evidence, for each specific clinical question. For this purpose, we will search systematic reviews using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.

ETHICS AND DISSEMINATION

No ethical approval is required to conduct this project. We will submit our results in both peer-reviewed journals and scientific conferences. We will aim to disseminate our findings through different academic platforms, social media, local press, among others. The final results will be communicated to local funding agencies and government stakeholders.

DISCUSSION

We aim to provide an accurate and up-to-date picture of the research gaps-to be filled by new future findings-and the identification of redundant research, which will constitute relevant information for local decision-makers.

Knowledge translation and evidence generation to increase the impact of vector control in Burkina Faso, Cameroon and Malawi

Lack of context-specific evidence and inadequate evidence-use for decision-making contribute to poor health. This paper reports on our work aimed at addressing the knowledge translation (KT) gap between evidence generators and users. We present our experiences of strengthening KT via technical advisory groups (TAGs) in parallel with increasing evidence generation through research fellowships and operational research. Vectorborne diseases (VBDs) impose substantial health and economic burdens in sub-Saharan Africa despite being preventable with vector control. The Partnership for Increasing the Impact of Vector Control aimed to reduce the burden of VBDs in Burkina Faso, Cameroon, Malawi and at regional and global levels. TAGs can promote evidence-use in policy and practice by engaging relevant stakeholders in both research and policy processes. TAGs and related activities are best facilitated by a coordinator with skills in research and policy. Contextual factors should influence the design and governance of TAGs, which will likely evolve over time. Relevant national stakeholders should be included in TAGs and be actively involved in developing research agendas to increase the relevance and acceptability of research findings for decision-making. The countries present three differing contexts with longer-term research and evaluation necessary to draw lessons on impact.

A comprehensive monitoring and evaluation framework for evidence to policy networks

OBJECTIVE

To describe the development of a framework for monitoring and evaluating knowledge translation (KT) networks.

METHOD

The framework was developed using mixed methods over four phases, including i) a targeted literature review of KT networks, activities and indicators, ii) two scoping reviews to further enhance the set of indicators, iii) peer-reviews by international KT experts and an online expert consultation, and iv) piloting.

RESULTS

A comprehensive theory of change (ToC) and indicators, both for the Network Secretariat and its participating member countries, were identified to develop the monitoring and evaluation framework. The framework includes (i) a ToC, including three key indicator domains across the results chain (outputs, short term outcomes, intermediate outcomes), and (ii) indicators for the three key domains, that can be selected depending on the stage of network maturity, along with suggested data collection methods. The three key indicator domains are 1) KT capacity and skill building; 2) network (structure, governance and leadership); and 3) KT/evidence-informed policy value and culture.

CONCLUSION

The monitoring and evaluation framework that links KT activities with policy and health outcomes fills an important gap in optimizing KT procedures, generating lessons learned and increasing accountability of major multipartner KT networks.

A framework of evidence-based decision-making in health system management: a best-fit framework synthesis

BACKGROUND

Scientific evidence is the basis for improving public health; decision-making without sufficient attention to evidence may lead to unpleasant consequences. Despite efforts to create comprehensive guidelines and models for evidence-based decision-making (EBDM), there isn`t any to make the best decisions concerning scarce resources and unlimited needs. The present study aimed to develop a comprehensive applied framework for EBDM.

METHODS

This was a Best-Fit Framework (BFF) synthesis conducted in 2020. A comprehensive systematic review was done via six main databases including PUBMED, Scopus, Web of Science, Science Direct, EMBASE, and ProQuest using related keywords. After the evidence quality appraisal, data were extracted and analyzed via thematic analysis. Results of the thematic analysis and the concepts generated by the research team were then synthesized to achieve the best-fit framework applying Carroll et al. (2013) approach.

RESULTS

Four thousand six hundred thirteen studies were retrieved, and due to the full-text screening of the studies, 17 final articles were selected for extracting the components and steps of EBDM in Health System Management (HSM). After collecting, synthesizing, and categorizing key information, the framework of EBDM in HSM was developed in the form of four general scopes. These comprised inquiring, inspecting, implementing, and integrating, which included 10 main steps and 47 sub-steps.

CONCLUSIONS

The present framework provided a comprehensive guideline that can be well adapted for implementing EBDM in health systems and related organizations especially in underdeveloped and developing countries where there is usually a lag in updating and applying evidence in their decision-making process. In addition, this framework by providing a complete, well-detailed, and the sequential process can be tested in the organizational decision-making process by developed countries to improve their EBDM cycle.

A framework for the promotion of ethical benefit sharing in health research

There is an increasing recognition of the importance of including benefit sharing in research programmes in order to ensure equitable and just distribution of the benefits arising from research. Whilst there are global efforts to promote benefit sharing when using non-human biological resources, benefit sharing plans and implementation do not yet feature prominently in research programmes, funding applications or requirements by ethics review boards. Whilst many research stakeholders may agree with the concept of benefit sharing, it can be difficult to operationalise benefit sharing within research programmes. We present a framework designed to assist with identifying benefit sharing opportunities in research programmes. The framework has two dimensions: the first represents microlevel, mesolevel and macrolevel stakeholders as defined using a socioecological model; and the second identifies nine different types of benefit sharing that might be achieved during a research programme. We provide an example matrix identifying different types of benefit sharing that might be undertaken during genomics research, and present a case study evaluating benefit sharing in Africa during the SARS-CoV-2 pandemic. This framework, with examples, is intended as a practical tool to assist research stakeholders with identifying opportunities for benefit sharing, and inculcating intentional benefit sharing in their research programmes from inception.

Integrating citizen engagement into evidence-informed health policy-making in eastern Europe and central Asia: scoping study and future research priorities

BACKGROUND

The perspectives of citizens are an important and often overlooked source of evidence for informing health policy. Despite growing encouragement for its adoption, little is known regarding how citizen engagement may be integrated into evidence-informed health policy-making in low- and middle-income counties (LMICs) and newly democratic states (NDSs). We aimed to identify the factors and variables affecting the potential integration of citizen engagement into evidence-informed health policy-making in LMICs and NDSs and understand whether its implementation may require a different approach outside of high-income western democracies. Further, we assessed the context-specific considerations for the practical implementation of citizen engagement in one focus region-eastern Europe and central Asia.

METHODS

First, adopting a scoping review methodology, we conducted and updated searches of six electronic databases, as well as a comprehensive grey literature search, on citizen engagement in LMICs and NDSs, published before December 2019. We extracted insights about the approaches to citizen engagement, as well as implementation considerations (facilitators and barriers) and additional political factors, in developing an analysis framework. Second, we undertook exploratory methods to identify relevant literature on the socio-political environment of the focus region, before subjecting these sources to the same analysis framework.

RESULTS

Our searches identified 479 unique sources, of which 28 were adjudged to be relevant. The effective integration of citizen engagement within policy-making processes in LMICs and NDSs was found to be predominantly dependent upon the willingness and capacity of citizens and policy-makers. In the focus region, the implementation of citizen engagement within evidence-informed health policy-making is constrained by a lack of mutual trust between citizens and policy-makers. This is exacerbated by inadequate incentives and capacity for either side to engage.

CONCLUSIONS

This research found no reason why citizen engagement could not adopt the same form in LMICs and NDSs as it does in high-income western democracies. However, it is recognized that certain political contexts may require additional support in developing and implementing citizen engagement, such as through trialling mechanisms at subnational scales. While specifically outlining the potential for citizen engagement, this study highlights the need for further research on its practical implementation.

Policy options to increase motivation for improving evidence-informed health policy-making in Iran

BACKGROUND

Current incentive programmes are not sufficient to motivate researchers and policy-makers to use research evidence in policy-making. We conducted a mixed-methods design to identify context-based policy options for strengthening motivations among health researchers and policy-makers to support evidence-informed health policy-making (EIHP) in Iran.

METHODS

This study was conducted in 2019 in two phases. In the first phase, we conducted a scoping review to extract interventions implemented or proposed to strengthen motivations to support EIHP. Additionally, we employed a comparative case study design for reviewing the performance evaluation (PE) processes in Iran and other selected countries to determine the current individual and organizational incentives to encourage EIHP. In the second phase, we developed two policy briefs and then convened two policy dialogues, with 12 and 8 key informants, respectively, where the briefs were discussed. Data were analysed using manifest content analysis in order to propose contextualized policy options.

RESULTS

The policy options identified to motivate health researchers and policy-makers to support EIHP in Iran were: revising the criteria of academic PE; designing appropriate incentive programmes for nonacademic researchers; developing an indicator for the evaluation of research impact on policy-making or health outcomes; revising the current policies of scientific journals; revising existing funding mechanisms; presenting the knowledge translation plan when submitting a research proposal, as a mandatory condition; encouraging and supporting mechanisms for increasing interactions between policy-makers and researchers; and revising some administrative processes (e.g. managers and staff PEs; selection, appointment, and changing managers and reward mechanisms).

CONCLUSIONS

The current individual or organizational incentives are mainly focused on publications, rather than encouraging researchers and policy-makers to support EIHP. Relying more on incentives that consider the other impacts of research (e.g. impacts on health system and policy, or health outcomes) is recommended. These incentives may encourage individuals and organizations to be more involved in conducting research evidence, resulting in promoting EIHP.

TRIAL REGISTRATION

NA.

[Institutionalizing evidence-based policies in BrazilInstitucionalización de las políticas informadas por evidencia]

OBJECTIVE

To evaluate capacities, organizational arrangements, and barriers to the implementation of Evidence Centers (NEvs) as part of Brazil's Evidence-Informed Policy Network (EVIPNet).

METHOD

A mixed methods descriptive-analytical, multiple-case exploratory study was performed. Coordinators of active NEvs answered a questionnaire in three parts: participant characteristics, assessment of the capacity to "acquire, assess, adapt, and apply" evidence (4A), and open questions addressing organizational arrangements and barriers to the implementation of NEvs.

RESULTS

The study included 15 NEvs, mostly from the Midwest; 73.3% were based in universities, while 20% were installed in state/city health departments or in the Ministry of Health. All coordinators had completed graduate training and 80% reported 1 to 5 years' experience with evidence-based policies as well as proficiency in English. None of the participants worked exclusively as NEv coordinator. NEv teams included health care professionals, students (undergraduate/graduate), professors, and civil servants from health departments. The data revealed high capacity to "acquire" and "assess" evidence, and low capacity to "adapt" and "apply" evidence. On average, three activities or products were developed yearly by each NEv, especially knowledge translation initiatives (systematic reviews and deliberative dialogues) and training for health care professionals, managers and undergraduate/graduate students. Five barrier categories were described: 1) financing, 2) network integration, 3) institutionalization of demands, 4) adaptive capacity, and 5) research communication skills to recommend actions at the local level.

CONCLUSIONS

Trained human resources associated with academic and research institutions are available to support evidence-informed policies. However, the sustainability of NEvs depends on coordinated action to ensure the capacity to adapt and apply evidence.