The implementation of embedded researchers in policy, public services, and commercial settings: a systematic evidence and gap map

BACKGROUND

Embedding researchers into policy and other settings may enhance research capacity within organisations to enable them to become more research active. We aimed to generate an evidence map on evaluations of embedded researcher interventions to (i) identify where systematic reviews and primary research are needed and (ii) develop conceptual understandings of 'embedded researchers'. We define 'embedded researchers' through a set of principles that incorporate elements such as the aim of activities, the types of relationships and learning involved, and the affiliations and identities adopted.

METHODS

We included studies published across all sectors, searching fourteen databases, other web sources and two journals for evaluations published between 1991 and spring 2021. Data were extracted using a coding tool developed for this study. We identified new typologies of embedded researcher interventions through undertaking Latent Class Analysis.

RESULTS

The map describes 229 evaluations spanning a variety of contexts. Our set of principles allowed us to move beyond a narrow focus on embedded researchers in name alone, towards consideration of the wide range of roles, activities, identities, and affiliations related to embedded researchers. We identified 108 different allied terms describing an embedded researcher. Embedded researcher activity spanned a continuum across lines of physical, cultural, institutional, and procedural embeddedness (from weaker to more intense forms of embeddedness) and took a range of forms that bridge or blur boundaries between academia and policy/practice.

CONCLUSIONS

We developed a broad map of international embedded researcher activity in a wide range of sectors. The map suggests that embedded researcher interventions occupy a broader suite of models than previously acknowledged and our findings also offer insight on the type and nature of this literature. Given the clear policy interest in this area, a better understanding of the processes involved with becoming embedded within an organisation is needed. Further work is also necessary to address the challenges of evaluating the work of embedded researchers, including consideration for which outcome measures are most appropriate, to better understand their influence.

Production and use of rapid responses during the COVID-19 pandemic in Quebec (Canada): perspectives from evidence synthesis producers and decision makers

BACKGROUND

The COVID-19 pandemic has required evidence to be made available more rapidly than usual, in order to meet the needs of decision makers in a timely manner. These exceptional circumstances have caused significant challenges for organizations and teams responsible for evidence synthesis. They had to adapt to provide rapid responses to support decision-making. This study aimed to document (1) the challenges and adaptations made to produce rapid responses during the pandemic, (2) their perceived usefulness, reported use and factors influencing their use and (3) the methodological adaptations made to produce rapid responses.

METHODS

A qualitative study was conducted in 2021 with eight organizations in the health and social services system in Quebec (Canada), including three institutes with a provincial mandate. Data collection included focus groups (n = 9 groups in 8 organizations with 64 participants), interviews with decision makers (n = 12), and a document analysis of COVID-19 rapid responses (n = 128). A thematic analysis of qualitative data (objectives 1 and 2) and a descriptive analysis of documents (objective 3) were conducted.

RESULTS

The results highlight the teams and organizations' agility to deal with the many challenges encountered during the pandemic (e.g., increased their workloads, adoption of new technological tools or work processes, improved collaboration, development of scientific monitoring, adaptation of evidence synthesis methodologies and products). The challenge of balancing rigor and speed was reported by teams and organizations. When available at the right time, rapid responses have been reported as a useful tool for informing or justifying decisions in a context of uncertainty. Several factors that may influence their use were identified (e.g., clearly identify needs, interactions with producers, perceived rigor and credibility, precise and feasible recommendations). Certain trends in the methodological approaches used to speed up the evidence synthesis process were identified.

CONCLUSIONS

This study documented rapid responses producers' experiences during the COVID-19 pandemic in Quebec, and decision makers who requested, consulted, or used these products. Potential areas of improvements are identified such as reinforce coordination, improve communication loops, clarify guidelines or methodological benchmarks, and enhance utility of rapid response products for decision makers.

Supporting National Immunization Technical Advisory Groups (NITAGs) in development of evidence-based vaccine recommendations and NITAG assessments - New tools and approaches

Increasing opportunities for prevention of infectious diseases by new, effective vaccines and the expansion of global immunization programs across the life course highlight the importance and value of evidence-informed decision-making (EIDM) by National Immunization Technical Advisory Groups (NITAGs). The U.S. Centers for Disease Control and Prevention (CDC) and Task Force for Global Health (TFGH) have developed and made available new tools to support NITAGs in EIDM. These include a toolkit for conducting facilitated training of NITAGs, Secretariats, or work groups on the use of the Evidence to Recommendations (EtR) approach to advise Ministries of Health (MoH) on specific vaccine policies, and an eLearning module on the EtR approach for NITAG members, Secretariat and others. The CDC and TFGH have also supported final development and implementation of the NITAG Maturity Assessment Tool (NMAT) for assessing maturity of NITAG capabilities in seven functional domains. The EtR toolkit and eLearning have been widely promoted in collaboration with the World Health Organization (WHO) Headquarters and Regional Offices through workshops engaging over 30 countries to date, and the NMAT assessment tool used in most countries in 3 WHO regions (Americas, Eastern Mediterranean, African). Important lessons have been learned regarding planning and conducting trainings for multiple countries and additional ways to support countries in applying the EtR approach to complete vaccine recommendations. Priorities for future work include the need to evaluate the impact of EtR training and NMAT assessments, working with partners to expand and adapt these tools for wider use, synergizing with other approaches for NITAG strengthening, and developing the best approaches to empower NITAGs to use the EtR approach.

Assessing the impact of an evidence- and consensus-based guideline for controlling SARS-CoV-2 transmission in German schools on decision-making processes: a multi-component qualitative analysis

INTRODUCTION

During the COVID-19 pandemic, decision-making on measures to reduce or prevent transmission of SARS-CoV-2 in schools was rendered difficult by a rapidly evolving and uncertain evidence base regarding their effectiveness and unintended consequences. To support decision-makers, an interdisciplinary panel of scientific experts, public health and school authorities as well as those directly affected by school measures, was convened in an unprecedented effort to develop an evidence- and consensus-based public health guideline for German schools. This study sought to assess whether and how this guideline impacted decision-making processes.

METHODS

This study comprised three components: (1) we sent inquiries according to the Freedom of Information Acts of each Federal State to ministries of education, family, and health. (2) We conducted semi-structured interviews with individuals involved in decision-making regarding school measures in two Federal States, and (3) we undertook semi-structured interviews with members of the guideline panel. The content of response letters in component 1 was analysed descriptively; data for components 2 and 3 were analysed using deductive-inductive thematic qualitative content analysis according to Kuckartz.

RESULTS

Responses to the Freedom of Information Act inquiries showed that the guideline was recognised as a relevant source of information by ministries of education in nine out of 16 Federal States and used as a reference to check existing directives for school measures in five Federal States. All participants (20 interviews) emphasised the value of the guideline given its evidence- and consensus-based development process but also noted limitations in its usability and usefulness, e.g., lack of context-specificity. It was consulted by participants who advised policy-makers (5 interviews) alongside other sources of evidence. Overall, perceptions regarding the guideline's impact were mixed.

CONCLUSIONS

Our findings suggest that the guideline was relatively well-known in Federal States' decision-making bodies and that it was considered alongside other forms of evidence in some of these. We suggest that further research to evaluate the impact of public health guidelines on (political) decision-making is warranted. Guideline development processes may need to be adapted to account for the realities of decision-making during public health emergencies and beyond.

Knowledge management tools and mechanisms for evidence-informed decision-making in the WHO European Region: a scoping review

BACKGROUND

Knowledge management (KM) emerged as a strategy to promote evidence-informed decision-making. This scoping review aims to map existing KM tools and mechanisms used to promote evidence-informed health decision-making in the WHO European Region and identify knowledge gaps.

METHODS

Following the Joanna Briggs Institute (JBI) guidance for conducting scoping reviews, we searched Medline, PubMed, EMBASE, the Cochrane library, and Open Grey. We conducted a descriptive analysis of the general characteristics of the included papers and conducted narrative analysis of the included studies and categorized studies according to KM type and phase.

RESULTS

Out of 9541 citations identified, we included 141 studies. The KM tools mostly assessed are evidence networks, surveillance tools, observatories, data platforms and registries, with most examining KM tools in high-income countries of the WHO European region. Findings suggest that KM tools can identify health problems, inform health planning and resource allocation, increase the use of evidence by policymakers and stimulate policy discussion.

CONCLUSION

Policymakers and funding agencies are called to support capacity-building activities, and future studies to strengthen KM in the WHO European region particularly in Eastern Europe and Central Asia. An updated over-arching strategy to coordinate KM activities in the WHO European region will be useful in these efforts.

An evidence-informed policymaking (EIPM) competency profile for the Brazilian Health System developed through consensus: process and outcomes

BACKGROUND

Evidence-informed policymaking (EIPM) requires a set of individual and organizational knowledge, skills and attitudes that should be articulated with background factors and needs. In this regard, the development of an EIPM competency profile is important to support the diagnosis, planning and implementation of EIPM.

PURPOSE

To present the process and outcomes of the development of an EIPM competency profile by an expert committee, to be applied in different contexts of the Brazilian Health System.

METHODS

A committee of experts in EIPM shared different views, experiences and opinions to develop an EIPM competency profile for Brazil. In six consensus workshops mediated by facilitators, the committee defined from macro problems to key actions and performances essential for the competency profile. The development steps consisted of: (1) Constitution of the committee, including researchers, professionals with practical experience, managers, and educators; (2) Development of a rapid review on EIPM competency profiles; (3) Agreement on commitments and responsibilities in the processes; (4) Identification and definition of macro problems relating to the scope of the competency profile; and (5) Outlining of general and specific capacities, to be incorporated into the competency profile, categorized by key actions.

RESULTS

The development of the EIPM competency profile was guided by the following macro problems: (1) lack of systematic and transparent decision-making processes in health policy management; (2) underdeveloped institutional capacity for knowledge management and translation; and (3) incipient use of scientific evidence in the formulation and implementation of health policies. A general framework of key actions and performances of the EIPM Competency Profile for Brazil was developed, including 42 specific and general key actions distributed by area of activity (Health Management, Scientific Research, Civil Society, Knowledge Translation, and Cross-sectional areas).

CONCLUSIONS

The competency profile presented in this article can be used in different contexts as a key tool for the institutionalization of EIPM.

Anxiety increased among children and adolescents during pandemic-related school closures in Europe: a systematic review and meta-analysis

BACKGROUND

Considering the heterogenous evidence, a systematic review of the change in anxiety in European children and adolescents associated with the COVID-19 pandemic is lacking. We therefore assessed the change compared with pre-pandemic baselines stratified by gender and age as well as evaluated the impact of country-specific restriction policies.

METHODS

A registration on the 'International Prospective Register of Systematic Reviews' (PROSPERO) occurred and an a priori protocol was published. We searched six databases (PubMed, Embase, PsycINFO, Cochrane Central Register of Controlled Trials, Web of Science, WHO COVID-19) using a peer-reviewed search string with citation tracking and grey literature screening. Primary outcomes were: (1) general anxiety symptoms; and (2) clinically relevant anxiety rates. We used the Oxford COVID-19 Stringency Index as an indicator of pandemic-related restrictions. Screening of title/abstract and full text as well as assessing risk of bias (using the 'Risk of Bias in Non-randomized Studies of Exposure' [ROBINS-E]) and certainty of evidence (using the 'Grading of Recommendations Assessment, Development and Evaluation' [GRADE]) was done in duplicate. We pooled data using a random effects model. Reporting is in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) statement.

RESULTS

Of 7,422 non-duplicate records, 18 studies with data from 752,532 pre-pandemic and 763,582 pandemic participants met full inclusion criteria. For general anxiety symptoms the total change effect estimate yielded a standardised mean difference (SMD) of 0.34 (95% confidence interval [CI], 0.17-0.51) and for clinically relevant anxiety rates we observed an odds ratio of 1.08 (95%-CI, 0.98-1.19). Increase in general anxiety symptoms was highest in the 11-15 years age group. Effect estimates were higher when pandemic-related restrictions were more stringent (Oxford Stringency Index > 60: SMD, 0.52 [95%-CI, 0.30-0.73]) and when school closures (School Closure Index ≥ 2: SMD, 0.44 [95%-CI, 0.23-0.65]) occurred.

CONCLUSION

General anxiety symptoms among children and adolescents in Europe increased in a pre/during comparison of the COVID-19 pandemic; particularly for males aged 11-15 years. In periods of stringent pandemic-related restrictions and/or school closures a considerable increase in general anxiety symptoms could be documented. PROSPERO registration: CRD42022303714.

Decision-makers' experiences with rapid evidence summaries to support real-time evidence informed decision-making in crises: a mixed methods study

BACKGROUND

There is a clear need for research evidence to drive policymaking and emergency responses so that lives are saved and resources are not wasted. The need for evidence support for health and humanitarian crisis is even more pertinent because of the time and practical constraints that decision-makers in these settings face. To improve the use of research evidence in policy and practice, it is important to provide evidence resources tailored to the target audience. This study aims to gain real-world insights from decision-makers about how they use evidence summaries to inform real-time decision-making in crisis-settings, and to use our findings to improve the format of evidence summaries.

METHODS

This study used an explanatory sequential mixed method study design. First, we used a survey to identify the views and experiences of those who were directly involved in crisis response in different contexts, and who may or may not have used evidence summaries. Second, we used the insights generated from the survey to help inform qualitative interviews with decision-makers in crisis-settings to derive an in-depth understanding of how they use evidence summaries and their desired format for evidence summaries.

RESULTS

We interviewed 26 decision-makers working in health and humanitarian emergencies. The study identified challenges decision-makers face when trying to find and use research evidence in crises, including insufficient time and increased burden of responsibilities during crises, limited access to reliable internet connection, large volume of data not translated into user friendly summaries, and little information available on preparedness and response measures. Decision-makers preferred the following components in evidence summaries: title, target audience, presentation of key findings in an actionable checklist or infographic format, implementation considerations, assessment of the quality of evidence presented, citation and hyperlink to the full review, funding sources, language of full review, and other sources of information on the topic. Our study developed an evidence summary template with accompanying training material to inform real-time decision-making in crisis-settings.

CONCLUSIONS

Our study provided a deeper understanding of the preferences of decision-makers working in health and humanitarian emergencies about the format of evidence summaries to enable real-time evidence informed decision-making.

Competency profiles for evidence-informed policy-making (EIPM): a rapid review

BACKGROUND

Evidence-informed policy-making (EIPM) requires a set of individual and organizational capacities, linked with background factors and needs. The identification of essential knowledge, skills and attitudes for EIPM can support the development of competency profiles and their application in different contexts.

PURPOSE

To identify elements of competency (knowledge, skills and attitudes) for EIPM, according to different professional profiles (researcher, health professional, decision-maker and citizen).

METHODS

Rapid umbrella review. A structured search was conducted and later updated in two comprehensive repositories (BVSalud and PubMed). Review studies with distinctive designs were included, published from 2010 onwards, without language restrictions. Assessment of the methodological quality of the studies was not performed. A meta-aggregative narrative synthesis was used to report the findings.

RESULTS

Ten reviews were included. A total of 37 elements of competency were identified, eight were categorized as knowledge, 19 as skills and 10 as attitudes. These elements were aggregated into four competency profiles: researcher, health professional, decision-maker and citizen. The competency profiles included different sets of EIPM-related knowledge, skills and attitudes.

STRENGTHS AND LIMITATIONS

This study is innovative because it aggregates different profiles of competency from a practical perspective, favouring the application of its results in different contexts to support EIPM. Methodological limitations are related to the shortcuts adopted in this review: complementary searches of the grey literature were not performed, and the study selection and data extraction were not conducted in duplicate.

FINAL CONSIDERATIONS

CONCLUSIONS AND IMPLICATIONS OF THE FINDINGS: EIPM requires the development of individual and organizational capacities. This rapid review contributes to the discussion on the institutionalization of EIPM in health systems. The competency profiles presented here can support discussions about the availability of capacity and the need for its development in different contexts.

Improving institutional platforms for evidence-informed decision-making: getting beyond technical solutions

Purely technical interventions aimed at enhancing evidence-informed decision-making (EIDM) have rarely translated into organizational institutionalization or systems change. A panel of four presentations at the Health Systems Global 2020 conference provides a basis for inference about contextual factors that influence the establishment and sustainability of institutional platforms to support EIDM. These cases include local structures such as citizen panels in Uganda, regional knowledge translation structures such as the West African Health Organization, global multilateral initiatives such as the "One Health" Quadrapartite and regional public health networks in South-East Asia. They point to the importance of political economy as well as technical capability determinants of evidence uptake and utilization at institutional, organizational and individual levels. The cases also lend support to evidence that third-party (broker and intermediary) supportive institutions can facilitate EIDM processes. The involvement of third-party supranational organizations, however, poses challenges in terms of legitimacy and accountability.

Evidence attack in public health: Diverse actors' experiences with translating controversial or misrepresented evidence in health policy and systems research

Bringing evidence into policy and practice discussions is political; more so when evidence from health studies or programme data are deemed controversial or unexpected, or when results are manipulated and misrepresented. Furthermore, opinion and misinformation in recent years has challenged our notions about how to achieve evidence-informed decision-making (EIDM). Health policy and systems (HPS) researchers and practitioners are battling misrepresentation that only serves to detract from important health issues or, worse, benefit powerful interests. This paper describes cases of politically and socially controversial evidence presented by researchers, practitioners and journalists during the Health Systems Research Symposium 2020. These cases cut across global contexts and range from public debates on vaccination, comprehensive sexual education, and tobacco to more inward debates around performance-based financing and EIDM in refugee policy. The consequences of engaging in controversial research include threats to commercial profit, perceived assaults on moral beliefs, censorship, fear of reprisal, and infodemics. Consequences for public health include research(er) hesitancy, contribution to corruption and leakage, researcher reflexivity, and ethical concerns within the HPS research and EIDM fields. Recommendations for supporting researchers, practitioners and advocates include better training and support structures for responding to controversy, safe spaces for sharing experiences, and modifying incentive structures.

Lessons from bright-spots for advancing knowledge exchange at the interface of marine science and policy

Evidence-informed decision-making is in increasing demand given growing pressures on marine environments. A way to facilitate this is by knowledge exchange among marine scientists and decision-makers. While many barriers are reported in the literature, there are also examples whereby research has successfully informed marine decision-making (i.e., 'bright-spots'). Here, we identify and analyze 25 bright-spots from a wide range of marine fields, contexts, and locations to provide insights into how to improve knowledge exchange at the interface of marine science and policy. Through qualitative surveys we investigate what initiated the bright-spots, their goals, and approaches to knowledge exchange. We also seek to identify what outcomes/impacts have been achieved, the enablers of success, and what lessons can be learnt to guide future knowledge exchange efforts. Results show that a diversity of approaches were used for knowledge exchange, from consultative engagement to genuine knowledge co-production. We show that diverse successes at the interface of marine science and policy are achievable and include impacts on policy, people, and governance. Such successes were enabled by factors related to the actors, processes, support, context, and timing. For example, the importance of involving diverse actors and managing positive relationships is a key lesson for success. However, enabling routine success will require: 1) transforming the ways in which we train scientists to include a greater focus on interpersonal skills, 2) institutionalizing and supporting knowledge exchange activities in organizational agendas, 3) conceptualizing and implementing broader research impact metrics, and 4) transforming funding mechanisms to focus on need-based interventions, impact planning, and an acknowledgement of the required time and effort that underpin knowledge exchange activities.

University-Based Researchers as Knowledge Brokers for Climate Policies and Action

Responding effectively to climate crisis requires strong science-policy links to be put in place. Past research on the research-policy interface indicates longstanding challenges that have become more acute in the case of climate science, since this requires multi-disciplinary approaches and faces distinctive political challenges in linking knowledge with policy. What can be learned from the experiences of university-based researchers seeking to influence policy as they try to operate in the brokering space? With this in mind, an empirical study was designed to capture the detailed views and experiences of forty researchers in four universities across four countries-Bangladesh, Germany, Uganda and UK. It found a wide range of different researcher attitudes to policy engagement, diverse methods of engaging, a preference for working with government and civil society over private sector policy actors, and a perceived need for more university support. The findings suggest a need to rethink conditions for engagement to create spaces for knowledge exchange and cooperation that can contribute to policies for societal transformation. More attention also needs to be paid to interdisciplinary research approaches, improving research connections with private sector actors, and strengthening university research links with local communities. Finally, the position of university based researchers in the Global South will require strengthening to improve North-South knowledge exchange, capacity development, and incentives for policy engagement.

Strategies to promote evidence use for health programme improvement: learning from the experiences of embedded implementation research teams in Latin America and the Caribbean

Background

To achieve global health targets, innovative approaches are needed to strengthen the implementation of efficacious interventions. New approaches in implementation research that bring together health system decision-makers alongside researchers to collaboratively design, produce and apply research evidence are gaining traction. Embedded implementation research (EIR) approaches led by decision-maker principal investigators (DM PIs) appear promising in this regard. Our aim is to describe the strategies study teams employ in the post-research phase of EIR to promote evidence-informed programme or policy improvement.

Methods

We conducted a prospective, comparative case study of an EIR initiative in Bolivia, Colombia and Dominican Republic. Guided by a conceptual framework on EIR, we used semi-structured key informant interviews (n = 51) and document reviews (n = 20) to examine three decision-maker-led study teams (“cases”). Focusing on three processes (communication/dissemination, stakeholder engagement with evidence, integrating evidence in decision-making) and the main outcome (enacting improvements), we used thematic analysis to identify associated strategies and enabling or hindering factors.

Results

Across cases, we observed diverse strategies, shaped substantially by whether the DM PI was positioned to lead the response to study findings within their sphere of work. We found two primary change pathways: (1) DM PIs implement remedial measures directly, and (2) DM PIs seek to influence other stakeholders to respond to study findings. Throughout the post-research phase, EIR teams adapted research use strategies based on the evolving context.

Conclusions

EIR led by well-positioned DM PIs can facilitate impactful research translation efforts. We draw lessons around the importance of (1) understanding DM PI positionality, (2) ongoing assessment of the evolving context and stakeholders and (3) iterative adaptation to dynamic, uncertain circumstances. Findings may guide EIR practitioners in planning and conducting fit-for-purpose and context-sensitive strategies to advance the use of evidence for programme improvement.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00834-1.

Development of a knowledge broker group to support evidence-informed policy: lessons learned from Myanmar

Background

Globally, policy-makers face challenges to using evidence in health decision-making, particularly lack of interaction between research and policy. Knowledge-brokering mechanisms can fill research–policy gaps and facilitate evidence-informed policy-making. In Myanmar, the need to promote evidence-informed policy is significant, and thus a mechanism was set up for this purpose. This paper discusses lessons learned from the development of the Knowledge Broker Group–Myanmar (KBG-M), supported by the Johns Hopkins Bloomberg School of Public Health’s Applied Mental Health Research Group (JHU) and Community Partners International (CPI).

Methods

Sixteen stakeholders were interviewed to explore challenges in formulating evidence-informed policy. Two workshops were held: the first to further understand the needs of policy-makers and discuss knowledge-brokering approaches, and the second to co-create the KBG-M structure and process. The KBG-M was then envisioned as an independent body, with former officials of the Ministry of Health and Sports (MoHS) and representatives from the nongovernmental sector actively engaging in the health sector, with an official collaboration with the MoHS.

Results

A development task force that served as an advisory committee was established. Then, steps were taken to establish the KBG-M and obtain official recognition from the MoHS. Finally, when the technical agreement with the MoHS was nearly complete, the process stopped because of the military coup on 1 February 2021, and is now on hold indefinitely.

Conclusions

Learning from this process may be helpful for future or current knowledge-brokering efforts, particularly in fragile, conflict-affected settings. Experienced and committed advisory committee members enhanced stakeholder relationships. Responsive coordination mechanisms allowed for adjustments to a changing bureaucratic landscape. Coordination with similar initiatives avoided overlap and identified areas needing technical support. Recommendations to continue the work of the KBG-M itself or similar platforms include the following: increase resilience to contextual changes by ensuring diverse partnerships, maintain advisory committee members experienced and influential in the policy-making process, ensure strong organizational and funding support for effective functioning and sustainability, have budget and timeline flexibility to allow sufficient time and resources for establishment, organize ongoing needs assessments to identify areas needing technical support and to develop responsive corrective approaches, and conduct information sharing and collaboration between stakeholders to ensure alignment.

Methods to support evidence-informed decision-making in the midst of COVID-19: creation and evolution of a rapid review service from the National Collaborating Centre for Methods and Tools

Background

The COVID-19 public health crisis has produced an immense and quickly evolving body of evidence. This research speed and volume, along with variability in quality, could overwhelm public health decision-makers striving to make timely decisions based on the best available evidence. In response to this challenge, the National Collaborating Centre for Methods and Tools developed a Rapid Evidence Service, building on internationally accepted rapid review methodologies, to address priority COVID-19 public health questions.

Results

Each week, the Rapid Evidence Service team receives requests from public health decision-makers, prioritizes questions received, and frames the prioritized topics into searchable questions. We develop and conduct a comprehensive search strategy and critically appraise all relevant evidence using validated tools. We synthesize the findings into a final report that includes key messages, with a rating of the certainty of the evidence using GRADE, as well as an overview of evidence and remaining knowledge gaps. Rapid reviews are typically completed and disseminated within two weeks. From May 2020 to July 21, 2021, we have answered more than 31 distinct questions and completed 32 updates as new evidence emerged. Reviews receive an average of 213 downloads per week, with some reaching over 7700. To date reviews have been accessed and cited around the world, and a more fulsome evaluation of impact on decision-making is planned.

Conclusions

The development, evolution, and lessons learned from our process, presented here, provides a real-world example of how review-level evidence can be made available – rapidly and rigorously, and in response to decision-makers’ needs – during an unprecedented public health crisis.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01436-1.

Health economic evidence in clinical guidelines in South Africa: a mixed-methods study

BACKGROUND

Evidence-informed clinical practice guidelines (CPGs) are useful tools to inform transparent healthcare decision-making. Consideration of health economic evidence (HEE) during CPG development in a structured manner remains a challenge globally and locally. This study explored the views, current practice, training needs and challenges faced by CPG developers in the production and use of HEE for CPGs in South Africa.

METHODS

This mixed-methods study comprised an online survey and a focus group discussion. The survey was piloted and subsequently sent to CPG role players - evidence reviewers, CPG panellists, academics involved with training in relevant disciplines like health economics and public health, implementers and funders. The focus group participants hold strategic roles in CPG development and health economic activities nationally. The survey evaluated mean values, measures of variability, and percentages for Likert scales, while narrative components were thematically analysed. Focus group data were manually coded, thematically analysed and verified.

RESULTS

The survey (n = 55 respondents to 245 surveys distributed) and one focus group (n = 5 participants from 10 people invited) occurred between October 2018 and February 2019. We found the most consistent reason why HEE should inform CPG decisions was 'making more efficient use of limited financial resources'. This was explained by numerous context and methodological barriers. Focus groups participants noted that consideration of complex HEE are not achievable without bolstering skills in applying evidence-based medicine principles. Further concerns include lack of clarity of standard methods; inequitable and opaque topic selection across private and public sectors; inadequate skills of CPG panel members to use HEE; and the ability of health economists to communicate results in accessible ways. Overall, in the absence of clarity about process and methods, politics and interests may drive CPG decisions about which interventions to implement.

CONCLUSIONS

HEE should ideally be considered in CPG decisions in South Africa. However, this will remain hampered until the CPG community agree on methods and processes for using HEE in CPGs. Focused investment by national government to address the challenges identified by the study is imperative for a better return on investment as National Health Insurance moves forward.

Integrated knowledge translation to advance noncommunicable disease policy and practice in South Africa: application of the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework

Background

In response to the “know–do” gap, several initiatives have been implemented to enhance evidence-informed decision-making (EIDM). These include individual training, organizational culture change management, and legislative changes. The importance of relationships and stakeholder engagement in EIDM has led to an evolution of models and approaches including integrated knowledge translation (IKT). IKT has emerged as a key strategy for ensuring that engagement is equitable, demand-driven, and responsive. As a result, the African-German Collaboration for Evidence-Based Healthcare and Public Health in Africa (CEBHA+) incorporated an IKT approach to influence noncommunicable diseases (NCD) policy and practice. We documented the phased process of developing, implementing, and monitoring the IKT approach in South Africa; and explored the appropriateness of using the exploration, preparation, implementation, and sustainment (EPIS) framework for this purpose.

Methods

We mapped the South Africa IKT approach onto the EPIS framework using a framework analysis approach. Notes of team meetings, stakeholder matrices, and engagement strategies were analysed and purposefully plotted against the four phases of the framework in order to populate the different constructs. We discussed and finalized the analysis in a series of online iterations until consensus was reached.

Results

The mapping exercise revealed an IKT approach that was much more iterative, dynamic, and engaging than initially thought. Several constructs (phase-agnostic) remained important and stable across EPIS phases: stable and supportive funding; committed and competent leadership; skilled and dedicated IKT champions; diverse and established personal networks; a conducive and enabling policy environment; and boundary-spanning intermediaries. Constructs such as “innovations” constantly evolved and adapted to the changing inner and outer contexts (phase-specific).

Conclusions

Using the EPIS framework to interrogate, reflect on, and document our IKT experiences proved extremely relevant and useful. Phase-agnostic constructs proved critical to ensure resilience and agility of NCD deliberations and policies in the face of highly dynamic and changing local contexts, particularly in view of the current coronavirus disease 2019 (COVID-19) pandemic. Bridging IKT with a framework from implementation science helps to reflect on this process and can guide the development and planning of similar interventions and strategies.

Understanding the decision to immunize: insights into the information needs and priorities of people who have utilized an online human papillomavirus (HPV) vaccine decision aid tool

Setting

People in Alberta are more likely to seek information about cancer prevention online than they are to have this conversation with their primary care provider. As people turn to the internet to support health decision-making, it is critical that we improve the supportiveness of the virtual health setting for cancer prevention.

Intervention

In 2014, the Alberta Cancer Prevention Legacy Fund launched an online HPV Decision Aid Tool to support evidence-informed decision-making in response to suboptimal uptake of the HPV (human papillomavirus) vaccine. Google Analytics data from approximately 2000 recent interactions with this tool have yielded insights into the concerns that impact people’s decision-making about the vaccine.

Outcomes

Most users of this tool are adults interested in the vaccine for themselves (69%), rather than parents considering immunizing their children (31%). No differences were found in the information-seeking behaviour of parents of girls compared with parents of boys, suggesting that mental models among those who are considering the HPV vaccine may have shifted in recent years. Concerns differed by respondent; cost was the most important concern among adults (62.0%), while parents were most concerned about vaccine safety (61.5%). Only 23% of users asked “what is HPV”, suggesting that many people in Alberta now have basic knowledge about the virus.

Implications

Results provide a real-time “pulse” on knowledge and attitudes towards HPV immunization, which informs our approach to tailoring messaging with the aim of increasing vaccine uptake in Alberta. Outcomes will provide evidence needed to inform new interventions aimed at increasing HPV immunization rates.

Intelligent Tools for Precision Public Health

The idea of "precision public health" (PPH) was proposed as an alternative to a one-size-fits-all approach to improving population health, which is not always effective. PPH aims to develop and apply interventions in a customized way, taking into account the detailed information about the target group. To enable the implementation of PPH in practice, we are developing an ontology-driven software platform that provides: a) access to detailed up-to-date information about population health, b) a structured machine-readable repository of evidence about public health interventions, and c) a set of intelligent tools to facilitate the assessment of evidence transferability, i.e. to determine how well certain interventions fit a given population.

Measures of evidence-informed decision-making competence attributes: a psychometric systematic review

Background

The current state of evidence regarding measures that assess evidence-informed decision-making (EIDM) competence attributes (i.e., knowledge, skills, attitudes/beliefs, behaviours) among nurses is unknown. This systematic review provides a narrative synthesis of the psychometric properties and general characteristics of EIDM competence attribute measures in nursing.

Methods

The search strategy included online databases, hand searches, grey literature, and content experts. To align with the Cochrane Handbook of Systematic Reviews, psychometric outcome data (i.e., acceptability, reliability, validity) were extracted in duplicate, while all remaining data (i.e., study and measure characteristics) were extracted by one team member and checked by a second member for accuracy. Acceptability data was defined as measure completion time and overall rate of missing data. The Standards for Educational and Psychological Testing was used as the guiding framework to define reliability, and validity evidence, identified as a unified concept comprised of four validity sources: content, response process, internal structure and relationships to other variables. A narrative synthesis of measure and study characteristics, and psychometric outcomes is presented across measures and settings.

Results

A total of 5883 citations were screened with 103 studies and 35 unique measures included in the review. Measures were used or tested in acute care (n = 31 measures), public health (n = 4 measures), home health (n = 4 measures), and long-term care (n = 1 measure). Half of the measures assessed a single competence attribute (n = 19; 54.3%). Three measures (9%) assessed four competence attributes of knowledge, skills, attitudes/beliefs and behaviours. Regarding acceptability, overall missing data ranged from 1.6–25.6% across 11 measures and completion times ranged from 5 to 25 min (n = 4 measures). Internal consistency reliability was commonly reported (21 measures), with Cronbach’s alphas ranging from 0.45–0.98. Two measures reported four sources of validity evidence, and over half (n = 19; 54%) reported one source of validity evidence.

Conclusions

This review highlights a gap in the testing and use of competence attribute measures related to evidence-informed decision making in community-based and long-term care settings. Further development of measures is needed conceptually and psychometrically, as most measures assess only a single competence attribute, and lack assessment and evidence of reliability and sources of established validity evidence.

Registration

PROSPERO #CRD42018088754.

Experts' perceptions on the use of visual analytics for complex mental healthcare planning: an exploratory study

BACKGROUND

Health experts including planners and policy-makers face complex decisions in diverse and constantly changing healthcare systems. Visual analytics may play a critical role in supporting analysis of complex healthcare data and decision-making. The purpose of this study was to examine the real-world experience that experts in mental healthcare planning have with visual analytics tools, investigate how well current visualisation techniques meet their needs, and suggest priorities for the future development of visual analytics tools of practical benefit to mental healthcare policy and decision-making.

METHODS

Health expert experience was assessed by an online exploratory survey consisting of a mix of multiple choice and open-ended questions. Health experts were sampled from an international pool of policy-makers, health agency directors, and researchers with extensive and direct experience of using visual analytics tools for complex mental healthcare systems planning. We invited them to the survey, and the experts' responses were analysed using statistical and text mining approaches.

RESULTS

The forty respondents who took part in the study recognised the complexity of healthcare systems data, but had most experience with and preference for relatively simple and familiar visualisations such as bar charts, scatter plots, and geographical maps. Sixty-five percent rated visual analytics as important to their field for evidence-informed decision-making processes. Fifty-five percent indicated that more advanced visual analytics tools were needed for their data analysis, and 67.5% stated their willingness to learn new tools. This was reflected in text mining and qualitative synthesis of open-ended responses.

CONCLUSIONS

This exploratory research provides readers with the first self-report insight into expert experience with visual analytics in mental healthcare systems research and policy. In spite of the awareness of their importance for complex healthcare planning, the majority of experts use simple, readily available visualisation tools. We conclude that co-creation and co-development strategies will be required to support advanced visual analytics tools and skills, which will become essential in the future of healthcare.

Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about treatment effects: one-year follow up of a randomised trial

INTRODUCTION

Earlier, we designed and evaluated an educational mass media intervention for improving people's ability to think more critically and to assess the trustworthiness of claims (assertions) about the benefits and harms (effects) of treatments. The overall aims of this follow-up study were to evaluate the impact of our intervention 1 year after it was administered, and to assess retention of learning and behaviour regarding claims about treatments.

METHODS

We randomly allocated consenting parents to listen to either the Informed Health Choices podcast (intervention) or typical public service announcements about health issues (control) over 7-10 weeks. Each intervention episode explained how the trustworthiness of treatment claims can be assessed by using relevant key concepts of evidence-informed decision-making. Participants listened to two episodes per week, delivered by research assistants. We evaluated outcomes immediately, and a year after the intervention. Primary outcomes were mean score and the proportion with a score indicating a basic ability to apply the key concepts (> 11 out of 18 correct answers) on a tool measuring people's ability to critically appraise the trustworthiness of treatment claims. Skills decay/retention was estimated by calculating the relative difference between the follow-up and initial results in the intervention group, adjusting for chance. Statistical analyses were performed using R (R Core Team, Vienna, Austria; version 3.4.3).

RESULTS

After 1 year, the mean score for parents in the intervention group was 58.9% correct answers, compared to 52.6% in the control (adjusted mean difference of 6.7% (95% CI 3.3% to 10.1%)). In the intervention group, 47.2% of 267 parents had a score indicating a basic ability to assess treatment claims compared to 39.5% of 256 parents in the control (adjusted difference of 9.8% more parents (95% CI 0.9% to 18.9%). These represent relative reductions of 29% in the mean scores and 33% in the proportion of parents with a score indicating a basic ability to assess the trustworthiness of claims about treatment effects.

CONCLUSIONS

Although listening to the Informed Health Choices podcast initially led to a large improvement in the ability of parents to assess claims about the effects of treatments, our findings show that these skills decreased substantially over 1 year. More active practice could address the substantial skills decay observed over 1 year.

TRIAL REGISTRATION

Pan African Clinical Trial Registry (www.pactr.org), PACTR201606001676150. Registered on 12 June 2016.

A description of a tailored knowledge translation intervention delivered by knowledge brokers within public health departments in Canada

Background

While there is an expectation to demonstrate evidence-informed public health there is an ongoing need for capacity development. The purpose of this paper is to provide a description of a tailored knowledge translation intervention implemented by knowledge brokers (KBs), and reflections on the factors that facilitated or hindered its implementation.

Methods

The 22-month knowledge translation intervention, implemented by two KBs, sought to facilitate evidence-informed public health decision-making. Data on outcomes were collected using a knowledge, skills and behavioural assessment survey. In addition, the KBs maintained reflective journals noting which activities appeared successful or not, as well as factors related to the individual or the organisation that facilitated or hindered evidence-informed decision-making.

Results

Tailoring of the knowledge translation intervention to address the needs, preferences and structure of each organisation resulted in three unique interventions being implemented. A consistent finding across organisations was that each site needed to determine where evidence-informed decision-making ‘fit’ within pre-existing organisational processes. Components of the intervention consistent across the three organisations included one-to-one mentoring of teams through rapid evidence reviews, large group workshops and regular meetings with senior management. Components that varied included the frequency of the KB being physically onsite, the amount of time staff spent with the KB and proportion of time spent one-to-one with a KB versus in workshops. Key facilitating factors for implementation included strong leadership, influential power of champions, supportive infrastructure, committed resources and staff enthusiasm.

Conclusions

The results of this study illustrate the importance of working collaboratively with organisations to tailor knowledge translation interventions to best meet unique needs, preferences, organisational structures and contexts. Organisational factors such as leadership, champions and supportive infrastructure play a key role in determining the impact of the knowledge translation interventions. Future studies should explore how these factors can be fostered and/or developed within organisations. While KBs implemented the knowledge translation intervention in this study, more research is needed to understand the impact of all change agent roles including KBs, as well as how these roles can be maintained in the long-term if proven effective.

Understanding evidence-informed decision-making: a rural interorganizational breastfeeding network

Background

Networks are a vehicle for mobilizing knowledge, but there is little research about evidence-informed decision-making in community settings. Breastfeeding is a powerful intervention for population health; combined system and community interventions can increase exclusive breastfeeding rates by 2.5 times. This study examined evidence-informed decision-making within an interorganizational network, including the facilitators and barriers to achieving network goals.

Methods

A mixed method case study design was used. The primary sources of data were focus group discussion and questionnaire administration. Data were analyzed concurrently using framework analysis and social network analysis.

Results

Key findings were at the interorganizational and external levels: 1) Relationships and trust are connected to knowledge exchange 2) Need for multiple levels of leadership.

Conclusions

The findings of this study have potential implications for enhancing the use of evidence-informed decision-making as other networks work toward Baby Friendly Initiative (BFI) designation and also highlights the potential for network maps to be used as a knowledge mobilization tool.

Who can you trust? A review of free online sources of "trustworthy" information about treatment effects for patients and the public

BACKGROUND

Information about effects of treatments based on unsystematic reviews of research evidence may be misleading. However, finding trustworthy information about the effects of treatments based on systematic reviews, which is accessible to patients and the public can be difficult. The objectives of this study were to identify and evaluate free sources of health information for patients and the public that provide information about effects of treatments based on systematic reviews.

METHODS

We reviewed websites that we and our colleagues knew of, searched for government sponsored health information websites, and searched for online sources of health information that provide evidence-based information. To be included in our review, a website had to be available in English, freely accessible, and intended for patients and the public. In addition, it had to have a broad scope, not limited to specific conditions or types of treatments. It had to include a description of how the information is prepared and the description had to include a statement about using systematic reviews. We compared the included websites by searching for information about the effects of eight treatments.

RESULTS

Three websites met our inclusion criteria: Cochrane Evidence, Informed Health, and PubMed Health. The first two websites produce content, whereas PubMed Health aggregated content. A fourth website that met our inclusion criteria, CureFacts, was under development. Cochrane Evidence provides plain language summaries of Cochrane Reviews (i.e. summaries that are intended for patients and the public). They are translated to several other languages. No information besides treatment effects is provided. Informed Health provides information about treatment effects together with other information for a wide range of topics. PubMed Health was discontinued in October 2018. It included a large number of systematic reviews of treatment effects with plain language summaries for Cochrane Reviews and some other reviews. None of the three websites included links to ongoing trials, and information about treatment effects was not reported consistently on any of the websites.

CONCLUSION

It is possible for patients and the public to access trustworthy information about the effects of treatments using the two of the websites included in this review.

Lessons learned from Evidence-Informed Decision-Making in Nutrition & Health (EVIDENT) in Africa: a project evaluation

Background

Evidence-informed Decision-making in Nutrition and Health (EVIDENT) is an international partnership that seeks to identify information needs in nutrition and health in Africa and build local capacity in knowledge management to help translate the best available evidence into context-appropriate recommendations aligned to the priorities of decision-makers. This study evaluates the extent to which EVIDENT achieved its intended activities, documents the lessons learned and draws on these lessons learned to inform future activities of EVIDENT, as well as in evidence-informed decision-making (EIDM) in nutrition overall.

Methods

Purposive and snowball sampling were used to identify participants that were either directly or indirectly involved with EVIDENT. An analytical framework of five key elements was developed to guide data collection from EVIDENT’s documentation, in-depth interviews (n = 20), online surveys (n = 26) and a participatory discussion. Interviews were transcribed verbatim and coded in NVivo 11, using deductive thematic content analysis and a phenomenological approach. Online surveys were analysed using Stata 14. Data were triangulated to address both objectives under each element of the analytical framework.

Results

EVIDENT succeeded in establishing a collaborative partnership, within which it delivered four short courses in EIDM. This capacity complemented case study activities in four partner African countries where EIDM processes were implemented and assessed. Identified barriers to these processes included little experience in EIDM, difficulties in engaging stakeholders, challenging local environments (e.g. donor influence, bureaucracy, inaccessibility to scientific research, poor internet connectivity), and limited time and funding. However, EVIDENT activities were driven by a local need for EIDM, a sheer interest and commitment to the cause, and the opportunity for the Global North and South to work together and build relationships. Future activities of EVIDENT, and EIDM in nutrition overall, should focus on sustained capacity-building in EIDM processes, leadership and functional skills across the Global South, investment in stakeholder engagement, context-specific EIDM, enhanced communication and linking, and strengthening relationships with existing stakeholder organisations.

Conclusions

In its first 3 years, EVIDENT developed and strengthened partnership, capacity and visibility on EIDM in Africa. Innovative and long-term capacity-building, dedicated leadership, further stakeholder engagement and sustainable financing, are needed for future activities of EVIDENT and EIDM in nutrition.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0413-6) contains supplementary material, which is available to authorized users.

Advancing the evaluation of integrated knowledge translation

Background

Integrated knowledge translation (IKT) flows from the premise that knowledge co-produced with decision-makers is more likely to inform subsequent decisions. However, evaluations of manager/policy-maker-focused IKT often concentrate on intermediate outcomes, stopping short of assessing whether research findings have contributed to identifiable organisational action. Such hesitancy may reflect the difficulty of tracing the causes of this distal, multifactorial outcome. This paper elucidates how an approach based on realistic evaluation could advance the field.

Main Text

Realistic evaluation views outcomes as a joint product of intervention mechanisms and context. Through identification of context–mechanism–outcome configurations, it enables the systematic testing and refinement of ‘mid-range theory’ applicable to diverse interventions that share a similar underlying logic of action. The ‘context-sensitive causal chain’ diagram, a tool adapted from the broader theory-based evaluation literature, offers a useful means of visualising the posited chain from activities to outcomes via mechanisms, and the context factors that facilitate or disrupt each linkage (e.g. activity–mechanism, mechanism–outcome).

Drawing on relevant literature, this paper proposes a context-sensitive causal chain by which IKT may generate instrumental use of research findings (i.e. direct use to make a concrete decision) and identifies an existing tool to assess this outcome, then adapts the chain to describe a more subtle, indirect pathway of influence. Key mechanisms include capacity- and relationship-building among researchers and decision-makers, changes in the (perceived) credibility and usability of findings, changes in decision-makers’ beliefs and attitudes, and incorporation of new knowledge in an actual decision. Project-specific context factors may impinge upon each linkage; equally important is the organisation’s absorptive capacity, namely its overall ability to acquire, assimilate and apply knowledge. Given a sufficiently poor decision-making environment, even well-implemented IKT that triggers important mechanisms may fall short of its desired outcomes. Further research may identify additional mechanisms and context factors.

Conclusion

By investigating ‘what it is about an intervention that works, for whom, under what conditions’, realistic evaluation addresses questions of causality head-on without sacrificing complexity. A realist approach could contribute greatly to our ability to assess – and, ultimately, to increase – the value of IKT.

Measures assessing attributes of evidence-informed decision-making (EIDM) competence among nurses: a systematic review protocol

BACKGROUND

There are growing professional expectations for nurses to engage in and develop competence in evidence-informed decision-making (EIDM) due to opportunities for improved client and community outcomes and provision of the highest quality of care. However, EIDM is underdeveloped, with low implementation rates among nurses. The use of indicators to assess EIDM performance has potential to encourage nurses' engagement in EIDM through competence recognition and support assessment of strengths and competency gaps for individual nurses and organizations. Currently, the state of evidence regarding measures that assess EIDM competence attributes (i.e., knowledge, skills, beliefs/values, behaviors) among nurses is unknown. This systematic review aims to address this gap through a narrative synthesis of the characteristics and psychometric properties of EIDM competence measures.

METHODS

The search strategy, developed in consultation with a Health Sciences Librarian, consists of online databases, contacting experts, hand searching reference lists, key journals, websites, conference proceedings, and grey literature. Studies will be included if the following criteria are met: (1) sample includes practicing nurses and data for nurses are reported separately; (2) conducted in any healthcare setting; (3) quantitative or mixed-methods design; (4) reports use or testing of a measure assessing EIDM competence attributes (i.e., knowledge, skills, attitudes/values, and/or behaviors); and (5) published in English. Screening will be conducted independently by two reviewers using a two-stage process: (1) title and abstract level; and (2) full-text level. Data extraction of study characteristics (e.g., sample, setting) will be conducted by a single reviewer and checked for accuracy by a second reviewer. Psychometric properties of acceptability, reliability, and validity evidence for each measure will be independently extracted by two reviewers. Data on measures will be synthesized narratively according to acceptability, number of validity evidence sources established, and reliability of scores. Data pertaining to population and healthcare setting will also be reported for each measure.

DISCUSSION

This systematic review will provide a current understanding about the state of evidence with respect to EIDM competence measures in nursing to assist in determining potentially relevant and robust measures for use in different nursing practice settings.

SYSTEMATIC REVIEW REGISTRATION

Protocol registered in PROSPERO Registration #: CRD42018088754.

Factors affecting engagement between academic faculty and decision-makers: learnings and priorities for a school of public health

Background

Schools of public health (SPHs) are increasingly being recognised as important contributors of human, social and intellectual capital relevant to health policy and decision-making. Few studies within the implementation science literature have systematically examined knowledge exchange experiences within this specific organisational context. The purpose of this study was therefore to elicit whether documented facilitators and barriers to engaging with government decision-makers resonates within an academic SPH context. We sought to understand the variations in such experiences at four different levels of government decision-making. Furthermore, we sought to elicit intervention priorities as identified by faculty.

Methods

Between May and December 2016, 211 (34%) of 627 eligible full-time faculty across one SPH in the United States of America participated in a survey on engagement with decision-makers at the city, state, federal and global government levels. Surveys were administered face-to-face or via Skype. Descriptive data as well as tests of association and logistic regression analyses were conducted using STATA.

Results

Over three-quarters of respondents identified colleagues with ties to decision-makers, institutional affiliation and conducting policy-relevant research as the highest facilitators. Several identified time constraints, academic incentives and financial support as important contributors to engagement. Faculty characteristics, such as research areas of expertise, career track and faculty rank, were found to be statistically significantly associated with facilitators. The top three intervention priorities that emerged were (1) creating incentives for engagement, (2) providing funding for engagement and (3) inculcating an institutional culture around engagement.

Conclusions

The data suggest that five principal categories of factors – individual characteristics, institutional environment, relational dynamics, research focus and funder policies – affect the willingness and ability of academic faculty to engage with government decision-makers. This study suggests that SPHs could enhance the relevance of their role in health policy decision-making by (1) periodically measuring engagement with decision-makers; (2) enhancing individual capacity in knowledge translation and communication, taking faculty characteristics into account; (3) institutionalising a culture that supports policies and practices for engagement in decision-making processes; and (4) creating a strategy to expand and nurture trusted, relevant networks and relationships with decision-makers.

Implementation of evidence-based weekend service recommendations for allied health managers: a cluster randomised controlled trial protocol

BACKGROUND

It is widely acknowledged that health policy and practice do not always reflect current research evidence. Whether knowledge transfer from research to practice is more successful when specific implementation approaches are used remains unclear. A model to assist engagement of allied health managers and clinicians with research implementation could involve disseminating evidence-based policy recommendations, along with the use of knowledge brokers. We developed such a model to aid decision-making for the provision of weekend allied health services. This protocol outlines the design and methods for a multi-centre cluster randomised controlled trial to evaluate the success of research implementation strategies to promote evidence-informed weekend allied health resource allocation decisions, especially in hospital managers.

METHODS

This multi-centre study will be a three-group parallel cluster randomised controlled trial. Allied health managers from Australian and New Zealand hospitals will be randomised to receive either (1) an evidence-based policy recommendation document to guide weekend allied health resource allocation decisions, (2) the same policy recommendation document with support from a knowledge broker to help implement weekend allied health policy recommendations, or (3) a usual practice control group. The primary outcome will be alignment of weekend allied health service provision with policy recommendations. This will be measured by the number of allied health service events (occasions of service) occurring on weekends as a proportion of total allied health service events for the relevant hospital wards at baseline and 12-month follow-up.

DISCUSSION

Evidence-based policy recommendation documents communicate key research findings in an accessible format. This comparatively low-cost research implementation strategy could be combined with using a knowledge broker to work collaboratively with decision-makers to promote knowledge transfer. The results will assist managers to make decisions on resource allocation, based on evidence. More generally, the findings will inform the development of an allied health model for translating research into practice.

TRIAL REGISTRATION

This trial is registered with the Australian New Zealand Clinical Trials Registry (ANZCTR) ( ACTRN12618000029291 ). Universal Trial Number (UTN): U1111-1205-2621.

Improving interpretation of publically reported statistics on health and healthcare: the Figure Interpretation Assessment Tool (FIAT-Health)

BACKGROUND

Policy-makers, managers, scientists, patients and the general public are confronted daily with figures on health and healthcare through public reporting in newspapers, webpages and press releases. However, information on the key characteristics of these figures necessary for their correct interpretation is often not adequately communicated, which can lead to misinterpretation and misinformed decision-making. The objective of this research was to map the key characteristics relevant to the interpretation of figures on health and healthcare, and to develop a Figure Interpretation Assessment Tool-Health (FIAT-Health) through which figures on health and healthcare can be systematically assessed, allowing for a better interpretation of these figures.

METHODS

The abovementioned key characteristics of figures on health and healthcare were identified through systematic expert consultations in the Netherlands on four topic categories of figures, namely morbidity, healthcare expenditure, healthcare outcomes and lifestyle. The identified characteristics were used as a frame for the development of the FIAT-Health. Development of the tool and its content was supported and validated through regular review by a sounding board of potential users.

RESULTS

Identified characteristics relevant for the interpretation of figures in the four categories relate to the figures' origin, credibility, expression, subject matter, population and geographical focus, time period, and underlying data collection methods. The characteristics were translated into a set of 13 dichotomous and 4-point Likert scale questions constituting the FIAT-Health, and two final assessment statements. Users of the FIAT-Health were provided with a summary overview of their answers to support a final assessment of the correctness of a figure and the appropriateness of its reporting.

CONCLUSIONS

FIAT-Health can support policy-makers, managers, scientists, patients and the general public to systematically assess the quality of publicly reported figures on health and healthcare. It also has the potential to support the producers of health and healthcare data in clearly communicating their data to different audiences. Future research should focus on the further validation of the tool in practice.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 8: developing, implementing and evaluating an evidence dissemination service in a local healthcare setting

BACKGROUND

This is the eighth in a series of papers reporting Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. The SHARE Program was a systematic, integrated, evidence-based program for disinvestment within a large Australian health service. One of the aims was to explore methods to deliver existing high quality synthesised evidence directly to decision-makers to drive decision-making proactively. An Evidence Dissemination Service (EDS) was proposed. While this was conceived as a method to identify disinvestment opportunities, it became clear that it could also be a way to review all practices for consistency with current evidence. This paper reports the development, implementation and evaluation of two models of an in-house EDS.

METHODS

Frameworks for development of complex interventions, implementation of evidence-based change, and evaluation and explication of processes and outcomes were adapted and/or applied. Mixed methods including a literature review, surveys, interviews, workshops, audits, document analysis and action research were used to capture barriers, enablers and local needs; identify effective strategies; develop and refine proposals; ascertain feedback and measure outcomes.

RESULTS

Methods to identify, capture, classify, store, repackage, disseminate and facilitate use of synthesised research evidence were investigated. In Model 1, emails containing links to multiple publications were sent to all self-selected participants who were asked to determine whether they were the relevant decision-maker for any of the topics presented, whether change was required, and to take the relevant action. This voluntary framework did not achieve the aim of ensuring practice was consistent with current evidence. In Model 2, the need for change was established prior to dissemination, then a summary of the evidence was sent to the decision-maker responsible for practice in the relevant area who was required to take appropriate action and report the outcome. This mandatory governance framework was successful. The factors influencing decisions, processes and outcomes were identified.

CONCLUSION

An in-house EDS holds promise as a method of identifying disinvestment opportunities and/or reviewing local practice for consistency with current evidence. The resource-intensive nature of delivery of the EDS is a potential barrier. The findings from this study will inform further exploration.

Implementation of evidence-informed practice through central network actors; a case study of three public health units in Canada

BACKGROUND

Workforce development is an important aspect of evidence-informed decision making (EIDM) interventions. The social position of individuals in formal and informal social networks, and the relevance of formal roles in relation to EIDM are important factors identifying key EIDM players in public health organizations. We assessed the role of central actors in information sharing networks in promoting the adoption of EIDM by the staff of three public health units in Canada, over a two-year period during which an organization-wide intervention was implemented.

METHODS

A multi-faceted and tailored intervention to train select staff applying research evidence in practice was implemented in three public health units in Canada from 2011 to 2013. Staff (n = 572) were asked to identify those in the health unit whom they turned to get help using research in practice, whom they considered as experts in EIDM, and friends. We developed multi-level linear regression models to predict the change in EIDM behavior scores predicted by being connected to peers who were central in networks and were engaged in the intervention.

RESULTS

Only the group of highly engaged central actors who were connected to each other, and the staff who were not engaged in the intervention but were connected to highly engaged central actors significantly improved their EIDM behavior scores. Among the latter group, the staff who were also friends with their information sources showed a larger improvement in EIDM behavior.

CONCLUSIONS

If engaged, central network actors use their formal and informal connections to promote EIDM. Central actors themselves are more likely to adopt EIDM if they communicate with each other. These social communications should be reinforced and supported through the implementation of training interventions as a means to promoting EIDM.

What are the best methodologies for rapid reviews of the research evidence for evidence-informed decision making in health policy and practice: a rapid review

Background

Rapid reviews have the potential to overcome a key barrier to the use of research evidence in decision making, namely that of the lack of timely and relevant research. This rapid review of systematic reviews and primary studies sought to answer the question: What are the best methodologies to enable a rapid review of research evidence for evidence-informed decision making in health policy and practice?

Methods

This rapid review utilised systematic review methods and was conducted according to a pre-defined protocol including clear inclusion criteria (PROSPERO registration: CRD42015015998). A comprehensive search strategy was used, including published and grey literature, written in English, French, Portuguese or Spanish, from 2004 onwards. Eleven databases and two websites were searched. Two review authors independently applied the eligibility criteria. Data extraction was done by one reviewer and checked by a second. The methodological quality of included studies was assessed independently by two reviewers. A narrative summary of the results is presented.

Results

Five systematic reviews and one randomised controlled trial (RCT) that investigated methodologies for rapid reviews met the inclusion criteria. None of the systematic reviews were of sufficient quality to allow firm conclusions to be made. Thus, the findings need to be treated with caution. There is no agreed definition of rapid reviews in the literature and no agreed methodology for conducting rapid reviews. While a wide range of ‘shortcuts’ are used to make rapid reviews faster than a full systematic review, the included studies found little empirical evidence of their impact on the conclusions of either rapid or systematic reviews. There is some evidence from the included RCT (that had a low risk of bias) that rapid reviews may improve clarity and accessibility of research evidence for decision makers.

Conclusions

Greater care needs to be taken in improving the transparency of the methods used in rapid review products. There is no evidence available to suggest that rapid reviews should not be done or that they are misleading in any way. We offer an improved definition of rapid reviews to guide future research as well as clearer guidance for policy and practice.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-016-0155-7) contains supplementary material, which is available to authorized users.

Moving knowledge about family violence into public health policy and practice: a mixed method study of a deliberative dialogue

BACKGROUND

There is a need to understand scientific evidence in light of the context within which it will be used. Deliberative dialogues are a promising strategy that can be used to meet this evidence interpretation challenge.

METHODS

We evaluated a deliberative dialogue held by a transnational violence prevention network. The deliberative dialogue included researchers and knowledge user partners of the Preventing Violence Across the Lifespan (PreVAiL) Research Network and was incorporated into a biennial full-team meeting. The dialogue included pre- and post-meeting activities, as well as deliberations embedded within the meeting agenda. The deliberations included a preparatory plenary session, small group sessions and a synthesizing plenary. The challenge addressed through the process was how to mobilize research to orient health and social service systems to prevent family violence and its consequences. The deliberations focused on the challenge, potential solutions for addressing it and implementation factors. Using a mixed-methods approach, data were collected via questionnaires, meeting minutes, dialogue documents and follow-up telephone interviews.

RESULTS

Forty-four individuals (all known to each other and from diverse professional roles, settings and countries) participated in the deliberative dialogue. Ten of the 12 features of the deliberative dialogue were rated favourably by all respondents. The mean behavioural intention score was 5.7 on a scale from 1 (strongly disagree) to 7 (strongly agree), suggesting that many participants intended to use what they learned in their future decision-making. Interviews provided further insight into what might be done to facilitate the use of research in the violence prevention arena.

CONCLUSION

Findings suggest that participants will use dialogue learnings to influence practice and policy change. Deliberative dialogues may be a viable strategy for collaborative sensemaking of research related to family violence prevention, and other public health topics.

Do academic knowledge brokers exist? Using social network analysis to explore academic research-to-policy networks from six schools of public health in Kenya

The potential for academic research institutions to facilitate knowledge exchange and influence evidence-informed decision-making has been gaining ground. Schools of public health (SPHs) may play a key knowledge brokering role—serving as agencies of and for development. Understanding academic-policymaker networks can facilitate the enhancement of links between policymakers and academic faculty at SPHs, as well as assist in identifying academic knowledge brokers (KBs). Using a census approach, we administered a sociometric survey to academic faculty across six SPHs in Kenya to construct academic-policymaker networks. We identified academic KBs using social network analysis (SNA) in a two-step approach: First, we ranked individuals based on (1) number of policymakers in their network; (2) number of academic peers who report seeking them out for advice on knowledge translation and (3) their network position as ‘inter-group connectors’. Second, we triangulated the three scores and re-ranked individuals. Academic faculty scoring within the top decile across all three measures were classified as KBs. Results indicate that each SPH commands a variety of unique as well as overlapping relationships with national ministries in Kenya. Of 124 full-time faculty, we identified 7 KBs in 4 of the 6 SPHs. Those scoring high on the first measure were not necessarily the same individuals scoring high on the second. KBs were also situated in a wide range along the ‘connector/betweenness’ measure. We propose that a composite score rather than traditional ‘betweenness centrality’, provides an alternative means of identifying KBs within these networks. In conclusion, SNA is a valuable tool for identifying academic-policymaker networks in Kenya. More efforts to conduct similar network studies would permit SPH leadership to identify existing linkages between faculty and policymakers, shared linkages with other SPHs and gaps so as to contribute to evidence-informed health policies.

Is dengue a disease of poverty? A systematic review

Policy prescriptions for combating dengue fever tend to focus on addressing environmental and social conditions of poverty. However, while poverty has long been considered a determinant of dengue, the research evidence for such a relationship is not well established. Results of a systematic review of the research literature designed to identify and assess the current state of the empirical evidence for the dengue-poverty link reveal a mixed story. Of 260 peer-reviewed articles referencing dengue-poverty relationships, only 12 English-language studies empirically assessed these relationships. Our analysis covering various social and economic conditions of poverty showed no clear associations with dengue rates. While nine of the 12 studies demonstrated some positive associations between measures of dengue and poverty (measured inconsistently through income, education, structural housing condition, overcrowding, and socioeconomic status), nine also presented null results and five with negative results. Of the five studies relating to access to water and sanitation, four reported null associations. Income and physical housing conditions were more consistently correlated with dengue outcomes than other poverty indicators. The small size of this sample, and the heterogeneity of measures and scales used to capture conditions of poverty, make it difficult to assess the strength and consistency of associations between various poverty indicators and dengue outcomes. At present, the global body of eligible English-language peer-reviewed literature investigating dengue-poverty relationships is too small to support a definitive relationship. We conclude that more research, particularly using standardized measures of both outcomes and indicators, is needed to support evidence-informed policies and approaches.