What factors affect evidence-informed policymaking in public health? Protocol for a systematic review of qualitative evidence using thematic synthesis

Strategies, facilitators, and barriers to interaction between health researchers and policy makers: Protocol for a systematic review

This systematic review protocol is developed with the objective to identify the strategies, facilitators, and barriers to interaction between researchers and policy makers to use research evidence in health policy making. It seems that review of interactive methods between researchers and policy makers can help to understand the role of researchers on evidence-informed policy making. Moreover, identifying barriers and facilitators of these interactions can help universities and institutions associated to health policy making in planning to improve the interaction between researchers and policy makers to facilitate evidence-informed policy making.

Mapping the Qualitative Evidence Base on the Use of Research Evidence in Health Policy-Making: A Systematic Review

BACKGROUND

The use of research evidence in health policy-making is a popular line of inquiry for scholars of public health and policy studies, with qualitative methods constituting the dominant strategy in this area. Research on this subject has been criticized for, among other things, disproportionately focusing on high-income countries; overemphasizing 'barriers and facilitators' related to evidence use to the neglect of other, less descriptive concerns; relying on descriptive, rather than in-depth explanatory designs; and failing to draw on insights from political/policy studies theories and concepts. We aimed to comprehensively map the global, peer-reviewed qualitative literature on the use of research evidence in health policy-making and to provide a descriptive overview of the geographic, temporal, methodological, and theoretical characteristics of this body of literature.

METHODS

We conducted a systematic review following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We searched nine electronic databases, hand-searched 11 health- and policy-related journals, and systematically scanned the reference lists of included studies and previous reviews. No language, date or geographic limitations were imposed.

RESULTS

The review identified 319 qualitative studies on a diverse array of topics related to the use of evidence in health policy-making, spanning 72 countries and published over a nearly 40 year period. A majority of these studies were conducted in high-income countries, but a growing proportion of the research output in this area is now coming from low- and middle-income countries, especially from sub-Saharan Africa. While over half of all studies did not use an identifiable theory or framework, and only one fifth of studies used a theory or conceptual framework drawn from policy studies or political science, we found some evidence that theory-driven and explanatory (eg, comparative case study) designs are becoming more common in this literature. Investigations of the barriers and facilitators related to evidence use constitute a large proportion but by no means a majority of the work in this area.

CONCLUSION

This review provides a bird's eye mapping of the peer reviewed qualitative research on evidence-to-policy processes, and has identified key features of - and gaps within - this body of literature that will hopefully inform, and improve, research in this area moving forward.

A comprehensive monitoring and evaluation framework for evidence to policy networks

OBJECTIVE

To describe the development of a framework for monitoring and evaluating knowledge translation (KT) networks.

METHOD

The framework was developed using mixed methods over four phases, including i) a targeted literature review of KT networks, activities and indicators, ii) two scoping reviews to further enhance the set of indicators, iii) peer-reviews by international KT experts and an online expert consultation, and iv) piloting.

RESULTS

A comprehensive theory of change (ToC) and indicators, both for the Network Secretariat and its participating member countries, were identified to develop the monitoring and evaluation framework. The framework includes (i) a ToC, including three key indicator domains across the results chain (outputs, short term outcomes, intermediate outcomes), and (ii) indicators for the three key domains, that can be selected depending on the stage of network maturity, along with suggested data collection methods. The three key indicator domains are 1) KT capacity and skill building; 2) network (structure, governance and leadership); and 3) KT/evidence-informed policy value and culture.

CONCLUSION

The monitoring and evaluation framework that links KT activities with policy and health outcomes fills an important gap in optimizing KT procedures, generating lessons learned and increasing accountability of major multipartner KT networks.

Experts prioritize osteoarthritis non-surgical interventions from Cochrane systematic reviews for translation into "Evidence4Equity" summaries

OBJECTIVE

Osteoarthritis generates substantial health and socioeconomic burden, which is particularly marked in marginalized groups. It is imperative that practitioners have ready access to summaries of evidence-based interventions for osteoarthritis that incorporate equity considerations. Summaries of systematic reviews can provide this. The present study surveyed experts to prioritize a selection ofinterventions, from which equity focused summaries will be generated. Specifically, the prioritized interventions will be developed into Cochrane Evidence4Equity (E4E) summaries.

METHODS

Twenty-seven systematic reviews of OA interventions were found. From these, twenty-nine non-surgical treatments for osteoarthritis were identified, based on statistically significant findings for desired outcome variables or adverse events. Key findings from these studies were summarised and provided to 9 experts in the field of osteoarthritis.. Expert participants were asked to rate interventions based on feasibility, health system effects, universality, impact on inequities, and priority for translation into equity based E4E summaries. Expert participants were also encouraged to make comments to provide context for each rating. Free text responses were coded inductively and grouped into subthemes and themes.

RESULTS

Expert participants rated the intervention home land-based exercise for knee OA highest for priority for translation into an E4E summaries, followed by the interventions individual land-based exercise for knee OA, class land-based exercise for knee OA, exercise for hand OA and land-based exercise for hip OA. Upon qualitative analysis of the expert participants' comments, fifteen subthemes were identified and grouped into three overall themes: (1) this intervention or an aspect of this intervention is unnecessary or unsafe; (2) this intervention or an aspect of this intervention may increase health inequities; and (3) experts noted difficulties completing rating exercise.

CONCLUSION

The list of priority interventions and corresponding expert commentary generated information that will be used to direct and support knowledge translation efforts.

The research burden of randomized controlled trial participation: a systematic thematic synthesis of qualitative evidence

BACKGROUND

Participation in randomized controlled trials (RCTs) may be quite demanding and could represent an important burden for patients. We aimed to explore this research burden (i.e., the psychological, physical, and financial burdens) experienced by patients through their participation in a RCT.

METHODS

We conducted a systematic review of qualitative studies exploring adult patients' experiences with RCT participation. We searched MEDLINE (PubMed), CINAHL, PSYCHINFO, and Embase (search date March 2018) for eligible reports. Qualitative data coding and indexing were assisted by NVivo. The quality of reports was assessed by using the Critical Appraisal Skills Program (CASP) tool.

RESULTS

We included 45 qualitative studies that involved 1732 RCT participants. Important psychological burdens were identified at every stage of the trial process. Participants reported feeling anxiety and being afraid of "being a 'guinea pig'" and described undergoing randomization and allocation to a placebo as particularly difficult resulting in disappointment, anger, and depression. Patients' follow-up and trial closure were also responsible for a wide range of psychological, physical, and financial burdens. Furthermore, factors related to burdensome impacts and consequences were discerned. These factors involved trial information, poorly organized and too-demanding follow-up, and lack of appropriate management when the patient's participation ended. Trial participation was also associated with beneficial effects such as the satisfaction of feeling "useful," gaining "a sense of control," and receiving special attention.

CONCLUSIONS

Our finding provides a detailed description of research burden across the whole RCT process. Many of the burdens described could be anticipated, and some avoided in a movement toward minimally disruptive clinical research. Such an approach could improve trial recruitment and retention.

REVIEW REGISTRATION

PROSPERO CRD42018098994.

Using narratives to impact health policy-making: a systematic review

BACKGROUND

There is increased interest in using narratives or storytelling to influence health policies. We aimed to systematically review the evidence on the use of narratives to impact the health policy-making process.

METHODS

Eligible study designs included randomised studies, non-randomised studies, process evaluation studies, economic studies, qualitative studies, stakeholder analyses, policy analyses, and case studies. The MEDLINE, PsycINFO, Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), WHO Global Health Library, Communication and Mass Media Complete, and Google Scholar databases were searched. We followed standard systematic review methodology for study selection, data abstraction and risk of bias assessment. We synthesised the findings narratively and presented the results stratified according to the following stages of the policy cycle: (1) agenda-setting, (2) policy formulation, (3) policy adoption, (4) policy implementation and (5) policy evaluation. Additionally, we presented the knowledge gaps relevant to using narrative to impact health policy-making.

RESULTS

Eighteen studies met the eligibility criteria, and included case studies (n = 15), participatory action research (n = 1), documentary analysis (n = 1) and biographical method (n = 1). The majority were of very low methodological quality. In addition, none of the studies formally evaluated the effectiveness of the narrative-based interventions. Findings suggest that narratives may have a positive influence when used as inspiration and empowerment tools to stimulate policy inquiries, as educational and awareness tools to initiate policy discussions and gain public support, and as advocacy and lobbying tools to formulate, adopt or implement policy. There is also evidence of undesirable effects of using narratives. In one case study, narrative use led to widespread insurance reimbursement of a therapy for breast cancer that was later proven to be ineffective. Another case study described how the use of narrative inappropriately exaggerated the perceived risk of a procedure, which led to limiting its use and preventing a large number of patients from its benefits. A third case study described how optimistic 'cure' or 'hope' stories of children with cancer were selectively used to raise money for cancer research that ignored the negative realities. The majority of included studies did not provide information on the definition or content of narratives, the theoretical framework underlying the narrative intervention or the possible predictors of the success of narrative interventions.

CONCLUSION

The existing evidence base precludes any robust inferences about the impact of narrative interventions on health policy-making. We discuss the implications of the findings for research and policy.

TRIAL REGISTRATION

The review protocol is registered in PROSPERO International prospective register of systematic reviews (ID =  CRD42018085011 ).

Barriers and facilitators to the integration of mental health services into primary health care: a systematic review

BACKGROUND

The objective of the review was to synthesize evidence of barriers and facilitators to the integration of mental health services into PHC from existing literature. The structure of the review was guided by the SPIDER framework which involves the following: Sample or population of interest-primary care providers (PCPs); Phenomenon of Interest-integration of mental health services into primary health care (PHC); Design-influenced robustness and analysis of the study; Evaluation-outcomes included subjective outcomes (views and attitudes); and Research type-qualitative, quantitative, and mixed methods studies.

METHODS

Studies that described mental health integration in PHC settings, involved primary care providers, and presented barriers/facilitators of mental health integration into PHC were included in the review. The sources of information included PubMed, PsycINFO, Cochrane Central Register of Controlled trials, the WHO website, and OpenGrey. Assessment of bias and quality was done using two separate tools: the Critical Appraisal Skills Program (CASP) qualitative checklist and the Effective Public Health Practice Project Quality Assessment Tool for Quantitative Studies.

RESULTS

Twenty studies met the inclusion criteria out of the 3353 search results. The most frequently reported barriers to integration of mental health services into PHC were (i) attitudes regarding program acceptability, appropriateness, and credibility; (ii) knowledge and skills; (iii) motivation to change; (iv) management and/or leadership; and (v) financial resources. In order to come up with an actionable approach to addressing the barriers, these factors were further analyzed along a behavior change theory.

DISCUSSION

We have shown that the integration of mental health services into PHC has been carried out by various countries. The analysis from this review provides evidence to inform policy on the existing barriers and facilitators to the implementation of the mental health integration policy option. Not all databases may have been exhausted.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO 2016 (Registration Number: CRD42016052000 ) and published in BMC Systematic Reviews August 2017.

Barriers and facilitators to implementing evidence-based interventions among third sector organisations: a systematic review

BACKGROUND

The third sector is becoming a growing provider of public, social, and health services. However, there is little evidence on the effectiveness of third sector organisations (TSOs), and their capacity to implement evidence-based interventions (EBIs). Understanding implementation aspects of service delivery remains an important issue in clinical practice, but is poorly understood in the context of TSOs. This is problematic, since implementation issues are known to be critical for effective intervention outcomes.

OBJECTIVES

To identify and synthesise existing research on what barriers and facilitators influence the implementation process of TSOs delivering EBIs.

METHODS

This review is reported according to PRISMA guidelines and was pre-registered in PROSPERO. Key databases were searched using relevant terms, experts in the field were contacted, and websites were reviewed. All identified studies were double-screened, and data were extracted independently by two authors. Included studies were synthesised using thematic analysis and were quality appraised.

RESULTS

Thirty-one studies were included, most of which were conducted in North America. The thematic synthesis identified resource limitations, in particular staff and finance, to be the most reported barrier to TSOs implementing EBIs. Organisational culture, including factors such as alignment between the mission of the TSO and EBI, and support/prioritisation of the implementation process were the most reported facilitators. These findings generalise across the included studies and are robust to study quality assessment.

CONCLUSIONS

While it is often assumed that good outcomes follow when implementing interventions that have been developed and tested according to best practice, little attention has been paid to how EBIs are best transported, contextualised, and implemented by third sector providers. This systematic review found that TSOs faced considerable challenges in implementing EBIs, which were primarily a lack of support and expertise, and unclear/insufficient guidelines on how to adapt EBIs to different populations. To address these challenges, it is important to engage with central stakeholders, such as funders, researchers, policymakers, and practitioners, to discuss how these needs can be met.

TRIAL REGISTRATION

PROSPERO: CRD42017073090 .

Characteristics of healthcare organisations struggling to improve quality: results from a systematic review of qualitative studies

BACKGROUND

Identifying characteristics associated with struggling healthcare organisations may help inform improvement. Thus, we systematically reviewed the literature to: (1) Identify organisational factors associated with struggling healthcare organisations and (2) Summarise these factors into actionable domains.

METHODS

Systematic review of qualitative studies that evaluated organisational characteristics of healthcare organisations that were struggling as defined by below-average patient outcomes (eg, mortality) or quality of care metrics (eg, Patient Safety Indicators). Searches were conducted in MEDLINE (via Ovid), EMBASE, Cochrane Library, CINAHL, and Web of Science from database inception through February 8 2018. Qualitative data were analysed using framework-based synthesis and summarised into key domains. Study quality was evaluated using the Critical Appraisal Skills Program tool.

RESULTS

Thirty studies (33 articles) from multiple countries and settings (eg, acute care, outpatient) with a diverse range of interviewees (eg, nurses, leadership, staff) were included in the final analysis. Five domains characterised struggling healthcare organisations: poor organisational culture (limited ownership, not collaborative, hierarchical, with disconnected leadership), inadequate infrastructure (limited quality improvement, staffing, information technology or resources), lack of a cohesive mission (mission conflicts with other missions, is externally motivated, poorly defined or promotes mediocrity), system shocks (ie, events such as leadership turnover, new electronic health record system or organisational scandals that detract from daily operations), and dysfunctional external relations with other hospitals, stakeholders, or governing bodies.

CONCLUSIONS

Struggling healthcare organisations share characteristics that may affect their ability to provide optimal care. Understanding and identifying these characteristics may provide a first step to helping low performers address organisational challenges to improvement.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO: CRD42017067367.

What are the barriers and facilitators for third sector organisations (non-profits) to evaluate their services? A systematic review

BACKGROUND

The third sector is becoming a more common provider of social and health services, but little is known about how third sector organisations (TSOs) evaluate their activities. Past research has reported that the third sector is under increasing pressure to evaluate its impact and performance by government and other commissioning bodies. However, in responding to this increased pressure to undertake evaluation, research suggests that many TSOs struggle to evaluate their activities following the principles of evidence-based practice (EBP). Yet, there has been no systematic effort to investigate why the third sector is struggling to provide good quality evidence of its effects.

METHODS

This systematic review is reported following the PRISMA guidelines. Ten interdisciplinary databases were searched using a search string developed following best practice and in consultation with an information systems expert. Included studies were primary research of any research design investigating barriers to and facilitators of the evaluation process of TSOs as identified by practitioners. All studies were quality appraised, and the results were synthesised as a thematic summary.

RESULTS

Twenty-four studies were included, which mainly investigated TSOs working within health and social services. The thematic summary identified the main barriers for TSOs to undertake evaluation to be related to the (1) lack of financial resources, (2) lack of technical capability and evaluation literacy and (3) challenges around identifying relevant evaluation systems and outcome indicators. Key facilitating factors involved (1) getting the appropriate support, (2) having an organisational culture that supports evaluation and (3) the motivation to be accountable to stakeholders. These findings were robust to study quality.

CONCLUSIONS

This review constitutes the first systematic effort to synthesise existing literature on factors supporting and preventing evaluation by TSOs. The prevalence of factors revolving around the lack of support, resources and clarity on appropriate outcome indicators suggests that many of the identified challenges may be met by applying evidence-based and stakeholder-inclusive strategies to develop shared evaluation requirements. Future efforts should address the application of EBP as part of the commissioning process of TSOs.

Clinical reminder alert fatigue in healthcare: a systematic literature review protocol using qualitative evidence

BACKGROUND

Integrated reminders within clinical systems have become more prevalent due to the use of electronic health records and evidence demonstrating an increase in compliance within practice. Clinical reminders are assessed for effectiveness on an individual basis, rather than in combination with existing prompts for other conditions. The growing number of prompts may be counter-productive as healthcare professionals are increasingly suffering from "reminder fatigue" meaning many reminders are ignored. This work will review the qualitative evidence to identify barriers and enablers of existing prompts found within computerised decision support systems. Our focus will be on primary care where clinicians have to negotiate a plethora of reminders as they deal with increasingly complex patients and sophisticated treatment regimes. The review will provide a greater understanding of existing systems and the way clinicians interact with them to inform the development of more effective and targeted clinical reminders.

METHODS

A comprehensive search using piloted terms will be used to identify relevant literature from 1960 (or commencement of database) to 2017. MEDLINE, MEDLINE In Process, EMBASE, HMIC, PsycINFO, CDSR DARE, HTA, CINAHL and CPCI, will be searched, as well as grey literature and references and citations of included papers. Manuscripts will be assessed for eligibility, bias and quality using the CASP tool with narrative data being included and questionnaire based studies excluded. Inductive thematic analysis will be performed in order to produce a conceptual framework defining the key barriers around integrated clinical reminders.

DISCUSSION

Indications of alert and reminder fatigue are found throughout the current literature. However, this has not been fully investigated using a robust qualitative approach, particularly in a rapidly growing body of evidence. This review will aid people forming new clinical systems so that alerts can be incorporated appropriately.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO: CRD42016029418.

Ancillary health effects of climate mitigation scenarios as drivers of policy uptake: a review of air quality, transportation and diet co-benefits modeling studies

Background

Significant mitigation efforts beyond the Nationally Determined Commitments (NDCs) coming out of the 2015 Paris Climate Agreement are required to avoid warming of 2°C above pre-industrial temperatures. Health co-benefits represent selected near term, positive consequences of climate policies that can offset mitigation costs in the short term before the beneficial impacts of those policies on the magnitude of climate change are evident. The diversity of approaches to modeling mitigation options and their health effects inhibits meta-analyses and syntheses of results useful in policy-making.

Methods/Design

We evaluated the range of methods and choices in modeling health co-benefits of climate mitigation to identify opportunities for increased consistency and collaboration that could better inform policy-making. We reviewed studies quantifying the health co-benefits of climate change mitigation related to air quality, transportation, and diet published since the 2009 Lancet Commission 'Managing the health effects of climate change' through January 2017. We documented approaches, methods, scenarios, health-related exposures, and health outcomes.

Results/Synthesis

Forty-two studies met the inclusion criteria. Air quality, transportation, and diet scenarios ranged from specific policy proposals to hypothetical scenarios, and from global recommendations to stakeholder-informed local guidance. Geographic and temporal scope as well as validity of scenarios determined policy relevance. More recent studies tended to use more sophisticated methods to address complexity in the relevant policy system.

Discussion

Most studies indicated significant, nearer term, local ancillary health benefits providing impetus for policy uptake and net cost savings. However, studies were more suited to describing the interaction of climate policy and health and the magnitude of potential outcomes than to providing specific accurate estimates of health co-benefits. Modeling the health co-benefits of climate policy provides policy-relevant information when the scenarios are reasonable, relevant, and thorough, and the model adequately addresses complexity. Greater consistency in selected modeling choices across the health co-benefits of climate mitigation research would facilitate evaluation of mitigation options particularly as they apply to the NDCs and promote policy uptake.

Barriers and facilitators to the integration of mental health services into primary health care: a systematic review protocol

BACKGROUND

Mental health is an integral part of health and well-being and yet health systems have not adequately responded to the burden of mental disorders. Integrating mental health services into primary health care (PHC) is the most viable way of closing the treatment gap and ensuring that people get the mental health care they need. PHC was formally adapted by the World Health Organization (WHO), and they have since invested enormous amounts of resources across the globe to ensure that integration of mental health services into PHC works.

METHODS

This review will use the SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) framework approach to identify experiences of mental health integration into PHC; the findings will be reported using the "Best fit" framework synthesis. PubMed, EMBASE, PsycINFO, and Cochrane Central Register of Controlled trials (CENTRAL) will be searched including other sources like the WHO website and OpenGrey database. Assessment of bias and quality will be done at study level using two separate tools to check for the quality of evidence presented. Data synthesis will take on two synergistic approaches (qualitative and quantitative studies). Synthesizing evidence from countries across the globe will provide useful insights into the experiences of integrating mental health services into PHC and how the barriers and challenges have been handled. The findings will be useful to a wide array of stakeholders involved in the implementation of the mental health integration into PHC.

DISCUSSION

The SPIDER framework has been chosen for this review because of its suitable application to qualitative and mixed methods research and will be used as a guide when selecting articles for inclusion. Data extracted will be synthesized using the "Best fit" framework because it has been used before and proved its suitability in producing new conceptual models for explaining decision-making and possible behaviors. Synthesizing evidence from countries across the globe will provide useful insights into the experiences of integrating mental health services into PHC and how the barriers and challenges have been handled.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016052000.