Evidence-based priority-setting: what do the decision-makers think?

Barriers and facilitators to implementing priority setting and resource allocation tools in hospital decisions: A systematic review

Health care budgets in high-income countries are having issues coping with unsustainable growth in demand, particularly in the hospital setting. Despite this, implementing tools systematising priority setting and resource allocation decisions has been challenging. This study answers two questions: (1) what are the barriers and facilitators to implementing priority setting tools in the hospital setting of high-income countries? and (2) what is their fidelity? A systematic review using the Cochrane methods was conducted including studies of hospital-related priority setting tools reporting barriers or facilitators for implementation, published after the year 2000. Barriers and facilitators were classified using the Consolidated Framework for Implementation Research (CFIR). Fidelity was assessed using priority setting tool's standards. Out of thirty studies, ten reported program budgeting and marginal analysis (PBMA), twelve multi-criteria decision analysis (MCDA), six health technology assessment (HTA) related frameworks, and two, an ad hoc tool. Barriers and facilitators were outlined across all CFIR domains. Implementation factors not frequently observed, such as 'evidence of previous successful tool application', 'knowledge and beliefs about the intervention' or 'external policy and incentives' were reported. Conversely, some constructs did not yield any barrier or facilitator including 'intervention source' or 'peer pressure'. PBMA studies satisfied the fidelity criteria between 86% and 100%, for MCDA it varied between 36% and 100%, and for HTA it was between 27% and 80%. However, fidelity was not related to implementation. This study is the first to use an implementation science approach. Results represent the starting point for organisations wishing to use priority setting tools in the hospital setting by providing an overview of barriers and facilitators. These factors can be used to assess readiness for implementation or to serve as the foundation for process evaluations. Through our findings, we aim to improve the uptake of priority setting tools and support their sustainable use.

Reinforcing Science and Policy, With Suggestions for Future Research Comment on "Evidence-Informed Deliberative Processes for Health Benefit Package Design - Part II: A Practical Guide"

Oortwijn et al continue their guide to good practice in the use of deliberative processes in health technology assessment (HTA) based on a survey of international practice. This is useful, and I applaud their care in maintaining objectivity, especially regarding the treatment of moral and politically controversial issues, in reporting how jurisdictions have handled such matters in designing HTA procedures and in their execution. To their suggestions for future research, I add: the historical development of deliberation in healthcare decision-making and in other fields of public choice, with comparisons of methods, successes and failures; development of guidance on the design and use of deliberative processes that enhance decision-making when there is no consensus amongst the decision-makers; ways of identifying and managing context-free and context-sensitive evidence; and a review of high-level capacity building to raise awareness of HTA and the use of knowledge translation and exchange (KTE) and deliberation amongst policy makers, especially in low and middle-income countries.

Determinants of evidence use by frontline maternal, newborn and child health staff in selected health facilities in Ghana

Background

The current paper examines the level of use of evidence and factors affecting the use of evidence by frontline maternal, newborn and child health (MNCH) and reproductive and child health (RCH) staff in practice decisions in selected health facilities in Ghana.

Methods

Data on use of evidence and its correlates was collected from 509 frontline healthcare staff drawn from 44 health facilities in three regions in Ghana. Means were used to examine the level of use of evidence, whiles cross-tabulations and Partial Least Squares-based regression were used to examine factors associated with the use of evidence in practice decisions by frontline MNCH/RCH staff.

Findings

The findings suggest a high level of use of evidence by frontline MNCH/RCH staff in practice decisions (score of 3.98 out of 5), albeit that evidence use is skewed towards the use of practice guidelines and policies. For the antecedents of evidence use, attitude had the highest score (3.99), followed by knowledge (3.8), access to evidence (3.77) and organizational structure (3.57), using a threshold of 5. The regression results indicate that attitudes and knowledge of frontline MNCH/RCH staff, organizational structure (strongest association), years of experience, being a male and working in a mission health facility are significantly positive correlated with evidence use, whiles working in a private health facility or in the post-natal clinic is negatively correlated with the use of evidence.

Conclusion

We argue that any effort to improve the use of evidence by frontline MNCH/RCH staff in practice decisions should focus on improving attitudes and knowledge of staff as well as challenges related to the structure of the organisation. Given however that the score for attitude was relatively high, emphases to improve evidence use should be on access to evidence and organizational structure in particular, which had the lowest score even though it has the strongest association with the use of evidence.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00881-8.

Developing Public Health Emergency Response Leaders in Incident Management: A Scoping Review of Educational Interventions

During emergency responses, public health leaders frequently serve in incident management roles that differ from their routine job functions. Leaders' familiarity with incident management principles and functions can influence response outcomes. Therefore, training and exercises in incident management are often required for public health leaders. To describe existing methods of incident management training and exercises in the literature, we queried 6 English language databases and found 786 relevant articles. Five themes emerged: (1) experiential learning as an established approach to foster engaging and interactive learning environments and optimize training design; (2) technology-aided decision support tools are increasingly common for crisis decision-making; (3) integration of leadership training in the education continuum is needed for developing public health response leaders; (4) equal emphasis on competency and character is needed for developing capable and adaptable leaders; and (5) consistent evaluation methodologies and metrics are needed to assess the effectiveness of educational interventions.These findings offer important strategic and practical considerations for improving the design and delivery of educational interventions to develop public health emergency response leaders. This review and ongoing real-world events could facilitate further exploration of current practices, emerging trends, and challenges for continuous improvements in developing public health emergency response leaders.

Priority Setting in the Polish Health Care System According to Patients' Perspective

Identification of health priorities is concerned with equitable distribution of resources and is an important part of strategic planning in the health care system. The aim of this article is to describe health priorities in the Polish health care system from the patients’ perspective. The study included 533 patients hospitalized in the Lodz region. The average age of the respondents was 48.5 years and one third (36.6%) had university education. Most of the respondents (64.9%) negatively assessed the functioning of the health care system in Poland. Most of them claimed the following aspects require improvements: financing health services (85.8%), determining priorities in health care (80.3%), the role of health insurance (80.3%), and medical education (70.8%). Over 70% of the respondents agreed the role of politicians in designing and implementing health system reforms should be limited. The fact that the respondents so negatively assessed the Polish health care system implies there is a need for full discussion on redefining health priorities.

Ethics education and moral decision-making in clinical commissioning: an interview study

BACKGROUND

Clinical commissioning involves ethically challenging decisions about health resource allocation. However, commissioners come from a range of professional backgrounds with varying levels of training and expertise in ethical decision-making. Hence, they may lack the relevant training and resources to feel fully prepared for this increasingly demanding role.

AIM

This study aims to provide insight into how prepared commissioners feel in making ethical decisions; what ethics learning needs they might have; and how these might be addressed.

DESIGN AND SETTING

This qualitative interview study explored the experiences of commissioners working for clinical commissioning groups (CCGs) in England.

METHOD

Eighteen participants were interviewed between December 2017 and July 2018 using a purposive sampling approach to participant selection. Transcriptions were coded and analysed using the constant comparative method of thematic analysis.

RESULTS

Most participants had not received ethics training in preparation for, or during, their commissioning role, and reported difficulties identifying and analysing ethical issues. Participants often felt uncomfortable about decisions they were involved in, attributing this to a number of factors: a sense of moral unease; concerns that CCGs' decision-making processes were not sufficiently transparent; and that CCGs were not fully accountable to the population served.

CONCLUSION

Commissioners face complex decisions involving ethical issues, and associated moral unease is exacerbated by a lack of ethics training and lack of confidence in identifying and analysing these. This study shows a clear need for additional support and ethics training for commissioners to support them in this area of decision-making.

How hospitals select their patient safety priorities: An exploratory study of four Veterans Health Administration hospitals

BACKGROUND

Hospitals face ongoing pressure to reduce patient safety events. However, given resource constraints, hospitals must prioritize their safety improvements. There is limited literature on how hospitals select their safety priorities.

PURPOSE

The aim of this research was to describe and compare the approaches used by Veterans Health Administration (VA) hospitals to select their safety priorities.

METHODOLOGY

Semistructured telephone interviews with key informants (n = 16) were used to collect data on safety priorities in four VA hospitals from May to December 2016. We conducted a directed content analysis of the interview notes using an organizational learning perspective. We coded for descriptive data on the approaches (e.g., set of cues, circumstances, and activities) used to select safety priorities, a priori organizational learning capabilities (learning processes, learning environment, and learning-oriented leadership), and emergent domains. For cross-site comparisons, we examined the coded data for patterns.

RESULTS

All hospitals used multiple approaches to select their safety priorities; these approaches used varied across hospitals. Although no single approach was reported as particularly influential, all hospitals used approaches that addressed system level or national requirements (i.e., externally required activities). Additional approaches used by hospitals (e.g., responding to staff concerns of patient safety issues, conducting a multidisciplinary team investigation) were less connected to externally required activities and demonstrated organizational learning capabilities in learning processes (e.g., performance monitoring), learning environment (e.g., staff's psychological safety), and learning-oriented leadership (e.g., establishing a nonpunitive culture).

PRACTICE IMPLICATIONS

Leaders should examine the approaches used to select safety priorities and the role of organizational learning in these selection approaches. Exclusively relying on approaches focused on externally required activities may fail to identify safety priorities that are locally relevant but not established as significant at the system or national levels. Organizational learning may promote hospitals' use of varied approaches to guide their selection of safety priorities and thereby benefit hospital safety improvement efforts.

Learning to lead: a review and synthesis of literature examining health care managers' use of knowledge

Background

Scholarship cites health care managers (HCMs) as not using research evidence in their management practice. The purpose of this review was to evaluate the effectiveness of interventions to enhance HCMs use of research evidence in practice.

Methods

We carried out a systematic review and focus groups to validate the review findings. We searched 10 electronic databases for studies reporting on interventions for HCMs to enhance research utilization in their practice. Qualitative studies were analysed using Hoon’s approach to meta-synthesis.

Results

Seven, primarily qualitative, studies of varying quality (reported in 11 articles) met our inclusion criteria. Interventions to enhance research use by HCMs included: informal and formal training, computer-based application, executive-level knowledge translation activities and residency programmes. Studies did not report efficacy of interventions or impacts of increasing managers’ use of research on staff or patient outcomes. Meta-synthesis yielded four contextual factors influencing the perceived effectiveness of interventions to enhance research use by HCMs: organizational culture, competing priorities, time as a resource and capacity building. Included studies differed in how they defined research and demonstrated varying understandings of research among HCMs, limiting the generalizability of work in this field.

Conclusions

Healthcare managers are increasingly called upon to make evidence-based decisions in practice, but the small number of studies and diverse strategies employed hinder our ability to identify any intervention to increase use of evidence as superior. Future studies in this area should clearly articulate the definition of research evidence they base their decisions on. Registration: PROSPERO (CRD42014006256)

A mixed-method systematic review of the effectiveness and acceptability of preoperative psychological preparation programmes to reduce paediatric preoperative anxiety in elective surgery

AIM

To explore the effectiveness of preoperative psychological preparation programmes aimed to reduce paediatric preoperative anxiety and the potential factors that could have an impact on parent and children's acceptance of such interventions.

BACKGROUND

Various preoperative psychological preparation programmes are available to address paediatric preoperative anxiety. No mixed-method review has been conducted to explore the effectiveness and acceptability of these programmes.

DESIGN

A mixed-method systematic review.

DATA SOURCES

Seven bibliographic databases were searched from inception to September 2016, complemented by hand searching of key journals, the reference lists of relevant reviews, search for grey literature and the contacting of associated experts.

REVIEW METHODS

The review process was conducted based on the framework developed by the Evidence for Policy and Practice Information and Co-ordinating Centre. A narrative summary and a thematic synthesis were developed to synthesize the quantitative and qualitative data respectively, followed by a third synthesis to combine the previous syntheses.

RESULTS

Nineteen controlled trials and eleven qualitative studies were included for data synthesis. The controlled trials reveal that educational multimedia applications and web-based programmes may reduce paediatric preoperative anxiety, while the effectiveness of therapeutic play and books remains uncertain. Qualitative studies showed parent-child dyads seek different levels of information.

CONCLUSIONS

Providing matched information provision to each parent and child, actively involving children and their parents and teaching them coping skills, may be the essential hallmarks of a successful preoperative psychological preparation. Further research is necessary to confirm the effectiveness of therapeutic play and books.

Identifying public health policymakers' sources of information: comparing survey and network analyses

Background

Research suggests that policymakers often use personal contacts to find information and advice. However, the main sources of information for public health policymakers are not known. This study aims to describe policymakers' sources of information. A questionnaire survey of public health policymakers across Greater Manchester (GM) was carried out (response rate 48%). All policy actors above Director level involved in public health policy (finding, analyzing or producing information, producing or implementing policy) in GM were included in the sampling frame. Respondents were provided with a list of sources of information and asked which they used (categorical data) and to name specific individuals who acted as sources of information (network data). Data were analyzed using frequencies and network analysis. The most frequently chosen sources of information from the categorical data were NICE, government websites and Directors of Public Health. However, the network data showed that the main sources of information in the network were actually mid-level managers in the NHS, who had no direct expertise in public health. Academics and researchers did not feature in the network. Both survey and network analyses provide useful insights into how policymakers access information. Network analysis offers practical and theoretical contributions to the evidence-based policy debate. Identifying individuals who act as key users and producers of evidence allows academics to target actors likely to use and disseminate their work.

Assessment of policy makers' individual and organizational capacity to acquire, assess, adapt and apply research evidence for maternal and child health policy making in Nigeria: a cross-sectional quantitative survey

Background

Throughout the world, there is increasing awareness and acknowledgement of the value of research evidence in the development of effective health policy and in quality health care practice and administration. Among the major challenges associated with the lack of uptake of research evidence into policy and practice in Nigeria is the capacity constraints of policymakers to use research evidence in policy making.

Objective

To assess the capacity of maternal and child health policy makers to acquire, access, adapt and apply available research evidence.

Methods

This cross-sectional quantitative survey was conducted at a national maternal, newborn and child health (MNCH) stakeholders' engagement event. An evidence to policy self-assessment questionnaire was used to assess the capacity of forty MNCH policy makers to acquire, assess, adapt and apply research evidence for policy making.

Results

Low mean ratings were observed ranging from 2.68–3.53 on a scale of 5 for knowledge about initiating/conducting research and capacity to assess authenticity, validity, reliability, relevance and applicability of research evidence and for organizational capacity for promoting and using of research for policy making.

Conclusion

There is need to institute policy makers' capacity development programmes to improve evidence-informed policymaking.

Assessing the capacity of ministries of health to use research in decision-making: conceptual framework and tool

Background

The capacity to demand and use research is critical for governments if they are to develop policies that are informed by evidence. Existing tools designed to assess how government officials use evidence in decision-making have significant limitations for low- and middle-income countries (LMICs); they are rarely tested in LMICs and focus only on individual capacity. This paper introduces an instrument that was developed to assess Ministry of Health (MoH) capacity to demand and use research evidence for decision-making, which was tested for reliability and validity in eight LMICs (Bangladesh, Fiji, India, Lebanon, Moldova, Pakistan, South Africa, Zambia).

Methods

Instrument development was based on a new conceptual framework that addresses individual, organisational and systems capacities, and items were drawn from existing instruments and a literature review. After initial item development and pre-testing to address face validity and item phrasing, the instrument was reduced to 54 items for further validation and item reduction. In-country study teams interviewed a systematic sample of 203 MoH officials. Exploratory factor analysis was used in addition to standard reliability and validity measures to further assess the items.

Results

Thirty items divided between two factors representing organisational and individual capacity constructs were identified. South Africa and Zambia demonstrated the highest level of organisational capacity to use research, whereas Pakistan and Bangladesh were the lowest two. In contrast, individual capacity was highest in Pakistan, followed by South Africa, whereas Bangladesh and Lebanon were the lowest.

Conclusion

The framework and related instrument represent a new opportunity for MoHs to identify ways to understand and improve capacities to incorporate research evidence in decision-making, as well as to provide a basis for tracking change.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-017-0227-3) contains supplementary material, which is available to authorized users.

Implementing priority setting frameworks: Insights from leading researchers

In spite of a substantial literature developing frameworks for policymakers to use in resource allocation decisions in healthcare, there remains limited published work reporting on the implementation or evaluation of such frameworks in practice. This paper presents findings of a targeted survey of 18 leading researchers around the implementation and evaluation of priority-setting exercises. Approximately one third of respondents knew of situations where recommendations of priority-setting exercises had been implemented, one third knew that recommendations had not been implemented and the final third responded that they did not know whether recommendations had been adopted. The lack of evidence linking the implementation of priority-setting recommendations to equity and efficiency outcomes was highlighted by all respondents. Features identified as facilitating successful implementation of priority-setting recommendations included having a climate ready to accept priority-setting, good leadership or a 'champion' for the priority-setting process and having a health economist to guide the process. Successful disinvestment was very uncommon in the experience of the researchers surveyed. Recommendations emerging from Program Budgeting and Marginal Analysis exercises appeared to be more widely implemented than those coming from alternative processes. Identifying if the process was repeated following the initial process was suggested as a means to measure success.

Setting budgets and priorities at state health agencies

CONTEXT

State health departments across the country are responsible for assuring and improving the health of the public, and yet financial constraints grow only more acute, and resource allocation decisions become even more challenging. Little empirical evidence exists regarding how officials working in state health departments make these tough allocation decisions.

DESIGN

Through a mixed-methods process, we attempted to address this gap in knowledge and characterize issues of resource allocation at state health agencies (SHAs). First, we conducted 45 semistructured interviews across 6 states. Next, a Web-based survey was sent to 355 public health leaders within all states and District of Columbia. In total, 207 leaders responded to the survey (66% response rate).

PARTICIPANTS

Leaders of SHAs.

RESULTS

The data suggest that state public health leaders are highly consultative internally while making resource allocation decisions, but they also frequently engage with the governor's office and the legislature-much more so at the executive level than at the division director level. Respondents reported that increasing and decreasing funding for certain activities occur frequently and have a moderate impact on the agency or division budget. Agencies continue to "thin the soup," or prefer cutting broadly to cutting deeply.

CONCLUSIONS

Public health leaders report facing significant tradeoffs in the course of priority-setting. The authorizing environment continues to force public health leaders to make challenging tradeoffs between unmet need and political considerations, and among vulnerable groups.

High performance in healthcare priority setting and resource allocation: A literature- and case study-based framework in the Canadian context

Priority setting and resource allocation, or PSRA, are key functions of executive teams in healthcare organizations. Yet decision-makers often base their choices on historical patterns of resource distribution or political pressures. Our aim was to provide leaders with guidance on how to improve PSRA practice, by creating organizational contexts which enable high performance. We carried out in-depth case studies of six Canadian healthcare organizations to obtain from healthcare leaders their understanding of the concept of high performance in PSRA and the factors which contribute to its achievement. Individual and group interviews were carried out (n = 62) with senior managers, middle managers and Board members. Site observations and document review were used to assist researchers in interpreting the interview data. Qualitative data were analyzed iteratively with the literature on empirical examples of PSRA practice, in order to develop a framework of high performance in PSRA. The framework consists of four domains - structures, processes, attitudes and behaviours, and outcomes - within which are 19 specific elements. The emergent themes derive from case studies in different kinds of health organizations (urban/rural, small/large) across Canada. The elements can serve as a checklist for 'high performance' in PSRA. This framework provides a means by which decision-makers in healthcare might assess their practice and identify key areas for improvement. The findings are likely generalizable, certainly within Canada but also across countries. This work constitutes, to our knowledge, the first attempt to present a full package of elements comprising high performance in health care PSRA.

Evidence based policy making and the 'art' of commissioning - how English healthcare commissioners access and use information and academic research in 'real life' decision-making: an empirical qualitative study

BACKGROUND

Policymakers such as English healthcare commissioners are encouraged to adopt 'evidence-based policy-making', with 'evidence' defined by researchers as academic research. To learn how academic research can influence policy, researchers need to know more about commissioning, commissioners' information seeking behaviour and the role of research in their decisions.

METHODS

In case studies of four commissioning organisations, we interviewed 52 people including clinical and managerial commissioners, observed 14 commissioning meetings and collected documentation e.g. meeting minutes and reports. Using constant comparison, data were coded, summarised and analysed to facilitate cross case comparison.

RESULTS

The 'art of commissioning' entails juggling competing agendas, priorities, power relationships, demands and personal inclinations to build a persuasive, compelling case. Policymakers sought information to identify options, navigate ways through, justify decisions and convince others to approve and/or follow the suggested course. 'Evidence-based policy-making' usually meant pragmatic selection of 'evidence' such as best practice guidance, clinicians' and users' views of services and innovations from elsewhere. Inconclusive or negative research was unhelpful in developing policymaking plans and did not inform disinvestment decisions. Information was exchanged through conversations and stories, which were fast, flexible and suited the rapidly changing world of policymaking. Local data often trumped national or research-based evidence. Local evaluations were more useful than academic research.

DISCUSSION

Commissioners are highly pragmatic and will only use information that helps them create a compelling case for action.Therefore, researchers need to start producing more useful information.

CONCLUSIONS

To influence policymakers' decisions, researchers need to 1) learn more about local policymakers' priorities 2) develop relationships of mutual benefit 3) use verbal instead of writtencommunication 4) work with intermediaries such as public health consultants and 5) co-produce local evaluations.

Priority setting in Indigenous health: assessing priority setting process and criteria that should guide the health system to improve Indigenous Australian health

INTRODUCTION

The health of Indigenous Australians is worse than that of other Australians. Most of the determinants of health are preventable and the poor health outcomes are inequitable. The Australian Government recently pledged to close that health gap. One possible way is to improve the priority setting process to ensure transparency and the use of evidence such as epidemiology, equity and economic evaluation.The purpose of this research was to elicit the perceptions of Indigenous and non-Indigenous decision-makers on several issues related to priority setting in Indigenous-specific health care services. Specifically, we aimed to:1. identify the criteria used to set priorities in Indigenous-specific health care services;2. determine the level of uptake of economic evaluation evidence by decision-makers and how to improve its uptake; and 3. identify how the priority setting process can be improved from the perspective of decision-makers.

METHODS

We used a paper survey instrument, adapted from Mitton and colleagues' work, and a face-to-face interview approach to elicit decision-makers' perceptions in Indigenous-specific health care in Victoria, Australia. We used mixed methods to analyse data from the survey. Responses were summarised using descriptive statistics and content analysis. Results were reported as numbers and percentages.

RESULTS

The size of the health burden; sustainability and acceptability of interventions; historical trends/patterns; and efficiency are key criteria for making choices in Indigenous health in Victoria. There is a need for an explicit priority setting approach, which is systematic, and is able to use available data/evidence, such as economic evaluation evidence. The involvement of Indigenous Australians in the process would potentially make the process acceptable.

CONCLUSIONS

An economic approach to priority setting is a potentially acceptable and useful tool for Aboriginal Community Controlled Health Services (ACCHS). It has the ability to use evidence and ensure due process at the same time. The use of evidence can ensure that health outcomes for Indigenous peoples can be maximised - hence, increase the potential for 'closing the gap' between Indigenous and other Australians.

Changing priority setting practice: the role of implementation in practice change

BACKGROUND

Programme budgeting and marginal analysis (PBMA) is a priority setting approach that assists decision makers in choosing among resource demands. This paper describes and evaluates the process of implementing PBMA in a Canadian regional health authority, and draws out key lessons learned from this experience.

METHODS

Qualitative data were collected through semi-structured participant interviews (twelve post year-1; nine post year-2), meeting attendance, and document review. Interview transcripts were analyzed using a constant comparison technique. Other data were analyzed to evaluate PBMA implementation.

RESULTS

Desire for more clarity and for PBMA adaptations emerged as overarching themes. Participants desired greater clarity of their roles and how PBMA should be used to achieve PBMA's potential benefits. They argued that each PBMA stage should be useful independent of the others so that implementation could be adapted. To help improve clarity and ensure that resources were available to support PBMA, participants requested an organizational readiness and capacity assessment.

CONCLUSION

We suggest tactics by which PBMA may be more closely aligned with real-world priority setting practice. Our results also contribute to the literature on PBMA use in various healthcare settings. Highlighting implementation issues and potential responses to these should be of interest to decision makers implementing PBMA and other evidence-informed practices.

Barriers and facilitators to evidence-use in program management: a systematic review of the literature

Background

The use of evidence in decision-making at the program management level is a priority in health care organizations. The objective of this study was to identify potential barriers and facilitators experienced by managers to the use of evidence in program management within health care organizations.

Methods

The authors conducted a comprehensive search for published, peer-reviewed and grey literature that explores the use of evidence in program management. Two reviewers selected relevant studies from which data was extracted using a standard data abstraction form and tabulated for qualitative analysis. The results were summarized through narrative review. The quality of the included studies was assessed using published criteria for the critical appraisal of qualitative, quantitative and mixed methods research.

Results

Fourteen papers were included in the review. Barriers and facilitators were categorized into five main thematic areas: (1) Information, (2) Organization – Structure and Process, (3) Organization – Culture, (4) Individual, and (5) Interaction.

Conclusion

This paper reviews the literature on barriers and facilitators to evidence-informed decision-making experienced by program management decision-makers within health care organizations. The multidimensional solutions required to promote evidence-informed program management can be developed through an understanding of the existing barriers and facilitators of evidence-use.

Experiences and attitudes towards evidence-informed policy-making among research and policy stakeholders in the Canadian agri-food public health sector

Policy-makers working at the interface of agri-food and public health often deal with complex and cross-cutting issues that have broad health impacts and socio-economic implications. They have a responsibility to ensure that policy-making based on these issues is accountable and informed by the best available scientific evidence. We conducted a qualitative descriptive study of agri-food public health policy-makers and research and policy analysts in Ontario, Canada, to understand their perspectives on how the policy-making process is currently informed by scientific evidence and how to facilitate this process. Five focus groups of 3-7 participants and five-one-to-one interviews were held in 2012 with participants from federal and provincial government departments and industry organizations in the agri-food public health sector. We conducted a thematic analysis of the focus group and interview transcripts to identify overarching themes. Participants indicated that the following six key principles are necessary to enable and demonstrate evidence-informed policy-making (EIPM) in this sector: (i) establish and clarify the policy objectives and context; (ii) support policy-making with credible scientific evidence from different sources; (iii) integrate scientific evidence with other diverse policy inputs (e.g. economics, local applicability and stakeholder interests); (iv) ensure that scientific evidence is communicated by research and policy stakeholders in relevant and user-friendly formats; (V) create and foster interdisciplinary relationships and networks across research and policy communities; and (VI) enhance organizational capacity and individual skills for EIPM. Ongoing and planned efforts in these areas, a supportive culture, and additional education and training in both research and policy realms are important to facilitate evidence-informed policy-making in this sector. Future research should explore these findings further in other countries and contexts.

Decision maker perceptions of resource allocation processes in Canadian health care organizations: a national survey

BACKGROUND

Resource allocation is a key challenge for healthcare decision makers. While several case studies of organizational practice exist, there have been few large-scale cross-organization comparisons.

METHODS

Between January and April 2011, we conducted an on-line survey of senior decision makers within regional health authorities (and closely equivalent organizations) across all Canadian provinces and territories. We received returns from 92 individual managers, from 60 out of 89 organizations in total. The survey inquired about structures, process features, and behaviours related to organization-wide resource allocation decisions. We focus here on three main aspects: type of process, perceived fairness, and overall rating.

RESULTS

About one-half of respondents indicated that their organization used a formal process for resource allocation, while the others reported that political or historical factors were predominant. Seventy percent (70%) of respondents self-reported that their resource allocation process was fair and just over one-half assessed their process as 'good' or 'very good'. This paper explores these findings in greater detail and assesses them in context of the larger literature.

CONCLUSION

Data from this large-scale cross-jurisdictional survey helps to illustrate common challenges and areas of positive performance among Canada's health system leadership teams.

Affordability as a discursive accomplishment in a changing National Health Service

Health systems worldwide face the challenges of rationing. The English National Health Service (NHS) was founded on three core principles: universality, comprehensiveness, and free at the point of delivery. Yet patients are increasingly hearing that some treatments are unaffordable on the NHS. We considered affordability as a social accomplishment and sought to explore how those charged with allocating NHS resources achieved this in practice. We undertook a linguistic ethnography to examine the work practices of resource allocation committees in three Primary Care Trusts (PCTs) in England between 2005 and 2012, specifically deliberations over 'individual funding requests' (IFRs)--requests by patients and their doctors for the PCT to support a treatment not routinely funded. We collected and analysed a diverse dataset comprising policy documents, legal judgements, audio recordings, ethnographic field notes and emails from PCT committee meetings, interviews and a focus group with committee members. We found that the fundamental values of universality and comprehensiveness strongly influenced the culture of these NHS organisations, and that in this context, accomplishing affordability was not easy. Four discursive practices served to confer legitimacy on affordability as a guiding value of NHS health care: (1) categorising certain treatments as only eligible for NHS funding if patients could prove 'exceptional' circumstances; (2) representing resource allocation decisions as being not (primarily) about money; (3) indexical labelling of affordability as an ethical principle, and (4) recontextualising legal judgements supporting refusal of NHS treatment on affordability grounds as 'rational'. The overall effect of these discursive practices was that denying treatment to patients became reasonable and rational for an organisation even while it continued to espouse traditional NHS values. We conclude that deliberations about the funding of treatments at the margins of NHS care have powerful consequences both for patients and for redrawing the ideological landscape of NHS care.

Prioritizing strategies for comprehensive liver cancer control in Asia: a conjoint analysis

Background

Liver cancer is a complex and burdensome disease, with Asia accounting for 75% of known cases. Comprehensive cancer control requires the use of multiple strategies, but various stakeholders may have different views as to which strategies should have the highest priority. This study identified priorities across multiple strategies for comprehensive liver cancer control (CLCC) from the perspective of liver cancer clinical, policy, and advocacy stakeholders in China, Japan, South Korea and Taiwan. Concordance of priorities was assessed across the region and across respondent roles.

Methods

Priorities for CLCC were examined as part of a cross-sectional survey of liver cancer experts. Respondents completed several conjoint-analysis choice tasks to prioritize 11 strategies. In each task, respondents judged which of two competing CLCC plans, consisting of mutually exclusive and exhaustive subsets of the strategies, would have the greatest impact. The dependent variable was the chosen plan, which was then regressed on the strategies of different plans. The restricted least squares (RLS) method was utilized to compare aggregate and stratified models, and t-tests and Wald tests were used to test for significance and concordance, respectively.

Results

Eighty respondents (69.6%) were eligible and completed the survey. Their primary interests were hepatitis (26%), hepatocellular carcinoma (HCC) (58%), metastatic liver cancer (10%) and transplantation (6%). The most preferred strategies were monitoring at-risk populations (p<0.001), clinician education (p<0.001), and national guidelines (p<0.001). Most priorities were concordant across sites except for three strategies: transplantation infrastructure (p=0.009) was valued lower in China, measuring social burden (p=0.037) was valued higher in Taiwan, and national guidelines (p=0.025) was valued higher in China. Priorities did not differ across stakeholder groups (p=0.438).

Conclusions

Priorities for CLCC in Asia include monitoring at-risk populations, clinician education, national guidelines, multidisciplinary management, public awareness and centers of excellence. As most priorities are relatively concordant across the region, multilateral approaches to addressing comprehensive liver cancer would be beneficial. However, where priorities are discordant among sites, such as transplantation infrastructure, strategies should be tailored to local needs.

A framework to improve evidence-informed decision-making in health service management

OBJECTIVE

The objective of this paper is to present and provide justification for a framework to improve evidence-informed management decision-making among health service managers. Three research questions informed the study: How have different perspectives influenced how evidence has been defined? What are the barriers to the practice of evidence-informed decision-making (EIDM)? What are the factors that may encourage the application of evidence to guide management decision-making processes?

METHODS

A literature review was conducted to identify studies that examined the practice of EIDM among health service managers. Information relevant to the three research questions was collectively analysed, compared and contrasted based on their relevance to the EIDM process.

CONCLUSION

Several factors have played different but significant roles in affecting the practice of EIDM among health service managers. Although interaction between these factors is complex, the framework developed in this paper may guide the development of strategies to encourage and improve the utilisation of evidence in management decision-making process.

Identifying and prioritizing strategies for comprehensive liver cancer control in Asia

BACKGROUND

Liver cancer is both common and burdensome in Asia. Effective liver cancer control, however, is hindered by a complex etiology and a lack of coordination across clinical disciplines. We sought to identify strategies for inclusion in a comprehensive liver cancer control for Asia and to compare qualitative and quantitative methods for prioritization.

METHODS

Qualitative interviews (N = 20) with international liver cancer experts were used to identify strategies using Interpretative Phenomenological Analysis and to formulate an initial prioritization through frequency analysis. Conjoint analysis, a quantitative stated-preference method, was then applied among Asian liver cancer experts (N = 20) who completed 12 choice tasks that divided these strategies into two mutually exclusive and exhaustive subsets. Respondents' preferred plan was the primary outcome in a choice model, estimated using ordinary least squares (OLS) and logistic regression. Priorities were then compared using Spearman's Rho.

RESULTS

Eleven strategies were identified: Access to treatments; Centers of excellence; Clinical education; Measuring social burden; Monitoring of at-risk populations; Multidisciplinary management; National guidelines; Public awareness; Research infrastructure; Risk-assessment and referral; and Transplantation infrastructure. Qualitative frequency analysis indicated that Risk-assessment and referral (85%), National guidelines (80%) and Monitoring of at-risk populations (80%) received the highest priority, while conjoint analysis pointed to Monitoring of at-risk populations (p < 0.001), Centers of excellence (p = 0.002), and Access to treatments (p = 0.004) as priorities, while Risk-assessment and referral was the lowest priority (p = 0.645). We find moderate concordance between the qualitative and quantitative methods (rho = 0.20), albeit insignificant (p = 0.554), and a strong concordance between the OLS and logistic regressions (rho = 0.979; p < 0.0001).

CONCLUSIONS

Identified strategies can be conceptualized as the ABCs of comprehensive liver cancer control as they focus on Antecedents, Better care and Connections within a national strategy. Some concordance was found between the qualitative and quantitative methods (e.g. Monitoring of at-risk populations), but substantial differences were also identified (e.g. qualitative methods gave highest priority to risk-assessment and referral, but it was the lowest for the quantitative methods), which may be attributed to differences between the methods and study populations, and potential framing effects in choice tasks. Continued research will provide more generalizable estimates of priorities and account for variation across stakeholders and countries.

The use of research evidence in public health decision making processes: systematic review

Background

The use of research evidence to underpin public health policy is strongly promoted. However, its implementation has not been straightforward. The objectives of this systematic review were to synthesise empirical evidence on the use of research evidence by public health decision makers in settings with universal health care systems.

Methods

To locate eligible studies, 13 bibliographic databases were screened, organisational websites were scanned, key informants were contacted and bibliographies of included studies were scrutinised. Two reviewers independently assessed studies for inclusion, extracted data and assessed methodological quality. Data were synthesised as a narrative review.

Findings

18 studies were included: 15 qualitative studies, and three surveys. Their methodological quality was mixed. They were set in a range of country and decision making settings. Study participants included 1063 public health decision makers, 72 researchers, and 174 with overlapping roles. Decision making processes varied widely between settings, and were viewed differently by key players. A range of research evidence was accessed. However, there was no reliable evidence on the extent of its use. Its impact was often indirect, competing with other influences. Barriers to the use of research evidence included: decision makers' perceptions of research evidence; the gulf between researchers and decision makers; the culture of decision making; competing influences on decision making; and practical constraints. Suggested (but largely untested) ways of overcoming these barriers included: research targeted at the needs of decision makers; research clearly highlighting key messages; and capacity building. There was little evidence on the role of research evidence in decision making to reduce inequalities.

Conclusions

To more effectively implement research informed public health policy, action is required by decision makers and researchers to address the barriers identified in this systematic review. There is an urgent need for evidence to support the use of research evidence to inform public health decision making to reduce inequalities.

Managing resources in NHS dentistry: using health economics to inform commissioning decisions

BACKGROUND

The aim of this study is to develop, apply and evaluate an economics-based framework to assist commissioners in their management of finite resources for local dental services. In April 2006, Primary Care Trusts in England were charged with managing finite dental budgets for the first time, yet several independent reports have since criticised the variability in commissioning skills within these organisations. The study will explore the views of stakeholders (dentists, patients and commissioners) regarding priority setting and the criteria used for decision-making and resource allocation. Two inter-related case studies will explore the dental commissioning and resource allocation processes through the application of a pragmatic economics-based framework known as Programme Budgeting and Marginal Analysis.

METHODS/DESIGN

The study will adopt an action research approach. Qualitative methods including semi-structured interviews, focus groups, field notes and document analysis will record the views of participants and their involvement in the research process. The first case study will be based within a Primary Care Trust where mixed methods will record the views of dentists, patients and dental commissioners on issues, priorities and processes associated with managing local dental services. A Programme Budgeting and Marginal Analysis framework will be applied to determine the potential value of economic principles to the decision-making process. A further case study will be conducted in a secondary care dental teaching hospital using the same approach. Qualitative data will be analysed using thematic analysis and managed using a framework approach.

DISCUSSION

The recent announcement by government regarding the proposed abolition of Primary Care Trusts may pose challenges for the research team regarding their engagement with the research study. However, whichever commissioning organisations are responsible for resource allocation for dental services in the future; resource scarcity is highly likely to remain an issue. Wider understanding of the complexities of priority setting and resource allocation at local levels are important considerations in the development of dental commissioning processes, national oral health policy and the future new dental contract which is expected to be implemented in April 2014.

Knowledge in process? Exploring barriers between epidemiological research and local health policy development

BACKGROUND

In the Netherlands municipalities are legally required to draw up a Local Health Policy Memorandum every four years. This policy memorandum should be based on (local) epidemiological research as performed by the Regional Health Services. However, it is largely unknown if and in what way epidemiological research is used during local policy development. As part of a larger study on knowledge utilization at the local level in The Netherlands, an analytical framework on the use of epidemiological research in local health policy development in the Netherlands is presented here.

METHOD

Based on a literature search and a short inventory on experiences from Regional Health Services, we made a description of existing research utilization models and concepts about research utilization. Subsequently we mapped different barriers in research transmission.

RESULTS

The interaction model is regarded as the main explanatory model. It acknowledges the interactive and incremental nature of policy development, which takes place in a context and includes diversity within the groups of researchers and policymakers. This fits well in the dynamic and complex setting of local Dutch health policy.For the conceptual framework we propose a network approach, in which we "extend" the interaction model. We not only focus on the one-to-one relation between an individual researcher and policymaker but include interactions between several actors participating in the research and policy process.In this model interaction between actors in the research and the policy network is expected to improve research utilization. Interaction can obstruct or promote four clusters of barriers between research and policy: expectations, transfer issues, acceptance, and interpretation. These elements of interactions and barriers provide an actual explanation of research utilization. Research utilization itself can be measured on the individual level of actors and on a policy process level.

CONCLUSION

The developed framework has added value on existing models on research utilization because it emphasizes on the 'logic' of the context of the research and policy networks. The framework will contribute to a better understanding of the impact of epidemiological research in local health policy development, however further operationalisation of the concepts mentioned in the framework remains necessary.

Patient choice and evidence based decisions: The case of complementary therapies

OBJECTIVE

Current government policies simultaneously pursue the development of 'patient-led' and 'evidence-based' approaches to healthcare. The objective of this study was to explore how primary care clinicians and Primary Care Trust (PCT) managers balance these potentially competing tensions when considering popular, controversial treatments, like complementary therapies, in consultations (clinicians) or funding decisions (PCT managers).

SETTING AND PARTICIPANTS

We selected two case sites where complementary therapies were offered on NHS premises in England. We interviewed 18 PCT managers and clinicians, conducted an observation of a PCT meeting on complementary therapies and collected documentary data from referral databases and service funding bids. All interviews were taped, transcribed and analysed thematically. Interview, observation and documentary data were used to compare reported beliefs and behaviour to observed and documented behaviour.

RESULTS

The majority of clinicians and PCT managers claimed that research evidence guided their decisions; those who did not felt increasingly marginalized. However, discrepancies between reported and observed behaviour suggest that perceptions of research evidence, rather than fact based knowledge, predominated when considering complementary therapies.

CONCLUSION

In the case of NHS complementary therapy service provision, patient preference may be largely insignificant in clinician and PCT managerial decisions, with decisions based mainly on 'evidence rhetoric' devised from collectively agreed, unchallenged, tacit perceptions of research literature. If a patient-led NHS is to become a reality, NHS professionals need to cede the power that they wield with evidence rhetoric and acknowledge the legitimacy of patient preferences, views and alternative sources of evidence.

Priority setting: what constitutes success? A conceptual framework for successful priority setting

Background

The sustainability of healthcare systems worldwide is threatened by a growing demand for services and expensive innovative technologies. Decision makers struggle in this environment to set priorities appropriately, particularly because they lack consensus about which values should guide their decisions. One way to approach this problem is to determine what all relevant stakeholders understand successful priority setting to mean. The goal of this research was to develop a conceptual framework for successful priority setting.

Methods

Three separate empirical studies were completed using qualitative data collection methods (one-on-one interviews with healthcare decision makers from across Canada; focus groups with representation of patients, caregivers and policy makers; and Delphi study including scholars and decision makers from five countries).

Results

This paper synthesizes the findings from three studies into a framework of ten separate but interconnected elements germane to successful priority setting: stakeholder understanding, shifted priorities/reallocation of resources, decision making quality, stakeholder acceptance and satisfaction, positive externalities, stakeholder engagement, use of explicit process, information management, consideration of values and context, and revision or appeals mechanism.

Conclusion

The ten elements specify both quantitative and qualitative dimensions of priority setting and relate to both process and outcome components. To our knowledge, this is the first framework that describes successful priority setting. The ten elements identified in this research provide guidance for decision makers and a common language to discuss priority setting success and work toward improving priority setting efforts.

Commissioning NHS dentistry in England: issues for decision-makers managing the new contract with finite resources

OBJECTIVES

To explore the views of dental decision-makers in Primary Care Organisations with regard to the management of NHS dental services, and to gauge participants' awareness of economics-based approaches including programme budgeting and marginal analysis, with which to potentially structure commissioning decisions.

METHODS

Recorded semi-structured interviews were conducted with 18 NHS dental decision-makers (mixed clinical and finance backgrounds) predominantly across Primary Care Trusts in England. Data were analysed using qualitative methods and the constant comparative approach.

RESULTS

Participants were generally involved with contracting rather than commissioning new dental services at the time of interview. It was unclear how oral health needs assessments would guide future resource shifts and how commissioners would ensure the efficient use of finite resources. Whilst many participants thought that economic approaches would assist their commissioning decisions, few participants were aware of programme budgeting and marginal analysis as an alternative economics-based approach.

CONCLUSIONS

An assessment of the extent to which finite resources actually maximise the oral health of local populations is timely. Pragmatic economic approaches such as programme budgeting and marginal analysis may offer a framework to guide decision-makers through commissioning and the stages which lie beyond oral health needs assessments.

Factors that influence case managers' resource allocation decisions in pediatric home care: an ethnographic study

BACKGROUND

Case managers make decisions that directly affect the amount and type of services home care clients receive and subsequently affect the overall available health care resources of home care programs. A recent systematic review of the literature identified significant knowledge gaps with respect to resource allocation decision-making in home care.

METHODS

Using Spradley's methodology, we designed an ethnographic study of a children's home care program in Western Canada. The sample included 11 case managers and program leaders. Data sources included interviews, card sorts, and participant observation over a 5-month period. Data analyses included open coding, domain, taxonomic, and componential analysis.

RESULTS

One of the key findings was a taxonomy of factors that influence case manager resource allocation decisions. The factors were grouped into one of four main categories: system-related, home care program-related, family related, or client-related. Family related factors have not been previously reported as influencing case manager resource allocation decision-making and nor has the team's role been reported as an influencing factor.

CONCLUSION

The findings of this study are examined in light of Daniels and Sabin's Accountability for Reasonableness framework, which may be useful for future knowledge development about micro-level resource allocation theory.

Knowledge transfer and exchange: review and synthesis of the literature

Knowledge transfer and exchange (KTE) is as an interactive process involving the interchange of knowledge between research users and researcher producers. Despite many strategies for KTE, it is not clear which ones should be used in which contexts. This article is a review and synthesis of the KTE literature on health care policy. The review examined and summarized KTE's current evidence base for KTE. It found that about 20 percent of the studies reported on a real-world application of a KTE strategy, and fewer had been formally evaluated. At this time there is an inadequate evidence base for doing "evidence-based" KTE for health policy decision making. Either KTE must be reconceptualized, or strategies must be evaluated more rigorously to produce a richer evidence base for future activity.

Research-based-decision-making in Canadian health organizations: a behavioural approach

Decision making in Health sector is affected by a several elements such as economic constraints, political agendas, epidemiologic events, managers' values and environment... These competing elements create a complex environment for decision making. Research-Based-Decision-Making (RBDM) offers an opportunity to reduce the generated uncertainty and to ensure efficacy and efficiency in health administrations. We assume that RBDM is dependant on decision makers' behaviour and the identification of the determinants of this behaviour can help to enhance research results utilization in health sector decision making. This paper explores the determinants of RBDM as a personal behaviour among managers and professionals in health administrations in Canada. From the behavioural theories and the existing literature, we build a model measuring "RBDM" as an index based on five items. These items refer to the steps accomplished by a decision maker while developing a decision which is based on evidence. The determinants of RBDM behaviour are identified using data collected from 942 health care decision makers in Canadian health organizations. Linear regression is used to model the behaviour RBDM. Determinants of this behaviour are derived from Triandis Theory and Bandura's construct "self-efficacy." The results suggest that to improve research use among managers in Canadian governmental health organizations, strategies should focus on enhancing exposition to evidence through facilitating communication networks, partnerships and links between researchers and decision makers, with the key long-term objective of developing a culture that supports and values the contribution that research can make to decision making in governmental health organizations. Nevertheless, depending on the organizational level, determinants of RBDM are different. This difference has to be taken into account if RBDM adoption is desired. Decision makers in Canadian health organizations (CHO) can help to build networks, develop partnerships between professionals locally, regionally and nationally, and also act as change agents in the dissemination and adoption of knowledge and innovations in health services. However, the research focused on knowledge use as a support to decision-making, further research is needed to identify and evaluate effective incentives and strategies to implement so as to enhance RBDM adoption among health decision makers and more theoretical development are to complete in this perspective.

Priority setting in the provincial health services authority: survey of key decision makers

Background

In recent years, decision makers in Canada and elsewhere have expressed a desire for more explicit, evidence-based approaches to priority setting. To achieve this aim within health care organizations, knowledge of both the organizational context and stakeholder attitudes towards priority setting are required. The current work adds to a limited yet growing body of international literature describing priority setting practices in health organizations.

Methods

A qualitative study was conducted using in-depth, face-to-face interviews with 25 key decision makers of the Provincial Health Services Authority (PHSA) of British Columbia. Major themes and sub-themes were identified through content analysis.

Results

Priorities were described by decision makers as being set in an ad hoc manner, with resources generally allocated along historical lines. Participants identified the Strategic Plan and a strong research base as strengths of the organization. The main areas for improvement were a desire to have a more transparent process for priority setting, a need to develop a culture which supports explicit priority setting, and a focus on fairness in decision making. Barriers to an explicit allocation process included the challenge of providing specialized services for disparate patient groups, and a lack of formal training in priority setting amongst decision makers.

Conclusion

This study identified factors important to understanding organizational context and informed next steps for explicit priority setting for a provincial health authority. While the PHSA is unique in its organizational structure in Canada, lessons about priority setting should be transferable to other contexts.

Managing to manage healthcare resources in the English NHS? What can health economics teach? What can health economics learn?

OBJECTIVES

To provide a 'thick description' of how decision-makers understand and manage healthcare prioritisation decisions, and to explore the potential for using economic frameworks in the context of the NHS in England.

METHODS

Interviews were conducted with 22 key decision-makers from six Primary Care Trusts (PCTs) in northern England. A constant comparative approach was used to identify broad themes and sub-themes.

RESULTS

Six broad themes emerged from the analysis. In summary, decision-makers recognised the concepts of resources scarcity, competing claims, and the need for choices and trade-offs to be made. Decision-makers even went on to identify a common set of principles that ought to guide commissioning decisions. However, the process of commissioning was dominated by political, historical and clinical methods of commissioning which, failed to recognise these concepts in practice, and departed from the principles. As a result, the commissioning process was viewed as not being systematic or transparent and, therefore, seen as underperforming.

CONCLUSIONS

Health economists need to acknowledge the importance of contextual factors and the realities of priority setting. Our research suggests that the emphasis should be on integrating principles of economics into a management process rather than expecting decision-makers to apply the output of ever more seemingly 'technically sound' health economic methods which cannot reflect the dominating and driving complexities of the commissioning process.

Resource allocation in orthopaedics: economic evaluation to priority setting

In 2000 the World Health Organization launched the "Bone and Joint Decade" campaign in part to promote cost-effective treatments. This will impact the organization, delivery, and funding of health care as the population ages. However, it is well recognized that resources in health care are limited and it is essential the resources available are used to best effect. Thus, there has been greater emphasis in the orthopaedic literature on the use of economics. Still, there is little discussion of whether and how the results of these economic methods can be further used to aid resource allocation decisions. We discuss the suitability of economic methods for priority setting in orthopaedic surgery, arguing economic evaluation alone is not sufficient for addressing resource allocation decisions. We also describe an alternative approach to priority setting that has been steadily gaining prominence within health economics--program budgeting and marginal analysis--and use a working example from the United Kingdom National Health Service to illustrate its application within orthopaedic surgery.

State-of-the-evidence reviews: advantages and challenges of including grey literature

BACKGROUND

Increasingly, health policy decision-makers and professionals are turning to research-based evidence to support decisions about policy and practice. Systematic reviews are useful for gathering, summarizing, and synthesizing published and unpublished research about clearly defined interventions. State-of-the-evidence reviews are broader than traditional systematic reviews and may include not only published and unpublished research, but also published and unpublished non-research literature. Decisions about whether to include this "grey literature" in a review are challenging and lead to many questions about whether the advantages outweigh the challenges.

AIMS

The primary purpose of this article is to describe what constitutes grey literature, and methods to locate it and assess its quality. The secondary purpose is to discuss the core issues to consider when making decisions to include grey literature in a state-of-the-evidence review.

METHODS

A recent state-of-the-evidence review is used as an exemplar to present advantages and challenges related to including grey literature in a review.

RESULTS

Despite the challenges, in the exemplar, inclusion of grey literature was useful to validate the results of a research-based literature search.

CONCLUSION

Decisions about whether to include grey literature in a state-of-the-evidence review are complex. A checklist to assist in decision-making was created as a tool to assist the researcher in determining whether it is advantageous to include grey literature in a review.

Informing Research in Primary Dental Care: Setting Priorities

This paper suggests that priorities for research in primary dental care should follow the examples set in other areas of primary healthcare. It reviews the history of research in primary dental care, since 1990, and goes on to explain how the Delphi exercise, initiated by the Faculty of General Dental Practice (UK), has identified five priority areas for research in primary dental care. These areas are:

1. Research into the application of evidence-based dentistry into practice.

2. The effects of different systems of remuneration on treatment patterns in practice.

3. The oral health assessment on determining recall intervals and its effect on oral health.

4. Factors that influence and affect dentists’ treatment modalities.

5. The evaluation of the cost benefits of whole team training.

Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field

Policy-makers and managers have always used a wide range of sources of evidence in making decisions about policy and the organization of services. However, they are under increasing pressure to adopt a more systematic approach to the utilization of the complex evidence base. Decision-makers must address complicated questions about the nature and significance of the problem to be addressed; the nature of proposed interventions; their differential impact; cost-effectiveness; acceptability and so on. This means that Cochrane-style reviews alone are not sufficient. Rather, they require access to syntheses of high-quality evidence that include research and non-research sources, and both qualitative and quantitative research findings. There is no single, agreed framework for synthesizing such diverse forms of evidence and many of the approaches potentially applicable to such an endeavour were devised for either qualitative or quantitative synthesis and/or for analysing primary data. This paper describes the key stages in reviewing and synthesizing qualitative and quantitative evidence for decision-making and looks at various strategies that could offer a way forward. We identify four basic approaches: narrative (including traditional 'literature reviews' and more methodologically explicit approaches such as 'thematic analysis', 'narrative synthesis', 'realist synthesis' and 'meta-narrative mapping'), qualitative (which convert all available evidence into qualitative form using techniques such as 'meta-ethnography' and 'qualitative cross-case analysis'), quantitative (which convert all evidence into quantitative form using techniques such as 'quantitative case survey' or 'content analysis') and Bayesian meta-analysis and decision analysis (which can convert qualitative evidence such as preferences about different outcomes into quantitative form or 'weights' to use in quantitative synthesis). The choice of approach will be contingent on the aim of the review and nature of the available evidence, and often more than one approach will be required.

Economic evaluation: what does a nurse manager need to know?

This paper considers how health economists can assist nurse managers, using the concepts and tools of economic evaluation. We aim to clarify these and also explode some of the myths about economic evaluation and its role in health care decision-making. Economic evaluation techniques compare alternative courses of action in terms of their costs and consequences. There are four principal methods; cost-minimization, cost-effectiveness, cost-utility and cost-benefit analysis, all of which synthesize costs and outcomes, at different levels of outcome. Economic evaluation is an intrinsic part of national decision-making about the efficient provision of effective treatments and services, and increasingly, organizational matters. In the UK, such technology evaluation is disseminated in guidelines from the National Institute for Clinical Effectiveness (NICE), having a top-down impact on the nurse manager. But economic evaluation is increasingly relevant to the nurse manager at local level, through newer techniques such as Programme Budgeting Marginal Analysis (PBMA), which facilitates explicit, transparent decisions, from the bottom-up. Nurse managers need to weigh up competing demands on resources and decide in ways which maximize health gain. Economic evaluation can help here because it presents evidence to challenge or support existing allocations, and provides a systematic framework to analyse health care decisions. In the current context of competition for scarce resources, we suggest that nurse managers need to embrace these techniques, or be marginalized from the resource allocation process.