International validation of quality indicators for evaluating priority setting in low income countries: process and key lessons

Priority setting in times of crises: an analysis of priority setting for the COVID-19 response in the Western Pacific Region

BACKGROUND

While priority setting is recognized as critical for promoting accountability and transparency in health system planning, its role in supporting rational, equitable and fair pandemic planning and responses is less well understood. This study aims to describe how priority setting was used to support planning in the initial stage of the pandemic response in a subset of countries in the Western Pacific Region (WPR).

METHODS

We purposively sampled a subset of countries from WPR and undertook a critical document review of the initial national COVID-19 pandemic response plans. A pre-specified tool guided data extraction and the analysis examined the use of quality parameters of priority setting, and equity considerations.

RESULTS

Nine plans were included in this analysis, from the following countries: Papua New Guinea, Tonga, The Philippines, Fiji, China, Australia, New Zealand, Japan, and Taiwan. Most commonly the plans described strong political will to respond swiftly, resource needs, stakeholder engagement, and defined the roles of institutions that guided COVID-19 response decision-making. The initial plans did not reflect strong evidence of public engagement or considerations of equity informing the early responses to the pandemic.

CONCLUSION

This study advances an understanding of how priority setting and equity considerations were integrated to support the development of the initial COVID-19 responses in nine countries in WPR and contributes to the literature on health system planning during emergencies. This baseline assessment reveals evidence of the common priority setting parameters that were deployed in the initial responses, the prioritized resources and equity considerations and reinforces the importance of strengthening health system capacity for priority setting to support future pandemic preparedness.

Stakeholder participation in the COVID-19 pandemic preparedness and response plans: A synthesis of findings from 70 countries

Stakeholder participation is a key component of a fair and equitable priority-setting in health. The COVID-19 pandemic highlighted the need for fair and equitable priority setting, and hence, stakeholder participation. To date, there is limited literature on stakeholder participation in the development of the pandemic plans (including the priority setting plans) that were rapidly developed during the pandemic. Drawing on a global study of national COVID-19 preparedness and response plans, we present a secondary analysis of COVID-19 national plans from 70 countries from the six WHO regions, focusing on stakeholder participation. We found that most plans were prepared by the Ministry of Health and acknowledged WHO guidance, however less than half mentioned that additional stakeholders were involved. Few plans described a strategy for stakeholder participation and/or accounted for public participation in the plan preparation. However, diverse stakeholders (including multiple governmental, non-governmental, and international organizations) were proposed to participate in the implementation of the plans. Overall, there was a lack of transparency about who participated in decision-making and limited evidence of meaningful participation of the community, including marginalized groups. The critical relevance of stakeholder participation in priority setting requires that governments develop strategies for meaningful participation of diverse stakeholders during pandemics such as COVID-19, and in routine healthcare priority setting.

A global comparative analysis of the the inclusion of priority setting in national COVID-19 pandemic plans: A reflection on the methods and the accessibility of the plans

BACKGROUND

Despite the swift governments' response to the COVID-19 pandemic, there remains a paucity of literature assessing the degree to which; priority setting (PS) was included in the pandemic plans and the pandemic plans were publicly accessible. This paper reflects on the methods employed in a global comparative analysis of the degree to which countries integrated PS into their COVID-19 pandemic plans based on Kapiriri & Martin's framework. We also assessed if the accessibility of the plans was related to the country's transparency index.

METHODS

Through a three stage search strategy, we accessed and reviewed 86 national COVID-19 pandemic plans (and 11 Canadian provinces and territories). Secondary analysis assessed any alignment between the readily accessible plans and the country's transparency index.

RESULTS AND CONCLUSION

71 national plans were readily accessible while 43 were not. There were no systematic differences between the countries whose plans were readily available and those whose plans were 'missing'. However, most of the countries with 'missing' plans tended to have a low transparency index. The framework was adapted to the pandemic context by adding a parameter on the need to plan for continuity of priority routine services. While document review may be the most feasible and appropriate approach to conducting policy analysis during health emergencies, interviews and follow up document review would assess policy implementation.

A global comparative analysis of the criteria and equity considerations included in eighty-six national COVID-19 plans

Systematic priority setting (PS), based on explicit criteria, is thought to improve the quality and consistency of the PS decisions. Among the PS criteria, there is increased focus on the importance of equity considerations and vulnerable populations. This paper discusses the PS criteria that were included in the national COVID-19 pandemic plans, with specific focus on equity and on the vulnerable populations considered. Secondary synthesis of data, from a global comparative study that examined the degree to which the COVID-19 plans included PS, was conducted. Only 32 % of the plans identified explicit criteria. Severity of the disease and/or disease burden were the commonly mentioned criteria. With regards to equity considerations and prioritizing vulnerable populations, 22 countries identified people with co-morbidities others mentioned children, women etc. Low social-economic status and internally displaced population were not identified in any of the reviewed national plans. The limited inclusion of explicit criteria and equity considerations highlight a need for policy makers, in all contexts, to consider instituting and equipping PS institutions who can engage diverse stakeholders in identifying the relevant PS criteria during the post pandemic period. While vulnerability will vary with the type of health emergency- awareness of this and having mechanisms for identifying and prioritizing the most vulnerable will support equitable pandemic responses.

Assessing the interactions of people and policy-makers in social participation for health: an inventory of participatory governance measures from a rapid systematic literature review

Social participation, also termed stakeholder voice, is an important component of health system governance. Increased interactions between the community and policy makers could facilitate a more responsive health system that targets the needs of the community better. Recently, the World Health Organization (WHO) published a handbook on social participation that identified five key themes for ministries of health to consider when engaging the input of the community. In this rapid systematic literature review, we aimed to identify quantitative and qualitative measures that have been used to assess aspects of social participation involving people and policy makers. We identified 172 measures from 48 studies from countries in all six WHO regions. These measures were categorized by all five themes from the handbook on social participation and these measures are linked to 27 concepts. This rapid review found that the focus of measures is largely on the existence of participation-be it by the general population or specific vulnerable groups-rather than on the quality of their participation. The measures in this inventory may be useful for ministries of health and other key stakeholders to use when developing methods to assess and encourage social participation in their context.

Was priority setting included in the Canadian COVID-19 pandemic planning and preparedness? A comparative analysis of COVID-19 pandemic plans from Eight provinces and Three territories

Background

Variation in priorities during pandemic planning among the federal, provincial and territorial jurisdictions are thought to have impacted Canada's ability to effectively control the spread of the COVID-19 virus, and protect the most vulnerable. The potential influence of diverse and divergent political, cultural, and behavioural factors, regarding inclusion of priority setting (PS) in pandemic preparedness planning across the country is not well understood. This study aimed to examine how the Canadian federal, provincial and territorial COVID-19 pandemic preparedness planning documents integrated PS.

Methods

A documentary analysis of the federal, eight provincial, three territorial COVID-19 preparedness and response plans. We assessed the degree to which the documented PS processes fulfilled established quality requirements of effective PS using the Kapiriri & Martin framework.

Results

While the federal plan included most of the parameters of effective PS, the provinces and territories reflected few. The lack of obligation for the provinces and territories to emulate the federal plan is one of the possible reasons for the varying inclusion of these parameters. The parameters included did not vary systematically with the jurisdiction's context.

Conclusion

Provinces could consider using the framework of the federal plan and the WHO guidelines to guide future pandemic planning. Regular evaluation of the instituted PS would provide a mechanism through which lessons can be harnessed and improvement strategies developed. Future studies should describe and evaluate what PS mechanisms were implemented.

Priority setting for pandemic preparedness and response: A comparative analysis of COVID-19 pandemic plans in 12 countries in the Eastern Mediterranean Region

Background

The COVID-19 pandemic has significantly disrupted health systems and exacerbated pre-existing resource gaps in the Eastern Mediterranean Region (WHO-EMRO). Active humanitarian and refugee crises have led to mass population displacement and increased health system fragility, which has implication for equitable priority setting (PS). We examine whether and how PS was included in national COVID-19 pandemic plans within EMRO.

Methods

An analysis of COVID-19 pandemic response and preparedness planning documents from a sample of 12/22 countries in WHO-EMRO. We assessed the degree to which documented PS processes adhere to twenty established quality parameters of effective PS.

Results

While all reviewed plans addressed some aspect of PS, none included all quality parameters. Yemen's plan included the highest number (9) of quality parameters, while Egypt's addressed the lowest (3). Most plans used evidence in their planning processes. While no plans explicitly identify equity as a criterion to guide PS; many identified vulnerable populations - a key component of equitable PS. Despite high concentrations of refugees, migrants, and IDPs in EMRO, only a quarter of the plans identified them as vulnerable.

Conclusion

PS setting challenges are exacerbated by conflict and the resulting health system fragmentation. Systematic and quality PS is essential to tackle long-term health implications of COVID-19 for vulnerable populations in this region, and to support effective PS and equitable resource allocation.

Examining priority setting in the national COVID-19 pandemic plans: A case study from countries in the WHO- South-East Asia Region (WHO-SEARO)

Background

The World Health Organization- South-East Asia Region (WHO-SEARO) accounted for almost 17% of all the confirmed cases and deaths of COVID-19 worldwide. While the literature has documented a weak COVID-19 response in the WHO-SEARO, there has been no discussion of the degree to which this could have been influenced/ mitigated with the integration of priority setting (PS) in the region's COVID-19 response. The purpose of this paper is to describe the degree to which the COVID-19 plans from a sample of WHO-SEARO countries included priority setting.

Methods

The study was based on an analysis of national COVID-19 pandemic response and preparedness planning documents from a sample of seven (of the eleven) countries in WHO-SEARO. We described the degree to which the documented priority setting processes adhered to twenty established quality indicators of effective PS and conducted a cross-country comparison.

Results

All of the reviewed plans described the required resources during the COVID-19 pandemic. Most, but not all of the plans demonstrated political will, and described stakeholder involvement. However, none of the plans presented a clear description of the PS process including a formal PS framework, and PS criteria. Overall, most of the plans included only a limited number of quality indicators for effective PS.

Discussion and conclusion

There was wide variation in the parameters of effective PS in the reviewed plans. However, there were no systematic variations between the parameters presented in the plans and the country's economic, health system and pandemic and PS context and experiences. The political nature of the pandemic, and its high resource demands could have influenced the inclusion of the parameters that were apparent in all the plans. The finding that the plans did not include most of the evidence-based parameters of effective PS highlights the need for further research on how countries operationalize priority setting in their respective contexts as well as deeper understanding of the parameters that are deemed relevant. Further research should explore and describe the experiences of implementing defined priorities and the impact of this decision-making on the pandemic outcomes in each country.

An analysis of how health systems integrated priority-setting in the pandemic planning in a sample of Latin America and the Caribbean countries

BACKGROUND

Latin America and the Caribbean (LAC) are among those regions most affected by the COVID-19 pandemic worldwide. The COVID-19 pandemic has strained health systems in the region. In this context of severe healthcare resource constraints, there is a need for systematic priority-setting to support decision-making which ensures the best use of resources while considering the needs of the most vulnerable groups. The aim of this paper was to provide a critical description and analysis of how health systems considered priority-setting in the COVID-19 response and preparedness plans of a sample of 14 LAC countries; and to identify the associated research gaps.

METHODS

A documentary analysis of COVID-19 preparedness and response plans was performed in a sample of 14 countries in the LAC region. We assessed the degree to which the documented priority-setting processes adhered to established quality indicators of effective priority-setting included in the Kapiriri and Martin framework. We conducted a descriptive analysis of the degree to which the reports addressed the quality parameters for each individual country, as well as a cross-country comparison to explore whether parameters varied according to independent variables.

RESULTS

While all plans were led and supported by the national governments, most included only a limited number of quality indicators for effective priority-setting. There was no systematic pattern between the number of quality indicators and the country's health system and political contexts; however, the countries that had the least number of quality indicators tended to be economically disadvantaged.

CONCLUSION

This study adds to the literature by providing the first descriptive analysis of the inclusion of priority-setting during a pandemic, using the case of COVID-19 response and preparedness plans in the LAC region. The analysis found that despite the strong evidence of political will and stakeholder participation, none of the plans presented a clear priority-setting process, or used a formal priority-setting framework, to define interventions, populations, geographical regions, healthcare setting or resources prioritized. There is need for case studies that analyse how priority-setting actually occurred during the COVID-19 pandemic and the degree to which the implementation reflected the plans and the parameters of effective priority-setting, as well as the impact of the prioritization processes on population health, with a focus on the most vulnerable groups.

Priority Setting and Equity in COVID-19 Pandemic Plans: A Comparative Analysis of eighteen African Countries

Priority setting represents an even bigger challenge during public health emergencies than routine times. This is because such emergencies compete with routine programmes for the available health resources, strain health systems and shift health-care attention and resources towards containing the spread of the epidemic and treating those that fall seriously ill. This paper is part of a larger global study, the aim of which is to evaluate the degree to which national COVID-19 preparedness and response plans incorporated priority setting concepts. It provides important insights into what and how priority decisions were made in the context of a pandemic. Specifically, with a focus on a sample of 18 African countries’ pandemic plans, the paper aims to: (1) explore the degree to which the documented priority setting processes adhere to established quality indicators of effective priority setting and (2) examine if there is a relationship between the number of quality indicators present in the pandemic plans and the country’s economic context, health system and prior experiences with disease outbreaks. All the reviewed plans contained some aspects of expected priority setting processes but none of the national plans addressed all quality parameters. Most of the parameters were mentioned by less than 10 of the 18 country plans reviewed, and several plans identified one or two aspects of fair priority setting processes. Very few plans identified equity as a criterion for priority setting. Since the parameters are relevant to the quality of priority setting that is implemented during public health emergencies and most of the countries have pre-existing pandemic plans; it would be advisable that, for the future (if not already happening), countries consider priority setting as a critical part of their routine health emergency and disease outbreak plans. Such an approach would ensure that priority setting is integral to pandemic planning, response and recovery.

Priority setting for health system strengthening in low income countries. A qualitative case study illustrating the complexities

Health systems are critical to the realisation of Universal Health Coverage. There has been insufficient attention to the evaluation of priority setting for health system strengthening within low income countries, including evaluation of the local capacity to implement priorities. This study evaluated the extent to which health system strengthening was prioritized in Uganda. The Kapiriri & Martin framework was used to evaluate health system priority setting from 2005-2015. A document analysis was triangulated with interview data (n = 67) from global, national and subnational stakeholders and analysed using content analysis. Health system strengthening was perceived to be circumvented by a lack of resources as well as influential actors with disease focused, rather than system-oriented, interests. There were defined processes with explicit criteria for identifying priorities and evidence was highly valued. But sub-optimal transparency and weak accountability often compromised the integrity of priority setting and contributed to stalling progress on health system strengthening and achieving health system outcomes. The strengths in the current planning processes should be harnessed. In addition, a systematic approach to priority setting, potentially through the establishment of an independent body, and stronger oversight mechanisms, would strengthen health system planning in this setting.

Implementing evidence-informed deliberative processes in health technology assessment: a low income country perspective

The purpose of this paper is to discuss the potential feasibility and utility of evidence-informed deliberative processes (EPDs) in low income country (LIC) contexts. EDPs are implemented in high and middle income countries and thought to improve the quality, consistency, and transparency of decisions informed by health technology assessment (HTA). Together these would ultimately improve the legitimacy of any decision making process. We argue-based on our previous work and in light of the priority setting literature-that EDPs are relevant and feasible within LICs. The extreme lack of resources necessitates making tough decisions which may mean depriving populations of potentially valuable health technologies. It is critical that the decisions and the decision making bodies are perceived as fair and legitimate by the people that are most affected by the decisions. EDPs are well aligned with the political infrastructure in some LICs, which encourages public participation in decision making. Furthermore, many countries are committed to evidence-informed decision making. However, the application of EDPs may be hampered by the limited availability of evidence of good quality, lack of interest in transparency and accountability (in some LICs), limited capacity to conduct HTA, as well as limited time and financial resources to invest in a deliberative process. While EDPs would potentially benefit many LICs, mitigating the identified potential barriers would strengthen their applicability. We believe that implementation studies in LICs, documenting the contextualized enablers and barriers will facilitate the development of context specific improvement strategies for EDPs.

Priority setting for maternal, newborn and child health in Uganda: a qualitative study evaluating actual practice

Background

Despite continued investment, Maternal, Newborn and Child Health (MNCH) indicators in low and middle income countries have remained relatively poor. This could, in part, be explained by inadequate resources to adequately address these problems, inappropriate allocation of the available resources, or lack of implementation of the most effective interventions. Systematic priority setting and resource allocation could contribute to alleviating these limitations. There is a paucity of literature that follows through MNCH prioritization processes to implementation, making it difficult for policy makers to understand the impact of their decision-making on population health. The overall objective of this paper was to describe and evaluate priority setting for maternal, newborn and child health interventions in Uganda.

Methods

Fifty-four key informant interviews and a review of policies and media reports were used to describe priority setting for MNCH in Uganda. Kapiriri and Martin’s conceptual framework was used to evaluate priority setting for MNCH.

Results

There were three main prioritization exercises for maternal, newborn and child health in Uganda. The processes were participatory and were guided by explicit tools, evidence, and criteria, however, the public and the districts were insufficiently involved in the priority setting process. While there were conducive contextual factors including strong political support, implementation was constrained by the presence of competing actors, with varying priorities, an unequal allocation of resources between child health and maternal health interventions, limited financial and human resources, a weak health system and limited institutional capacity.

Conclusions

Stronger institutional capacity at the Ministry of Health and equitable engagement of key stakeholders in decision-making processes, especially the public, and implementers, would improve understanding, satisfaction and compliance with the priority setting process. Availability of financial and human resources that are appropriately allocated would facilitate the implementation of well-developed policies.

Electronic supplementary material

The online version of this article (10.1186/s12913-019-4170-6) contains supplementary material, which is available to authorized users.

Beyond cost-effectiveness, morbidity and mortality: a comprehensive evaluation of priority setting for HIV programming in Uganda

BACKGROUND

While there has been progress in controlling the HIV epidemic, HIV still remains a disease of global concern. Some of the progress has been attributed to increased public awareness and uptake of public health interventions, as well as increased access to anti- retroviral treatment and the prevention of vertical HIV transmission. These interventions would not have been possible without substantial investments in HIV programs. However, donor fatigue introduces the need for low income countries to maximize the benefits of the available resources. This necessitates identification of priorities that should be funded. Evaluating prioritization processes would enable decision makers to assess the effectiveness of their processes, thereby designing intervention strategies. To date most evaluations have focused on cost-benefit analyses, which overlooks additional critical impacts of priority setting decisions. Kapiriri & Martin (2010) developed and validated a comprehensive framework for evaluating PS in low income countries. The objective of this paper report findings from a comprehensive evaluation of priority setting for HIV in Uganda, using the framework; and to identify lessons of good practice and areas for improvement.

METHODS

This was a qualitative study based on forty interviews with decision makers and policy document review. Data were analysed using INVIVO 10, and based on the parameters in Kapiriri et al's evaluation framework.

RESULTS

We found that HIV enjoys political support, which contributes to the availability of resources, strong planning institutions, and participatory prioritization process based on some criteria. Some of the identified limitations included; undue donor and political influence, priorities not being publicized, and lack of mechanisms for appealing the decisions. HIV prioritization had both positive and negative impacts on the health system.

CONCLUSIONS

The framework facilitated a more comprehensive evaluation of HIV priority setting. While there were successful areas, the process could be strengthened by minimizing undue influence of external actors, and support the legitimate institutions to set priorities and implement them. These should also institute mechanisms for publicizing the decisions, appeals and increased accountability. While this paper looked at HIV, the framework is flexible enough to be used in evaluating priority setting for other health programs within similar context.

Priority setting for disease outbreaks in Uganda: A case study evaluating the process

Priority setting (PS) and resource allocation during health emergencies are key factors influencing an effective response. However, there is limited understanding of how priorities and resource allocation during disease outbreaks occur and the extent to which these processes are successful. This paper, based on 23 in-depth interviews with policy makers and a review of policy and emergency preparedness documents, used a PS evaluation framework to evaluate PS for disease outbreaks in Uganda. With regard to PS for disease outbreaks in Uganda, we identified a conducive socio-political-economical context, credible institutions, formal participatory prioritisation processes, evidence informed the processes, demonstrated implementation capacity, institutional strengthening and positive health outcomes. Factors that compromised the success of PS included limited resources - especially in between disease outbreaks and unfair processes. Investment in sustaining the established prioritisation infrastructure to oversee preparedness activities between the outbreaks would strengthen the prioritisation process. This should be supported with health system strengthening. The framework enabled us to evaluate some aspects of PS during disease outbreaks. The framework's inability to evaluate all aspects, and reported as opposed to actual PS calls for the integration of evaluation throughout the planning and implementation process to ensure validity and continuous implementation of improvement strategies.

The unfunded priorities: an evaluation of priority setting for noncommunicable disease control in Uganda

BACKGROUND

The double burden of infectious diseases coupled with noncommunicable diseases poses unique challenges for priority setting and for achieving equitable action to address the major causes of disease burden in health systems already impacted by limited resources. Noncommunicable disease control is an important global health and development priority. However, there are challenges for translating this global priority into local priorities and action. The aim of this study was to evaluate the influence of national, sub-national and global factors on priority setting for noncommunicable disease control in Uganda and examine the extent to which priority setting was successful.

METHODS

A mixed methods design that used the Kapiriri & Martin framework for evaluating priority setting in low income countries. The evaluation period was 2005-2015. Data collection included a document review (policy documents (n = 19); meeting minutes (n = 28)), media analysis (n = 114) and stakeholder interviews (n = 9). Data were analysed according to the Kapiriri & Martin (2010) framework.

RESULTS

Priority setting for noncommunicable diseases was not entirely fair nor successful. While there were explicit processes that incorporated relevant criteria, evidence and wide stakeholder involvement, these criteria were not used systematically or consistently in the contemplation of noncommunicable diseases. There were insufficient resources for noncommunicable diseases, despite being a priority area. There were weaknesses in the priority setting institutions, and insufficient mechanisms to ensure accountability for decision-making. Priority setting was influenced by the priorities of major stakeholders (i.e. development assistance partners) which were not always aligned with national priorities. There were major delays in the implementation of noncommunicable disease-related priorities and in many cases, a failure to implement.

CONCLUSIONS

This evaluation revealed the challenges that low income countries are grappling with in prioritizing noncommunicable diseases in the context of a double disease burden with limited resources. Strengthening local capacity for priority setting would help to support the development of sustainable and implementable noncommunicable disease-related priorities. Global support (i.e. aid) to low income countries for noncommunicable diseases must also catch up to align with NCDs as a global health priority.

How have systematic priority setting approaches influenced policy making? A synthesis of the current literature

BACKGROUND

There is a growing body of literature on systematic approaches to healthcare priority setting from various countries and different levels of decision making. This paper synthesizes the current literature in order to assess the extent to which program budgeting and marginal analysis (PBMA), burden of disease & cost-effectiveness analysis (BOD/CEA), multi-criteria decision analysis (MCDA), and accountability for reasonableness (A4R), are reported to have been institutionalized and influenced policy making and practice.

METHODS

We searched for English language publications on health care priority setting approaches (2000-2017). Our sources of literature included PubMed and Ovid databases (including Embase, Global Health, Medline, PsycINFO, EconLit).

FINDINGS

Of the four approaches PBMA and A4R were commonly applied in high income countries while BOD/CEA was exclusively applied in low income countries. PBMA and BOD/CEA were most commonly reported to have influenced policy making. The explanations for limited adoption of an approach were related to its complexity, poor policy maker understanding and resource requirements.

CONCLUSIONS

While systematic approaches have the potential to improve healthcare priority setting; most have not been adopted in routine policy making. The identified barriers call for sustained knowledge exchange between researchers and policy-makers and development of practical guidelines to ensure that these frameworks are more accessible, applicable and sustainable in informing policy making.