Donor-funded project's sustainability assessment: a qualitative case study of a results-based financing pilot in Koulikoro region, Mali

Integrating interventions supported by development assistance for health into local health system: evidence from a China-World Bank-UK rural health system strengthening project (1998-2007)

INTRODUCTION

To empirically investigate sustainability of development assistance for health (DAH), we conducted a retrospective case study on the Basic Health Services Project (BHSP) for rural health system strengthening, supported by the World Bank and the UK in China between 1998 and 2007. Specifically, this study examines the integration of the BHSP interventions into China's health system.

METHODS

From December 2021 to December 2022, we reviewed 64 published papers and project or policy documents, and conducted semistructured interviews with 22 key informants, ranging from managers of donor agencies and the government to township-level hospital directors. From February to March 2023, the data were analysed under an analytical framework for integration of targeted health interventions into health systems.

RESULTS

Evidence of the BHSP shows that the integration outcomes can vary by the levels of integration (national or subnational), geographical coverage (project areas or both project and non-project areas) and approach to integration (policy or routinisation). The country's health system reform facilitated the integration of the interventions relevant to the reform policies, as the BHSP was one of the pilot schemes. However, interventions incompatible with this broad context were integrated to a limited extent. This integration occurred through embedding the project within the existing system, with a higher degree of embeddedness leading to smoother integration. Cross-sectoral leading groups and a technical support system heightened the project visibility and enabled contextualised local adaptation, contributing to the smooth integration of the project interventions.

CONCLUSION

The DAH-supported interventions can achieve sustainability by being integrated into the local health system. This integration can take various forms to improve health outcomes, including being accepted and internalised, modified as well as innovated and expanded. The host country and development partners can promote DAH sustainability by contextually integrating these interventions within the project scope.

A conceptualisation of scale-up and sustainability of social innovations in global health: a narrative review and integrative framework for action

BACKGROUND

The scale-up and sustainability of social innovations for health have received increased interest in global health research in recent years; however, these ambiguous concepts are poorly defined and insufficiently theorised and studied. Researchers, policymakers, and practitioners lack conceptual clarity and integrated frameworks for the scale-up and sustainability of global health innovations. Often, the frameworks developed are conceived in a linear and deterministic or consequentialist vision of the diffusion of innovations. This approach limits the consideration of complexity in scaling up and sustaining innovations.

OBJECTIVE

By using a systems theory lens and conducting a narrative review, this manuscript aims to produce an evidence-based integrative conceptual framework for the scale-up and sustainability of global health innovations.

METHOD

We conducted a hermeneutic narrative review to synthetise different definitions of scale-up and sustainability to model an integrative definition of these concepts for global health. We have summarised the literature on the determinants that influence the conditions for innovation success or failure while noting the interconnections between internal and external innovation environments.

RESULTS

The internal innovation environment includes innovation characteristics (effectiveness and testability, monitoring and evaluation systems, simplification processes, resource requirements) and organisational characteristics (leadership and governance, organisational change, and organisational viability). The external innovation environment refers to receptive and transformative environments; the values, cultures, norms, and practices of individuals, communities, organisations, and systems; and other contextual characteristics relevant to innovation development.

CONCLUSION

From these syntheses, we proposed an interconnected framework for action to better guide innovation researchers, practitioners, and policymakers in incorporating complexity and systemic interactions between internal and external innovation environments in global health.

Built to last? Barriers and facilitators of healthcare program sustainability: a systematic integrative review

OBJECTIVE

To identify barriers and facilitators associated with the sustainability of implemented and evaluated improvement programs in healthcare delivery systems.

DATA SOURCES AND STUDY SETTING

Six academic databases were searched to identify relevant peer-reviewed journal articles published in English between July 2011 and June 2022. Studies were included if they reported on healthcare program sustainability and explicitly identified barriers to, and facilitators of, sustainability.

STUDY DESIGN

A systematic integrative review guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. Study quality was appraised using Hawker's Quality Assessment Tool.

DATA COLLECTION/EXTRACTION METHODS

A team of reviewers screened eligible studies against the inclusion criteria and extracted the data independently using a purpose-designed Excel spreadsheet. Barriers and facilitators were extracted and mapped to the Integrated Sustainability Framework (ISF). Frequency counts of reported barriers/facilitators were performed across the included studies.

RESULTS

Of the 124 studies included in this review, almost half utilised qualitative designs (n = 52; 41.9%) and roughly one third were conducted in the USA (n = 43; 34.7%). Few studies (n = 29; 23.4%) reported on program sustainability beyond 5 years of program implementation and only 16 of them (55.2%) defined sustainability. Factors related to the ISF categories of inner setting (n = 99; 79.8%), process (n = 99; 79.8%) and intervention characteristics (n = 72; 58.1%) were most frequently reported. Leadership/support (n = 61; 49.2%), training/support/supervision (n = 54; 43.5%) and staffing/turnover (n = 50; 40.3%) were commonly identified barriers or facilitators of sustainability across included studies. Forty-six (37.1%) studies reported on the outer setting category: funding (n = 26; 56.5%), external leadership by stakeholders (n = 16; 34.8%), and socio-political context (n = 14; 30.4%). Eight studies (6.5%) reported on discontinued programs, with factors including funding and resourcing, poor fit, limited planning, and intervention complexity contributing to discontinuation.

CONCLUSIONS

This review highlights the importance of taking into consideration the inner setting, processes, intervention characteristics and outer setting factors when sustaining healthcare programs, and the need for long-term program evaluations. There is a need to apply consistent definitions and implementation frameworks across studies to strengthen evidence in this area.

TRIAL REGISTRATION

https://bmjopen.bmj.com/content/7/11/e018568 .

Using systems thinking to understand the scale-up and sustainability of health innovation: a case study of seasonal malaria chemoprevention processes in Burkina Faso

BACKGROUND

Scale-up and sustainability are often studied separately, with few studies examining the interdependencies between these two processes and the implementation contexts of innovations towards malaria prevention and control. Researchers and implementers offer much more attention to the content of innovations, as they focus on the technological dimensions and the conditions for expansion. Researchers have often considered innovation a linear sequence in which scaling up and sustainability represented the last stages. Using systems thinking in this manuscript, we analyze complex scaling and sustainability processes through adopting and implementing seasonal malaria chemoprevention (SMC) in Burkina Faso from 2014 to 2018.

METHODS

We conducted a qualitative case study involving 141 retrospective secondary data (administrative, press, scientific, tools and registries, and verbatim) spanning from 2012 to 2018. We complemented these data with primary data collected between February and March 2018 in the form of 15 personal semi-structured interviews with SMC stakeholders and non-participant observations. Processual analysis permitted us to conceptualize scale-up and sustainability processes over time according to different vertical and horizontal levels of analysis and their interconnections.

RESULTS

Our results indicated six internal and external determinants of SMC that may negatively or positively influence its scale-up and sustainability. These determinants are effectiveness, monitoring and evaluation systems, resources (financial, material, and human), leadership and governance, adaptation to the local context, and other external elements. Our results revealed that donors and implementing actors prioritized financial resources over other determinants. In contrast, our study clearly showed that the sustainability of the innovation, as well as its scaling up, depends significantly on the consideration of the interconnectedness of the determinants. Each determinant can concurrently constitute an opportunity and a challenge for the success of the innovation.

CONCLUSION

Our findings highlight the usefulness of the systemic perspective to consider all contexts (international, national, subnational, and local) to achieve large-scale improvements in the quality, equity, and effectiveness of global health interventions. Thus, complex and systems thinking have made it possible to observe emergent and dynamic innovation behaviors and the dynamics particular to sustainability and scaling up processes.

Sustainability of health services in refugee hosting districts: a qualitative study of health services in three west Nile refugee hosting districts, Uganda

BACKGROUND

Humanitarian health assistance programmes have expanded from temporary approaches addressing short-term needs to providing long-term interventions in emergency settings. Measuring sustainability of humanitarian health services is important towards improving the quality of health services in refugee settings.

OBJECTIVE

To explore the sustainability of health services following the repatriation of refugees from the west Nile districts of Arua, Adjumani and Moyo.

METHODS

This was a qualitative comparative case study conducted in three west Nile refugee-hosting districts of Arua, Adjumani, and Moyo. In-depth interviews were conducted with 28 purposefully selected respondents in each of the three districts. Respondents included health workers and managers, district civic leaders, planners, chief administrative officers, district health officers, project staff of aid agencies, refugee health focal persons and community development officers.

RESULTS

The study shows that in terms of organization capacity, the District Health Teams provided health services to both refugee and host communities with minimal support from aid agencies. Health services were available in most former refugee hosting areas in Adjumani, Arua and Moyo districts. However, there were several disruptions notably reduction and inadequate services due to shortage of drugs and essential supplies, lack of health workers, and closure or relocation of health facilities in around former settlements. To minimize disruptions the district health office reorganized health services. In restructuring health services, the district local governments closed or upgraded health facilities to address reduced capacity and catchment population. Health workers employed by aid agencies were recruited into government services while others who were deemed excess or unqualified were laid off. Equipment and machinery including machines and vehicles were transferred to the district health office in specific health facilities. Funding for health services was mainly provided by the Government of Uganda through the Primary Health Care Grant. Aid agencies, however, continued to provide minimal support health services for refugees who remained in Adjumani district.

CONCLUSION

Our study showed that while humanitarian health services are not designed for sustainability, several interventions continued at the end of the refugee emergency in the three districts. The embeddedness of the refugee health services in the district health systems ensured health services continued through public service delivery structures. It is important to strengthen the capacity of the local service delivery structures and ensure health assistance programmes are integrated into local health systems to promote sustainability.

What improves access to primary healthcare services in rural communities? A systematic review

BACKGROUND

To compile key strategies from the international experiences to improve access to primary healthcare (PHC) services in rural communities. Different innovative approaches have been practiced in different parts of the world to improve access to essential healthcare services in rural communities. Systematically collecting and combining best experiences all over the world is important to suggest effective strategies to improve access to healthcare in developing countries. Accordingly, this systematic review of literature was undertaken to identify key approaches from international experiences to enhance access to PHC services in rural communities.

METHODS

All published and unpublished qualitative and/or mixed method studies conducted to improvement access to PHC services were searched from MEDLINE, Scopus, Web of Science, WHO Global Health Library, and Google Scholar. Articles published other than English language, citations with no abstracts and/or full texts, and duplicate studies were excluded. We included all articles available in different electronic databases regardless of their publication years. We assessed the methodological quality of the included studies using mixed methods appraisal tool (MMAT) version 2018 to minimize the risk of bias. Data were extracted using JBI mixed methods data extraction form. Data were qualitatively analyzed using emergent thematic analysis approach to identify key concepts and coded them into related non-mutually exclusive themes.

RESULTS

Our analysis of 110 full-text articles resulted in ten key strategies to improve access to PHC services. Community health programs or community-directed interventions, school-based healthcare services, student-led healthcare services, outreach services or mobile clinics, family health program, empanelment, community health funding schemes, telemedicine, working with traditional healers, working with non-profit private sectors and non-governmental organizations including faith-based organizations are the key strategies identified from international experiences.

CONCLUSION

This review identified key strategies from international experiences to improve access to PHC services in rural communities. These strategies can play roles in achieving universal health coverage and reducing disparities in health outcomes among rural communities and enabling them to get healthcare when and where they want.

Identifying existing approaches used to evaluate the sustainability of evidence-based interventions in healthcare: an integrative review

BACKGROUND

There is limited evidence to evaluate the sustainability of evidence-based interventions (EBIs) for healthcare improvement. Through an integrative review, we aimed to identify approaches to evaluate the sustainability of evidence-based interventions (EBIs) and sustainability outcomes.

METHODS

Following Whittemore and Knafl's methodological process: (1) problem identification; (2) literature search; (3) data evaluation; (4) data analysis; and (5) presentation, a comprehensive search strategy was applied across five databases. Included studies were not restricted by research design; and had to evaluate the sustainability of an EBI in a healthcare context. We assessed the methodological quality of studies using the Mixed Methods Appraisal Tool.

RESULTS

Of 18,783 articles retrieved, 64 fit the inclusion criteria. Qualitative designs were most commonly used for evaluation (48%), with individual interviews as the predominant data collection method. Timing of data collection varied widely with post-intervention data collection most frequent (89%). Of the 64 studies, 44% used a framework, 26% used a model, 11% used a tool, 5% used an instrument, and 14% used theory as their primary approach to evaluate sustainability. Most studies (77%) did not measure sustainability outcomes, rather these studies focused on sustainability determinants.

DISCUSSION

It is unclear which approach/approaches are most effective for evaluating sustainability and what measures and outcomes are most commonly used. There is a disconnect between evaluating the factors that may shape sustainability and the outcomes approaches employed to measure sustainability. Our review offers methodological recommendations for sustainability evaluation research and highlights the importance in understanding mechanisms of sustainability to advance the field.

La pérennité d’un projet de prévention et de prise en charge de la malnutrition au Niger

OBJECTIVE

This article is a case study of a capacity-building project to reduce malnutrition, implemented by a non-governmental organization (NGO) in Niger. It aims to generate a set of lessons learned on the processes of sustaining health interventions in vulnerable contexts.

METHODS

Using a theoretical framework on sustainability, we carried out a thematic analysis on semi-structured individual interviews (n = 124) collected, between 2016 and 2017, during the evaluative research of the NGO project.

RESULTS

The data analysis revealed several risks and opportunities to sustain the project activities, classified according to the characteristics of a routine identified in the theoretical framework: memory, routines, values, and rules. Our findings show a low level of sustainability of the project. Several activities were stopped at the end of the project and others may not continue in the long term. Indeed, local realities-poverty and political fragility-combined with the 'short-term' and vertical logics of the project's implementation have hindered the project activities' sustainability.

CONCLUSION

In vulnerable environments, sustainability should not simply be seen as an outcome to be achieved at the end of the project. The sustainability process must be activated by key stakeholders, focusing not only on the resources stabilizing but also on the resilience of the structures and systems in which interventions are embedded.

Sustainability of Donor-Funded Health-Related Programs Beyond the Funding Lifecycle in Africa: A Systematic Review

This study aimed to determine if implemented donor-funded health related-programs in Africa were sustained beyond the funding lifecycle and determine their facilitators and impeders. A systematic review was conducted after the documentation of a study protocol. A database search was done across three databases namely Google Scholar, PubMed, and Medline between January 27 and February 15, 2022. All peer-reviewed articles on sustainability of health interventions in Africa published between 2015 and 2021 that provided one or more context-relevant definitions of sustainability were included. Publications with no use of quantitative or qualitative methods and studies with no information on project evaluation after initial implementation were excluded. Screening of titles and abstracts was done, and the full texts of all relevant articles were retrieved. The risk of bias in systematic reviews (ROBIS) tool was used to assess the risk of bias in the systematic review. Overall, 4,876 articles were retrieved, and only nine articles were eligible for inclusion in the review following the removal of duplicates. Overall, sustainability was described in only three of the five regions in Africa. Donor-funded projects were sustained beyond the funding lifecycle in seven (77.8%) studies. Facilitators of sustainability in Africa included community ownership of the project through the engagement of community stakeholders in the design and implementation of such projects, use of locally available resources, sound infrastructure, and the constitution of interdisciplinary team to facilitate capacity building. Impeders to the sustainability of donor-funded projects included weak health systems exemplified in poor documentation and integration of records, lack of financial leadership, shortage of resources, political interference, poor feedback mechanism, and weak donor-community interactions. From the ROBIS tool, a low risk of bias existed in the studies included in the review. Although the included studies appropriately considered the review’s research question, seven studies had a low risk of bias in the domains one to three, and two studies had high risk of bias in domain four. To derive maximum benefits from donor-funded health interventions, sustainability of such projects is key. During program planning phase, context-based facilitators of sustainability should be promoted, while impeders are immediately addressed.

Recognizing and mediating bureaucratic barriers: increasing access to care through small and medium-sized private providers in Kenya

Background: Equitable access to health services can be constrained in countries where private practitioners make up a large portion of primary care providers. Expanding purchasing arrangements has helped many countries integrate private providers into government-supported payment schemes, reducing financial barriers to care. However, private providers often must go through an onerous accreditation process to enroll in these schemes. The difficulties of this process are exacerbated where health policy is changed often and low-level bureaucrats must navigate these shifts at their own discretion. This paper analyzes one initiative to increase private provider accreditation with social health insurance (SHI) in Kenya by creating an intermediary between providers and “street-level” SHI bureaucrats.

Methods: This paper draws on 126 semi-structured interviews about SHI accreditation experience with private providers who were members of a franchise network in Kenya. It also draws on four focus group discussions conducted with franchise representatives who provided accreditation support to the providers and served as liaisons between the franchised providers and local SHI offices. There was a total of 20 participants across all four focus groups.

Results: In a governance environment where regulations are weak and impermanent, street-level bureaucrats often created an accreditation process that was inconsistent and opaque. Support from the implementing organizations increased communication between SHI officials and providers, which clarified rules and increased providers’ confidence in the system. The intermediaries also reduced bureaucrats’ ability to apply regulations at will and helped to standardize the accreditation process for both providers and bureaucrats.

Conclusions: We conclude that intermediary organizations can mitigate institutional weaknesses and facilitate process efficiency. However, intermediaries only have a temporary role to play where there is potential to: 1) directly increase private providers’ power in a complex regulatory system; 2) reform the system itself to be more responsive to the limitations of on-the-ground implementation.

Health actor approaches to financing universal coverage strategies for pneumococcal and rotavirus immunisation programmes in low-income and middle-income countries: a scoping review protocol

INTRODUCTION

Sustainable financing of immunisation programmes is an important step towards universal coverage of life-saving vaccines. Yet, financing mechanisms for health programmes could have consequences on the design of universal approaches to immunisation coverage. Effective implementation of immunisation interventions necessitates investigating the roles of institutions and power on interventions. This review aims to understand how sustainable financing and equitable immunisation are conceptualised by health actors like Gavi, and government-related entities across low-income and middle-income countries (LMICs) and how financing mechanisms can affect universal coverage of vaccines.

METHODS AND ANALYSIS

This study protocol outline a scoping review of the peer-reviewed and the grey literature, using established methodological framework for scoping review. Literature will be identified through a comprehensive search of multiple databases and grey literature. All peer-reviewed implementation research studies from the year 2002 addressing financing and universal coverage of immunisation programmes for the pneumococcal conjugated vaccine and rotavirus vaccines immunisation interventions will be included and grey literature published in/after the year 2015. For the study scope, population, concept and context are defined: Population as international and national health stakeholders financing immunisation programmes; Concept as implementation research on pneumococcal conjugate and rotavirus vaccination interventions; and Context as LMICs. Findings will be quantitatively summarised to provide an overview and narratively synthesised and analysed. Studies that do not use implementation research approaches, frameworks or models will be excluded.

ETHICS AND DISSEMINATION

Ethics approval is not required for this scoping review. Findings and recommendations will be presented to implementation researchers and health stakeholders.

Recognizing and mediating bureaucratic barriers: increasing access to care through small and medium-sized private providers in Kenya

Background: Equitable access to health services can be constrained in countries where private practitioners make up a large portion of primary care providers. Expanding purchasing arrangements has helped many countries integrate private providers into government-supported payment schemes, reducing financial barriers to care. However, private providers often must go through an onerous accreditation process to enroll in these schemes. The difficulties of this process are exacerbated where health policy is changed often and low-level bureaucrats must navigate these shifts at their own discretion. This paper analyzes one initiative to increase private provider accreditation with social health insurance (SHI) in Kenya by creating an intermediary between providers and “street-level” SHI bureaucrats. Methods: This paper draws on 126 semi-structured interviews about SHI accreditation experience with private providers who were members of a franchise network in Kenya. It also draws on four focus group discussions conducted with franchise representatives who provided accreditation support to the providers and served as liaisons between the franchised providers and local SHI offices. There was a total of 20 participants across all four focus groups. Results: In a governance environment where regulations are weak and impermanent, street-level bureaucrats often created an accreditation process that was inconsistent and opaque. Support from the implementing organizations increased communication between SHI officials and providers, which clarified rules and increased providers’ confidence in the system. The intermediaries also reduced bureaucrats’ ability to apply regulations at will and helped to standardize the accreditation process for both providers and bureaucrats. Conclusions: We conclude that intermediary organizations can mitigate institutional weaknesses and facilitate process efficiency. However, intermediaries only have a temporary role to play where there is potential to: 1) directly increase private providers’ power in a complex regulatory system; 2) reform the system itself to be more responsive to the limitations of on-the-ground implementation.

Sustainability determinants of the Burkinabe performance-based financing project

PURPOSE

A performance-based financing (PBF) pilot project was implemented in 2011 in Burkina Faso. After more than five years of implementation (data collection in 2016), the project's sustainability was not guaranteed. This study's objective is to assess this project's sustainability in 2016 by identifying the presence/absence of different determinants of sustainability according to the conceptual framework of Seppey et al. (2017).

DESIGN/METHODOLOGY/APPROACH

It uses a case study approach using in-depth interviews with various actors at the local, district/regional and national levels. Participants (n = 37) included health practitioners, management team members, implementers and senior members of health directions. A thematic analysis based on the conceptual framework was conducted, as well as an inductive analysis.

FINDINGS

Results show the project's sustainability level was weak according to an unequal presence of sustainability's determinants; some activities are being maintained but not fully routinised. Discrepancies between the project and the context's values appeared to be important barriers towards sustainability. Project's ownership by key stakeholders also seemed superficial despite the implementers' leadership towards its success. The project's objective towards greater autonomy for health centres was also directly confronting the Burkinabe's hierarchical health system.

ORIGINALITY/VALUE

This study reveals many fits and misfits between a PBF project and its context affecting its ability to sustain activities through time. It also underlines the importance of using a conceptual framework in implementing and evaluating interventions. These results could be interesting for decision-makers and implementers in further assessing PBF projects elsewhere.

Does the gap between health workers' expectations and the realities of implementing a performance-based financing project in Mali create frustration?

Background

Performance-Based Financing (PBF), an innovative health financing initiative, was recently implemented in Mali. PBF aims to improve quality of care by motivating health workers. The purpose of this research was to identify and understand how health workers’ expectations related to their experiences of the first cycle of payment of PBF subsidies, and how this experience affected their motivation and sentiments towards the intervention. We pose the research question, “how does the process of PBF subsidies impact the motivation of health workers in Mali?”

Methods

We adopted a qualitative approach using multiple case studies. We chose three district hospitals (DH 1, 2 and 3) in three health districts (district 1, 2 and 3) among the ten in the Koulikoro region. Our cases correspond to the three DHs. We followed the principle of data source triangulation; we used 53 semi-directive interviews conducted with health workers (to follow the principle of saturuation), field notes, and documents relating to the distribution grids of subsidies for each DH. We analyzed data in a mixed deductive and inductive manner.

Results

The results show that the PBF subsidies led to health workers feeling more motivated to perform their tasks overall. Beyond financial motivation, this was primarily due to PBF allowing them to work more efficiently. However, respondents perceived a discrepancy between the efforts made and the subsidies received. The fact that their expectations were not met led to a sense of frustration and disappointment. Similarly, the way in which the subsidies were distributed and the lack of transparency in the distribution process led to feelings of unfairness among the vast majority of respondents. The results show that frustrations can build up in the early days of the intervention.

Conclusion

The PBF implementation in Mali left health workers frustrated. The short overall implementation period did not allow actors to adjust their initial expectations and motivational responses, neither positive nor negative. This underlines how short-term interventions might not just lack impact, but instil negative sentiments likely to carry on into the future.

No effects of pilot performance-based intervention implementation and withdrawal on the coverage of maternal and child health services in the Koulikoro region, Mali: an interrupted time series analysis

Performance-based financing (PBF) has been promoted and increasingly implemented across low- and middle-income countries to increase the utilization and quality of primary health care. However, the evidence of the impact of PBF is mixed and varies substantially across settings. Thus, further rigorous investigation is needed to be able to draw broader conclusions about the effects of this health financing reform. We examined the effects of the implementation and subsequent withdrawal of the PBF pilot programme in the Koulikoro region of Mali on a range of relevant maternal and child health indicators targeted by the programme. We relied on a control interrupted time series design to examine the trend in maternal and child health service utilization rates prior to the PBF intervention, during its implementation and after its withdrawal in 26 intervention health centres. The results for these 26 intervention centres were compared with those for 95 control health centres, with an observation window that covered 27 quarters. Using a mixed-effects negative binomial model combined with a linear spline regression model and covariates adjustment, we found that neither the introduction nor the withdrawal of the pilot PBF programme bore a significant impact in the trend of maternal and child health service use indicators in the Koulikoro region of Mali. The absence of significant effects in the health facilities could be explained by the context, by the weaknesses in the intervention design and by the causal hypothesis and implementation. Further inquiry is required in order to provide policymakers and practitioners with vital information about the lack of effects detected by our quantitative analysis.

Factors related to excessive out-of-pocket expenditures among the ultra-poor after discontinuity of PBF: a cross-sectional study in Burkina Faso

BACKGROUND

Measuring progress towards financial risk protection for the poorest is essential within the framework of Universal Health Coverage. The study assessed the level of out-of-pocket expenditure and factors associated with excessive out-of-pocket expenditure among the ultra-poor who had been targeted and exempted within the context of the performance-based financing intervention in Burkina Faso. Ultra-poor were selected based on a community-based approach and provided with an exemption card allowing them to access healthcare services free of charge.

METHODS

We performed a descriptive analysis of the level of out-of-pocket expenditure on formal healthcare services using data from a cross-sectional study conducted in Diébougou district. Multivariate logistic regression was performed to investigate the factors related to excessive out-of-pocket expenditure among the ultra-poor. The analysis was restricted to individuals who reported formal health service utilisation for an illness-episode within the last six months. Excessive spending was defined as having expenditure greater than or equal to two times the median out-of-pocket expenditure.

RESULTS

Exemption card ownership was reported by 83.64% of the respondents. With an average of FCFA 23051.62 (USD 39.18), the ultra-poor had to supplement a significant amount of out-of-pocket expenditure to receive formal healthcare services at public health facilities which were supposed to be free. The probability of incurring excessive out-of-pocket expenditure was negatively associated with being female (β = - 2.072, p = 0.00, ME = - 0.324; p = 0.000) and having an exemption card (β = - 1.787, p = 0.025; ME = - 0.279, p = 0.014).

CONCLUSIONS

User fee exemptions are associated with reduced out-of-pocket expenditure for the ultra-poor. Our results demonstrate the importance of free care and better implementation of existing exemption policies. The ultra-poor's elevated risk due to multi-morbidities and severity of illness need to be considered when allocating resources to better address existing inequalities and improve financial risk protection.

Can complex programs be sustained? A mixed methods sustainability evaluation of a national infant and young child feeding program in Bangladesh and Vietnam

Background

Poor early-life nutrition is a major barrier to good health and cognitive development, and is a global health priority. Alive & Thrive (A&T) was a multi-pronged initiative to improve infant and young child feeding behaviors. It aimed to achieve at-scale child health and nutrition improvements via a comprehensive approach that included nutrition counseling by health workers, policy change, social mobilization and mass media activities. This study evaluated the sustainability of activities introduced during A&T implementation (2009–2014) in Bangladesh and Vietnam.

Methods

This was a mixed methods study that used a quasi-experimental design. Quantitative data (surveys with 668 health workers, and 269 service observations) were collected in 2017; and analysis compared outcomes (primarily dose and fidelity of activities, and capacity) in former A&T intervention areas versus areas that did not receive the full A&T intervention. Additionally, we conducted interviews and focus groups with 218 stakeholders to explore their impressions about the determinants of sustainability, based on a multi-level conceptual framework.

Results

After program conclusion, stakeholders perceive declines in mass media campaigns, policy and advocacy activities, and social mobilization activities – but counseling activities were institutionalized and continued in both countries. Quantitative data show a persisting modest intervention effect: health workers in intervention areas had significantly higher child feeding knowledge, and in Bangladesh greater self-efficacy and job satisfaction, compared to their counterparts who did not receive the full package of A&T activities. While elements of the program were integrated into routine services, stakeholders noted dilution of the program focus due to competing priorities. Qualitative data suggest that some elements, such as training, monitoring, and evaluation, which were seen as essential to A&T’s success, have declined in frequency, quality, coverage, or were eliminated altogether.

Conclusions

The inclusion of multiple activities in A&T and efforts to integrate the program into existing institutions were seen as crucial to its success but also made it difficult to sustain, particularly given unstable financial support and human resource constraints. Future complex programs should carefully plan for institutionalization in advance of the program by cultivating champions across the health system, and designing unique and complementary roles for all stakeholders including donors.

Does performance-based financing curb stock-outs of essential medicines? Results from a randomised controlled trial in Cameroon

OBJECTIVE

In 2011, the government of Cameroon launched its performance-based financing (PBF) scheme. Our study examined the effects of the PBF intervention on the availability of essential medicines (EM).

METHODS

Randomised control trial whereby PBF and three distinct comparison groups were randomised in a total of 205 health facilities across three regions. Baseline data were collected between March and May 2012 and endline data 36 months later. We defined availability of multiple EM groups by assessing stock-outs for at least one day over the 30 days prior to the survey date and estimated changes attributable to PBF using a series of difference-in-difference regression models, adjusted for relevant facility-level covariates. Data were analysed stratified by region and area to assess effect heterogeneity.

RESULTS

Our estimates suggest that PBF intervention had no effect on the stock-outs of antenatal care drugs (P = 0.160), vaccines (P = 0.396), integrated management of childhood illness drugs (P = 0.681) and labour and delivery drugs (P = 0.589). However, the intervention was associated with a significant reduction of 34% in stock-outs of family planning medicines (P = 0.028). We observed effect heterogeneity across regions and areas, with significant decreases in stock-outs of family planning products in North-West region (P = 0.065) and in rural areas (P = 0.043).

CONCLUSIONS

The PBF intervention in Cameroon had limited effects on the reduction of EMs stock-outs. These poor results were likely the consequence of partial implementation failure, ranging from disruption and discontinuation of services to limited facility autonomy in managing decision-making and considerable delay in performance payment.

Implementing performance-based financing in peripheral health centres in Mali: what can we learn from it?

Introduction

Numerous sub-Saharan African countries have experimented with performance-based financing (PBF) with the goal of improving health system performance. To date, few articles have examined the implementation of this type of complex intervention in Francophone West Africa. This qualitative research aims to understand the process of implementing a PBF pilot project in Mali's Koulikoro region.

Method

We conducted a contrasted multiple case study of performance in 12 community health centres in three districts. We collected 161 semi-structured interviews, 69 informal interviews and 96 non-participant observation sessions. Data collection and analysis were guided by the Consolidated Framework for Implementation Research adapted to the research topic and local context.

Results

Our analysis revealed that the internal context of the PBF implementation played a key role in the process. High-performing centres exercised leadership and commitment more strongly than low-performing ones. These two characteristics were associated with taking initiatives to promote PBF implementation and strengthening team spirit. Information regarding the intervention was best appropriated by qualified health professionals. However, the limited duration of the implementation did not allow for the emergence of networks or champions. The enthusiasm initially generated by PBF quickly dissipated, mainly due to delays in the implementation schedule and the payment modalities.

Conclusion

PBF is a complex intervention in which many actors intervene in diverse contexts. The initial level of performance and the internal and external contexts of primary healthcare facilities influence the implementation of PBF. Future work in this area would benefit from an interdisciplinary approach combining public health and anthropology to better understand such an intervention. The deductive–inductive approach must be the stepping-stone of such a methodological approach.

Built to last? The sustainability of healthcare system improvements, programmes and interventions: a systematic integrative review

INTRODUCTION

The sustainability of healthcare delivery systems is challenged by ageing populations, complex systems, increasing rates of chronic disease, increasing costs associated with new medical technologies and growing expectations by healthcare consumers. Healthcare programmes, innovations and interventions are increasingly implemented at the front lines of care to increase effectiveness and efficiency; however, little is known about how sustainability is conceptualised and measured in programme evaluations.

OBJECTIVES

We aimed to describe theoretical frameworks, definitions and measures of sustainability, as applied in published evaluations of healthcare improvement programmes and interventions.

DESIGN

Systematic integrative review.

METHODS

We searched six academic databases, CINAHL, Embase, Ovid MEDLINE, Emerald Management, Scopus and Web of Science, for peer-reviewed English journal articles (July 2011-March 2018). Articles were included if they assessed programme sustainability or sustained outcomes of a programme at the healthcare system level. Six reviewers conducted the abstract and full-text review. Data were extracted on study characteristics, definitions, terminology, theoretical frameworks, methods and tools. Hawker's Quality Assessment Tool was applied to included studies.

RESULTS

Of the 92 included studies, 75.0% were classified as high quality. Twenty-seven (29.3%) studies provided 32 different definitions of sustainability. Terms used interchangeably for sustainability included continuation, maintenance, follow-up or long term. Eighty studies (87.0%) clearly reported the timepoints at which sustainability was evaluated: 43.0% at 1-2 years and 11.3% at <12 months. Eighteen studies (19.6%) used a theoretical framework to conceptualise or assess programme sustainability, including frameworks that were not specifically designed to assess sustainability.

CONCLUSIONS

The body of literature is limited by the use of inconsistent definitions and measures of programme sustainability. Evaluations of service improvement programmes and interventions seldom used theoretical frameworks. Embedding implementation science and healthcare service researchers into the healthcare system is a promising strategy to improve the rigour of programme sustainability evaluations.

Using implementation science theories and frameworks in global health

In global health, researchers and decision makers, many of whom have medical, epidemiology or biostatistics background, are increasingly interested in evaluating the implementation of health interventions. Implementation science, particularly for the study of public policies, has existed since at least the 1930s. This science makes compelling use of explicit theories and analytic frameworks that ensure research quality and rigour. Our objective is to inform researchers and decision makers who are not familiar with this research branch about these theories and analytic frameworks. We define four models of causation used in implementation science: intervention theory, frameworks, middle-range theory and grand theory. We then explain how scientists apply these models for three main implementation studies: fidelity assessment, process evaluation and complex evaluation. For each study, we provide concrete examples from research in Cuba and Africa to better understand the implementation of health interventions in global health context. Global health researchers and decision makers with a quantitative background will not become implementation scientists after reading this article. However, we believe they will be more aware of the need for rigorous implementation evaluations of global health interventions, alongside impact evaluations, and in collaboration with social scientists.

Stakeholder Perceptions and Context of the Implementation of Performance-Based Financing in District Hospitals in Mali

BACKGROUND

To improve the performance of the healthcare system, Mali's government implemented a pilot project of performance-based financing (PBF) in the field of reproductive health. It was established in the Koulikoro region. This research analyses the process of implementing PBF at district hospital (DH) level, something which has rarely been done in Africa.

METHODS

This qualitative research is based on a multiple, explanatory, and contrasting case study with nested levels of analysis. It covered three of the 10 DHs in the Koulikoro region. We conducted 36 interviews: 12 per DH with council of circle's members (2) and health personnel (10). We also conducted 24 non-participant observation sessions, 16 informal interviews, and performed a literature review. We performed data analysis using the Consolidated Framework for Implementation Research (CFIR).

RESULTS

Stakeholders perceived the PBF pilot project as a vertical intervention from outside that focused solely on reproductive health. Local actors were not involved in the design of the PBF model. Several difficulties regarding the quality of its design and implementation were highlighted: too short duration of the intervention (8 months), choice and insufficient number of indicators according to the priority of the donors, and impossibility of making changes to the model during its implementation. All health workers adhered to the principles of PBF intervention. Except for members of the district health management team (DHMT) involved in the implementation, respondents only had partial knowledge of the PBF intervention. The implementation of PBF appeared to be easier in District 3 Hospital compared to District 1 and District 2 because it benefited from a pre-pilot project and had good leadership.

CONCLUSION

The PBF programme offered an opportunity to improve the quality of care provided to the population through the motivation of health personnel in Mali. However, several obstacles were observed during the implementation of the PBF pilot project in DHs. When designing and implementing PBF in DHs, it is necessary to consider factors that can influence the implementation of a complex intervention.

From Amsterdam to Bamako: a qualitative case study on diffusion entrepreneurs’ contribution to performance-based financing propagation in Mali

For the past 15 years, several donors have promoted performance-based financing (PBF) in Africa for improving health services provision. European and African experts known as ‘diffusion entrepreneurs’ (DEs) assist with PBF pilot testing. In Mali, after participating in a first pilot PBF in 2012–13, the Ministry of Health and Public Hygiene included PBF in its national strategic plan. It piloted this strategy again in 2016–17. We investigated the interactions between foreign experts and domestic actors towards PBF diffusion in Mali from 2009 to 2018. Drawing on the framework on DEs (Gautier et al., 2018), we examine the characteristics of DEs acting at the global, continental and (sub)national levels; and their contribution to policy framing, emulation, experimentation and learning, across locations of PBF implementation. Using an interpretive approach, this longitudinal qualitative case study analyses data from observations (N = 5), interviews (N = 33) and policy documentation (N = 19). DEs framed PBF as the logical continuation of decentralization, contracting policies and existing policies. Policy emulation started with foreign DEs inspiring domestic actors’ interest, and succeeded thanks to longstanding relationships and work together. Learning was initiated by European DEs through training sessions and study tours outside Mali, and by African DEs transferring their passion and tacit knowledge to PBF implementers. However, the short-time frame and numerous implementation gaps of the PBF pilot project led to incomplete policy learning. Despite the many pitfalls of the region-wide pilot project, policy actors in Mali decided to pursue this policy in Mali. Future research should further investigate the making of successful African DEs by foreign DEs advocating for a given policy.