Comparative effectiveness of two disparate policies on child health: experimental evidence from the Philippines

Measurement of benefits in economic evaluations of nutrition interventions in low‐ and middle‐income countries: A systematic review

Economic evaluation of nutrition interventions that compares the costs to benefits is essential to priority‐setting. However, there are unique challenges to synthesizing the findings of multi‐sectoral nutrition interventions due to the diversity of potential benefits and the methodological differences among sectors in measuring them. This systematic review summarises literature on the interventions, sectors, benefit terminology and benefit types included in cost‐effectiveness, cost‐utility and benefit‐cost analyses (CEA, CUA and BCA, respectively) of nutrition interventions in low‐ and middle‐income countries. A systematic search of five databases published from January 2010 to September 2019 with expert consultation yielded 2794 studies, of which 93 met all inclusion criteria. Eighty‐seven per cent of the included studies included interventions delivered from only one sector, with almost half from the health sector (43%), followed by food/agriculture (27%), water, sanitation and hygiene (WASH) (10%), and social protection (8%). Only 9% of studies assessed programmes involving more than one sector (health, food/agriculture, social protection and/or WASH). Eighty‐one per cent of studies used more than one term to refer to intervention benefits. The included studies calculated 128 economic evaluation ratios (57 CEAs, 39 CUAs and 32 BCAs), and the benefits they included varied by sector. Nearly 60% measured a single benefit category, most frequently nutritional status improvements; other health benefits, cognitive/education gains, dietary diversity, food security, knowledge/attitudes/practices and income were included in less than 10% of all ratios. Additional economic evaluation of non‐health and multi‐sector interventions, and incorporation of benefits beyond nutritional improvements (including cost savings) in future economic evaluations is recommended.

Current economic evaluations often underestimate the total sum of benefits that can arise from nutrition interventions. Comprehensive benefit measurement of some nutrition programmes may require further methodological research.

In the near‐term, economic evaluations of multi‐sectoral nutrition interventions should include potential cost savings from improved nutrition in their calculations and assess the potential for benefits unrelated to nutrition. If the range of benefits is diverse and can be monetised, benefit‐cost analysis may be the preferred evaluation method.

Economic evaluations of nutrition‐sensitive interventions from agriculture, water, sanitation and hygiene (WASH), and gender empowerment sectors, are needed to fill an evidence gap on costs and benefits of multisectoral approaches to improved maternal and child health and nutrition.

Measuring and Improving Evidence-Based Patient Care Using a Web-Based Gamified Approach in Primary Care (QualityIQ): Randomized Controlled Trial

BACKGROUND

Unwarranted variability in clinical practice is a challenging problem in practice today, leading to poor outcomes for patients and low-value care for providers, payers, and patients.

OBJECTIVE

In this study, we introduced a novel tool, QualityIQ, and determined the extent to which it helps primary care physicians to align care decisions with the latest best practices included in the Merit-Based Incentive Payment System (MIPS).

METHODS

We developed the fully automated QualityIQ patient simulation platform with real-time evidence-based feedback and gamified peer benchmarking. Each case included workup, diagnosis, and management questions with explicit evidence-based scoring criteria. We recruited practicing primary care physicians across the United States into the study via the web and conducted a cross-sectional study of clinical decisions among a national sample of primary care physicians, randomized to continuing medical education (CME) and non-CME study arms. Physicians "cared" for 8 weekly cases that covered typical primary care scenarios. We measured participation rates, changes in quality scores (including MIPS scores), self-reported practice change, and physician satisfaction with the tool. The primary outcomes for this study were evidence-based care scores within each case, adherence to MIPS measures, and variation in clinical decision-making among the primary care providers caring for the same patient.

RESULTS

We found strong, scalable engagement with the tool, with 75% of participants (61 non-CME and 59 CME) completing at least 6 of 8 total cases. We saw significant improvement in evidence-based clinical decisions across multiple conditions, such as diabetes (+8.3%, P<.001) and osteoarthritis (+7.6%, P=.003) and with MIPS-related quality measures, such as diabetes eye examinations (+22%, P<.001), depression screening (+11%, P<.001), and asthma medications (+33%, P<.001). Although the CME availability did not increase enrollment in the study, participants who were offered CME credits were more likely to complete at least 6 of the 8 cases.

CONCLUSIONS

Although CME availability did not prove to be important, the short, clinically detailed case simulations with real-time feedback and gamified peer benchmarking did lead to significant improvements in evidence-based care decisions among all practicing physicians.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03800901; https://clinicaltrials.gov/ct2/show/NCT03800901.

Impact of global budget combined with pay-for-performance on the quality of care in county hospitals: a difference-in-differences study design with a propaensity-score-matched control group using data from Guizhou province, China

BACKGROUND

Provider payment system has a profound impact on health system performance. In 2016, a number of counties in rural Guizhou, China, implemented global budget (GB) for county hospitals with quality control measures. The aim of this study is to measure the impact of GB combined with pay-for-performance on the quality of care of inpatients in county-level hospitals in China.

METHODS

Inpatient cases of four diseases, including pneumonia, chronic asthma, acute myocardial infarction and stroke, from 16 county-level hospitals in Guizhou province that implemented GB in 2016 were selected as the intervention group, and similar inpatient cases from 10 county-level hospitals that still implemented fee-for-services were used as the control group. Propensity matching score (PSM) was used for data matching to control for age factors, and difference-in-differences (DID) models were constructed using the matched samples to perform regression analysis on quality of care for the four diseases.

RESULTS

After the implementation of GB, rate of sputum culture in patients with pneumonia, rate of aspirin at discharge, rate of discharge with β-blocker and rate of smoking cessation advice in patients with acute myocardial infarction increased. Rate of oxygenation index assessment in patient with chronic asthma decreased 20.3%. There are no significant changes in other indicators of process quality.

CONCLUSIONS

The inclusion of pay-for-performance in the global budget payment system will help to reduce the quality risks associated with the reform of the payment system and improve the quality of care. Future reform should also consider the inclusion of the pay-for-performance mechanism.

A nationwide program to improve clinical care quality in the Kyrgyz Republic

Background

To assess baseline quality of care in the Kyrgyz Republic in 2019 and determine the effect of online simulated patients in changing doctors’ practice in three specific disease areas: non-communicable disease, neonatal/child health, and maternal health.

Methods

Over 2000 family health, pediatric, neonatology, therapy, and obstetric-gynecologic doctors from every rayon (district) hospital and at least one associated family health (Primary) care clinic participated. To adequately scale the project, the Ministry of Health used online simulated Clinical Performance and Value (CPV) vignettes. All doctors cared for the same set of patients in their clinical area. Over eight months in 2019, we gathered three rounds of CPV data in seven oblasts.

Results

Overall quality scores were highly variable at baseline (59.2% + 13.5%). After three rounds the average score increased 6.5% (P < 0.001). By the end of round three, the lowest scoring oblast was providing higher quality care compared to the highest scoring oblast in the initial round (64.2% in round 3 vs 62.4% in round 1), indicating greater adherence to the evidence base. Additionally, family health doctors ordered 26% fewer unnecessary tests (P < 0.05), while specialists ordered 39% fewer unnecessary tests (P < 0.05). If trends continue, this translates into a net annual savings of 63 million Kyrgyz som.

Conclusions

This study demonstrates serial measurement of care provided by over 2000 physicians in the Kyrgyz Republic can be improved as measured by CPVs. This project may be a useful template to improve health care quality at a national level in other low- and middle-income country settings.

Paying for performance to improve the delivery of health interventions in low- and middle-income countries

BACKGROUND

There is growing interest in paying for performance (P4P) as a means to align the incentives of healthcare providers with public health goals. Rigorous evidence on the effectiveness of these strategies in improving health care and health in low- and middle-income countries (LMICs) is lacking; this is an update of the 2012 review on this topic.

OBJECTIVES

To assess the effects of paying for performance on the provision of health care and health outcomes in low- and middle-income countries.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, and 10 other databases between April and June 2018. We also searched two trial registries, websites, online resources of international agencies, organizations and universities, and contacted experts in the field. Studies identified from rerunning searches in 2020 are under 'Studies awaiting classification.'

SELECTION CRITERIA

We included randomized or non-randomized trials, controlled before-after studies, or interrupted time series studies conducted in LMICs (as defined by the World Bank in 2018). P4P refers to the transfer of money or material goods conditional on taking a measurable action or achieving a predetermined performance target. To be included, a study had to report at least one of the following outcomes: patient health outcomes, changes in targeted measures of provider performance (such as the delivery of healthcare services), unintended effects, or changes in resource use.

DATA COLLECTION AND ANALYSIS

We extracted data as per original review protocol and narratively synthesised findings. We used standard methodological procedures expected by Cochrane. Given diversity and variability in intervention types, patient populations, analyses and outcome reporting, we deemed meta-analysis inappropriate. We noted the range of effects associated with P4P against each outcome of interest. Based on intervention descriptions provided in documents, we classified design schemes and explored variation in effect by scheme design.

MAIN RESULTS

We included 59 studies: controlled before-after studies (19), non-randomized (16) or cluster randomized trials (14);  and interrupted time-series studies (9). One study included both an interrupted time series and a controlled before-after study. Studies focused on a wide range of P4P interventions, including target payments and payment for outputs as modified by quality (or quality and equity assessments). Only one study assessed results-based aid. Many schemes were funded by national governments (23 studies) with the World Bank funding most externally funded schemes (11 studies). Targeted services varied; however, most interventions focused on reproductive, maternal and child health indicators. Participants were predominantly located in public or in a mix of public, non-governmental and faith-based facilities (54 studies). P4P was assessed predominantly at health facility level, though districts and other levels were also involved. Most studies assessed the effects of P4P against a status quo control (49 studies); however, some studies assessed effects against comparator interventions (predominantly enhanced financing intended to match P4P funds (17 studies)). Four studies reported intervention effects against both comparator and status quo. Controlled before-after studies were at higher risk of bias than other study designs. However, some randomised trials were also downgraded due to risk of bias. The interrupted time-series studies provided insufficient information on other concurrent changes in the study context. P4P compared to a status quo control For health services that are specifically targeted, P4P may slightly improve health outcomes (low certainty evidence), but few studies assessed this. P4P may also improve service quality overall (low certainty evidence); and probably increases the availability of health workers, medicines and well-functioning infrastructure and equipment (moderate certainty evidence). P4P may have mixed effects on the delivery and use of services (low certainty evidence) and may have few or no distorting unintended effects on outcomes that were not targeted (low-certainty evidence), but few studies assessed these. For secondary outcomes, P4P may make little or no difference to provider absenteeism, motivation or satisfaction (low certainty evidence); but may improve patient satisfaction and acceptability (low certainty evidence); and may positively affect facility managerial autonomy (low certainty evidence). P4P probably makes little to no difference to management quality or facility governance (low certainty evidence). Impacts on equity were mixed (low certainty evidence). For health services that are untargeted, P4P probably improves some health outcomes (moderate certainty evidence); may improve the delivery, use and quality of some health services but may make little or no difference to others (low certainty evidence); and may have few or no distorting unintended effects (low certainty evidence). The effects of P4P on the availability of medicines and other resources are uncertain (very low certainty evidence). P4P compared to other strategies For health outcomes and services that are specifically targeted, P4P may make little or no difference to health outcomes (low certainty evidence), but few studies assessed this. P4P may improve service quality (low certainty evidence); and may have mixed effects on the delivery and use of health services and on the availability of equipment and medicines (low certainty evidence). For health outcomes and services that are untargeted, P4P may make little or no difference to health outcomes and to the delivery and use of health services (low certainty evidence). The effects of P4P on service quality, resource availability and unintended effects are uncertain (very low certainty evidence). Findings of subgroup analyses Results-based aid, and schemes using payment per output adjusted for service quality, appeared to yield the greatest positive effects on outcomes. However, only one study evaluated results-based aid, so the effects may be spurious. Overall, schemes adjusting both for quality of service and rewarding equitable delivery of services appeared to perform best in relation to service utilization outcomes.

AUTHORS' CONCLUSIONS

The evidence base on the impacts of P4P schemes has grown considerably, with study quality gradually increasing. P4P schemes may have mixed effects on outcomes of interest, and there is high heterogeneity in the types of schemes implemented and evaluations conducted. P4P is not a uniform intervention, but rather a range of approaches. Its effects depend on the interaction of several variables, including the design of the intervention (e.g., who receives payments ), the amount of additional funding,  ancillary components (such as technical support) and contextual factors (including organizational context).

How are pay-for-performance schemes in healthcare designed in low- and middle-income countries? Typology and systematic literature review

BACKGROUND

Pay for performance (P4P) schemes provide financial incentives to health workers or facilities based on the achievement of pre-specified performance targets and have been widely implemented in health systems across low and middle-income countries (LMICs). The growing evidence base on P4P highlights that (i) there is substantial variation in the effect of P4P schemes on outcomes and (ii) there appears to be heterogeneity in incentive design. Even though scheme design is likely a key determinant of scheme effectiveness, we currently lack systematic evidence on how P4P schemes are designed in LMICs.

METHODS

We develop a typology to classify the design of P4P schemes in LMICs, which highlights different design features that are a priori likely to affect the behaviour of incentivised actors. We then use results from a systematic literature review to classify and describe the design of P4P schemes that have been evaluated in LMICs. To capture academic publications, Medline, Embase, and EconLit databases were searched. To include relevant grey literature, Google Scholar, Emerald Insight, and websites of the World Bank, WHO, Cordaid, Norad, DfID, USAID and PEPFAR were searched.

RESULTS

We identify 41 different P4P schemes implemented in 29 LMICs. We find that there is substantial heterogeneity in the design of P4P schemes in LMICs and pinpoint precisely how scheme design varies across settings. Our results also highlight that incentive design is not adequately being reported on in the literature - with many studies failing to report key design features.

CONCLUSIONS

We encourage authors to make a greater effort to report information on P4P scheme design in the future and suggest using the typology laid out in this paper as a starting point.

Large-Scale Evaluation of Quality of Care in 6 Countries of Eastern Europe and Central Asia Using Clinical Performance and Value Vignettes

When providers in 6 different countries were asked how they would care for the same patient, there was wide variation within and between countries. Nevertheless, 11% of the physicians scored over 80%, suggesting good quality of care is possible even with resource constraints. Use of validated clinical vignettes, which can be applied affordably at scale, could help improve quality of services in low- and middle-income countries.

Background:

A significant determinant of population health outcomes is the quality of care provided for noncommunicable diseases, obstetric, and pediatric care. We present results on clinical practice quality in these areas as measured among nearly 4,000 providers working at more than 1,000 facilities in 6 Eastern European and Central Asian countries.

Methods:

This study was conducted between March 2011 and April 2013 in Albania, Armenia, Georgia, Kazakhstan, Kirov Province in Russia, and Tajikistan. Using a probability proportional-to-size sampling technique, based on number of hospital beds, we randomly selected within each country 42 hospitals and their associated primary health care clinics. Physicians and midwives within each clinical area of interest were randomly selected from each hospital and clinic and asked how they would care for simulated patients using Clinical Performance and Value (CPV) vignettes. Facility administrators were also asked to complete a facility survey to collect structural measures of quality. CPV vignettes were scored on a scale of 0% to 100% for each provider. We used descriptive statistics and t tests to identify significant differences in CPV scores between hospitals and clinics and rural vs. urban facilities, and ANOVA to identify significant differences in CPV scores across countries.

Results:

We found that quality of care, as concurrently measured by performance on CPV vignettes, was generally poor and widely variable within and between countries. Providers in Kirov Province, Russia, had the highest overall performance, with an average score of 70.8%, while providers in Albania and Tajikistan had the lowest average score, each at 50.8%. The CPV vignettes with the lowest scores were for multiple noncommunicable disease risk factors and birth asphyxia. A considerable proportion (11%) of providers performed well on the CPV vignettes, regardless of country, facility, or structural resources available to them.

Conclusions:

Countries of Eastern Europe and Central Asia are challenged by poor performance as measured by clinical care vignettes, but there is potential for provision of high-quality care by a sizable proportion of providers. Large-scale assessments of quality of care have been hampered by the lack of effective measurement tools that provide generalizable and reliable results across diverse economic, cultural, and social settings. The feasibility of quality measurement using CPV vignettes in these 6 countries and the ability to combine results with individual feedback could significantly enhance strategies to improve quality of care, and ultimately population health.