Improving district facility readiness: a 12-month evaluation of a data-driven health systems strengthening intervention in rural Rwanda

Data-driven evaluation framework for the effectiveness of rural vitalization in China: an empirical case study of Hubei Province

Rural vitalization (RV) has attracted more and more attention in China, especially since the Rural Vitalization Strategy (RVS) was proposed to restrict rural decline in 2017. The evaluation of RV is an effective means to objectively identify the characteristics and problems of rural development, so exploring scientific and rational evaluation methods is important for sustainable rural development. Therefore, this study builds a data-driven evaluation framework from a "bottom-up" perspective, and selects Hubei Province as the object to evaluate the effectiveness of RV. The evaluation index system is formed based on the concept and connotation of RV, which contains six dimensions, namely thriving businesses (TB), pleasant living environments (PLE), social etiquette and civility (SEC), effective governance (EG), living in prosperity (LP), and organization system (OS). The empirical results indicate that there is a low level of variation of the total scores but an obvious disparity in the dimensional scores in 13 prefecture-level and 83 county-level regions. At county-level, the regional development stage has an impact on the effectiveness of RV, and regions with a higher economy or endowed with better resources perform better. The results of spatial analysis further reveal that there is regional agglomeration as well as differences in various dimensions, and regions with characteristic industries or policy support perform better. Compared with the traditional evaluation method, differentiated evaluation objectives and diversified data are considered in the evaluation process of this study. The results and discussion shown in this study could provide empirical evidence for policymakers to effectively promote RV in the future.

Readiness to provide child health services in rural Uttar Pradesh, India: mapping, monitoring and ongoing supportive supervision

BACKGROUND

In 2018, 875 000 under-five children died in India with children from poor families and rural communities disproportionately affected. Community health centres are positioned to improve access to quality child health services but capacity is often low and the systems for improvements are weak.

METHODS

Secondary analysis of child health program data from the Uttar Pradesh Technical Support Unit was used to delineate how program activities were temporally related to public facility readiness to provide child health services including inpatient admissions. Fifteen community health centres were mapped regarding capacity to provide child health services in July 2015. Mapped domains included human resources and training, infrastructure, equipment, drugs/supplies and child health services. Results were disseminated to district health managers. Six months following dissemination, Clinical Support Officers began regular supportive supervision and gaps were discussed monthly with health managers. Senior pediatric residents mentored medical officers over a three-month period. Improvements were assessed using a composite score of facility readiness for child health services in July 2016. Usage of outpatient and inpatient services by under-five children was also assessed.

RESULTS

The median essential composition score increased from 0.59 to 0.78 between July 2015 and July 2016 (maximum score of 1) and the median desirable composite increased from 0.44 to 0.58. The components contributing most to the change were equipment, drugs and supplies and service provision. Scores for trained human resources and infrastructure did not change between assessments. The number of facilities providing some admission services for sick children increased from 1 in July 2015 to 9 in October 2016.

CONCLUSIONS

Facility readiness for the provision of child health services in Uttar Pradesh was improved with relatively low inputs and targeted assessment. However, these improvements were only translated into admissions for sick children when clinical mentoring was included in the support provided to facilities.

Factors associated with the performance of routine health information system in Yaoundé-Cameroon: a cross-sectional survey

Background

Routine Health Information Systems (RHIS) of low-income countries function below the globally expected standard, characterised by the production and use of poor-quality data, or the non-use of good quality data for informed decision making. This has negatively influenced the health service delivery and uptake. This study focuses on identifying the factors associated with the performance of RHIS of the health facilities (HF) in Yaoundé, so as to guide targeted RHIS strengthening.

Methods

A HF-based cross-sectional study in the 6 health districts (HDs) of Yaoundé was conducted. HFs were chosen using stratified sampling with probability proportional to size per HD. Data were collected, entered into Microsoft Excel 2013 and analysed with IBM- SPSS version 25. Consistency of the questionnaire was measured using Cronbach’s alpha coefficient. Pearson’s chi-square (and Fisher exact where relevant) tests were used to establish relationships between qualitative variables. Associations were further quantified using unadjusted Odd ratio (OR) for univariable analysis and adjusted odds ratio (aOR) for multivariable analysis with 95% confidence interval (CI). A p-value of less than 0.05 was considered statistically significant.

Results

Of 111 selected HFs; 16 (14.4%) were public and 95 (85.6%) private. Respondents aged 24–60 years with an average of 38.3 ± 9.3 years; 58 (52.3%) males and 53(47.7%) females. Cronbach’s alpha was 0.96 (95%CI: 0.95–0.98, p < 0.001), proving that the questionnaire was reliable in measuring RHIS performances. At univariable level, the following factors were positively associated with good performances: supportive supervision (OR = 3.03 (1.1, 8.3); p = 0.02), receiving feedback from hierarchy (OR = 3.6 (0.99, 13.2); p = 0.05), having received training on health information (OR = 5.0 (1.6, 16.0); p = 0.003), and presence of a performance evaluation plan (OR = 3.3 (1.4, 8.2), p = 0.007). At multivariable level, the only significantly associated factor was having received training on health information (aOR = 3.3 (1.01, 11.1), p = 0.04).

Conclusion

Training of health staff in the RHIS favors RHIS good performance. Hence, emphasis should be laid on training and empowering staff, frequent and regular RHIS supervision, and frequent and regular feedback, for an efficient RHIS strengthening in Yaoundé.

Advancing the health of women and newborns: predictors of patient satisfaction among women attending antenatal and maternity care in rural Rwanda

Objective

Identify predictors of patient satisfaction with antenatal care (ANC) and maternity services in rural Rwanda.

Design

Cross-sectional.

Setting

Twenty-six health facilities in Southern Kayonza (SK) and Kirehe districts.

Participants

Sample of women ≥ 16 years old receiving antenatal and delivery care between November and December 2013.

Intervention

Survey of patient satisfaction with antenatal and delivery care to inform quality improvement (QI) initiatives aimed at reducing neonatal mortality.

Main Outcome Measure

Overall satisfaction with antenatal and delivery care (reported as excellent or very good).

Results

In multivariate logistic regression analysis, high perceived quality [odds ratio (OR) = 3.03, 95% confidence intervals (CI): 1.565.88], respect [OR = 4.13, 95% CI: 2.16-7.89], and confidentiality [SK: OR = 7.50, 95% CI: 2.16-26.01], [Kirehe: OR = 1.54, 95% CI: 0.60-3.94] were associated with higher overall satisfaction with ANC, while having ≥1 child compared to none [OR = 0.46, 95% CI: 0.25-0.84] was associated with lower satisfaction. For maternity services, <5 years of school versus ≥5 years [OR = 0.13, 95% CI: 0.026-0.69] and higher cleanliness [OR = 19.23, 95% CI: 2.22-166.83], self-reported quality [OR = 10.52, 95% CI: 1.81-61.22], communication [OR = 8.78, 95%CI: 1.95-39.59], and confidentiality [OR = 8.66, 95% CI: 1.20-62.64] were all positively associated with high satisfaction. Higher comfort [OR: 0.050, 95% CI: 0.0034-0.71] and Kirehe vs. SK district [OR: 0.21, 95% CI: 0.042-1.01] were associated with lower satisfaction.

Conclusions

Patient-centeredness (including interpersonal relationships), organizational factors, and location are important individual determinants of satisfaction for women seeking maternal care at study facilities. Understanding variation in these factors should inform QI efforts in maternal and newborn health programs.

Successful implementation of a combined learning collaborative and mentoring intervention to improve neonatal quality of care in rural Rwanda

Background

Globally, neonatal mortality remains high despite interventions known to reduce neonatal deaths. The All Babies Count (ABC) initiative was a comprehensive health systems strengthening intervention designed by Partners In Health in collaboration with the Rwanda Ministry of Health to improve neonatal care in rural public facilities. ABC included provision of training, essential equipment, and a quality improvement (QI) initiative which combined clinical and QI mentorship within a learning collaborative. We describe ABC implementation outcomes, including development of a QI change package.

Methods

ABC was implemented over 18 months from 2013 to 2015 in two Rwandan districts of Kirehe and Southern Kayonza, serving approximately 500,000 people with 24 nurse-led health centers and 2 district hospitals. A process evaluation of ABC implementation and its impact on healthcare worker (HCW) attitudes and QI practice was done using program documents, standardized surveys and focus groups with facility QI team members attending ABC Learning Sessions. The Change Package was developed using mixed methods to identify projects with significant change according to quantitative indicators and qualitative feedback obtained during focus group discussions. Outcome measures included ABC implementation process measures, HCW-reported impact on attitudes and practice of QI, and resulting change package developed for antenatal care, delivery management and postnatal care.

Results

ABC was implemented across all 26 facilities with an average of 0.76 mentorship visits/facility/month and 118 tested QI change ideas. HCWs reported a reduction in barriers to quality care delivery related to training (p = 0.018); increased QI capacity (knowledge 37 to 89%, p <  0.001); confidence (47 to 89%, p <  0.001), QI leadership (59 to 91%, p <  0.001); and peer-to-peer learning (37 to 66%, p = 0.024). The final change package included 46 change ideas. Themes associated with higher impact changes included provision of mentorship and facility readiness support through equipment provision.

Conclusions

ABC provides a feasible model of an integrated approach to QI in rural Rwanda. This model resulted in increases in HCW and facility capacity to design and implement effective QI projects and facilitated peer-to-peer learning. ABC and the change package are being scaled to accelerate improvement in neonatal outcomes.

A Comparison of Health Achievements in Rwanda and Burundi

Strong primary health care systems are essential for implementing universal health coverage and fulfilling health rights entitlements, but disagreement exists over how best to create them. Comparing countries with similar histories, lifestyle practices, and geography but divergent health outcomes can yield insights into possible mechanisms for improvement. Rwanda and Burundi are two such countries. Both faced protracted periods of violence in the 1990s, leading to significant societal upheaval. In subsequent years, Rwanda's improvement in health has been far greater than Burundi's. To understand how this divergence occurred, we studied trends in life expectancy following the periods of instability in both countries, as well as the health policies implemented after these conflicts. We used the World Bank's World Development Indicators to assess trends in life expectancy in the two countries and then evaluated health policy reforms using Walt and Gilson's framework. Following both countries' implementation of health sector policies in 2005, we found a statistically significant increase in life expectancy in Rwanda after adjusting for GDP per capita (14.7 years, 95% CI: 11.4-18.0), relative to Burundi (4.6 years, 95% CI: 1.8-7.5). Strong public sector leadership, investments in health information systems, equity-driven policies, and the use of foreign aid to invest in local capacity helped Rwanda achieve greater health gains compared to Burundi.

Economic evaluation of a mentorship and enhanced supervision program to improve quality of integrated management of childhood illness care in rural Rwanda

Background

Integrated management of childhood illness (IMCI) can reduce under-5 morbidity and mortality in low-income settings. A program to strengthen IMCI practices through Mentorship and Enhanced Supervision at Health centers (MESH) was implemented in two rural districts in eastern Rwanda in 2010.

Methods

We estimated cost per improvement in quality of care as measured by the difference in correct diagnosis and correct treatment at baseline and 12 months of MESH. Costs of developing and implementing MESH were estimated in 2011 United States Dollars (USD) from the provider perspective using both top-down and bottom-up approaches, from programmatic financial records and site-level data. Improvement in quality of care attributed to MESH was measured through case management observations (n = 292 cases at baseline, 413 cases at 12 months), with outcomes from the intervention already published. Sensitivity analyses were conducted to assess uncertainty under different assumptions of quality of care and patient volume.

Results

The total annual cost of MESH was US$ 27,955.74 and the average cost added by MESH per IMCI patient was US$1.06. Salary and benefits accounted for the majority of total annual costs (US$22,400 /year). Improvements in quality of care after 12 months of MESH implementation cost US$2.95 per additional child correctly diagnosed and $5.30 per additional child correctly treated.

Conclusions

The incremental costs per additional child correctly diagnosed and child correctly treated suggest that MESH could be an affordable method for improving IMCI quality of care elsewhere in Rwanda and similar settings. Integrating MESH into existing supervision systems would further reduce costs, increasing potential for spread.

Data-driven quality improvement in low-and middle-income country health systems: lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia

BACKGROUND

Well-functioning health systems need to utilize data at all levels, from the provider, to local and national-level decision makers, in order to make evidence-based and needed adjustments to improve the quality of care provided. Over the last 7 years, the Doris Duke Charitable Foundation's African Health Initiative funded health systems strengthening projects at the facility, district, and/or provincial level to improve population health. Increasing data-driven decision making was a common strategy in Mozambique, Rwanda and Zambia. This paper describes the similar and divergent approaches to increase data-driven quality of care improvements (QI) and implementation challenge and opportunities encountered in these three countries.

METHODS

Eight semi-structured in-depth interviews (IDIs) were administered to program staff working in each country. IDIs for this paper included principal investigators of each project, key program implementers (medically-trained support staff, data managers and statisticians, and country directors), as well as Ministry of Health counterparts. IDI data were collected through field notes; interviews were not audio recorded. Data were analyzed using thematic analysis but no systematic coding was conducted. IDIs were supplemented through donor report abstractions, a structured questionnaire, one-on-one phone calls, and email exchanges with country program leaders to clarify and expand on key themes emerging from IDIs.

RESULTS

Project successes ranged from over 450 collaborative action-plans developed, implemented, and evaluated in Mozambique, to an increase from <10% to >80% of basic clinical protocols followed in intervention facilities in rural Zambia, and a shift from a lack of awareness of health data among health system staff to collaborative ownership of data and using data to drive change in Rwanda.

CONCLUSION

Based on common successes across the country experiences, we recommend future data-driven QI interventions begin with data quality assessments to promote that rapid health system improvement is possible, ensure confidence in available data, serve as the first step in data-driven targeted improvements, and improve staff data analysis and visualization skills. Explicit Ministry of Health collaborative engagement can ensure performance review is collaborative and internally-driven rather than viewed as an external "audit."

Impact of a district-wide health center strengthening intervention on healthcare utilization in rural Rwanda: Use of interrupted time series analysis

Background

Evaluations of health systems strengthening (HSS) interventions using observational data are rarely used for causal inference due to limited data availability. Routinely collected national data allow use of quasi-experimental designs such as interrupted time series (ITS). Rwanda has invested in a robust electronic health management information system (HMIS) that captures monthly healthcare utilization data. We used ITS to evaluate impact of an HSS intervention to improve primary health care facility readiness on health service utilization in two rural districts of Rwanda.

Methods

We used controlled ITS analysis to compare changes in healthcare utilization at health centers (HC) that received the intervention (n = 13) to propensity score matched non-intervention health centers in Rwanda (n = 86) from January 2008 to December 2012. HC support included infrastructure renovation, salary support, medical equipment, referral network strengthening, and clinical training. Baseline quarterly mean outpatient visit rates and population density were used to model propensity scores. The intervention began in May 2010 and was implemented over a twelve-month period. We used monthly healthcare utilization data from the national Rwandan HMIS to study changes in the (1) number of facility deliveries per 10,000 women, (2) number of referrals for high risk pregnancy per 100,000 women, and (3) the number of outpatient visits performed per 1,000 catchment population.

Results

PHIT HC experienced significantly higher monthly delivery rates post-HSS during the April-June season than comparison (3.19/10,000, 95% CI: [0.27, 6.10]). In 2010, this represented a 13% relative increase, and in 2011, this represented a 23% relative increase. The post-HSS change in monthly rate of high-risk pregnancies referred increased slightly in intervention compared to control HC (0.03/10,000, 95% CI: [-0.007, 0.06]). There was a small immediate post-HSS increase in outpatient visit rates in intervention compared to control HC (6.64/1,000, 95% CI: [-13.52, 26.81]).

Conclusion

We failed to find strong evidence of post-HSS increases in outpatient visit rates or referral rates at health centers, which could be explained by small sample size and high baseline nation-wide health service coverage. However, our findings demonstrate that high quality routinely collected health facility data combined with ITS can be used for rigorous policy evaluation in resource-limited settings.

Barriers to utilization of childbirth services of a rural birthing center in Nepal: A qualitative study

Background

Maternal mortality and morbidity are public health problems in Nepal. In rural communities, many women give birth at home without the support of a skilled birth attendant, despite the existence of rural birthing centers. The aim of this study was to explore the barriers and provide pragmatic recommendations for better service delivery and use of rural birthing centers.

Methods

We conducted 26 in-depth interviews with service users and providers, and three focus group discussions with community key informants in a rural community of Rukum district. We used the Adithya Cattamanchi logic model as a guiding framework for data analysis.

Results

Irregular and poor quality services, inadequate human and capital resources, and poor governance were health system challenges which prevented service delivery. Contextual barriers including difficult geography, poor birth preparedness practices, harmful culture practices and traditions and low level of trust were also found to contribute to underutilization of the birthing center.

Conclusion

The rural birthing center was not providing quality services when women were in need, which meant women did not use the available services properly because of systematic and contextual barriers. Approaches such as awareness-raising activities, local resource mobilization, ensuring access to skilled providers and equipment and other long-term infrastructure development works could improve the quality and utilization of childbirth services in the rural birthing center. This has resonance for other centers in Nepal and similar countries.

Opportunities and challenges in conducting secondary analysis of HIV programmes using data from routine health information systems and personal health information

INTRODUCTION

HIV programme data from routine health information systems (RHIS) and personal health information (PHI) provide ample opportunities for secondary data analysis. However, these data pose unique opportunities and challenges for use in health system monitoring, along with process and impact evaluations.

METHODS

Analyses focused on retrospective case reviews of four of the HIV-related studies published in this JIAS supplement. We identify specific opportunities and challenges with respect to the secondary analysis of RHIS and PHI data.

RESULTS

Challenges working with both HIV-related RHIS and PHI included missing, inconsistent and implausible data; rapidly changing indicators; systematic differences in the utilization of services; and patient linkages over time and different data sources. Specific challenges among RHIS data included numerous registries and indicators, inconsistent data entry, gaps in data transmission, duplicate registry of information, numerator-denominator incompatibility and infrequent use of data for decision-making. Challenges specific to PHI included the time burden for busy providers, the culture of lax charting, overflowing archives for paper charts and infrequent chart review.

CONCLUSIONS

Many of the challenges that undermine effective use of RHIS and PHI data for analyses are related to the processes and context of collecting the data, excessive data requirements, lack of knowledge of the purpose of data and the limited use of data among those generating the data. Recommendations include simplifying data sources, analysis and reporting; conducting systematic data quality audits; enhancing the use of data for decision-making; promoting routine chart review linked with simple patient tracking systems; and encouraging open access to RHIS and PHI data for increased use.