Building district-level capacity for continuous improvement in maternal and newborn health

Effectiveness of in-service training plus the collaborative improvement strategy on the quality of routine malaria surveillance data: results of a pilot study in Kayunga District, Uganda

BACKGROUND

Surveillance data are essential for malaria control, but quality is often poor. The aim of the study was to evaluate the effectiveness of the novel combination of training plus an innovative quality improvement method-collaborative improvement (CI)-on the quality of malaria surveillance data in Uganda.

METHODS

The intervention (training plus CI, or TCI), including brief in-service training and CI, was delivered in 5 health facilities (HFs) in Kayunga District from November 2015 to August 2016. HF teams monitored data quality, conducted plan-do-study-act cycles to test changes, attended periodic learning sessions, and received CI coaching. An independent evaluation was conducted to assess data completeness, accuracy, and timeliness. Using an interrupted time series design without a separate control group, data were abstracted from 156,707 outpatient department (OPD) records, laboratory registers, and aggregated monthly reports (MR) for 4 time periods: baseline-12 months, TCI scale-up-5 months; CI implementation-9 months; post-intervention-4 months. Monthly OPD register completeness was measured as the proportion of patient records with a malaria diagnosis with: (1) all data fields completed, and (2) all clinically-relevant fields completed. Accuracy was the relative difference between: (1) number of monthly malaria patients reported in OPD register versus MR, and (2) proportion of positive malaria tests reported in the laboratory register versus MR. Data were analysed with segmented linear regression modelling.

RESULTS

Data completeness increased substantially following TCI. Compared to baseline, all-field completeness increased by 60.1%-points (95% confidence interval [CI]: 46.9-73.2%) at mid-point, and clinically-relevant completeness increased by 61.6%-points (95% CI: 56.6-66.7%). A relative - 57.4%-point (95% confidence interval: - 105.5, - 9.3%) change, indicating an improvement in accuracy of malaria test positivity reporting, but no effect on data accuracy for monthly malaria patients, were observed. Cost per additional malaria patient, for whom complete clinically-relevant data were recorded in the OPD register, was $3.53 (95% confidence interval: $3.03, $4.15).

CONCLUSIONS

TCI improved malaria surveillance completeness considerably, with limited impact on accuracy. Although these results are promising, the intervention's effectiveness should be evaluated in more HFs, with longer follow-up, ideally in a randomized trial, before recommending CI for wide-scale use.

Continuous quality improvement as a tool to implement evidence-informed problem solving: experiences from the district and health facility level in Uganda

Background

Continuous quality improvement processes in health care were developed for use at health facility level, and that is where they have been used the most, often addressing defined care processes. However, in different settings different factors have been important to support institutionalization. This study explores how continuous quality improvement processes were institutionalized at the district level and at the health facility level in Uganda.

Methods

This qualitative study was carried out in seven districts in Uganda. Semi-structured interviews with key informants from the district health management teams and document review were conducted. Thematic analysis was used to analyze the data.

Results

All districts that participated in the study formed Continuous Quality Improvement (CQI) teams both at the district level and at the health facilities. The district CQI teams comprised of members from different departments within the district health office. District level CQI teams were mandated to take the lead in addressing management gaps and follow up CQI activities at the health facility level. Acceptability of quality improvement processes by the district leadership was identified across districts as supporting the successful implementation of CQI. However, high turnover of staff at health facility level was also reported as a detrimental to the successful implementation of quality improvement processes. Also the district health management teams did not engage much in addressing their own roles using continuous quality improvement.

Conclusion

The leadership and management provided by the district health management team was an important factor for the use of Continuous Quality Improvement principles within the district. The key roles of the district health team revolved around the institutionalisation of CQI at different levels of the health system, monitoring results of continuous quality improvement implementation, mobilising resources and health care delivery hence promoting the culture of quality, direct implementation of CQI, and creating an enabling environment for the lower-level health facilities to engage in CQI. High turnover of staff at health facility level was also reported as one of the challenges to the successful implementation of continuous quality improvement. The DHT did not engage much in addressing gaps in their own roles using continuous quality improvement.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06061-8.

What factors do make quality improvement work in primary health care? Experiences of maternal health quality improvement teams in three Puskesmas in Indonesia

BACKGROUND

Indonesia has been shifting from ensuring access to health services towards improving service quality. Accreditation has been used as quality assurance (QA) mechanism, first in hospitals and subsequently in primary health care facilities, including Puskesmas (community health centres). QA provides measures of whether services meet quality targets, but quality improvement (QI) is needed to make change and achieve improvements. QI is a cyclical process with cycles of problem identification, solution testing and observation. We investigated the factors which influenced the process of QI based on experience of maternal health QI teams in three Puskesmas in Cianjur district, West Java province, Indonesia.

METHODS

Qualitative data were collected using 28 in-depth interviews at two points of time: pre- (April 2016) and post- QI intervention (April 2017), involving national, provincial, district and Puskesmas managers; and Puskesmas QI team members. Thematic analysis of transcripts was conducted.

RESULTS

We found four main factors contributed to the process of QI: 1) leadership, including awareness and attitude of leader(s) towards QI, involvement of leader(s) in the QI process and decision-making in budget allocation for QI; 2) staff enthusiasm and multidisciplinary collaboration; 3) a culture where QI is integrated in existing responsibilities; and 4) the ongoing Puskesmas accreditation process, which increased the value of QI to the organisation.

CONCLUSION

Making QI a success in the decentralised Indonesian system requires action at four levels. At individual level, leadership attributes can create an internal quality environment and drive organisational cultural change. At team level, staff enthusiasm and collaboration can be triggered through engaging and tasking everyone in the QI process and having a shared vision of what quality should look like. At organisational level, QI should be integrated in planned activities, ensuring financial and human resources. Lastly, QI can be encouraged when it is implemented by the wider health system as part of national accreditation programmes.

Evaluation of the Safe Care, Saving Lives (SCSL) quality improvement collaborative for neonatal health in Telangana and Andhra Pradesh, India: a study protocol

Background: The collaborative quality improvement approach proposed by the Institute for Healthcare Improvement has the potential to improve coverage of evidence-based maternal and newborn health practices. The Safe Care, Saving Lives initiative supported the implementation of 20 evidence-based maternal and newborn care practices, targeting labour wards and neonatal care units in 85 public and private hospitals in Telangana and Andhra Pradesh, India.

Objective: We present a protocol for the evaluation of this programme which aims to (a) estimate the effect of the initiative on evidence-based care practices and mortality; (b) evaluate the mechanisms leading to changes in adherence to evidence-based practices, and their relationship with contextual factors; (c) explore the feasibility of scaling-up the approach.

Methods: The mixed-method evaluation is based on a plausibility design nested within a phased implementation. The 29 non-randomly selected hospitals comprising wave II of the programme were compared to the 31 remaining hospitals where the quality improvement approach started later. We assessed mortality and adherence to evidence-based practices at baseline and endline using abstraction of registers, checklists, observations and interviews in intervention and comparison hospitals. We also explored the mechanisms and drivers of change in adherence to evidence-based practices. Qualitative methods investigated the mechanisms of change in purposefully selected case study hospitals. A readiness assessment complemented the analysis of what works and why. We used a difference-in-difference approach to estimate the effects of the intervention on mortality and coverage. Thematic analysis was used for the qualitative data.

Discussion: This is the first quality improvement collaborative targeting neonatal health in secondary and tertiary hospitals in a middle-income country linked to a government health insurance scheme. Our process evaluation is theory driven and will refine hypotheses about how this quality improvement approach contributes to institutionalization of evidence-based practices.

Is quality affordable for community health systems? Costs of integrating quality improvement into close-to-community health programmes in five low-income and middle-income countries

Introduction

Countries aspiring to universal health coverage view close-to-community (CTC) providers as a low-cost means of increasing coverage. However, due to lack of coordination and unreliable funding, the quality of large-scale CTC healthcare provision is highly variable and routine data about service quality are not trustworthy. Quality improvement (QI) approaches are a means of addressing these issues, yet neither the costs nor the budget impact of integrating QI approaches into CTC programme costs have been assessed.

Methods

This paper examines the costs and budget impact of integrating QI into existing CTC health programmes in five countries (Ethiopia, Indonesia, Kenya, Malawi, Mozambique) between 2015 and 2017. The intervention involved: (1) QI team formation; (2) Phased training interspersed with supportive supervision; which resulted in (3) QI teams independently collecting and analysing data to conduct QI interventions. Project costs were collected using an ingredients approach from a health systems perspective. Based on project costs, costs of local adoption of the intervention were modelled under three implementation scenarios.

Results

Annualised economic unit costs ranged from $62 in Mozambique to $254 in Ethiopia per CTC provider supervised, driven by the context, type of community health model and the intensity of the intervention. The budget impact of Ministry-led QI for community health is estimated at 0.53% or less of the general government expenditure on health in all countries (and below 0.03% in three of the five countries).

Conclusion

CTC provision is a key component of healthcare delivery in many settings, so QI has huge potential impact. The impact is difficult to establish conclusively, but as a first step we have provided evidence to assess affordability of QI for community health. Further research is needed to assess whether QI can achieve the level of benefits that would justify the required investment.

Improving immunization capacity in Ethiopia through continuous quality improvement interventions: a prospective quasi-experimental study

BACKGROUND

Strong scientific evidence is needed to support low-income countries in building effective and sustainable immunization programs and proactively engaging in global vaccine development and implementation initiatives. This study aimed to implement and evaluate the effectiveness of system-wide continuous quality improvement (CQI) interventions to improve national immunization programme performance in Ethiopia.

METHODS

The study used a prospective, quasi-experimental design with an interrupted time-series analysis to collect data from 781 government health sectors (556 healthcare facilities, 196 district health offices, and 29 zonal health departments) selected from developing and emerging regions in Ethiopia. Procedures included baseline quality assessment of immunization programme and services using structured checklists; immunization systems strengthening using onsite technical support, training, and supportive supervision interventions in a Plan-Do-Check-Act cycle over 12 months; and collection and analysis of data at baseline and at the 6th and 12th month of interventions using statistical process control and the t-test. Outcome measures were the coverage of the vaccines pentavalent 3, measles, Bacillus Calmette-Guérin vaccine (BCG), Pneumococcal Conjugate Vaccine (PCV), as well as full vaccination status; while process measures were changes in human resources, planning, service delivery, logistics and supply, documentation, coordination and collaboration, and monitoring and evaluation. Analysis and interpretation of data adhered to SQUIRE 2.0 guidelines.

RESULTS

Prior to the interventions, vaccination coverage was low and all seven process indicators had an aggregate score of below 50%, with significant differences in performance at healthcare facility level between developing and emerging regions (P = 0.0001). Following the interventions, vaccination coverage improved significantly from 63.6% at baseline to 79.3% for pentavalent (P = 0.0001), 62.5 to 72.8% for measles (P = 0.009), 62.4 to 73.5% for BCG (P = 0.0001), 65.3 to 81.0% for PCV (P = 0.02), and insignificantly from 56.2 to 74.2% for full vaccination. All seven process indicators scored above 75% in all regions, with no significant differences found in performance between developing and emerging regions.

CONCLUSIONS

The CQI interventions improved immunization capacity and vaccination coverage in Ethiopia, where the unstable transmission patterns and intensity of infectious diseases necessitate for a state of readiness of the health system at all times. The approach was found to empower zone, district, and facility-level health sectors to exercise accountability and share ownership of immunization outcomes. While universal approaches can improve routine immunization, local innovative interventions that target local problems and dynamics are also necessary to achieve optimal coverage.

Experiences of healthcare middle managers in developing capacity and capability to manage complexity: a systematic review protocol

REVIEW QUESTION/OBJECTIVE

The objective of this review is to explore the experiences of healthcare middle managers in developing capacity and capability to manage in a leadership role characterized by high complexity.

Implementing the World Health Organization safe childbirth checklist in a district Hospital in Rwanda: a pre- and post-intervention study

Background

Worldwide maternal mortality remains high, with approximately 830 maternal deaths occurring each day. About 90% of these deaths occur in low-income countries. Evidenced-based essential birth practices administered during routine obstetrical care and childbirth are key to reducing maternal and neonatal deaths. The WHO Safe Childbirth Checklist (SCC) is a low-cost tool designed to ensure birth attendants perform 29 essential birth practices (EBP) at four critical periods in the birth continuum. This study aimed to evaluate compliance with EBP in Masaka District Hospital both before and after the implementation of the WHO-SCC.

Methods

This quality improvement project took place in the Masaka District Hospital in Rwanda. Observations of the 29 EBPs were done before and after WHO SCC implementation. The implementation process consisted of providing training in the use of the checklist to all clinical staff and posting SCC posters at different locations in the maternity unit.

Results

A total 391 birth events were observed pre-intervention and 389 post-intervention. The overall EBP compliance rate increased from 46% pre-intervention to 56% post-intervention (P = 0.005). Significant improvements were seen in 11 out of 29 EBPs.

Conclusion

The implementation of the WHO SCC improved the overall EBP compliance rate in Masaka District Hospital. Determining the root cause of low compliance rate of some EBP may allow for more successful implementation of EBP interventions in the future. After further study, the SCC should be considered for scale up.

Effects of the EQUIP quasi-experimental study testing a collaborative quality improvement approach for maternal and newborn health care in Tanzania and Uganda

BACKGROUND

Quality improvement is a recommended strategy to improve implementation levels for evidence-based essential interventions, but experience of and evidence for its effects in low-resource settings are limited. We hypothesised that a systemic and collaborative quality improvement approach covering district, facility and community levels, supported by report cards generated through continuous household and health facility surveys, could improve the implementation levels and have a measurable population-level impact on coverage and quality of essential services.

METHODS

Collaborative quality improvement teams tested self-identified strategies (change ideas) to support the implementation of essential maternal and newborn interventions recommended by the World Health Organization. In Tanzania and Uganda, we used a plausibility design to compare the changes over time in one intervention district with those in a comparison district in each country. Evaluation included indicators of process, coverage and implementation practice analysed with a difference-of-differences and a time-series approach, using data from independent continuous household and health facility surveys from 2011 to 2014. Primary outcomes for both countries were birth in health facilities, breastfeeding within 1 h after birth, oxytocin administration after birth and knowledge of danger signs for mothers and babies. Interpretation of the results considered contextual factors.

RESULTS

The intervention was associated with improvements on one of four primary outcomes. We observed a 26-percentage-point increase (95% CI 25-28%) in the proportion of live births where mothers received uterotonics within 1 min after birth in the intervention compared to the comparison district in Tanzania and an 8-percentage-point increase (95% CI 6-9%) in Uganda. The other primary indicators showed no evidence of improvement. In Tanzania, we saw positive changes for two other outcomes reflecting locally identified improvement topics. The intervention was associated with an increase in preparation of clean birth kits for home deliveries (31 percentage points, 95% CI 2-60%) and an increase in health facility supervision by district staff (14 percentage points, 95% CI 0-28%).

CONCLUSIONS

The systemic quality improvement approach was associated with improvements of only one of four primary outcomes, as well as two Tanzania-specific secondary outcomes. Reasons for the lack of effects included limited implementation strength as well a relatively short follow-up period in combination with a 1-year recall period for population-based estimates and a limited power of the study to detect changes smaller than 10 percentage points.

TRIAL REGISTRATION

Pan African Clinical Trials Registry: PACTR201311000681314.

Evaluating investment in quality improvement capacity building: a systematic review

PURPOSE

Leading health systems have invested in substantial quality improvement (QI) capacity building, but little is known about the aggregate effect of these investments at the health system level. We conducted a systematic review to identify key steps and elements that should be considered for system-level evaluations of investment in QI capacity building.

METHODS

We searched for evaluations of QI capacity building and evaluations of QI training programmes. We included the most relevant indexed databases in the field and a strategic search of the grey literature. The latter included direct electronic scanning of 85 relevant government and institutional websites internationally. Data were extracted regarding evaluation design and common assessment themes and components.

RESULTS

48 articles met the inclusion criteria. 46 articles described initiative-level non-economic evaluations of QI capacity building/training, while 2 studies included economic evaluations of QI capacity building/training, also at the initiative level. No system-level QI capacity building/training evaluations were found. We identified 17 evaluation components that fit within 5 overarching dimensions (characteristics of QI training; characteristics of QI activity; individual capacity; organisational capacity and impact) that should be considered in evaluations of QI capacity building. 8 key steps in return-on-investment (ROI) assessments in QI capacity building were identified: (1) planning-stakeholder perspective; (2) planning-temporal perspective; (3) identifying costs; (4) identifying benefits; (5) identifying intangible benefits that will not be included in the ROI estimation; (6) discerning attribution; (7) ROI calculations; (8) sensitivity analysis.

CONCLUSIONS

The literature on QI capacity building evaluation is limited in the number and scope of studies. Our findings, summarised in a Framework to Guide Evaluations of QI Capacity Building, can be used to start closing this knowledge gap.

Learning before leaping: integration of an adaptive study design process prior to initiation of BetterBirth, a large-scale randomized controlled trial in Uttar Pradesh, India

Background

Pragmatic and adaptive trial designs are increasingly used in quality improvement (QI) interventions to provide the strongest evidence for effective implementation and impact prior to broader scale-up. We previously showed that an on-site coaching intervention focused on the World Health Organization Safe Childbirth Checklist (SCC) improved performance of essential birth practices (EBPs) in one facility in Karnataka, India. We report on the process and outcomes of adapting the intervention prior to larger-scale implementation in a randomized controlled trial in Uttar Pradesh (UP), India.

Methods

Initially, we trained a local team of physicians and nurses to coach birth attendants in SCC use at two public facilities for 4–6 weeks. Trained observers evaluated adherence to EBPs before and after coaching. Using mixed methods and a systematic adaptation process, we modified and strengthened the intervention. The modified intervention was implemented in three additional facilities. Pre/post-change in EBP prevalence aggregated across facilities was analyzed.

Results

In the first two facilities, limited improvement was seen in EBPs with the exception of post-partum oxytocin. Checklists were used <25 % of observations. We identified challenges in physicians coaching nurses, need to engage district and facility leadership to address system gaps, and inadequate strategy for motivating SCC uptake. Revisions included change to peer-to-peer coaching (nurse to nurse, physician to physician); strengthened coach training on behavior and system change; adapted strategy for effective leadership engagement; and an explicit motivation strategy to enhance professional pride and effectiveness. These modifications resulted in improvement in multiple EBPs from baseline including taking maternal blood pressure (0 to 16 %), post-partum oxytocin (36 to 97 %), early breastfeeding initiation (3 to 64 %), as well as checklist use (range 32 to 88 %), all p < 0.01. Further adaptations were implemented to increase the effectiveness prior to full trial launch.

Conclusions

The adaptive study design of implementation, evaluation, and feedback drove iterative redesign and successfully developed a SCC-focused coaching intervention that improved EBPs in UP facilities. This work was critical to develop a replicable BetterBirth package tailored to the local context. The multi-center pragmatic trial is underway measuring impact of the BetterBirth program on EBP and maternal-neonatal morbidity and mortality.

Trial registration

Clinical trials identifier: NCT02148952.

The effect of community maternal and newborn health family meetings on type of birth attendant and completeness of maternal and newborn care received during birth and the early postnatal period in rural Ethiopia

INTRODUCTION

Maternal and newborn deaths occur predominantly in low-resource settings. Community-based packages of evidence-based interventions and skilled birth attendance can reduce these deaths. The Maternal and Newborn Health in Ethiopia Partnership (MaNHEP) used community-level health workers to conduct prenatal Community Maternal and Newborn Health family meetings to build skills and care-seeking behaviors among pregnant women and family caregivers.

METHODS

Baseline and endline surveys provided data on a random sample of women with a birth in the prior year. An intention-to-treat analysis, plausible net effect calculation, and dose-response analysis examined increases in completeness of care (mean percentage of 17 maternal and newborn health care elements performed) over time and by meeting participation. Regression models assessed the relationship between meeting participation, completeness of care, and use of skilled providers or health extension workers for birth care-controlling for sociodemographic and health service utilization factors.

RESULTS

A 151% increase in care completeness occurred from baseline to endline. At endline, women who participated in 2 or more meetings had more complete care than women who participated in fewer than 2 meetings (89% vs 76% of care elements; P < .001). A positive dose-response relationship existed between the number of meetings attended and greater care completeness (P < .001). Women with any antenatal care were nearly 3 times more likely to have used a skilled provider or health extension worker for birth care. Women who had additionally attended 2 or more meetings with family members were over 5 times as likely to have used these providers, compared to women without antenatal care and who attended fewer than 2 meetings (odds ratio, 5.19; 95% confidence interval, 2.88-9.36; P < .001).

DISCUSSION

MaNHEP's family meetings complemented routine antenatal care by engaging women and family caregivers in self-care and care-seeking, resulting in greater completeness of care and more highly skilled birth care.

Barriers and facilitators to accessing skilled birth attendants in Afar region, Ethiopia

OBJECTIVE

to explore barriers and facilitators that enable women to access skilled birth attendance in Afar Region, Ethiopia.

DESIGN

researchers used a Key Informant Research approach (KIR), whereby Health Extension Workers participated in an intensive training workshop and conducted interviews with Afar women in their communities. Data was also collected from health-care workers through questionnaires, interviews and focus groups.

PARTICIPANTS

fourteen health extension workers were key informants and interviewers; 33 women and eight other health-care workers with a range of experience in caring for Afar childbearing women provided data as individuals and in focus groups.

FINDINGS

participants identified friendly service, female skilled birth attendants (SBA) and the introduction of the ambulance service as facilitators to SBA. There are many barriers to accessing SBA, including women׳s low status and restricted opportunities for decision making, lack of confidence in health-care facilities, long distances, cost, domestic workload, and traditional practices which include a preference for birthing at home with a traditional birth attendant.

KEY CONCLUSIONS

many Afar men and women expressed a lack of confidence in the services provided at health-care facilities which impacts on skilled birth attendance utilisation.

IMPLICATIONS FOR PRACTICE

ambulance services that are free of charge to women are effective as a means to transfer women to a hospital for emergency care if required and expansion of ambulance services would be a powerful facilitator to increasing institutional birth. Skilled birth attendants working in institutions need to ensure their practice is culturally, physically and emotionally safe if more Afar women are to accept their midwifery care. Adequate equipping and staffing of institutions providing emergency obstetric and newborn care will assist in improving community perceptions of these services. Most importantly, mutual respect and collaboration between traditional birth attendants (Afar women׳s preferred caregiver), health extension workers and skilled birth attendants will help ensure timely consultation and referral and reduce delay for women if they require emergency maternity care.