Limitations of using administratively reported immunization data for monitoring routine immunization system performance in Nigeria

Cognizance and mitigation of falsified immunization documentation: Analyzing the consequences for public health in Nigeria, with a focus on counterfeited COVID-19 vaccination cards: A case report

Background and Aims

The global immunization campaign against COVID-19 has mandated vaccination certificates, leading to a surge in fake documentation. In Nigeria, the proliferation of counterfeit COVID-19 vaccination cards, facilitated by unscrupulous health workers, raises critical public health concerns. This research spotlights various forms of this malpractice, analyzes the factors contributing to the circulation of fake vaccination cards, their implications on public health, and provides recommendations for addressing the issue.

Methods

Extensive literature review and analysis were conducted to investigate the emergence of fake COVID-19 vaccination cards in Nigeria. Perspectives from health workers and reports from reputable undercover investigations were examined to identify factors contributing to the circulation of fraudulent records. The research also delved into corruption within the health sector and the impact of low salaries on healthcare workers. Additionally, global instances of fake vaccination cards were explored to provide a comprehensive understanding of the issue.

Results

Healthcare workers' vaccine hesitancy, corruption, and inadequate salaries were identified as key contributors to the circulation of fake vaccination cards in Nigeria. Instances of health workers accepting bribes to issue cards without administering vaccines were uncovered. The implications on public health included threats to herd immunity, compromised disease surveillance, erosion of public trust, and reinforcement of vaccine hesitancy. The research also highlighted global challenges with fake vaccination cards, emphasizing the need for international cooperation.

Conclusion

Fake vaccination cards in Nigeria poses challenges to public health, affecting the reliability of immunization data and jeopardizing disease control efforts. It is crucial to strengthen healthcare worker engagement, tackle corruption through increased transparency and improved policies, and implement digital vaccination verification systems. International collaboration is essential to establish standardized security measures and verification checkpoints. Addressing flawed vaccination records requires urgent action to enhance vaccination efforts, and safeguard the population from the resurgence of vaccine-preventable diseases.

Evaluation of Container Clinics as an Urban Immunization Strategy: Findings from the First Year of Implementation in Ghana, 2017-2018

BACKGROUND

In 2017, the Expanded Programme on Immunization in Ghana opened two container clinics in Accra, which were cargo containers outfitted to deliver immunizations. At each clinic, we assessed performance and clinic acceptance during the first 12 months of implementation.

METHODS

We employed a descriptive mixed-method design using monthly administrative immunization data, exit interviews with caregivers of children of <5 years (N = 107), focus group discussions (FGDs) with caregivers (n = 6 FGDs) and nurses (n = 2 FGDs), and in-depth interviews (IDIs) with community leaders (n = 3) and health authorities (n = 3).

RESULTS

Monthly administrative data showed that administered vaccine doses increased from 94 during the opening month to 376 in the 12th month across both clinics. Each clinic exceeded its target doses for the 12-23 month population (second dose of measles). Almost all (98%) exit interview participants stated that the clinics made it easier to receive child health services compared to previous health service interactions. The accessibility and acceptability of the container clinics were also supported from health worker and community perspectives.

CONCLUSIONS

Our initial data support container clinics as an acceptable strategy for delivering immunization services in urban populations, at least in the short term. They can be rapidly deployed and designed to serve working mothers in strategic areas.

Stakeholders' perspectives on internal accountability within a sub-national immunization program: A qualitative study in Enugu State, South-East Nigeria

BACKGROUND

Weak accountability hinders the effectiveness of routine immunization (RI) systems in low- and middle-income countries, yet studies on accountability of immunization programs are scarce. Aim: The study explored stakeholders' perspectives on the functioning of internal accountability within the National Program on Immunization in Enugu State, southeast Nigeria.

SUBJECTS AND METHODS

We used semi-structured in-depth interviews to collect data from RI officials at state government, local government, and health facility levels (n = 35) in Enugu State between June and July 2021. We adopted maximum variation sampling to purposively select individuals with roles in immunization. The interview guide was developed based on an accountability framework with three dimensions-the axes of power, ability, and justice. Data were analyzed thematically using NVivo software (version 11). The major themes were role clarity, performance standards, supervision, data use, human resources, funding, motivation, sanctions, political influence, and community engagement.

RESULTS

Performance targets for immunization coverage and reporting timeline were not always met due to multiple accountability failures. Weaknesses in the formal rules that distribute roles among the immunization workforce comprise a lack of deployment letters, unavailability of job descriptions, and inadequate staff orientation. Local officials have a narrow decision space regarding staff posting, transfer, and discipline. Performance accountability was constrained by staff shortages, uneven staff distribution, absenteeism, infrequent supervision, weak data monitoring system, and underfunding. Despite being motivated by job recognition and accomplishments, low motivation from an insecure working environment and lack of financial incentives undermined the constructive agency of service delivery actors. The sanctions framework exists but is weakly enforced due to fear of victimization. Political commitment to the immunization program was low. Yet, political decision-makers interfered with staff recruitment, distribution, and discipline. Community engagement improved resource availability through paid volunteer health workers and maintenance of facilities. However, health facility committees were poorly resourced, non-functional, and lacked the power to sanction erring health workers.

CONCLUSIONS

Immunization service delivery actors can be held accountable for program performance when there are sufficient formal instruments that provide roles and responsibilities, needed resources, motivated and supervised staff, an effective sanctions framework, genuine political participation, and strong community engagement.

Maternal Service Coverage and Its Relationship To Health Information System Performance: A Linked Facility and Population-Based Survey in Ethiopia

Coverage for most maternal services showed promising performance. Improving the health information system performance can further improve maternal service uptake and quality.

Background:

Studies in Ethiopia show an increasing trend in maternal health service use, such as having at least 4 visits of antenatal care (ANC4+) and skilled birth attendance (SBA). Improving the health information system (HIS) is an intervention that can improve service uptake and quality. We conducted a baseline study to measure current maternal service coverage, HIS performance status, and their relationship.

Methods:

We conducted a linked health facility-level and population-based survey from September 2020 to October 2020. The study covers all regions of Ethiopia. For the population-based survey, 3,016 mothers were included. Overall, 81 health posts, 71 health centers, and 15 hospitals were selected for the facility survey. A two-stage sampling procedure was applied to select target households. The study used modified Performance of Routine Information System Management tools for the facility survey and a structured questionnaire for the household survey. Multilevel logistic regression was employed to account for clustering and control for likely confounders.

Results:

Maternal service indicators, ANC4+ visits (54.0%), SBA (75.8%), postnatal care (70.6%), and cesarean delivery (9%) showed good service uptake. All data quality and use indicators showed lower performance compared to the national target of 90%. Maternal education and higher levels of wealth index were significantly and positively associated with all selected maternal service indicators. Longer distance from health facilities was significantly and negatively associated with SBA and the maternal care composite indicator. Among HIS-related indicators, availability of electronic HIS tools was significantly associated with maternal care composite indicator and ANC4+.

Conclusions:

Maternal service indicators showed promising performance. However, current HIS performance is suboptimal. Both service user and HIS-related factors were associated with maternal service uptake. Conducting similar research outside of the project sites will be helpful to have a wider understanding and better coverage.

Estimating population-based coverage of reproductive, maternal, newborn, and child health (RMNCH) interventions from health management information systems: a comprehensive review

Background

Routinely collected health facility data usually captured and stored in Health Management Information Systems (HMIS) are potential sources of data for frequent and local disaggregated estimation of the coverage of reproductive, maternal, newborn, and child health interventions (RMNCH), but have been under-utilized due to concerns over data quality. We reviewed methods for estimation of national or subnational coverage of RMNCH interventions using HMIS data exclusively or in conjunction with survey data from low- and middle-income countries (LMICs).

Methods

We conducted a comprehensive review of studies indexed in PubMed and Scopus to identify potential papers based on predefined search terms. Two reviewers screened the papers using defined inclusion and exclusion criteria. Following sequences of title, abstract and full paper reviews, we retained 18 relevant papers.

Results

12 papers used only HMIS data and 6 used both HMIS and survey data. There is enormous lack of standards in the existing methods for estimating RMNCH intervention coverage; all appearing to be highly author dependent. The denominators for coverage measures were estimated using census, non-census and combined projection-based methods. No satisfactory methods were found for treatment-based coverage indicators for which the estimation of target population requires the population prevalence of underlying conditions. The estimates of numerators for the coverage measures were obtained from the count of users or visits and in some cases correction for completeness of reporting in the HMIS following an assessment of data quality.

Conclusions

Standard methods for correcting numerators from HMIS data for accurate estimation of coverage of RMNCH interventions are needed to expand the use of these data. More research and investments are required to improve denominators for health facility-derived statistics. Improvement in routine data quality and analytical methods would allow for timely estimation of RMNCH intervention coverage at the national and subnational levels.

Measuring coverage of maternal and child health services using routine health facility data: a Sierra Leone case study

BACKGROUND

There are limited existing approaches to generate estimates from Routine Health Information Systems (RHIS) data, despite the growing interest to these data. We calculated and assessed the consistency of maternal and child health service coverage estimates from RHIS data, using census-based and health service-based denominators in Sierra Leone.

METHODS

We used Sierra Leone 2016 RHIS data to calculate coverage of first antenatal care contact (ANC1), institutional delivery and diphtheria-pertussis-tetanus 3 (DPT3) immunization service provision. For each indicator, national and district level coverages were calculated using denominators derived from two census-based and three health service-based methods. We compared the coverage estimates from RHIS data to estimates from MICS 2017. We considered the agreement adequate when estimates from RHIS fell within the 95% confidence interval of the survey estimate.

RESULTS

We found an overall poor consistency of the coverage estimates calculated from the census-based methods. ANC1 and institutional delivery coverage estimates from these methods were greater than 100% in about half of the fourteen districts, and only 3 of the 14 districts had estimates consistent with the survey data. Health service-based methods generated better estimates. For institutional delivery coverage, five districts met the agreement criteria using BCG service-based method. We found better agreement for DPT3 coverage estimates using DPT1 service-based method as national coverage was close to survey data, and estimates were consistent for 8 out of 14 districts. DPT3 estimates were consistent in almost half of the districts (6/14) using ANC1 service-based method.

CONCLUSION

The study highlighted the challenge in determining an appropriate denominator for RHIS-based coverage estimates. Systematic and transparent data quality check and correction, as well as rigorous approaches to determining denominators are key considerations to generate accurate coverage statistics using RHIS data.

Estimation of Ethiopia's immunization coverage - 20 years of discrepancies

Background

Coverage with the third dose of diphtheria-pertussis-tetanus-containing vaccine (DPT3) is a widely used measure of the performance of routine immunization systems. Since 2015, data reported by Ethiopia’s health facilities have suggested DPT3 coverage to be greater than 95%. Yet, Demographic and Health Surveys in 2016 and 2019 found DPT3 coverage to be 53 and 61% respectively for years during this period. This case study reviews the last 20 years of administrative (based on facility data), survey and United Nations (UN) estimates of Ethiopia’s nationwide immunization coverage to document long-standing discrepancies in these statistics.

Methods

Published estimates were compiled of Ethiopia’s nationwide DPT3 coverage from 1999 to 2018. These estimates come from the Joint Reporting Form submitted annually to WHO and UNICEF, a series of 8 population-based surveys and the annual reports of the WHO/UNICEF Estimates of National Immunization Coverage (WUENIC). Possible reasons for variation in survey findings were explored through secondary analysis of data from the 2012 immunization coverage survey. In addition, selected health officials involved with management of the immunization program were interviewed to obtain their perspectives on the reliability of various methods for estimation of immunization coverage.

Findings

Comparison of Ethiopia’s estimates for the same year from different sources shows major and persistent discrepancies between administrative, survey and WUENIC estimates. Moreover, the estimates from each of these sources have repeatedly shown erratic year-to-year fluctuations. Those who were interviewed expressed scepticism of Demographic and Health Survey (DHS) statistics. Officials of the national immunization programme have repeatedly shown a tendency to overlook all survey statistics when reporting on programme performance.

Conclusions

The present case study raises important questions, not only about the estimation methods of national and UN agencies, but about the reliability and comparability of widely trusted coverage surveys. Ethiopia provides an important example of a country where no data source provides a truly robust “gold standard” for estimation of immunization coverage. It is essential to identify and address the reasons for these discrepancies and arrive at a consensus on how to improve the reliability and acceptability of each data source and how best to “triangulate” between them.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06568-0.

Can routine health facility data be used to monitor subnational coverage of maternal, newborn and child health services in Uganda?

Background

Routine health facility data are a critical source of local monitoring of progress and performance at the subnational level. Uganda has been using district health statistics from facility data for many years. We aimed to systematically assess data quality and examine different methods to obtain plausible subnational estimates of coverage for maternal, newborn and child health interventions.

Methods

Annual data from the Uganda routine health facility information system 2015–2019 for all 135 districts were used, as well as national surveys for external comparison and the identification of near-universal coverage interventions. The quality of reported data on antenatal and delivery care and child immunization was assessed through completeness of facility reporting, presence of extreme outliers and internal data consistencies. Adjustments were made when necessary. The denominators for the coverage indicators were derived from population projections and health facility data on near-universal coverage interventions. The coverage results with different denominators were compared with the results from household surveys.

Results

Uganda’s completeness of reporting by facilities was near 100% and extreme outliers were rare. Inconsistencies in reported events, measured by annual fluctuations and between intervention consistency, were common and more among the 135 districts than the 15 subregions. The reported numbers of vaccinations were improbably high compared to the projected population of births or first antenatal visits – and especially so in 2015–2016. There were also inconsistencies between the population projections and the expected target population based on reported numbers of antenatal visits or immunizations. An alternative approach with denominators derived from facility data gave results that were more plausible and more consistent with survey results than based on population projections, although inconsistent results remained for substantive number of subregions and districts.

Conclusion

Our systematic assessment of the quality of routine reports of key events and denominators shows that computation of district health statistics is possible with transparent adjustments and methods, providing a general idea of levels and trends for most districts and subregions, but that improvements in data quality are essential to obtain more accurate monitoring.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06554-6.

Establishing evidence-based decision-making mechanism in a health eco-system and its linkages with health service coverage in 25 high-priority districts of Uttar Pradesh, India

Background

Achievement of successful health outcomes depends on evidence-based programming and implementation of effective health interventions. Routine Health Management Information System is one of the most valuable data sets to support evidence-based programming, however, evidence on systemic use of routine monitoring data for problem-solving and improving health outcomes remain negligible. We attempt to understand the effects of systematic evidence-based review mechanism on improving health outcomes in Uttar Pradesh, India.

Methods

Data comes from decision-tracking system and routine health management information system for period Nov-2017 to Mar-2019 covering 6963 health facilities across 25 high-priority districts of the state. Decision-tracking data captured pattern of decisions taken, actions planned and completed, while the latter one provided information on service coverage outcomes over time. Three service coverage indicators, namely, pregnant women receiving 4 or more times ANC and haemoglobin testing during pregnancy, delivered at the health facility, and receive post-partum care within 48 h of delivery were used as outcomes. Univariate and bivariate analyses were conducted.

Results

Total 412 decisions were taken during the study reference period and a majority were related to ante-natal care services (31%) followed by delivery (16%) and post-natal services (16%). About 21% decisions-taken were focused on improving data quality. By 1 year, 67% of actions planned based on these decisions were completed, 26% were in progress, and the remaining 7% were not completed. We found that, over a year, districts witnessing > 20 percentage-point increase in outcomes were also the districts with significantly higher action completion rates (> 80%) compared to the districts with < 10 percentage-point increase in outcomes having completion of action plans around 50–70%.

Conclusions

Findings revealed a significantly higher improvement in coverage outcomes among the districts which used routine health management data to conduct monthly review meetings and had high actions completion rates. A data-based review-mechanisms could specifically identify programmatic gaps in service delivery leading to strategic decision making by district authorities to bridge the programmatic gaps. Going forward, establishing systematic evidence-based review platforms can be an important strategy to improve health outcomes and promote the use of routine health monitoring system data in any setting.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06172-2.

Factors associated with incomplete immunisation in children aged 12-23 months at subnational level, Nigeria: a cross-sectional study

OBJECTIVES

National immunisation coverage rate masks subnational immunisation coverage gaps at the state and local district levels. The objective of the current study was to determine the sociodemographic factors associated with incomplete immunisation in children at a sub-national level.

DESIGN

Cross-sectional study using the WHO sampling method (2018 Reference Manual).

SETTING

Fifty randomly selected clusters (wards) in four districts (two urban and two rural) in Enugu state, Nigeria.

PARTICIPANTS

1254 mothers of children aged 12-23 months in July 2020.

PRIMARY AND SECONDARY OUTCOME MEASURES

Fully immunised children and not fully immunised children.

RESULTS

Full immunisation coverage (FIC) rate in Enugu state was 78.9% (95% CI 76.5% to 81.1%). However, stark difference exists in FIC rate in urban versus rural districts. Only 55.5% of children in rural communities are fully immunised compared with 94.5% in urban communities. Significant factors associated with incomplete immunisation are: children of single mothers (aOR=5.74, 95% CI 1.45 to 22.76), children delivered without skilled birth attendant present (aOR=1.93, 95% CI 1.24 to 2.99), children of mothers who did not receive postnatal care (aOR=6.53, 95% CI 4.17 to 10.22), children of mothers with poor knowledge of routine immunisation (aOR=1.76, 95% CI 1.09 to 2.87), dwelling in rural district (aOR=7.49, 95% CI 4.84 to 11.59), low-income families (aOR=1.56, 95% CI 1.17 to 2.81) and living further than 30 min from the nearest vaccination facility (aOR=2.15, 95% CI 1.31 to 3.52).

CONCLUSIONS

Although the proportion of fully immunised children in Enugu state is low, it is significantly lower in rural districts. Study findings suggest the need for innovative solutions to improve geographical accessibility and reinforce the importance of reporting vaccination coverage at local district level to identify districts for more targeted interventions.

An epidemiological investigation of the 2019 suspected pertussis outbreak in northwestern Nigeria

Introduction:

In 2019, we investigated the profile of the cases and controls and the determinants of pertussis transmission in Kebbi State, Northwestern Nigeria, to inform better immunization and surveillance strategies.

Methods:

Community-based unmatched case–control study and review of the 2019 pertussis routine surveillance data in the affected settlements in the state were conducted. A total of 52 suspected cases of pertussis and 107 control from two local government areas in Kebbi State were recruited. Data were analyzed using descriptive and inferential statistics.

Results:

The highest attack rate was observed among between 1- and 4-year age group followed by children less than 1-year old, and the least attack rate was among those above 15 years. The overall attack rate and the case fatality rate were 2.10% and 0.10%, respectively. A higher attack rate was observed among women, whereas the case fatality rate was more among males. From the community survey, we observed that the cases were less likely to have pertussis vaccination history (adjusted odds ratio = 0.28, 95% confidence interval = 0.11–0.74) compared with the controls. Knowing pertussis prevention methods were found protective for pertussis transmission (adjusted odds ratio = 0.14, 95% confidence interval = 0.04–0.45).

Conclusion:

This study showed the vulnerability of children under 5 years, especially under 1 year, to vaccine-preventable diseases in rural populations, where “real” immunization coverage is sub-optimal, and the dominant socio-demographic factors are supportive of disease transmission. We found immunization and knowledge of the preventive measures to be protective against pertussis outbreaks. Therefore, routine immunization services must be intensified to improve coverage and prevent future pertussis outbreak(s).

Engaging traditional barbers to identify and refer newborns for routine immunization services in Sokoto, Nigeria: a mixed methods evaluation

OBJECTIVES

This study evaluates the effectiveness of an intervention that engaged traditional barbers to inform parents about the importance of vaccination and then refer newborns for vaccination services.

METHODS

We conducted a pre-post quasi-experimental study (n = 2639) to evaluate changes in the coverage of three birth antigens among children aged 0-5 months in response to the intervention. We also conducted in-depth interviews and focus group discussions to assess the enabling factors and challenges associated with implementation.

RESULTS

We found mothers who received a yellow referral card from a traditional barber were two to three times more likely to vaccinate their children with the three birth antigens. Qualitative findings indicated that the intervention influenced parent's decision to vaccinate their newborn because the barbers were considered a trusted community advisor. Challenges stemmed from the low levels of literacy among community leaders and barbers that resulted in the need for continuous training, low-literacy training materials and supervision.

CONCLUSIONS

Efforts to increase vaccine coverage rates in northern Nigeria should consider expanding the role of traditional barbers to encourage parents to accept vaccines.

Improving the quality and use of immunization and surveillance data: Summary report of the Working Group of the Strategic Advisory Group of Experts on Immunization

Concerns about the quality and use of immunization and vaccine-preventable disease (VPD) surveillance data have been highlighted on the global agenda for over two decades. In August 2017, the Strategic Advisory Group of Experts (SAGE) established a Working Group (WG) onthe Quality and Use of Global Immunization and Surveillance Data to review the current status and evidence to make recommendations, which were presented to SAGE in October 2019. The WG synthesized evidence from landscape analyses, literature reviews, country case-studies, a data triangulation analysis, as well as surveys of experts. Data quality (DQ) was defined as data that are accurate, precise, relevant, complete, and timely enough for the intended purpose (fit-for-purpose), and data use as the degree to which data are actually used for defined purposes, e.g., immunization programme management, performance monitoring, decision-making. The WG outlined roles and responsibilities for immunization and surveillance DQ and use by programme level. The WG found that while DQ is dependent on quality data collection at health facilities, many interventions have targeted national and subnational levels, or have focused on new technologies, rather than the people and enabling environments required for functional information systems. The WG concluded that sustainable improvements in immunization and surveillance DQ and use will require efforts across the health system - governance, people, tools, and processes, including use of data for continuous quality improvement (CQI) - and that the approaches need to be context-specific, country-owned and driven from the frontline up. At the country level, major efforts are needed to: (1) embed monitoring DQ and use alongside monitoring of immunization and surveillance performance, (2) increase workforce capacity and capability for DQ and use, starting at the facility level, (3) improve the accuracy of immunization programme targets (denominators), (4) enhance use of existing data for tailored programme action (e.g., immunization programme planning, management and policy-change), (5) adopt a data-driven CQI approach as part of health system strengthening, (6) strengthen governance around piloting and implementation of new information and communication technology tools, and (7) improve data sharing and knowledge management across areas and organizations for improved transparency and efficiency. Global and regional partners are requested to support countries in adopting relevant recommendations for their setting and to continue strengthening the reporting and monitoring of immunization and VPD surveillance data through processes periodic needs assessment and revision processes. This summary of the WG's findings and recommendations can support "data-guided" implementation of the new Immunization Agenda 2030.

Impact of state weights on national vaccination coverage estimates from household surveys in Nigeria

National vaccination coverage estimates from household surveys are widely used in monitoring and planning of immunization programs. In Nigeria, survey-reported national coverage estimates have shown large fluctuations in the past few years. In this paper, we examine the impact of state-level survey weighting on Nigeria's national vaccination coverage estimation. In particular, we focus three vaccination-related outcomes among children aged 12-23 months: the coverage of the third dose of diphtheria, pertussis, and tetanus vaccine (DPT3); the coverage of the first dose of measles-containing vaccine (MCV1); and the availability rate of home-based vaccination record (HBR). We compare the sample selection and weight assignment of three major survey programs in Nigeria, and show that considerable portions of the changes in survey-reported national coverage estimates can be explained by shifts in state-level weights. Our analysis demonstrates the importance of state weighting method in estimating aggregated national coverage figures and provides important context for interpreting changes in coverage estimates between surveys in the future.

Generating statistics from health facility data: the state of routine health information systems in Eastern and Southern Africa

Health facility data are a critical source of local and continuous health statistics. Countries have introduced web-based information systems that facilitate data management, analysis, use and visualisation of health facility data. Working with teams of Ministry of Health and country public health institutions analysts from 14 countries in Eastern and Southern Africa, we explored data quality using national-level and subnational-level (mostly district) data for the period 2013-2017. The focus was on endline analysis where reported health facility and other data are compiled, assessed and adjusted for data quality, primarily to inform planning and assessments of progress and performance. The analyses showed that although completeness of reporting was generally high, there were persistent data quality issues that were common across the 14 countries, especially at the subnational level. These included the presence of extreme outliers, lack of consistency of the reported data over time and between indicators (such as vaccination and antenatal care), and challenges related to projected target populations, which are used as denominators in the computation of coverage statistics. Continuous efforts to improve recording and reporting of events by health facilities, systematic examination and reporting of data quality issues, feedback and communication mechanisms between programme managers, care providers and data officers, and transparent corrections and adjustments will be critical to improve the quality of health statistics generated from health facility data.

Accuracy and quality of immunization data in Iran: findings from data quality self-assessment survey in 2017

BACKGROUND

The aim of this study was to assess the accuracy and quality of immunization data on the pentavalent (diphtheria, pertussis, tetanus, hepatitis B and Haemophilus influenzae type B (Hib)) and MMR vaccines as the administrative data of the expanded program on immunization (EPI) in Iran.

METHODS

We conducted a Data Quality Self-assessment (DQS) survey from October to December 2017. Standardized DQS tools were used to assess the accuracy of reported immunizations data and quality of the immunization monitoring system at the provincial level of the healthcare system including health houses, health posts, rural and urban health centers and district health centers. Multistage cluster random sampling with proportional to size (PPS) weights was used to select target provinces and related health units. Accuracy ratio, quality index (QI), completeness and relevant quality indices of first dose of MMR (MMR1) and third dose of pentavalent vaccines were reported. Corresponding period of the survey was limited to reported administrative immunization data during the first 6 months of 2016.

RESULTS

In relation to accuracy ratio, there was some evidence of under reporting of pentavalent (3rd dose) and MMR1 vaccines in health house units which were 100.94 and 101.1%, respectively. Completeness of reporting for both vaccines at different provincial levels was near 100%. However, the corresponding value for pentavalent (3rd dose) and MMR1 vaccines at the level of urban health centers was 96.67 and 94.17% respectively. Among the five components of a monitoring system data usage and core output had the lowest QI scores in either rural or urban as well as district healthcare centers.

CONCLUSIONS

Findings from our DQS survey reveals that administrative reporting of the immunization data was adequate at provincial and district levels of the healthcare centers. Although, addressing the existing concerns regarding timelines of the reporting by health authorities and staffs of EPI is warranted.

Routine immunization community surveys as a tool for guiding program implementation in Kaduna state, Nigeria 2015-2016

Background

Routine childhood immunization remains an important strategy for achieving polio eradication and maintaining a polio-free world. To address gaps in reported administrative coverage data, community surveys were conducted to verify coverage, and guide strategic interventions for improved coverage.

Methods

We reviewed the conduct of community surveys by World Health Organization (WHO) field volunteers deployed as part of the surge capacity to Kaduna state and the use of survey results between July 2015 and June 2016. Monthly and quarterly collation and use of these data to guide the deployment of various interventions aimed at strengthening routine immunization in the state.

Results

Over 97,000 children aged 0–11 months were surveyed by 138 field volunteers across 237 of the 255 wards in Kaduna state. Fully or appropriately immunized children increased from 67% in the fourth quarter of 2015 to 76% by the end of the second quarter of 2016. Within the period reviewed, the number of local government areas with < 80% coverage reduced from eight to zero.

Conclusions

The routine conduct of community surveys by volunteers to inform interventions has shown an improvement in the vaccination status of children 0–11 months in Kaduna state and remains a useful tool in addressing administrative data quality issues.

Data for decision making: using a dashboard to strengthen routine immunisation in Nigeria

Availability of reliable data has for a long time been a challenge for health programmes in Nigeria. Routine immunisation (RI) data have always been characterised by conflicting coverage figures for the same vaccine across different routine data reporting platforms.

Following the adoption of District Health Information System version 2 (DHIS2) as a national electronic data management platform, the DHIS2 RI Dashboard Project was initiated to address the absence of some RI-specific indicators on DHIS2. The project was also intended to improve visibility and monitoring of RI indicators as well as strengthen the broader national health management information system by promoting the use of routine data for decision making at all governance levels. This paper documents the process, challenges and lessons learnt in implementing the project in Nigeria.

A multistakeholder technical working group developed an implementation framework with clear preimplementation; implementation and postimplementation activities. Beginning with a pilot in Kano state in 2014, the project has been scaled up countrywide.

Nearly 34 000 health workers at all administrative levels were trained on RI data tools and DHIS2 use. The project contributed to the improvement in completeness of reports on DHIS2 from 53 % in first quarter 2014 to 81 % in second quarter 2017.

The project faced challenges relating to primary healthcare governance structures at the subnational level, infrastructure and human resource capacity. Our experience highlights the need for early and sustained advocacy to stakeholders in a decentralised health system to promote ownership and sustainability of a centrally coordinated systems strengthening initiative.

Enhancing Workforce Capacity to Improve Vaccination Data Quality, Uganda

In Uganda, vaccine dose administration data are often not available or are of insufficient quality to optimally plan, monitor, and evaluate program performance. A collaboration of partners aimed to address these key issues by deploying data improvement teams (DITs) to improve data collection, management, analysis, and use in district health offices and health facilities. During November 2014–September 2016, DITs visited all districts and 89% of health facilities in Uganda. DITs identified gaps in awareness and processes, assessed accuracy of data, and provided on-the-job training to strengthen systems and improve healthcare workers’ knowledge and skills in data quality. Inaccurate data were observed primarily at the health facility level. Improvements in data management and collection practices were observed, although routine follow-up and accountability will be needed to sustain change. The DIT strategy offers a useful approach to enhancing the quality of health data.

Caregiver recall in childhood vaccination surveys: Systematic review of recall quality and use in low- and middle-income settings

INTRODUCTION

High population coverage is key to the impact of vaccines. However, vaccine coverage estimates in low- and middle-income countries (LMICs) have repeatedly been shown to be of poor quality. LMICs often rely on 'caregiver recall' of vaccination, the validity and collection method of which remains uncertain. We aimed to critique the quality of caregiver recall and make recommendations for its collection and use.

METHODS

We performed a systematic review for methods assessing childhood vaccination coverage in LMICs. We searched Medline using variations of the key terms: (child) AND (vaccinat^∗) AND (survey OR recall OR coverage) AND (reliab^∗ OR valid^∗). We selected articles assessing the quality of recall in LMICs and extracted reported validity, reliability and completeness. We synthesised recommendations on collecting, analysing and presenting caregiver recall for varying resource availabilities.

RESULTS

Of 1268 articles, 134 full texts were screened and eight were included for review. There was heterogeneity in study designs, ways of incorporating recall data and outcomes measured. Sensitivity of recall was 41-98%; specificity was 12-80%. There was a dearth of reliability measures and no consistent method for dealing with data incompleteness.

CONCLUSION

There are quality concerns with caregiver recall and difficulty in assessing it given the lack of a 'gold standard' for vaccine status. To improve coverage estimates and the impact of vaccines, caregiver recall should be used. Other recommendations include: recall is included for those presenting vaccine records; missing data is imputed; recall and record quality are assessed in a sub-sample; and sensitivity analyses are performed.

Oral Cholera Vaccine Coverage during an Outbreak and Humanitarian Crisis, Iraq, 2015

During November–December 2015, as part of the 2015 cholera outbreak response in Iraq, the Iraqi Ministry of Health targeted ≈255,000 displaced persons >1 year of age with 2 doses of oral cholera vaccine (OCV). All persons who received vaccines were living in selected refugee camps, internally displaced persons camps, and collective centers. We conducted a multistage cluster survey to obtain OCV coverage estimates in 10 governorates that were targeted during the campaign. In total, 1,226 household and 5,007 individual interviews were conducted. Overall, 2-dose OCV coverage in the targeted camps was 87% (95% CI 85%–89%). Two-dose OCV coverage in the 3 northern governorates (91%; 95% CI 87%–94%) was higher than that in the 7 southern and central governorates (80%; 95% CI 77%–82%). The experience in Iraq demonstrates that OCV campaigns can be successfully implemented as part of a comprehensive response to cholera outbreaks among high-risk populations in conflict settings.

A pilot study of routine immunization data quality in Bunza Local Government area: causes and possible remedies

Introduction

As a result of poor quality administrative data for routine immunisation (RI) in Nigeria, the real coverage of RI remains unknown, constituting a setback in curtailing vaccine preventable diseases (VPDs). Consequently, the purpose of this pilot study is to identify source(s) and evaluate the magnitude of poor data quality as well as propose recommendations to address the problem.

Methods

The authors conducted a cross-sectional study in which 5 out of the 22 health facilities providing routine immunization services in Bunza Local Government Area (LGA), Kebbi State, Nigeria, were selected for data quality assessment. The reported coverage of RI in August and September, 2016 was the primary element of evaluation in the selected Health Facilities (HFs). Administered questionnaires were adapted from WHO Data Quality Assurance and RI monitoring tools to generate data from the HFs, as well as standardised community survey tool for household surveys.

Results

Data inconsistency was detected in 100% of the selected HFs. Maximum difference between HF monthly summary and RI registration book for penta 3 data quality report analysis was 820% and 767% in MCH Bunza and PHC Balu respectively. However, a minimum difference of 3% was observed at Loko Dispensary. Maximum difference between HF summary and RI registration for measles was 614% at MCH Bunza and 43% minimum difference at Loko. In contrast to the administrative coverage, 60-80% of the children sampled from households were either not immunised or partially immunised. Further, the main sources of poor data quality include heavy workload on RI providers, over-reliance on administrative coverage report, and lack of understanding of the significance of high data quality by RI providers.

Conclusion

Substantial data discrepancies were observed in RI reports from all the Health Facilities which is indicative of poor data quality at the LGA level. Community surveys also revealed an over-reporting from administrative coverage data. Consequently, efforts should be geared towards achieving good data quality by immunisation stakeholders as it has implication on disease prevention and control efforts.

Comparison of administrative and survey data for estimating vitamin A supplementation and deworming coverage of children under five years of age in Sub-Saharan Africa

OBJECTIVE

To compare administrative coverage data with results from household coverage surveys for vitamin A supplementation (VAS) and deworming campaigns conducted during 2010-2015 in 12 African countries.

METHODS

Paired t-tests examined differences between administrative and survey coverage for 52 VAS and 34 deworming dyads. Independent t-tests measured VAS and deworming coverage differences between data sources for door-to-door and fixed-site delivery strategies and VAS coverage differences between 6- to 11-month and 12- to 59-month age group.

RESULTS

For VAS, administrative coverage was higher than survey estimates in 47 of 52 (90%) campaign rounds, with a mean difference of 16.1% (95% CI: 9.5-22.7; P < 0.001). For deworming, administrative coverage exceeded survey estimates in 31 of 34 (91%) comparisons, with a mean difference of 29.8% (95% CI: 16.9-42.6; P < 0.001). Mean ± SD differences in coverage between administrative and survey data were 12.2% ± 22.5% for the door-to-door delivery strategy and 25.9% ± 24.7% for the fixed-site model (P = 0.06). For deworming, mean ± SD differences in coverage between data sources were 28.1% ± 43.5% and 33.1% ± 17.9% for door-to-door and fixed-site distribution, respectively (P = 0.64). VAS administrative coverage was higher than survey estimates in 37 of 49 (76%) comparisons for the 6- to 11-month age group and 45 of 48 (94%) comparisons for the 12- to 59-month age group.

CONCLUSION

Reliance on health facility data alone for calculating VAS and deworming coverage may mask low coverage and prevent measures to improve programmes. Countries should periodically validate administrative coverage estimates with population-based methods.

Coverage and determinants of childhood immunization in Nigeria: A systematic review and meta-analysis

INTRODUCTION

The proportion of fully immunized children in Nigeria is reportedly low. There are concerns over national immunization data quality, with this possibly limiting country-wide response. We reviewed publicly available evidence on routine immunization across Nigeria to estimate national and zonal coverage of childhood immunization and associated determinants.

METHODS

A systematic search of Medline, EMBASE, Global Health and African Journals Online (AJOL) was conducted. We included population-based studies on childhood immunization in Nigeria. A random effects meta-analysis was conducted on extracted crude rates to arrive at national and zonal pooled estimates for the country.

RESULTS

Our search returned 646 hits. 21 studies covering 25 sites and 26,960 children were selected. The estimated proportion of fully immunized children in Nigeria was 34.4% (95% confidence interval [CI]: 27.0-41.9), with South-south zone having the highest at 51.5% (95% CI: 20.5-82.6), and North-west the lowest at 9.5% (95% CI: 4.6-14.4). Mother's social engagements (OR=4.0, 95% CI: 1.9-8.1) and vaccines unavailability (OR=3.9, 95% CI: 1.2-12.3) were mostly reported for low coverage. Other leading determinants were vaccine safety concerns (OR=3.0, 95% CI: 0.9-9.4), mother's low education (OR=2.5, 95% CI: 1.8-3.6) and poor information (OR=2.0, 95% CI: 0.8-4.7).

CONCLUSION

Our study suggests a low coverage of childhood immunization in Nigeria. Due to the paucity of data in the Northern states, we are still uncertain of the quality of evidence presented. It is hoped that this study will prompt the needed research, public health and policy changes toward increased evenly-spread coverage of childhood immunization in the country.

Routine Vaccination Coverage in Northern Nigeria: Results from 40 District-Level Cluster Surveys, 2014-2015

Background

Despite recent success towards controlling poliovirus transmission, Nigeria has struggled to achieve uniformly high routine vaccination coverage. A lack of reliable vaccination coverage data at the operational level makes it challenging to target program improvement. To reliably estimate vaccination coverage, we conducted district-level vaccine coverage surveys using a pre-existing infrastructure of polio technical staff in northern Nigeria.

Methods

Household-level cluster surveys were conducted in 40 polio high risk districts of Nigeria during 2014–2015. Global positioning system technology and intensive supervision by a pool of qualified technical staff were used to ensure high survey quality. Vaccination status of children aged 12–23 months was documented based on vaccination card or caretaker’s recall. District-level coverage estimates were calculated using survey methods.

Results

Data from 7,815 children across 40 districts were analyzed. District-level coverage with the third dose of diphtheria-pertussis-tetanus vaccine (DPT3) ranged widely from 1–63%, with all districts having DPT3 coverage below the target of 80%. Median coverage across all districts for each of eight vaccine doses (1 Bacille Calmette-Guérin dose, 3 DPT doses, 3 oral poliovirus vaccine doses, and 1 measles vaccine dose) was <50%. DPT3 coverage by survey was substantially lower (range: 28%–139%) than the 2013 administrative coverage reported among children aged <12 months. Common reported reasons for non-vaccination included lack of knowledge about vaccines and vaccination services (50%) and factors related to access to routine immunization services (15%).

Conclusions

Survey results highlighted vaccine coverage gaps that were systematically underestimated by administrative reporting across 40 polio high risk districts in northern Nigeria. Given the limitations of administrative coverage data, our approach to conducting quality district-level coverage surveys and providing data to assess and remediate issues contributing to poor vaccination coverage could serve as an example in countries with sub-optimal vaccination coverage, similar to Nigeria.

Sparking, supporting and steering change: grounding an accountability framework with viewpoints from Nigerian routine immunization and primary health care government officials

Existing accountability efforts in Nigeria primarily serve as retrospective policing. To enable accountability to guide change prospectively and preemptively, we drew from a literature review to develop a framework that highlights mutually reinforcing dimensions of accountability in health systems along three counterbalancing axes. The axis of power sparks change by wielding 'sticks' that curb the potential abuse of power, but also by offering 'carrots' that motivate constructive agency. The axis of ability supports change by enabling service delivery actors with formal rules that appropriately expand their authority to act, but also the informal norms and inputs for improved performance. Last, the axis of justice orients the strategic direction of change, balancing political representation, community ownership and social equity, so that accountability measures are progressive, rather than being captured by self-interests. We consulted Nigerian government officials to understand their viewpoints on accountability and mapped their responses to our evolving framework. All government officials (n = 36) participating in three zonal workshops on routine immunization filled out questionnaires that listed the top three opportunities and challenges to strengthening accountability. Thematically coded responses highlighted dimensions of accountability within the axes of ability and power: clarifying formal roles and responsibilities; transparency, data and monitoring systems; availability of skilled health personnel that are motivated and supervised; addressing informal norms and behaviours; and availability of inputs regarding funding and supplies. Other dimensions of accountability were mentioned but were not as critical from their viewpoints: managerial discretion; sanctions and enforcements; political influence and community engagement. Strikingly, almost no respondents mentioned social equity as being an important aspect of accountability, although a few mentioned broad development concerns that reflected community perspectives. Reframing accountability as a means of sparking, supporting and steering change can highlight different dimensions of health systems that need reform, particularly depending on the positionality of the viewpoints consulted.