Improving public health information: a data quality intervention in KwaZulu-Natal, South Africa

Systematizing Information Use to Address Determinants of Health Worker Health in South Africa: A Cross-sectional Mixed Method Study

Background

Recognizing that access to safe and healthy working conditions is a human right, the World Health Organization (WHO) calls for specific occupational safety and health (OSH) programs for health workers (HWs). The WHO health systems' building blocks, and the International Labour Organization (ILO), highlight the importance of information as part of effective systems. This study examined how OSH stakeholders access, use, and value an occupational health information system (OHIS).

Methods

A cross-sectional survey of OSH stakeholders was conducted as part of a larger quasi experimental study in four teaching hospitals. The study hospitals and participants were purposefully selected and data collected using a modified questionnaire with both closed and open-ended questions. Quantitative analysis was conducted and themes identified for qualitative analysis. Ethics approval was provided by the University of Pretoria and University of British Columbia.

Results

There were 71 participants comprised of hospital managers, health and safety representatives, trade unions representatives and OSH professionals. At least 42% reported poor accessibility and poor timeliness of OHIS for decision-making. Only 50% had access to computers and 27% reported poor computer skills. When existing, OHIS was poorly organized and needed upgrades, with 85% reporting the need for significant reforms. Only 45% reported use of OHIS for decision-making in their OSH role.

Conclusion

Given the gap in access and utilization of information needed to protect worker's rights to a safe and healthy workplace, more attention is warranted to OHIS development and use as well as education and training in South Africa and beyond.

Lessons learned from over a decade of data audits in international observational HIV cohorts in Latin America and East Africa

Introduction

Routine patient care data are increasingly used for biomedical research, but such "secondary use" data have known limitations, including their quality. When leveraging routine care data for observational research, developing audit protocols that can maximize informational return and minimize costs is paramount.

Methods

For more than a decade, the Latin America and East Africa regions of the International epidemiology Databases to Evaluate AIDS (IeDEA) consortium have been auditing the observational data drawn from participating human immunodeficiency virus clinics. Since our earliest audits, where external auditors used paper forms to record audit findings from paper medical records, we have streamlined our protocols to obtain more efficient and informative audits that keep up with advancing technology while reducing travel obligations and associated costs.

Results

We present five key lessons learned from conducting data audits of secondary-use data from resource-limited settings for more than 10 years and share eight recommendations for other consortia looking to implement data quality initiatives.

Conclusion

After completing multiple audit cycles in both the Latin America and East Africa regions of the IeDEA consortium, we have established a rich reference for data quality in our cohorts, as well as large, audited analytical datasets that can be used to answer important clinical questions with confidence. By sharing our audit processes and how they have been adapted over time, we hope that others can develop protocols informed by our lessons learned from more than a decade of experience in these large, diverse cohorts.

Development of novel composite data quality scores to evaluate facility-level data quality in electronic data in Kenya: a nationwide retrospective cohort study

BACKGROUND

In this evaluation, we aim to strengthen Routine Health Information Systems (RHIS) through the digitization of data quality assessment (DQA) processes. We leverage electronic data from the Kenya Health Information System (KHIS) which is based on the District Health Information System version 2 (DHIS2) to perform DQAs at scale. We provide a systematic guide to developing composite data quality scores and use these scores to assess data quality in Kenya.

METHODS

We evaluated 187 HIV care facilities with electronic medical records across Kenya. Using quarterly, longitudinal KHIS data from January 2011 to June 2018 (total N = 30 quarters), we extracted indicators encompassing general HIV services including services to prevent mother-to-child transmission (PMTCT). We assessed the accuracy (the extent to which data were correct and free of error) of these data using three data-driven composite scores: 1) completeness score; 2) consistency score; and 3) discrepancy score. Completeness refers to the presence of the appropriate amount of data. Consistency refers to uniformity of data across multiple indicators. Discrepancy (measured on a Z-scale) refers to the degree of alignment (or lack thereof) of data with rules that defined the possible valid values for the data.

RESULTS

A total of 5,610 unique facility-quarters were extracted from KHIS. The mean completeness score was 61.1% [standard deviation (SD) = 27%]. The mean consistency score was 80% (SD = 16.4%). The mean discrepancy score was 0.07 (SD = 0.22). A strong and positive correlation was identified between the consistency score and discrepancy score (correlation coefficient = 0.77), whereas the correlation of either score with the completeness score was low with a correlation coefficient of -0.12 (with consistency score) and -0.36 (with discrepancy score). General HIV indicators were more complete, but less consistent, and less plausible than PMTCT indicators.

CONCLUSION

We observed a lack of correlation between the completeness score and the other two scores. As such, for a holistic DQA, completeness assessment should be paired with the measurement of either consistency or discrepancy to reflect distinct dimensions of data quality. Given the complexity of the discrepancy score, we recommend the simpler consistency score, since they were highly correlated. Routine use of composite scores on KHIS data could enhance efficiencies in DQA at scale as digitization of health information expands and could be applied to other health sectors beyondHIV clinics.

Evaluation of the rubella surveillance system in South Africa, 2016-2018: A cross-sectional study

BACKGROUND

Rubella is a leading vaccine-preventable cause of birth defects. We conducted this study to evaluate the rubella surveillance system in South Africa from 2016 to 2018. The rubella surveillance system had not been evaluated since its inception; therefore, a formal evaluation is necessary to assess key attributes and to ascertain the extent to which the system achieves its objectives.

METHODS

We conducted a cross-sectional study to assess the usefulness, simplicity, positive predictive value, timeliness, and data quality of the rubella surveillance system from 2016 to 2018. We reviewed retrospective rubella surveillance data and conducted a survey with key stakeholders of the system. We compiled a summary report from the survey and calculated the annualized detection rate of rubella and non-rubella febrile rash, positive predictive value, the proportion of complete records, and timeliness between the surveillance steps. We compared our results with recommended performance indicators from the 2015 revised World Health Organization African regional guidelines for measles and rubella surveillance.

RESULTS

The rubella surveillance system was useful but weak in terms of simplicity. The annualized detection rate of rubella febrile rash was 1.5 per 100,000 populations in 2016, 4.4 in 2017, and 2.1 in 2018. The positive predictive value was 29.1% in 2016, 40.9% in 2017, and 32.9% in 2018. The system did not meet the timeliness goal in the health facility component but met this goal in the laboratory component. The system had poor data quality, particularly in the health facility component.

CONCLUSIONS

The rubella surveillance system was useful, although it was not simple to use and had low PPV, poor timeliness, and poor data quality. Efforts should be made to improve the system's simplicity, PPV, timeliness, and data quality at the facility level.

Improving birth weight measurement and recording practices in Kenya and Tanzania: a prospective intervention study with historical controls

BACKGROUND

Low birth weight (LBW) is a significant public health concern given its association with early-life mortality and other adverse health consequences that can impact the entire life cycle. In many countries, accurate estimates of LBW prevalence are lacking due to inaccuracies in collection and gaps in available data. Our study aimed to determine LBW prevalence among facility-born infants in selected areas of Kenya and Tanzania and to assess whether the introduction of an intervention to improve the accuracy of birth weight measurement would result in a meaningfully different estimate of LBW prevalence than current practice.

METHODS

We carried out a historically controlled intervention study in 22 health facilities in Kenya and three health facilities in Tanzania. The intervention included: provision of high-quality digital scales, training of nursing staff on accurate birth weight measurement, recording and scale calibration practices, and quality maintenance support that consisted of enhanced supervision and feedback (prospective arm). The historically controlled data were birth weights from the same facilities recorded in maternity registers for the same calendar months from the previous year measured using routine practices and manual scales. We calculated mean birth weight (95% confidence interval CI), mean difference in LBW prevalence, and respective risk ratio (95% CI) between study arms.

RESULTS

Between October 2019 and February 2020, we prospectively collected birth weights from 8441 newborns in Kenya and 4294 in Tanzania. Historical data were available from 9318 newborns in Kenya and 12,007 in Tanzania. In the prospective sample, the prevalence of LBW was 12.6% (95% confidence intervals [CI]: 10.9%-14.4%) in Kenya and 18.2% (12.2%-24.2%) in Tanzania. In the historical sample, the corresponding prevalence estimates were 7.8% (6.5%-9.2%) and 10.0% (8.6%-11.4%). Compared to the retrospective sample, the LBW prevalence in the prospective sample was 4.8% points (3.2%-6.4%) higher in Kenya and 8.2% points (2.3%-14.0%) higher in Tanzania, corresponding to a risk ratio of 1.61 (1.38-1.88) in Kenya and 1.81 (1.30-2.52) in Tanzania.

CONCLUSION

Routine birth weight records underestimate the risk of LBW among facility-born infants in Kenya and Tanzania. The quality of birth weight data can be improved by a simple intervention consisting of provision of digital scales and supportive training.

Prevalence of locoregional recurrence and survival post-treatment of head and neck cancers in Africa: a systematic review and meta-analysis

Background

Recurrent cancers of the head and neck are associated with poor survival outcome. Yet, their burden in Africa is not reliably known. We therefore aimed to estimate the prevalence of recurrence and the 5-year overall survival among patients treated for head and neck cancers (HNC) in Africa.

Method

In this systematic review and meta-analysis, we searched four electronic databases (Pubmed, CINAHL, MEDLINE, and Web of Science) and the grey literature for studies reporting the prevalence of HNC recurrence and 5-year overall survival post treatment, published between January 1, 2002, and December 31, 2022. We contacted corresponding authors of relevant studies. Searches were extended to reference lists of review articles and other relevant sources for potentially eligible studies. Each record was assessed for inclusion or exclusion by two independent reviewers. Records with individual-level data on recurrence and survival conducted in Africa were included while exclusion was based on the study design and availability of relevant data. Data were independently extracted by three reviewers from eligible studies, and summary estimates were sought. Our primary outcomes were recurrence and 5-year overall survival of patients who have been treated for HNC, and our secondary outcomes included risk factors, tumor site, squamous cell histology, clinical stage of tumor, and treatment options received. Only records selected for primary outcomes were assessed for secondary outcome data extraction. Random-effects meta-analysis was conducted for each outcome. Meta-regression models were used in addressing sample heterogeneity among the studies. Protocol for this study was registered with PROSPERO, CRD42022372307.

Findings

This systematic review and meta-analysis returned 3998 records, yielding 28 included studies after exclusion. Eighteen studies reported on the prevalence of HNC recurrence while 24 articles reported on the 5-year overall survival. Of the pooled total study population, 7199 (70.5%) of 10,218 patients were males while 2603 (25.5%) were females. We found that the prevalence of HNC recurrence was 15.4% (I² = 96.2%; 95% CI: 9.5-22.3; n = 3214; k = 18), and the 5-year overall survival was 54.4% (I² = 99.5%; 95% CI: 40.1-68.4; n = 9798; k = 24). We also found that the prevalence of smoking and alcohol consumption as risk factors for HNC were 42.6% (I² = 98.8%; 95% CI: 25.2-61.0; n = 4374; k = 15) and 35.8% (I² = 98.9%; 95% CI: 21.7-51.4; n = 4110; k = 11) respectively. The pooled current prevalence for advanced HNC (clinical stages III-IV) was 80.0% (I² = 99.2%; 95% CI: 68.6-89.5; n = 7624; k = 18) compared to 12.2% (I² = 96.4%; 95% CI: 6.2-19.8; n = 7624; k = 18) in early disease (clinical stages I-II).

Interpretation

The results showed significantly high prevalence of cancer recurrence, poor 5-year overall survival and very high prevalence of advanced cancers at time of diagnosis. This study provides robust evidence for strategies towards prompt diagnosis and appropriate management of HNC to improve patients' outcome in the African continent.

Funding

This study was not supported by any funding.

Assessing the effect of electronic health information exchange on the completeness and validity of data for measuring viral load testing turnaround time in Nigeria

INTRODUCTION

Implementation of health information exchange has been shown to result in several benefits which includes the improvement in the completeness and timeliness of data for public health program monitoring and surveillance.

OBJECTIVE

The objective of this study was to assess the effect of implementing an electronic health information exchange (HIE) on the quality of data available to measure HIV viral load testing turnaround time (TAT) in Nigeria.

METHODS

We measured viral load data validity and completeness before the implementation of electronic health information exchange, and 6 months after implementation. Records of specimens collected at 30 healthcare facilities and tested in 3 Polymerase Chain Reaction (PCR) labs were analyzed. We define data completeness as the percentage of non-missing values and measured this value by specimens and by data elements in the dataset for calculating TAT. To examine data validity, we classified TAT segments with negative values and date fields that were not in International Organization for Standardization(ISO) standard date format as invalid. Validity was measured by specimens and by each TAT segment. Pearson's chi square was used to assess for improvements in validity and completeness post implementation of HIE.

RESULTS

15,226 records of specimens were analyzed at baseline and 18,022 records of specimens analyzed at endline. Data completeness for all specimens recorded increased significantly from 47% before HIE implementation to 67% six months after implementation (p < 0.01). Data validity also increased from 90% before implementation to 91% after implementation (p < 0.01) CONCLUSION: Our study demonstrated evidence of significant improvement in the quality of data available to measure viral load turnaround time with the implementation of HIE.

Rapid-cycle evaluation and learning for the effective delivery of integrated interventions in early childhood in rural India

Background

Effective and real-time data analytics plays an essential role in understanding gaps and improving the quality and coverage of complex public health interventions. Studies of public health information systems identify problems with data quality, such as incomplete records and untimely reporting. Effective data collection and real-time analysis systems for rapid-cycle learning are necessary to monitor public health programs and take timely evidence-based decisions. Early childhood development (ECD) programs are very diverse. Rapid-cycle evaluation and learning (REAL) guides the implementation process of such complex interventions in real time. Stepping stones was one such early childhood development program implemented in Central India.

Objective

The objective was to improve the delivery of complex, integrated public health interventions for early childhood development in remote areas of rural India.

Methodology

The program was developed according to the principles of inclusion and community-centeredness, which can be tested quickly and iteratively. To enhance the decision-making process and improve delivery and coverage, the core team implemented an information system for rapid-cycle learning. We developed performance indicators and a performance measurement matrix after defining the specific needs. Following that, we trained staff to collect complete data using electronic data collection tools and transfer it the same day to the server for quality review and further analysis. A variety of data/information was triangulated to address the gaps in intervention delivery, and those decisions were subsequently implemented.

Results

We observed that the quality of data collection improved, and errors were reduced by 50% in the third quarter. The quality of the narrative was also enhanced; it became more elaborate and reflective. Sharing their field output in meetings and improving the quality of the narrative enhanced the self-reflection skills of field staff and consequently improved the quality of the intervention delivery. Refresher training and mentoring by supervisors helped to improve the data quality over time.

Conclusion

Rapid-cycle evaluation and learning (REAL) can be implemented in resource-limited settings to improve the quality and coverage of integrated intervention in early childhood. It nurtures a reinforcing ecosystem that integrates providers, community, and family perspectives and guides interactions among stakeholders by integrating data from all available sources.

Data quality assessment in emergency medical services: an objective approach

BACKGROUND

In emergency medical services, high quality data are of great importance for patient care. Due to the unique nature of this type of services, the purpose of this study was to assess data quality in emergency medical services using an objective approach.

METHODS

This was a retrospective quantitative study conducted in 2019. The research sample included the emergency medical records of patients who referred to three emergency departments by the pre-hospital emergency care services (n = 384). Initially a checklist was designed based on the data elements of the triage form, pre-hospital emergency care form, and emergency medical records. Then, data completeness, accuracy and timeliness were assessed.

RESULTS

Data completeness in the triage form, pre-hospital emergency care form, and emergency medical records was 52.3%, 70% and 57.3%, respectively. Regarding data accuracy, most of the data elements were consistent. Measuring data timeliness showed that in some cases, paper-based ordering and computer-based data entry was not sequential.

CONCLUSION

Data quality in emergency medical services was not satisfactory and there were some weaknesses in the documentation processes. The results of this study can inform the clinical and administrative staff to pay more attentions to these weaknesses and plan for data quality improvement.

Evaluating the effectiveness of a mobile application to improve the quality, collection, and usability of forensic documentation of sexual violence

BACKGROUND

Survivors of sexual violence deserve timely and high-quality forensic examination, evidence collection, and documentation as part of comprehensive care. However, in many countries, the quality of medical-legal documentation is severely limited. MediCapt is an innovative digital application that enables clinicians to document forensic medical evidence as well as capture and securely store forensic photographs of injuries. This study evaluated the effectiveness and usability of MediCapt to document forensic medical evidence of sexual violence.

METHODS

This mixed-methods evaluation involved key-informant interviews, usability questionnaires, and forensic record reviews. Participants included clinicians, medical records personnel, information technology personnel, and health facility administrators, as well as law enforcement and legal professionals in Kenya.

RESULTS

The Physicians for Human Rights (PHR) data quality checklist found that using MediCapt led to significantly higher data-quality scores compared to paper-based forms. MediCapt forms scored higher on 23 of 26 checklist items. While a wide difference in quality was seen among paper-based forms, MediCapt appeared to both standardize and improve quality of documentation across sites. MediCapt strengths included data security and confidentiality, accuracy and efficiency, and supplemental documentation with photography. Weaknesses included infrastructure challenges, required technological proficiencies, and time to learn the new system. Although it is early to assess the impact of MediCapt on prosecutions, providers and law and justice sector professionals were optimistic about its usefulness. They identified MediCapt as appropriate for use with survivors of sexual violence and reported MediCapt's legibility and photography features had already been commended by the court.

CONCLUSION

MediCapt was well-received across all sectors, its use was perceived as feasible and sustainable, and it significantly improved the quality of collected forensic data. It is anticipated this improvement in forensic documentation will increase successful prosecutions, strengthen accountability for perpetrators, and improve justice for survivors.

Development and Testing Requirements for an Integrated Maternal and Child Health Information System in Iran: A Design Thinking Case Study

BACKGROUND

 Management of child health care can be negatively affected by incomplete recording, low data quality, and lack of data integration of health management information systems to support decision making and public health program needs. Given the importance of identifying key determinants of child health via capturing and integrating accurate and high-quality information, we aim to address this gap through the development and testing requirements for an integrated child health information system.

SUBJECTS AND METHODS

 A five-phase design thinking approach including empathizing, defining, ideation, prototyping, and testing was applied. We employed observations and interviews with the health workers at the primary health care network to identify end-users' challenges and needs using tools in human-centered design and focus group discussion. Then, a potential solution to the identified problems was developed as an integrated maternal and child health information system (IMCHIS) prototype and tested using Software Quality Requirements and Evaluation Model (SQuaRE) ISO/IEC 25000.

RESULTS

 IMCHIS was developed as a web-based system with 74 data elements and seven maternal and child health care requirements. The requirements of "child disease" with weight (0.26), "child nutrition" with weight (0.20), and "prenatal care" with weight (0.16) acquired the maximum weight coefficient. In the testing phase, the highest score with the weight coefficient of 0.48 and 0.73 was attributed to efficiency and functionality characteristics, focusing on software capability to fulfill the tasks that meet users' needs.

CONCLUSION

 Implementing a successful child health care system integrates both maternal and child health care information systems to track the effect of maternal conditions on child health and support managing performance and optimizing service delivery. The highest quality score of IMCHIS in efficiency and functionality characteristics confirms that it owns the capability to identify key determinants of child health.

Investigating the quality of HIV rapid testing practices in public antenatal health care facilities, South Africa

Monitoring HIV prevalence using antenatal HIV sentinel surveillance is important for efficient epidemic tracking, programme planning and resource allocation. HIV sentinel surveillance usually employs unlinked anonymous HIV testing which raises ethical, epidemiological and public health challenges in the current era of universal test and treat. The World Health Organization (WHO) recommends that countries should consider using routine prevention of mother-to-child transmission of HIV (PMTCT) data for surveillance. We audited antenatal care clinics to assess the quality of HIV rapid testing practices as the first step to assess whether South Africa is ready to utilize PMTCT programme data for antenatal HIV surveillance. In 2017, we conducted a cross-sectional survey in 360 randomly sampled antenatal care clinics using the adapted WHO Stepwise-Process-for-Improving-the-Quality-of-HIV-Rapid-Testing (SPI-RT) checklist. We calculated median percentage scores within a domain (domain-specific median score), and across all domains (overall median percentage scores). The latter was used to classify sites according to five implementation levels; (from 0:<40% to 4: 90% or higher). Of 346 (96.1%) facilities assessed, an overall median percentage score of 62.1% (inter-quartile range (IQR): 50.8–71.9%) was obtained. The lowest domain-specific median percentage scores were obtained under training/certification (35% IQR: 10.0–50.0%) and external quality assurance (12.5% IQR: 0.0–50.0%), respectively. The majority (89%) of sites had an overall median score at level 2 or below; of these, 37% required improvement in specific areas and 6.4% in all areas. Facilities in districts implementing the HIV Rapid Test Quality Improvement Initiative and supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) had significantly higher median overall scores (65.6% IQR: 53.9–74.2%) (P-value from rank sum test: <0.001) compared with non–PEPFAR–supported facilities (56.6% IQR:47.7–66.0%). We found sub-optimal implementation of HIV rapid testing practices. We recommend the expansion of the PEPFAR-funded Rapid Test Continuous Quality Improvement (RTCQI) support to all antenatal care testing sites.

District-Level Health Management and Health System Performance: The Ethiopia Primary Healthcare Transformation Initiative

BACKGROUND

Despite a wide range of interventions to improve district health management capacity in low-income settings, evidence of the impact of these investments on system-wide management capacity and primary healthcare systems performance is limited. To address this gap, we conducted a longitudinal study of the 36 rural districts (woredas), including 229 health centers, participating in the Primary Healthcare Transformation Initiative (PTI) in Ethiopia.

METHODS

Between 2015 and 2017, we collected quantitative measures of management capacity at the district and health center levels and a primary healthcare key performance indicator (KPI) summary score based on antenatal care (ANC) coverage, contraception use, skilled birth attendance, infant immunization, and availability of essential medications. We conducted repeated measures analysis of variance (ANOVA) to assess (1) changes in management capacities at the district health office level and health center level, (2) changes in health systems performance, and (3) the differential effects of more vs less intensive intervention models.

RESULTS

Adherence to management standards at both district and health center levels improved during the intervention, and the most prominent improvement was achieved during district managers' exposure to intensive mentorship and education. We did not observe similar patterns of change in KPI summary score.

CONCLUSION

The district health office is a valuable entry point for primary healthcare reform, and district- and facility-level management capacity can be measured and improved in a relatively short period of time. A combination of intensive mentorship and structured team-based education can serve as both an accelerator for change and a mechanism to inform broader reform efforts.

Did an Intervention Programme Aimed at Strengthening the Maternal and Child Health Services in Nigeria Improve the Completeness of Routine Health Data Within the Health Management Information System?

BACKGROUND

During 2012-2015, the Federal Government of Nigeria launched the Subsidy Reinvestment and Empowerment Programme, a health system strengthening (HSS) programme with a Maternal and Child Health component (Subsidy Reinvestment and Empowerment Programme [SURE-P]/MCH), which was monitored using the Health Management Information Systems (HMIS) data reporting tools. Good quality data is essential for health policy and planning decisions yet, little is known on whether and how broad health systems strengthening programmes affect quality of data. This paper explores the effects of the SURE-P/MCH on completeness of MCH data in the National HMIS.

METHODS

This mixed-methods study was undertaken in Anambra state, southeast Nigeria. A standardized proforma was used to collect facility-level data from the facility registers on MCH services to assess the completeness of data from 2 interventions and one control clusters. The facility data was collected to cover before, during, and after the SURE-P intervention activities. Qualitative in-depth interviews were conducted with purposefully-identified health facility workers to identify their views and experiences of changes in data quality throughout the above 3 periods.

RESULTS

Quantitative analysis of the facility data showed that data completeness improved substantially, starting before SURE-P and continuing during SURE-P but across all clusters (ie, including the control). Also health workers felt data completeness were improved during the SURE-P, but declined with the cessation of the programme. We also found that challenges to data completeness are dependent on many variables including a high burden on providers for data collection, many variables to be filled in the data collection tools, and lack of health worker incentives.

CONCLUSION

Quantitative analysis showed improved data completeness and health workers believed the SURE-P/MCH had contributed to the improvement. The functioning of national HMIS are inevitably linked with other health systems components. While health systems strengthening programmes have a great potential for improved overall systems performance, a more granular understanding of their implications on the specific components such as the resultant quality of HMIS data, is needed.

Behavioral Determinants of Routine Health Information System Data Use in Senegal: A Qualitative Inquiry Based on the Integrated Behavioral Model

Although behavioral factors are thought to be important barriers to routine data use, they remain understudied particularly in low-income country settings. We show that the integrated behavior model can be a valuable theoretical framework for targeted communication strategies and capacity-building interventions aimed at promoting a culture of data use.

Routine health information system (RHIS) data are essential in driving decision making and planning in health systems as well as health programs. However, despite their importance, these data are underutilized, and the underlying individual-level facilitators and barriers to use remain understudied. In this research, we applied the Integrated Behavior Model (IBM) to examine how attitudes toward RHIS data, perceived norms concerning RHIS data use, and the ability to use RHIS data influence the demand and use of RHIS data among stakeholders in Senegal. Using data from interviews with respondents working at national levels of malaria, HIV, and TB control programs in Senegal, we used a framework analysis approach to apply the IBM behavioral constructs and identify their linkages to RHIS data use. We found that attitudes about the quality, availability, and relevance of RHIS data for decision making were important in driving data use among respondents. Institutional expectations, organizational protocols, policies, and practices around RHIS data ultimately shape social norms around the use of the data. Although we found that perceived ability and self-efficacy to use RHIS data were not barriers to RHIS data use among stakeholders at the strategic levels of their respective organizations, these were reported to be barriers at lower levels of the health system. Low perceived control of the RHIS data production process ultimately reduced RHIS data use for decision making among the strategic-level respondents. We recommend context-specific reexamination of existing RHIS interventions with a renewed emphasis on behavioral aspects of data use. The IBM can help guide practitioners, policy makers, and academics to address multiple socioecological factors that influence data use behavior when recommending RHIS and data use solutions.

Quality of routine health facility data used for newborn indicators in low- and middle-income countries: A systematic review

Background

High-quality data are fundamental for effective monitoring of newborn morbidity and mortality, particularly in high burden low- and middle-income countries (LMIC).

Methods

We conducted a systematic review on the quality of routine health facility data used for newborn indicators in LMIC, including measures employed. Five databases were searched from inception to February 2021 for relevant observational studies (excluding case-control studies, case series, and case reports) and baseline or control group data from interventional studies, with no language limits. An adapted version (19-point scale) of the Critical Appraisal Tool to assess the Quality of Cross-Sectional Studies (AXIS) was used to assess methodological quality, and results were synthesized using descriptive analysis.

Results

From the 19 572 records retrieved, 34 studies in 16 LMIC countries were included. Methodological quality was high (>14/19) in 32 studies and moderate (10-14/19) in two. Studies were mostly from African (n = 30, 88.2%) and South-East Asian (n = 24, 70.6%) World Health Organization (WHO) regions, with very few from Eastern Mediterranean (n = 2, 5.9%) and Western Pacific (n = 1, 2.9%) ones. We found that only data elements used to calculate neonatal indicators had been assessed, not the indicators themselves. 41 data elements were assessed, most frequently birth outcome. 20 measures of data quality were used, most along three dimensions: 1) completeness and timeliness, 2) internal consistency, and 3) external consistency. Data completeness was very heterogeneous across 26 studies, ranging from 0%-100% in routine facility registers, 0%-100% in patient case notes, and 20%-68% in aggregate reports. One study reported on the timeliness of aggregate reports. Internal consistency ranged from 0% to 96.2% in four studies. External consistency (21 studies) varied widely in measurement and findings, with specificity (6.4%-100%), sensitivity (23.6%-97.6%), and percent agreement (24.6%-99.4%) most frequently reported.

Conclusions

This systematic review highlights a gap in the published literature on the quality of routine LMIC health facility data for newborn indicators. Robust evidence is crucial in driving data quality initiatives at national and international levels. The findings of this review indicate that good quality data collection is achievable even in high-burden LMIC settings, but more efforts are needed to ensure uniformly high data quality for neonatal indicators.

Converting the existing disease surveillance from a paper-based to an electronic-based system using district health information system (DHIS-2) for real-time information: the Lebanese experience

INTRODUCTION

The Ministry of Public Health in Lebanon is in the process of converting the surveillance reporting from a cumbersome paper-based system to a web-based electronic platform (DHIS-2) to have real-time information for early detection of alerts and outbreaks and for initiating a prompt response.

OBJECTIVES

This paper aimed to document the Lebanese experience in implementing DHIS-2 for the disease surveillance system. It also targets to assess the improvement of reporting rates and timeliness of the reported data and to disclose the encountered challenges and opportunities.

METHODOLOGY

This is a retrospective description of processes involved in the implementation of the DHIS-2 tool in Lebanon. Initially, it was piloted for the school-based surveillance in 2014; then its use was extended in May 2017 to cover other specific surveillance systems. This included all surveillance programs collecting aggregate data from hospitals, medical centers, dispensaries, or laboratories at the first stage. As part of the national roll-out process, the online application was developed. The customized aggregated-based datasets, organization units, user accounts, specific and generic dashboards were generated. More than 80 training sessions were conducted throughout the country targeting 1290 end-users including health officers at the national and provincial levels, focal persons who were working in all public and private hospitals, laboratories, and medical centers as well. Completeness and timeliness of reported data were compared before and after the implementation of DHIS-2. The unveiled challenges and the main lessons learned during the roll-out process were discussed.

RESULTS

For laboratory-based surveillance, completeness of reporting increased from 70.8% in May to 89.6% in October. Timeliness has improved from 25 to 74%. For medical centers, an improvement of 8.1% for completeness and 9.4% in timeliness was recorded before and after training sessions. For zero reporting, completeness remains the same (88%) and timeliness has improved from 74 to 87%. The main challenges faced during the implementation of DHIS-2 were mainly infrastructural and system-related in addition to poor internet connectivity, limited workforce, and frequent changes to DHIS-2 versions.

CONCLUSION

Implementation of DHIS-2 improved timeliness and completeness for aggregated data reporting. Continued on-site support, monitoring, and system enhancement are needed to improve the performance of DHIS-2.

Contribution of health information system to child immunization services in Ethiopia: baseline study of 33 woredas

Background

Monitoring progress using appropriate data, with a functional health information system (HIS), believed to be very crucial for success of immunization program. Baseline study was conducted to assess, immunization service coverage, HIS performance status, and their relationships.

Methods

A linked facility and population-based survey was conducted concurrently from September 21 to October 15, 2020. A total of 3016 households were reached to interview mothers having children aged 12–23 months in the 33 woredas. Overall 81 health posts, 71 health centers, and 15 hospitals were selected for the facility survey. The study used modified Performance of Routine Information System Management (PRISM) tools for the facility survey and a structured questionnaire for the household survey. Using STATA 14.0 software, mixed effect modeling was employed to control the effect of clustering and potential confounders.

Results

The proportion of fully immunized children was 58%. Coverages of measles (at least one dose) and penta3 immunization (received all 3 doses of DPT-HepB-Hib vaccine) were 86%, and 85% respectively. About 27% of mothers had missed their child immunization card mainly due to misplacing or lost. Except ‘source document completeness’ (85%) and ‘use of data for planning and target setting’ (84%), other data quality and use indicators like ‘data accuracy’ (63%), ‘data use for performance review and evidence-based decision making’ (50%), and ‘data use to produce analytical reports’ (31%) show low performance. The odds of fully immunized children is 37% lower in Muslims compared with Orthodox Christians (AOR, 0.63; 95%CI: 0.46, 0.88), higher by 42% with father’s secondary education compared with no education (AOR, 1.42; 95%CI: 1.05, 1.92), and highest wealth quintile compared with lowest quintile (AOR, 2.49; 95%CI: 1.54, 4.03). For each additional score of HIS infrastructure availability, the odds of fully immunized children increased by 22% (AOR:1.22; 95% CI: 1.03, 1.44).

Conclusions

Child immunization coverages are promising However, the current HIS performance is suboptimal. Both service user and HIS related factors are important for immunization service uptake. Documenting required information and advising mothers to keep immunization cards by health workers, and working to have functional HIS are recommended.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01796-8.

Direct financial assistance for improved maternal and child health data: a pilot study supporting the health management information system in Malawi

Background

The health management information system (HMIS) is an integral component of a strong health care system. Despite its importance for decision-making, the quality of HMIS data remains of concern in low- and middle-income countries. To address challenges with the quality of maternal and child health (MCH) data gathered within Malawi’s HMIS, we conducted a pilot study evaluating different support modalities to district-level HMIS offices. We hypothesized that providing regular, direct financial assistance to HMIS offices would enable staff to establish strategies and priorities based on local context, resulting in more accurate, timely, and complete MCH data.

Methods

The pilot intervention was implemented in Mwanza district, while Chikwawa, Neno, and Ntchisi districts served as control sites given support received from other institutions. The intervention consisted of providing direct financial assistance to Mwanza’s HMIS office following the submission of detailed budgets and lists of planned activities. In the control districts, we performed interviews with the HMIS officers to track the HMIS-related activities. We evaluated the intervention by comparing data quality between the post- and pre-intervention periods in the intervention and control districts. Additionally, we conducted interviews with Mwanza’s HMIS office staff to determine the acceptability and appropriateness of the intervention.

Results

Following the 10-month intervention period, we observed improvements in MCH data quality in Mwanza. The availability and completeness of MCH data collected in the registers increased by 22 and 18 percentage points, respectively. The consistency of MCH data between summary reports and electronic HMIS also improved. In contrast, 2/3 control districts noted minimal changes or reductions in data quality after 10 months. The qualitative interviews confirmed that, despite some challenges, the intervention was well received by the participating HMIS office. HMIS staff preferred our strategy to other conventional strategies that fail to give them the independence to make decisions.

Conclusions

This pilot intervention demonstrated an alternative approach to support HMIS offices in their daily efforts to improve data quality. Given the Ministry of Health’s (MoH) interest in strengthening its HMIS, our intervention provides a strategy that the MoH and local and international partners could consider to rapidly improve HMIS data with minimal oversight.

At the frontlines of digitisation: a qualitative study on the challenges and opportunities in maintaining accurate, complete and timely digital health records in India's government health system

OBJECTIVES

To understand factors underpinning the accuracy and timeliness of mobile phone numbers and other health information captured in India's government registry for pregnant and postpartum women. Accurate and timely registration of mobile phone numbers is necessary for beneficiaries to receive mobile health services.

SETTING

Madhya Pradesh and Rajasthan states in India at the community, clinical, and administrative levels of the health system.

PARTICIPANTS

Interviews (n=59) with frontline health workers (FLHWs), data entry operators, and higher level officials. Focus group discussions (n=12) with pregnant women to discuss experiences with sharing data in the health system. Observations (n=9) of the process of digitization and of interactions between stakeholders for data collection.

PRIMARY AND SECONDARY OUTCOME MEASURES

Thematic analysis identified how key actors experienced the data collection and digitisation process, reasons for late or inaccurate data, and mechanisms that can bolster timeliness and accuracy.

RESULTS

Pregnant women were comfortable sharing mobile numbers with health workers, but many were unaware that their data moved beyond their FLHW. FLHWs valued knowing up-to-date beneficiary mobile numbers, but felt little incentive to ensure accuracy in the digital record system. Delays in registering pregnant women in the online portal were attributed to slow movement of paper records into the digital system and difficulties in gathering required documents from beneficiaries. Data, including women's phone numbers, were handwritten and copied multiple times by beneficiaries and health workers with variable literacy. Supervision tended to focus on completeness rather than accuracy. Health system actors noted challenges with the digital system but valued the broader project of digitisation.

CONCLUSIONS

Increased focus on training, supportive supervision, and user-friendly data processes that prioritise accuracy and timeliness should be considered. These inputs can build on existing positive patient-provider relationships and health system actors' enthusiasm for digitisation.

Development of a Quality Assurance Score for the Nigeria AIDS Indicator and Impact Survey (NAIIS) Database: Validation Study

BACKGROUND

In 2018, Nigeria implemented the world's largest HIV survey, the Nigeria AIDS Indicator and Impact Survey (NAIIS), with the overarching goal of obtaining more reliable metrics regarding the national scope of HIV epidemic control in Nigeria.

OBJECTIVE

This study aimed to (1) describe the processes involved in the development of a new database evaluation tool (Database Quality Assurance Score [dQAS]) and (2) assess the application of the dQAS in the evaluation and validation of the NAIIS database.

METHODS

The dQAS tool was created using an online, electronic Delphi (e-Delphi) methodology with the assistance of expert review panelists. Thematic categories were developed to form superordinate categories that grouped themes together. Subordinate categories were then created that decomposed themes for more specificity. A validation score using dQAS was employed to assess the technical performance of the NAIIS database.

RESULTS

The finalized dQAS tool was composed of 34 items, with a total score of 81. The tool had 2 sections: validation item section, which contains 5 subsections, and quality assessment score section, with a score of "1" for "Yes" to indicate that the performance measure item was present and "0" for "No" to indicate that the measure was absent. There were also additional scaling scores ranging from "0" to a maximum of "4" depending on the measure. The NAIIS database achieved 78 out of the maximum total score of 81, yielding an overall technical performance score of 96.3%, which placed it in the highest category denoted as "Exceptional."

CONCLUSIONS

This study showed the feasibility of remote internet-based collaboration for the development of dQAS-a tool to assess the validity of a locally created database infrastructure for a resource-limited setting. Using dQAS, the NAIIS database was found to be valid, reliable, and a valuable source of data for future population-based, HIV-related studies.

Assessing routine health information system performance during the tenth outbreak of Ebola virus disease (2018-2020) in the Democratic Republic of the Congo: A qualitative study in North Kivu

The Democratic Republic of Congo has implemented reforms to its national routine health information system (RHIS) to improve timeliness, completeness, and use of quality data. However, outbreaks can undermine efforts to strengthen it. We assessed the functioning of the RHIS during the 2018-2020 outbreak of Ebola Virus Disease (EVD) to identify opportunities for future development. We conducted a qualitative study in North Kivu, from March to May 2020. Semi-structured interviews were conducted with 34 key informants purposively selected from among the personnel involved in the production of RHIS data. The topics discussed included RHIS functioning, tools, compilation, validation, quality, sharing, and the use of data. Audio recordings were transcribed verbatim and thematic analysis was used to study the interviewees' lived experience. The RHIS retained its structure, tools, and flow during the outbreak. The need for other types of data to inform the EVD response created other parallel systems to the RHIS. This included data from Ebola treatment centers, vaccination against Ebola, points of entry surveillance, and safe and dignified burial. The informants indicated that the availability of weekly surveillance data had improved, while timeliness and quality of monthly RHIS reporting declined. The compilation of data was late and validation meetings were irregular. The upsurge of patients following the implementation of the free care policy, the departure of healthcare workers for better-paid jobs, and the high prioritization of the outbreak response over routine activities led to RHIS disruptions. Delays in decision-making were one of the consequences of the decline in data timeliness. Adequate allocation of human resources, equitable salary policy, coordination, and integration of the response with local structures are necessary to ensure optimal functioning of the RHIS during an outbreak. Future research should assess the scale of data quality changes during outbreaks.

Pooled Prevalence of Adverse Pregnancy and Neonatal Outcomes in Malawi, South Africa, Uganda, and Zimbabwe: Results From a Systematic Review and Meta-Analyses to Inform Trials of Novel HIV Prevention Interventions During Pregnancy

Background: Robust data summarizing the prevalence of pregnancy and neonatal outcomes in low- and middle-income countries are critically important for studies evaluating investigational products for HIV prevention and treatment in pregnant and breastfeeding women. In preparation for studies evaluating the safety of the dapivirine vaginal ring for HIV prevention in pregnancy, we conducted a systematic literature review and meta-analyses to summarize the prevalence of pregnancy and neonatal outcomes in Malawi, South Africa, Uganda, and Zimbabwe.

Methods: Ten individual systematic literature reviews were conducted to identify manuscripts presenting prevalence data for 12 pregnancy and neonatal outcomes [pregnancy loss, stillbirth, preterm birth, low birthweight (LBW), neonatal mortality, congenital anomaly, chorioamnionitis, postpartum endometritis, postpartum hemorrhage, gestational hypertension, preeclampsia/eclampsia, and preterm premature rupture of membranes (PPROM)]. Studies included in the meta-analyses were published between January 1, 1998, and July 11, 2018, provided numerator and denominator data to support prevalence estimation, and included women of any HIV serostatus. Random-effects meta-analyses were conducted to estimate the pooled prevalence and 95% confidence interval (CI) for each outcome overall, by country, and by HIV status.

Results: A total of 152 manuscripts were included across the 12 outcomes. Overall, the frequency of stillbirth (n = 75 estimates), LBW (n = 68), and preterm birth (n = 67) were the most often reported. However, fewer than 10 total manuscripts reported prevalence estimates for chorioamnionitis, endometritis, or PPROM. The outcomes with the highest pooled prevalence were preterm birth (12.7%, 95%CI 11.2–14.3), LBW (11.7%, 95%CI 10.6–12.9), and gestational hypertension (11.4%, 95%CI 7.8–15.7). Among the outcomes with the lowest pooled prevalence estimates were neonatal mortality (1.7%, 95%CI 1.4–2.1), pregnancy loss [1.9%, 95%CI 1.1–2.8, predominately studies (23/29) assessing losses occurring after the first trimester], PPROM (2.2%, 95%CI 1.5–3.2), and stillbirth (2.5%, 95%CI 2.2–2.7).

Conclusions: Although this review identified numerous prevalence estimates for some outcomes, data were lacking for other important pregnancy-related conditions. Additional research in pregnant populations is needed for a thorough evaluation of investigational products, including for HIV prevention and treatment, and to inform better estimates of the burden of adverse pregnancy outcomes globally.

Effect of birthweight measurement quality improvement on low birthweight prevalence in rural Ethiopia

BACKGROUND

Low birthweight (LBW) (< 2500 g) is a significant determinant of infant morbidity and mortality worldwide. In low-income settings, the quality of birthweight data suffers from measurement and recording errors, inconsistent data reporting systems, and missing data from non-facility births. This paper describes birthweight data quality and the prevalence of LBW before and after implementation of a birthweight quality improvement (QI) initiative in Amhara region, Ethiopia.

METHODS

A comparative pre-post study was performed in selected rural health facilities located in West Gojjam and South Gondar zones. At baseline, a retrospective review of delivery records from February to May 2018 was performed in 14 health centers to collect birthweight data. A birthweight QI initiative was introduced in August 2019, which included provision of high-quality digital infant weight scales (precision 5 g), routine calibration, training in birth weighing and data recording, and routine field supervision. After the QI implementation, birthweight data were prospectively collected from late August to early September 2019, and December 2019 to June 2020. Data quality, as measured by heaping (weights at exact multiples of 500 g) and rounding to the nearest 100 g, and the prevalence of LBW were calculated before and after QI implementation.

RESULTS

We retrospectively reviewed 1383 delivery records before the QI implementation and prospectively measured 1371 newborn weights after QI implementation. Heaping was most frequently observed at 3000 g and declined from 26% pre-initiative to 6.7% post-initiative. Heaping at 2500 g decreased from 5.4% pre-QI to 2.2% post-QI. The percentage of rounding to the nearest 100 g was reduced from 100% pre-initiative to 36.5% post-initiative. Before the QI initiative, the prevalence of recognized LBW was 2.2% (95% confidence interval [CI]: 1.5-3.1) and after the QI initiative increased to 11.7% (95% CI: 10.1-13.5).

CONCLUSIONS

A QI intervention can improve the quality of birthweight measurements, and data measurement quality may substantially affect estimates of LBW prevalence.

Can routine health facility data be used to monitor subnational coverage of maternal, newborn and child health services in Uganda?

Background

Routine health facility data are a critical source of local monitoring of progress and performance at the subnational level. Uganda has been using district health statistics from facility data for many years. We aimed to systematically assess data quality and examine different methods to obtain plausible subnational estimates of coverage for maternal, newborn and child health interventions.

Methods

Annual data from the Uganda routine health facility information system 2015–2019 for all 135 districts were used, as well as national surveys for external comparison and the identification of near-universal coverage interventions. The quality of reported data on antenatal and delivery care and child immunization was assessed through completeness of facility reporting, presence of extreme outliers and internal data consistencies. Adjustments were made when necessary. The denominators for the coverage indicators were derived from population projections and health facility data on near-universal coverage interventions. The coverage results with different denominators were compared with the results from household surveys.

Results

Uganda’s completeness of reporting by facilities was near 100% and extreme outliers were rare. Inconsistencies in reported events, measured by annual fluctuations and between intervention consistency, were common and more among the 135 districts than the 15 subregions. The reported numbers of vaccinations were improbably high compared to the projected population of births or first antenatal visits – and especially so in 2015–2016. There were also inconsistencies between the population projections and the expected target population based on reported numbers of antenatal visits or immunizations. An alternative approach with denominators derived from facility data gave results that were more plausible and more consistent with survey results than based on population projections, although inconsistent results remained for substantive number of subregions and districts.

Conclusion

Our systematic assessment of the quality of routine reports of key events and denominators shows that computation of district health statistics is possible with transparent adjustments and methods, providing a general idea of levels and trends for most districts and subregions, but that improvements in data quality are essential to obtain more accurate monitoring.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06554-6.

Opening the 'black box' of collaborative improvement: a qualitative evaluation of a pilot intervention to improve quality of malaria surveillance data in public health centres in Uganda

BACKGROUND

Demand for high-quality surveillance data for malaria, and other diseases, is greater than ever before. In Uganda, the primary source of malaria surveillance data is the Health Management Information System (HMIS). However, HMIS data may be incomplete, inaccurate or delayed. Collaborative improvement (CI) is a quality improvement intervention developed in high-income countries, which has been advocated for low-resource settings. In Kayunga, Uganda, a pilot study of CI was conducted in five public health centres, documenting a positive effect on the quality of HMIS and malaria surveillance data. A qualitative evaluation was conducted concurrently to investigate the mechanisms of effect and unintended consequences of the intervention, aiming to inform future implementation of CI.

METHODS

The study intervention targeted health workers, including brief in-service training, plus CI with 'plan-do-study-act' (PDSA) cycles emphasizing self-reflection and group action, periodic learning sessions, and coaching from a CI mentor. Health workers collected data on standard HMIS out-patient registers. The qualitative evaluation (July 2015 to September 2016) included ethnographic observations at each health centre (over 12-14 weeks), in-depth interviews with health workers and stakeholders (n = 20), and focus group discussions with health workers (n = 6).

RESULTS

The results suggest that the intervention did facilitate improvement in data quality, but through unexpected mechanisms. The CI intervention was implemented as planned, but the PDSA cycles were driven largely by the CI mentor, not the health workers. In this context, characterized by a rigid hierarchy within the health system of limited culture of self-reflection and inadequate training and supervision, CI became an effective form of high-quality training with frequent supervisory visits. Health workers appeared motivated to improve data collection habits by their loyalty to the CI mentor and the potential for economic benefits, rather than a desire for self-improvement.

CONCLUSIONS

CI is a promising method of quality improvement and could have a positive impact on malaria surveillance data. However, successful scale-up of CI in similar settings may require deployment of highly skilled mentors. Further research, focusing on the effectiveness of 'real world' mentors using robust study designs, will be required to determine whether CI can be translated effectively and sustainably to low-resource settings.

Key informant perspectives on the challenges and opportunities for using routine health data for decision-making in Senegal

Background

Increasing the performance of routine health information systems (RHIS) is an important policy priority both globally and in Senegal. As RHIS data become increasingly important in driving decision-making in Senegal, it is imperative to understand the factors that determine their use.

Methods

Semi-structured interviews were conducted with 18 high- and mid-level key informants active in the malaria, tuberculosis and HIV programmatic areas in Senegal. Key informants were employed in the relevant divisions of the Senegal Ministry of Health or nongovernmental / civil society organizations. We asked respondents questions related to the flow, quality and use of RHIS data in their organizations. A framework approach was used to analyze the qualitative data.

Results

Although the respondents worked at the strategic levels of their respective organizations, they consistently indicated that data quality and data use issues began at the operational level of the health system before the data made its way to the central level. We classify the main identified barriers and facilitators to the use of routine data into six categories and attempt to describe their interrelated nature. We find that data quality is a central and direct determinant of RHIS data use. We report that a number of upstream factors in the Senegal context interact to influence the quality of routine data produced. We identify the sociopolitical, financial and system design determinants of RHIS data collection, dissemination and use. We also discuss the organizational and infrastructural factors that influence the use of RHIS data.

Conclusions

We recommend specific prescriptive actions with potential to improve RHIS performance in Senegal, the quality of the data produced and their use. These actions include addressing sociopolitical factors that often interrupt RHIS functioning in Senegal, supporting and motivating staff that maintain RHIS data systems as well as ensuring RHIS data completeness and representativeness. We argue for improved coordination between the various stakeholders in order to streamline RHIS data processes and improve transparency. Finally, we recommend the promotion of a sustained culture of data quality assessment and use.

Data and information needs of policymakers for palliative cancer care: a multi-country qualitative study

Background

Despite regional efforts to address concerns regarding the burden of advanced cancer in Africa, urgent attention is still required. Widespread issues include late symptom presentation, inaccessibility of palliative care services, limited resources, poor data quality, disparity in data availability, and lack of stakeholder engagement. One way of helping to address these issues is by understanding and meeting the data and information needs of policymakers in palliative cancer care.

Aims

To explore the views of policymakers regarding data availability, data gaps and preferred data formats to support policy and decision making for palliative cancer care in Nigeria, Uganda and Zimbabwe.

Methods

A secondary analysis of interview data collected as part of a cross-sectional qualitative study that aimed to explore the data and information needs of patients, policymakers and caregivers in Nigeria, Uganda and Zimbabwe. Framework analysis, guided by the MEASURE evaluation framework, was used to qualitatively analyse the data.

Results

Twenty-six policymakers were recruited. The policymakers data and information concerns are aligned to the MEASURE evaluation framework of data and information use and include; assessing and improving data use (e.g. low prioritisation of cancer); identifying and engaging the data user (e.g. data processes); improving data quality (e.g. manual data collection processes); improving data availability (e.g. the accessibility of data); identifying information needs (e.g. what is ‘need to know’?); capacity building in core competencies (e.g. skills gaps); strengthening organisational data demand and use (e.g. policy frameworks); monitoring, evaluating and communicating of data demand and use (e.g. trustworthiness of data).

Conclusions

We present evidence of data sources, challenges to their access and use, guidance on data needs for policymakers, and opportunities for better engagement between data producers, brokers and users. This framework of evidence should inform the development of strategies to improve data access and use for policy and decision making to improve palliative cancer services in participating countries with relevance to the wider region.

The impact of data quality assurance and control solutions on the completeness, accuracy, and consistency of data in a national spinal cord injury registry of Iran (NSCIR-IR)

STUDY DESIGN

Descriptive study.

OBJECTIVE

This study aimed to develop and evaluate a systematic arrangement for improvement and monitoring of data quality of the National Spinal Cord (and Column) Injury Registry of Iran (NSCIR-IR)-a multicenter hospital-based registry.

SETTING

SCI community in Iran.

METHODS

Quality assurance and quality control were the primary objectives in improving overall quality of data that were considered in designing a paper-based and computerized case report. To prevent incorrect data entry, we implemented several validation algorithms, including 70 semantic rules, 18 syntactic rules, seven temporal rules, and 13 rules for acceptable value range. Qualified and trained staff members were also employed to review and identify any defect, inaccuracy, or inconsistency in the data to improve data quality. A set of functions were implemented in the software to cross-validate, and feedback on data was provided by reviewers and registrars.

RESULTS

Socio-demographic data items were 100% complete, except for national ID and education level, which were 97% and 92.3% complete, respectively. Completeness of admission data and emergency medical services data were 100% except for arrival and transfer time (99.4%) and oxygen saturation (48.9%). Evaluation of data received from two centers located in Tehran proved to be 100% accurate following validation by quality reviewers. All data was also found to be 100% consistent.

CONCLUSIONS

This approach to quality assurance and consistency validation proved to be effective. Our solutions resulted in a significant decrease in the number of missing data.

Interventions to improve district-level routine health data in low-income and middle-income countries: a systematic review

BACKGROUND

Routine health information system(s) (RHIS) facilitate the collection of health data at all levels of the health system allowing estimates of disease prevalence, treatment and preventive intervention coverage, and risk factors to guide disease control strategies. This core health system pillar remains underdeveloped in many low-income and middle-income countries. Efforts to improve RHIS data coverage, quality and timeliness were launched over 10 years ago.

METHODS

A systematic review was performed across 12 databases and literature search engines for both peer-reviewed articles and grey literature reports on RHIS interventions. Studies were analysed in three stages: (1) categorisation of RHIS intervention components and processes; (2) comparison of intervention component effectiveness and (3) whether the post-intervention outcome improved above the WHO integrated disease surveillance response framework data quality standard of 80% or above.

RESULTS

5294 references were screened, resulting in 56 studies. Three key performance determinants-technical, organisational and behavioural-were proposed as critical to RHIS strengthening. Seventy-seven per cent [77%] of studies identified addressed all three determinants. The most frequently implemented intervention components were 'providing training' and 'using an electronic health management information systems'. Ninety-three per cent [93%] of pre-post or controlled trial studies showed improvements in one or more data quality outputs, but after applying a standard threshold of >80% post-intervention, this number reduced to 68%. There was an observed benefit of multi-component interventions that either conducted data quality training or that addressed improvement across multiple processes and determinants of RHIS.

CONCLUSION

Holistic data quality interventions that address multiple determinants should be continuously practised for strengthening RHIS. Studies with clearly defined and pragmatic outcomes are required for future RHIS improvement interventions. These should be accompanied by qualitative studies and cost analyses to understand which investments are needed to sustain high-quality RHIS in low-income and middle-income countries.

Assessing pregnancy and neonatal outcomes in Malawi, South Africa, Uganda, and Zimbabwe: Results from a systematic chart review

A systematic chart review was performed to estimate the frequency of pregnancy outcomes, pregnancy complications and neonatal outcomes at facilities in Blantyre, Malawi; Johannesburg, South Africa; Kampala, Uganda; and Chitungwiza and Harare, Zimbabwe to provide comparisons with estimates from an ongoing clinical trial evaluating the safety of two biomedical HIV prevention interventions in pregnancy. A multi-site, cross-sectional chart review was conducted at Maternal Obstetric Units and hospitals where women participating in the ongoing clinical trial would be expected to deliver. All individuals delivering at the designated facilities or admitted for postpartum care within seven days of a delivery elsewhere (home, health clinic, etc.) were included in the review. Data were abstracted for pregnancy outcomes, pregnancy complications, maternal and neonatal death, and congenital anomalies. Data from 10,138 records were abstracted across all four sites (Blantyre n = 2,384; Johannesburg n = 1,888; Kampala n = 3,708; Chitungwiza and Harare n = 2,158), which included 10,426 pregnancy outcomes. The prevalence of preterm birth was 13% (range across sites: 10.4-20.7) and 4.1% of deliveries resulted in stillbirth (range: 3.1-5.5). The most commonly noted pregnancy complication was gestational hypertension, reported among 4.4% of pregnancies. Among pregnancies resulting in a live birth, 15.5% were low birthweight (range: 13.8-17.4) and 2.0% resulted in neonatal death (range:1.2-3.2). Suspected congenital anomalies were noted in 1.2% of pregnancies. This study provides systematically collected data on background rates of pregnancy outcomes, pregnancy complications and neonatal outcomes that can be used as a reference in support of ongoing HIV prevention studies. In addition, estimates from this study provide important background data for future studies of investigational products evaluated in pregnancy in these urban settings.

Evaluation of neonatal mortality data completeness and accuracy in Ghana

Background

Cause-specific mortality data are required to set interventions to reduce neonatal mortality. However, in many developing countries, these data are either lacking or of low quality. We assessed the completeness and accuracy of cause of death (COD) data for neonates in Ghana to assess their usability for monitoring the effectiveness of health system interventions aimed at improving neonatal survival.

Methods

A lot quality assurance sampling survey was conducted in 20 hospitals in the public sector across four regions of Ghana. Institutional neonatal deaths (IND) occurring from 2014 through 2017 were divided into lots, defined as neonatal deaths occurring in a selected facility in a calendar year. A total of 52 eligible lots were selected: 10 from Ashanti region, and 14 each from Brong Ahafo, Eastern and Volta region. Nine lots were from 2014, 11 from 2015 and 16 each were from 2016 and 2017. The cause of death (COD) of 20 IND per lot were abstracted from admission and discharge (A&D) registers and validated against the COD recorded in death certificates, clinician’s notes or neonatal death audit reports for consistency. With the error threshold set at 5%, ≥ 17 correctly matched diagnoses in a sample of 20 deaths would make the lot accurate for COD diagnosis. Completeness of COD data was measured by calculating the proportion of IND that had death certificates completed.

Results

Nineteen out of 52 eligible (36.5%) lots had accurate COD diagnoses recorded in their A&D registers. The regional distribution of lots with accurate COD data is as follows: Ashanti (4, 21.2%), Brong Ahafo (7, 36.8%), Eastern (4, 21.1%) and Volta (4, 21.1%). Majority (9, 47.4%) of lots with accurate data were from 2016, followed by 2015 and 2017 with four (21.1%) lots. Two (10.5%) lots had accurate COD data in 2014. Only 22% (239/1040) of sampled IND had completed death certificates.

Conclusion

Death certificates were not reliably completed for IND in a sample of health facilities in Ghana from 2014 through 2017. The accuracy of cause-specific mortality data recorded in A&D registers was also below the desired target. Thus, recorded IND data in public sector health facilities in Ghana are not valid enough for decision-making or planning. Periodic data quality assessments can determine the magnitude of the data quality concerns and guide site-specific improvements in mortality data management.

Efficient semiparametric inference for two-phase studies with outcome and covariate measurement errors

In modern observational studies using electronic health records or other routinely collected data, both the outcome and covariates of interest can be error-prone and their errors often correlated. A cost-effective solution is the two-phase design, under which the error-prone outcome and covariates are observed for all subjects during the first phase and that information is used to select a validation subsample for accurate measurements of these variables in the second phase. Previous research on two-phase measurement error problems largely focused on scenarios where there are errors in covariates only or the validation sample is a simple random sample of study subjects. Herein, we propose a semiparametric approach to general two-phase measurement error problems with a quantitative outcome, allowing for correlated errors in the outcome and covariates and arbitrary second-phase selection. We devise a computationally efficient and numerically stable expectation-maximization algorithm to maximize the nonparametric likelihood function. The resulting estimators possess desired statistical properties. We demonstrate the superiority of the proposed methods over existing approaches through extensive simulation studies, and we illustrate their use in an observational HIV study.

Improving the quality of routine maternal and newborn data captured in primary health facilities in Gombe State, Northeastern Nigeria: a before-and-after study

OBJECTIVES

Primary objective: to assess nine data quality metrics for 14 maternal and newborn health data elements, following implementation of an integrated, district-focused data quality intervention.

SECONDARY OBJECTIVE

to consider whether assessing the data quality metrics beyond completeness and accuracy of facility reporting offered new insight into reviewing routine data quality.

DESIGN

Before-and-after study design.

SETTING

Primary health facilities in Gombe State, Northeastern Nigeria.

PARTICIPANTS

Monitoring and evaluation officers and maternal, newborn and child health coordinators for state-level and all 11 local government areas (district-equivalent) overseeing 492 primary care facilities offering maternal and newborn care services.

INTERVENTION

Between April 2017 and December 2018, we implemented an integrated data quality intervention which included: introduction of job aids and regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media.

OUTCOME MEASURES

9 metrics for the data quality dimensions of completeness and timeliness, internal consistency of reported data, and external consistency.

RESULTS

The data quality intervention was associated with improvements in seven of nine data quality metrics assessed including availability and timeliness of reporting, completeness of data elements, accuracy of facility reporting, consistency between related data elements, and frequency of outliers reported. Improvement differed by data element type, with content of care and commodity-related data improving more than contact-related data. Increases in the consistency between related data elements demonstrated improved internal consistency within and across facility documentation.

CONCLUSIONS

An integrated district-focused data quality intervention-including regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media-can increase the completeness, accuracy and internal consistency of facility-based routine data.

Evaluation of the malaria reporting system supported by the District Health Information System 2 in Solomon Islands

Background

District Health Information Systems 2 (DHIS2) is used for supporting health information management in 67 countries, including Solomon Islands. However, there have been few published evaluations of the performance of DHIS2-enhanced disease reporting systems, in particular for monitoring infectious diseases such as malaria. The aim of this study was to evaluate DHIS2 supported malaria reporting in Solomon Islands and to develop recommendations for improving the system.

Methods

The evaluation was conducted in three administrative areas of Solomon Islands: Honoria City Council, and Malaita and Guadalcanal Provinces. Records of nine malaria indicators including report submission date, total malaria cases, Plasmodium falciparum case record, Plasmodium vivax case record, clinical malaria, malaria diagnosed with microscopy, malaria diagnosed with (rapid diagnostic test) (RDT), record of drug stocks and records of RDT stocks from 1st January to 31st December 2016 were extracted from the DHIS2 database. The indicators permitted assessment in four core areas: availability, completeness, timeliness and reliability. To explore perceptions and point of view of the stakeholders on the performance of the malaria case reporting system, focus group discussions were conducted with health centre nurses, whilst in-depth interviews were conducted with stakeholder representatives from government (province and national) staff and World Health Organization officials who were users of DHIS2.

Results

Data were extracted from nine health centres in Honoria City Council and 64 health centres in Malaita Province. The completeness and timeliness from the two provinces of all nine indicators were 28.2% and 5.1%, respectively. The most reliable indicator in DHIS2 was ‘clinical malaria’ (i.e. numbers of clinically diagnosed malaria cases) with 62.4% reliability. Challenges to completeness were a lack of supervision, limited feedback, high workload, and a lack of training and refresher courses. Health centres located in geographically remote areas, a lack of regular transport, high workload and too many variables in the reporting forms led to delays in timely reporting. Reliability of reports was impacted by a lack of technical professionals such as statisticians and unavailability of tally sheets and reporting forms.

Conclusion

The availability, completeness, timeliness and reliability of nine malaria indicators collected in DHIS2 were variable within the study area, but generally low. Continued onsite support, supervision, feedback and additional enhancements, such as electronic reporting will be required to further improve the malaria reporting system.

Assessing coverage of essential maternal and child health interventions using health-facility data in Uganda

BACKGROUND

Nationally representative household surveys are the gold standard for tracking progress in coverage of life-saving maternal and child interventions, but often do not provide timely information on coverage at the local and health facility level. Electronic routine health information system (RHIS) data could help provide this information, but there are currently concerns about data quality. This analysis seeks to improve the usability of and confidence in electronic RHIS data by using adjustments to calculate more accurate numerators and denominators for essential interventions.

METHODS

Data from three sources (Ugandan Demographic and Health (UDHS) survey, electronic RHIS, and census) were used to provide estimates of essential maternal (> 4 antenatal care visits (ANC), skilled delivery, and postnatal care visit (PNC)) and child health interventions (diphtheria, pertussis, tetanus, and hepatitis B and Haemophilus influenzae type b and polio vaccination series, measles vaccination, and vitamin A). Electronic RHIS data was checked for quality and both numerators and denominators were adjusted to improve accuracy. Estimates were compared between the three sources.

RESULTS

Estimates of maternal health interventions from adjusted electronic RHIS data were lower than those of the UDHS, while child intervention estimates were typically higher. Adjustment of electronic RHIS data generally improved accuracy compared with no adjustment. There was considerable agreement between estimates from adjusted, electronic RHIS data, and UDHS for skilled delivery and first dose of childhood vaccination series, but lesser agreement for ANC visits and second and third doses of childhood vaccinations.

CONCLUSIONS

Nationally representative household surveys will likely continue being the gold standard of coverage estimates of maternal and child health interventions, but this analysis shows that current approaches to adjusting health facility estimate works better for some indications than others. Further efforts to improve accuracy of estimates from RHIS sources are needed.

Improving quality and use of routine health information system data in low- and middle-income countries: A scoping review

BACKGROUND

A routine health information system is one of the essential components of a health system. Interventions to improve routine health information system data quality and use for decision-making in low- and middle-income countries differ in design, methods, and scope. There have been limited efforts to synthesise the knowledge across the currently available intervention studies. Thus, this scoping review synthesised published results from interventions that aimed at improving data quality and use in routine health information systems in low- and middle-income countries.

METHOD

We included articles on intervention studies that aimed to improve data quality and use within routine health information systems in low- and middle-income countries, published in English from January 2008 to February 2020. We searched the literature in the databases Medline/PubMed, Web of Science, Embase, and Global Health. After a meticulous screening, we identified 20 articles on data quality and 16 on data use. We prepared and presented the results as a narrative.

RESULTS

Most of the studies were from Sub-Saharan Africa and designed as case studies. Interventions enhancing the quality of data targeted health facilities and staff within districts, and district health managers for improved data use. Combinations of technology enhancement along with capacity building activities, and data quality assessment and feedback system were found useful in improving data quality. Interventions facilitating data availability combined with technology enhancement increased the use of data for planning.

CONCLUSION

The studies in this scoping review showed that a combination of interventions, addressing both behavioural and technical factors, improved data quality and use. Interventions addressing organisational factors were non-existent, but these factors were reported to pose challenges to the implementation and performance of reported interventions.

The All Babies Count Initiative: Impact of a Health System Improvement Approach on Neonatal Care and Outcomes in Rwanda

A health system improvement program combining facility readiness support, clinical training/mentoring, and improvement collaboratives increased quality improvement capacity, improved maternal and newborn quality of care, and reduced neonatal mortality. These results can be used to inform system improvement approach design to transform quality of care and outcomes for newborns.

Introduction:

Poor-quality care contributes to a significant portion of neonatal deaths globally. The All Babies Count (ABC) initiative was an 18-month district-wide approach designed to improve clinical and system performance across 2 rural Rwandan districts.

Methods:

This pre-post intervention study measured change in maternal and newborn health (MNH) quality of care and neonatal mortality. Data from the facility and community health management information system and newly introduced indicators were extracted from facility registers. Medians and interquartile ranges were calculated for the health facility to assess changes over time, and a mixed-effects logistic regression model was created for neonatal mortality. A difference-in-differences analysis was conducted to compare the change in district neonatal mortality with the rest of rural Rwanda.

Results:

Improvements were seen in multiple measures of facility readiness and MNH quality of care, including antenatal care coverage, preterm labor management, and postnatal care quality. District hospital case fatality decreased, with a statistically significant reduction in district neonatal mortality (odds ratio [OR]=0.54; 95% confidence interval [CI]=0.36, 0.83) and among preterm/low birth weight neonates (OR=0.47; 95% CI=0.25, 0.90). Neonatal mortality was reduced from 30.1 to 19.6 deaths/1,000 live births in the intervention districts and remained relatively stable in the rest of rural Rwanda (difference in differences −12.9).

Conclusion:

The ABC initiative contributed to improved MNH quality of care and outcomes in rural Rwanda. A combined clinical and health system improvement approach could be an effective strategy to improve quality and reduce neonatal mortality.

Factors That Influence Data Use to Improve Health Service Delivery in Low- and Middle-Income Countries

BACKGROUND

Health service delivery indicators are designed to reveal how well health services meet a community's needs. Effective use of the data can enable targeted improvements in health service delivery. We conducted a systematic review to identify the factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income settings.

METHODS

We reviewed empirical studies published in 2005 or later that provided evidence on the use of health service delivery data at the primary care level in low- and middle-income countries. We searched Scopus, Medline, the Cochrane Library, and citations of included studies. We also searched the gray literature, using a separate strategy. We extracted information on study design, setting, study population, study objective, key findings, and any identified lessons learned.

RESULTS

Twelve studies met the inclusion criteria. This small number of studies suggests there is insufficient evidence to draw reliable conclusions. However, a content analysis identified the following potentially influential factors, which we classified into 3 categories: governance (leadership, participatory monitoring, regular review of data); production of information (presentation of findings, data quality, qualitative data); and health information system resources (electronic health management information systems, organizational structure, training). Contextual factors and performance-based financing were also each found to have a role; however, discussing these as mediating factors may not be practical in terms of promoting data use.

CONCLUSION

Scant evidence exists regarding factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income countries. However, the existing evidence highlights some factors that may have a role in improving data use. Further research may benefit from comparing data use factors across different types of program indicators or using our classification as a framework for field experiments.

Determinants of improved data consistency across routine immunization data tools for health facilities in Kano State, Nigeria

Introduction

in this study, determinants of improved data consistency for routine immunization information at health facilities was measured to identify associated factors.

Methods

between June and August 2015, 1055 HFs were visited across 44 Local Government Areas in Kano state. We assessed data consistency, frequency of supportive supervision visits, availability of trained staff and attendance to monthly LGA RI review meetings. We compared RI monthly summary forms (MSF) versus national health management information system summary form (NHMIS) and vaccine management form 1a (VM1a) versus HF vaccine utilization summary monthly summary (HFVUM) for consistency. Data consistency at HF was determined at <+10% between number of children reportedly immunized, and doses of vaccine opened using 3 antigens (BCG, Penta and Measles). Levels of discrepancy <10% were considered as good data consistency. Bivariate and multivariate analysis used to determine association.

Results

data Consistency was observed in 195 (18.5%) HFs between (MSF vs NHMIS) and 90 (8.5%) HFs between (VM1a vs HFVUM). Consistency between MSF vs NHMIS was associated with receiving one or more SS visits in the previous month (p=0.001), data collection tools availability (p=0.001), recent attendance to monthly LGA RI review meeting and availability of trained staff. Data consistency between VM1a form and the HF VU summary was associated with a recent documented SS visit (p=0.05) and availability of trained staff (p=0.05).

Conclusion

low level of data consistency was observed in Kano. Enhanced SS visits and availability of trained staff are associated with improved data quality.

Using routine health information data for research in low- and middle-income countries: a systematic review

BACKGROUND

Routine health information systems (RHISs) support resource allocation and management decisions at all levels of the health system, as well as strategy development and policy-making in many low- and middle-income countries (LMICs). Although RHIS data represent a rich source of information, such data are currently underused for research purposes, largely due to concerns over data quality. Given that substantial investments have been made in strengthening RHISs in LMICs in recent years, and that there is a growing demand for more real-time data from researchers, this systematic review builds upon the existing literature to summarize the extent to which RHIS data have been used in peer-reviewed research publications.

METHODS

Using terms 'routine health information system', 'health information system', or 'health management information system' and a list of LMICs, four electronic peer-review literature databases were searched from inception to February 202,019: PubMed, Scopus, EMBASE, and EconLit. Articles were assessed for inclusion based on pre-determined eligibility criteria and study characteristics were extracted from included articles using a piloted data extraction form.

RESULTS

We identified 132 studies that met our inclusion criteria, originating in 37 different countries. Overall, the majority of the studies identified were from Sub-Saharan Africa and were published within the last 5 years. Malaria and maternal health were the most commonly studied health conditions, although a number of other health conditions and health services were also explored.

CONCLUSIONS

Our study identified an increasing use of RHIS data for research purposes, with many studies applying rigorous study designs and analytic methods to advance program evaluation, monitoring and assessing services, and epidemiological studies in LMICs. RHIS data represent an underused source of data and should be made more available and further embraced by the research community in LMIC health systems.

Effect of data quality improvement intervention on health management information system data accuracy: An interrupted time series analysis

Background

As part of a partnership between the Institute for Healthcare Improvement and the Ethiopian Federal Ministry of Health, woreda-based quality improvement collaboratives took place between November 2016 and December 2017 aiming to accelerate reduction of maternal and neonatal mortality in Lemu Bilbilu, Tanqua Abergele and Duguna Fango woredas. Before starting the collaboratives, assessments found inaccuracies in core measures obtained from Health Management Information System reports.

Methods and results

Building on the quality improvement collaborative design, data quality improvement activities were added and we used the World Health Organization review methodology to drive a verification factor for the core measures of number of pregnant women that received their first antenatal care visit, number of pregnant women that received antenatal care on at least four visits, number of pregnant women tested for syphilis and number of births attended by skilled health personnel. Impact of the data quality improvement was assessed using interrupted time series analysis. We found accurate data across all time periods for Tanqua Abergele. In Lemu Bilbilu and Duguna Fango, data quality improved for all core metrics over time. In Duguna Fango, the verification factor for number of pregnant women that received their first antenatal care visit improved from 0.794 (95%CI 0.753, 0.836; p<0.001) pre-intervention by 0.173 (95%CI 0.128, 0.219; p<0.001) during the collaborative; and the verification factor for number of pregnant women tested for syphilis improved from 0.472 (95%CI 0.390, 0.554; p<0.001) pre-intervention by 0.460 (95%CI 0.369, 0.552; p<0.001) during the collaborative. In Lemu Bilbilu, the verification factor for number of pregnant women receiving a fourth antenatal visit rose from 0.589 (95%CI 0.513, 0.664; p<0.001) at baseline by 0.358 (95%CI 0.258, 0.458; p<0.001) post-intervention; and skilled birth attendance rose from 0.917 (95%CI 0.869, 0.965) at baseline by 0.083 (95%CI 0.030, 0.136; p<0.001) during the collaborative.

Conclusions

A Data quality improvement initiative embedded within woreda clinical improvement collaborative improved accuracy of data used to monitor maternal and newborn health services in Ethiopia.

A, Salim N, Zaman SB, Shamba D, Blencowe H, Ruysen H, El Arifeen S, Boggs D, Gordeev VS, Rahman QSU, Hossain T, Joshi E, Thapa S, Poudel RP, Poudel D, Chaudhary P, Karki R, Chitrakar B, Mkopi N, Wisiko A, Kitende AP, Shirati MR, Chingalo C, Semhando AO, Mtei C, Mwenisongole V, Bakuza JM, Kombo J, Mbaruku G, Lawn JE. Labour and delivery ward register data availability, quality, and utility - Every Newborn - birth indicators research tracking in hospitals (EN-BIRTH) study baseline analysis in three countries

BACKGROUND

Countries with the highest burden of maternal and newborn deaths and stillbirths often have little information on these deaths. Since over 81% of births worldwide now occur in facilities, using routine facility data could reduce this data gap. We assessed the availability, quality, and utility of routine labour and delivery ward register data in five hospitals in Bangladesh, Nepal, and Tanzania. This paper forms the baseline register assessment for the Every Newborn-Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study.

METHODS

We extracted 21 data elements from routine hospital labour ward registers, useful to calculate selected maternal and newborn health (MNH) indicators. The study sites were five public hospitals during a one-year period (2016-17). We measured 1) availability: completeness of data elements by register design, 2) data quality: implausibility, internal consistency, and heaping of birthweight and explored 3) utility by calculating selected MNH indicators using the available data.

RESULTS

Data were extracted for 20,075 births. Register design was different between the five hospitals with 10-17 of the 21 selected MNH data elements available. More data were available for health outcomes than interventions. Nearly all available data elements were > 95% complete in four of the five hospitals and implausible values were rare. Data elements captured in specific columns were 85.2% highly complete compared to 25.0% captured in non-specific columns. Birthweight data were less complete for stillbirths than live births at two hospitals, and significant heaping was found in all sites, especially at 2500g and 3000g. All five hospitals recorded count data required to calculate impact indicators including; stillbirth rate, low birthweight rate, Caesarean section rate, and mortality rates.

CONCLUSIONS

Data needed to calculate MNH indicators are mostly available and highly complete in EN-BIRTH study hospital routine labour ward registers in Bangladesh, Nepal and Tanzania. Register designs need to include interventions for coverage measurement. There is potential to improve data quality if Health Management Information Systems utilization with feedback loops can be strengthened. Routine health facility data could contribute to reduce the coverage and impact data gap around the time of birth.

Health management information system (HMIS) data verification: A case study in four districts in Rwanda

INTRODUCTION

Reliable Health Management and Information System (HMIS) data can be used with minimal cost to identify areas for improvement and to measure impact of healthcare delivery. However, variable HMIS data quality in low- and middle-income countries limits its value in monitoring, evaluation and research. We aimed to review the quality of Rwandan HMIS data for maternal and newborn health (MNH) based on consistency of HMIS reports with facility source documents.

METHODS

We conducted a cross-sectional study in 76 health facilities (HFs) in four Rwandan districts. For 14 MNH data elements, we compared HMIS data to facility register data recounted by study staff for a three-month period in 2017. A HF was excluded from a specific comparison if the service was not offered, source documents were unavailable or at least one HMIS report was missing for the study period. World Health Organization guidelines on HMIS data verification were used: a verification factor (VF) was defined as the ratio of register over HMIS data. A VF<0.90 or VF>1.10 indicated over- and under-reporting in HMIS, respectively.

RESULTS

High proportions of HFs achieved acceptable VFs for data on the number of deliveries (98.7%;75/76), antenatal care (ANC1) new registrants (95.7%;66/69), live births (94.7%;72/76), and newborns who received first postnatal care within 24 hours (81.5%;53/65). This was slightly lower for the number of women who received iron/folic acid (78.3%;47/60) and tested for syphilis in ANC1 (67.6%;45/68) and was the lowest for the number of women with ANC1 standard visit (25.0%;17/68) and fourth standard visit (ANC4) (17.4%;12/69). The majority of HFs over-reported on ANC4 (76.8%;53/69) and ANC1 (64.7%;44/68) standard visits.

CONCLUSION

There was variable HMIS data quality by data element, with some indicators with high quality and also consistency in reporting trends across districts. Over-reporting was observed for ANC-related data requiring more complex calculations, i.e., knowledge of gestational age, scheduling to determine ANC standard visits, as well as quality indicators in ANC. Ongoing data quality assessments and training to address gaps could help improve HMIS data quality.

Effect and feasibility of district level scale up of maternal, newborn and child health interventions in Pakistan: a quasi-experimental study

INTRODUCTION

Pakistan has a high burden of maternal, newborn and child morbidity and mortality. Several factors including weak scale-up of evidence-based interventions within the existing health system; lack of community awareness regarding health conditions; and poverty contribute to poor outcomes. Deaths and morbidity are largely preventable if a combination of community and facility-based interventions are rolled out at scale.

METHODS AND ANALYSIS

Umeed-e-Nau (UeN) (New Hope) project aims is to improve maternal, newborn and child health (MNCH) in eight high-burden districts of Pakistan by scaling up of evidence-based interventions. The project will assess interventions focused on, first, improving the quality of MNCH care at primary level and secondary level. Second, interventions targeting demand generation such as community mobilisation, creating awareness of healthy practices and expanding coverage of outreach services will be evaluated. Third, we will also evaluate interventions targeting the improvement in quality of routine health information and promotion of use of the data for decision-making. Hypothesis of the project is that roll out of evidence-based interventions at scale will lead to at least 20% reduction in perinatal mortality and 30% decrease in diarrhoea and pneumonia case fatality in the target districts whereas two intervention groups will serve as internal controls. Monitoring and evaluation of the programme will be undertaken through conducting periodical population level surveys and quality of care assessments. Descriptive and multivariate analytical methods will be used for assessing the association between different factors, and difference in difference estimates will be used to assess the impact of the intervention on outcomes.

ETHICS AND DISSEMINATION

The ethics approval was obtained from the Aga Khan University Ethics Review Committee. The findings of the project will be shared with relevant stakeholders and disseminated through open access peer-reviewed journal articles.

TRIAL REGISTRATION NUMBER

NCT04184544; Pre-results.

Factors limiting data quality in the expanded programme on immunization in low and middle-income countries: A scoping review

Few public health interventions can match the immense achievements of immunization in terms of mortality and morbidity reduction. However, progress in reaching global coverage goals and achieving universal immunization coverage have stalled; with key stakeholders concerned about the accuracy of reported coverage figures. Incomplete and incorrect data has made it challenging to obtain an accurate overview of immunization coverage, particularly in low- and middle-income countries (LMIC). To date, only one literature review concerning immunization data quality exists. However, it only included articles from Gavi-eligible countries, did not go deep into the characteristics of the data quality problems, and used a narrow 'data quality' definition. This scoping review builds upon that work; exploring the "state of data quality" in LMIC, factors affecting data quality in these settings and potential means to improve it. Only a small volume of literature addressing immunization data quality in LMIC was found and definitions of 'data quality' varied widely. Data quality was, on the whole, considered poor in the articles included. Coverage numerators were seen to be inflated for official reports and denominators were inaccurate and infrequently adjusted. Numerous factors related to these deficiencies were reported, including health information system fragmentation, overreliance on targets and poor data management processes. Factors associated with health workers were noted most frequently. Authors suggested that data quality could be improved by ensuring proper data collection tools, increasing workers' capacities and motivation through training and supervision, whilst also ensuring adequate and timely feedback on the data collected. The findings of this scoping review can serve as the basis to identify and address barriers to good quality immunization data in LMICs. Overcoming said barriers is essential if immunization's historic successes are to continue.

Self-audits as alternatives to travel-audits for improving data quality in the Caribbean, Central and South America network for HIV epidemiology

INTRODUCTION

Audits play a critical role in maintaining the integrity of observational cohort data. While previous work has validated the audit process, sending trained auditors to sites ("travel-audits") can be costly. We investigate the efficacy of training sites to conduct "self-audits."

METHODS

In 2017, eight research groups in the Caribbean, Central, and South America network for HIV Epidemiology each audited a subset of their patient records randomly selected by the data coordinating center at Vanderbilt. Designated investigators at each site compared abstracted research data to the original clinical source documents and captured audit findings electronically. Additionally, two Vanderbilt investigators performed on-site travel-audits at three randomly selected sites (one adult and two pediatric) in late summer 2017.

RESULTS

Self- and travel-auditors, respectively, reported that 93% and 92% of 8919 data entries, captured across 28 unique clinical variables on 65 patients, were entered correctly. Across all entries, 8409 (94%) received the same assessment from self- and travel-auditors (7988 correct and 421 incorrect). Of 421 entries mutually assessed as "incorrect," 304 (82%) were corrected by both self- and travel-auditors and 250 of these (72%) received the same corrections. Reason for changing antiretroviral therapy (ART) regimen, ART end date, viral load value, CD4%, and HIV diagnosis date had the most mismatched corrections.

CONCLUSIONS

With similar overall error rates, findings suggest that data audits conducted by trained local investigators could provide an alternative to on-site audits by external auditors to ensure continued data quality. However, discrepancies observed between corrections illustrate challenges in determining correct values even with audits.

Social franchising of community-based HIV counselling and testing services to increase HIV testing and linkage to care in Tshwane, South Africa: study protocol for a non-randomised implementation trial

BACKGROUND

Meeting the ambitious UN 90-90-90 HIV testing, treatment and viral load suppression targets requires innovative strategies and approaches in Sub-Saharan Africa. To date no known interventions have been tested with community health workers (counsellors) as social franchisees or owner-managed businesses in Community-based HIV counselling and testing (CBCT) work. The aim of this methods paper is to describe a Social franchise (SF) CBCT implementation trial to increase HIV testing and linkage to care for individuals at community levels in comparison with an existing CBCT programme methods.

METHODS/DESIGN

This is a two arm non-randomised community implementation trial with a once off round of post-test follow-up per HIV positive participant to assess linkage to care in low income communities. The intervention arm is a social franchise CBCT in which unemployed, self-employed or employed community members are recruited, contracted and incentivised to test at least 100 people per month, identifying at least 5 HIV positive tests and linking to care at least 4 of them. Social franchisees receive approximately $3.20 per HIV test and $8 per client linked to care. In the control arm, full-time employed HIV counsellors conduct CBCT on a fixed monthly salary. Primary study outcomes are HIV testing uptake rate, HIV positivity, Linkage to care and treatment rate and average counsellors' remuneration cost. Data collection will be conducted using both paper-based and electronic data applications by CBCT or SF counsellors. Data analysis will compare proportions of HIV testing, positivity, linkage to HIV care and treatment rates and counsellors' cost in the two study arms.

DISCUSSION

The study will provide important insight into whether the SF-delivered CBCT programme increases testing coverage and linkage to care as well as reducing CBCT cost per HIV test and per HIV positive person linked to care.

TRIAL REGISTRATION

Pan African Clinical Trial Registry PACTR201809873079121. The trial was retrospectively registered on 11 September 2018.

Evaluating the impact of a hospital scale-up phase of a quality improvement intervention in Ghana on mortality for children under five

OBJECTIVE

To evaluate the scale-up phase of a national quality improvement initiative across hospitals in Southern Ghana.

DESIGN

This evaluation used a comparison of pre- and post-intervention means to assess changes in outcomes over time. Multivariable interrupted time series analyses were performed to determine whether change categories (interventions) tested were associated with improvements in the outcomes.

SETTING

Hospitals in Southern Ghana.

PARTICIPANTS

The data sources were monthly outcome data from intervention hospitals along with program records.

INTERVENTION

The project used a quality improvement approach whereby process failures were identified by health staff and process changes were implemented in hospitals and their corresponding communities. The three change categories were: timely care-seeking, prompt provision of care and adherence to protocols.

MAIN OUTCOME MEASURES

Facility-level neonatal mortality, facility-level postneonatal infant mortality and facility-level postneonatal under-five mortality.

RESULTS

There were significant improvements for two outcomes from the pre-intervention to the post-intervention phase. Postneonatal infant mortality dropped from 44.3 to 21.1 postneonatal infant deaths per 1000 admissions, while postneonatal under-five mortality fell from 23.1 to 11.8 postneonatal under-five deaths per 1000 admissions. The multivariable interrupted time series analysis indicated that over the long-term the prompt provision of care change category was significantly associated with reduced postneonatal under five mortality (β = -0.0024, 95% CI -0.0051, 0.0003, P < 0.10).

CONCLUSIONS

The reduced postneonatal under-five mortality achieved in this project gives support to the promotion of quality improvement as a means to achieve health impacts at scale.

Impact of Implementing Antenatal Syphilis Point-of-Care Testing on Maternal Mortality in KwaZulu-Natal, South Africa: An Interrupted Time Series Analysis

Background: Syphilis infection has been associated with an increased risk of HIV infection during pregnancy which poses greater risk for maternal mortality, and antenatal syphilis point-of-care (POC) testing has been introduced to improve maternal and child health outcomes. There is limited evidence on the impact of syphilis POC testing on maternal outcomes in high HIV prevalent settings. We used syphilis POC testing as a model to evaluate the impact of POC diagnostics on the improvement of maternal mortality in KwaZulu-Natal, South Africa. Methods: We extracted 132 monthly data points on the number of maternal deaths in facilities and number of live births in facilities for 12 tertiary healthcare facilities in KwaZulu-Natal (KZN), South Africa from 2004 to 2014 from District Health Information System (DHIS) health facility archived. We employed segmented Poisson regression analysis of interrupted time series to assess the impact of the exposure on maternal mortality ratio (MMR) before and after the implementation of antenatal syphilis POC testing. We processed and analyzed data using Stata Statistical Software: Release 13. (Stata, Corp LP, College Station, TX, USA). Results: The provincial average annual maternal mortality ratio (MMR) was estimated at 176.09 ± 43.92 ranging from a minimum of 68.48 to maximum of 225.49 per 100,000 live births. The data comprised 36 temporal points before the introduction of syphilis POC test exposure and 84 after the introduction in primary health care clinics in KZN. The average annual MMR for KZN from 2004 to 2014 was estimated at 176.09 ± 43.92. A decrease in MMR level was observed during 2008 after syphilis POC test implementation, followed by a rise during 2009. Analysis of the MMR trend estimates a significant 1.5% increase in MMR trends during the period before implementation and 1.3% increase after implementation of syphilis POC testing (p < 0.001). Conclusion: Although our finding suggests a brief reduction in the MMR trend after the implementation of antenatal syphilis POC testing, a continued increase in syphilis rates is seen in KwaZulu-Natal, South Africa. The study used one of the most powerful quasi-experimental research methods, segmented Poisson regression analysis of interrupted time series to model the impact of syphilis POC on maternal outcome. The study finding requires confirmation by use of more rigorous primary study design.