Data quality assessment in the routine health information system: an application of the Lot Quality Assurance Sampling in Benin

Routine health information system utilization and associated factors among health professionals in public health facilities in Dire Dawa, eastern Ethiopia: A cross-sectional study

Background

Using reliable evidence from routine health information system (RHIS) over time is a vital aid to improve health outcome, tackling disparities, enhancing efficiency, and encouraging innovation. In Ethiopia, utilization of routine health data for improving the performance and quality of care was not well-studied in grassroot health facilities.

Objective

This study was conducted to determine the level of RHIS utilization and associated factors among health professionals in public health facilities of Dire Dawa, eastern Ethiopia.

Methods

An institution-based cross-sectional study was conducted among 378 health professionals from June 10 to July 20, 2020. Self-administered pretested-structured questionnaire was used to collect data from the participants. Data were entered using EpiData 3.1 and analyzed using Stata 16.0. Descriptive statistics was used to describe the basic characteristics of the participants, and multivariable logistic regression analysis was conducted to identify factors associated with RHIS utilization. Adjusted odds ratio (AOR) (95% CI) was used to report association and significance declared at a P-value <0.05.

Results

Good RHIS utilization among health professionals was 57.7% (95% CI: 52.6%, 62.6%). Good organizational support (AOR = 3.91, 95% CI: 2.01, 7.61), low perceived complexity of RHIS formats (AOR = 2.20, 95% CI: 1.23, 3.97), good self-efficacy (AOR = 2.52, 95% CI: 1.25, 5.10), and good decision-making autonomy (AOR = 3.97, 95% CI: 2.12, 7.43) were important factors associated with good RHIS utilization.

Conclusions

The level of good RHIS utilization among health professionals was low. Lack of self-confidence and empowerment, complexity of RHIS formats, and poor organizational support were significantly reducing RHIS utilization. Therefore, improving self-efficacy and decision-making capacity of health professionals through comprehensive training, empowerment, and organizational support would be essential.

From real-world individuals’ data to national health indicators: a pilot study in Gabon (Preprint)

Background

Achieving health goals requires informed decision-making supported by transparent, reliable, and relevant health information. This helps decision makers, such as health managers, to better understand the functioning of their health system and improve their ability to respond quickly to health demands. To achieve this, the health system needs to be supported by a digitized decision-making information system. In Sub-Saharan African countries, inadequate digital infrastructure, including limited internet connectivity and insufficient access to appropriate computer software, makes it difficult to collect, process, and analyze data for health statistics. The processing of data is done manually in this case; however, this situation affects the quality of the health statistics produced and compromises the quality of health intervention choices in these countries.

Objective

This study aimed to describe the conceptual approach of a data production and dissemination platform model proposed and implemented in Gabon. More precisely, it aimed to present the approach applied for the multidimensional analysis of the data production and dissemination process in the existing information system and present the results of an evaluation of the proposed model implemented in a real context.

Methods

The research was carried out in 3 phases. First, a platform was designed and developed based on the examination of the various data production and indicator generation procedures. Then, the platform was implemented in chosen health facilities in Gabon. Finally, a platform evaluation was carried out with actual end users.

Results

A total of 14 users with 12 years of average experience in health data management were interviewed. The results show that the use of the proposed model significantly improved the completeness, timeliness, and accuracy of data compared with the traditional system (93% vs 12%, P<.001; 96% vs 18%, P<.001; and 100% vs 18%, P<.001; respectively).

Conclusions

The proposed model contributes significantly to the improvement of health data quality in Gabon.

Health management information system (HMIS) data quality and associated factors in Massaguet district, Chad

Background

Quality data from Health Management Information Systems (HMIS) are important for tracking the effectiveness of malaria control interventions. However, HMIS data in many resource-limited settings do not currently meet standards set by the World Health Organization (WHO). We aimed to assess HMIS data quality and associated factors in Chad.

Methods

A cross-sectional study was conducted in 14 health facilities in Massaguet district. Data on children under 15 years were obtained from the HMIS and from the external patient register covering the period January–December 2018. An additional questionnaire was administered to 16 health centre managers to collect data on contextual variables. Patient registry data were aggregated and compared with the HMIS database at district and health centre level. Completeness and accuracy indicators were calculated as per WHO guidelines. Multivariate logistic regressions were performed on the Verification Factor for attendance, suspected and confirmed malaria cases for three age groups (1 to < 12 months, 1 to < 5 years and 5 to < 15 years) to identify associations between health centre characteristics and data accuracy.

Results

Health centres achieved a high level of data completeness in HMIS. Malaria data were over-reported in HMIS for children aged under 15 years. There was an association between workload and higher odds of inaccuracy in reporting of attendance among children aged 1 to < 5 years (Odds ratio [OR]: 10.57, 95% CI 2.32–48.19) and 5– < 15 years (OR: 6.64, 95% CI 1.38–32.04). Similar association was found between workload and stock-outs in register books, and inaccuracy in reporting of malaria confirmed cases. Meanwhile, we found that presence of a health technician, and of dedicated staff for data management, were associated with lower inaccuracy in reporting of clinic attendance in children aged under five years.

Conclusion

Data completeness was high while the accuracy was low. Factors associated with data inaccuracy included high workload and the unavailability of required data collection tools. The results suggest that improvement in working conditions for clinic personnel may improve HMIS data quality. Upgrading from paper-based forms to a web-based HMIS may provide a solution for improving data accuracy and its utility for future evaluations of health interventions. Results from this study can inform the Ministry of Health and it partners on the precautions to be taken in the use of HMIS data and inform initiatives for improving its quality.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01684-7.

Twenty years of integrated disease surveillance and response in Sub-Saharan Africa: challenges and opportunities for effective management of infectious disease epidemics

INTRODUCTION

This systematic review aimed to analyse the performance of the Integrated Disease Surveillance and Response (IDSR) strategy in Sub-Saharan Africa (SSA) and how its implementation has embraced advancement in information technology, big data analytics techniques and wealth of data sources.

METHODS

HINARI, PubMed, and advanced Google Scholar databases were searched for eligible articles. The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols.

RESULTS

A total of 1,809 articles were identified and screened at two stages. Forty-five studies met the inclusion criteria, of which 35 were country-specific, seven covered the SSA region, and three covered 3-4 countries. Twenty-six studies assessed the IDSR core functions, 43 the support functions, while 24 addressed both functions. Most of the studies involved Tanzania (9), Ghana (6) and Uganda (5). The routine Health Management Information System (HMIS), which collects data from health care facilities, has remained the primary source of IDSR data. However, the system is characterised by inadequate data completeness, timeliness, quality, analysis and utilisation, and lack of integration of data from other sources. Under-use of advanced and big data analytical technologies in performing disease surveillance and relating multiple indicators minimises the optimisation of clinical and practice evidence-based decision-making.

CONCLUSIONS

This review indicates that most countries in SSA rely mainly on traditional indicator-based disease surveillance utilising data from healthcare facilities with limited use of data from other sources. It is high time that SSA countries consider and adopt multi-sectoral, multi-disease and multi-indicator platforms that integrate other sources of health information to provide support to effective detection and prompt response to public health threats.

Assessment of RHIS Quality Assurance Practices in Tarkwa Submunicipal Health Directorate, Ghana

Background. Routine health information system (RHIS) quality assurance has become an important issue, not only because of its significance in promoting high standard of patient care, but also because of its impact on government budgets for the maintenance of health services. Routine health information system comprises healthcare data collection, compilation, storage, analysis, report generation, and dissemination on routine basis at the various healthcare settings. The data from RHIS give a representation of health status, health services, and health resources. The sources of RHIS data are normally individual health records, records of services delivered, and records of health resources. Using reliable information from routine health information systems is fundamental in the healthcare delivery system. Quality assurance practices are measures that are put in places to ensure the health data that are collected meet required quality standards. Routine health information system quality assurance practices ensure that data that are generated from the system are fit for use. This study considered quality assurance practices in the RHIS processes. Methods. A cross-sectional study was conducted in eight health facilities in Tarkwa Submunicipal health service in the western region of Ghana. The study involved routine quality assurance practices among the 90-health staff and management selected from facilities in Tarkwa Submunicipal who collect or use data routinely from 24th December, 2019, to 20th January, 2020. Results. Generally, Tarkwa Submunicipal health service appears to practice quality assurance during data collection, compilation, storage, analysis, and dissemination. The results show some achievement in quality control performance in report dissemination (77.6%), data analysis (68.0%), data compilation (67.4%), report compilation (66.3%), data storage (66.3%), and collection (61.1%). Conclusions. Even though Tarkwa Submunicipal health directorate engages some control measures to ensure data quality, there is the need to strengthen the process to achieve the targeted percentage of performance (90.0%). There was significant shortfall in quality assurance practices performance especially during data collection, with respect to the expected performance.

Data utilisation and factors influencing the performance of the health management information system in Tanzania

BACKGROUND

Health Management Information System (HMIS) is a set of data regularly collected at health care facilities to meet the needs of statistics on health services. This study aimed to determine the utilisation of HMIS data and factors influencing the health system's performance at the district and primary health care facility levels in Tanzania.

METHODS

This cross-sectional study was carried out in 11 districts and involved 115 health care facilities in Tanzania. Data were collected using a semi-structured questionnaire administered to health workers at facility and district levels and documented using an observational checklist. Thematic content analysis approach was used to synthesise and triangulate the responses and observations to extract essential information.

RESULTS

A total of 93 healthcare facility workers and 13 district officials were interviewed. About two-thirds (60%) of the facility respondents reported using the HMIS data, while only five out of 13 district respondents (38.5%) reported analysing HMIS data routinely. The HMIS data were mainly used for comparing performance in terms of services coverage (53%), monitoring of disease trends over time (50%), and providing evidence for community health education and promotion programmes (55%). The majority (41.4%) of the facility's personnel had not received any training on data management related to HMIS during the past 12 months prior to the survey. Less than half (42%) of the health facilities had received supervisory visits from the district office 3 months before this assessment. Nine district respondents (69.2%) reported systematically receiving feedback on the quality of their reports monthly and quarterly from higher authorities. Patient load was described to affect staff performance on data collection and management frequently.

CONCLUSION

Inadequate analysis and poor data utilisation practices were common in most districts and health facilities in Tanzania. Inadequate human and financial resources, lack of incentives and supervision, and lack of standard operating procedures on data management were the significant challenges affecting the HMIS performance in Tanzania.

Evaluation of neonatal mortality data completeness and accuracy in Ghana

Background

Cause-specific mortality data are required to set interventions to reduce neonatal mortality. However, in many developing countries, these data are either lacking or of low quality. We assessed the completeness and accuracy of cause of death (COD) data for neonates in Ghana to assess their usability for monitoring the effectiveness of health system interventions aimed at improving neonatal survival.

Methods

A lot quality assurance sampling survey was conducted in 20 hospitals in the public sector across four regions of Ghana. Institutional neonatal deaths (IND) occurring from 2014 through 2017 were divided into lots, defined as neonatal deaths occurring in a selected facility in a calendar year. A total of 52 eligible lots were selected: 10 from Ashanti region, and 14 each from Brong Ahafo, Eastern and Volta region. Nine lots were from 2014, 11 from 2015 and 16 each were from 2016 and 2017. The cause of death (COD) of 20 IND per lot were abstracted from admission and discharge (A&D) registers and validated against the COD recorded in death certificates, clinician’s notes or neonatal death audit reports for consistency. With the error threshold set at 5%, ≥ 17 correctly matched diagnoses in a sample of 20 deaths would make the lot accurate for COD diagnosis. Completeness of COD data was measured by calculating the proportion of IND that had death certificates completed.

Results

Nineteen out of 52 eligible (36.5%) lots had accurate COD diagnoses recorded in their A&D registers. The regional distribution of lots with accurate COD data is as follows: Ashanti (4, 21.2%), Brong Ahafo (7, 36.8%), Eastern (4, 21.1%) and Volta (4, 21.1%). Majority (9, 47.4%) of lots with accurate data were from 2016, followed by 2015 and 2017 with four (21.1%) lots. Two (10.5%) lots had accurate COD data in 2014. Only 22% (239/1040) of sampled IND had completed death certificates.

Conclusion

Death certificates were not reliably completed for IND in a sample of health facilities in Ghana from 2014 through 2017. The accuracy of cause-specific mortality data recorded in A&D registers was also below the desired target. Thus, recorded IND data in public sector health facilities in Ghana are not valid enough for decision-making or planning. Periodic data quality assessments can determine the magnitude of the data quality concerns and guide site-specific improvements in mortality data management.

Data quality of the routine health management information system at the primary healthcare facility and district levels in Tanzania

BACKGROUND

Effective planning for disease prevention and control requires accurate, adequately-analysed, interpreted and communicated data. In recent years, efforts have been put in strengthening health management information systems (HMIS) in Sub-Saharan Africa to improve data accessibility to decision-makers. This study assessed the quality of routine HMIS data at primary healthcare facility (HF) and district levels in Tanzania.

METHODS

This cross-sectional study involved reviews of documents, information systems and databases, and collection of primary data from facility-level registers, tally sheets and monthly summary reports. Thirty-four indicators from Outpatient, Inpatient, Antenatal care, Family Planning, Post-natal care, Labour and Delivery, and Provider-Initiated Testing and Counselling service areas were assessed. Indicator records were tracked and compared across the process of data collection, compilation and submission to the district office. Copies of monthly report forms submitted by facilities to the district were also reviewed. The availability and utilization of HMIS tools were assessed, while completeness and data accuracy levels were quantified for each phase of the reporting system.

RESULTS

A total of 115 HFs (including hospitals, health centres, dispensaries) in 11 districts were involved. Registers (availability rate = 91.1%; interquartile range (IQR) 66.7-100%) and report forms (86.9%; IQR 62.2-100%) were the most utilized tools. There was a limited use of tally-sheets (77.8%; IQR 35.6-100%). Tools availability at the dispensary was 91.1%, health centre 82.2% and hospital 77.8%, and was low in urban districts. The availability rate at the district level was 65% (IQR 48-75%). Wrongly filled or empty cells in registers and poor adherence to the coding procedures were observed. Reports were highly over-represented in comparison to registers' records, with large differences observed at the HF phase of the reporting system. The OPD and IPD areas indicated the highest levels of mismatch between data source and district office. Indicators with large number of clients, multiple variables, disease categorization, or those linked with dispensing medicine performed poorly.

CONCLUSION

There are high variations in the tool utilisation and data accuracy at facility and district levels. The routine HMIS is weak and data at district level inaccurately reflects what is available at the source. These results highlight the need to design tailored and inter-service strategies for improving data quality.

Improving the quality of routine maternal and newborn data captured in primary health facilities in Gombe State, Northeastern Nigeria: a before-and-after study

OBJECTIVES

Primary objective: to assess nine data quality metrics for 14 maternal and newborn health data elements, following implementation of an integrated, district-focused data quality intervention.

SECONDARY OBJECTIVE

to consider whether assessing the data quality metrics beyond completeness and accuracy of facility reporting offered new insight into reviewing routine data quality.

DESIGN

Before-and-after study design.

SETTING

Primary health facilities in Gombe State, Northeastern Nigeria.

PARTICIPANTS

Monitoring and evaluation officers and maternal, newborn and child health coordinators for state-level and all 11 local government areas (district-equivalent) overseeing 492 primary care facilities offering maternal and newborn care services.

INTERVENTION

Between April 2017 and December 2018, we implemented an integrated data quality intervention which included: introduction of job aids and regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media.

OUTCOME MEASURES

9 metrics for the data quality dimensions of completeness and timeliness, internal consistency of reported data, and external consistency.

RESULTS

The data quality intervention was associated with improvements in seven of nine data quality metrics assessed including availability and timeliness of reporting, completeness of data elements, accuracy of facility reporting, consistency between related data elements, and frequency of outliers reported. Improvement differed by data element type, with content of care and commodity-related data improving more than contact-related data. Increases in the consistency between related data elements demonstrated improved internal consistency within and across facility documentation.

CONCLUSIONS

An integrated district-focused data quality intervention-including regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media-can increase the completeness, accuracy and internal consistency of facility-based routine data.

Assessing cardiovascular disease risk factor screening inequalities in India using Lot Quality Assurance Sampling

Background

Cardiovascular diseases (CVDs) are the leading cause of mortality in India. India has rolled out Comprehensive Primary Health Care (CPHC) reforms including population based screening for hypertension and diabetes, facilitated by frontline health workers. Our study assessed blood pressure and blood sugar coverage achieved by frontline workers using Lot Quality Assurance Sampling (LQAS).

Methods

LQAS Supervision Areas were defined as catchments covered by frontline workers in primary health centres in two districts each of Uttar Pradesh and Delhi. In each Area, 19 households for each of four sampling universes (males, females, Above Poverty Line (APL) and Below Poverty Line (BPL)) were visited using probability proportional to size sampling. Following written informed consent procedures, a short questionnaire was administered to individuals aged 30 or older using tablets related to screening for diabetes and hypertension. Using the LQAS hand tally method, coverage across Supervision Areas was determined.

Results

A sample of 2052 individuals was surveyed, median ages ranging from 42 to 45 years. Caste affiliation, education levels, and occupation varied by location; the sample was largely married and Hindu. Awareness of and interaction with frontline health workers was reported in Uttar Pradesh and mixed in Delhi. Greater coverage of CVD risk factor screening (especially blood pressure) was seen among females, as compared to males. No clear pattern of inequality was seen by poverty status; some SAs did not have adequate BPL samples. Overall, blood pressure and blood sugar screening coverage by frontline health workers fell short of targeted coverage levels at the aggregate level, but in all sites, at least one area was crossing this threshold level.

Conclusion

CVD screening coverage levels at this early stage are low. More emphasis may be needed on reaching males. Sex and poverty related inequalities must be addressed by more closely studying the local context and models of service delivery where the threshold of screening is being met. LQAS is a pragmatic method for measuring program inequalities, in resource-constrained settings, although possibly not for spatially segregated population sub-groups.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-020-05914-y.

Factors That Influence Data Use to Improve Health Service Delivery in Low- and Middle-Income Countries

BACKGROUND

Health service delivery indicators are designed to reveal how well health services meet a community's needs. Effective use of the data can enable targeted improvements in health service delivery. We conducted a systematic review to identify the factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income settings.

METHODS

We reviewed empirical studies published in 2005 or later that provided evidence on the use of health service delivery data at the primary care level in low- and middle-income countries. We searched Scopus, Medline, the Cochrane Library, and citations of included studies. We also searched the gray literature, using a separate strategy. We extracted information on study design, setting, study population, study objective, key findings, and any identified lessons learned.

RESULTS

Twelve studies met the inclusion criteria. This small number of studies suggests there is insufficient evidence to draw reliable conclusions. However, a content analysis identified the following potentially influential factors, which we classified into 3 categories: governance (leadership, participatory monitoring, regular review of data); production of information (presentation of findings, data quality, qualitative data); and health information system resources (electronic health management information systems, organizational structure, training). Contextual factors and performance-based financing were also each found to have a role; however, discussing these as mediating factors may not be practical in terms of promoting data use.

CONCLUSION

Scant evidence exists regarding factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income countries. However, the existing evidence highlights some factors that may have a role in improving data use. Further research may benefit from comparing data use factors across different types of program indicators or using our classification as a framework for field experiments.

Factors limiting data quality in the expanded programme on immunization in low and middle-income countries: A scoping review

Few public health interventions can match the immense achievements of immunization in terms of mortality and morbidity reduction. However, progress in reaching global coverage goals and achieving universal immunization coverage have stalled; with key stakeholders concerned about the accuracy of reported coverage figures. Incomplete and incorrect data has made it challenging to obtain an accurate overview of immunization coverage, particularly in low- and middle-income countries (LMIC). To date, only one literature review concerning immunization data quality exists. However, it only included articles from Gavi-eligible countries, did not go deep into the characteristics of the data quality problems, and used a narrow 'data quality' definition. This scoping review builds upon that work; exploring the "state of data quality" in LMIC, factors affecting data quality in these settings and potential means to improve it. Only a small volume of literature addressing immunization data quality in LMIC was found and definitions of 'data quality' varied widely. Data quality was, on the whole, considered poor in the articles included. Coverage numerators were seen to be inflated for official reports and denominators were inaccurate and infrequently adjusted. Numerous factors related to these deficiencies were reported, including health information system fragmentation, overreliance on targets and poor data management processes. Factors associated with health workers were noted most frequently. Authors suggested that data quality could be improved by ensuring proper data collection tools, increasing workers' capacities and motivation through training and supervision, whilst also ensuring adequate and timely feedback on the data collected. The findings of this scoping review can serve as the basis to identify and address barriers to good quality immunization data in LMICs. Overcoming said barriers is essential if immunization's historic successes are to continue.

Data quality self-assessment of child health and sexual reproductive health indicators in Botswana, 2016-2017

There is no published data on quality of administrative data for various health indicators in Botswana, yet such data are used for policy making and future planning. This article reports on quality of data on child health and sexual and reproductive health (SRH) indicators in Botswana. The main objective of the study was to assess the quality of administrative data from Expanded Immunization Program (EPI) and condom use, Depo-Provera uptake and domiciliary care attendance in Botswana. This was a retrospective study entailing a review of data retrieved from district health records and District Health Information System (DHIS). A total of 30 clinics and health posts were randomly selected from two cities, a town and three rural villages which makes up 6 districts commonly denoted urban, semi-urban and rural respectively. Through a stratified random sampling health facilities were selected. EPI data (Penta 3- third dose of pentavalent vaccine and Measles vaccine) and SRH data (condom use, Depo-Provera uptake and Domiciliary care) were assessed for completeness, discrepancies and verification factor using WHO Routine data quality (RDQA) assessment tool. A verification score of less than 90%% was considered as underreporting while more than 110% is over reporting. However, the score which is within +-10% is acceptable, reliable and a good indicator of data quality and reporting system. About 56% (9/16) SRH indicators had a verification factor score outside the accepted range and 87% (13/15) discrepancy value outside the accepted range. For immunization, 10% (1/10) had a verification factor score outside the accepted range and 33% (3/9) had a discrepancy value outside the accepted range. The level of completeness was high for both Penta3 and Measles coverage and it was lowest for condom. Our findings highlight a poorer data quality for SRH indicators compared to child health indicators. A comprehensive program review drawing lessons from the child health indicators is required to improve the quality of administrative data in Botswana.

Strengthening immunization service delivery post Ebola virus disease (EVD) outbreak in Liberia 2015-2017

Introduction

The Ebola virus disease (EVD) outbreak in Liberia from 2014-2015 setback the already fragile health system which was recovering from the effects of civil unrest. This led to significant decline in immunization coverage and key polio free certification indicators. The Liberia investment plan was developed to restore immunization service delivery and overall health system.

Methods

We conducted a desk review to summarize performance of immunization coverage, polio eradication, measles control, new vaccines and technologies. Data sources include program reports, scientific and grey literature, District Health Information System (DHIS2), Integrated Diseases Surveillance and Response (IDSR) database, auto visual AFP detection and reporting (AVADAR) and ONA Servers. Data analysis was done using Microsoft excel spreadsheets, ONA software and Arc GIS.

Results

There was a 36% increase in national coverage for Penta 3 in 2017 compared to 2014 from WUENIC data. Penta 3 dropout rate reduced by 2.5 fold from 15.3% in 2016 to 6.4% in 2017; while MCV1 coverage improved by 23% from 64% in 2015 to 87% in 2017. There was a rebound of non-polio AFP rate (NPAFP) rate from 1.2 in 2015 to 4.3 in 2017. Furthermore, there was a 2-fold increase in the number of AFP cases receiving 3 or more doses of OPV from 36% in 2015 to 61% in 2017.

Conclusion

Liberia demonstrated strong rebound of immunization services following the largest and most devastating EVD outbreak in West Africa in 2014 - 2015. Immunization coverage improved and dropout rates reduced. However, there are still opportunities for improvement in the immunization program both at national and sub-national levels.

A quality assessment of Health Management Information System (HMIS) data for maternal and child health in Jimma Zone, Ethiopia

Health management information system (HMIS) data are important for guiding the attainment of health targets in low- and middle-income countries. However, the quality of HMIS data is often poor. High-quality information is especially important for populations experiencing high burdens of disease and mortality, such as pregnant women, newborns, and children. The purpose of this study was to assess the quality of maternal and child health (MCH) data collected through the Ethiopian Ministry of Health’s HMIS in three districts of Jimma Zone, Oromiya Region, Ethiopia over a 12-month period from July 2014 to June 2015. Considering data quality constructs from the World Health Organization’s data quality report card, we appraised the completeness, timeliness, and internal consistency of eight key MCH indicators collected for all the primary health care units (PHCUs) located within three districts of Jimma Zone (Gomma, Kersa and Seka Chekorsa). We further evaluated the agreement between MCH service coverage estimates from the HMIS and estimates obtained from a population-based cross-sectional survey conducted with 3,784 women who were pregnant in the year preceding the survey, using Pearson correlation coefficients, intraclass correlation coefficients (ICC), and Bland-Altman plots. We found that the completeness and timeliness of facility reporting were highest in Gomma (75% and 70%, respectively) and lowest in Kersa (34% and 32%, respectively), and observed very few zero/missing values and moderate/extreme outliers for each MCH indicator. We found that the reporting of MCH indicators improved over time for all PHCUs, however the internal consistency between MCH indicators was low for several PHCUs. We found poor agreement between MCH estimates obtained from the HMIS and the survey, indicating that the HMIS may over-report the coverage of key MCH services, namely, antenatal care, skilled birth attendance and postnatal care. The quality of MCH data within the HMIS at the zonal level in Jimma, Ethiopia, could be improved to inform MCH research and programmatic efforts.

Quality of routine facility data for monitoring priority maternal and newborn indicators in DHIS2: A case study from Gombe State, Nigeria

INTRODUCTION

Routine health information systems are critical for monitoring service delivery. District Heath Information System, version 2 (DHIS2) is an open source software platform used in more than 60 countries, on which global initiatives increasingly rely for such monitoring. We used facility-reported data in DHIS2 for Gombe State, north-eastern Nigeria, to present a case study of data quality to monitor priority maternal and neonatal health indicators.

METHODS

For all health facilities in DHIS2 offering antenatal and postnatal care services (n = 497) and labor and delivery services (n = 486), we assessed the quality of data for July 2016-June 2017 according to the World Health Organization data quality review guidance. Using data from DHIS2 as well as external facility-level and population-level household surveys, we reviewed three data quality dimensions-completeness and timeliness, internal consistency, and external consistency-and considered the opportunities for improvement.

RESULTS

Of 14 priority maternal and neonatal health indicators that could be tracked through facility-based data, 12 were included in Gombe's DHIS2. During July 2016-June 2017, facility-reported data in DHIS2 were incomplete at least 40% of the time, under-reported 10%-60% of the events documented in facility registers, and showed inconsistencies over time, between related indicators, and with an external data source. The best quality data elements were those that aligned with Gombe's health program priorities, particularly older health programs, and those that reflected contact indicators rather than indicators related to the provision of commodities or content of care.

CONCLUSION

This case study from Gombe State, Nigeria, demonstrates the high potential for effective monitoring of maternal and neonatal health using DHIS2. However, coordinated action at multiple levels of the health system is needed to maximize reporting of existing data; rationalize data flow; routinize data quality review, feedback, and supervision; and ensure ongoing maintenance of DHIS2.

Hospital mortality statistics in Tanzania: availability, accessibility, and quality 2006-2015

Background

Accurate and reliable hospital information on the pattern and causes of death is important to monitor and evaluate the effectiveness of health policies and programs. The objective of this study was to assess the availability, accessibility, and quality of hospital mortality data in Tanzania.

Methods

This cross-sectional study involved selected hospitals of Tanzania and was carried out from July to October 2016. Review of hospital death registers and forms was carried out to cover a period of 10 years (2006–2015). Interviews with hospital staff were conducted to seek information as regards to tools used to record mortality data, staff involved in recording and availability of data storage and archiving facilities.

Results

A total of 247,976 death records were reviewed. The death register was the most (92.3%) common source of mortality data. Other sources included the International Classification of Diseases (ICD) report forms, Inpatient registers, and hospital administrative reports. Death registers were available throughout the 10-year period while ICD-10 forms were available for the period of 2013–2015. In the years between 2006 and 2010 and 2011–2015, the use of death register increased from 82 to 94.9%. Three years after the introduction of ICD-10 procedure, the forms were available and used in 28% (11/39) hospitals. The level of acceptable data increased from 69% in 2006 to 97% in 2015. Inconsistency in the language used, use of non-standard nomenclature for causes of death, use of abbreviations, poorly and unreadable handwriting, and missing variables were common data quality challenges. About 6.3% (n = 15,719) of the records had no patient age, 3.5% (n = 8790) had no cause of death and ~ 1% had no sex indicated. The frequency of missing sex variable was most common among under-5 children. Data storage and archiving in most hospitals was generally poor. Registers and forms were stored in several different locations, making accessibility difficult.

Conclusion

Overall, this study demonstrates gaps in hospital mortality data availability, accessibility, and quality, and highlights the need for capacity strengthening in data management and periodic record reviews. Policy guidelines on the data management including archiving are necessary to improve data.

Perceptions about data-informed decisions: an assessment of information-use in high HIV-prevalence settings in South Africa

Background

Information-use is an integral component of a routine health information system and essential to influence policy-making, program actions and research. Despite an increased amount of routine data collected, planning and resource-allocation decisions made by health managers for managing HIV programs are often not based on data. This study investigated the use of information, and barriers to using routine data for monitoring the prevention of mother-to-child transmission of HIV (PMTCT) programs in two high HIV-prevalence districts in South Africa.

Methods

We undertook an observational study using a multi-method approach, including an inventory of facility records and reports. The performance of routine information systems management (PRISM) diagnostic ‘Use of Information’ tool was used to assess the PMTCT information system for evidence of data use in 57 health facilities in two districts. Twenty-two in-depth interviews were conducted with key informants to investigate barriers to information use in decision-making. Participants were purposively selected based on their positions and experience with either producing PMTCT data and/or using data for management purposes. We computed descriptive statistics and used a general inductive approach to analyze the qualitative data.

Results

Despite the availability of mechanisms and processes to facilitate information-use in about two-thirds of the facilities, evidence of information-use (i.e., indication of some form of information-use in available RHIS reports) was demonstrated in 53% of the facilities. Information was inadequately used at district and facility levels to inform decisions and planning, but was selectively used for reporting and monitoring program outputs at the provincial level. The inadequate use of information stemmed from organizational issues such as the lack of a culture of information-use, lack of trust in the data, and the inability of program and facility managers to analyze, interpret and use information.

Conclusions

Managers’ inability to use information implied that decisions for program implementation and improving service delivery were not always based on data. This lack of data use could influence the delivery of health care services negatively. Facility and program managers should be provided with opportunities for capacity development as well as practice-based, in-service training, and be supported to use information for planning, management and decision-making.

Perceptions about data-informed decisions: an assessment of information-use in high HIV-prevalence settings in South Africa

BACKGROUND

Information-use is an integral component of a routine health information system and essential to influence policy-making, program actions and research. Despite an increased amount of routine data collected, planning and resource-allocation decisions made by health managers for managing HIV programs are often not based on data. This study investigated the use of information, and barriers to using routine data for monitoring the prevention of mother-to-child transmission of HIV (PMTCT) programs in two high HIV-prevalence districts in South Africa.

METHODS

We undertook an observational study using a multi-method approach, including an inventory of facility records and reports. The performance of routine information systems management (PRISM) diagnostic 'Use of Information' tool was used to assess the PMTCT information system for evidence of data use in 57 health facilities in two districts. Twenty-two in-depth interviews were conducted with key informants to investigate barriers to information use in decision-making. Participants were purposively selected based on their positions and experience with either producing PMTCT data and/or using data for management purposes. We computed descriptive statistics and used a general inductive approach to analyze the qualitative data.

RESULTS

Despite the availability of mechanisms and processes to facilitate information-use in about two-thirds of the facilities, evidence of information-use (i.e., indication of some form of information-use in available RHIS reports) was demonstrated in 53% of the facilities. Information was inadequately used at district and facility levels to inform decisions and planning, but was selectively used for reporting and monitoring program outputs at the provincial level. The inadequate use of information stemmed from organizational issues such as the lack of a culture of information-use, lack of trust in the data, and the inability of program and facility managers to analyze, interpret and use information.

CONCLUSIONS

Managers' inability to use information implied that decisions for program implementation and improving service delivery were not always based on data. This lack of data use could influence the delivery of health care services negatively. Facility and program managers should be provided with opportunities for capacity development as well as practice-based, in-service training, and be supported to use information for planning, management and decision-making.

Assessing the quality of routine data for the prevention of mother-to-child transmission of HIV: An analytical observational study in two health districts with high HIV prevalence in South Africa

BACKGROUND

The prevention of mother-to-child transmission of HIV (PMTCT) is a key maternal and child-health intervention in the context of the HIV/AIDS pandemic in South Africa. Accordingly, the PMTCT programmes have been incorporated in the routine District Health Management Information System (DHMIS) which collects monthly facility-based data to support the management of public-health services. To date, there has been no comprehensive evaluation of the PMTCT information system.

OBJECTIVES

This study seeks to evaluate the quality of output indicators for monitoring PMTCT interventions in two health districts with high HIV prevalence.

METHODS

An analytical observational study was undertaken based on the Performance of Routine Information System Management (PRISM) framework and tools, including an assessment of the routine PMTCT data for quality in terms of accuracy and completeness. Data were collected from 57 public health facilities for six pre-defined PMTCT data elements by comparing the source registers with the routine monthly report (RMR), and the RMR with the DMHIS for January and April 2012. This was supplemented by the analysis of the monthly data reported routinely in the DMHIS for the period 2009-2012. Descriptive statistics, analysis of variance (ANOVA) and Bland Altman analysis were conducted using STATA^® Version 13.

RESULTS

Although completeness was relatively high at 91% (95% CI: 78-100%) at facility level and 96% (95% CI: 92-100%) at district level, the study revealed considerable data quality concerns for the PMTCT information with an average accuracy between the register and RMR of 51% (95% CI: 44-58%) and between the RMR and DHMIS database of 84% (95% CI: 78-91%). We observed differences in the data accuracy by organisational authority. The poor quality of the data was attributed partly to insufficient competencies of health information personnel.

CONCLUSIONS

The study suggests that the primary point of departure for accurate data transfer is during the collation process. Institutional capacity to improve data quality at the facility level and ensure core competencies for routine health information system (RHIS)-related tasks are needed. Further exploration of the possible factors that may influence data accuracy, such as supervision, RHIS processes, training and leadership are needed. In particular understanding is needed about how individual actions can bring about changes in institutional routines.

An assessment of data quality in a multi-site electronic medical record system in Haiti

OBJECTIVES

Strong data quality (DQ) is a precursor to strong data use. In resource limited settings, routine DQ assessment (DQA) within electronic medical record (EMR) systems can be resource-intensive using manual methods such as audit and chart review; automated queries offer an efficient alternative. This DQA focused on Haiti's national EMR - iSanté - and included longitudinal data for over 100,000 persons living with HIV (PLHIV) enrolled in HIV care and treatment services at 95 health care facilities (HCF).

METHODS

This mixed-methods evaluation used a qualitative Delphi process to identify DQ priorities among local stakeholders, followed by a quantitative DQA on these priority areas. The quantitative DQA examined 13 indicators of completeness, accuracy, and timeliness of retrospective data collected from 2005 to 2013. We described levels of DQ for each indicator over time, and examined the consistency of within-HCF performance and associations between DQ and HCF and EMR system characteristics.

RESULTS

Over all iSanté data, age was incomplete in <1% of cases, while height, pregnancy status, TB status, and ART eligibility were more incomplete (approximately 20-40%). Suspicious data flags were present for <3% of cases of male sex, ART dispenses, CD4 values, and visit dates, but for 26% of cases of age. Discontinuation forms were available for about half of all patients without visits for 180 or more days, and >60% of encounter forms were entered late. For most indicators, DQ tended to improve over time. DQ was highly variable across HCF, and within HCFs DQ was variable across indicators. In adjusted analyses, HCF and system factors with generally favorable and statistically significant associations with DQ were University hospital category, private sector governance, presence of local iSante server, greater HCF experience with the EMR, greater maturity of the EMR itself, and having more system users but fewer new users. In qualitative feedback, local stakeholders emphasized lack of stable power supply as a key challenge to data quality and use of the iSanté EMR.

CONCLUSIONS

Variable performance on key DQ indicators across HCF suggests that excellent DQ is achievable in Haiti, but further effort is needed to systematize and routinize DQ approaches within HCFs. A dynamic, interactive "DQ dashboard" within iSanté could bring transparency and motivate improvement. While the results of the study are specific to Haiti's iSanté data system, the study's methods and thematic lessons learned holdgeneralized relevance for other large-scale EMR systems in resource-limited countries.