Improving quality and use of data through data-use workshops: Zanzibar, United Republic of Tanzania

Barriers and benefits of mHealth for community health workers in integrated community case management of childhood diseases in Banda Parish, Kampala, Uganda: a cross-sectional study

BACKGROUND

Low-quality data presents a significant challenge for community health workers (CHWs) in low and middle-income countries (LMICs). Mobile health (mHealth) applications offer a solution by enabling CHWs to record and submit data electronically. However, the barriers and benefits of mHealth usage among CHWs in informal urban settlements remain poorly understood. This study sought to determine the barriers and benefits of mHealth among CHWs in Banda parish, Kampala.

METHODS

This qualitative study involved 12 key informant interviews (KIIs) among focal persons from Kampala City Council Authority (KCCA) and NGOs involved in data collected by CHWs, and officials from the Ministry of Health (MOH) and two mixed-sex Focused Group Discussions (FGDs) of CHWs from Banda parish, Kampala district. Data analysis utilised Atlas Ti Version 7.5.7. Thematic analysis was conducted, and themes were aligned with the social-ecological model.

RESULTS

Three themes of institutional and policy, community and interpersonal, and individual aligning to the Social ecological model highlighted the factors contributing to barriers and the benefits of mHealth among CHWs for iCCM. The key barriers to usability, acceptability and sustainability included high training costs, CHW demotivation, infrastructure limitations, data security concerns, community awareness deficits, and skill deficiencies. Conversely, mHealth offers benefits such as timely data submission, enhanced data quality, geo-mapping capabilities, improved CHW performance monitoring, community health surveillance, cost-effective reporting, and CHW empowering with technology.

CONCLUSION

Despite limited mHealth experience, CHWs expressed enthusiasm for its potential. Implementation was viewed as a solution to multiple challenges, facilitating access to health information, efficient data reporting, and administrative processes, particularly in resource-constrained settings. Successful mHealth implementation requires addressing CHWs' demotivation, ensuring reliable power and network connectivity, and enhancing capacity for digital data ethics and management. By overcoming these barriers, mHealth can significantly enhance healthcare delivery at the community level, leveraging technology to optimize resource utilization and improve health outcomes. mHealth holds promise for transforming CHW practices, yet its effective integration necessitates targeted interventions to address systemic challenges and ensure sustainable implementation in LMIC contexts.

Development of novel composite data quality scores to evaluate facility-level data quality in electronic data in Kenya: a nationwide retrospective cohort study

BACKGROUND

In this evaluation, we aim to strengthen Routine Health Information Systems (RHIS) through the digitization of data quality assessment (DQA) processes. We leverage electronic data from the Kenya Health Information System (KHIS) which is based on the District Health Information System version 2 (DHIS2) to perform DQAs at scale. We provide a systematic guide to developing composite data quality scores and use these scores to assess data quality in Kenya.

METHODS

We evaluated 187 HIV care facilities with electronic medical records across Kenya. Using quarterly, longitudinal KHIS data from January 2011 to June 2018 (total N = 30 quarters), we extracted indicators encompassing general HIV services including services to prevent mother-to-child transmission (PMTCT). We assessed the accuracy (the extent to which data were correct and free of error) of these data using three data-driven composite scores: 1) completeness score; 2) consistency score; and 3) discrepancy score. Completeness refers to the presence of the appropriate amount of data. Consistency refers to uniformity of data across multiple indicators. Discrepancy (measured on a Z-scale) refers to the degree of alignment (or lack thereof) of data with rules that defined the possible valid values for the data.

RESULTS

A total of 5,610 unique facility-quarters were extracted from KHIS. The mean completeness score was 61.1% [standard deviation (SD) = 27%]. The mean consistency score was 80% (SD = 16.4%). The mean discrepancy score was 0.07 (SD = 0.22). A strong and positive correlation was identified between the consistency score and discrepancy score (correlation coefficient = 0.77), whereas the correlation of either score with the completeness score was low with a correlation coefficient of -0.12 (with consistency score) and -0.36 (with discrepancy score). General HIV indicators were more complete, but less consistent, and less plausible than PMTCT indicators.

CONCLUSION

We observed a lack of correlation between the completeness score and the other two scores. As such, for a holistic DQA, completeness assessment should be paired with the measurement of either consistency or discrepancy to reflect distinct dimensions of data quality. Given the complexity of the discrepancy score, we recommend the simpler consistency score, since they were highly correlated. Routine use of composite scores on KHIS data could enhance efficiencies in DQA at scale as digitization of health information expands and could be applied to other health sectors beyondHIV clinics.

Identifying barriers to the production and use of routine health information in Western Province, Zambia

Recent decades of improvements to routine health information systems in low- and middle-income countries (LMICs) have increased the volume of health data collected. However, countries continue to face several challenges with quality production and use of information for decision-making at sub-national levels, limiting the value of health information for policy, planning and research. Improving the quality of data production and information use is thus a priority in many LMICs to improve decision-making and health outcomes. This qualitative study identified the challenges of producing and using routine health information in Western Province, Zambia. We analysed the interview responses from 37 health and social sector professionals at the national, provincial, district and facility levels to understand the barriers to using data from the Zambian health management information system (HMIS). Respondents raised several challenges that we categorized into four themes: governance and health system organization, geographic barriers, technical and procedural barriers, and challenges with human resource capacity and staff training. Staff at the facility and district levels were arguably the most impacted by these barriers as they are responsible for much of the labour to collect and report routine data. However, facility and district staff had the least authority and ability to mitigate the barriers to data production and information use. Expectations for information use should therefore be clearly outlined for each level of the health system. Further research is needed to understand to what extent the available HMIS data address the needs and purposes of the staff at facilities and districts.

Refining the Performance of Routine Information System Management (PRISM) framework for data use at the local level: An integrative review

INTRODUCTION

Foundational to a well-functioning health system is a strong routine health information system (RHIS) that informs decisions and actions at all levels of the health system. In the context of decentralization across low- and middle-income countries, RHIS has the promise of supporting sub-national health staff to take data-informed actions to improve health system performance. However, there is wide variation in how "RHIS data use" is defined and measured in the literature, impeding the development and evaluation of interventions that effectively promote RHIS data use.

METHODS

An integrative review methodology was used to: (1) synthesize the state of the literature on how RHIS data use in low- and middle-income countries is conceptualized and measured; (2) propose a refined RHIS data use framework and develop a common definition for RHIS data use; and (3) propose improved approaches to measure RHIS data use. Four electronic databases were searched for peer-reviewed articles published between 2009 and 2021 investigating RHIS data use.

RESULTS

A total of 45 articles, including 24 articles measuring RHIS data use, met the inclusion criteria. Less than half of included articles (42%) explicitly defined RHIS data use. There were differences across the literature whether RHIS data tasks such as data analysis preceded or were a part of RHIS data use; there was broad consensus that data-informed decisions and actions were essential steps within the RHIS data use process. Based on the synthesis, the Performance of Routine Information System Management (PRISM) framework was refined to specify the steps of the RHIS data use process.

CONCLUSION

Conceptualizing RHIS data use as a process that includes data-informed actions emphasizes the importance of actions in improving health system performance. Future studies and implementation strategies should be designed with consideration for the different support needs for each step of the RHIS data use process.

Feasibility, usability and acceptability of a novel digital hybrid-system for reporting of routine maternal health information in Southern Tanzania: A mixed-methods study

Health information systems are important for health planning and progress monitoring. Still, data from health facilities are often of limited quality in Low-and-Middle-Income Countries. Quality deficits are partially rooted in the fact that paper-based documentation is still the norm at facility level, leading to mistakes in summarizing and manual copying. Digitization of data at facility level would allow automatization of these procedural steps. Here we aimed to evaluate the feasibility, usability and acceptability of a scanning innovation called Smart Paper Technology for digital data processing. We used a mixed-methods design to understand users' engagement with Smart Paper Technology and identify potential positive and negative effects of this innovation in three health facilities in Southern Tanzania. Eight focus group discussions and 11 in-depth interviews with users were conducted. We quantified time used by health care providers for documentation and patient care using time-motion methods. Thematic analysis was used to analyze qualitative data. Descriptive statistics and multivariable linear models were generated to compare the difference before and after introduction and adjust for confounders. Health care providers and health care managers appreciated the forms' simple design features and perceived Smart Paper Technology as time-saving and easy to use. The time-motion study with 273.3 and 224.0 hours of observations before and after introduction of Smart Paper Technology, respectively, confirmed that working time spent on documentation did not increase (27.0% at baseline and 26.4% post-introduction; adjusted p = 0.763). Time spent on patient care was not negatively impacted (26.9% at baseline and 37.1% at post-intervention; adjusted p = 0.001). Health care providers described positive effects on their accountability for data and service provision relating to the fact that individually signed forms were filled. Health care providers perceived Smart Paper Technology as feasible, easy to integrate and acceptable in their setting, particularly as it did not add time to documentation.

Improving disease surveillance data analysis, interpretation, and use at the district level in Tanzania

An effective disease surveillance system is critical for early detection and response to disease epidemics. This study aimed to assess the capacity to manage and utilize disease surveillance data and implement an intervention to improve data analysis and use at the district level in Tanzania. Mapping, in-depth interview and desk review were employed for data collection in Ilala and Kinondoni districts in Tanzania. Interviews were conducted with members of the council health management teams (CHMT) to assess attitudes, motivation and practices related to surveillance data analysis and use. Based on identified gaps, an intervention package was developed on basic data analysis, interpretation and use. The effectiveness of the intervention package was assessed using pre-and post-intervention tests. Individual interviews involved 21 CHMT members (females = 10; males = 11) with an overall median age of 44.5 years (IQR = 37, 53). Over half of the participants regarded their data analytical capacities and skills as excellent. Analytical capacity was higher in Kinondoni (61%) than Ilala (52%). Agreement on the availability of the opportunities to enhance capacity and skills was reported by 68% and 91% of the participants from Ilala and Kinondoni, respectively. Reported challenges in disease surveillance included data incompleteness and difficulties in storage and accessibility. Training related to enhancement of data management was reported to be infrequently done. In terms of data interpretation and use, despite reporting of incidence of viral haemorrhagic fevers for five years, no actions were taken to either investigate or mitigate, indicating poor use of surveillance data in monitoring disease occurrence. The overall percentage increase on surveillance knowledge between pre-and post-training was 37.6% for Ilala and 20.4% for Kinondoni indicating a positive impact on of the training. Most of CHMT members had limited skills and practices on data analysis, interpretation and use. The training in data analysis and interpretation significantly improved skills of the participants.

Assessment of health staff's proficiency and quality of key malaria indicators in rural district of Ghana

BACKGROUND

Routine Health Information Systems (RHIS) are important for not just sure enough control of malaria, but its elimination as well. If these systems are working, they can extensively provide accurate data on reported malaria cases instead of presenting modelled approximations of malaria burden. Queries are raised on both the quality and use of generated malaria data. Some issues of concern include inaccurate reporting of malaria cases as well as treatment plans, wrongly categorizing malaria cases in registers used to collate data and misplacing data or registers for reporting. This study analyses data quality concerning health staff's proficiency, timeliness, availability and data accuracy in the Sissala East Municipal Health Directorate (MHD).

METHODS

A cross-sectional design was used to collect data from 15 facilities and 50 health staff members who offered clinical related care for malaria cases in the Sissala East MHD from 24th August 2020 to 17th September 2020. Fifteen health facilities were randomly selected from the 56 health facilities in the municipality that were implementing the malarial control programme, and they were included in the study.

RESULTS

On the question of when did staff receive any training on malaria-related health information management in the past six months prior to the survey, as minimal as 13 out of 50(26%) claimed to have been trained, whereas the majority 37 out of 50 (74%) had no training. In terms of proficiency in malaria indicators (MI), the majority (68% - 82%) of the respondents could not demonstrate the correct calculations of the indicators. Nevertheless, the MHD recorded monthly average timeliness of the 5th day [range: 4.7-5.7] within the reporting year. However, the MHD had a worse average performance of 5.4th and 5.7th days in July and September respectively. Furthermore, results indicated that 14 out of 15(93.3%) facilities exceeded the target to accomplish report availability (> = 90%) and data completeness (> = 90%). However, the verification factor (VF) of the overall malaria indicator showed that the MHD neither over-reported nor under-reported actual cases, with the corresponding level of data quality as Good (+/-5%).

CONCLUSIONS

The Majority of staff had not received any training on malaria-related RHIS. Some staff members did not know the correct definitions of some of MI used in the malaria programme, while the majority of them could not demonstrate the correct calculations of MI. Timeliness of reporting was below the target, nevertheless, copies of data that were submitted were available and completed. There should be training, supervision and monitoring to enhance staff proficiency and improve the quality of MI.

Routine use of DHIS2 data: a scoping review

Background

In regard to health service planning and delivery, the use of information at different levels in the health system is vital, ranging from the influencing of policy to the programming of action to the ensuring of evidence-informed practices. However, neither ownership of, nor access to, good quality data guarantees actual use of these data. For information to be used, relevant data need to be collected, processed and analysed in an accessible format. This problem of underused data, and indeed the absence of data use entirely, is widespread and has been evident for decades.

The DHIS2 software platform supports routine health management for an estimated 2.4 billion people, in over 70 countries worldwide. It is by far the largest and most widespread software for this purpose and adopts a holistic, socio-technical approach to development and implementation. Given this approach, and the rapid and extensive scaling of DHIS2, we questioned whether or not there has been a parallel increase in the scaling of improved information use. To date, there has been no rigorous review of the documentation on how exactly DHIS2 data is routinely being used for decision-making and subsequent programming of action. This scoping review addresses this review gap.

Methods

The five-stage approach of Arksey and O’Malley progressed by Levac et al. and Peters was followed. Three databases (PubMed, Web of Science and Embase) were searched, along with relevant conference proceedings and postgraduate theses. In total, over 500 documents were reviewed and data from 19 documents were extracted.

Results

Overall, DHIS2 data are being used but there are few detailed descriptions of this usage in peer reviewed or grey literature. We find that, commonly, there exists a centralised versus decentralised pattern of use in terms of access to data and the reporting of data ‘up’ in the system. We also find that the different conceptualisations of data use and how data use is conceptualised are not made explicit.

Conclusions

We conclude with some suggestions for a way forward, namely: i) the need to document in more detail and share how data are being used, ii) the need to investigate how data were created and who uses such data, iii) the need to design systems based on work practices, and in tandem develop and promote forums in which ‘conversations’ around data can take place.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08598-8.

Barriers and Facilitators to Data Use for Decision Making: The Experience of the African Health Initiative Partnerships in Ethiopia, Ghana, and Mozambique

BACKGROUND

Three African Health Initiative (AHI) partnership projects in Ethiopia, Ghana, and Mozambique implemented strategies to improve the quality and evaluation of routinely collected data at the primary health care level and stimulate its use in evidence-based decision making. We compare how these programs designed and carried out data for decision-making (DDM) strategies, elaborate on barriers and facilitators to implementation success, and offer recommendations for future DDM programming.

METHODS

Researchers from each project collaboratively wrote a cross-country protocol based on these objectives. By adapting the Consolidated Framework for Implementation Research (CFIR) through a qualitative theme reduction process, they harmonized lines of inquiry on the design of the respective DDM strategies and the barriers and facilitators of effective implementation. We conducted in-depth interviews and focus group discussions with stakeholders from the primary health care systems in each country, and we carried out multistage, thematic analyses using a deductive lens.

RESULTS

Effective implementation of DDM depended on whether implementers felt that DDM was adaptable to context, feasible to trial, and easy to introduce and maintain. The prevailing policy and political environment in the wider health system, learning climate and absorptive capacity for evidence-based change in DDM settings, engagement of external change agents and internal change leaders, and promotion of opportunities and means for team-based reflection and evaluations of what works influenced the success or failure of DDM strategies.

CONCLUSION

Opportunities for team-based capacity building and individual mentorship led to effective DDM programming. External policies and associated incentives bolstered this but occasionally led to unintended consequences. Leadership engagement and availability of resources to act on recommendations; respond to capacity-building needs; and facilitate collaborations between peers, within hierarchies, and across the local health system proved crucial to DDM, as was encouraging adaptation and opportunities for iterative on-the-job learning.

The Effectiveness of the Capacity Building and Mentorship Program in Improving Evidence-Based Decision-making in the Amhara Region, Northwest Ethiopia: Difference-in-Differences Study

Background

Weak health information systems (HISs) hobble countries’ abilities to effectively manage and distribute their resources to match the burden of disease. The Capacity Building and Mentorship Program (CBMP) was implemented in select districts of the Amhara region of Ethiopia to improve HIS performance; however, evidence about the effectiveness of the intervention was meager.

Objective

This study aimed to determine the effectiveness of routine health information use for evidence-based decision-making among health facility and department heads in the Amhara region, Northwest Ethiopia.

Methods

The study was conducted in 10 districts of the Amhara region: five were in the intervention group and five were in the comparison group. We employed a quasi-experimental study design in the form of a pretest-posttest comparison group. Data were collected from June to July 2020 from the heads of departments and facilities in 36 intervention and 43 comparison facilities. The sample size was calculated using the double population formula, and we recruited 172 participants from each group. We applied a difference-in-differences analysis approach to determine the effectiveness of the intervention. Heterogeneity of program effect among subgroups was assessed using a triple differences method (ie, difference-in-difference-in-differences [DIDID] method). Thus, the β coefficients, 95% CIs, and P values were calculated for each parameter, and we determined that the program was effective if the interaction term was significant at P<.05.

Results

Data were collected using the endpoint survey from 155 out of 172 (90.1%) participants in the intervention group and 166 out of 172 (96.5%) participants in the comparison group. The average level of information use for the comparison group was 37.3% (95% CI 31.1%-43.6%) at baseline and 43.7% (95% CI 37.9%-49.5%) at study endpoint. The average level of information use for the intervention group was 52.2% (95% CI 46.2%-58.3%) at baseline and 75.8% (95% CI 71.6%-80.0%) at study endpoint. The study indicated that the net program change over time was 17% (95% CI 5%-28%; P=.003). The subgroup analysis also indicated that location showed significant program effect heterogeneity, with a DIDID estimate equal to 0.16 (95% CI 0.026-0.29; P=.02). However, sex, age, educational level, salary, and experience did not show significant heterogeneity in program effect, with DIDID estimates of 0.046 (95% CI –0.089 to 0.182), –0.002 (95% CI –0.015 to 0.009), –0.055 (95% CI –0.190 to 0.079), –1.63 (95% CI –5.22 to 1.95), and –0.006 (95% CI –0.017 to 0.005), respectively.

Conclusions

The CBMP was effective at enhancing the capacity of study participants in using the routine HIS for decision-making. We noted that urban facilities had benefited more than their counterparts. The intervention has been shown to produce positive outcomes and should be scaled up to be used in other districts. Moreover, the mentorship modalities for rural facilities should be redesigned to maximize the benefits.

Trial Registration

Pan African Clinical Trials Registry PACTR202001559723931; https://tinyurl.com/3j7e5ka5

Utilisation of health management information and its determinant factors among health professionals working at public health facilities in North Wollo Zone, Northeast Ethiopia: a cross-sectional study

OBJECTIVE

The study aimed to assess health management information utilisation and associated factors among health professionals working at public health facilities in North Wollo Zone, Northeast Ethiopia.

SETTING

The study was conducted at public health facilities in the North Wollo Zone, Northeast Ethiopia.

PARTICIPANTS

A total of 664 (56.3% male and 43.7% female) health professionals participated in the study. All health professionals permanently working in North Wollo Zone were included in this study. However, health professionals who were not present during the data collection period by any means and who had less than 6 months of experience were not included in this study.

PRIMARY AND SECONDARY OUTCOME MEASURES

The main outcome measure was health management information utilisation.

RESULT

About 58.4% (n=388) (95% CI: 54.4% to 62.0%) of the study participants use health management information. The multivariable logistic regression model indicated that participants who had managerial positions are more likely to use health management information with an adjusted OR (AOR) of 3.11 and 95% CI 1.84 to 5.24. Similarly, having a good motivation level (AOR=4.42 (95% CI: 2.82 to 6.93)), perceived good culture of health information (AOR=6.17 (95% CI: 3.35 to 11.36)), a standard set of indicators (AOR=4.11 (95% CI: 2.65 to 6.38)), having good governance of health information system (AOR=1.75 (95% CI:1.13 to 2.72)) and health management information system (HMIS) training (AOR=3.10 (95% CI: 1.89 to 5.07)) were the predictors positively associated with higher utilisation of health management information.

CONCLUSION

This study revealed that utilisation of health management information was still inadequate. Enhancing motivation, building a culture of information use, having standardised indicators, strengthening the governance of health information systems and comprehensive HMIS training were measures to be taken to improve utilisation of health management information in this study setting.

Use of a district health information system 2 routine immunization dashboard for immunization program monitoring and decision making, Kano State, Nigeria

Introduction

a district health information system 2 tool with a customized routine immunization (RI) module and indicator dashboard was introduced in Kano State, Nigeria, in November 2014 to improve data management and analysis of RI services. We assessed the use of the module for program monitoring and decision-making, as well as the enabling factors and barriers to data collection and use.

Methods

a mixed-methods approach was used to assess user experience with the RI data module and dashboard, including 1) a semi-structured survey questionnaire administered at 60 health facilities administering vaccinations and 2) focus group discussions and 16 in-depth interviews conducted with immunization program staff members at the local government area (LGA) and state levels.

Results

in health facilities, a RI monitoring chart was used to review progress toward meeting vaccination coverage targets. At the LGA, staff members used RI dashboard data to prioritize health facilities for additional support. At the State level, immunization program staff members use RI data to make policy decisions. They viewed the provision of real-time data through the RI dashboard as a "game changer". Use of immunization data is facilitated through review meetings and supportive supervision visits. Barriers to data use among LGA staff members included inadequate understanding of the data collection tools and computer illiteracy.

Conclusion

the routine immunization data dashboard facilitated access to and use of data for decision-making at the LGA, State and national levels, however, use at the health facility level remains limited. Ongoing data review meetings and training on computer skills and data collection tools are recommended.

Pilot implementation of a routine immunization module of the district health information system version 2 in Kano State, Nigeria, 2014 - 2015

Introduction

Timely and accurate data are necessary for informing sound decision-making and developing effective routine immunization (RI) programs. We launched a pilot project in Kano State to strengthen routine immunization (RI) data reporting through the immunization module of the District Health Information System version 2 (DHIS2). We examined the completeness and timeliness of reporting monthly RI data one year before and one year after DHIS2 module pilot in the State.

Methods

The first phase of the DHIS2 RI module pilot in Kano included training on RI data tools in November 2014 and in January 2015 for 36 state and zonal personnels, 276 local government area (LGA) personnel, and 2,423 health facility (HF) staff. A RI-focused dashboard to display core RI accountability framework indicators, such as completeness and timeliness of reporting, planned immunization sessions conducted, coverage and dropout was implemented. Report completeness was ratio of submitted reports to number of health facilities while report timeliness was ratio of reports on the DHIS2 by 14th of the month to number of expected.

Results

Completeness of data reporting increase from 70% in 2014 to 87% in 2015, while timeliness of reporting increase from 64% to 87% over the same period. Challenges encountered during the implementation process included limited access to internet, power outages, health workers strike, staff attrition and competing state activities.

Conclusion

The pilot implementation of the DHIS2 immunization module in Kano State led to modest improvement in the reporting of RI services. Several lessons learned were used to guide scale-up to other states in the country.

Assessment of quality of routine health information system data and associated factors among departments in public health facilities of Harari region, Ethiopia

Background

Despite the improvements in the knowledge and understanding of the role of health information in the global health system, the quality of data generated by a routine health information system is still very poor in low and middle-income countries. There is a paucity of studies as to what determines data quality in health facilities in the study area. Therefore, this study was aimed to assess the quality of routine health information system data and associated factors in public health facilities of Harari region, Ethiopia.

Methods

A cross-sectional study was conducted in all public health facilities in the Harari region of Ethiopia. The department-level data were collected from respective department heads through document reviews, interviews, and observation checklists. Descriptive statistics were used to data quality and multivariate logistic regression was run to identify factors influencing data quality. The level of significance was declared at P value < 0.05.

Result

The study found good quality data in 51.35% (95% CI 44.6–58.1) of the departments in public health facilities in the Harari Region. Departments found in the health centers were 2.5 times more likely to have good quality data as compared to those found in the health posts. The presence of trained staffs able to fill reporting formats (AOR = 2.474; 95% CI 1.124–5.445) and provisions of feedbacks (AOR = 3.083; 95% CI 1.549–6.135) were also significantly associated with data quality.

Conclusion

The level of good data quality in the public health facilities was less than the expected national level. Lack of trained personnel able to fill the reporting format and feedback were the factors that are found to be affecting data quality. Therefore, training should be provided to increase the knowledge and skills of the health workers. Regular supportive supervision and feedback should also be maintained.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01651-2.

Measuring coverage of maternal and child health services using routine health facility data: a Sierra Leone case study

BACKGROUND

There are limited existing approaches to generate estimates from Routine Health Information Systems (RHIS) data, despite the growing interest to these data. We calculated and assessed the consistency of maternal and child health service coverage estimates from RHIS data, using census-based and health service-based denominators in Sierra Leone.

METHODS

We used Sierra Leone 2016 RHIS data to calculate coverage of first antenatal care contact (ANC1), institutional delivery and diphtheria-pertussis-tetanus 3 (DPT3) immunization service provision. For each indicator, national and district level coverages were calculated using denominators derived from two census-based and three health service-based methods. We compared the coverage estimates from RHIS data to estimates from MICS 2017. We considered the agreement adequate when estimates from RHIS fell within the 95% confidence interval of the survey estimate.

RESULTS

We found an overall poor consistency of the coverage estimates calculated from the census-based methods. ANC1 and institutional delivery coverage estimates from these methods were greater than 100% in about half of the fourteen districts, and only 3 of the 14 districts had estimates consistent with the survey data. Health service-based methods generated better estimates. For institutional delivery coverage, five districts met the agreement criteria using BCG service-based method. We found better agreement for DPT3 coverage estimates using DPT1 service-based method as national coverage was close to survey data, and estimates were consistent for 8 out of 14 districts. DPT3 estimates were consistent in almost half of the districts (6/14) using ANC1 service-based method.

CONCLUSION

The study highlighted the challenge in determining an appropriate denominator for RHIS-based coverage estimates. Systematic and transparent data quality check and correction, as well as rigorous approaches to determining denominators are key considerations to generate accurate coverage statistics using RHIS data.

The burden of recording and reporting health data in primary health care facilities in five low- and lower-middle income countries

Background

Recording and reporting health data in facilities is the backbone of routine health information systems which provide data collected by health facility workers during service provision. Data is firstly collected in a register, to record patient health data and care process, and tallied into nationally designed reporting forms. While there is anecdotal evidence of large numbers of registers and reporting forms for primary health care (PHC) facilities, there are few systematic studies to document this potential burden on health workers. This multi-country study aimed to document the numbers of registers and reporting forms use at the PHC level and to estimate the time it requires for health workers to meet data demands.

Methods

In Cambodia, Ghana, Mozambique, Nigeria and Tanzania, a desk review was conducted to document registers and reporting forms mandated at the PHC level. In each country, visits to 16 randomly selected public PHC facilities followed to assess the time spent on paper-based recording and reporting. Information was collected through self-reports of estimated time use by health workers, and observation of 1360 provider-patient interactions. Data was primarily collected in outpatient care (OPD), antenatal care (ANC), immunization (EPI), family planning (FP), HIV and Tuberculosis (TB) services.

Result

Cross-countries, the average number of registers was 34 (ranging between 16 and 48). Of those, 77% were verified in use and each register line had at least 20 cells to be completed per patient. The mean time spent on recording was about one-third the total consultation time for OPD, FP, ANC and EPI services combined. Cross-countries, the average number of monthly reporting forms was 35 (ranging between 19 and 52) of which 78% were verified in use. The estimated time to complete monthly reporting forms was 9 h (ranging between 4 to 15 h) per month per health worker.

Conclusions

PHC facilities are mandated to use many registers and reporting forms pausing a considerable burden to health workers. Service delivery systems are expected to vary, however an imperative need remains to invest in international standards of facility-based registers and reporting forms, to ensure regular, comparable, quality-driven facility data collection and use.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06652-5.

Can routine health facility data be used to monitor subnational coverage of maternal, newborn and child health services in Uganda?

Background

Routine health facility data are a critical source of local monitoring of progress and performance at the subnational level. Uganda has been using district health statistics from facility data for many years. We aimed to systematically assess data quality and examine different methods to obtain plausible subnational estimates of coverage for maternal, newborn and child health interventions.

Methods

Annual data from the Uganda routine health facility information system 2015–2019 for all 135 districts were used, as well as national surveys for external comparison and the identification of near-universal coverage interventions. The quality of reported data on antenatal and delivery care and child immunization was assessed through completeness of facility reporting, presence of extreme outliers and internal data consistencies. Adjustments were made when necessary. The denominators for the coverage indicators were derived from population projections and health facility data on near-universal coverage interventions. The coverage results with different denominators were compared with the results from household surveys.

Results

Uganda’s completeness of reporting by facilities was near 100% and extreme outliers were rare. Inconsistencies in reported events, measured by annual fluctuations and between intervention consistency, were common and more among the 135 districts than the 15 subregions. The reported numbers of vaccinations were improbably high compared to the projected population of births or first antenatal visits – and especially so in 2015–2016. There were also inconsistencies between the population projections and the expected target population based on reported numbers of antenatal visits or immunizations. An alternative approach with denominators derived from facility data gave results that were more plausible and more consistent with survey results than based on population projections, although inconsistent results remained for substantive number of subregions and districts.

Conclusion

Our systematic assessment of the quality of routine reports of key events and denominators shows that computation of district health statistics is possible with transparent adjustments and methods, providing a general idea of levels and trends for most districts and subregions, but that improvements in data quality are essential to obtain more accurate monitoring.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06554-6.

Examining policy intentions and actual implementation practices: How organizational factors influence health management information systems in Uttar Pradesh, India

This study investigates the implementation of a recent health management information systems (HMIS) policy reform in Uttar Pradesh, India, which aims to improve the quality and use of HMIS data in decision-making. Through in-depth interviews, meeting observations and a policy document review, this study sought to capture the experiences of district-level staff (street-level bureaucrats) who were responsible for HMIS policy implementation. Findings revealed that issues of weak HMIS implementation were partly due to human resources shortages both in number and technical skill. Delays in recruitment and the presence of inactive staff overburdened existing staff and weakened the implementation of HMIS activities at the block- and district-levels. District staff also explained how inadequate computer literacy and limited technical understanding further contributed to low HMIS data quality. The organizational culture was even more constraining: working within a very rigid and hierarchical organization was challenging for district data staff, who were expected to manage day-to-day HMIS activities, but lacked the discretion and authority to do so effectively. Consequently, they had to escalate minor issues to district leadership for action and were expected to follow their supervisors' directives- even if they contradicted HMIS policy guidelines. High performance pressures associated with achieving top district rankings deviated focus away from HMIS data quality issues. Many district-level respondents described their superiors' "fixation" with becoming a top-ranking district often resulted in disregard for the quality of data informing district rankings. Furthermore, the review of district rankings only partially encouraged district-level leadership to investigate reasons for low-performing indicators. Instead, low district rankings often resulted in punitive action. The study recommends the importance of incorporating the perspectives of district staff, and recognizing their discretion, and authority when designing policy implementation processes, and finally concludes with potential strategies for strengthening the current HMIS policy reform.

Using health data for decision-making at each level of the health system to achieve universal health coverage in Ethiopia: the case of an immunization programme in a low-resource setting

Background

For evidence-based decision-making, there is a need for quality, timely, relevant and accessible information at each level of the health system. Limited use of local data at each level of the health system is reported to be a main challenge for evidence-based decision-making in low- and middle-income countries. Although evidence is available on the timeliness and quality of local data, we know little about how it is used for decision-making at different levels of the health system. Therefore, this study aimed to assess the level of data use and its effect on data quality and shared accountability at different levels of the health system.

Methods

An implementation science study was conducted using key informants and document reviews between January and September 2017. A total of 21 key informants were selected from community representatives, data producers, data users and decision-makers from the community to the regional level. Reviewed documents include facility reports, district reports, zonal reports and feedback in supervision from the district. Thematic content analysis was performed for the qualitative data.

Results

Respondents reported that routine data use for routine decision-making was low. All health facilities and health offices have a performance monitoring team, but these were not always functional. Awareness gaps, lack of motivating incentives, irregularity of supportive supervision, lack of community engagement in health report verification as well as poor technical capacity of health professionals were found to be the major barriers to data use. The study also revealed that there are no institutional or national-level regulations or policies on the accountability mechanisms related to health data. The community-level Health Development Army programme was found to be a strong community engagement approach that can be leveraged for data verification at the source of community data.

Conclusion

The culture of using routine data for decision-making at the local level was found to be low. Strengthening the capacity of health workers and performance monitoring teams, introducing incentive mechanisms for data use, engaging the community in data verification and introducing accountability mechanisms for health data are essential to improve data use and quality.

A realist systematic review of evidence from low- and middle-income countries of interventions to improve immunization data use

BACKGROUND

The use of routine immunization data by health care professionals in low- and middle-income countries remains an underutilized resource in decision-making. Despite the significant resources invested in developing national health information systems, systematic reviews of the effectiveness of data use interventions are lacking. Applying a realist review methodology, this study synthesized evidence of effective interventions for improving data use in decision-making.

METHODS

We searched PubMed, POPLINE, Centre for Agriculture and Biosciences International Global Health, and African Journals Online for published literature. Grey literature was obtained from conference, implementer, and technical agency websites and requested from implementing organizations. Articles were included if they reported on an intervention designed to improve routine data use or reported outcomes related to data use, and targeted health care professionals as the principal data users. We developed a theory of change a priori for how we expect data use interventions to influence data use. Evidence was then synthesized according to data use intervention type and level of the health system targeted by the intervention.

RESULTS

The searches yielded 549 articles, of which 102 met our inclusion criteria, including 49 from peer-reviewed journals and 53 from grey literature. A total of 66 articles reported on immunization data use interventions and 36 articles reported on data use interventions for other health sectors. We categorized 68 articles as research evidence and 34 articles as promising strategies. We identified ten primary intervention categories, including electronic immunization registries, which were the most reported intervention type (n = 14). Among the research evidence from the immunization sector, 32 articles reported intermediate outcomes related to data quality and availability, data analysis, synthesis, interpretation, and review. Seventeen articles reported data-informed decision-making as an intervention outcome, which could be explained by the lack of consensus around how to define and measure data use.

CONCLUSIONS

Few immunization data use interventions have been rigorously studied or evaluated. The review highlights gaps in the evidence base, which future research and better measures for assessing data use should attempt to address.

One country's journey to interoperability: Tanzania's experience developing and implementing a national health information exchange

BACKGROUND

Robust, flexible, and integrated health information (HIS) systems are essential to achieving national and international goals in health and development. Such systems are still uncommon in most low and middle income countries. This article describes a first-phase activity in Tanzania to integrate the country's vertical health management information system with the help of an interoperability layer that enables cross-program data exchange.

METHODS

From 2014 to 2019, the Tanzanian government and partners implemented a five-step procedure based on the "Mind the GAPS" (governance, architecture, program management, and standards) framework and using both proprietary and open-source tools. In collaboration with multiple stakeholders, the team developed the system to address major data challenges via four fully documented "use case scenarios" addressing data exchange among hospitals, between services and the supply chain, across digital data systems, and within the supply chain reporting system. This work included developing the architecture for health system data exchange, putting a middleware interoperability layer in place to facilitate the exchange, and training to support use of the system and the data it generates.

RESULTS

Tanzania successfully completed the five-step procedure for all four use cases. Data exchange is currently enabled among 15 separate information systems, and has resulted in improved data availability and significant time savings. The government has adopted the health information exchange within the national strategy for health care information, and the system is being operated and managed by Tanzanian officials.

CONCLUSION

Developing an integrated HIS requires a significant time investment; but ultimately benefit both programs and patients. Tanzania's experience may interest countries that are developing their HIS programs.

Barriers and enablers to routine register data collection for newborns and mothers: EN-BIRTH multi-country validation study

BACKGROUND

Policymakers need regular high-quality coverage data on care around the time of birth to accelerate progress for ending preventable maternal and newborn deaths and stillbirths. With increasing facility births, routine Health Management Information System (HMIS) data have potential to track coverage. Identifying barriers and enablers faced by frontline health workers recording HMIS source data in registers is important to improve data for use.

METHODS

The EN-BIRTH study was a mixed-methods observational study in five hospitals in Bangladesh, Nepal and Tanzania to assess measurement validity for selected Every Newborn coverage indicators. We described data elements required in labour ward registers to track these indicators. To evaluate barriers and enablers for correct recording of data in registers, we designed three interview tools: a) semi-structured in-depth interview (IDI) guide b) semi-structured focus group discussion (FGD) guide, and c) checklist assessing care-to-documentation. We interviewed two groups of respondents (January 2018-March 2019): hospital nurse-midwives and doctors who fill ward registers after birth (n = 40 IDI and n = 5 FGD); and data collectors (n = 65). Qualitative data were analysed thematically by categorising pre-identified codes. Common emerging themes of barriers or enablers across all five hospitals were identified relating to three conceptual framework categories.

RESULTS

Similar themes emerged as both barriers and enablers. First, register design was recognised as crucial, yet perceived as complex, and not always standardised for necessary data elements. Second, register filling was performed by over-stretched nurse-midwives with variable training, limited supervision, and availability of logistical resources. Documentation complexity across parallel documents was time-consuming and delayed because of low staff numbers. Complete data were valued more than correct data. Third, use of register data included clinical handover and monthly reporting, but little feedback was given from data users.

CONCLUSION

Health workers invest major time recording register data for maternal and newborn core health indicators. Improving data quality requires standardised register designs streamlined to capture only necessary data elements. Consistent implementation processes are also needed. Two-way feedback between HMIS levels is critical to improve performance and accurately track progress towards agreed health goals.

Data quality of the routine health management information system at the primary healthcare facility and district levels in Tanzania

BACKGROUND

Effective planning for disease prevention and control requires accurate, adequately-analysed, interpreted and communicated data. In recent years, efforts have been put in strengthening health management information systems (HMIS) in Sub-Saharan Africa to improve data accessibility to decision-makers. This study assessed the quality of routine HMIS data at primary healthcare facility (HF) and district levels in Tanzania.

METHODS

This cross-sectional study involved reviews of documents, information systems and databases, and collection of primary data from facility-level registers, tally sheets and monthly summary reports. Thirty-four indicators from Outpatient, Inpatient, Antenatal care, Family Planning, Post-natal care, Labour and Delivery, and Provider-Initiated Testing and Counselling service areas were assessed. Indicator records were tracked and compared across the process of data collection, compilation and submission to the district office. Copies of monthly report forms submitted by facilities to the district were also reviewed. The availability and utilization of HMIS tools were assessed, while completeness and data accuracy levels were quantified for each phase of the reporting system.

RESULTS

A total of 115 HFs (including hospitals, health centres, dispensaries) in 11 districts were involved. Registers (availability rate = 91.1%; interquartile range (IQR) 66.7-100%) and report forms (86.9%; IQR 62.2-100%) were the most utilized tools. There was a limited use of tally-sheets (77.8%; IQR 35.6-100%). Tools availability at the dispensary was 91.1%, health centre 82.2% and hospital 77.8%, and was low in urban districts. The availability rate at the district level was 65% (IQR 48-75%). Wrongly filled or empty cells in registers and poor adherence to the coding procedures were observed. Reports were highly over-represented in comparison to registers' records, with large differences observed at the HF phase of the reporting system. The OPD and IPD areas indicated the highest levels of mismatch between data source and district office. Indicators with large number of clients, multiple variables, disease categorization, or those linked with dispensing medicine performed poorly.

CONCLUSION

There are high variations in the tool utilisation and data accuracy at facility and district levels. The routine HMIS is weak and data at district level inaccurately reflects what is available at the source. These results highlight the need to design tailored and inter-service strategies for improving data quality.

Assessment of the validity of the measurement of newborn and maternal health-care coverage in hospitals (EN-BIRTH): an observational study

BACKGROUND

Progress in reducing maternal and neonatal deaths and stillbirths is impeded by data gaps, especially regarding coverage and quality of care in hospitals. We aimed to assess the validity of indicators of maternal and newborn health-care coverage around the time of birth in survey data and routine facility register data.

METHODS

Every Newborn-BIRTH Indicators Research Tracking in Hospitals was an observational study in five hospitals in Bangladesh, Nepal, and Tanzania. We included women and their newborn babies who consented on admission to hospital. Exclusion critiera at admission were no fetal heartbeat heard or imminent birth. For coverage of uterotonics to prevent post-partum haemorrhage, early initiation of breastfeeding (within 1 h), neonatal bag-mask ventilation, kangaroo mother care (KMC), and antibiotics for clinically defined neonatal infection (sepsis, pneumonia, or meningitis), we collected time-stamped, direct observation or case note verification data as gold standard. We compared data reported via hospital exit surveys and via hospital registers to the gold standard, pooled using random effects meta-analysis. We calculated population-level validity ratios (measured coverage to observed coverage) plus individual-level validity metrics.

FINDINGS

We observed 23 471 births and 840 mother-baby KMC pairs, and verified the case notes of 1015 admitted newborn babies regarding antibiotic treatment. Exit-survey-reported coverage for KMC was 99·9% (95% CI 98·3-100) compared with observed coverage of 100% (99·9-100), but exit surveys underestimated coverage for uterotonics (84·7% [79·1-89·5]) vs 99·4% [98·7-99·8] observed), bag-mask ventilation (0·8% [0·4-1·4]) vs 4·4% [1·9-8·1]), and antibiotics for neonatal infection (74·7% [55·3-90·1] vs 96·4% [94·0-98·6] observed). Early breastfeeding coverage was overestimated in exit surveys (53·2% [39·4-66·8) vs 10·9% [3·8-21·0] observed). "Don't know" responses concerning clinical interventions were more common in the exit survey after caesarean birth. Register data underestimated coverage of uterotonics (77·9% [37·8-99·5] vs 99·2% [98·6-99·7] observed), bag-mask ventilation (4·3% [2·1-7·3] vs 5·1% [2·0-9·6] observed), KMC (92·9% [84·2-98·5] vs 100% [99·9-100] observed), and overestimated early breastfeeding (85·9% (58·1-99·6) vs 12·5% [4·6-23·6] observed). Inter-hospital heterogeneity was higher for register-recorded coverage than for exit survey report. Even with the same register design, accuracy varied between hospitals.

INTERPRETATION

Coverage indicators for newborn and maternal health care in exit surveys had low accuracy for specific clinical interventions, except for self-report of KMC, which had high sensitivity after admission to a KMC ward or corner and could be considered for further assessment. Hospital register design and completion are less standardised than surveys, resulting in variable data quality, with good validity for the best performing sites. Because approximately 80% of births worldwide take place in facilities, standardising register design and information systems has the potential to sustainably improve the quality of data on care at birth.

FUNDING

Children's Investment Fund Foundation and Swedish Research Council.

Improving the quality of routine maternal and newborn data captured in primary health facilities in Gombe State, Northeastern Nigeria: a before-and-after study

OBJECTIVES

Primary objective: to assess nine data quality metrics for 14 maternal and newborn health data elements, following implementation of an integrated, district-focused data quality intervention.

SECONDARY OBJECTIVE

to consider whether assessing the data quality metrics beyond completeness and accuracy of facility reporting offered new insight into reviewing routine data quality.

DESIGN

Before-and-after study design.

SETTING

Primary health facilities in Gombe State, Northeastern Nigeria.

PARTICIPANTS

Monitoring and evaluation officers and maternal, newborn and child health coordinators for state-level and all 11 local government areas (district-equivalent) overseeing 492 primary care facilities offering maternal and newborn care services.

INTERVENTION

Between April 2017 and December 2018, we implemented an integrated data quality intervention which included: introduction of job aids and regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media.

OUTCOME MEASURES

9 metrics for the data quality dimensions of completeness and timeliness, internal consistency of reported data, and external consistency.

RESULTS

The data quality intervention was associated with improvements in seven of nine data quality metrics assessed including availability and timeliness of reporting, completeness of data elements, accuracy of facility reporting, consistency between related data elements, and frequency of outliers reported. Improvement differed by data element type, with content of care and commodity-related data improving more than contact-related data. Increases in the consistency between related data elements demonstrated improved internal consistency within and across facility documentation.

CONCLUSIONS

An integrated district-focused data quality intervention-including regular self-assessment of data quality, peer-review and feedback, learning workshops, work planning for improvement, and ongoing support through social media-can increase the completeness, accuracy and internal consistency of facility-based routine data.

Evaluation of the malaria reporting system supported by the District Health Information System 2 in Solomon Islands

Background

District Health Information Systems 2 (DHIS2) is used for supporting health information management in 67 countries, including Solomon Islands. However, there have been few published evaluations of the performance of DHIS2-enhanced disease reporting systems, in particular for monitoring infectious diseases such as malaria. The aim of this study was to evaluate DHIS2 supported malaria reporting in Solomon Islands and to develop recommendations for improving the system.

Methods

The evaluation was conducted in three administrative areas of Solomon Islands: Honoria City Council, and Malaita and Guadalcanal Provinces. Records of nine malaria indicators including report submission date, total malaria cases, Plasmodium falciparum case record, Plasmodium vivax case record, clinical malaria, malaria diagnosed with microscopy, malaria diagnosed with (rapid diagnostic test) (RDT), record of drug stocks and records of RDT stocks from 1st January to 31st December 2016 were extracted from the DHIS2 database. The indicators permitted assessment in four core areas: availability, completeness, timeliness and reliability. To explore perceptions and point of view of the stakeholders on the performance of the malaria case reporting system, focus group discussions were conducted with health centre nurses, whilst in-depth interviews were conducted with stakeholder representatives from government (province and national) staff and World Health Organization officials who were users of DHIS2.

Results

Data were extracted from nine health centres in Honoria City Council and 64 health centres in Malaita Province. The completeness and timeliness from the two provinces of all nine indicators were 28.2% and 5.1%, respectively. The most reliable indicator in DHIS2 was ‘clinical malaria’ (i.e. numbers of clinically diagnosed malaria cases) with 62.4% reliability. Challenges to completeness were a lack of supervision, limited feedback, high workload, and a lack of training and refresher courses. Health centres located in geographically remote areas, a lack of regular transport, high workload and too many variables in the reporting forms led to delays in timely reporting. Reliability of reports was impacted by a lack of technical professionals such as statisticians and unavailability of tally sheets and reporting forms.

Conclusion

The availability, completeness, timeliness and reliability of nine malaria indicators collected in DHIS2 were variable within the study area, but generally low. Continued onsite support, supervision, feedback and additional enhancements, such as electronic reporting will be required to further improve the malaria reporting system.

Improving quality and use of routine health information system data in low- and middle-income countries: A scoping review

BACKGROUND

A routine health information system is one of the essential components of a health system. Interventions to improve routine health information system data quality and use for decision-making in low- and middle-income countries differ in design, methods, and scope. There have been limited efforts to synthesise the knowledge across the currently available intervention studies. Thus, this scoping review synthesised published results from interventions that aimed at improving data quality and use in routine health information systems in low- and middle-income countries.

METHOD

We included articles on intervention studies that aimed to improve data quality and use within routine health information systems in low- and middle-income countries, published in English from January 2008 to February 2020. We searched the literature in the databases Medline/PubMed, Web of Science, Embase, and Global Health. After a meticulous screening, we identified 20 articles on data quality and 16 on data use. We prepared and presented the results as a narrative.

RESULTS

Most of the studies were from Sub-Saharan Africa and designed as case studies. Interventions enhancing the quality of data targeted health facilities and staff within districts, and district health managers for improved data use. Combinations of technology enhancement along with capacity building activities, and data quality assessment and feedback system were found useful in improving data quality. Interventions facilitating data availability combined with technology enhancement increased the use of data for planning.

CONCLUSION

The studies in this scoping review showed that a combination of interventions, addressing both behavioural and technical factors, improved data quality and use. Interventions addressing organisational factors were non-existent, but these factors were reported to pose challenges to the implementation and performance of reported interventions.

Factors That Influence Data Use to Improve Health Service Delivery in Low- and Middle-Income Countries

BACKGROUND

Health service delivery indicators are designed to reveal how well health services meet a community's needs. Effective use of the data can enable targeted improvements in health service delivery. We conducted a systematic review to identify the factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income settings.

METHODS

We reviewed empirical studies published in 2005 or later that provided evidence on the use of health service delivery data at the primary care level in low- and middle-income countries. We searched Scopus, Medline, the Cochrane Library, and citations of included studies. We also searched the gray literature, using a separate strategy. We extracted information on study design, setting, study population, study objective, key findings, and any identified lessons learned.

RESULTS

Twelve studies met the inclusion criteria. This small number of studies suggests there is insufficient evidence to draw reliable conclusions. However, a content analysis identified the following potentially influential factors, which we classified into 3 categories: governance (leadership, participatory monitoring, regular review of data); production of information (presentation of findings, data quality, qualitative data); and health information system resources (electronic health management information systems, organizational structure, training). Contextual factors and performance-based financing were also each found to have a role; however, discussing these as mediating factors may not be practical in terms of promoting data use.

CONCLUSION

Scant evidence exists regarding factors that influence the use of health service delivery indicators to improve delivery of primary health care services in low- and middle-income countries. However, the existing evidence highlights some factors that may have a role in improving data use. Further research may benefit from comparing data use factors across different types of program indicators or using our classification as a framework for field experiments.

Effect of data quality improvement intervention on health management information system data accuracy: An interrupted time series analysis

Background

As part of a partnership between the Institute for Healthcare Improvement and the Ethiopian Federal Ministry of Health, woreda-based quality improvement collaboratives took place between November 2016 and December 2017 aiming to accelerate reduction of maternal and neonatal mortality in Lemu Bilbilu, Tanqua Abergele and Duguna Fango woredas. Before starting the collaboratives, assessments found inaccuracies in core measures obtained from Health Management Information System reports.

Methods and results

Building on the quality improvement collaborative design, data quality improvement activities were added and we used the World Health Organization review methodology to drive a verification factor for the core measures of number of pregnant women that received their first antenatal care visit, number of pregnant women that received antenatal care on at least four visits, number of pregnant women tested for syphilis and number of births attended by skilled health personnel. Impact of the data quality improvement was assessed using interrupted time series analysis. We found accurate data across all time periods for Tanqua Abergele. In Lemu Bilbilu and Duguna Fango, data quality improved for all core metrics over time. In Duguna Fango, the verification factor for number of pregnant women that received their first antenatal care visit improved from 0.794 (95%CI 0.753, 0.836; p<0.001) pre-intervention by 0.173 (95%CI 0.128, 0.219; p<0.001) during the collaborative; and the verification factor for number of pregnant women tested for syphilis improved from 0.472 (95%CI 0.390, 0.554; p<0.001) pre-intervention by 0.460 (95%CI 0.369, 0.552; p<0.001) during the collaborative. In Lemu Bilbilu, the verification factor for number of pregnant women receiving a fourth antenatal visit rose from 0.589 (95%CI 0.513, 0.664; p<0.001) at baseline by 0.358 (95%CI 0.258, 0.458; p<0.001) post-intervention; and skilled birth attendance rose from 0.917 (95%CI 0.869, 0.965) at baseline by 0.083 (95%CI 0.030, 0.136; p<0.001) during the collaborative.

Conclusions

A Data quality improvement initiative embedded within woreda clinical improvement collaborative improved accuracy of data used to monitor maternal and newborn health services in Ethiopia.

Labour and delivery ward register data availability, quality, and utility - Every Newborn - birth indicators research tracking in hospitals (EN-BIRTH) study baseline analysis in three countries

BACKGROUND

Countries with the highest burden of maternal and newborn deaths and stillbirths often have little information on these deaths. Since over 81% of births worldwide now occur in facilities, using routine facility data could reduce this data gap. We assessed the availability, quality, and utility of routine labour and delivery ward register data in five hospitals in Bangladesh, Nepal, and Tanzania. This paper forms the baseline register assessment for the Every Newborn-Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study.

METHODS

We extracted 21 data elements from routine hospital labour ward registers, useful to calculate selected maternal and newborn health (MNH) indicators. The study sites were five public hospitals during a one-year period (2016-17). We measured 1) availability: completeness of data elements by register design, 2) data quality: implausibility, internal consistency, and heaping of birthweight and explored 3) utility by calculating selected MNH indicators using the available data.

RESULTS

Data were extracted for 20,075 births. Register design was different between the five hospitals with 10-17 of the 21 selected MNH data elements available. More data were available for health outcomes than interventions. Nearly all available data elements were > 95% complete in four of the five hospitals and implausible values were rare. Data elements captured in specific columns were 85.2% highly complete compared to 25.0% captured in non-specific columns. Birthweight data were less complete for stillbirths than live births at two hospitals, and significant heaping was found in all sites, especially at 2500g and 3000g. All five hospitals recorded count data required to calculate impact indicators including; stillbirth rate, low birthweight rate, Caesarean section rate, and mortality rates.

CONCLUSIONS

Data needed to calculate MNH indicators are mostly available and highly complete in EN-BIRTH study hospital routine labour ward registers in Bangladesh, Nepal and Tanzania. Register designs need to include interventions for coverage measurement. There is potential to improve data quality if Health Management Information Systems utilization with feedback loops can be strengthened. Routine health facility data could contribute to reduce the coverage and impact data gap around the time of birth.

A learning agenda to build the evidence base for strengthening global health information systems

BACKGROUND

Evidence-based interventions are necessary for planning and investing in health information systems (HIS) and for strengthening those systems to collect, manage, sort and analyse health data to support informed decision-making. However, evidence and guidance on HIS strengthening in low- and middle-income countries have been historically lacking.

OBJECTIVE

This article describes the approach, methods, lessons learned and recommendations from 5 years of applying our learning agenda to strengthen the evidence base for effective HIS interventions.

METHODS

The first step was to define key questions about characteristics, stages of progression, and factors and conditions of HIS performance progress. We established a team and larger advisory group to guide the implementation of activities to build the evidence base to answer questions. We strengthened learning networks to share information.

RESULTS

The process of applying the learning agenda provided a unique opportunity to learn by doing, strategically collecting information about monitoring and evaluating HIS strengthening interventions and building a body of evidence. There are now models and tools to strengthen HIS, improved indicators and measures, country HIS profiles, documentation of interventions, a searchable database of HIS assessment tools and evidence generated through syntheses and evaluation results.

CONCLUSION

The systematic application of learning agenda processes and activities resulted in increased evidence, information, guidance and tools for HIS strengthening and a resource centre, making that information accessible and available globally.

IMPLICATIONS

We describe the inputs, processes and lessons learned, so that others interested in designing a successful learning agenda have access to evidence of how to do so.

Understanding the challenges associated with the use of data from routine health information systems in low- and middle-income countries: A systematic review

BACKGROUND

Routine health information systems (RHISs) are crucial to informing decision-making at all levels of the health system. However, the use of RHIS data in low- and middle-income countries (LMICs) is limited due to concerns regarding quality, accuracy, timeliness, completeness and representativeness.

OBJECTIVE

This study systematically reviewed technical, behavioural and organisational/environmental challenges that hinder the use of RHIS data in LMICs and strategies implemented to overcome these challenges.

METHOD

Four electronic databases were searched for studies describing challenges associated with the use of RHIS data and/or strategies implemented to circumvent these challenges in LMICs. Identified articles were screened against inclusion and exclusion criteria by two independent reviewers.

RESULTS

Sixty studies met the inclusion criteria and were included in this review, 55 of which described challenges in using RHIS data and 20 of which focused on strategies to address these challenges. Identified challenges and strategies were organised by their technical, behavioural and organisational/environmental determinants and by the core steps of the data process. Organisational/environmental challenges were the most commonly reported barriers to data use, while technical challenges were the most commonly addressed with strategies.

CONCLUSION

Despite the known benefits of RHIS data for health system strengthening, numerous challenges continue to impede their use in practice.

IMPLICATIONS

Additional research is needed to identify effective strategies for addressing the determinants of RHIS use, particularly given the disconnect identified between the type of challenge most commonly described in the literature and the type of challenge most commonly targeted for interventions.

Evaluation of quality and use of health management information system in primary health care units of east Wollega zone, Oromia regional state, Ethiopia

Background

Health care practice relies on evidence-based decisions and needs the use of quality health care data. Health management information system (HMIS) is among the core elements of health system building blocks. In our study setting, there is a lack of adequate information on the quality of health information data. This study aimed at exploring the quality of health management information system data in terms of timeliness, completeness, and accuracy. The specific objectives were to evaluate the quality and use of the health management information system in Primary health care units of East Wollega zone, Ethiopia.

Methods

A cross-sectional study was conducted from April to June 2016 on 316 health professionals/health information technicians. The sample was obtained by simple random sampling technique. Qualitative data were obtained from 16 purposefully selected key informants by Focus group discussion (FGD). We observed 50 selected health facilities using an observation checklist. We analyzed quantitative data by SPSS version 20 using descriptive and logistic regression analysis techniques. we applied a thematic analysis approach to analyze qualitative data.

Results

Timeliness of report, registration completeness, report completeness, and data accuracy level of the selected facilities were 70, 78.2, 86, and 48%, respectively. All results are below the expected national standards. Commonly reported reasons for the poor practice of data quality were; poor support of management, lack of accountability for the false report, poor supportive supervision, and lack of separate and responsible unit for health information management.

Conclusion

The Health information management system is poorly coordinated at the primary health units. Accountability should be assured through continuous in-service training, supportive supervision, and concrete feedbacks. Electronic management of health information should be available in primary health care units.

Factors limiting data quality in the expanded programme on immunization in low and middle-income countries: A scoping review

Few public health interventions can match the immense achievements of immunization in terms of mortality and morbidity reduction. However, progress in reaching global coverage goals and achieving universal immunization coverage have stalled; with key stakeholders concerned about the accuracy of reported coverage figures. Incomplete and incorrect data has made it challenging to obtain an accurate overview of immunization coverage, particularly in low- and middle-income countries (LMIC). To date, only one literature review concerning immunization data quality exists. However, it only included articles from Gavi-eligible countries, did not go deep into the characteristics of the data quality problems, and used a narrow 'data quality' definition. This scoping review builds upon that work; exploring the "state of data quality" in LMIC, factors affecting data quality in these settings and potential means to improve it. Only a small volume of literature addressing immunization data quality in LMIC was found and definitions of 'data quality' varied widely. Data quality was, on the whole, considered poor in the articles included. Coverage numerators were seen to be inflated for official reports and denominators were inaccurate and infrequently adjusted. Numerous factors related to these deficiencies were reported, including health information system fragmentation, overreliance on targets and poor data management processes. Factors associated with health workers were noted most frequently. Authors suggested that data quality could be improved by ensuring proper data collection tools, increasing workers' capacities and motivation through training and supervision, whilst also ensuring adequate and timely feedback on the data collected. The findings of this scoping review can serve as the basis to identify and address barriers to good quality immunization data in LMICs. Overcoming said barriers is essential if immunization's historic successes are to continue.

A Qualitative Study to Examine Approaches used to Manage Data about Health Facilities and their Challenges: A Case of Uganda

Availability of an accurate and complete health facility list is fundamental in producing quality and timely data that is sufficient to aid evidence-based decision, resource allocation and planning within the healthcare ecosystem. This study aimed at examining the approaches used in Uganda to manage data about health facilities and the challenges they are facing. We conducted a qualitative study involving 32 interviews with participants from Ministry of Health, government regulatory organizations, district local government, general public, academia, implementing partners and healthcare providers. Our analysis identified four divergent approaches that had five common challenges, namely; lack of a health facility unique identifier, non-standardized, incomplete, inaccurate data, difficulty accessing and using data. Establishing a national central health facility registry to manage the national health facility list would improve patient referrals, facility look-ups, health information exchange, data curation and access and health information system integration.

Generating statistics from health facility data: the state of routine health information systems in Eastern and Southern Africa

Health facility data are a critical source of local and continuous health statistics. Countries have introduced web-based information systems that facilitate data management, analysis, use and visualisation of health facility data. Working with teams of Ministry of Health and country public health institutions analysts from 14 countries in Eastern and Southern Africa, we explored data quality using national-level and subnational-level (mostly district) data for the period 2013-2017. The focus was on endline analysis where reported health facility and other data are compiled, assessed and adjusted for data quality, primarily to inform planning and assessments of progress and performance. The analyses showed that although completeness of reporting was generally high, there were persistent data quality issues that were common across the 14 countries, especially at the subnational level. These included the presence of extreme outliers, lack of consistency of the reported data over time and between indicators (such as vaccination and antenatal care), and challenges related to projected target populations, which are used as denominators in the computation of coverage statistics. Continuous efforts to improve recording and reporting of events by health facilities, systematic examination and reporting of data quality issues, feedback and communication mechanisms between programme managers, care providers and data officers, and transparent corrections and adjustments will be critical to improve the quality of health statistics generated from health facility data.

Optimizing data quality of pharmaceutical information systems in public health care in resource limited settings

BACKGROUND

Robust pharmaceutical management information systems (PMIS) strengthen healthcare planning and delivery. Few nationwide studies in resource limited settings in Africa validate the data quality of PMIS in public healthcare.

OBJECTIVE

To determine predictors and quality of data in a nationwide PMIS database in Namibia.

METHODS

A population-level analysis of the quality of data i.e. completeness, accuracy and consistency in a nationwide PMIS database, 2007-2015. Data quality of the PMIS was determined by three domains, completeness, accuracy and consistency. Data completeness was determined by level of missing data in SPSSv25, with acceptable level set at <5%. Data accuracy was determined by proportion of PMIS indicators with extreme outliers. Data consistence was determined by patterns of missingness, i.e. random or systematic. Predictors of data quality were determined using logistic regression modelling.

RESULTS

A total of 544 entries and 12 indicators were registered in the PMIS at 38 public health facilities. All the PMIS indicators had missing data and 50% (n = 6) had inaccurate data i.e. extreme values. The data for most PMIS indicators (75%, n = 12) were consistent with the pattern of missing completely at random (MCAR, i.e. missingness <5%). Incompleteness of PMIS data was highest for average number of prescriptions 6%, annual expenditure per capita for pharmaceuticals 5% and population per pharmacist's assistant 5%. The main predictors of poor quality of PMIS data were year of reporting of PMIS data (p = 0.035), level of health facility (p < 0.001), vital reference materials available at the pharmacy (p = 0.002), and pharmacists' posts filled (p = 0.013).

CONCLUSIONS

The data quality of PMIS in public health care in Namibia is sub-optimal and widely varies by reporting period, level of health facility and region. The integration of data quality assurance systems is required to strengthen quality of PMIS data to optimize quality of PMIS data in public health care.

Identification of the essential components of quality in the data collection process for public health information systems

This study identifies essential components in the data collection process for public health information systems based on appraisal and synthesis of the reported factors affecting this process in the literature. Extant process assessment instruments and studies of public health data collection from electronic databases and the relevant institutional websites were reviewed and analyzed following a five-stage framework. Four dimensions covering 12 factors and 149 indicators were identified. The first dimension, data collection management, includes data collection system and quality assurance. The second dimension, data collector, is described by staffing pattern, skill or competence, communication and attitude toward data collection. The third, information system, is assessed by function and technology support, integration of different data collection systems, and device. The fourth dimension, data collection environment, comprises training, leadership, and funding. With empirical testing and contextual analysis, these essential components can be further used to develop a framework for measuring the quality of the data collection process for public health information systems.

Facilitators, best practices and barriers to integrating family planning data in Uganda's health management information system

Background

Health management information systems (HMIS) are instrumental in addressing health delivery problems and strengthening health sectors by generating credible evidence about the health status of clients. There is paucity of studies which have explored possibilities for integrating family planning data from the public and private health sectors in Uganda’s national HMIS. This study sought to investigate the facilitators, best practices and barriers of integrating family planning data into the district and national HMIS in Uganda.

Methods

We conducted a qualitative study in Kampala, Jinja, and Hoima Districts of Uganda, based on 16 key informant interviews and a multi-stakeholder dialogue workshop with 11 participants. Deductive and inductive thematic methods were used to analyze the data.

Results

The technical facilitators of integrating family planning data from public and private facilities in the national and district HMIS were user-friendly software; web-based and integrated reporting; and availability of resources, including computers. Organizational facilitators included prioritizing family planning data; training staff; supportive supervision; and quarterly performance review meetings. Key behavioral facilitators were motivation and competence of staff. Collaborative networks with implementing partners were also found to be essential for improving performance and sustainability.

Significant technical barriers included limited supply of computers in lower level health facilities, complex forms, double and therefore tedious entry of data, and web-reporting challenges. Organizational barriers included limited human resources; high levels of staff attrition in private facilities; inadequate training in data collection and use; poor culture of information use; and frequent stock outs of paper-based forms. Behavioral barriers were low use of family planning data for planning purposes by district and health facility staff.

Conclusion

Family planning data collection and reporting are integrated in Uganda’s district and national HMIS. Best practices included integrated reporting and performance review, among others. Limited priority and attention is given to family planning data collection at the facility and national levels. Data are not used by the health facilities that collect them. We recommend reviewing and tailoring data collection forms and ensuring their availability at health facilities. All staff involved in data reporting should be trained and regularly supervised.

Electronic supplementary material

The online version of this article (10.1186/s12913-019-4151-9) contains supplementary material, which is available to authorized users.

Hospital mortality statistics in Tanzania: availability, accessibility, and quality 2006-2015

Background

Accurate and reliable hospital information on the pattern and causes of death is important to monitor and evaluate the effectiveness of health policies and programs. The objective of this study was to assess the availability, accessibility, and quality of hospital mortality data in Tanzania.

Methods

This cross-sectional study involved selected hospitals of Tanzania and was carried out from July to October 2016. Review of hospital death registers and forms was carried out to cover a period of 10 years (2006–2015). Interviews with hospital staff were conducted to seek information as regards to tools used to record mortality data, staff involved in recording and availability of data storage and archiving facilities.

Results

A total of 247,976 death records were reviewed. The death register was the most (92.3%) common source of mortality data. Other sources included the International Classification of Diseases (ICD) report forms, Inpatient registers, and hospital administrative reports. Death registers were available throughout the 10-year period while ICD-10 forms were available for the period of 2013–2015. In the years between 2006 and 2010 and 2011–2015, the use of death register increased from 82 to 94.9%. Three years after the introduction of ICD-10 procedure, the forms were available and used in 28% (11/39) hospitals. The level of acceptable data increased from 69% in 2006 to 97% in 2015. Inconsistency in the language used, use of non-standard nomenclature for causes of death, use of abbreviations, poorly and unreadable handwriting, and missing variables were common data quality challenges. About 6.3% (n = 15,719) of the records had no patient age, 3.5% (n = 8790) had no cause of death and ~ 1% had no sex indicated. The frequency of missing sex variable was most common among under-5 children. Data storage and archiving in most hospitals was generally poor. Registers and forms were stored in several different locations, making accessibility difficult.

Conclusion

Overall, this study demonstrates gaps in hospital mortality data availability, accessibility, and quality, and highlights the need for capacity strengthening in data management and periodic record reviews. Policy guidelines on the data management including archiving are necessary to improve data.

Assessing primary care data quality

Purpose The purpose of this paper is to assess National Medical Care Survey data quality. Design/methodology/approach Data completeness and representativeness were computed for all observations while other data quality measures were assessed using a 10 per cent sample from the National Medical Care Survey database; i.e., 12,569 primary care records from 189 public and private practices were included in the analysis. Findings Data field completion ranged from 69 to 100 per cent. Error rates for data transfer from paper to web-based application varied between 0.5 and 6.1 per cent. Error rates arising from diagnosis and clinical process coding were higher than medication coding. Data fields that involved free text entry were more prone to errors than those involving selection from menus. The authors found that completeness, accuracy, coding reliability and representativeness were generally good, while data timeliness needs to be improved. Research limitations/implications Only data entered into a web-based application were examined. Data omissions and errors in the original questionnaires were not covered. Practical implications Results from this study provided informative and practicable approaches to improve primary health care data completeness and accuracy especially in developing nations where resources are limited. Originality/value Primary care data quality studies in developing nations are limited. Understanding errors and missing data enables researchers and health service administrators to prevent quality-related problems in primary care data.

Understanding health worker data use in a South African antiretroviral therapy register

OBJECTIVE

To evaluate how electronic data management systems affect data use practices in antiretroviral therapy (ART) programs within local health districts, and individual health facilities.

METHODS

We used a data quality audit to establish a baseline of the quality of data in the electronic register alongside in-depth interviews with health workers and managers, to understand perceptions of data quality, data use by facility staff and challenges affecting data use.

RESULTS

The findings provide a four-level continuum of data use that can be applied to other settings and recommendations for optimising facility-level data use.

CONCLUSION

By defining four levels of data use our findings suggest the potential to encourage a structured process of moving from passive data use, to more active and engaged data use, where data could be used to anticipate patient behaviour and link that behaviour to differentiated care plans.

Can training of health care workers improve data management practice in health management information systems: a case study of primary health care facilities in Kaduna State, Nigeria

Introduction

Reliable and accurate public health information is essential for monitoring, evaluating and improving the delivery of healthcare services. The objective of this study was to assess the effect of training health care workers on data management practice in health management information systems in primary health care (PHC) centers in Kaduna state.

Methods

The study was quasi-experimental with baseline, intervention and end point components. It was carried out in two local government areas, a study and a control. Eleven PHC facilities were selected in each LGA. The intervention was carried out among 76 PHC workers in the study LGA. Data were collected using a health facility checklist and a focused group discussion (FGD) guide. Data analysis was done using SPSS version 20.0 and statistical significance of the difference between baseline and end-line data were determined using chi-square or fisher's exact test where applicable at p < 0.05.

Results

There was a statistically significant increase in completeness of reporting (p = 0.02), overall accuracy rate (p < 0.001), timeliness rate of reporting (p = <0.001) and feedback (p = 0.012). No improvement was found in the control group. During the baseline FGDs, PHC workers in both study and control LGAs expressed difficulty in filling registers/forms, data analysis and use of data. At end point, those in the study LGA said their practice had improved but those in the control LGA still expressed difficulty in data management.

Conclusion

Health management information system training achieved an improvement in the data management practice of PHC workers. In-service training and re-training should be done to improve data management practice of health workers.

Ending preventable maternal mortality: phase II of a multi-step process to develop a monitoring framework, 2016-2030

BACKGROUND

In February 2015, the World Health Organization (WHO) released "Strategies toward ending preventable maternal mortality (EPMM)" (EPMM Strategies), a direction-setting report outlining global targets and strategies for reducing maternal mortality in the Sustainable Development Goal (SDG) period. In May 2015, the EPMM Working Group outlined a plan to develop a comprehensive monitoring framework to track progress toward the achievement of these targets and priorities. This monitoring framework was developed in two phases. Phase I, which focused on identifying indicators related to the proximal causes of maternal mortality, was completed in October 2015. This paper describes the process and results of Phase II, which was completed in November 2016 and aimed to build consensus on a set of indicators that capture information on the social, political, and economic determinants of maternal health and mortality.

FINDINGS

A total of 150 experts from more than 78 organizations worldwide participated in this second phase of the process to develop a comprehensive monitoring framework for EPMM. The experts considered a total of 118 indicators grouped into the 11 key themes outlined in the EPMM report, ultimately reaching consensus on a set of 25 indicators, five equity stratifiers, and one transparency stratifier.

CONCLUSION

The indicators identified in Phase II will be used along with the Phase I indicators to monitor progress towards ending preventable maternal deaths. Together, they provide a means for monitoring not only the essential clinical interventions needed to save lives but also the equally important political, social, economic and health system determinants of maternal health and survival. These distal factors are essential to creating the enabling environment and high-performing health systems needed to ensure high-quality clinical care at the point of service for every woman, her fetus and newborn. They complement and support other monitoring efforts, in particular the "Survive, Thrive, and Transform" agenda laid out by the Global Strategy for Women's, Children's and Adolescents' Health (2016-2030) and the SDG3 global target on maternal mortality.

Data-informed decision-making for life-saving commodities investments in Malawi: A qualitative case study

Background

During the last 15 years, Malawi has made remarkable progress in reducing child mortality. However, maternal and newborn mortality remains persistently high. To help address these entrenched challenges, the Reproductive, Maternal, Newborn and Child Health (RMNCH) Trust Fund provided short-term catalytic financing of $11.5 million (2013-2016) to support country plans to advance the RMNCH and commodity agenda.

Objectives

(1) To document how Malawi (ministries, partners, working groups) used evidence to inform decision-making and RMNCH investments, (2) To identify barriers to utilizing information and evidence in the planning and prioritization process at national and sub-national levels, and (3) To assess the utility of the RMNCH Landscape Synthesis, which uses existing information to review life-saving RMNCH commodities and services.

Methods

This was a qualitative case study utilizing a Rapid Appraisal (RA) approach, where semi-structured interviews were conducted with staff members from UN agencies, development partners and the Ministry of Health (MoH) at national and district level. The analysis enlists a framework approach for manual qualitative content analysis.

Results

Led by the MoH, the RMNCH Trust Fund grant proposal utilized an evidence-based and equity-focused process for prioritization of investments. Data-informed decision-making permeates similar commodity-focused working groups. However, common health information system (HIS) weaknesses, such as data quality and collection burden, persist and are more prevalent at district-level. The collation of evidence in the RMNCH Landscape Synthesis was a useful and sustainable tool to support planning.

Conclusions

The evidence-based, equity-focused decision-making process for the RMNCH Trust Fund proposal provides an effective model for inter-agency investment prioritization. Strengthening data-informed decision-making will require financial and political commitments to HIS and capacity building for data use, particularly at the district-level. New initiatives (e.g. Health Data Collaborative and QED Network to Improve Quality of Care) provide opportunities to further improve evidence-informed decision-making.

How can the use of data within the immunisation programme be increased in order to improve data quality and ensure greater accountability in the health system? A protocol for implementation science study

BACKGROUND

Immunisation remains one of the most important and cost-effective interventions to reduce vaccine-preventable child morbidity, disability and mortality. Health programmes like the Expanded Program of Immunization rely on complex decision-making and strong local level evidence is important to effectively and efficiently utilise limited resources. Lack of data use for decision-making at each level of the health system remains the main challenge in most developing countries. While there is much evidence on data quality and how to improve it, there is a lack of sufficient evidence on why the use of data for decision-making at each level of the health system is low. Herein, we describe a comprehensive implementation science study that will be conducted to identify organisational, technical and individual level factors affecting local data use at each level of the Ethiopian health system.

METHODS

We will apply a mixed methods approach using key informant interviews and document reviews. The qualitative data will be gathered through key informant interviews using a semi-structured guide with open- and closed-ended questions with four categories of respondents, namely decision-makers, data producers, data users and community representatives at the federal, regional, zonal, woreda and community levels of the health system. The document review will be conducted on selected reports and feedback documented at different levels of the health system. Data will be collected from July 2017 to March 2018. Descriptive statistics will be analysed for the quantitative study using SPSS version 20 software and thematic content analysis will be performed for the qualitative part using NVivo software.

DISCUSSION

Appropriate and timely use of health and health-related information for decision-making is an essential element in the process of transforming the health sector. The findings of the study will inform stakeholders at different levels on the institutionalisation of evidence-based practice in immunisation programmes.

"Every day they keep adding new tools but they don't take any away": Producing indicators for intermittent preventive treatment for malaria in pregnancy (IPTp) from routine data in Kenya

BACKGROUND

Intermittent preventive treatment for malaria in pregnancy (IPTp) is part of a multi-pronged strategy aimed at preventing malaria in pregnancy in areas of moderate to high transmission in sub-Saharan Africa. Despite being formally adopted as a malaria prevention policy over a decade ago, IPTp coverage has remained low. Recent demands for action have incorporated calls to strengthen IPTp monitoring and evaluation systems, including the use of routine data, to measure coverage, track implementation and identify roadblocks to improving uptake. Concerns about the quality of malaria indicators reported through routine information systems are well recognized, but there are few data on the realities of IPTp recording practices in frontline facilities or their entry into District Health Information Software (DHIS2).

METHODS

Drawing on fieldwork conducted in two malaria endemic sub-counties in Kenya, we explore how local adaptations and innovations employed by health workers and sub-country managers to cope with a range of health system constraints, shape recording practices and in turn, the measurement of IPTp. Data were collected through observations, interviews, and document reviews. Data analysis and interpretation was guided by thematic analysis approach.

RESULTS

Measurement of IPTp was undermined by health system constraints such as stock-out of drugs and human resource shortages. Coping strategies adopted by health workers to address these challenges ensured continuity in service delivery and IPTp data generation but had variable consequences on IPTp data quality. Unclear recording and reporting instructions also led to lack of standardization in IPTp data generation. The use of redundant tools created significant data burdens which undermined service delivery in general.

CONCLUSIONS

There is need to integrate monthly reporting forms so as to remove redundancies which exacerbates workload for health workers and disrupts service delivery. Similarly, data collection instructions in registers and reporting forms need to be clarified to standardize IPTp data generation across health facilities. There is also need to address broader contextual factors such as stock-out of commodities and human resource shortages which undermine IPTp data generation process.