"Real-Time" Monitoring of Under-Five Mortality: A Vision Tempered by Reality

An evaluation of truncated birth histories for the rapid measurement of fertility and child survival

BACKGROUND

Full birth histories (FBHs) are a key tool for estimating fertility and child mortality in low- and middle-income countries, but they are lengthy to collect. This is not desirable, especially for rapid turnaround surveys that ought to be short (e.g., mobile phone surveys). To reduce the length of the interview, some surveys resort to truncated birth histories (TBHs), where questions are asked only on recent births.

METHODS

We used 32 Malaria Indicator Surveys that included TBHs from 18 countries in sub-Saharan Africa. Each set of TBHs was paired and compared to an overlapping set of FBHs (typically from a standard Demographic and Health Survey). We conducted a variety of data checks, including a comparison of the proportion of children reported in the reference period and a comparison of the fertility and mortality estimates.

RESULTS

Fertility and mortality estimates from TBHs are lower than those based on FBHs. These differences are driven by the omission of events and the displacement of births backward and out of the reference period.

CONCLUSIONS

TBHs are prone to misreporting errors that will bias both fertility and mortality estimates. While we find a few significant associations between outcomes measured and interviewer's characteristics, data quality markers correlate more consistently with respondent attributes, suggesting that truncation creates confusion among mothers being interviewed. Rigorous data quality checks should be put in place when collecting data through this instrument in future surveys.

Comparing two data collection methods to track vital events in maternal and child health via community health workers in rural Nepal

BACKGROUND

Timely tracking of health outcomes is difficult in low- and middle-income countries without comprehensive vital registration systems. Community health workers (CHWs) are increasingly collecting vital events data while delivering routine care in low-resource settings. It is necessary, however, to assess whether routine programmatic data collected by CHWs are sufficiently reliable for timely monitoring and evaluation of health interventions. To study this, we assessed the consistency of vital events data recorded by CHWs using two methodologies-routine data collected while delivering an integrated maternal and child health intervention, and data from a birth history census approach at the same site in rural Nepal.

METHODS

We linked individual records from routine programmatic data from June 2017 to May 2018 with those from census data, both collected by CHWs at the same site using a mobile platform. We categorized each vital event over a one-year period as 'recorded by both methods,' 'census alone,' or 'programmatic alone.' We further assessed whether vital events data recorded by both methods were classified consistently.

RESULTS

From June 2017 to May 2018, we identified a total of 713 unique births collectively from the census (birth history) and programmatic maternal 'post-delivery' data. Three-fourths of these births (n = 526) were identified by both. There was high consistency in birth location classification among the 526 births identified by both methods. Upon including additional programmatic 'child registry' data, we identified 746 total births, of which 572 births were identified by both census and programmatic methods. Programmatic data (maternal 'post-delivery' and 'child registry' combined) captured more births than census data (723 vs. 595). Both methods consistently classified most infants as 'living,' while infant deaths and stillbirths were largely classified inconsistently or recorded by only one method. Programmatic data identified five infant deaths and five stillbirths not recorded in census data.

CONCLUSIONS

Our findings suggest that data collected by CHWs from routinely tracking pregnancies, births, and deaths are promising for timely program monitoring and evaluation. Despite some limitations, programmatic data may be more sensitive in detecting vital events than cross-sectional census surveys asking women to recall these events.

Testing survey-based methods for rapid monitoring of child mortality, with implications for summary birth history data

INTRODUCTION

Under-five mortality estimates are increasingly used in low and middle income countries to target interventions and measure performance against global development goals. Two new methods to rapidly estimate under-5 mortality based on Summary Birth Histories (SBH) were described in a previous paper and tested with data available. This analysis tests the methods using data appropriate to each method from 5 countries that lack vital registration systems. SBH data are collected across many countries through censuses and surveys, and indirect methods often rely upon their quality to estimate mortality rates.

METHODS AND FINDINGS

The Birth History Imputation method imputes data from a recent Full Birth History (FBH) onto the birth, death and age distribution of the SBH to produce estimates based on the resulting distribution of child mortality. DHS FBHs and MICS SBHs are used for all five countries. In the implementation, 43 of 70 estimates are within 20% of validation estimates (61%). Mean Absolute Relative Error is 17.7.%. 1 of 7 countries produces acceptable estimates. The Cohort Change method considers the differences in births and deaths between repeated Summary Birth Histories at 1 or 2-year intervals to estimate the mortality rate in that period. SBHs are taken from Brazil's PNAD Surveys 2004-2011 and validated against IGME estimates. 2 of 10 estimates are within 10% of validation estimates. Mean absolute relative error is greater than 100%.

CONCLUSIONS

Appropriate testing of these new methods demonstrates that they do not produce sufficiently good estimates based on the data available. We conclude this is due to the poor quality of most SBH data included in the study. This has wider implications for the next round of censuses and future household surveys across many low- and middle- income countries.

Understanding Misclassification between Neonatal Deaths and Stillbirths: Empirical Evidence from Malawi

Improving the counting of stillbirths and neonatal deaths is important to tracking Sustainable Development Goal 3.2 and improving vital statistics in low- and middle-income countries (LMICs). However, the validity of self-reported stillbirths and neonatal deaths in surveys is often threatened by misclassification errors between the two birth outcomes. We assessed the extent and correlates of stillbirths being misclassified as neonatal deaths by comparing two recent and linked population surveys conducted in Malawi, one being a full birth history (FBH) survey, and the other a follow-up verbal/social autopsy (VASA) survey. We found that one-fifth of 365 neonatal deaths identified in the FBH survey were classified as stillbirths in the VASA survey. Neonatal deaths with signs of movements in the last few days before delivery reported were less likely to be misclassified stillbirths (OR = 0.08, p<0.05). Having signs of birth injury was found to be associated with higher odds of misclassification (OR = 6.17, p<0.05). We recommend replicating our study with larger sample size in other settings. Additionally, we recommend conducting validation studies to confirm accuracy and completeness of live births and neonatal deaths reported in household surveys with events reported in a full birth history and the extent of underestimation of neonatal mortality resulting from misclassifications. Questions on fetal movement, signs of life at delivery and improved probing among older mother may be useful to improve accuracy of reported events.

Can Community Health Workers Report Accurately on Births and Deaths? Results of Field Assessments in Ethiopia, Malawi and Mali

Introduction

Most low-income countries lack complete and accurate vital registration systems. As a result, measures of under-five mortality rates rely mostly on household surveys. In collaboration with partners in Ethiopia, Ghana, Malawi, and Mali, we assessed the completeness and accuracy of reporting of births and deaths by community-based health workers, and the accuracy of annualized under-five mortality rate estimates derived from these data. Here we report on results from Ethiopia, Malawi and Mali.

Method

In all three countries, community health workers (CHWs) were trained, equipped and supported to report pregnancies, births and deaths within defined geographic areas over a period of at least fifteen months. In-country institutions collected these data every month. At each study site, we administered a full birth history (FBH) or full pregnancy history (FPH), to women of reproductive age via a census of households in Mali and via household surveys in Ethiopia and Malawi. Using these FBHs/FPHs as a validation data source, we assessed the completeness of the counts of births and deaths and the accuracy of under-five, infant, and neonatal mortality rates from the community-based method against the retrospective FBH/FPH for rolling twelve-month periods. For each method we calculated total cost, average annual cost per 1,000 population, and average cost per vital event reported.

Results

On average, CHWs submitted monthly vital event reports for over 95 percent of catchment areas in Ethiopia and Malawi, and for 100 percent of catchment areas in Mali. The completeness of vital events reporting by CHWs varied: we estimated that 30%-90% of annualized expected births (i.e. the number of births estimated using a FPH) were documented by CHWs and 22%-91% of annualized expected under-five deaths were documented by CHWs. Resulting annualized under-five mortality rates based on the CHW vital events reporting were, on average, under-estimated by 28% in Ethiopia, 32% in Malawi, and 9% in Mali relative to comparable FPHs. Costs per vital event reported ranged from $21 in Malawi to $149 in Mali.

Discussion

Our findings in Mali suggest that CHWs can collect complete and high-quality vital events data useful for monitoring annual changes in under-five mortality rates. Both the supervision of CHWs in Mali and the rigor of the associated field-based data quality checks were of a high standard, and the size of the pilot area in Mali was small (comprising of approximately 53,205 residents in 4,200 households). Hence, there are remaining questions about whether this level of vital events reporting completeness and data quality could be maintained if the approach was implemented at scale. Our experience in Malawi and Ethiopia suggests that, in some settings, establishing and maintaining the completeness and quality of vital events reporting by CHWs over time is challenging. In this sense, our evaluation in Mali falls closer to that of an efficacy study, whereas our evaluations in Ethiopia and Malawi are more akin to an effectiveness study. Our overall findings suggest that no one-size-fits-all approach will be successful in guaranteeing complete and accurate reporting of vital events by CHWs.

Evaluation of a mHealth Data Quality Intervention to Improve Documentation of Pregnancy Outcomes by Health Surveillance Assistants in Malawi: A Cluster Randomized Trial

BACKGROUND

While community health workers are being recognized as an integral work force with growing responsibilities, increased demands can potentially affect motivation and performance. The ubiquity of mobile phones, even in hard-to-reach communities, has facilitated the pursuit of novel approaches to support community health workers beyond traditional modes of supervision, job aids, in-service training, and material compensation. We tested whether supportive short message services (SMS) could improve reporting of pregnancies and pregnancy outcomes among community health workers (Health Surveillance Assistants, or HSAs) in Malawi.

METHODS AND FINDINGS

We designed a set of one-way SMS that were sent to HSAs on a regular basis during a 12-month period. We tested the effectiveness of the cluster-randomized intervention in improving the complete documentation of a pregnancy. We defined complete documentation as a pregnancy for which a specific outcome was recorded. HSAs in the treatment group received motivational and data quality SMS. HSAs in the control group received only motivational SMS. During baseline and intervention periods, we matched reported pregnancies to reported outcomes to determine if reporting of matched pregnancies differed between groups and by period. The trial is registered as ISCTRN24785657.

CONCLUSIONS

Study results show that the mHealth intervention improved the documentation of matched pregnancies in both the treatment (OR 1.31, 95% CI: 1.10-1.55, p<0.01) and control (OR 1.46, 95% CI: 1.11-1.91, p = 0.01) groups relative to the baseline period, despite differences in SMS content between groups. The results should be interpreted with caution given that the study was underpowered. We did not find a statistically significant difference in matched pregnancy documentation between groups during the intervention period (OR 0.94, 95% CI: 0.63-1.38, p = 0.74). mHealth applications have the potential to improve the tracking and data quality of pregnancies and pregnancy outcomes, particularly in low-resource settings.

"Real-Time" Monitoring of Under-Five Mortality: Lessons for Strengthened Vital Statistics Systems

Bryce and colleagues, reflect on lessons that can be learned from the Real-Time Monitoring of Under-Five Mortality Collection.

Monitoring Change in Child Mortality through Household Surveys

BACKGROUND

Most low- and middle-income countries lack fully functional civil registration systems. Measures of under-five mortality are typically derived from periodic household surveys collecting detailed information from women on births and child deaths. However, such surveys are expensive and are not appropriate for monitoring short-term changes in child mortality. We explored and tested the validity of two new analysis methods for less-expensive summary histories of births and child deaths for such monitoring in five African countries.

METHODS AND FINDINGS

The first method we explored uses individual-level survey data on births and child deaths to impute full birth histories from an earlier survey onto summary histories from a more recent survey. The second method uses cohort changes between two surveys in the average number of children born and the number of children dead by single year of age to estimate under-five mortality for the inter-survey period. The first method produces acceptable annual estimates of under-five mortality for two out of six applications to available data sets; the second method produced an acceptable estimate in only one of five applications, though none of the applications used ideal data sets.

CONCLUSIONS

The methods we tested were not able to produce consistently good quality estimates of annual under-five mortality from summary birth history data. The key problem we identified was not with the methods themselves, but with the underlying quality of the summary birth histories. If summary birth histories are to be included in general household surveys, considerable emphasis must be placed on quality control.