Depends Who's Asking: Interviewer Effects in Demographic and Health Surveys Abortion Data

Interviewer Effects on the Measurement of Physical Performance in a Cross-National Biosocial Survey

Biosocial surveys increasingly use interviewers to collect objective physical health measures (or "biomeasures") in respondents' homes. While interviewers play an important role, their high involvement can lead to unintended interviewer effects on the collected measurements. Such interviewer effects add uncertainty to population estimates and have the potential to lead to erroneous inferences. This study examines interviewer effects on the measurement of physical performance in a cross-national and longitudinal setting using data from the Survey of Health, Ageing and Retirement in Europe. The analyzed biomeasures exhibited moderate-to-large interviewer effects on the measurements, which varied across biomeasure types and across countries. Our findings demonstrate the necessity to better understand the origin of interviewer-related measurement errors in biomeasure collection and account for these errors in statistical analyses of biomeasure data.

Tanzanian adolescents' attitudes toward abortion: innovating video vignettes in survey research on health topics

BACKGROUND

The purpose of this study was to pilot an innovative cartoon video vignette survey methodology to learn about young people's perspectives on abortion and sexual relationships in Tanzania. The Animating Children's Views methodology used videos shown on tablets to engage young people in conversations. Such conversations are complicated because abortion is highly stigmatized, inaccessible, and illegal in Tanzania.

METHODS

The cartoon video vignette methodology was conducted as a part of a quantitative survey using tablet computers. Hypothetical situations and euphemistic expressions were tested in order to engage adolescents on sensitive topics in low-risk ways. Qualitative interviews and focus groups validated and further explored the perspectives of the young respondents.

RESULTS

Results indicate that 12-17 year-olds usually understand euphemistic expressions for abortion and are aware of social stigma and contradictory norms surrounding abortion from as young as age twelve. Despite the risks involved with abortion, this study finds adolescents sometimes view abortion as a reasonable solution to allow a girl to remain in school. Additional findings show that as adolescents wrestle with how to respond to a schoolgirl's pregnancy, they are considering both the (un)affordability of healthcare services and also expectations for gender roles.

CONCLUSIONS

Digital data collection, such as the Animating Children's Views cartoon video vignettes used in this study, allows researchers to better understand girls' and boys' own perspectives on their experiences and reproductive health.

Increases in child marriage among the poorest in Mali: 'Reverse policies' or data quality issues?

Child marriage is associated with adverse outcomes related to women's well-being. Many countries have introduced laws banning this practice, and a number of studies have evaluated their impact. Scant research has focused on instances where countries have lowered the legal minimum age at marriage, even though such 'reverse policies' could result in stalled or uneven progress in eradicating child marriage. Using visualization techniques, regression analyses, and multiple robustness checks, we document changes in the prevalence of child marriage in Mali, where in 2011 the general minimum age at marriage of 18 was lowered to 16. Since 2011, the prevalence of child marriage has progressively increased among women with no education and women living in communities characterized by low local development. We reflect on the role that data collection processes may play in explaining some of these findings and stress how repealing existing provisions aiming to protect girls can have adverse consequences on the most vulnerable social strata.

Gone and Forgotten? Predictors of Birth History Omissions in India

Background

Fertility histories are subject to measurement errors such as incorrect birth dates, incorrect birth orders, incorrect sex, and omissions. These errors can bias demographic estimates such as fertility rates and child mortality rates.

Objective

We focus on births missing in fertility histories. We estimate the prevalence of such omissions and study their associated factors.

Methods

We leverage a panel survey (the India Human Development Survey) where the same women were interviewed in two waves several years apart. We compare data across waves and identify omitted births. Omissions in the second wave are modeled as a function of several child, mother, household, and survey interviewer variables. Models are fit separately to omissions reported alive or dead in the first wave.

Results

We conservatively estimate the prevalence of omissions at 4%. A large majority of omitted births are those of dead children, especially infants, with children in poorer households at greater risk of being omitted. For children alive in wave 1, female children are much more likely to be omitted in wave 2 compared to male children. Interviewers can detect respondent behaviors associated with omissions.

Conclusions

Omissions in fertility histories are non-ignorable. They do not randomly occur but affect some population sub-groups and some interview contexts more than others.

Contributions

We investigate the understudied but important phenomenon of omitted births in fertility histories. We bring attention to possible biases in demographic estimates. We shed light on the survey process and propose strategies for minimizing the bias through improved survey design.

The effect of interviewer-respondent age difference on the reporting of sexual activity in the Demographic and Health Surveys: Analysis of data from 21 countries

Background

Interviewer effects can have consequential impacts on survey data, particularly for reporting sensitive attitudes and behaviours such as sexual activity and drug use, yet these effects remain understudied in low- and middle-income countries. The Demographic and Health Surveys (DHS) present a unique opportunity to study interviewer effects on the self-report of sensitive topics in low- and middle-income countries by including interviewer characteristics data. This paper aims to narrow the gap in research on interviewer effects by studying the effects that age difference between interviewer and respondent and interviewer survey experience have on the reporting of ever having sexual intercourse.

Methods

We used DHS data from 91 066 women and 56 336 men in 21 countries where the standard DHS was implemented among all women of reproductive age, and interviewer characteristics were included in the data set. Using a Bayesian cross-classified model with random intercepts for interviewer and cluster, we assessed whether the effect of an age difference of 10 years or greater was associated with a difference in self-report of ever having sexual intercourse, adjusting for respondent demographics.

Results

There was a meaningful association between an age difference of greater than ten years and reporting of ever having had sexual intercourse in most countries for both genders after adjusting for interviewer age and experience, rural or urban cluster, and individual-level characteristics. Among women, the marginal posterior probability of reporting ever having sexual intercourse if the interviewer was ten years or more years older was lower for 17 of 19 countries (countries ranged from -12.50 to 3.90 percentage points). Among men, the marginal posterior probability was lower for 16 of 20 countries, ranging from -18.30 to 17.10 percentage points.

Conclusions

In most countries, women and men were less likely to report ever having sexual activity if the interviewer was ten or more years older than them, adjusting for potential confounders. These findings have important implications for interpreting numerous sexual health indicators, such as unmet family planning needs and human immunodeficiency virus (HIV)/acquired immunodeficiency syndrome (AIDS) risk. Survey administrators may consider more careful interviewer-respondent characteristic matching or novel approaches like Audio Computer Assisted Self Interview to minimize interviewer-induced variance.

Randomized experimental testing of new survey approaches to improve abortion reporting in the United States

CONTEXT

Abortions are substantially underreported in surveys due to social stigma, compromising the study of abortion, pregnancy, fertility, and related demographic and health outcomes.

METHODS

In this study, we evaluated six methodological approaches identified through formative mixed-methods research to improve the measurement of abortion in surveys. These approaches included altering the placement of abortion items in the survey, the order of pregnancy outcome questions, the level of detail, the introduction to the abortion question, and the context of the abortion question, and using graduated sensitivity. We embedded a preregistered randomized experiment in a newly designed online survey about sexual and reproductive health behaviors (N = 6536). We randomized respondents to experimental arms in a fully crossed factorial design; we estimated an average treatment effect using standardized estimators from logistic regression models, adjusted for demographic covariates associated with reporting.

RESULTS

None of the experimental arms significantly improved abortion reporting compared to the control condition.

CONCLUSION

More work is needed to improve reporting of abortion in future surveys, particularly as abortion access becomes increasingly restricted in the United States. Despite this study's null results, it provides a promising path for future efforts to improve abortion measurement. It is proof of concept for testing new approaches in a less expensive, faster, and more flexible format than embedding changes in existing national fertility surveys.

Survey implementation process and interviewer effects on skipping sequence of maternal and child health indicators from National Family Health Survey: An application of cross-classified multilevel model

Implementing a large-scale survey involves a string of intricate procedures exposed to numerous types of survey errors. Uniform and systematic training protocols, comprehensive survey manuals, and multilayer supervision during survey implementation help reduce survey errors, providing a consistent fieldwork environment that should not result in any variation in the quality of data collected across interviewers and teams. With this background, the present study attempts to delineate the effect of field investigator (FI) teams and survey implementation design on the selected outcomes. Data on four of the bigger Empowered Action Group (EAG) states of India, namely Uttar Pradesh, Madhya Pradesh, Bihar, and Rajasthan, were obtained from the fourth round of the National Family Health Survey (NFHS-4) for analysis. A fixed-effect binary logistic regression model was used to assess the effect of FI teams and survey implementation design on the selected outcomes. To study the variation in the outcome variables at the interviewer level, a cross-classified multilevel model was used. Since one interviewer had worked in more than one primary sampling unit (PSU) & district and did not follow a perfect hierarchical structure, the cross-classified multilevel model was deemed suitable. In addition, since NFHS-4 used a two-stage stratified sampling design, two-level weights were adjusted for the models to compute unbiased estimates. This study demonstrated the presence of interviewer-level variation in the selected outcomes at both inter- and intra-field agencies across the selected states. The interviewer-level intra-class correlation coefficient (ICC) for women who had not availed antenatal care (ANC) was the highest for eastern Madhya Pradesh (0.23) and central Uttar Pradesh (0.20). For ‘immunisation card not seen’, Rajasthan (0.16) and western Uttar Pradesh (0.13) had higher interviewer-level ICC. Interviewer-level variations were insignificant for women who gave birth at home across all regions of Uttar Pradesh. Eastern Madhya Pradesh, Rajasthan, and Bihar showed higher interviewer-level variation across the selected outcomes, underlining the critical role of agencies and skilled interviewers in different survey implementation designs. The analysis highlights non-uniform adherence to survey protocols, which implies that not all interviewers and agencies performed in a similar manner in the field. This study recommends a refined mechanism for field implementation and supervision, including focused training on the challenges faced by FIs, random vigilance, and morale building. In addition, examining interviewer-level characteristics, field challenges, and field agency effects may also highlight the roots of interviewer-level variation in the data. However, based on the interviewer's performance in the field, the present study offers an intriguing insight into interviewer-level variations in the quality of data.

•

With uniform survey implementation strategies, the interviewer should not have any effect in explaining the maternal and child health variables.

•

Skipping of selected maternal and child health indicators curtails informativeness of the survey. Results confirms that information on vaccination card, antenatal care, maternal policy information, delivery cost and postnatal checkups have been skipped with negative response to opening questions.

•

Cross-classified multilevel model confirms the presence of interviewer-level variation. The interviewer-level intra-class correlation coefficient (ICC) for ‘immunization card not seen’ was found to be highest in Rajasthan and western Uttar Pradesh.

•

Interviewer-level variations were found to be not significant for women who gave birth at home across all the regions of Uttar Pradesh whereas the interviewer-level variations for women who had not availed ANC was found to be highest for eastern Madhya Pradesh central Uttar Pradesh. Results emphasizes that not all interviewers and agencies performed in a similar manner in the field.

•

The study recommends a refined mechanism for field implementation and supervision, including focused training on challenges faced by field investigators, random vigilance, and morale building.

Mapping local variation in household overcrowding across Africa from 2000 to 2018: a modelling study

BACKGROUND

Household overcrowding is a serious public health threat associated with high morbidity and mortality. Rapid population growth and urbanisation contribute to overcrowding and poor sanitation in low-income and middle- income countries, and are risk factors for the spread of infectious diseases, including COVID-19, and antimicrobial resistance. Many countries do not have adequate surveillance capacity to monitor household overcrowding. Geostatistical models are therefore useful tools for estimating household overcrowding. In this study, we aimed to estimate household overcrowding in Africa between 2000 and 2018 by combining available household survey data, population censuses, and other country-specific household surveys within a geostatistical framework.

METHODS

We used data from household surveys and population censuses to generate a Bayesian geostatistical model of household overcrowding in Africa for the 19-year period between 2000 and 2018. Additional sociodemographic and health-related covariates informed the model, which covered 54 African countries.

FINDINGS

We analysed 287 surveys and population censuses, covering 78 695 991 households. Spatial and temporal variability arose in household overcrowding estimates over time. In 2018, the highest overcrowding estimates were observed in the Horn of Africa region (median proportion 62% [IQR 57-63]); the lowest regional median proportion was estimated for the north of Africa region (16% [14-19]). Overall, 474·4 million (95% uncertainty interval [UI] 250·1 million-740·7 million) people were estimated to be living in overcrowded conditions in Africa in 2018, a 62·7% increase from the estimated 291·5 million (180·8 million-417·3 million) people who lived in overcrowded conditions in the year 2000. 48·5% (229·9 million) of people living in overcrowded conditions came from six African countries (Nigeria, Ethiopia, Democratic Republic of the Congo, Sudan, Uganda, and Kenya), with a combined population of 538·3 million people.

INTERPRETATION

This study incorporated survey and population censuses data and used geostatistical modelling to estimate continent-wide overcrowding over a 19-year period. Our analysis identified countries and areas with high numbers of people living in overcrowded conditions, thereby providing a benchmark for policy planning and the implementation of interventions such as in infectious disease control.

FUNDING

UK Department of Health and Social Care, Wellcome Trust, Bill & Melinda Gates Foundation.

How Interviewers Affect Responses to Sensitive Questions on the Justification for Wife Beating, the Refusal to have Conjugal Sex, and Domestic Violence in India

Despite a general understanding that interviewers might cause measurement errors on sensitive questions in sample surveys, there is relatively little research on interviewer effects on responses to questions on women justifying a woman's refusal to have sex with her husband, women justifying wife beating, women's experience of physical and sexual violence, and whether the woman's father ever beat her mother. This study examines interviewer effects on these indicators that were collected in two large-scale National Family Health Surveys (NFHS) in India (2005-2006 and 2015-2016). We use cross-classified random intercept multivariable multilevel logit models to examine interviewer effects. In both surveys, we find large interviewer effects on questions about the justification of a woman refusing to have sex with her husband (32-33% in NFHS-3 and 45-46% in NFHS-4) and the justification of wife beating (27-28% in NFHS-3 and 33-34% in NFHS-4). The interviewer effects were much larger in the 2015-2016 survey than in the 2005-2006 survey. Such large interviewer effects should be considered when interpreting trends and patterns on these topics, especially since the interviewer effects might have changed between survey rounds. Understanding interviewer effects is important given the wide use of these surveys in policy formulation and monitoring in India.

Interviewer effects on abortion reporting: a multilevel analysis of household survey responses in Côte d'Ivoire, Nigeria and Rajasthan, India

OBJECTIVES

The analysis aimed to assess the scale of interviewer effects on abortion survey responses, to compare interviewer effects between different question wordings and between direct and indirect approaches, and to identify interviewer and interview characteristics that explain interviewer effects on abortion reporting.

SETTING

2018 Performance Monitoring for Action nationally representative household surveys from Côte d'Ivoire, Nigeria and Rajasthan, India.

PARTICIPANTS

Survey data from 20 016 interviews with reproductive age (15-49) women, selected using multistage stratified cluster sampling. Data from self-administered interviewer surveys and from a sample of health service delivery points that serve the female survey participants were also included.

PRIMARY OUTCOME MEASURES

Outcomes were the respondent's own experience of ever 'removing a pregnancy', their closest confidante's experience of pregnancy removal and the respondent's own experience of period regulation.

RESULTS

Substantial interviewer effects were observed, ranging from 7% in Côte d'Ivoire to 24% in Nigeria for pregnancy removal. Interviewer effects for survey questions that were designed to ask about abortion in a less stigmatising way were either similar to (9%-26% for confidante-reporting) or higher than (17%-32% for a question about period regulation) the pregnancy removal question. Interviewer and interview characteristics associated with abortion reporting included respondent-interviewer familiarity, the language of interview and the interviewer's comfort asking questions about abortion.

CONCLUSION

This study highlights that questions designed to be less stigmatising may increase interviewer effects due to lower comprehension among respondents. Further work is needed to assess question wordings for different contexts. Selecting and training interviewers to ensure comfort asking questions about abortion is important for reproductive health surveys. Challenges for the use of 'insider' interviewers and the management of surveys in countries with high linguistic diversity are also identified.