Evaluation of Mechanisms to Improve Performance of Mobile Phone Surveys in Low- and Middle-Income Countries: Research Protocol

A nationwide mobile phone survey for tobacco use in Tanzania: Sample quality and representativeness compared to a household survey

We investigated the feasibility of an interactive voice response (IVR) survey in Tanzania and compared its prevalence estimates for tobacco use to the estimates of the 'Global Adult Tobacco Survey (GATS) 2018'. IVR participants were enrolled by random digit dialing. Quota sampling was employed to achieve the required sample sizes of age-sex strata: sex (male/female) and age (18-29-, 30-44-, 45-59-, and ≥60-year-olds). GATS was a nationally representative survey and used a multistage stratified cluster sampling design. The IVR sample's weights were generated using the inverse proportional weighting (IPW) method with a logit model and the standard age-sex distribution of Tanzania. The IVR and GATS had 2362 and 4555 participants, respectively. Compared to GATS, the unweighted IVR sample had a higher proportion of males (58.7 % vs. 43.2 %), educated people (secondary/above education: 43.3 % vs. 21.1 %), and urban residents (56.5 % vs. 40 %). The weighted prevalence (95 % confidence interval (CI)) of current smoking was 4.99 % (4.11-6.04), 5.22 % (4.36-6.24), and 7.36 % (6.51-8.31) among IVR (IPW), IVR (age-sex standard), and GATS samples, respectively; the weighted prevalence (95 % CI) of smokeless tobacco use was similar: 3.54 % (2.73-4.57), 3.58 % (2.80-4.56), and 2.43 % (1.98-2.98), respectively. Most differences in point estimates for tobacco indicators were small (<2%). Overall, the odds of tobacco smoking indicators were lower in IVR than in GATS; however, the odds of smokeless tobacco use were reversed. Although we found under-/over-estimation of the prevalence of tobacco use in IVR than GATS, the estimates were close. Further research is required to increase the representativeness of IVR.

Developing digital tools for health surveys in low- and middle-income countries: Comparing findings of two mobile phone surveys with a nationally representative in-person survey in Bangladesh

Non-communicable disease (NCD) risk factor data from low- and middle-income countries (LMICs) are inadequate, mostly due to the cost and burden of collecting in-person population-level estimates. High-income countries regularly use phone-based surveys, and with increasing mobile phone subscription in developing countries, mobile phone surveys (MPS) could complement in-person surveys in LMICs. We compared the representativeness and prevalence estimates of two MPS (i.e., interactive voice response (IVR) and computer-assisted telephone interview (CATI)) with a nationally representative household survey in Bangladesh-the STEPwise approach to NCD risk factor surveillance (STEPs) 2018. This cross-sectional study included 18-69-year-old respondents. CATI and IVR recruitments were done by random digit dialing, while STEPs used multistage cluster sampling design. The prevalence of NCD risk factors related to tobacco, alcohol, diet, and hypertension was reported and compared by prevalence differences (PD) and prevalence ratios (PR). We included 2355 (57% males), 1942 (62% males), and 8185 (47% males) respondents in the CATI, IVR, and STEPs, respectively. CATI (28%) and IVR (52%) had a higher proportion of secondary/above-educated people than STEPs (13%). Most prevalence estimates differed by survey mode; however, CATI estimates were closer to STEPs than IVR. For instance, in CATI, IVR, and STEPs, respectively, the prevalence was 21.4%, 17.9%, and 23.5% for current smoking; and 1.6%, 2.2%, and 1.5% for alcohol drinking in past month. Compared to STEPs, the PD ranged from '-56.6% to 0.4%' in CATI and '-41.0% to 8.4%' in IVR; the PR ranged from '0.3 to 1.1' in CATI and '0.3 to 1.6' in IVR. There were some differences and some similarities in NCD indicators produced by MPS and STEPs with differences likely due to differences in socioeconomic characteristics between survey participants.

Improving success of non-communicable diseases mobile phone surveys: Results of two randomized trials testing interviewer gender and message valence in Bangladesh and Uganda

INTRODUCTION

Although interactive voice response (IVR) is a promising mobile phone survey (MPS) method for public health data collection in low- and middle-income countries (LMICs), participation rates for this method remain lower than traditional methods. This study tested whether using different introductory messages increases the participation rates of IVR surveys in two LMICs, Bangladesh and Uganda.

METHODS

We conducted two randomized, controlled micro-trials using fully-automated random digit dialing to test the impact of (1) the gender of the speaker recording the survey (i.e., survey voice); and (2) the valence of the invitation to participate in the survey (i.e., survey introduction) on response and cooperation rates. Participants indicated their consent by using the keypad of cellphones. Four study arms were compared: (1) male and informational (MI); (2) female and information (FI); (3) male and motivational (MM); and (4) female and motivational (FM).

RESULTS

Bangladesh and Uganda had 1705 and 1732 complete surveys, respectively. In both countries, a majority of the respondents were males, young adults (i.e., 18-29-year-olds), urban residents, and had O-level/above education level. In Bangladesh, the contact rate was higher in FI (48.9%), MM (50.0%), and FM (55.2%) groups than in MI (43.0%); the response rate was higher in FI (32.3%) and FM (33.1%) but not in MM (27.2%) and MI (27.1%). Some differences in cooperation and refusal rates were also observed. In Uganda, MM (65.4%) and FM (67.9%) had higher contact rates than MI (60.8%). The response rate was only higher in MI (52.5%) compared to MI (45.9%). Refusal and cooperation rates were similar. In Bangladesh, after pooling by introductions, female arms had higher contact (52.1% vs 46.5%), response (32.7% vs 27.1%), and cooperation (47.8% vs 40.4%) rates than male arms. Pooling by gender showed higher contact (52.3% vs 45.6%) and refusal (22.5% vs 16.3%) rates but lower cooperation rate (40.0% vs 48.2%) in motivational arms than informational arms. In Uganda, pooling intros did not show any difference in survey rates by gender; however, pooling by intros showed higher contact (66.5% vs 61.5%) and response (50.0% vs 45.2%) rates in motivational arms than informational arms.

CONCLUSION

Overall, we found higher survey rates among female voice and motivational introduction arms compared to male voice and informational introduction arm in Bangladesh. However, Uganda had higher rates for motivational intro arms only compared to informational arms. Gender and valence must be considered for successful IVR surveys.

TRIAL REGISTRATION

Name of the registry: ClinicalTrials.gov. Trial registration number: NCT03772431. Date of registration: 12/11/2018, Retrospectively Registered. URL of trial registry record: https://clinicaltrials.gov/ct2/show/NCT03772431?term=03772431&cond=Non-Communicable+Disease&draw=2&rank=1. Protocol Availability: https://www.researchprotocols.org/2017/5/e81.

Reduced Diarrhea Prevalence and Improvements in Handwashing with Soap and Stored Drinking Water Quality Associated with Diarrheal Disease Awareness Measured by Interactive Voice Response Messages in the CHoBI7 Mobile Health Program

The Cholera-Hospital-Based-Intervention-for-7-Days (CHoBI7) mobile health program promotes water, sanitation, and hygiene (WASH) behaviors through interactive voice response (IVR), voice, and text messages to reduce diarrheal diseases in Bangladesh. The objective of this study was to investigate the relationship between responses to CHoBI7 WASH IVR quiz messages and subsequent diarrhea and WASH behaviors. Fourteen CHoBI7 IVR quiz messages on handwashing with soap and treatment of stored water were sent to 517 households with 1,777 participants during the 12-month program period. IVR message responses were classified as correct answer, incorrect answer, no response (did not press 1 or 2), and failed (did not answer the phone). Diarrhea prevalence was assessed through self-reported monthly clinical surveillance visits. Handwashing with soap was assessed by a 5-hour structured observation, and stored water quality was defined by Escherichia coli concentration. Households that responded correctly to a CHoBI7 IVR quiz message had significantly lower odds of diarrhea for all age groups (adults and children) at the subsequent visit 1 month later (odds ratio [OR], 0.73; 95% CI, 0.54-0.98), and significantly greater odds of handwashing with soap after stool-related events (OR, 2.48; 95% CI, 1.12-5.49) and E. coli levels < 100 colony forming units (CFU)/100 mL (World Health Organization high-risk cutoff) in the stored household water (OR, 2.04; 95% CI, 1.25-3.33) compared with households that did not answer CHoBI7 IVR quiz calls. Correct responses to CHoBI7 IVR quizzes were associated with decreased diarrhea prevalence and improved stored drinking water quality and handwashing with soap behaviors at the subsequent visits. These findings suggest engagement in the CHoBI7 mobile health (mHealth) program and awareness of diarrheal disease prevention can reduce diarrhea and facilitate changes in WASH behaviors.

Remote consent approaches for mobile phone surveys of non-communicable disease risk factors in Colombia and Uganda: A randomized study

INTRODUCTION

Automated mobile phone surveys (MPS) can be used to collect public health data of various types to inform health policy and programs globally. One challenge in administering MPS is identification of an appropriate and effective participant consent process. This study investigated the impact of different survey consent approaches on participant disposition (response characteristics and understanding of the purpose of the survey) within the context of an MPS that measured noncommunicable disease (NCD) risk factors across Colombia and Uganda.

METHODS

Participants were randomized to one of five consent approaches, with consent modules varying by the consent disclosure and mode of authorization. The control arm consisted of a standard consent disclosure and a combined opt-in/opt-out mode of authorization. The other four arms consist of a modified consent disclosure and one of four different forms of authorization (i.e., opt-in, opt-out, combined opt-in/opt-out, or implied). Data related to respondent disposition and respondent understanding of the survey purpose were analyzed.

RESULTS

Among 1889 completed surveys in Colombia, differences in contact, response, refusal, and cooperation rates by study arms were found. About 68% of respondents correctly identified the survey purpose, with no significant difference by study arm. Participants reporting higher levels of education and urban residency were more likely to identify the purpose correctly. Participants were also more likely to accurately identify the survey purpose after completing several survey modules, compared to immediately following the consent disclosure (78.8% vs 54.2% correct, p<0.001). In Uganda, 1890 completed surveys were collected. Though there were differences in contact, refusal, and cooperation rates by study arm, response rates were similar across arms. About 37% of respondents identified the survey purpose correctly, with no difference by arm. Those with higher levels of education and who completed the survey in English were able to more accurately identify the survey purpose. Again, participants were more likely to accurately identify the purpose of the survey after completing several NCD modules, compared to immediately following the consent module (42.0% vs 32.2% correct, p = 0.013).

CONCLUSION

This study contributes to the limited available evidence regarding consent procedures for automated MPS. Future studies should develop and trial additional interventions to enhance consent for automated public health surveys, and measure other dimensions of participant engagement and understanding.

Promised and Lottery Airtime Incentives to Improve Interactive Voice Response Survey Participation Among Adults in Bangladesh and Uganda: Randomized Controlled Trial

Background

Increased mobile phone penetration allows the interviewing of respondents using interactive voice response surveys in low- and middle-income countries. However, there has been little investigation of the best type of incentive to obtain data from a representative sample in these countries.

Objective

We assessed the effect of different airtime incentives options on cooperation and response rates of an interactive voice response survey in Bangladesh and Uganda.

Methods

The open-label randomized controlled trial had three arms: (1) no incentive (control), (2) promised airtime incentive of 50 Bangladeshi Taka (US $0.60; 1 BDT is approximately equivalent to US $0.012) or 5000 Ugandan Shilling (US $1.35; 1 UGX is approximately equivalent to US $0.00028), and (3) lottery incentive (500 BDT and 100,000 UGX), in which the odds of winning were 1:20. Fully automated random-digit dialing was used to sample eligible participants aged ≥18 years. The risk ratios (RRs) with 95% confidence intervals for primary outcomes of response and cooperation rates were obtained using log-binomial regression.

Results

Between June 14 and July 14, 2017, a total of 546,746 phone calls were made in Bangladesh, with 1165 complete interviews being conducted. Between March 26 and April 22, 2017, a total of 178,572 phone calls were made in Uganda, with 1248 complete interviews being conducted. Cooperation rates were significantly higher for the promised incentive (Bangladesh: 39.3%; RR 1.38, 95% CI 1.24-1.55, P<.001; Uganda: 59.9%; RR 1.47, 95% CI 1.33-1.62, P<.001) and the lottery incentive arms (Bangladesh: 36.6%; RR 1.28, 95% CI 1.15-1.45, P<.001; Uganda: 54.6%; RR 1.34, 95% CI 1.21-1.48, P<.001) than those for the control arm (Bangladesh: 28.4%; Uganda: 40.9%). Similarly, response rates were significantly higher for the promised incentive (Bangladesh: 26.5%%; RR 1.26, 95% CI 1.14-1.39, P<.001; Uganda: 41.2%; RR 1.27, 95% CI 1.16-1.39, P<.001) and lottery incentive arms (Bangladesh: 24.5%%; RR 1.17, 95% CI 1.06-1.29, P=.002; Uganda: 37.9%%; RR 1.17, 95% CI 1.06-1.29, P=.001) than those for the control arm (Bangladesh: 21.0%; Uganda: 32.4%).

Conclusions

Promised or lottery airtime incentives improved survey participation and facilitated a large sample within a short period in 2 countries.

Trial Registration

ClinicalTrials.gov NCT03773146; http://clinicaltrials.gov/ct2/show/NCT03773146

Assessing the reliability of phone surveys to measure reproductive, maternal and child health knowledge among pregnant women in rural India: a feasibility study

OBJECTIVES

Efforts to understand the factors influencing the uptake of reproductive, maternal, newborn, child health and nutrition (RMNCH&N) services in high disease burden low-resource settings have often focused on face-to-face surveys or direct observations of service delivery. Increasing access to mobile phones has led to growing interest in phone surveys as a rapid, low-cost alternatives to face-to-face surveys. We assess determinants of RMNCH&N knowledge among pregnant women with access to phones and examine the reliability of alternative modalities of survey delivery.

PARTICIPANTS

Women 5-7 months pregnant with access to a phone.

SETTING

Four districts of Madhya Pradesh, India.

DESIGN

Cross-sectional surveys administered face-to-face and within 2 weeks, the same surveys were repeated among two random subsamples of the original sample: face-to-face (n=205) and caller-attended telephone interviews (n=375). Bivariate analyses, multivariable linear regression, and prevalence and bias-adjusted kappa scores are presented.

RESULTS

Knowledge scores were low across domains: 52% for maternal nutrition and pregnancy danger signs, 58% for family planning, 47% for essential newborn care, 56% infant and young child feeding, and 58% for infant and young child care. Higher knowledge (≥1 composite score) was associated with older age; higher levels of education and literacy; living in a nuclear family; primary health decision-making; greater attendance in antenatal care and satisfaction with accredited social health activist services. Survey questions had low inter-rater and intermodal reliability (kappa<0.70) with a few exceptions. Questions with the lowest reliability included true/false questions and those with unprompted, multiple response options. Reliability may have been hampered by the sensitivity of the content, lack of privacy, enumerators' and respondents' profile differences, rapport, social desirability bias, and/or enumerator's ability to adequately convey concepts or probe.

CONCLUSIONS

Phone surveys are a reliable modality for generating population-level estimates data about pregnant women's knowledge, however, should not be used for individual-level tracking.

TRIAL REGISTRATION NUMBER

NCT03576157.

Ethical considerations of phone-based interviews from three studies of COVID-19 impact in Bihar, India

Phone-based interviews present a range of ethical challenges, including how to ensure informed consent and privacy and maintain confidentiality. Our paper presents conceptual and practical ethical considerations taken into account across three telephone studies on the impact of COVID-19 conducted following India’s nationwide lockdown imposed in March 2020. Two studies captured COVID-19 response impact on primary-level Reproductive Maternal Neonatal and Child Health (RMNCH) services and on provider wellness, respectively. The third study focused on how the gendered experience of COVID-19 and the state’s response to control transmission impacted women’s lives, focusing on health services, livelihood, entitlements and social change, by interviewing individual women. The ethical challenges as well as the advantages of digital data collection are presented with recommendations for low-resource settings. Ethical considerations included the above challenges as well as avoiding posing unreasonable time burden on the respondents, framing questions with a gendered lens, considering emotional states given contagion concerns and economic uncertainties, and redressing pandemic-induced distress. Using scripted Hindi was challenging in consent-taking, as was protecting household respondents’ privacy and confidentiality during lockdown. Unanticipated positive ethical implications of using a telephone approach included providing respondents privacy and catharsis, respondents choosing convenient interview times and affording health providers more privacy than institutional inperson interviews. Internalising empathy, respect and appreciative enquiry are key to establishing rapport in the absence of prior relationships. Institutional Review Board (IRB) time limits on call duration need to be flexible to allow for ‘active listening’ and empathetic enquiry in surveys on the impact of COVID-19.

Participatory surveillance of COVID-19 in Lesotho via weekly calls: Protocol for cell phone data collection

Background

The increase in cell phone ownership in low- and middle-income countries (LMIC) has created an opportunity for low-cost, rapid data collection by calling participants on their cell phones. Cell phones can be mobilized for a myriad of data collection purposes, including surveillance. In LMIC, cell phone–based surveillance has been used to track Ebola, measles, acute flaccid paralysis, and diarrheal disease, as well as noncommunicable diseases. Phone-based surveillance in LMIC is a particularly pertinent, burgeoning approach in the context of the COVID-19 pandemic. Participatory surveillance via cell phone could allow governments to assess burden of disease and complements existing surveillance systems.

Objective

We describe the protocol for the LeCellPHIA (Lesotho Cell Phone PHIA) project, a cell phone surveillance system that collects weekly population-based data on influenza-like illness (ILI) in Lesotho by calling a representative sample of a recent face-to-face survey.

Methods

We established a phone-based surveillance system to collect ILI symptoms from approximately 1700 participants who had participated in a recent face-to-face survey in Lesotho, the Population-based HIV Impact Assessment (PHIA) Survey. Of the 15,267 PHIA participants who were over 18 years old, 11,975 (78.44%) consented to future research and provided a valid phone number. We followed the PHIA sample design and included 342 primary sampling units from 10 districts. We randomly selected 5 households from each primary sampling unit that had an eligible participant and sampled 1 person per household. We oversampled the elderly, as they are more likely to be affected by COVID-19. A 3-day Zoom training was conducted in June 2020 to train LeCellPHIA interviewers.

Results

The surveillance system launched July 1, 2020, beginning with a 2-week enrollment period followed by weekly calls that will continue until September 30, 2022. Of the 11,975 phone numbers that were in the sample frame, 3020 were sampled, and 1778 were enrolled.

Conclusions

The surveillance system will track COVID-19 in a resource-limited setting. The novel approach of a weekly cell phone–based surveillance system can be used to track other health outcomes, and this protocol provides information about how to implement such a system.

International Registered Report Identifier (IRRID)

DERR1-10.2196/31236

Phone-based monitoring to evaluate health policy and program implementation in Kenya

Monitoring and evaluating policies and programs in low- and middle-income countries are often difficult because of the lack of routine data. High mobile phone ownership in these countries presents an opportunity for efficient data collection through telephone interviews. This study examined the feasibility of collecting data on medicines through telephone interviews in Kenya. Data on the availability and prices of medicines at 137 health facilities and 639 patients were collected in September 2016 via in-person interviews. Between December 2016 and December 2017, monthly telephone interviews were conducted with health facilities and patients. An unannounced in-person interview was conducted with respondents to validate the telephone interview within 24 h. A bottom-up itemization costing approach was used to estimate the costs of telephone and in-person data collection. In-depth interviews were conducted with data collectors and respondents to explore their perceptions on both modes of data collection. The level of agreement between data on medicines availability collected through phone and in-person interviews was strong at the health facility level [kappa = 0.90; confidence interval (CI) 0.88–0.92] and moderate at the household level (kappa = 0.50, CI 0.39–0.60). Price data from telephone and in-person interviews showed strong intra-class correlation at health facilities [intra-class correlation coefficient (ICC) = 0.96] and moderate intra-class correlation at households (ICC = 0.47). The cost per phone interview at health facilities and households were $19.73 and $16.86, respectively, compared to $186.20 for a baseline in-person interview. Participants considered telephone interviews to be more convenient. In countries with high cell phone penetration, telephone data collection should be considered in monitoring and evaluating public health programs especially at health facilities. Additional strategies may be needed to optimize this mode of data collection at the household level. Variations in cell phone ownership, telecommunication network and data collection costs across different settings may limit the generalizability of the findings from this study.

A cost study for mobile phone health surveys using interactive voice response for assessing risk factors of noncommunicable diseases

Background

This is the first study to examine the costs of conducting a mobile phone survey (MPS) through interactive voice response (IVR) to collect information on risk factors for noncommunicable diseases (NCD) in three low- and middle-income countries (LMIC); Bangladesh, Colombia, and Uganda.

Methods

This is a micro-costing study conducted from the perspective of the payer/funder with a 1-year horizon. The study evaluates the fixed costs and variable costs of implementing one nationally representative MPS for NCD risk factors of the adult population. In this costing study, we estimated the sample size of calls required to achieve a population-representative survey and associated incentives. Cost inputs were obtained from direct economic costs incurred by a central study team, from country-specific collaborators, and from platform developers who participated in the deployment of these MPS during 2017. Costs were reported in US dollars (USD). A sensitivity analysis was conducted assessing different scenarios of pricing and incentive strategies. Also, costs were calculated for a survey deployed targeting only adults younger than 45 years.

Results

We estimated the fixed costs ranging between $47,000 USD and $74,000 USD. Variable costs were found to be between $32,000 USD and $129,000 USD per nationally representative survey. The main cost driver was the number of calls required to meet the sample size, and its variability largely depends on the extent of mobile phone coverage and access in the country. Therefore, a larger number of calls were estimated to survey specific harder-to-reach sub-populations.

Conclusion

Mobile phone surveys have the potential to be a relatively less expensive and timely method of collecting survey information than face-to-face surveys, allowing decision-makers to deploy survey-based monitoring or evaluation programs more frequently than it would be possible having only face-to-face contact. The main driver of variable costs is survey time, and most of the variability across countries is attributable to the sampling differences associated to reaching out to population subgroups with low mobile phone ownership or access.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12963-021-00258-z.

Adaptation of a mobile phone health survey for risk factors for noncommunicable diseases in Colombia: a qualitative study

Background

Data collection on noncommunicable disease (NCD) behavioral risk factors has traditionally been carried out through face-to-face surveys. However, its high costs and logistical difficulties can lead to lack of timely statistics for planning, particularly in low and middle-income countries. Mobile phone surveys (MPS) have the potential to fill these gaps.

Objective

This study explores perceptions, feasibility and strategies to increase the acceptability and response rate of health surveys administered through MPS using interactive voice response in Colombia.

Method

A sequential multimodal exploratory design was used. We conducted key informant interviews (KII) with stakeholders from government and academia; focus group discussions (FGDs) and user-group tests (UGTs) with young adults and elderly people living in rural and urban settings (men and women). The KII and FGDs explored perceptions of using mobile phones for NCD surveys. In the UGTs, participants were administered an IVR survey, and they provided feedback on its usability and potential improvement.

Results

Between February and November 2017, we conducted 7 KII, 6 FGDs (n = 54) and 4 UGTs (n = 34). Most participants consider MPS is a novel way to explore risk factors in NCDs. They also recognize challenges for their implementation including security issues, technological literacy and telecommunications coverage, especially in rural areas. It was recommended to promote the survey using mass media before its deployment and stressing its objectives, responsible institution and data privacy safeguards. The preferences in the survey administration relate to factors such as skills in the use of mobile phones, age, availability of time and educational level. The participants recommend questionnaires shorter than 10 minutes.

Conclusions

The possibility of obtaining data through MPS at a population level represents an opportunity to improve the availability of risk-factor data. Steps towards increasing the acceptability and overcoming technological and methodological challenges need to be taken.

Informed Consent for Mobile Phone Health Surveys in Colombia: A Qualitative Study

Public health surveys deployed through automated mobile phone calls raise a set of ethical challenges, including succinctly communicating information necessary to obtain respondent informed consent. This study aimed to capture the perspectives of key stakeholders, both experts and community members, on consent processes and preferences for participation in automated mobile phone surveys (MPS) of non-communicable disease risk factors in Colombia. We conducted semi-structured interviews with ethics and digital health experts and focus group discussions with community representatives. There was meaningful disagreement within both groups regarding the necessity of consent, when the purpose of a survey is to contribute to the formulation of public policies. Respondents who favored consent emphasized that consent communications ought to promote understanding and voluntariness, and implicitly suggested that information disclosure conform to a reasonable person standard. Given the automated and unsolicited nature of the phone calls and concerns regarding fraud, trust building was emphasized as important, especially for national MPS deployment. Community sensitization campaigns that provide relevant contextual information (such as the name of the administering institution) were thought to support trust-building. Additional ways to achieve the goals of consent while building trust in automated MPS for disease surveillance should be evaluated in order to inform ethical and effective practice.

User Perceptions and Experiences of an Interactive Voice Response Mobile Phone Survey Pilot in Uganda: Qualitative Study

Background

With the growing burden of noncommunicable diseases in low- and middle- income countries, the World Health Organization recommended a stepwise approach of surveillance for noncommunicable diseases. This is expensive to conduct on a frequent basis and using interactive voice response mobile phone surveys has been put forth as an alternative. However, there is limited evidence on how to design and deliver interactive voice response calls that are robust and acceptable to respondents.

Objective

This study aimed to explore user perceptions and experiences of receiving and responding to an interactive voice response call in Uganda in order to adapt and refine the instrument prior to national deployment.

Methods

A qualitative study design was used and comprised a locally translated audiorecorded interactive voice response survey delivered in 4 languages to 59 purposively selected participants' mobile phones in 5 survey rounds guided by data saturation. The interactive voice response survey had modules on sociodemographic characteristics, physical activity, fruit and vegetable consumption, diabetes, and hypertension. After the interactive voice response survey, study staff called participants back and used a semistructured interview to collect information on the participant’s perceptions of interactive voice response call audibility, instruction clarity, interview pace, language courtesy and appropriateness, the validity of questions, and the lottery incentive. Descriptive statistics were used for the interactive voice response survey, while a framework analysis was used to analyze qualitative data.

Results

Key findings that favored interactive voice response survey participation or completion included preference for brief surveys of 10 minutes or shorter, preference for evening calls between 6 PM and 10 PM, preference for courteous language, and favorable perceptions of the lottery-type incentive. While key findings curtailing participation were suspicion about the caller’s identity, unclear voice, confusing skip patterns, difficulty with the phone interface such as for selecting inappropriate digits for both ordinary and smartphones, and poor network connectivity for remote and rural participants.

Conclusions

Interactive voice response surveys should be as brief as possible and considerate of local preferences to increase completion rates. Caller credibility needs to be enhanced through either masking the caller or prior community mobilization. There is need to evaluate the preferred timing of interactive voice response calls, as the finding of evening call preference is inconclusive and might be contextual.

Process evaluation for the delivery of a water, sanitation and hygiene mobile health program: findings from the randomised controlled trial of the CHoBI7 mobile health program

OBJECTIVE

The Cholera-Hospital-Based Intervention for 7-days (CHoBI7) mobile health (mHealth) program delivers mobile messages to diarrhoea patient households promoting water treatment and handwashing with soap. The randomised controlled trial (RCT) of the CHoBI7 mHealth program demonstrated this intervention was effective in significantly reducing diarrhoea and stunting amoung young children. The objective of this study was to assess the implementation of the CHoBI7 mHealth program in delivering mHealth messages during this RCT.

METHODS

517 diarrhoea patient households with 1777 participants received weekly text, voice and interactive voice response (IVR) messages from the CHoBI7 mHealth program over the 12-month program period. The program process evaluation indicators were the following: the percentage of CHoBI7 mHealth messages received and fully listened to by program households (program fidelity and dose), and household members reporting receiving and sharing an mHealth message from the program in the past two weeks (program reach).

RESULTS

Ninety two percent of text messages were received by program households. Eighty three percent of voice and 86% of IVR messages sent were fully listened to by at least one household member. Eighty one percent of IVR quiz responses from households were answered correctly. Program households reported receiving a CHoBI7 mHealth message in the past two weeks at 79% of monthly household visits during the 12-month program. Seventy seven percent of participants reported sharing a program message with a spouse, 55% with a neighbour and 49% with a child during the program period.

CONCLUSION

There was high fidelity, dose and reach of mobile messages delivered for the CHoBI7 mHealth program. This study presents an approach for process evaluation that can be implemented to evaluate future mHealth programs.

Comparability of modern contraceptive use estimates between a face-to-face survey and a cellphone survey among women in Burkina Faso

Introduction

The proliferation of cell phone ownership in Sub-Saharan Africa (SSA) presents the opportunity to collect public health indicators at a lower cost compared to face-to-face (FTF) surveys. This analysis assesses the equivalence of modern contraceptive prevalence estimates between a nationally representative FTF survey and a cell phone survey using random digit dialing (RDD) among women of reproductive age in Burkina Faso.

Methods

We analyzed data from two surveys conducted in Burkina Faso between December 2017 and May 2018. The FTF survey conducted by Performance Monitoring and Accountability (PMA2020) comprised a nationally representative sample of 3,556 women of reproductive age (15–49 years). The RDD survey was conducted using computer-assisted telephone interviewing and included 2,379 women of reproductive age.

Results

Compared to FTF respondents, women in the RDD sample were younger, were more likely to have a secondary degree and to speak French. RDD respondents were more likely to report using modern contraceptive use (40%) compared to FTF respondents (26%) and the difference remained unchanged after applying post-stratification weights to the RDD sample (39%). This difference surpassed the equivalence margin of 4%. The RDD sample also produced higher estimates of contraceptive use than the subsample of women who owned a phone in the FTF sample (32%). After adjusting for women’s sociodemographic factors, the odds of contraceptive use were 1.9 times higher (95% CI: 1.6–2.2) in the RDD survey compared to the FTF survey and 1.6 times higher (95% CI: 1.3–1.8) compared to FTF phone owners.

Conclusions

Modern contraceptive prevalence in Burkina Faso is over-estimated when using a cell phone RDD survey, even after adjusting for a number of sociodemographic factors. Further research should explore causes of differential estimates of modern contraceptive use by survey modes.

Effect of airtime incentives on response and cooperation rates in non-communicable disease interactive voice response surveys: randomised controlled trials in Bangladesh and Uganda

Background

The global proliferation of mobile phones offers opportunity for improved non-communicable disease (NCD) data collection by interviewing participants using interactive voice response (IVR) surveys. We assessed whether airtime incentives can improve cooperation and response rates for an NCD IVR survey in Bangladesh and Uganda.

Methods

Participants were randomised to three arms: a) no incentive, b) 1X incentive or c) 2X incentive, where X was set to airtime of 50 Bangladesh Taka (US$0.60) and 5000 Ugandan Shillings (UGX; US$1.35). Adults aged 18 years and older who had a working mobile phone were sampled using random digit dialling. The primary outcomes, cooperation and response rates as defined by the American Association of Public Opinion Research, were analysed using log-binomial regression model.

Results

Between 14 June and 14 July 2017, 440 262 phone calls were made in Bangladesh. The cooperation and response rates were, respectively, 28.8% (353/1227) and 19.2% (580/3016) in control, 39.2% (370/945) and 23.9% (507/2120) in 50 Taka and 40.0% (362/906) and 24.8% (532/2148) in 100 Taka incentive groups. Cooperation and response rates, respectively, were significantly higher in both the 50 Taka (risk ratio (RR) 1.36, 95% CI 1.21 to 1.53) and (RR 1.24, 95% CI 1.12 to 1.38), and 100 Taka groups (RR 1.39, 95% CI 1.23 to 1.56) and (RR 1.29, 95% CI 1.16 to 1.43), as compared with the controls. In Uganda, 174 157 phone calls were made from 26 March to 22 April 2017. The cooperation and response rates were, respectively, 44.7% (377/844) and 35.2% (552/1570) in control, 57.6% (404/701) and 39.3% (508/1293) in 5000 UGX and 58.8% (421/716) and 40.3% (535/1328) in 10 000 UGX groups. Cooperation and response rates were significantly higher, respectively in the 5000 UGX (RR 1.29, 95% CI 1.17 to 1.42) and (RR 1.12, 95% CI 1.02 to 1.23), and 10 000 UGX groups (RR 1.32, 95% CI 1.19 to 1.45) and (RR 1.15, 95% CI 1.04 to 1.26), as compared with the control group.

Conclusion

In two diverse settings, the provision of an airtime incentive significantly improved both the cooperation and response rates of an IVR survey, with no significant difference between the two incentive amounts.

Trial registration number

NCT03768323.

Development of a Phone Survey Tool to Measure Respectful Maternity Care During Pregnancy and Childbirth in India: Study Protocol

Background

Respectful maternity care (RMC) is a key barometer of the underlying quality of care women receive during pregnancy and childbirth. Efforts to measure RMC have largely been qualitative, although validated quantitative tools are emerging. Available tools have been limited to the measurement of RMC during childbirth and confined to observational and face-to-face survey modes. Phone surveys are less invasive, low cost, and rapid alternatives to traditional face-to-face methods, yet little is known about their validity and reliability.

Objective

The primary objective of this study was to develop validated face-to-face and phone survey tools for measuring RMC during pregnancy and childbirth for use in India and other low resource settings. The secondary objective was to optimize strategies for improving the delivery of phone surveys for use in measuring RMC.

Methods

To develop face-to-face and phone surveys for measuring RMC, we describe procedures for assessing content, criterion, and construct validity as well as reliability analyses. To optimize the delivery of phone surveys, we outline plans for substudies, which aim to assess the effect of survey modality, and content on survey response, completion, and attrition rates.

Results

Data collection will be carried out in 4 districts of Madhya Pradesh, India, from July 2018 to March 2019.

Conclusions

To our knowledge, this is the first RMC phone survey tool developed for India, which may provide an opportunity for the rapid, routine collection of data essential for improving the quality of care during pregnancy and childbirth. Elsewhere, phone survey tools are emerging; however, efforts to develop these surveys are often not inclusive of rigorous pretesting activities essential for ensuring quality data, including cognitive, reliability, and validity testing. In the absence of these activities, emerging data could overestimate or underestimate the burden of disease and health care practices under assessment. In the context of RMC, poor quality data could have adverse consequences including the naming and shaming of providers. By outlining a blueprint of the minimum activities required to generate reliable and valid survey tools, we hope to improve efforts to develop and deploy face-to-face and phone surveys in the health sector.

International Registered Report Identifier (IRRID)

DERR1-10.2196/12173

Does mobile phone survey method matter? Reliability of computer-assisted telephone interviews and interactive voice response non-communicable diseases risk factor surveys in low and middle income countries

Introduction

Increased mobile phone subscribership in low- and middle-income countries (LMICs) provides novel opportunities to track population health. The objective of this study was to examine reliability of data in comparing participant responses collected using two mobile phone survey (MPS) delivery modalities, computer assisted telephone interviews (CATI) and interactive voice response (IVR) in Bangladesh (BGD) and Tanzania (TZA).

Methods

Using a cross-over design, we used random digit dialing (RDD) to call randomly generated mobile phone numbers and recruit survey participants to receive either a CATI or IVR survey on non-communicable disease (NCD) risk factors, followed 7 days later by the survey mode not received during first contact; either IVR or CATI. Respondents who received the first survey were designated as first contact (FC) and those who consented to being called a second time and subsequently answered the call were designated as follow-up (FU). We used the same questionnaire for both contacts, with response options modified to suit the delivery mode. Reliability of responses was analyzed using the Cohen’s kappa statistic for percent agreement between two modes.

Results

Self-reported data on demographic characteristics and NCD behavioral risk factors were collected from 482 (CATI-FC) and 653 (IVR-FC) age-eligible and consenting respondents in BGD, and from 387 (CATI-FC) and 674 (IVR-FC) respondents in TZA respectively. Survey follow-up rates were 30.7% (n = 482) for IVR-FU and 53.8% (n = 653) for CATI-FU in BGD; and 42.4% (n = 387) for IVR-FU and 49.9% (n = 674) for CATI-FU in TZA respectively. Overall, there was high consistency between delivery modalities for alcohol consumption in the past 30 days in both countries (kappa = 0.64 for CATI→IVR (BGD), kappa = 0.54 for IVR→CATI (BGD); kappa = 0.66 for CATI→IVR (TZA), kappa = 0.76 for IVR→CATI (TZA)), and current smoking (kappa = 0.68 for CATI→IVR (BGD), kappa = 0.69 for IVR→CATI (BGD); kappa = 0.39 for CATI→IVR (TZA), kappa = 0.50 for IVR→CATI (TZA)). There was moderate to substantial consistency in both countries for history of checking for hypertension and diabetes with kappa statistics ranging from 0.43 to 0.67. There was generally lower consistency in both countries for physical activity (vigorous and moderate) with kappa statistics ranging from 0.10 to 0.41, weekly fruit and vegetable with kappa ranging from 0.08 to 0.45, consumption of foods high in salt and efforts to limit salt with kappa generally below 0.3.

Conclusions

The study found that when respondents are re-interviewed, the reliability of answers to most demographic and NCD variables is similar whether starting with CATI or IVR. The study underscores the need for caution when selecting questions for mobile phone surveys. Careful design can help ensure clarity of questions to minimize cognitive burden for respondents, many of whom may not have prior experience in taking automated surveys. Further research should explore possible differences and determinants of survey reliability between delivery modes and ideally compare both IVR and CATI surveys to in-person face-to-face interviews. In addition, research is needed to better understand factors that influence survey cooperation, completion, refusal and attrition rates across populations and contexts.

Consent for mobile phone surveys of non-communicable disease risk factors in low-resource settings: an exploratory qualitative study in Uganda

BACKGROUND

Lack of data for timely decision-making around the prevention and control of non-communicable diseases (NCDs) presents special challenges for policy makers, especially in resource-limited settings. New data collection methods, including pre-recorded Interactive Voice Response (IVR) phone surveys, are being developed to support rapid compilation of population-level disease risk factor information in such settings. We aimed to identify information that could be used to optimize consent approaches for future mobile phone surveys (MPS) employed in Uganda and, possibly, similar contexts.

METHODS

We conducted an in-depth qualitative study with key stakeholders in Uganda about consent approaches, and potential challenges, for pre-recorded IVR NCD risk factor surveys. Semi-structured interviews were conducted with 14 key informants. A contextualized thematic approach was used to interpret the results supported by representative quotes.

RESULTS

Several potential challenges in designing consent approaches for MPS were identified, including low literacy and the lack of appropriate ways of assessing comprehension and documenting consent. Communication with potential respondents prior to the MPS and providing options for callbacks were suggested as possible strategies for improving comprehension within the consent process. "Opt-in" forms of authorization were preferred over "opt-out". There was particular concern about data security and confidentiality and how matters relating to this would be communicated to MPS respondents.

CONCLUSIONS

These local insights provide important information to support optimization of consent for MPS, whose use is increasing globally to advance public health surveillance and research in constructive ways.

Building the Evidence Base for Remote Data Collection in Low- and Middle-Income Countries: Comparing Reliability and Accuracy Across Survey Modalities

Background

Given the growing interest in mobile data collection due to the proliferation of mobile phone ownership and network coverage in low- and middle-income countries (LMICs), we synthesized the evidence comparing estimates of health outcomes from multiple modes of data collection. In particular, we reviewed studies that compared a mode of remote data collection with at least one other mode of data collection to identify mode effects and areas for further research.

Objective

The study systematically reviewed and summarized the findings from articles and reports that compare a mode of remote data collection to at least one other mode. The aim of this synthesis was to assess the reliability and accuracy of results.

Methods

Seven online databases were systematically searched for primary and grey literature pertaining to remote data collection in LMICs. Remote data collection included interactive voice response (IVR), computer-assisted telephone interviews (CATI), short message service (SMS), self-administered questionnaires (SAQ), and Web surveys. Two authors of this study reviewed the abstracts to identify articles which met the primary inclusion criteria. These criteria required that the survey collected the data from the respondent via mobile phone or landline. Articles that met the primary screening criteria were read in full and were screened using secondary inclusion criteria. The four secondary inclusion criteria were that two or more modes of data collection were compared, at least one mode of data collection in the study was a mobile phone survey, the study had to be conducted in a LMIC, and finally, the study should include a health component.

Results

Of the 11,568 articles screened, 10 articles were included in this study. Seven distinct modes of remote data collection were identified: CATI, SMS (singular sitting and modular design), IVR, SAQ, and Web surveys (mobile phone and personal computer). CATI was the most frequent remote mode (n=5 articles). Of the three in-person modes (face-to-face [FTF], in-person SAQ, and in-person IVR), FTF was the most common (n=11) mode. The 10 articles made 25 mode comparisons, of which 12 comparisons were from a single article. Six of the 10 articles included sensitive questions.

Conclusions

This literature review summarizes the existing research about remote data collection in LMICs. Due to both heterogeneity of outcomes and the limited number of comparisons, this literature review is best positioned to present the current evidence and knowledge gaps rather than attempt to draw conclusions. In order to advance the field of remote data collection, studies that employ standardized sampling methodologies and study designs are necessary to evaluate the potential for differences by survey modality.