Value of audio-enhanced handheld computers over paper surveys with adolescents

The impact of modes of administration on self-reports of offending: evidence from a methodological experiment with university students

Objectives

Current knowledge about the causes of offending behavior is heavily reliant on self-reports of offending (SRO). However, methodological research on the impact of modes of administration on SRO is very scarce. Further, the existing evidence conflicts with the general knowledge about responding to sensitive questions. In this study, we aimed to test whether SRO are affected by modes of administration.

Methods

We carried out a methodological experiment, with a 2 (interviewer-administered vs. self-administered surveys) × 2 (paper-and-pencil vs. computer- assisted surveys) factorial design. A total of 181 undergraduate students were randomly assigned to one of these conditions and completed the International Self-Report Delinquency 3 (ISRD3) questionnaire.

Results

Findings showed an increased odds of reporting offending behavior in self-administered surveys, compared to face-to-face interviews. Paper-and-pencil and computer-assisted modes resulted in comparable estimates of offending.

Conclusions

This experiment provides evidence that SRO provide more accurate estimates of offending behavior using self-administered surveys.

Enhancing validity, reliability and participation in self-reported health outcome measurement for children and young people: a systematic review of recall period, response scale format, and administration modality

Introduction

Self-report is the gold standard for measuring children’s health-related outcomes. Design of such measures is complex and challenging. This review aims to systematically appraise the evidence on recall period, response scale format, mode of administration and approaches needed to enable children and young people < 19 years to participate in valid and reliable self-reporting of their health outcomes.

Method

PsycInfo, Medline, CINAHL and Embase were searched from 1 January 1990 to 15 March 2020, and citation searching undertaken in Scopus. Articles were included if they were primary research or case reports of ≥ 3 participants reporting the following: recall period, response scale selection, administration modality. Quality was assessed using QualSyst, and results synthesised narratively. This review was conducted and reported according to PRISMA guidelines.

Results

81 of 13,215 retrieved articles met the inclusion criteria. Children < 5 years old cannot validly and reliably self-report health outcomes. Face scales demonstrate better psychometric properties than visual analogue or Likert scales. Computerised and paper scales generally show equivalent construct validity. Children prefer computerised measures. Children ≤ 7 years old think dichotomously so need two response options. Those > 8 years old can reliably use a 3-point scale.

Conclusion

The results of this review have both clinical and research implications. They can be used to inform appropriate choice of PROM for use with CYP in the clinical setting. We also give eight recommendations for future development of self-reported outcome measures for children and young people.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11136-021-02814-4.

Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods

BACKGROUND

Self-administered survey questionnaires are an important data collection tool in clinical practice, public health research and epidemiology. They are ideal for achieving a wide geographic coverage of the target population, dealing with sensitive topics and are less resource-intensive than other data collection methods. These survey questionnaires can be delivered electronically, which can maximise the scalability and speed of data collection while reducing cost. In recent years, the use of apps running on consumer smart devices (i.e., smartphones and tablets) for this purpose has received considerable attention. However, variation in the mode of delivering a survey questionnaire could affect the quality of the responses collected.

OBJECTIVES

To assess the impact that smartphone and tablet apps as a delivery mode have on the quality of survey questionnaire responses compared to any other alternative delivery mode: paper, laptop computer, tablet computer (manufactured before 2007), short message service (SMS) and plastic objects.

SEARCH METHODS

We searched MEDLINE, EMBASE, PsycINFO, IEEEXplore, Web of Science, CABI: CAB Abstracts, Current Contents Connect, ACM Digital, ERIC, Sociological Abstracts, Health Management Information Consortium, the Campbell Library and CENTRAL. We also searched registers of current and ongoing clinical trials such as ClinicalTrials.gov and the World Health Organization (WHO) International Clinical Trials Registry Platform. We also searched the grey literature in OpenGrey, Mobile Active and ProQuest Dissertation & Theses. Lastly, we searched Google Scholar and the reference lists of included studies and relevant systematic reviews. We performed all searches up to 12 and 13 April 2015.

SELECTION CRITERIA

We included parallel randomised controlled trials (RCTs), crossover trials and paired repeated measures studies that compared the electronic delivery of self-administered survey questionnaires via a smartphone or tablet app with any other delivery mode. We included data obtained from participants completing health-related self-administered survey questionnaire, both validated and non-validated. We also included data offered by both healthy volunteers and by those with any clinical diagnosis. We included studies that reported any of the following outcomes: data equivalence; data accuracy; data completeness; response rates; differences in the time taken to complete a survey questionnaire; differences in respondent's adherence to the original sampling protocol; and acceptability to respondents of the delivery mode. We included studies that were published in 2007 or after, as devices that became available during this time are compatible with the mobile operating system (OS) framework that focuses on apps.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data from the included studies using a standardised form created for this systematic review in REDCap. They then compared their forms to reach consensus. Through an initial systematic mapping on the included studies, we identified two settings in which survey completion took place: controlled and uncontrolled. These settings differed in terms of (i) the location where surveys were completed, (ii) the frequency and intensity of sampling protocols, and (iii) the level of control over potential confounders (e.g., type of technology, level of help offered to respondents). We conducted a narrative synthesis of the evidence because a meta-analysis was not appropriate due to high levels of clinical and methodological diversity. We reported our findings for each outcome according to the setting in which the studies were conducted.

MAIN RESULTS

We included 14 studies (15 records) with a total of 2275 participants; although we included only 2272 participants in the final analyses as there were missing data for three participants from one included study.Regarding data equivalence, in both controlled and uncontrolled settings, the included studies found no significant differences in the mean overall scores between apps and other delivery modes, and that all correlation coefficients exceeded the recommended thresholds for data equivalence. Concerning the time taken to complete a survey questionnaire in a controlled setting, one study found that an app was faster than paper, whereas the other study did not find a significant difference between the two delivery modes. In an uncontrolled setting, one study found that an app was faster than SMS. Data completeness and adherence to sampling protocols were only reported in uncontrolled settings. Regarding the former, an app was found to result in more complete records than paper, and in significantly more data entries than an SMS-based survey questionnaire. Regarding adherence to the sampling protocol, apps may be better than paper but no different from SMS. We identified multiple definitions of acceptability to respondents, with inconclusive results: preference; ease of use; willingness to use a delivery mode; satisfaction; effectiveness of the system informativeness; perceived time taken to complete the survey questionnaire; perceived benefit of a delivery mode; perceived usefulness of a delivery mode; perceived ability to complete a survey questionnaire; maximum length of time that participants would be willing to use a delivery mode; and reactivity to the delivery mode and its successful integration into respondents' daily routine. Finally, regardless of the study setting, none of the included studies reported data accuracy or response rates.

AUTHORS' CONCLUSIONS

Our results, based on a narrative synthesis of the evidence, suggest that apps might not affect data equivalence as long as the intended clinical application of the survey questionnaire, its intended frequency of administration and the setting in which it was validated remain unchanged. There were no data on data accuracy or response rates, and findings on the time taken to complete a self-administered survey questionnaire were contradictory. Furthermore, although apps might improve data completeness, there is not enough evidence to assess their impact on adherence to sampling protocols. None of the included studies assessed how elements of user interaction design, survey questionnaire design and intervention design might influence mode effects. Those conducting research in public health and epidemiology should not assume that mode effects relevant to other delivery modes apply to apps running on consumer smart devices. Those conducting methodological research might wish to explore the issues highlighted by this systematic review.

Using computer-assisted survey instruments instead of paper and pencil increased completeness of self-administered sexual behavior questionnaires

OBJECTIVES

To compare the data quality, logistics, and cost of a self-administered sexual behavior questionnaire administered either using a computer-assisted survey instrument (CASI) or by paper and pencil in a primary care clinic.

STUDY DESIGN AND SETTING

A self-administered sexual behavior questionnaire was administered to 16-29 year olds attending general practice. Questionnaires were administered by either paper and pencil (paper) or CASI. A personal digital assistant was used to self-administer the CASI.

RESULTS

A total of 4,491 people completed the questionnaire, with 46.9% responses via CASI and 53.2% by paper. Completion of questions was greater for CASI than for paper for sexual behavior questions: number of sexual partners [odds ratio (OR), 6.85; 95% confidence interval (CI): 3.32, 14.11] and ever having had sex with a person of the same gender (OR, 2.89; 95% CI: 1.52, 5.49). The median number of questions answered was higher for CASI than for paper (17.6 vs. 17.2; P < 0.01). CASI was cheaper to run at $8.18 per questionnaire compared with $11.83 for paper.

CONCLUSION

Electronic devices using CASI are a tool that can increase participants' questionnaire responses and deliver more complete data for a sexual behavior questionnaire in primary care clinics.