The effect of two lottery-style incentives on response rates to postal questionnaires in a prospective cohort study in preschool children at high risk of asthma: a randomized trial

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Self-administered questionnaires are widely used to collect data in epidemiological research, but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH METHODS

We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented.

SELECTION CRITERIA

Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre-defined criteria.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi2 test and quantified the degree of inconsistency between trial results using the I2 statistic.

MAIN RESULTS

Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follow-up (OR 1.69; 95% CI 0.93 to 3.06); using a more interesting/salient questionnaire (OR 1.73; 95% CI 1.12 to 2.66); and when mentioning an obligation to respond (OR 1.61; 95% CI 1.16 to 2.22). The odds of response also increased with: non-monetary incentives (OR 1.16; 95% CI 1.11 to 1.21; I2 = 80%); a larger monetary incentive (OR 1.24; 95% CI 1.15 to 1.33); a larger non-monetary incentive (OR 1.15; 95% CI 1.00 to 1.33); when a pen was included (OR 1.44; 95% CI 1.38 to 1.50); using personalised materials (OR 1.15; 95% CI 1.09 to 1.21; I2 = 57%); using a single-sided rather than a double-sided questionnaire (OR 1.13; 95% CI 1.02 to 1.25); using stamped return envelopes rather than franked return envelopes (OR 1.23; 95% CI 1.13 to 1.33; I2 = 69%), assuring confidentiality (OR 1.33; 95% CI 1.24 to 1.42); using first-class outward mailing (OR 1.11; 95% CI 1.02 to 1.21); and when questionnaires originated from a university (OR 1.32; 95% CI 1.13 to 1.54). The odds of response were reduced when the questionnaire included questions of a sensitive nature (OR 0.94; 95% CI 0.88 to 1.00). Electronic We found 88 eligible trials that evaluated over 30 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response tripled when: using a brief letter rather than a detailed letter (OR 3.26; 95% CI 1.79 to 5.94); and when a picture was included in an email (OR 3.05; 95% CI 1.84 to 5.06; I2 = 19%). The odds of response almost doubled when: using monetary incentives (OR 1.88; 95% CI 1.31 to 2.71; I2 = 79%); and using a more interesting topic (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by half when: using non-monetary incentives (OR 1.60; 95% CI 1.25 to 2.05); using shorter e-questionnaires (OR 1.51; 95% CI 1.06 to 2.16; I2 = 94%); and using a more interesting e-questionnaire (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third when: offering survey results as an incentive (OR 1.36; 95% CI 1.16 to 1.59); using a white background (OR 1.31; 95% CI 1.10 to 1.56); and when stressing the benefits to society of response (OR 1.38; 95% CI 1.07 to 1.78; I2 = 41%). The odds of response also increased with: personalised e-questionnaires (OR 1.24; 95% CI 1.17 to 1.32; I2 = 41%); using a simple header (OR 1.23; 95% CI 1.03 to 1.48); giving a deadline (OR 1.18; 95% CI 1.03 to 1.34); and by giving a longer time estimate for completion (OR 1.25; 95% CI 0.96 to 1.64). The odds of response were reduced when: "Survey" was mentioned in the e-mail subject (OR 0.81; 95% CI 0.67 to 0.97); when the email or the e-questionnaire was from a male investigator, or it included a male signature (OR 0.55; 95% CI 0.38 to 0.80); and by using university sponsorship (OR 0.84; 95%CI 0.69 to 1.01). The odds of response using a postal questionnaire were over twice those using an e-questionnaire (OR 2.33; 95% CI 2.25 to 2.42; I2 = 98%). Response also increased when: providing a choice of response mode (electronic or postal) rather than electronic only (OR 1.76 95% CI 1.67 to 1.85; I2 = 97%); and when administering the e-questionnaire by computer rather than by smartphone (OR 1.62 95% CI 1.36 to 1.94).

AUTHORS' CONCLUSIONS

Researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this Cochrane review.

Loss to 5-year follow-up in the population-based Telemark Study: risk factors and potential for bias

OBJECTIVES

This study aimed to characterise participants lost to follow-up and identify possible factors associated with non-participation in a prospective population-based study of respiratory health in Norway. We also aimed to analyse the impact of potentially biased risk estimates associated with a high proportion of non-responders.

DESIGN

Prospective 5-year follow-up study.

SETTING

Randomly selected inhabitants from the general population of Telemark County in south-eastern Norway were invited to fill in a postal questionnaire in 2013. Responders in 2013 were followed-up in 2018.

PARTICIPANTS

16 099 participants aged 16-50 years completed the baseline study. 7958 responded at the 5-year follow-up, while 7723 did not.

MAIN OUTCOME MEASURES

χ² test was performed to compare demographic and respiratory health-related characteristics between those who participated in 2018 and those who were lost to follow-up. Adjusted multivariable logistic regression models were used to assess the relationship between loss to follow-up, background variables, respiratory symptoms, occupational exposure and interactions, and to analyse whether loss to follow-up leads to biased risk estimates.

RESULTS

7723 (49%) participants were lost to follow-up. Loss to follow-up was significantly higher for male participants, those in the youngest age group (16-30 years), those in lowest education level category and among current smokers (all p<0.001). In multivariable logistic regression analysis, loss to follow-up was significantly associated with unemployment (OR 1.34, 95% CI 1.22 to 1.46), reduced work ability (1.48, 1.35 to 1.60), asthma (1.22, 1.10 to 1.35), being woken by chest tightness (1.22, 1.11 to 1.34) and chronic obstructive pulmonary disease (1.81, 1.30 to 2.52). Participants with more respiratory symptoms and exposure to vapour, gas, dust and fumes (VGDF) (1.07 to 1.00-1.15), low-molecular weight (LMW) agents (1.19, 1.00 to 1.41) and irritating agents (1.15, 1.05 to 1.26) were more likely to be lost to follow-up. We found no statistically significant association of wheezing and exposure to LMW agents for all participants at baseline (1.11, 0.90 to 1.36), responders in 2018 (1.12, 0.83 to 1.53) and those lost to follow-up (1.07, 0.81 to 1.42).

CONCLUSION

The risk factors for loss to 5-year follow-up were comparable to those reported in other population-based studies and included younger age, male gender, current smoking, lower educational level and higher symptom prevalence and morbidity. We found that exposure to VGDF, irritating and LMW agents can be risk factors associated with loss to follow-up. Results suggest that loss to follow-up did not affect estimates of occupational exposure as a risk factor for respiratory symptoms.

Provision of educational events and subsequent questionnaire response rates in a large-scale birth cohort study from Japan

OBJECTIVES

We examined whether providing educational events for participants in a birth cohort study would increase the response rates of study questionnaires.

DESIGN

Birth cohort study.

SETTING

Questionnaires were distributed and returned by post twice in 1 year. We developed and implemented two educational sessions; a Baby Food lecture for mothers with children around 8 months old (analysis 1) and a Eurythmic session for mothers with children around 1 year and 8 months old (analysis 2). Mothers with children over the target ages were not invited (not-invited group). The invited participants were divided into three groups: those who did not apply to attend (not-applied group), those who applied but did not attend (applied group), and those who applied and attended (attended group).

PARTICIPANTS

The participants were 5379 mother-child pairs registered with the Toyama Regional Center of the Japan Environment and Children's Study (JECS).

OUTCOME MEASURE

The outcome measure was return of the JECS questionnaire for 1 year old sent out after the Baby Food lecture and the JECS questionnaire for 2 years old sent out after the Eurythmic session. The questionnaires were returned to us by post.

RESULTS

The response rate for the attended group of the Baby Food lecture was 99.7%, and the odds ratio (OR) was significantly higher for this group than for the not-invited group (crude OR 24.54; 95% confidence interval (CI) 3.42 to 176.13; analysis 1). After the exclusion of participants who had previously attended the Baby Food lecture, the response rate for the attended group of the Eurythmic session was 97.8%, and the OR was significantly higher for this group than for the not-invited group (adjusted OR 5.66; 95% CI 1.93 to 16.54; analysis 2).

CONCLUSION

Providing educational events that are appropriate to the age and needs of the participants may increase questionnaire response rates in birth cohort studies.

TRIAL REGISTRATION NUMBER

UMIN 000030786.

Incentives and participation in a medical survey

BACKGROUND Questionnaire surveys are important for surveying the health and disease behaviour of the population, but recent years have seen a fall in participation. Our study tested whether incentives can increase participation in these surveys.MATERIAL AND METHOD We sent a questionnaire on risk factors for colorectal cancer (height, weight, smoking, self-reported diagnoses, family medical history) to non-screened participants in a randomised colonoscopy screening study for colorectal cancer: participants who were invited but did not attend for colonoscopy examination (screening-invited) and persons who were not offered colonoscopy (control group). The persons were randomised to three groups: no financial incentive, lottery scratch cards included with the form, or a prize draw for a tablet computer when they responded to the form. We followed up all the incentive groups with telephone reminder calls, and before the prize draw for the tablet computer.RESULTS Altogether 3 705 of 6 795 persons (54.5  %) responded to the questionnaire; 43.5  % of those invited for screening and 65.6  % of the control group (p < 0.001). The proportion that answered was not influenced by incentives, either among those invited for screening (42.4  % in the non-prize group, 45.5  % in the lottery scratch card group and 42.6  % in the prize draw group; p = 0.24), or in the control group (65.6  % in the non-prize group, 66.4  % in the lottery scratch card group and 64.7  % in the prize draw group; p = 0.69). Prior to reminder calls, 39.2  % responded. A further 15.3  % responded following telephone reminder calls (14.1  % of the screening-invited and 16.5  % of the control group; p < 0.001).INTERPRETATION Incentives did not increase participation in this medical questionnaire survey. Use of telephone reminder calls and telephone interviews increased participation, but whether this is more effective than other methods requires further study.

SMS text pre-notification and delivery of reminder e-mails to increase response rates to postal questionnaires in the SUSPEND trial: a factorial design, randomised controlled trial

BACKGROUND

Patient-reported outcomes are vital in informing randomised controlled trials (RCTs) and health-care interventions and policies from the patient's perspective. However, participant non-response may introduce bias and can affect the generalisability of the trial. This study evaluates two interventions aimed at increasing response rates to postal questionnaires within a large, UK-wide RCT: pre-notification via short messenger service (SMS) text prior to sending the initial mailing of trial questionnaires versus no pre-notification; for non-responders to the initial mailing of the questionnaires, an e-mail reminder (containing a hyperlink to complete the questionnaire online) versus a postal reminder.

METHODS

This study is a 2 × 2 partial factorial design RCT nested within an RCT of medical expulsive therapy for ureteric stone disease. Participants who supplied a mobile telephone number were randomly assigned to receive an SMS text pre-notification of questionnaire delivery or no pre-notification. Those who supplied an e-mail address were randomly assigned to receive a questionnaire reminder by e-mail or post. Participants could be randomly assigned to the pre-notification comparison or the reminder comparison or both. The primary outcome measure was response rate at each questionnaire time point.

RESULTS

Four hundred eighteen participants were randomly assigned to the SMS pre-notification comparison (80% were male, and the mean age was 41 years with a standard deviation (SD) of 11.1). The intervention had no effect on response rate at either questionnaire time point. In subgroup analyses, SMS pre-notification increased response rates in women but only at the first questionnaire time point. One hundred nineteen participants were randomly assigned to the reminder comparison (80% were male, and the mean age was 42 years with an SD of 12.1). There was no difference in response rate in those who received an e-mail reminder compared with those who received a postal reminder.

CONCLUSIONS

SMS text pre-notification of questionnaire delivery and email delivery of questionnaire reminders did not improve response rates. There was some evidence to suggest that SMS text pre-notification may be effective in women, and further studies to investigate this may be warranted. E-mail reminders for participants to return their postal questionnaire could be advantageous given that response rates were similar following either type of reminder and the low cost of delivering an e-mail compared with a postal reminder. This is a substudy of the SUSPEND trial (ISCTRN69423238) (18 Nov. 2010).

Cost-effectiveness of health research study participant recruitment strategies: a systematic review

BACKGROUND

A large fraction of the cost of conducting clinical trials is allocated to recruitment of participants. A synthesis of findings from studies that evaluate the cost and effectiveness of different recruitment strategies will inform investigators in designing cost-efficient clinical trials.

PURPOSE

To systematically identify, assess, and synthesize evidence from published comparisons of the cost and yield of strategies for recruitment of participants to health research studies.

METHODS

We included randomized studies in which two or more strategies for recruitment of participants had been compared. We focused our economic evaluation on studies that randomized participants to different recruitment strategies.

RESULTS

We identified 10 randomized studies that compared recruitment strategies, including monetary incentives (cash or prize), direct contact (letters or telephone call), and medical referral strategies. Only two of the 10 studies compared strategies for recruiting participants to clinical trials. We found that allocating additional resources to recruit participants using monetary incentives or direct contact yielded between 4% and 23% additional participants compared to using neither strategy. For medical referral, recruitment of prostate cancer patients by nurses was cost-saving compared to recruitment by consultant urologists. For all underlying study designs, monetary incentives cost more than direct contact with potential participants, with a median incremental cost per recruitment ratio of Int$72 (Int$-International dollar, a theoretical unit of currency) for monetary incentive strategy compared to Int$28 for direct contact strategy. Only monetary incentives and source of referral were evaluated for recruiting participants into clinical trials.

LIMITATIONS

We did not review studies that presented non-monetary cost or lost opportunity cost. We did not adjust for the number of study recruitment sites or the study duration in our economic evaluation analysis.

CONCLUSIONS

Systematic and explicit reporting of cost and effectiveness of recruitment strategies from randomized comparisons is required to aid investigators to select cost-efficient strategies for recruiting participants to health research studies including clinical trials.