Evaluation of the effects of an offer of a monetary incentive on the rate of questionnaire return during follow-up of a clinical trial: a randomised study within a trial

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Self-administered questionnaires are widely used to collect data in epidemiological research, but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH METHODS

We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented.

SELECTION CRITERIA

Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre-defined criteria.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi2 test and quantified the degree of inconsistency between trial results using the I2 statistic.

MAIN RESULTS

Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follow-up (OR 1.69; 95% CI 0.93 to 3.06); using a more interesting/salient questionnaire (OR 1.73; 95% CI 1.12 to 2.66); and when mentioning an obligation to respond (OR 1.61; 95% CI 1.16 to 2.22). The odds of response also increased with: non-monetary incentives (OR 1.16; 95% CI 1.11 to 1.21; I2 = 80%); a larger monetary incentive (OR 1.24; 95% CI 1.15 to 1.33); a larger non-monetary incentive (OR 1.15; 95% CI 1.00 to 1.33); when a pen was included (OR 1.44; 95% CI 1.38 to 1.50); using personalised materials (OR 1.15; 95% CI 1.09 to 1.21; I2 = 57%); using a single-sided rather than a double-sided questionnaire (OR 1.13; 95% CI 1.02 to 1.25); using stamped return envelopes rather than franked return envelopes (OR 1.23; 95% CI 1.13 to 1.33; I2 = 69%), assuring confidentiality (OR 1.33; 95% CI 1.24 to 1.42); using first-class outward mailing (OR 1.11; 95% CI 1.02 to 1.21); and when questionnaires originated from a university (OR 1.32; 95% CI 1.13 to 1.54). The odds of response were reduced when the questionnaire included questions of a sensitive nature (OR 0.94; 95% CI 0.88 to 1.00). Electronic We found 88 eligible trials that evaluated over 30 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response tripled when: using a brief letter rather than a detailed letter (OR 3.26; 95% CI 1.79 to 5.94); and when a picture was included in an email (OR 3.05; 95% CI 1.84 to 5.06; I2 = 19%). The odds of response almost doubled when: using monetary incentives (OR 1.88; 95% CI 1.31 to 2.71; I2 = 79%); and using a more interesting topic (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by half when: using non-monetary incentives (OR 1.60; 95% CI 1.25 to 2.05); using shorter e-questionnaires (OR 1.51; 95% CI 1.06 to 2.16; I2 = 94%); and using a more interesting e-questionnaire (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third when: offering survey results as an incentive (OR 1.36; 95% CI 1.16 to 1.59); using a white background (OR 1.31; 95% CI 1.10 to 1.56); and when stressing the benefits to society of response (OR 1.38; 95% CI 1.07 to 1.78; I2 = 41%). The odds of response also increased with: personalised e-questionnaires (OR 1.24; 95% CI 1.17 to 1.32; I2 = 41%); using a simple header (OR 1.23; 95% CI 1.03 to 1.48); giving a deadline (OR 1.18; 95% CI 1.03 to 1.34); and by giving a longer time estimate for completion (OR 1.25; 95% CI 0.96 to 1.64). The odds of response were reduced when: "Survey" was mentioned in the e-mail subject (OR 0.81; 95% CI 0.67 to 0.97); when the email or the e-questionnaire was from a male investigator, or it included a male signature (OR 0.55; 95% CI 0.38 to 0.80); and by using university sponsorship (OR 0.84; 95%CI 0.69 to 1.01). The odds of response using a postal questionnaire were over twice those using an e-questionnaire (OR 2.33; 95% CI 2.25 to 2.42; I2 = 98%). Response also increased when: providing a choice of response mode (electronic or postal) rather than electronic only (OR 1.76 95% CI 1.67 to 1.85; I2 = 97%); and when administering the e-questionnaire by computer rather than by smartphone (OR 1.62 95% CI 1.36 to 1.94).

AUTHORS' CONCLUSIONS

Researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this Cochrane review.

Evaluation of the effectiveness of an incentive strategy on the questionnaire response rate in parents of premature babies: a randomised controlled Study Within A Trial (SWAT) nested within SIFT

Background

Loss to follow-up resulting in missing outcomes compromises the validity of trial results by reducing statistical power, negatively affecting generalisability and undermining assumptions made at analysis, leading to potentially biased and misleading results. Evidence that incentives are effective at improving response rates exists, but there is little evidence regarding the best approach, especially in the field of perinatal medicine. The NIHR-funded SIFT trial follow-up of infants at 2 years of age provided an ideal opportunity to address this remaining uncertainty.

Methods

Participants: parents of infants from participating neonatal units in the UK and Ireland followed up for SIFT (multicentre RCT investigating two speeds of feeding in babies with gestational age at birth < 32 weeks and/or birthweight < 1500 g). Interventions: parents were randomly allocated to receive incentives (£15 gift voucher) before or after questionnaire return. The objective was to establish whether offering an unconditional incentive in advance or promising an incentive on completion of a questionnaire (conditional) improved the response rate in parents of premature babies. The primary outcome was questionnaire response rate. Permuted block randomisation was performed (variable size blocks), stratified by SIFT allocation (slower/faster feeds) and single/multiple birth. Multiple births were given the same incentives allocation. Parents were unaware that they were in an incentives SWAT; SIFT office staff were not blinded to allocation.

Results

Parents of 923 infants were randomised: 459 infants allocated to receive incentive before, 464 infants allocated to receive incentive after; analysis was by intention to treat. Allocation to the incentive before completion led to a significantly higher response rate, 83.0% (381/459) compared to the after-completion group, 76.1% (353/464); adjusted absolute difference of 6.8% (95% confidence interval 1.6% to 12.0%). Giving an incentive in advance is the more costly approach, but the mean difference of ~£3 per infant is small given the higher return.

Conclusions

An unconditional incentive in advance led to a significantly higher response rate compared to the promise of an incentive on completion. Against a backdrop of falling response rates to questionnaires, incentives can be an effective way to increase returns.

Trial registration

SIFT (ISRCTN76463425). Registered on March 5, 2013.; SWAT registration (SWAT 69 available from http://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/FileStore/Filetoupload,864297,en.pdf). Registered on June 27, 2016.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05515-y.

Strategies to improve retention in randomised trials

BACKGROUND

Poor retention of participants in randomised trials can lead to missing outcome data which can introduce bias and reduce study power, affecting the generalisability, validity and reliability of results. Many strategies are used to improve retention but few have been formally evaluated.

OBJECTIVES

To quantify the effect of strategies to improve retention of participants in randomised trials and to investigate if the effect varied by trial setting.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Scopus, PsycINFO, CINAHL, Web of Science Core Collection (SCI-expanded, SSCI, CPSI-S, CPCI-SSH and ESCI) either directly with a specified search strategy or indirectly through the ORRCA database. We also searched the SWAT repository to identify ongoing or recently completed retention trials. We did our most recent searches in January 2020.

SELECTION CRITERIA

We included eligible randomised or quasi-randomised trials of evaluations of strategies to increase retention that were embedded in 'host' randomised trials from all disease areas and healthcare settings. We excluded studies aiming to increase treatment compliance.

DATA COLLECTION AND ANALYSIS

We extracted data on: the retention strategy being evaluated; location of study; host trial setting; method of randomisation; numbers and proportions in each intervention and comparator group. We used a risk difference (RD) and 95% confidence interval (CI) to estimate the effectiveness of the strategies to improve retention. We assessed heterogeneity between trials. We applied GRADE to determine the certainty of the evidence within each comparison.

MAIN RESULTS

We identified 70 eligible papers that reported data from 81 retention trials. We included 69 studies with more than 100,000 participants in the final meta-analyses, of which 67 studies evaluated interventions aimed at trial participants and two evaluated interventions aimed at trial staff involved in retention. All studies were in health care and most aimed to improve postal questionnaire response. Interventions were categorised into broad comparison groups: Data collection; Participants; Sites and site staff; Central study management; and Study design. These intervention groups consisted of 52 comparisons, none of which were supported by high-certainty evidence as determined by GRADE assessment. There were four comparisons presenting moderate-certainty evidence, three supporting retention (self-sampling kits, monetary reward together with reminder or prenotification and giving a pen at recruitment) and one reducing retention (inclusion of a diary with usual follow-up compared to usual follow-up alone). Of the remaining studies, 20 presented GRADE low-certainty evidence and 28 presented very low-certainty evidence. Our findings do provide a priority list for future replication studies, especially with regard to comparisons that currently rely on a single study.

AUTHORS' CONCLUSIONS

Most of the interventions we identified aimed to improve retention in the form of postal questionnaire response. There were few evaluations of ways to improve participants returning to trial sites for trial follow-up. None of the comparisons are supported by high-certainty evidence. Comparisons in the review where the evidence certainty could be improved with the addition of well-done studies should be the focus for future evaluations.

Two-by-two factorial randomised study within a trial (SWAT) to evaluate strategies for follow-up in a randomised prevention trial

BACKGROUND

Failure to collect outcome data in randomised trials can result in bias and loss of statistical power. Further evaluations of strategies to increase retention are required. We assessed the effectiveness of two strategies for retention in a randomised prevention trial using a two-by-two factorial randomised study within a trial (SWAT).

METHODS

Parents of babies included in the host trial were randomised to (1) short message service (SMS) notification prior to sending questionnaires at 3, 6, 12 and 18 months versus no SMS notification and (2) a £10 voucher sent with the invitation letter for the primary follow-up visit at 24 months or given at the visit. The two co-primary outcomes were collection of host trial (1) questionnaire data at interim follow-up times and (2) primary outcome at 24 months during a home/clinic visit with a research nurse.

RESULTS

Between November 2014 and November 2016, 1394 participants were randomised: 350 to no SMS + voucher at visit, 345 to SMS + voucher at visit, 352 to no SMS + voucher before visit and 347 to SMS + voucher before visit. Overall questionnaire data was collected at interim follow-up times for 75% in both the group allocated to the prior SMS notification and the group allocated to no SMS notification (odds ratio (OR) SMS versus none 1.02, 95% CI 0.83 to 1.25). Host trial primary outcome data was collected at a visit for 557 (80%) allocated to the voucher before the visit in the invitation letter and for 566 (81%) whose parents were allocated to receive the voucher at the visit (OR before versus at visit 0.89, 95% CI 0.69 to 1.17).

CONCLUSION

There was no evidence of a difference in retention according to SMS notification or voucher timing. Future synthesis of SWAT results is required to be able to detect small but important incremental effects of retention strategies.

TRIAL REGISTRATION

ISRCTN registry, ID: ISRCTN21528841. Registered on 25 July 2014. SWAT Repository Store ID 25.

Unconditional and conditional monetary incentives to increase response to mailed questionnaires: A randomized controlled study within a trial (SWAT)

RATIONALE, AIMS, AND OBJECTIVES

High response rates to research questionnaires can help to ensure results are more representative of the population studied and provide increased statistical power, on which the study may have been predicated. Improving speed and quality of response can reduce costs.

METHOD

We conducted a randomized study within a trial (SWAT) to assess questionnaire response rates, reminders sent, and data completeness with unconditional compared with conditional monetary incentives. Eligible individuals were mailed a series of psychological questionnaires as a follow-up to a baseline host trial questionnaire. Half received a £5 gift voucher with questionnaires (unconditional), and half were promised the voucher after returning questionnaires (conditional).

RESULTS

Of 1079 individuals, response rates to the first follow-up questionnaire were 94.2% and 91.7% in the unconditional and conditional monetary incentive groups, respectively (OR 1.78; 95% CI, 0.85-3.72). There were significantly greater odds of returning repeat questionnaires in the unconditional group at 6 months (OR 2.97; 95% CI, 1.01-8.71; .047) but not at 12 months (OR 1.12; 95% CI, 0.44-2.85). Incentive condition had no impact at any time point on the proportion of sent questionnaires that needed reminders. Odds of incomplete questionnaires were significantly greater at 3 months in the unconditional compared with the conditional incentive group (OR 2.45; 95% CI, 1.32-4.55; .004).

CONCLUSIONS

Unconditional monetary incentives can produce a transitory greater likelihood of mailed questionnaire response in a clinical trial participant group, consistent with the direction of effect in other settings. However, this could have been a chance finding. The use of multiple strategies to promote response may have created a ceiling effect. This strategy has potential to reduce administrative and postage costs, weighed against the cost of incentives used, but could risk compromising the completeness of data.

Two speeds of increasing milk feeds for very preterm or very low-birthweight infants: the SIFT RCT

BACKGROUND

Observational data suggest that slowly advancing enteral feeds in preterm infants may reduce necrotising enterocolitis but increase late-onset sepsis. The Speed of Increasing milk Feeds Trial (SIFT) compared two rates of feed advancement.

OBJECTIVE

To determine if faster (30 ml/kg/day) or slower (18 ml/kg/day) daily feed increments improve survival without moderate or severe disability and other morbidities in very preterm or very low-birthweight infants.

DESIGN

This was a multicentre, two-arm, parallel-group, randomised controlled trial. Randomisation was via a web-hosted minimisation algorithm. It was not possible to safely and completely blind caregivers and parents.

SETTING

The setting was 55 UK neonatal units, from May 2013 to June 2015.

PARTICIPANTS

The participants were infants born at < 32 weeks' gestation or a weight of < 1500 g, who were receiving < 30 ml/kg/day of milk at trial enrolment.

INTERVENTIONS

When clinicians were ready to start advancing feed volumes, the infant was randomised to receive daily feed increments of either 30 ml/kg/day or 18 ml/kg/day. In total, 1400 infants were allocated to fast feeds and 1404 infants were allocated to slow feeds.

MAIN OUTCOME MEASURES

The primary outcome was survival without moderate or severe neurodevelopmental disability at 24 months of age, corrected for gestational age. The secondary outcomes were mortality; moderate or severe neurodevelopmental disability at 24 months corrected for gestational age; death before discharge home; microbiologically confirmed or clinically suspected late-onset sepsis; necrotising enterocolitis (Bell's stage 2 or 3); time taken to reach full milk feeds (tolerating 150 ml/kg/day for 3 consecutive days); growth from birth to discharge; duration of parenteral feeding; time in intensive care; duration of hospital stay; diagnosis of cerebral palsy by a doctor or other health professional; and individual components of the definition of moderate or severe neurodevelopmental disability.

RESULTS

The results showed that survival without moderate or severe neurodevelopmental disability at 24 months occurred in 802 out of 1224 (65.5%) infants allocated to faster increments and 848 out of 1246 (68.1%) infants allocated to slower increments (adjusted risk ratio 0.96, 95% confidence interval 0.92 to 1.01). There was no significant difference between groups in the risk of the individual components of the primary outcome or in the important hospital outcomes: late-onset sepsis (adjusted risk ratio 0.96, 95% confidence interval 0.86 to 1.07) or necrotising enterocolitis (adjusted risk ratio 0.88, 95% confidence interval 0.68 to 1.16). Cost-consequence analysis showed that the faster feed increment rate was less costly but also less effective than the slower rate in terms of achieving the primary outcome, so was therefore found to not be cost-effective. Four unexpected serious adverse events were reported, two in each group. None was assessed as being causally related to the intervention.

LIMITATIONS

The study could not be blinded, so care may have been affected by knowledge of allocation. Although well powered for comparisons of all infants, subgroup comparisons were underpowered.

CONCLUSIONS

No clear advantage was identified for the important outcomes in very preterm or very low-birthweight infants when milk feeds were advanced in daily volume increments of 30 ml/kg/day or 18 ml/kg/day. In terms of future work, the interaction of different milk types with increments merits further examination, as may different increments in infants at the extremes of gestation or birthweight.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN76463425.

FUNDING

This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 18. See the NIHR Journals Library website for further project information.

Effect of subgroup-specific multimodal therapy on chronic spinal back pain and function-a prospective inpatient multicentre clinical trial in Germany

Treatment modalities of spinal pain patients are discussed diversely, and different multimodal therapy programs have been developed. Purpose of the present study was to evaluate therapy outcome and effectiveness of an inpatient interdisciplinary and multimodal treatment program.This prospective multicentre clinical trial has been performed with patients from orthopedic hospitals receiving a functional musculoskeletal therapy pathway. Outcome measures were pain intensity and back-specific function (Oswestry Disability Index) before (T1) and after the intervention (T2) as well as after 6 and 12 months (T3, T4). Statistical approach included parametric (t test) and nonparametric (Wilcoxon-test) tests and the calculation of effect sizes. Additionally, a statistical subgroup analysis based on selected parameters (degree of pain chronicity, gender, and age) was performed using linear mixed models.In total, 249 patients (42.6% men, 57.4% women) with spinal pain were included, 133 patients were accessible for follow-up at T3 and 106 patients at T4.Average pain (AP) reduced significantly (P <.001) from T1 to T4 with an effect size of 0.99. Back-specific function also improved (P <.001) over all measuring time points (TP) (effect size: 0.63). Furthermore, the statistical subgroup analysis demonstrated the efficacy of the treatment concept within the subgroup parameters chronicity degree and age.A functional musculoskeletal therapy pathway including treatment of musculoskeletal dysfunctions appears to be beneficial in terms of treating pain and function. Pain chronicity and age seems to be factors influencing therapy outcome. Further studies are needed to examine the superiority of these inpatient programs for back pain including control groups.

A multicentre, randomised controlled trial of position during the late stages of labour in nulliparous women with an epidural: clinical effectiveness and an economic evaluation (BUMPES)

BACKGROUND

Epidural analgesia leads to increased risk of instrumental vaginal delivery (IVD). There is debate about whether or not posture in second-stage labour influences the incidence of spontaneous vaginal birth (SVB).

OBJECTIVES

In nulliparous women with epidural analgesia, does a policy of adopting an 'upright position' throughout second-stage labour increase the incidence of SVB compared with a policy of adopting a 'lying-down' position?

DESIGN

Two-arm randomised controlled trial.

SETTING

Maternity units in England and Wales.

PARTICIPANTS

Nulliparous women aged ≥ 16 years, at ≥ 37 weeks' gestation with singleton cephalic presentation and intended SVB, in second-stage labour with an epidural providing effective pain relief.

INTERVENTIONS

(1) Upright position to maintain the pelvis in as vertical a plane as possible; and (2) lying-down position to maintain the pelvis in as horizontal a plane as possible.

MAIN OUTCOME MEASURES

The primary outcome measure was incidence of SVB. Secondary outcomes included augmentation, interventions to maintain blood pressure, duration of labour, episiotomy, genital tract trauma, post-partum haemorrhage, maternal satisfaction, neonatal metabolic acidosis, 5-minute Apgar score of < 4, resuscitation at birth and admission to neonatal unit. At 1 year for (1) women: urinary or faecal incontinence, dyspareunia and health-related quality of life; (2) for infants: major morbidity. A cost-consequences analysis with a time horizon of 1 year after the birth from a NHS perspective.

RESULTS

Between October 2010 and January 2014, 3236 women were randomised from 41 centres in England and Wales. There was a statistically significant difference in the incidence of SVB between groups, with 35.2% of women achieving a SVB in the upright group, compared with 41.1% in the lying-down group (adjusted risk ratio 0.86, 95% confidence interval 0.78 to 0.94). There was no evidence of differences in most of the secondary maternal or neonatal outcomes, or in long-term outcomes at the 12-month follow-up. No significant overall cost differences were observed between upright and lying-down positions for mothers or their babies.

LIMITATIONS

Measurement of adherence was challenging in this unmasked trial, and adherence could be influenced by midwives' beliefs about the allocated positions. If adherence was poor, this would have diluted the difference between the two groups.

CONCLUSIONS

There is clear evidence of the benefit of adopting a lying-down position in second-stage labour in nulliparous women with epidural analgesia, with no apparent disadvantages in either short- or long-term outcomes for mother or baby, and this is cost neutral for the NHS.

FUTURE WORK

Questions remain about whether or not other positions could increase the incidence of SVB further in this group of women. The results also raise questions about the role of maternal position in second-stage labour in women without an epidural.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN35706297.

FUNDING

This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in Health Technology Assessment, Vol 21, No. 65. See the NIHR Journals Library website for further project information.

Recommendations for Randomised Trials in Surgical Oncology

Trials of surgical procedures in the treatment of malignant disease face a unique set of challenges. This review aimed to describe recommendations for the design, delivery and reporting of randomised trials in surgical oncology. A literature search was carried out without date limits to identify articles related to trial methodology research in surgery and surgical oncology. A narrative review was framed around two open National Institute of Health Research portfolio trials in colon and rectal cancer: the STAR-TREC trial (ISRCTN14240288) and the ROCCS trial (ISRCTN46330337). Twelve specific challenges were highlighted: standardisation of technique; pilot and feasibility studies; balancing treatments; the recruitment pathway; outcome measures; patient and public representation; trainee-led networks; randomisation; novel techniques and training; learning curves; blinding; follow-up. Evidence-based recommendations were made for the future design and conduct of surgical oncology trials. Better understanding of the challenges facing trials in the surgical treatment of cancer will accelerate high-quality evaluation and rapid adoption of innovation for the benefit of patient care.

Best practice guidance for the use of strategies to improve retention in randomized trials developed from two consensus workshops

Objectives

To develop best practice guidance for the use of retention strategies in randomized clinical trials (RCTs).

Study Design and Setting

Consensus development workshops conducted at two UK Clinical Trials Units. Sixty-six statisticians, clinicians, RCT coordinators, research scientists, research assistants, and data managers associated with RCTs participated. The consensus development workshops were based on the consensus development conference method used to develop best practice for treatment of medical conditions. Workshops commenced with a presentation of the evidence for incentives, communication, questionnaire format, behavioral, case management, and methodological retention strategies identified by a Cochrane review and associated qualitative study. Three simultaneous group discussions followed focused on (1) how convinced the workshop participants were by the evidence for retention strategies, (2) barriers to the use of effective retention strategies, (3) types of RCT follow-up that retention strategies could be used for, and (4) strategies for future research. Summaries of each group discussion were fed back to the workshop. Coded content for both workshops was compared for agreement and disagreement. Agreed consensus on best practice guidance for retention was identified.

Results

Workshop participants agreed best practice guidance for the use of small financial incentives to improve response to postal questionnaires in RCTs. Use of second-class post was thought to be adequate for postal communication with RCT participants. The most relevant validated questionnaire was considered best practice for collecting RCT data. Barriers identified for the use of effective retention strategies were: the small improvements seen in questionnaire response for the addition of monetary incentives, and perceptions among trialists that some communication strategies are outdated. Furthermore, there was resistance to change existing retention practices thought to be effective. Face-to-face and electronic follow-up technologies were identified as retention strategies for further research.

Conclusions

We developed best practice guidance for the use of retention strategies in RCTs and identified potential barriers to the use of effective strategies. The extent of agreement on best practice is limited by the variability in the currently available evidence. This guidance will need updating as new retention strategies are developed and evaluated.

The effectiveness of a monetary incentive offer on survey response rates and response completeness in a longitudinal study

Background

Achieving adequate response rates is an ongoing challenge for longitudinal studies. The World Trade Center Health Registry is a longitudinal health study that periodically surveys a cohort of ~71,000 people exposed to the 9/11 terrorist attacks in New York City. Since Wave 1, the Registry has conducted three follow-up surveys (Waves 2–4) every 3–4 years and utilized various strategies to increase survey participation. A promised monetary incentive was offered for the first time to survey non-respondents in the recent Wave 4 survey, conducted 13–14 years after 9/11.

Methods

We evaluated the effectiveness of a monetary incentive in improving the response rate five months after survey launch, and assessed whether or not response completeness was compromised due to incentive use. The study compared the likelihood of returning a survey for those who received an incentive offer to those who did not, using logistic regression models. Among those who returned surveys, we also examined whether those receiving an incentive notification had higher rate of response completeness than those who did not, using negative binomial regression models and logistic regression models.

Results

We found that a $10 monetary incentive offer was effective in increasing Wave 4 response rates. Specifically, the $10 incentive offer was useful in encouraging initially reluctant participants to respond to the survey. The likelihood of returning a survey increased by 30% for those who received an incentive offer (AOR = 1.3, 95% CI: 1.1, 1.4), and the incentive increased the number of returned surveys by 18%. Moreover, our results did not reveal any significant differences on response completeness between those who received an incentive offer and those who did not.

Conclusions

In the face of the growing challenge of maintaining a high response rate for the World Trade Center Health Registry follow-up surveys, this study showed the value of offering a monetary incentive as an additional refusal conversion strategy. Our findings also suggest that an incentive offer could be particularly useful near the end of data collection period when an immediate boost in response rate is needed.