A randomized trial of the total design method for the postal follow-up of women in a cancer prevention trial

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Self-administered questionnaires are widely used to collect data in epidemiological research, but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH METHODS

We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented.

SELECTION CRITERIA

Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre-defined criteria.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi2 test and quantified the degree of inconsistency between trial results using the I2 statistic.

MAIN RESULTS

Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follow-up (OR 1.69; 95% CI 0.93 to 3.06); using a more interesting/salient questionnaire (OR 1.73; 95% CI 1.12 to 2.66); and when mentioning an obligation to respond (OR 1.61; 95% CI 1.16 to 2.22). The odds of response also increased with: non-monetary incentives (OR 1.16; 95% CI 1.11 to 1.21; I2 = 80%); a larger monetary incentive (OR 1.24; 95% CI 1.15 to 1.33); a larger non-monetary incentive (OR 1.15; 95% CI 1.00 to 1.33); when a pen was included (OR 1.44; 95% CI 1.38 to 1.50); using personalised materials (OR 1.15; 95% CI 1.09 to 1.21; I2 = 57%); using a single-sided rather than a double-sided questionnaire (OR 1.13; 95% CI 1.02 to 1.25); using stamped return envelopes rather than franked return envelopes (OR 1.23; 95% CI 1.13 to 1.33; I2 = 69%), assuring confidentiality (OR 1.33; 95% CI 1.24 to 1.42); using first-class outward mailing (OR 1.11; 95% CI 1.02 to 1.21); and when questionnaires originated from a university (OR 1.32; 95% CI 1.13 to 1.54). The odds of response were reduced when the questionnaire included questions of a sensitive nature (OR 0.94; 95% CI 0.88 to 1.00). Electronic We found 88 eligible trials that evaluated over 30 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response tripled when: using a brief letter rather than a detailed letter (OR 3.26; 95% CI 1.79 to 5.94); and when a picture was included in an email (OR 3.05; 95% CI 1.84 to 5.06; I2 = 19%). The odds of response almost doubled when: using monetary incentives (OR 1.88; 95% CI 1.31 to 2.71; I2 = 79%); and using a more interesting topic (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by half when: using non-monetary incentives (OR 1.60; 95% CI 1.25 to 2.05); using shorter e-questionnaires (OR 1.51; 95% CI 1.06 to 2.16; I2 = 94%); and using a more interesting e-questionnaire (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third when: offering survey results as an incentive (OR 1.36; 95% CI 1.16 to 1.59); using a white background (OR 1.31; 95% CI 1.10 to 1.56); and when stressing the benefits to society of response (OR 1.38; 95% CI 1.07 to 1.78; I2 = 41%). The odds of response also increased with: personalised e-questionnaires (OR 1.24; 95% CI 1.17 to 1.32; I2 = 41%); using a simple header (OR 1.23; 95% CI 1.03 to 1.48); giving a deadline (OR 1.18; 95% CI 1.03 to 1.34); and by giving a longer time estimate for completion (OR 1.25; 95% CI 0.96 to 1.64). The odds of response were reduced when: "Survey" was mentioned in the e-mail subject (OR 0.81; 95% CI 0.67 to 0.97); when the email or the e-questionnaire was from a male investigator, or it included a male signature (OR 0.55; 95% CI 0.38 to 0.80); and by using university sponsorship (OR 0.84; 95%CI 0.69 to 1.01). The odds of response using a postal questionnaire were over twice those using an e-questionnaire (OR 2.33; 95% CI 2.25 to 2.42; I2 = 98%). Response also increased when: providing a choice of response mode (electronic or postal) rather than electronic only (OR 1.76 95% CI 1.67 to 1.85; I2 = 97%); and when administering the e-questionnaire by computer rather than by smartphone (OR 1.62 95% CI 1.36 to 1.94).

AUTHORS' CONCLUSIONS

Researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this Cochrane review.

Strategies to improve retention in randomised trials

BACKGROUND

Poor retention of participants in randomised trials can lead to missing outcome data which can introduce bias and reduce study power, affecting the generalisability, validity and reliability of results. Many strategies are used to improve retention but few have been formally evaluated.

OBJECTIVES

To quantify the effect of strategies to improve retention of participants in randomised trials and to investigate if the effect varied by trial setting.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Scopus, PsycINFO, CINAHL, Web of Science Core Collection (SCI-expanded, SSCI, CPSI-S, CPCI-SSH and ESCI) either directly with a specified search strategy or indirectly through the ORRCA database. We also searched the SWAT repository to identify ongoing or recently completed retention trials. We did our most recent searches in January 2020.

SELECTION CRITERIA

We included eligible randomised or quasi-randomised trials of evaluations of strategies to increase retention that were embedded in 'host' randomised trials from all disease areas and healthcare settings. We excluded studies aiming to increase treatment compliance.

DATA COLLECTION AND ANALYSIS

We extracted data on: the retention strategy being evaluated; location of study; host trial setting; method of randomisation; numbers and proportions in each intervention and comparator group. We used a risk difference (RD) and 95% confidence interval (CI) to estimate the effectiveness of the strategies to improve retention. We assessed heterogeneity between trials. We applied GRADE to determine the certainty of the evidence within each comparison.

MAIN RESULTS

We identified 70 eligible papers that reported data from 81 retention trials. We included 69 studies with more than 100,000 participants in the final meta-analyses, of which 67 studies evaluated interventions aimed at trial participants and two evaluated interventions aimed at trial staff involved in retention. All studies were in health care and most aimed to improve postal questionnaire response. Interventions were categorised into broad comparison groups: Data collection; Participants; Sites and site staff; Central study management; and Study design. These intervention groups consisted of 52 comparisons, none of which were supported by high-certainty evidence as determined by GRADE assessment. There were four comparisons presenting moderate-certainty evidence, three supporting retention (self-sampling kits, monetary reward together with reminder or prenotification and giving a pen at recruitment) and one reducing retention (inclusion of a diary with usual follow-up compared to usual follow-up alone). Of the remaining studies, 20 presented GRADE low-certainty evidence and 28 presented very low-certainty evidence. Our findings do provide a priority list for future replication studies, especially with regard to comparisons that currently rely on a single study.

AUTHORS' CONCLUSIONS

Most of the interventions we identified aimed to improve retention in the form of postal questionnaire response. There were few evaluations of ways to improve participants returning to trial sites for trial follow-up. None of the comparisons are supported by high-certainty evidence. Comparisons in the review where the evidence certainty could be improved with the addition of well-done studies should be the focus for future evaluations.

Evaluation of strategies for increasing response rates to postal questionnaires in quality control of nasal septal surgery

Background

Postal questionnaires are often used to assess the results of nasal septoplasty, but response rates vary widely. This study assesses strategies designed to increase the response rate.

Methods

Postoperative questionnaires using visual analogue scales (VAS) for nasal obstruction were mailed to 160 consecutive patients alternately allocated to one of two groups. Group A received the questionnaire in the usual manner and group B received a modified cover letter with hand-written name and signature and a hand-stamped return envelope.

Results

Of the 80 patients in each group, 47 (58.8%) in group A and 54 (67.5%) in group B returned the questionnaire (p = 0.25). There were no age or gender differences between the groups, nor did the pre- and postoperative VAS scores differ between the groups.

Conclusion

The strategies used in this study increased the response rate to postal questionnaires by 8.7% points, but this was not a statistically significant or clinically meaningful improvement.

Best practice guidance for the use of strategies to improve retention in randomized trials developed from two consensus workshops

Objectives

To develop best practice guidance for the use of retention strategies in randomized clinical trials (RCTs).

Study Design and Setting

Consensus development workshops conducted at two UK Clinical Trials Units. Sixty-six statisticians, clinicians, RCT coordinators, research scientists, research assistants, and data managers associated with RCTs participated. The consensus development workshops were based on the consensus development conference method used to develop best practice for treatment of medical conditions. Workshops commenced with a presentation of the evidence for incentives, communication, questionnaire format, behavioral, case management, and methodological retention strategies identified by a Cochrane review and associated qualitative study. Three simultaneous group discussions followed focused on (1) how convinced the workshop participants were by the evidence for retention strategies, (2) barriers to the use of effective retention strategies, (3) types of RCT follow-up that retention strategies could be used for, and (4) strategies for future research. Summaries of each group discussion were fed back to the workshop. Coded content for both workshops was compared for agreement and disagreement. Agreed consensus on best practice guidance for retention was identified.

Results

Workshop participants agreed best practice guidance for the use of small financial incentives to improve response to postal questionnaires in RCTs. Use of second-class post was thought to be adequate for postal communication with RCT participants. The most relevant validated questionnaire was considered best practice for collecting RCT data. Barriers identified for the use of effective retention strategies were: the small improvements seen in questionnaire response for the addition of monetary incentives, and perceptions among trialists that some communication strategies are outdated. Furthermore, there was resistance to change existing retention practices thought to be effective. Face-to-face and electronic follow-up technologies were identified as retention strategies for further research.

Conclusions

We developed best practice guidance for the use of retention strategies in RCTs and identified potential barriers to the use of effective strategies. The extent of agreement on best practice is limited by the variability in the currently available evidence. This guidance will need updating as new retention strategies are developed and evaluated.

Increasing follow-up questionnaire response rates in a randomized controlled trial of telehealth for depression: three embedded controlled studies

BACKGROUND

Attrition is problematic in trials, and may be exacerbated in longer studies, telehealth trials and participants with depression - three features of The Healthlines Study. Advance notification, including a photograph and using action-oriented email subject lines might increase response rates, but require further investigation. We examined the effectiveness of these interventions in three embedded Healthlines studies.

METHODS

Based in different trial sites, participants with depression were alternately allocated to be pre-called or not ahead of the 8-month follow-up questionnaire (Study 1), randomized to receive a research team photograph or not with their 12-month questionnaire (Study 2), and randomized to receive an action-oriented ('ACTION REQUIRED') or standard ('Questionnaire reminder') 12-month email reminder (Study 3). Participants could complete online or postal questionnaires, and received up to five questionnaire reminders. The primary outcome was completion of the Patient Health Questionnaire (PHQ-9). Secondary outcome measures were the number of reminders and time to questionnaire completion.

RESULTS

Of a total of 609 Healthlines depression participants, 190, 251 and 231 participants were included in Studies 1-3 (intervention: 95, 126 and 115), respectively. Outcome completion was ≥90 % across studies, with no differences between trial arms (Study 1: OR 0.38, 95 % CI 0.07-2.10; Study 2: OR 0.84, 95 % CI 0.26-2.66; Study 3: OR 0.53 95 % CI 0.19-1.49). Pre-called participants were less likely to require a reminder (48.4 % vs 62.1 %, OR 0.41, 95 % CI 0.21-0.78), required fewer reminders (adjusted difference in means -0.67, 95 % CI -1.13 to -0.20), and completed follow-up quicker (median 8 vs 15 days, HR 1.35, 95 % CI 1.00-1.82) than control subjects. There were no significant between-group differences in Studies 2 or 3.

CONCLUSIONS

Eventual response rates in this trial were high, with no further improvement from these interventions. While the photograph and email interventions were ineffective, pre-calling participants reduced time to completion. This strategy might be helpful when the timing of study completion is important. Researchers perceived a substantial benefit from the reduction in reminders with pre-calling, despite no overall decrease in net effort after accounting for pre-notification.

TRIAL REGISTRATION

Current Clinical Trials ISRCTN14172341.

Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis

OBJECTIVE

To quantify the effect of strategies to improve retention in randomised trials.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Sources searched: MEDLINE, EMBASE, PsycINFO, DARE, CENTRAL, CINAHL, C2-SPECTR, ERIC, PreMEDLINE, Cochrane Methodology Register, Current Controlled Trials metaRegister, WHO trials platform, Society for Clinical Trials (SCT) conference proceedings and a survey of all UK clinical trial research units.

REVIEW METHODS

Included trials were randomised evaluations of strategies to improve retention embedded within host randomised trials. The primary outcome was retention of trial participants. Data from trials were pooled using the fixed-effect model. Subgroup analyses were used to explore the heterogeneity and to determine whether there were any differences in effect by the type of strategy.

RESULTS

38 retention trials were identified. Six broad types of strategies were evaluated. Strategies that increased postal questionnaire responses were: adding, that is, giving a monetary incentive (RR 1.18; 95% CI 1.09 to 1.28) and higher valued incentives (RR 1.12; 95% CI 1.04 to 1.22). Offering a monetary incentive, that is, an incentive given on receipt of a completed questionnaire, also increased electronic questionnaire response (RR 1.25; 95% CI 1.14 to 1.38). The evidence for shorter questionnaires (RR 1.04; 95% CI 1.00 to 1.08) and questionnaires relevant to the disease/condition (RR 1.07; 95% CI 1.01 to 1.14) is less clear. On the basis of the results of single trials, the following strategies appeared effective at increasing questionnaire response: recorded delivery of questionnaires (RR 2.08; 95% CI 1.11 to 3.87); a 'package' of postal communication strategies (RR 1.43; 95% CI 1.22 to 1.67) and an open trial design (RR 1.37; 95% CI 1.16 to 1.63). There is no good evidence that the following strategies impact on trial response/retention: adding a non-monetary incentive (RR=1.00; 95% CI 0.98 to 1.02); offering a non-monetary incentive (RR=0.99; 95% CI 0.95 to 1.03); 'enhanced' letters (RR=1.01; 95% CI 0.97 to 1.05); monetary incentives compared with offering prize draw entry (RR=1.04; 95% CI 0.91 to 1.19); priority postal delivery (RR=1.02; 95% CI 0.95 to 1.09); behavioural motivational strategies (RR=1.08; 95% CI 0.93 to 1.24); additional reminders to participants (RR=1.03; 95% CI 0.99 to 1.06) and questionnaire question order (RR=1.00, 0.97 to 1.02). Also based on single trials, these strategies do not appear effective: a telephone survey compared with a monetary incentive plus questionnaire (RR=1.08; 95% CI 0.94 to 1.24); offering a charity donation (RR=1.02, 95% CI 0.78 to 1.32); sending sites reminders (RR=0.96; 95% CI 0.83 to 1.11); sending questionnaires early (RR=1.10; 95% CI 0.96 to 1.26); longer and clearer questionnaires (RR=1.01, 0.95 to 1.07) and participant case management by trial assistants (RR=1.00; 95% CI 0.97 to 1.04).

CONCLUSIONS

Most of the trials evaluated questionnaire response rather than ways to improve participants return to site for follow-up. Monetary incentives and offers of monetary incentives increase postal and electronic questionnaire response. Some strategies need further evaluation. Application of these results would depend on trial context and follow-up procedures.

Use of strategies to improve retention in primary care randomised trials: a qualitative study with in-depth interviews

OBJECTIVE

To explore the strategies used to improve retention in primary care randomised trials.

DESIGN

Qualitative in-depth interviews and thematic analysis.

PARTICIPANTS

29 UK primary care chief and principal investigators, trial managers and research nurses.

METHODS

In-depth face-to-face interviews.

RESULTS

Primary care researchers use incentive and communication strategies to improve retention in trials, but were unsure of their effect. Small monetary incentives were used to increase response to postal questionnaires. Non-monetary incentives were used although there was scepticism about the impact of these on retention. Nurses routinely used telephone communication to encourage participants to return for trial follow-up. Trial managers used first class post, shorter questionnaires and improved questionnaire designs with the aim of improving questionnaire response. Interviewees thought an open trial design could lead to biased results and were negative about using behavioural strategies to improve retention. There was consensus among the interviewees that effective communication and rapport with participants, participant altruism, respect for participant's time, flexibility of trial personnel and appointment schedules and trial information improve retention. Interviewees noted particular challenges with retention in mental health trials and those involving teenagers.

CONCLUSIONS

The findings of this qualitative study have allowed us to reflect on research practice around retention and highlight a gap between such practice and current evidence. Interviewees describe acting from experience without evidence from the literature, which supports the use of small monetary incentives to improve the questionnaire response. No such evidence exists for non-monetary incentives or first class post, use of which may need reconsideration. An exploration of barriers and facilitators to retention in other research contexts may be justified.

Strategies to improve retention in randomised trials

BACKGROUND

Loss to follow-up from randomised trials can introduce bias and reduce study power, affecting the generalisability, validity and reliability of results. Many strategies are used to reduce loss to follow-up and improve retention but few have been formally evaluated.

OBJECTIVES

To quantify the effect of strategies to improve retention on the proportion of participants retained in randomised trials and to investigate if the effect varied by trial strategy and trial setting.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, PreMEDLINE, EMBASE, PsycINFO, DARE, CINAHL, Campbell Collaboration's Social, Psychological, Educational and Criminological Trials Register, and ERIC. We handsearched conference proceedings and publication reference lists for eligible retention trials. We also surveyed all UK Clinical Trials Units to identify further studies.

SELECTION CRITERIA

We included eligible retention trials of randomised or quasi-randomised evaluations of strategies to increase retention that were embedded in 'host' randomised trials from all disease areas and healthcare settings. We excluded studies aiming to increase treatment compliance.

DATA COLLECTION AND ANALYSIS

We contacted authors to supplement or confirm data that we had extracted. For retention trials, we recorded data on the method of randomisation, type of strategy evaluated, comparator, primary outcome, planned sample size, numbers randomised and numbers retained. We used risk ratios (RR) to evaluate the effectiveness of the addition of strategies to improve retention. We assessed heterogeneity between trials using the Chi(2) and I(2) statistics. For main trials that hosted retention trials, we extracted data on disease area, intervention, population, healthcare setting, sequence generation and allocation concealment.

MAIN RESULTS

We identified 38 eligible retention trials. Included trials evaluated six broad types of strategies to improve retention. These were incentives, communication strategies, new questionnaire format, participant case management, behavioural and methodological interventions. For 34 of the included trials, retention was response to postal and electronic questionnaires with or without medical test kits. For four trials, retention was the number of participants remaining in the trial. Included trials were conducted across a spectrum of disease areas, countries, healthcare and community settings. Strategies that improved trial retention were addition of monetary incentives compared with no incentive for return of trial-related postal questionnaires (RR 1.18; 95% CI 1.09 to 1.28, P value < 0.0001), addition of an offer of monetary incentive compared with no offer for return of electronic questionnaires (RR 1.25; 95% CI 1.14 to 1.38, P value < 0.00001) and an offer of a GBP20 voucher compared with GBP10 for return of postal questionnaires and biomedical test kits (RR 1.12; 95% CI 1.04 to 1.22, P value < 0.005). The evidence that shorter questionnaires are better than longer questionnaires was unclear (RR 1.04; 95% CI 1.00 to 1.08, P value = 0.07) and the evidence for questionnaires relevant to the disease/condition was also unclear (RR 1.07; 95% CI 1.01 to 1.14). Although each was based on the results of a single trial, recorded delivery of questionnaires seemed to be more effective than telephone reminders (RR 2.08; 95% CI 1.11 to 3.87, P value = 0.02) and a 'package' of postal communication strategies with reminder letters appeared to be better than standard procedures (RR 1.43; 95% CI 1.22 to 1.67, P value < 0.0001). An open trial design also appeared more effective than a blind trial design for return of questionnaires in one fracture prevention trial (RR 1.37; 95% CI 1.16 to 1.63, P value = 0.0003).There was no good evidence that the addition of a non-monetary incentive, an offer of a non-monetary incentive, 'enhanced' letters, letters delivered by priority post, additional reminders, or questionnaire question order either increased or decreased trial questionnaire response/retention. There was also no evidence that a telephone survey was either more or less effective than a monetary incentive and a questionnaire. As our analyses are based on single trials, the effect on questionnaire response of using offers of charity donations, sending reminders to trial sites and when a questionnaire is sent, may need further evaluation. Case management and behavioural strategies used for trial retention may also warrant further evaluation.

AUTHORS' CONCLUSIONS

Most of the retention trials that we identified evaluated questionnaire response. There were few evaluations of ways to improve participants returning to trial sites for trial follow-up. Monetary incentives and offers of monetary incentives increased postal and electronic questionnaire response. Some other strategies evaluated in single trials looked promising but need further evaluation. Application of the findings of this review would depend on trial setting, population, disease area, data collection and follow-up procedures.

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Postal and electronic questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of health research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH STRATEGY

We searched 14 electronic databases to February 2008 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, we also contacted authors to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria.

SELECTION CRITERIA

Randomised controlled trials of methods to increase response to postal or electronic questionnaires.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity among trial odds ratios using a Chi(2) test and the degree of inconsistency between trial results was quantified using the I(2) statistic.

MAIN RESULTS

PostalWe found 481 eligible trials. The trials evaluated 110 different ways of increasing response to postal questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.87; 95% CI 1.73 to 2.04; heterogeneity P < 0.00001, I(2) = 84%), recorded delivery (1.76; 95% CI 1.43 to 2.18; P = 0.0001, I(2) = 71%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08; 95% CI 1.27 to 7.44) and a more interesting questionnaire topic (2.00; 95% CI 1.32 to 3.04; P = 0.06, I(2) = 80%). The odds of response were substantially higher with pre-notification (1.45; 95% CI 1.29 to 1.63; P < 0.00001, I(2) = 89%), follow-up contact (1.35; 95% CI 1.18 to 1.55; P < 0.00001, I(2) = 76%), unconditional incentives (1.61; 1.36 to 1.89; P < 0.00001, I(2) = 88%), shorter questionnaires (1.64; 95% CI 1.43 to 1.87; P < 0.00001, I(2) = 91%), providing a second copy of the questionnaire at follow up (1.46; 95% CI 1.13 to 1.90; P < 0.00001, I(2) = 82%), mentioning an obligation to respond (1.61; 95% CI 1.16 to 2.22; P = 0.98, I(2) = 0%) and university sponsorship (1.32; 95% CI 1.13 to 1.54; P < 0.00001, I(2) = 83%). The odds of response were also increased with non-monetary incentives (1.15; 95% CI 1.08 to 1.22; P < 0.00001, I(2) = 79%), personalised questionnaires (1.14; 95% CI 1.07 to 1.22; P < 0.00001, I(2) = 63%), use of hand-written addresses (1.25; 95% CI 1.08 to 1.45; P = 0.32, I(2) = 14%), use of stamped return envelopes as opposed to franked return envelopes (1.24; 95% CI 1.14 to 1.35; P < 0.00001, I(2) = 69%), an assurance of confidentiality (1.33; 95% CI 1.24 to 1.42) and first class outward mailing (1.11; 95% CI 1.02 to 1.21; P = 0.78, I(2) = 0%). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94; 95% CI 0.88 to 1.00; P = 0.51, I(2) = 0%).ElectronicWe found 32 eligible trials. The trials evaluated 27 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were increased by more than a half using non-monetary incentives (1.72; 95% CI 1.09 to 2.72; heterogeneity P < 0.00001, I(2) = 95%), shorter e-questionnaires (1.73; 1.40 to 2.13; P = 0.08, I(2) = 68%), including a statement that others had responded (1.52; 95% CI 1.36 to 1.70), and a more interesting topic (1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third using a lottery with immediate notification of results (1.37; 95% CI 1.13 to 1.65), an offer of survey results (1.36; 95% CI 1.15 to 1.61), and using a white background (1.31; 95% CI 1.10 to 1.56). The odds of response were also increased with personalised e-questionnaires (1.24; 95% CI 1.17 to 1.32; P = 0.07, I(2) = 41%), using a simple header (1.23; 95% CI 1.03 to 1.48), using textual representation of response categories (1.19; 95% CI 1.05 to 1.36), and giving a deadline (1.18; 95% CI 1.03 to 1.34). The odds of response tripled when a picture was included in an e-mail (3.05; 95% CI 1.84 to 5.06; P = 0.27, I(2) = 19%). The odds of response were reduced when "Survey" was mentioned in the e-mail subject line (0.81; 95% CI 0.67 to 0.97; P = 0.33, I(2) = 0%), and when the e-mail included a male signature (0.55; 95% CI 0.38 to 0.80; P = 0.96, I(2) = 0%).

AUTHORS' CONCLUSIONS

Health researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this systematic review.

Analysis of nonresponse in the assessment of health-related quality of life of childhood cancer survivors

The aim of this study was to compare the characteristics of respondents and nonrespondents in a survey of childhood cancer survivors recorded in the Childhood Cancer Registry of Piedmont and their current primary care general practitioners. Eligible subjects were identified from the Childhood Cancer Registry of Piedmont and the referring general practitioners were traced through the National Health Service. A postal questionnaire was sent both to childhood cancer survivors and to their general practitioners. Prevalence odds ratios were estimated for demographic and clinical characteristics in survivors and for demographic characteristics in general practitioners. A total of 1005 childhood cancer survivors and 857 general practitioners (132 of them had two or more cancer survivors in care) were included in the study. Completed questionnaires were obtained from 691 survivors (69%) and 615 general practitioners (72%). For survivors, the only associations with nonresponse were for age 35-44 years [prevalence odds ratio: 0.53 (95% confidence interval: 0.33-0.85)], being married [prevalence odds ratio: 1.45 (95% confidence interval: 0.96-2.18)] and diagnosis after 1977 [prevalence odds ratio: 0.66 (95% confidence interval: 0.42-1.03)]. For general practitioners, the only associations were for male sex [prevalence odds ratio: 1.62 (95% confidence interval: 1.13-2.32)] and place of work outside of the city of Turin [prevalence odds ratio: 1.93 (95% confidence interval: 1.07-3.47)]; furthermore associations were relatively weak. An association was also found between nonresponse in survivors and nonresponse in their general practitioners [prevalence odds ratio: 3.40 (95% confidence interval: 2.54-4.56)]. In conclusion, apart from age, marital status and period of diagnosis, there were little differences between respondent and nonrespondents, for the considered clinical and demographical characteristics. Participation of survivors and their general practitioners correlated, suggesting that involvement of the general practitioners in the study may be a method to increase participation of survivors of childhood cancers.

Actual vs. best practices for young children with cerebral palsy: a survey of paediatric occupational therapists and physical therapists in Quebec, Canada

RATIONALE

Cerebral palsy (CP) constitutes a substantial portion of paediatric rehabilitation, yet little is known regarding actual occupational therapy (OT) and physical therapy (PT) practices. This study describes OT and PT practices for young children with CP in Quebec, Canada.

METHODS

This was a cross-sectional survey. All eligible, consenting paediatric occupational therapists (OTs) and physical therapists (PTs) were interviewed using a structured telephone interview based on vignettes of two typical children with CP at two age points--18 months and 4 years. Reported practices were grouped according to the International Classification of Functioning, Disability and Health (ICF).

RESULTS

91.9% of PTs (n=62; 83.8% participation rate) and 67.1% of OTs (n=85; 91.4% participation rate) reported using at least one standardized paediatric assessment. OT and PT interventions focused primarily on impairments and primary function (such as gait function and activities of daily living). Both professions gave little attention to interventions related to play and recreation/leisure. Clinicians reported the need for more training and education specific to CP and to the use of research findings in clinical practice.

CONCLUSION

Wide variations and gaps were identified in clinicians' responses suggesting the need for a basic standard of OT and PT management as well as strategies to encourage knowledge dissemination regarding current best practice.

Methods to increase response rates to postal questionnaires

BACKGROUND

Postal questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response rates to postal questionnaires would improve the quality of health research.

OBJECTIVES

To identify effective strategies to increase response rates to postal questionnaires.

SEARCH STRATEGY

We aimed to find all randomised controlled trials of strategies to increase response rates to postal questionnaires. We searched 14 electronic databases to February 2003 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, authors were also contacted to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria.

SELECTION CRITERIA

Randomised controlled trials of methods to increase response rates to postal questionnaires.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios and 95% confidence intervals in a random effects model. Evidence for selection bias was assessed using Egger's weighted regression method and Begg's rank correlation test and funnel plot. Heterogeneity among trial odds ratios was assessed using a chi-square test at a 5% significance level and the degree of inconsistency between trial results was quantified using I(2).

MAIN RESULTS

We found 372 eligible trials. The trials evaluated 98 different ways of increasing response rates to postal questionnaires and for 62 of these the combined trials included over 1,000 participants. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.99, 95% CI 1.81 to 2.18; heterogeneity p<0.00001, I(2)=78%), recorded delivery (2.04, 1.60 to 2.61; p=0.0004, I(2)=69%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08, 1.27 to 7.44) and a more interesting questionnaire topic (2.44, 1.99 to 3.01; p=0.74, I(2)=0%). The odds of response were substantially higher with pre-notification (1.50, 1.29 to 1.74; p<0.00001, I(2)=90%), follow-up contact (1.44, 1.25 to 1.65; p<0.0001, I(2)=68%), unconditional incentives (1.61, 1.27 to 2.04; p<0.00001, I(2)=91%), shorter questionnaires (1.73, 1.47 to 2.03; p<0.00001, I(2)=93%), providing a second copy of the questionnaire at follow-up (1.51, 1.13 to 2.00; p<0.00001, I(2)=83%), mentioning an obligation to respond (1.61, 1.16 to 2.22; p=0.98, I(2)=0%) and university sponsorship (1.32, 1.13 to 1.54; p<0.00001, I(2)=83%). The odds of response were also increased with non-monetary incentives (1.13, 1.07 to 1.21; p<0.00001, I(2)=71%), personalised questionnaires (1.16, 1.07 to 1.26; p<0.00001, I(2)=67%), use of coloured as opposed to blue or black ink (1.39, 1.16 to 1.67), use of stamped return envelopes as opposed to franked return envelopes (1.29, 1.18 to 1.42; p<0.00001, I(2)=72%), an assurance of confidentiality (1.33, 1.24 to 1.42) and first class outward mailing (1.12, 1.02 to 1.23). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94, 0.88 to 1.00; p=0.51, I(2)=0%), when questionnaires began with the most general questions (0.80, 0.67 to 0.96), or when participants were offered the opportunity to opt out of the study (0.76, 0.65 to 0.89; p=0.46, I(2)=0%).

AUTHORS' CONCLUSIONS

Health researchers using postal questionnaires can increase response rates using the strategies shown to be effective in this systematic review.

Maximising response to postal questionnaires--a systematic review of randomised trials in health research

Background

Postal self-completion questionnaires offer one of the least expensive modes of collecting patient based outcomes in health care research. The purpose of this review is to assess the efficacy of methods of increasing response to postal questionnaires in health care studies on patient populations.

Methods

The following databases were searched: Medline, Embase, CENTRAL, CDSR, PsycINFO, NRR and ZETOC. Reference lists of relevant reviews and relevant journals were hand searched. Inclusion criteria were randomised trials of strategies to improve questionnaire response in health care research on patient populations. Response rate was defined as the percentage of questionnaires returned after all follow-up efforts. Study quality was assessed by two independent reviewers. The Mantel-Haenszel method was used to calculate the pooled odds ratios.

Results

Thirteen studies reporting fifteen trials were included. Implementation of reminder letters and telephone contact had the most significant effect on response rates (odds ratio 3.7, 95% confidence interval 2.30 to 5.97 p = <0.00001). Shorter questionnaires also improved response rates to a lesser degree (odds ratio 1.4, 95% confidence interval 1.19 to 1.54). No evidence was found that incentives, re-ordering of questions or including an information brochure with the questionnaire confer any additional advantage.

Conclusion

Implementing repeat mailing strategies and/or telephone reminders may improve response to postal questionnaires in health care research. Making the questionnaire shorter may also improve response rates. There is a lack of evidence to suggest that incentives are useful. In the context of health care research all strategies to improve response to postal questionnaires require further evaluation.

Mail Surveys Can Achieve High Response Rates in a Dermatology Patient Population

In order to understand the burden of skin disease on patient populations, researchers need to be able to measure exposures and outcomes of interest in a population-based study. One method of obtaining such information is mailed patient surveys. This method of obtaining information, however, raises concerns regarding possible low response rates, and to the best of our knowledge, this method has not been attempted in large dermatology patient populations with a high response rate. We sought to determine whether using a slightly modified version of the Dillman Total Design Method as a mailed survey protocol would result in a high response rate in a dermatology population. A mail survey was sent using a slightly modified version of Dillman's Total Design Method to 4894 patients seen in the Dermatology Department of the University of Pennsylvania, who were diagnosed with nonmelanoma skin cancer, dermatophytosis, acne rosacea, seborrheic keratosis, or warts; 74.1% of the subjects responded to the mailings; 69.8% (n=3203) of patients returned a completed survey. Response rates (both as overall response and as survey completers only) were high across a wide spectrum of dermatologic illnesses and did not vary significantly by entry diagnosis. Dillman's Total Design Method appears to be an effective tool for researchers studying the burden of skin disease in a large sample of dermatology patients.

Comparing different methods to detect and correct nonresponse bias in postal questionnaire studies

OBJECTIVES

This study compares methods for detecting and correcting the bias associated with nonresponse to postal questionnaires.

METHODS

Questionnaires were sent out in three sequential stages to parents of all 5-year-old children examined in a clinical survey. Each stage progressively targeted nonresponders. Data on dmft and area measures of socioeconomic status were available for all children. Estimates for whole population dmft were produced by different methodologies comparing the relationship between dmft and stage of response and three area measures of socioeconomic status.

RESULTS

A total of 1,776 children were examined and 1,437 questionnaires were obtained, a response rate of 80.9 percent. The mean dmft of the total population (1.49) was 17.3 percent more than responders (1.27). The dmft of the nonresponders was 2.41, 89.7 percent more than responders. There were significant linear trends in dmft and socioeconomic status across the mailing stages. The methodology using mailing stage regressed against dmft produced the most accurate adjusted dmft value (1.42). The methods using area measures of socioeconomic status produced nearly identical adjusted dmft values ranging from 1.31 to 1.32.

CONCLUSIONS

Even with an "acceptable" response rate, nonresponse bias can still be present. Researchers should report the outcomes of analyses to detect nonresponse bias when publishing questionnaire studies.

Developing a reliable, valid, and feasible plan for quality-of-care measurement for cancer: how should we measure?

BACKGROUND

Recent changes in the US health care delivery system have raised expectations that the medical marketplace will compete on quality and cost of care. This effort will require a systematic evaluation of the measurement of quality of care as it applies to cancer and other critical conditions.

OBJECTIVES

To articulate the components of the design of quality-of-care measurement systems that must be considered and optimally manipulated to generate feasible, reliable, and valid data pertinent to patients with cancer.

RESEARCH DESIGN

A synthesis of information obtained from literature reviews and experience.

MEASURES

Four key areas of design that influence quality-of-care measurement scores are discussed: case identification, data source, data-collection strategies, and the quality of the care-measurement model.

RESULTS

Challenges associated with these design and measurement strategies are defined and discussed.

CONCLUSIONS

Policy analyses vary as a function of measurement domains. The design of a quality-of-care measurement system should consider trade-offs between validity and burden by considering the intricate relations between domains of measurement.

The Pregnancy Risk Assessment Monitoring System (PRAMS): methods and 1996 response rates from 11 states

OBJECTIVES

To determine if the Pregnancy Risk Assessment Monitoring System (PRAMS) is a unique and valuable MCH data source and an effective mechanism for states to collect MCH data, and to assess if recent changes in it have improved efficiency and flexibility.

METHODS

Each component of the PRAMS methodology is described: sampling and stratification, data collection, questionnaire, and data management and weighting. To assess effectiveness, we calculated response rates, contact rates, cooperation rates, refusal rates, and questionnaire completion rates. Logistic regression was used to examine the relationship between maternal and infant characteristics and the likelihood of response. Four criteria were defined to measure improvement in PRAMS functioning.

RESULTS

Overall response rates for the 11 states in 1996 ranged from 66% to 80%. Cooperation rates were high (85-99%), with contact rates somewhat lower (73-87%). Response rates were higher for women who were older, White, married, had more education, were first-time mothers, and had a normal-birthweight infant. In all states, parity and education were the most consistent predictors of response, followed by marital status and race. Between 1988-1990 and 1996-1999, the number of states and areas participating in PRAMS increased from 6 to 23, response rates improved, and the time for a state to start data collection and to obtain a weighted dataset both decreased.

CONCLUSIONS

PRAMS is a unique and valuable MCH data source. The mail/telephone methodology used in PRAMS is an effective means of reaching most women who have recently given birth in the 11 states examined; however, some population subgroups are not reached as well as others. The system has become more efficient and flexible over time and more states now participate.