Developing an online, searchable database to systematically map and organise current literature on retention research (ORRCA2)

The overlap between randomised evaluations of recruitment and retention interventions: An updated review of recruitment (Online Resource for Recruitment in Clinical triAls) and retention (Online Resource for Retention in Clinical triAls) literature

BACKGROUND

The Online Resource for Recruitment in Clinical triAls (ORRCA) and the Online Resource for Retention in Clinical triAls (ORRCA2) were established to organise and map the literature addressing participant recruitment and retention within clinical research. The two databases are updated on an ongoing basis using separate but parallel systematic reviews. However, recruitment and retention of research participants is widely acknowledged to be interconnected. While interventions aimed at addressing recruitment challenges can impact retention and vice versa, it is not clear how well they are simultaneously considered within methodological research. This study aims to report the recent update of ORRCA and ORRCA2 with a special emphasis on assessing crossover of the databases and how frequently randomised studies of methodological interventions measure the impact on both recruitment and retention outcomes.

METHODS

Two parallel systematic reviews were conducted in line with previously reported methods updating ORRCA (recruitment) and ORRCA2 (retention) with publications from 2018 and 2019. Articles were categorised according to their evidence type (randomised evaluation, non-randomised evaluation, application and observation) and against the recruitment and retention domain frameworks. Articles categorised as randomised evaluations were compared to identify studies appearing in both databases. For randomised studies that were only in one database, domain categories were used to assess whether the methodological intervention was likely to impact on the alternate construct. For example, whether a recruitment intervention might also impact retention.

RESULTS

In total, 806 of 17,767 articles screened for the recruitment database and 175 of 18,656 articles screened for the retention database were added as result of the update. Of these, 89 articles were classified as 'randomised evaluation', of which 6 were systematic reviews and 83 were randomised evaluations of methodological interventions. Ten of the randomised studies assessed recruitment and retention and were included in both databases. Of the randomised studies only in the recruitment database, 48/55 (87%) assessed the content or format of participant information which could have an impact on retention. Of the randomised studies only in the retention database, 6/18 (33%) assessed monetary incentives, 4/18 (22%) assessed data collection location and methods and 3/18 (17%) assessed non-monetary incentives, all of which could have an impact on recruitment.

CONCLUSION

Only a small proportion of randomised studies of methodological interventions assessed the impact on both recruitment and retention despite having a potential impact on both outcomes. Where possible, an integrated approach analysing both constructs should be the new standard for these types of evaluations to ensure that improvements to recruitment are not at the expense of retention and vice versa.

Undertaking Studies Within A Trial to evaluate recruitment and retention strategies for randomised controlled trials: lessons learnt from the PROMETHEUS research programme

Background

Randomised controlled trials ('trials') are susceptible to poor participant recruitment and retention. Studies Within A Trial are the strongest methods for testing the effectiveness of strategies to improve recruitment and retention. However, relatively few of these have been conducted.

Objectives

PROMoting THE Use of Studies Within A Trial aimed to facilitate at least 25 Studies Within A Trial evaluating recruitment or retention strategies. We share our experience of delivering the PROMoting THE Use of Studies Within A Trial programme, and the lessons learnt for undertaking randomised Studies Within A Trial.

Design

A network of 10 Clinical Trials Units and 1 primary care research centre committed to conducting randomised controlled Studies Within A Trial of recruitment and/or retention strategies was established. Promising recruitment and retention strategies were identified from various sources including Cochrane systematic reviews, the Study Within A Trial Repository, and existing prioritisation exercises, which were reviewed by patient and public members to create an initial priority list of seven recruitment and eight retention interventions. Host trial teams could apply for funding and receive support from the PROMoting THE Use of Studies Within A Trial team to undertake Studies Within A Trial. We also tested the feasibility of undertaking co-ordinated Studies Within A Trial, across multiple host trials simultaneously.

Setting

Clinical trials unit-based trials recruiting or following up participants in any setting in the United Kingdom were eligible.

Participants

Clinical trials unit-based teams undertaking trials in any clinical context in the United Kingdom.

Interventions

Funding of up to £5000 and support from the PROMoting THE Use of Studies Within A Trial team to design, implement and report Studies Within A Trial.

Main outcome measures

Number of host trials funded.

Results

Forty-two Studies Within A Trial were funded (31 host trials), across 12 Clinical Trials Units. The mean cost of a Study Within A Trial was £3535. Twelve Studies Within A Trial tested the same strategy across multiple host trials using a co-ordinated Study Within A Trial design, and four used a factorial design. Two recruitment and five retention strategies were evaluated in more than one host trial. PROMoting THE Use of Studies Within A Trial will add 18% more Studies Within A Trial to the Cochrane systematic review of recruitment strategies, and 79% more Studies Within A Trial to the Cochrane review of retention strategies. For retention, we found that pre-notifying participants by card, letter or e-mail before sending questionnaires was effective, as was the use of pens, and sending personalised text messages to improve questionnaire response. We highlight key lessons learnt to guide others planning Studies Within A Trial, including involving patient and public involvement partners; prioritising and selecting strategies to evaluate and elements to consider when designing a Study Within A Trial; obtaining governance approvals; implementing Studies Within A Trial, including individual and co-ordinated Studies Within A Trials; and reporting Study Within A Trials.

Limitations

The COVID-19 pandemic negatively impacted five Studies Within A Trial, being either delayed (n = 2) or prematurely terminated (n = 3).

Conclusions

PROMoting THE Use of Studies Within A Trial significantly increased the evidence base for recruitment and retention strategies. When provided with both funding and practical support, host trial teams successfully implemented Studies Within A Trial.

Future work

Future research should identify and target gaps in the evidence base, including widening Study Within A Trial uptake, undertaking more complex Studies Within A Trial and translating Study Within A Trial evidence into practice.

Study registration

All Studies Within A Trial in the PROMoting THE Use of Studies Within A Trial programme had to be registered with the Northern Ireland Network for Trials Methodology Research Study Within A Trial Repository.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 13/55/80) and is published in full in Health Technology Assessment; Vol. 28, No. 2. See the NIHR Funding and Awards website for further award information.

An international study to explore the feasibility of collecting standardised outcome data for Complex Regional Pain Syndrome: recommendations for an international clinical research registry

Introduction

Complex Regional Pain Syndrome (CRPS) is a persistent pain condition with low prevalence. Multi-centre collaborative research is needed to attain sufficient sample sizes for meaningful studies. This international observational study: (1) tested the feasibility and acceptability of collecting outcome data using an agreed core measurement set (2) tested and refined an electronic data management system to collect and manage the data.

Methods

Adults with CRPS, meeting the Budapest diagnostic clinical criteria, were recruited to the study from 7 international research centres. After informed consent, a questionnaire comprising the core set outcome measures was completed: on paper at baseline (T1), and at 3 or 6 months (T2) using a paper or e-version. Participants and clinicians provided feedback on the data collection process. Clinicians completed the CRPS severity score at T1 and optionally, at T2. Ethical approval was obtained at each international centre.

Results

Ninety-eight adults were recruited (female n=66; mean age 46.6 years, range 19-89), of whom 32% chose to receive the T2 questionnaire in an electronic format. Fifty-five participants completed both T1 and T2. Eighteen participants and nine clinicians provided feedback on their data collection experience.

Conclusion

This study confirmed the questionnaire core outcome data are feasible and practicable to collect in clinical practice. The electronic data management system provided a robust means of collecting and managing the data across an international population. The findings have informed the final data collection tools and processes which will comprise the first international, clinical research registry and data bank for CRPS.

Participant retention in paediatric randomised controlled trials published in six major journals 2015-2019: systematic review and meta-analysis

BACKGROUND

The factors which influence participant retention in paediatric randomised controlled trials are under-researched. Retention may be more challenging due to child developmental stages, involving additional participants, and proxy-reporting of outcomes. This systematic review and meta-analysis explores the factors which may influence retention in paediatric trials.

METHODS

Using the MEDLINE database, paediatric randomised controlled trials published between 2015 and 2019 were identified from six general and specialist high-impact factor medical journals. The review outcome was participant retention for each reviewed trial's primary outcome. Context (e.g. population, disease) and design (e.g. length of trial) factors were extracted. Retention was examined for each context and design factor in turn, with evidence for an association being determined by a univariate random-effects meta-regression analysis.

RESULTS

Ninety-four trials were included, and the median total retention was 0.92 (inter-quartile range 0.83 to 0.98). Higher estimates of retention were seen for trials with five or more follow-up assessments before the primary outcome, those less than 6 months between randomisation and primary outcome, and those that used an inactive data collection method. Trials involving children aged 11 and over had the higher estimated retention compared with those involving younger children. Those trials which did not involve other participants also had higher retention, than those where they were involved. There was also evidence that a trial which used an active or placebo control treatment had higher estimated retention, than treatment-as-usual. Retention increased if at least one engagement method was used. Unlike reviews of trials including all ages of participants, we did not find any association between retention and the number of treatment groups, size of trial, or type of treatment.

CONCLUSIONS

Published paediatric RCTs rarely report the use of specific modifiable factors that improve retention. Including multiple, regular follow-ups with participants before the primary outcome may reduce attrition. Retention may be highest when the primary outcome is collected up to 6 months after a participant is recruited. Our findings suggest that qualitative research into improving retention when trials involve multiple participants such as young people, and their caregivers or teachers would be worthwhile. Those designing paediatric trials also need to consider the use of appropriate engagement methods. RESEARCH ON RESEARCH (ROR) REGISTRY: https://ror-hub.org/study/2561.

Effectiveness and cost analysis of methods used to recruit older adult sedative users to a deprescribing randomized controlled trial during the COVID-19 pandemic

Background

Recruitment to clinical trials is a challenge for researchers that became more pronounced because of COVID-19 public health protective measures, especially with respect to studies enrolling older adults. We completed an effectiveness and cost analysis of the recruitment methods used in The Your Answers When Needing Sleep in New Brunswick (YAWNS NB) study, a randomized controlled trial of a deprescribing intervention that recruited older adults with chronic use of sedatives during the pandemic.

Methods

Study recruitment began during the COVID-19 pandemic. Strategies included random digit dialing (RDD), a targeted mail campaign and advertising through newspapers, online platforms (Google and Facebook), and television. Other awareness raising and recruitment strategies involved seniors' organizations, pharmacies, television news stories, and referrals. Recruitment effectiveness and cost analysis involved enrollment rate (ER), cost per randomized participant (CPRP), fractional cost (FC), fractional enrollment (FE), fractional enrollment-cost ratio (FEC), and efficacy index (EI) calculations.

Results

There were 1295 interested older adults with 594 randomized into the study for an enrollment rate of 46%. The efficacy index (EI) was highest for Facebook ads (EI = 0.683) followed by television (EI = 0.426), and newsprint ads (EI = 0.298). The cost of RDD was highest per randomized participant at $1117.90 and produced the lowest EI (0.013).

Conclusion

Facebook ads had the best efficacy index for recruiting older adults to the YAWNS NB study during the COVID-19 pandemic and television ads produced the most enrollments. RDD was expensive and yielded few recruits. Recruitment costs can be significant for recruiting community-dwelling older adults. This experience can inform recruitment strategy and budget development for future community studies enrolling older adults, especially in the context of the COVID-19 pandemic.