Impact of on-site initiation visits on patient recruitment and data quality in a randomized trial of adjuvant chemotherapy for breast cancer

Describing the content of trial recruitment interventions using the TIDieR reporting checklist: a systematic methodology review

BACKGROUND

Recruiting participants to clinical trials is an ongoing challenge, and relatively little is known about what recruitment strategies lead to better recruitment. Recruitment interventions can be considered complex interventions, often involving multiple components, targeting a variety of groups, and tailoring to different groups. We used the Template for Intervention Description and Replication (TIDieR) reporting checklist (which comprises 12 items recommended for reporting complex interventions) to guide the assessment of how recruitment interventions are described. We aimed to (1) examine to what extent we could identify information about each TIDieR item within recruitment intervention studies, and (2) observe additional detail for each item to describe useful variation among these studies.

METHODS

We identified randomized, nested recruitment intervention studies providing recruitment or willingness to participate rates from two sources: a Cochrane review of trials evaluating strategies to improve recruitment to randomized trials, and the Online Resource for Research in Clinical triAls database. First, we assessed to what extent authors reported information about each TIDieR item. Second, we developed descriptive categorical variables for 7 TIDieR items and extracting relevant quotes for the other 5 items.

RESULTS

We assessed 122 recruitment intervention studies. We were able to extract information relevant to most TIDieR items (e.g., brief rationale, materials, procedure) with the exception of a few items that were only rarely reported (e.g., tailoring, modifications, planned/actual fidelity). The descriptive variables provided a useful overview of study characteristics, with most studies using various forms of informational interventions (55%) delivered at a single time point (90%), often by a member of the research team (59%) in a clinical care setting (41%).

CONCLUSIONS

Our TIDieR-based variables provide a useful description of the core elements of complex trial recruitment interventions. Recruitment intervention studies report core elements of complex interventions variably; some process elements (e.g., mode of delivery, location) are almost always described, while others (e.g., duration, fidelity) are reported infrequently, with little indication of a reason for their absence. Future research should explore whether these TIDieR-based variables can form the basis of an approach to better reporting of elements of successful recruitment interventions.

Do recruitment SWAT interventions have an impact on participant retention in randomised controlled trials? A systematic review

BACKGROUND

Evidence-based methods for randomised controlled trial recruitment and retention are extremely valuable. Despite increased testing of these through studies within a trial, there remains limited high-certainty evidence for effective strategies. In addition, there has been little consideration as to whether recruitment interventions also have an impact on participant retention.

METHODS

A systematic review was conducted. Studies were eligible if they were randomised controlled trials using a recruitment intervention and which also assessed the impact of this on retention at any time point. Searches were conducted through MEDLINE, EMBASE, Cochrane Library, and the Northern Ireland Hub for Trials Methodology Research SWAT Repository. Two independent reviewers screened the search results and extracted data for eligible studies using a piloted extraction form.

RESULTS

A total of 7815 records were identified, resulting in 10 studies being included in the review. Most studies (n = 6, 60%) focussed on the information given to participants (n = 6, 60%), with two (20%) focussing on incentives, and two focussing on trial design and recruiter interventions. Due to intervention heterogeneity, none of the interventions could be meta-analysed. Only one study found any statistically significant effect of letters including a photograph (odds ratio: 5.40, 95% CI 1.12-26.15, p = 0.04).

CONCLUSION

Assessment of the impacts of recruitment strategies, evaluated in a SWAT, on retention of participants in the host trial remains limited. Assessment of the impact of recruitment interventions on retention is recommended to minimise future research costs and waste.

Monitoring strategies for clinical intervention studies

BACKGROUND

Trial monitoring is an important component of good clinical practice to ensure the safety and rights of study participants, confidentiality of personal information, and quality of data. However, the effectiveness of various existing monitoring approaches is unclear. Information to guide the choice of monitoring methods in clinical intervention studies may help trialists, support units, and monitors to effectively adjust their approaches to current knowledge and evidence.

OBJECTIVES

To evaluate the advantages and disadvantages of different monitoring strategies (including risk-based strategies and others) for clinical intervention studies examined in prospective comparative studies of monitoring interventions.

SEARCH METHODS

We systematically searched CENTRAL, PubMed, and Embase via Ovid for relevant published literature up to March 2021. We searched the online 'Studies within A Trial' (SWAT) repository, grey literature, and trial registries for ongoing or unpublished studies.

SELECTION CRITERIA

We included randomized or non-randomized prospective, empirical evaluation studies of different monitoring strategies in one or more clinical intervention studies. We applied no restrictions for language or date of publication.

DATA COLLECTION AND ANALYSIS

We extracted data on the evaluated monitoring methods, countries involved, study population, study setting, randomization method, and numbers and proportions in each intervention group. Our primary outcome was critical and major monitoring findings in prospective intervention studies. Monitoring findings were classified according to different error domains (e.g. major eligibility violations) and the primary outcome measure was a composite of these domains. Secondary outcomes were individual error domains, participant recruitment and follow-up, and resource use. If we identified more than one study for a comparison and outcome definitions were similar across identified studies, we quantitatively summarized effects in a meta-analysis using a random-effects model. Otherwise, we qualitatively summarized the results of eligible studies stratified by different comparisons of monitoring strategies. We used the GRADE approach to assess the certainty of the evidence for different groups of comparisons.

MAIN RESULTS

We identified eight eligible studies, which we grouped into five comparisons. 1. Risk-based versus extensive on-site monitoring: based on two large studies, we found moderate certainty of evidence for the combined primary outcome of major or critical findings that risk-based monitoring is not inferior to extensive on-site monitoring. Although the risk ratio was close to 'no difference' (1.03 with a 95% confidence interval [CI] of 0.81 to 1.33, below 1.0 in favor of the risk-based strategy), the high imprecision in one study and the small number of eligible studies resulted in a wide CI of the summary estimate. Low certainty of evidence suggested that monitoring strategies with extensive on-site monitoring were associated with considerably higher resource use and costs (up to a factor of 3.4). Data on recruitment or retention of trial participants were not available. 2. Central monitoring with triggered on-site visits versus regular on-site visits: combining the results of two eligible studies yielded low certainty of evidence with a risk ratio of 1.83 (95% CI 0.51 to 6.55) in favor of triggered monitoring intervention. Data on recruitment, retention, and resource use were not available. 3. Central statistical monitoring and local monitoring performed by site staff with annual on-site visits versus central statistical monitoring and local monitoring only: based on one study, there was moderate certainty of evidence that a small number of major and critical findings were missed with the central monitoring approach without on-site visits: 3.8% of participants in the group without on-site visits and 6.4% in the group with on-site visits had a major or critical monitoring finding (odds ratio 1.7, 95% CI 1.1 to 2.7; P = 0.03). The absolute number of monitoring findings was very low, probably because defined major and critical findings were very study specific and central monitoring was present in both intervention groups. Very low certainty of evidence did not suggest a relevant effect on participant retention, and very low certainty evidence indicated an extra cost for on-site visits of USD 2,035,392. There were no data on recruitment. 4. Traditional 100% source data verification (SDV) versus targeted or remote SDV: the two studies assessing targeted and remote SDV reported findings only related to source documents. Compared to the final database obtained using the full SDV monitoring process, only a small proportion of remaining errors on overall data were identified using the targeted SDV process in the MONITORING study (absolute difference 1.47%, 95% CI 1.41% to 1.53%). Targeted SDV was effective in the verification of source documents, but increased the workload on data management. The other included study was a pilot study, which compared traditional on-site SDV versus remote SDV and found little difference in monitoring findings and the ability to locate data values despite marked differences in remote access in two clinical trial networks. There were no data on recruitment or retention. 5. Systematic on-site initiation visit versus on-site initiation visit upon request: very low certainty of evidence suggested no difference in retention and recruitment between the two approaches. There were no data on critical and major findings or on resource use.

AUTHORS' CONCLUSIONS

The evidence base is limited in terms of quantity and quality. Ideally, for each of the five identified comparisons, more prospective, comparative monitoring studies nested in clinical trials and measuring effects on all outcomes specified in this review are necessary to draw more reliable conclusions. However, the results suggesting risk-based, targeted, and mainly central monitoring as an efficient strategy are promising. The development of reliable triggers for on-site visits is ongoing; different triggers might be used in different settings. More evidence on risk indicators that identify sites with problems or the prognostic value of triggers is needed to further optimize central monitoring strategies. In particular, approaches with an initial assessment of trial-specific risks that need to be closely monitored centrally during trial conduct with triggered on-site visits should be evaluated in future research.

Moving Beyond the Momentum: Innovative Approaches to Clinical Trial Implementation

Despite efforts to enhance enrollment and the merger of national cooperative groups, < 5% of patients with cancer will enroll into a clinical trial. Additionally, clinical trials are affected by a lack of diversity inclusive of minority patients, rural residents, or low-income individuals. COVID-19 further exacerbated known barriers of reduced physician-patient interaction, physician availability, trial activation and enrollment, financial resources, and capacity for conducting research. Based on the cumulative insight of academic and community clinical researchers, we have created a white paper identifying existing challenges in clinical trial conduct and have provided specific recommendations of sustainable modifications to improve efficiency in the activation and conduct of clinical trials with an overarching goal of providing improved access and care to our patients with cancer.

Trainee Principal Investigator Could Improve Recruitment in Trauma Trials: Review of Literature and Experience From a Trauma Center

INTRODUCTION

Recruitment of patients to participate in randomized control trials (RCT) is a challenging task, especially for trauma trials in which the identification and recruitment are time-limited. Multiple strategies have been tried to improve the participation of doctors and the recruitment of patients. The aim was to study the effect of a trainee principal investigator (TPI) on the efficacy of recruitment for a multicenter hip fracture RCT.

METHODS

A retrospective study comparing the number of junior doctors participating in the WHiTE 8 COPAL RCT and patients recruited before and after the introduction of formal TPI role at a major trauma center in the UK. Data was collected for nine months "before" (Nov 2018-July 2019) and six months "after" (Sept 2019-Feb 2020) the role of TPI was assigned.

RESULTS

From November 2018 to February 2020, a total of 292 patients were eligible for recruitment into this trial, out of which 196 (67.12 %) were successfully recruited. Excluding the research team, there were seven junior doctors actively recruiting in the "before period" in comparison with 10 in the "after period." Significantly more patients were recruited by junior doctors after a TPI was assigned. Overall, more percentage of eligible patients were recruited into the trial after a TPI was assigned, and this was statistically significant.

CONCLUSION

The allocation of a formal TPI significantly improved the recruitment of patients in a national RCT. TPI can work alongside the principal investigator and research team to be a valuable link person coordinating and engaging local trainees to take part in trials.

Strategies for research participant engagement: A synthetic review and conceptual framework

BACKGROUND

Research participant engagement, which we define as recruitment and retention in clinical trials, is a costly and challenging issue in clinical research. Research teams have leveraged a variety of strategies to increase research participant engagement in clinical trials, although a framework and evidence for effective participant engagement strategies are lacking. We (1) developed a novel conceptual framework for strategies used to recruit and retain participants in clinical trials based on their underlying behavioral principles and (2) categorized empirically tested recruitment and retention strategies in this novel framework.

METHODS

We conducted a synthetic analysis of interventions tested in studies from two Cochrane reviews on clinical trial recruitment and retention, which included studies from 1986 to 2015. We developed a conceptual framework of behavioral strategies for increasing research participant engagement using deductive and inductive approaches with the studies included in the Cochrane reviews. Reviewed interventions were then categorized using this framework. We examined the results of reviewed interventions and categorized the effects on clinical trial recruitment and retention as significantly positive, null, or significantly negative; summary statistics are presented for the frequency and effects of each behavioral strategy type.

RESULTS

We analyzed 141 unique interventions across 96 studies: 91 interventions targeted clinical trial research participant recruitment and 50 targeted retention. Our framework included 14 behavioral strategies to improve research participant engagement grouped into four general approaches: changing attitudes by appealing to social motivators, changing attitudes by targeting individual psychology, reducing barriers and cognitive burdens, and providing incentives. The majority of interventions (54%) aimed to reduce barriers or cognitive burdens, with improving comprehension (27%) as the most common specific strategy identified. For recruitment, the most common behavioral strategies tested were building legitimacy or trust (38%) and framing risks and benefits (32%), while financial or material incentives (32%) and reducing financial, time, and social barriers (32%) were most common for retention interventions. Among interventions tested in randomized controlled trials, 51% had a null effect on research participant engagement, and 30% had a statistically significant positive effect.

DISCUSSION

Clinical researchers have tested a wide range of interventions that leverage distinct behavioral strategies to achieve improved research participant recruitment and retention. Common behavioral strategies include building legitimacy or trust between research teams and participants, as well as improving participant comprehension of trial objectives and procedures. The high frequency of null effects among tested interventions suggests challenges in selecting the optimal interventions for increasing research participant engagement, although the proposed behavioral strategy categories can serve as a conceptual framework for developing and testing future interventions.

Reducing Events of Noncompliance in Neurology Human Subjects Research: the Effect of Human Subjects Research Protection Training and Site Initiation Visits

In an effort to minimize protocol noncompliance in neurological research studies that can potentially compromise patient safety, delay completion of the study, and result in premature termination and added costs, we determined the effect of investigator trainings and site initiation visits (SIVs) on the occurrence of noncompliance events. Results of protocol audits conducted at the National Institute of Neurological Disorders and Stroke from 2003 to 2019 on 97 research protocols were retrospectively analyzed. Based on the depth of auditing and provision of investigator research training, audit data were separated into four arms: 1) Early Period, 2003 to 2012; 2) Middle Period, 2013 to 2016; and Late Period, 2017 to 2019, further divided into 3) Late Period without SIVs; and 4) Late Period with SIVs. Events of noncompliance were classified by the type of protocol deviation, the category, and the cause. In total, 952 events occurred across 1080 participants. Protocols audited during the Middle Period, compared to the Early Period, showed a decrease in the percentage of protocols with at least 1 noncompliance event. Protocols with SIVs had a further decrease in major, minor, procedural, eligibility, and policy events. Additionally, protocols audited during the Early Period had on average 0.46 major deviations per participant, compared to 0.26 events in protocols audited during the Middle Period, and 0.08 events in protocols audited during the Late Period with SIVs. Protocol deviations and noncompliance events in neurological clinical trials can be reduced by targeted investigator trainings and SIVs. These measures have major impacts on the integrity, safety, and effectiveness of human subjects research in neurology.

Strategies to improve retention in randomised trials

BACKGROUND

Poor retention of participants in randomised trials can lead to missing outcome data which can introduce bias and reduce study power, affecting the generalisability, validity and reliability of results. Many strategies are used to improve retention but few have been formally evaluated.

OBJECTIVES

To quantify the effect of strategies to improve retention of participants in randomised trials and to investigate if the effect varied by trial setting.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Scopus, PsycINFO, CINAHL, Web of Science Core Collection (SCI-expanded, SSCI, CPSI-S, CPCI-SSH and ESCI) either directly with a specified search strategy or indirectly through the ORRCA database. We also searched the SWAT repository to identify ongoing or recently completed retention trials. We did our most recent searches in January 2020.

SELECTION CRITERIA

We included eligible randomised or quasi-randomised trials of evaluations of strategies to increase retention that were embedded in 'host' randomised trials from all disease areas and healthcare settings. We excluded studies aiming to increase treatment compliance.

DATA COLLECTION AND ANALYSIS

We extracted data on: the retention strategy being evaluated; location of study; host trial setting; method of randomisation; numbers and proportions in each intervention and comparator group. We used a risk difference (RD) and 95% confidence interval (CI) to estimate the effectiveness of the strategies to improve retention. We assessed heterogeneity between trials. We applied GRADE to determine the certainty of the evidence within each comparison.

MAIN RESULTS

We identified 70 eligible papers that reported data from 81 retention trials. We included 69 studies with more than 100,000 participants in the final meta-analyses, of which 67 studies evaluated interventions aimed at trial participants and two evaluated interventions aimed at trial staff involved in retention. All studies were in health care and most aimed to improve postal questionnaire response. Interventions were categorised into broad comparison groups: Data collection; Participants; Sites and site staff; Central study management; and Study design. These intervention groups consisted of 52 comparisons, none of which were supported by high-certainty evidence as determined by GRADE assessment. There were four comparisons presenting moderate-certainty evidence, three supporting retention (self-sampling kits, monetary reward together with reminder or prenotification and giving a pen at recruitment) and one reducing retention (inclusion of a diary with usual follow-up compared to usual follow-up alone). Of the remaining studies, 20 presented GRADE low-certainty evidence and 28 presented very low-certainty evidence. Our findings do provide a priority list for future replication studies, especially with regard to comparisons that currently rely on a single study.

AUTHORS' CONCLUSIONS

Most of the interventions we identified aimed to improve retention in the form of postal questionnaire response. There were few evaluations of ways to improve participants returning to trial sites for trial follow-up. None of the comparisons are supported by high-certainty evidence. Comparisons in the review where the evidence certainty could be improved with the addition of well-done studies should be the focus for future evaluations.

Simulation Training to Improve Informed Consent and Pharmacokinetic/Pharmacodynamic Sampling in Pediatric Trials

Background: Pediatric trials to add missing data for evidence-based pharmacotherapy are still scarce. A tailored training concept appears to be a promising tool to cope with critical and complex situations before enrolling the very first patient and subsequently to ensure high-quality study conduct. The aim was to facilitate study success by optimizing the preparedness of the study staff shift. Method: An interdisciplinary faculty developed a simulation training focusing on the communication within the informed consent procedure and the conduct of the complex pharmacokinetic/pharmacodynamic (PK/PD) sampling within a simulation facility. Scenarios were video-debriefed by an audio-video system and manikins with artificial blood simulating patients were used. The training was evaluated by participants' self-assessment before and during trial recruitment. Results: The simulation training identified different optimization potentials for improved informed consent process and study conduct. It facilitated the reduction of avoidable errors, especially in the early phase of a clinical study. The knowledge gained through the intervention was used to train the study teams, improve the team composition and optimize the on-ward setting for the FP-7 funded "LENA" project (grant agreement no. 602295). Self-perceived ability to communicate core elements of the trial as well as its correct performance of sample preparation increased significantly (mean, 95% CI, p ≤ 0.0001) from 3 (2.5-3.5) to four points (4.0-4.5), and from 2 (1.5-2.5) to five points (4.0-5.0). Conclusion: An innovative training concept to optimize the informed consent process and study conduct was successfully developed and enabled high-quality conduct of the pediatric trials as of the very first patient visit.

Protocol for a factorial randomised controlled trial, embedded within WHiTE 8 COPAL, of an Enhanced Trainee Principal Investigator Package and Additional Digital Nudge to increase recruitment rates

Recruitment remains an issue when conducting randomised controlled trials (RCTs) with a significant proportion of studies failing to reach their target sample size. Studies evaluating interventions to improve recruitment aimed specifically at recruiters to the trial are limited in number. This factorial RCT will evaluate the effectiveness of an educational intervention to trainee principal investigators and a positive reinforcement intervention via an email nudge on increasing recruitment. The targeted recruiters will be in 20 centres nationally recruiting to one large orthopaedic randomised controlled trial, WHiTE 8 COPAL. Centres will be randomised via minimisation to one of four groups. The primary outcome is recruitment rate in the first six months that a centre is actively recruiting, with data being analysed via a Poisson regression model. Results will be presented as adjusted incidence rate ratios with 95% confidence intervals. Secondary outcomes relate to the feasibility and logistics of running the interventions.  We will also collect feedback regarding the educational programme set out for the trainee principal investigators. The study started in August 2018 with the anticipation of the primary objective endpoint by October 2019. The results of this study will be used to inform the design of future RCTs, particularly in orthopaedics in the UK, where the role of Trainee Principal Investigators is now a consistent one across different trials. Trial registration: 11600053, ISRCTN, 20/08/2018; SWAT 67, Northern Ireland Hub for Trials Methodology Research SWAT repository, 01/10/2017.

Remote or on-site visits were feasible for the initial setup meetings with hospitals in a multicenter surgical trial: an embedded randomized trial

OBJECTIVES

To investigate the effects, costs, and feasibility of providing on-site compared with remote meetings to set up hospital sites in a multicenter, surgical randomized controlled trial.

STUDY DESIGN AND SETTING

Hospitals were randomized to receive the initial trial setup meetings on-site (i.e., face-to-face) or remotely (i.e., via teleconference). Data were collected on site setup, recruitment, follow-up, and costs for the two methods. The hospital staff experience of trial setup was also surveyed.

RESULTS

Thirty-nine sites were randomized and 33 sites set up to recruit (19 on-site and 14 remote). For sites randomized to an on-site meeting compared with remote meeting respectively, the time from first contact to the first recruit was a median of 246 days (interquartile range [IQR] 196-346] vs. 212 days (IQR 154-266), mean recruitment was 10 participants (median 10, IQR 2-17) vs. 11 participants (median 6, IQR 5-23), and participant follow-up at 12 months was 81% vs. 82%. Sites allocated to an initial on-site visit cost on average £289.83 more to setup.

CONCLUSION

Remote or on-site visits are feasible for the initial setup meetings with hospitals in a multicenter surgical trial. This embedded trial should be replicated to improve generalizability and increase statistical power using meta-analysis. ISRCTN78899574.

Strategies to improve recruitment to randomised trials

BACKGROUND

Recruiting participants to trials can be extremely difficult. Identifying strategies that improve trial recruitment would benefit both trialists and health research.

OBJECTIVES

To quantify the effects of strategies for improving recruitment of participants to randomised trials. A secondary objective is to assess the evidence for the effect of the research setting (e.g. primary care versus secondary care) on recruitment.

SEARCH METHODS

We searched the Cochrane Methodology Review Group Specialised Register (CMR) in the Cochrane Library (July 2012, searched 11 February 2015); MEDLINE and MEDLINE In Process (OVID) (1946 to 10 February 2015); Embase (OVID) (1996 to 2015 Week 06); Science Citation Index & Social Science Citation Index (ISI) (2009 to 11 February 2015) and ERIC (EBSCO) (2009 to 11 February 2015).

SELECTION CRITERIA

Randomised and quasi-randomised trials of methods to increase recruitment to randomised trials. This includes non-healthcare studies and studies recruiting to hypothetical trials. We excluded studies aiming to increase response rates to questionnaires or trial retention and those evaluating incentives and disincentives for clinicians to recruit participants.

DATA COLLECTION AND ANALYSIS

We extracted data on: the method evaluated; country in which the study was carried out; nature of the population; nature of the study setting; nature of the study to be recruited into; randomisation or quasi-randomisation method; and numbers and proportions in each intervention group. We used a risk difference to estimate the absolute improvement and the 95% confidence interval (CI) to describe the effect in individual trials. We assessed heterogeneity between trial results. We used GRADE to judge the certainty we had in the evidence coming from each comparison.

MAIN RESULTS

We identified 68 eligible trials (24 new to this update) with more than 74,000 participants. There were 63 studies involving interventions aimed directly at trial participants, while five evaluated interventions aimed at people recruiting participants. All studies were in health care.We found 72 comparisons, but just three are supported by high-certainty evidence according to GRADE.1. Open trials rather than blinded, placebo trials. The absolute improvement was 10% (95% CI 7% to 13%).2. Telephone reminders to people who do not respond to a postal invitation. The absolute improvement was 6% (95% CI 3% to 9%). This result applies to trials that have low underlying recruitment. We are less certain for trials that start out with moderately good recruitment (i.e. over 10%).3. Using a particular, bespoke, user-testing approach to develop participant information leaflets. This method involved spending a lot of time working with the target population for recruitment to decide on the content, format and appearance of the participant information leaflet. This made little or no difference to recruitment: absolute improvement was 1% (95% CI -1% to 3%).We had moderate-certainty evidence for eight other comparisons; our confidence was reduced for most of these because the results came from a single study. Three of the methods were changes to trial management, three were changes to how potential participants received information, one was aimed at recruiters, and the last was a test of financial incentives. All of these comparisons would benefit from other researchers replicating the evaluation. There were no evaluations in paediatric trials.We had much less confidence in the other 61 comparisons because the studies had design flaws, were single studies, had very uncertain results or were hypothetical (mock) trials rather than real ones.

AUTHORS' CONCLUSIONS

The literature on interventions to improve recruitment to trials has plenty of variety but little depth. Only 3 of 72 comparisons are supported by high-certainty evidence according to GRADE: having an open trial and using telephone reminders to non-responders to postal interventions both increase recruitment; a specialised way of developing participant information leaflets had little or no effect. The methodology research community should improve the evidence base by replicating evaluations of existing strategies, rather than developing and testing new ones.

Identifying research priorities for effective retention strategies in clinical trials

BACKGROUND

The failure to retain patients or collect primary-outcome data is a common challenge for trials and reduces the statistical power and potentially introduces bias into the analysis. Identifying strategies to minimise missing data was the second highest methodological research priority in a Delphi survey of the Directors of UK Clinical Trial Units (CTUs) and is important to minimise waste in research. Our aim was to assess the current retention practices within the UK and priorities for future research to evaluate the effectiveness of strategies to reduce attrition.

METHODS

Seventy-five chief investigators of NIHR Health Technology Assessment (HTA)-funded trials starting between 2009 and 2012 were surveyed to elicit their awareness about causes of missing data within their trial and recommended practices for improving retention. Forty-seven CTUs registered within the UKCRC network were surveyed separately to identify approaches and strategies being used to mitigate missing data across trials. Responses from the current practice surveys were used to inform a subsequent two-round Delphi survey with registered CTUs. A consensus list of retention research strategies was produced and ranked by priority.

RESULTS

Fifty out of seventy-five (67%) chief investigators and 33/47 (70%) registered CTUs completed the current practice surveys. Seventy-eight percent of trialists were aware of retention challenges and implemented strategies at trial design. Patient-initiated withdrawal was the most common cause of missing data. Registered CTUs routinely used newsletters, timeline of participant visits, and telephone reminders to mitigate missing data. Whilst 36 out of 59 strategies presented had been formally or informally evaluated, some frequently used strategies, such as site initiation training, have had no research to inform practice. Thirty-five registered CTUs (74%) participated in the Delphi survey. Research into the effectiveness of site initiation training, frequency of patient contact during a trial, the use of routinely collected data, the frequency and timing of reminders, triggered site training and the time needed to complete questionnaires was deemed critical. Research into the effectiveness of Christmas cards for site staff was not of critical importance.

CONCLUSION

The surveys of current practices demonstrates that a variety of strategies are being used to mitigate missing data but with little evidence to support their use. Six retention strategies were deemed critically important within the Delphi survey and should be a primary focus of future retention research.

The impact of advertising patient and public involvement on trial recruitment: embedded cluster randomised recruitment trial

BACKGROUND

Patient and public involvement in research (PPIR) may improve trial recruitment rates, but it is unclear how. Where trials use PPIR to improve design and conduct, many do not communicate this clearly to potential participants. Better communication of PPIR might encourage patient enrolment, as trials may be perceived as more socially valid, relevant and trustworthy. We aimed to evaluate the impact on recruitment of directly advertising PPIR to potential trial participants.

METHODS

This is a cluster trial, embedded within a host trial ('EQUIP') recruiting service users diagnosed with severe mental illness. The intervention was informed by a systematic review, a qualitative study, social comparison theory and a stakeholder workshop including service users and carers. Adopting Participatory Design approaches, we co-designed the recruitment intervention with PPIR partners using a leaflet to advertise the PPIR in EQUIP and sent potential participants invitations with the leaflet (intervention group) or not (control group). Primary outcome was the proportion of patients enrolled in EQUIP. Secondary outcomes included the proportions of patients who positively responded to the trial invitation.

RESULTS

Thirty-four community mental health teams were randomised and 8182 service users invited. For the primary outcome, 4% of patients in the PPIR group were enrolled versus 5.3% of the control group. The intervention was not effective for improving recruitment rates (adjusted OR = 0.75, 95% CI = 0.53 to 1.07, p = 0.113). For the secondary outcome of positive response, the intervention was not effective, with 7.3% of potential participants in the intervention group responding positively versus 7.9% of the control group (adjusted OR = 0.74, 95% CI = 0.53 to 1.04, p = 0.082). We did not find a positive impact of directly advertising PPIR on any other outcomes.

CONCLUSION

To our knowledge, this is the largest ever embedded trial to evaluate a recruitment or PPIR intervention. Advertising PPIR did not improve enrolment rates or any other outcome. It is possible that rather than advertising PPIR being the means to improve recruitment, PPIR may have an alternative impact on trials by making them more attractive, acceptable and patient-centred. We discuss potential reasons for our findings and implications for recruitment practice and research.

TRIAL REGISTRATION NUMBERS

ISRCTN, ISRCTN16488358 . Registered on 14 May 2014. Study Within A Trial, SWAT-26 . Registered on 21 January 2016.

Strategies designed to help healthcare professionals to recruit participants to research studies

BACKGROUND

Identifying and approaching eligible participants for recruitment to research studies usually relies on healthcare professionals. This process is sometimes hampered by deliberate or inadvertent gatekeeping that can introduce bias into patient selection.

OBJECTIVES

Our primary objective was to identify and assess the effect of strategies designed to help healthcare professionals to recruit participants to research studies.

SEARCH METHODS

We performed searches on 5 January 2015 in the following electronic databases: Cochrane Methodology Register, CENTRAL, MEDLINE, EMBASE, CINAHL, British Nursing Index, PsycINFO, ASSIA and Web of Science (SSCI, SCI-EXPANDED) from 1985 onwards. We checked the reference lists of all included studies and relevant review articles and did citation tracking through Web of Science for all included studies.

SELECTION CRITERIA

We selected all studies that evaluated a strategy to identify and recruit participants for research via healthcare professionals and provided pre-post comparison data on recruitment rates.

DATA COLLECTION AND ANALYSIS

Two review authors independently screened search results for potential eligibility, read full papers, applied the selection criteria and extracted data. We calculated risk ratios for each study to indicate the effect of each strategy.

MAIN RESULTS

Eleven studies met our eligibility criteria and all were at medium or high risk of bias. Only five studies gave the total number of participants (totalling 7372 participants). Three studies used a randomised design, with the others using pre-post comparisons. Several different strategies were investigated. Four studies examined the impact of additional visits or information for the study site, with no increases in recruitment demonstrated. Increased recruitment rates were reported in two studies that used a dedicated clinical recruiter, and five studies that introduced an automated alert system for identifying eligible participants. The studies were embedded into trials evaluating care in oncology mainly but also in emergency departments, diabetes and lower back pain.

AUTHORS' CONCLUSIONS

There is no strong evidence for any single strategy to help healthcare professionals to recruit participants in research studies. Additional visits or information did not appear to increase recruitment by healthcare professionals. The most promising strategies appear to be those with a dedicated resource (e.g. a clinical recruiter or automated alert system) for identifying suitable participants that reduced the demand on healthcare professionals, but these were assessed in studies at high risk of bias.

Guidelines for reporting embedded recruitment trials

Background

Recruitment to clinical trials is difficult with many trials failing to recruit to target and within time. Embedding trials of recruitment interventions within host trials may provide a successful way to improve this. There are no guidelines for reporting such embedded methodology trials. As part of the Medical Research Council funded Systematic Techniques for Assisting Recruitment to Trials (MRC START) programme designed to test interventions to improve recruitment to trials, we developed guidelines for reporting embedded trials.

Methods

We followed a three-phase guideline development process: (1) pre-meeting literature review to generate items for the reporting guidelines; (2) face-to-face consensus meetings to draft the reporting guidelines; and (3) post-meeting feedback review, and pilot testing, followed by finalisation of the reporting guidelines.

Results

We developed a reporting checklist based on the Consolidated Standards for Reporting Trials (CONSORT) statement 2010. Embedded trials evaluating recruitment interventions should follow the CONSORT statement 2010 and report all items listed as essential. We used a number of examples to illustrate key issues that arise in embedded trials and how best to report them, including (a) how to deal with description of the host trial; (b) the importance of describing items that may differ in the host and embedded trials (such as the setting and the eligible population); and (c) the importance of identifying clearly the point at which the recruitment interventions were embedded in the host trial.

Conclusions

Implementation of these guidelines will improve the quality of reports of embedded recruitment trials while advancing the science, design and conduct of embedded trials as a whole.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-015-1126-y) contains supplementary material, which is available to authorized users.

The impact of clinical trial monitoring approaches on data integrity and cost--a review of current literature

PURPOSE

Monitoring is a costly requirement when conducting clinical trials. New regulatory guidance encourages the industry to consider alternative monitoring methods to the traditional 100 % source data verification (SDV) approach. The purpose of this literature review is to provide an overview of publications on different monitoring methods and their impact on subject safety data, data integrity, and monitoring cost.

METHODS

The literature search was performed by keyword searches in MEDLINE and hand search of key journals. All publications were reviewed for details on how a monitoring approach impacted subject safety data, data integrity, or monitoring costs.

RESULTS

Twenty-two publications were identified. Three publications showed that SDV has some value for detection of not initially reported adverse events and centralized statistical monitoring (CSM) captures atypical trends. Fourteen publications showed little objective evidence of improved data integrity with traditional monitoring such as 100 % SDV and sponsor queries as compared to reduced SDV, CSM, and remote monitoring. Eight publications proposed a potential for significant cost reductions of monitoring by reducing SDV without compromising the validity of the trial results.

CONCLUSIONS

One hundred percent SDV is not a rational method of ensuring data integrity and subject safety based on the high cost, and this literature review indicates that reduced SDV is a viable monitoring method. Alternative methods of monitoring such as centralized monitoring utilizing statistical tests are promising alternatives but have limitations as stand-alone tools. Reduced SDV combined with a centralized, risk-based approach may be the ideal solution to reduce monitoring costs while improving essential data quality.

Barriers and facilitators to recruitment in mental health services: care coordinators' expectations and experience of referring to a psychosis research trial

OBJECTIVE

High-quality research trials are necessary to provide evidence for the effective management of mental health difficulties, but successful recruitment can be challenging.

DESIGN

This qualitative study examines the perceived barriers and facilitators to referring mental health service users to research trials. Seven care coordinators (n = 7) who facilitated the recruitment of participants to a cognitive behaviour therapy - informed psychosis intervention trial were interviewed.

METHOD

Demographic information was collected by questionnaire and a semi-structured guide was used to explore barriers and facilitators to referring to a partially randomized participant preference trial. Qualitative data were thematically analysed.

RESULTS

Four key themes, each with a number of sub-themes, were identified: (1) engage the care coordinator in the recruitment process, (2) barriers to referring to research studies, (3) facilitators to referring to research studies; (4) organisational constraints impact on implementing research outcomes into routine clinical practice.

CONCLUSIONS

Understanding the barriers and facilitators to recruitment in mental health research could improve recruitment strategies. Our findings highlight the need for researchers' to closely consider their recruitment strategies as service users are not always given the choice to participate in research. Several key recommendations are made based on these findings in order to maximize successful recruitment to research studies. Overall, we recommend that researchers' adopt a flexible, tailor-made approach for each clinical team to ensure a collaborative relationship is developed between research staff and clinicians.

PRACTITIONER POINTS

A qualitative approach to understanding recruitment challenges provides a useful opportunity to explore the barriers and facilitators to recruiting participants to research studies. These findings have practical implications that highlight the need for a collaborative partnership between researchers and clinical services. Understanding the challenges and issues related to recruitment can help researchers consider strategies to overcome recruitment issues. More research with a larger sample, across a broader population and in different mental health services is required.

SWAT-1: The effectiveness of a 'site visit' intervention on recruitment rates in a multi-centre randomised trial

BACKGROUND

Recruitment rates in multi-centre randomised trials often fall below target recruitment rates, causing problems for study outcomes. The Studies Within A Trial (SWAT) Programme, established by the All-Ireland Hub for Trials Methodology Research in collaboration with the Medical Research Council Network of Hubs in the United Kingdom and others, is developing methods for evaluating aspects of trial methodology through the conduct of research within research. A recently published design for a SWAT-1 provides a protocol for evaluating the effect of a site visit by the principal investigator on recruitment in multi-centre trials.

METHODS

Using the SWAT-1 design, the effect of a site visit, with the sole purpose of discussing trial recruitment, on recruitment rates in a large multicentre trial in the Republic of Ireland was evaluated. A controlled before and after intervention comparison was used, where the date of the site visit provides the time point for the intervention, and for the comparison to control sites. Site A received the intervention. Site B and Site C acted as the controls. Z-scores for proportions were calculated to determine within site recruitment differences. Odds ratios and 95% confidence intervals were calculated to determine between site recruitment differences.

RESULTS

Recruitment rates were increased in Site A post-intervention (17% and 14% percentage point increases at 1 and 3 months, respectively). No differences in recruitment occurred in Site B or in Site C. Comparing between site differences, at 3 months post-intervention, a statistically significant difference was detected in favour of higher recruitment in Site A (34% versus 25%; odds ratio 1.57, 95% confidence interval 1.09 to 2.26).

CONCLUSIONS

This is the first reported example of a study in the SWAT programme.. It provides evidence that a site visit, combined with a scheduled meeting, increases recruitment in a clinical trial. Using this example, other researchers might be encouraged to consider conducting a similar study, allowing the findings of future SWAT-1s to be compared and combined, so that higher level evidence on the effect of a site visit by the principal investigator can be obtained.

THE ADCAR TRIAL

ISRCTN-96340041 ( www.controlled-trials.com ); date of registration: 25 March 2008.

A systematic review of strategies used to increase recruitment of people with cancer or organ failure into clinical trials: implications for palliative care research

CONTEXT

The challenges of palliative care clinical trial recruitment are well documented.

OBJECTIVES

The aim of the study was to review tested strategies to improve recruitment to trials of people with a range of conditions who may access palliative care services but are not explicitly stated to be "palliative."

METHODS

This was a systematic review with narrative description. The Cochrane, Embase, PubMed, PsycINFO, and CINAHL electronic databases were searched (English; January 2002 to February 2014) for quasi-experimental and randomized controlled trials (RCTs) testing the effect of recruitment strategies on accrual to clinical trials of people with organ failure and cancer. Titles, abstracts, and retrieved articles were screened by two researchers and categorized by recruitment challenge: 1) patients with reduced cognition, 2) those requiring emergency treatment, and 3) willingness of patients and clinical staff to contribute to trials.

RESULTS

Of 549 articles identified, 15 were included. Thirteen reported RCTs and two papers reported three quasi-experimental studies. Five were cluster RCTs of recruiting sites/institutions. One was a randomized cluster, crossover, feasibility study. Seven studies recruited patients with cancer. Others included patients with dementia, stroke, cardiovascular disease, diabetes, frail elderly, and bereaved carers. Some interventions improved recruitment: memory aid, contact before arrival, cluster consent, "opt out" consent. Others either reduced recruitment (formal mental capacity assessment) or made no difference (advance research directive; a variety of educational, supportive, and advertising interventions).

CONCLUSION

Successful strategies from other disciplines could be considered by palliative care researchers. Tailored, efficient, evidence-based strategies must be developed, acknowledging that strategies with face validity are not necessarily the most effective.

INVESTIGATING THE EFFICACY OF CLINICAL TRIAL MONITORING STRATEGIES: Design and Implementation of the Cluster Randomized START Monitoring Substudy

BACKGROUND

Trial monitoring protects participant safety and study integrity. While monitors commonly go on-site to verify source data, there is little evidence that this practice is efficient or effective. An ongoing international HIV treatment trial (START) provides an opportunity to explore the usefulness of different monitoring approaches.

METHODS

All START sites are centrally monitored and required to follow a local monitoring plan requiring specific quality assurance activities. Additionally, sites were randomized (1:1) to receive, or not receive, annual on-site monitoring. The study will determine if on-site monitoring increases the identification of major protocol deviations (eligibility or consent violations, improper study drug use, primary or serious event underreporting, data alteration or fraud).

RESULTS

The START study completed enrollment in December 2013, with planned follow-up through December 2016. The monitoring study is ongoing at 196 sites in 34 countries. Results are expected when the START study concludes in December 2016.

Researchers' and clinicians' perceptions of recruiting participants to clinical research: a thematic meta-synthesis

Background

Recruiting the desired number of research participants is frequently problematic with resulting financial and clinical implications. The views of individuals responsible for participant recruitment have not been previously reviewed. This systematic review and thematic meta-synthesis explores researchers’ and clinicians’ experiences and perceptions of recruiting participants to clinical research, with the aim of informing improved recruitment systems and strategies.

Methods

Studies published between January 1995 and May 2013 were identified from: Ovid MEDLINE, Ovid EMBASE, Ovid PSYCHINFO, ASSIA, British Nursing Index, Scopus, Web of Science, CINAHL and PubMed. Included studies were original peer reviewed research, with qualitative methodologies and an aim of exploring the views of clinicians and/or researchers on recruitment to clinical research. Studies discussing the recruitment of patients unable to give informed consent were excluded. The findings sections of the relevant studies were free coded to identify key concepts which were grouped into hierarchical themes. The quality of the identified studies was assessed and the relative contribution of each paper was checked to ensure individual studies did not dominate in any theme.

Results

Eighteen relevant papers were identified which examined the views of researchers and clinicians in 10 clinical specialties. Five main themes emerged: building a research community, securing resources, the nature of research, professional identities and recruitment strategies. The views of researchers and clinicians were similar, although the role of ‘researcher’ was inconsistently defined.

Conclusions

The general experience of recruiting participants to clinical research was one of competition and compromise. Competition arose over funding, staffing and participants, and between clinical and research responsibilities. Compromise was needed to create study designs that were acceptable to patients, clinicians and researchers. Forging relationships between clinical and research teams featured extensively, however the involvement of patients and the public within the research community was rarely discussed.

Meeting the challenges of recruitment to multicentre, community-based, lifestyle-change trials: a case study of the BeWEL trial

Background

Recruiting participants to multicentre, community-based trials is a challenge. This case study describes how this challenge was met for the BeWEL trial, which evaluated the impact of a diet and physical activity intervention on body weight in people who had had pre-cancerous bowel polyps.

Methods

The BeWEL trial was a community-based trial, involving centres linked to the Scottish National Health Service (NHS) colorectal cancer screening programme. BeWEL had a recruitment target of 316 and its primary recruitment route was the colonoscopy clinics of the Scottish Bowel Screening Programme.

Results

BeWEL exceeded its recruitment target but needed a 6-month no-cost extension from the funder to achieve this. The major causes of delay were lower consent rates (49% as opposed to 70% estimated from earlier work), the time taken for NHS research and development department approvals and the inclusion of two additional sites to increase recruitment, for which there were substantial bureaucratic delays. A range of specific interventions to increase recruitment, for example, telephone reminders and a shorter participant information leaflet, helped to increase the proportion of eligible individuals consenting and being randomized.

Conclusions

Recruitment to multicentre trials is a challenge but can be successfully achieved with a committed team. In a UK context, NHS research and development approval can be a substantial source of delay. Investigators should be cautious when estimating consent rates. If consent rates are less than expected, qualitative analysis might be beneficial, to try and identify the reason. Finally, investigators should select trial sites on the basis of a formal assessment of a site’s past performance and the likelihood of success in the trial being planned.

Trial registration

Current Controlled Trials ISRCTN53033856

Central site monitoring: Results from a test of accuracy in identifying trials and sites failing Food and Drug Administration inspection

Background

Site monitoring and source document verification account for 15%‒30% of clinical trial costs. An alternative is to streamline site monitoring to focus on correcting trial-specific risks identified by central data monitoring. This risk-based approach could preserve or even improve the quality of clinical trial data and human subject protection compared to site monitoring focused primarily on source document verification.

Purpose

To determine whether a central review by statisticians using data submitted to the Food and Drug Administration (FDA) by clinical trial sponsors can identify problem sites and trials that failed FDA site inspections.

Methods

An independent Analysis Center (AC) analyzed data from four anonymous new drug applications (NDAs) where FDA had performed site inspections overseen by FDA’s Office of Scientific Investigations (OSI). FDA team members in the OSI chose the four NDAs from among all NDAs with data in Study Data Tabulation Model (SDTM) format. Two of the NDAs had data that OSI had deemed unreliable in support of the application after FDA site inspections identified serious data integrity problems. The other two NDAs had clinical data that OSI deemed reliable after site inspections. At the outset, the AC knew only that the experimental design specified two NDAs with significant problems. FDA gave the AC no information about which NDAs had problems, how many sites were inspected, or how many were found to have problems until after the AC analysis was complete. The AC evaluated randomization balance, enrollment patterns, study visit scheduling, variability of reported data, and last digit reference. The AC classified sites as ‘High Concern’, ‘Moderate Concern’, ‘Mild Concern’, or ‘No Concern’.

Results

The AC correctly identified the two NDAs with data deemed unreliable by OSI. In addition, central data analysis correctly identified 5 of 6 (83%) sites for which FDA recommended rejection of data and 13 of 15 sites (87%) for which any regulatory deviations were identified during inspection. Of the six sites for which OSI reviewed inspections and found no deviations, the central process flagged four at the lowest level of concern, one at a moderate level, and one was not flagged.

Limitations

Central data monitoring during the conduct of a trial while data checking was in progress was not evaluated.

Conclusion

Systematic central monitoring of clinical trial data can identify problems at the same trials and sites identified during FDA site inspections. Central data monitoring in conjunction with an overall monitoring process that adapts to identify risks as a trial progresses has the potential to reduce the frequency of site visits while increasing data integrity and decreasing trial costs compared to processes that are dependent primarily on source documentation.

A systematic review of on-site monitoring methods for health-care randomised controlled trials

BACKGROUND

Monitoring the conduct of clinical trials is recommended by International Conference of Harmonisation Good Clinical Practice (ICH GCP) guidelines and is integral to trial quality assurance. On-site monitoring, that is, visiting trial sites, is one part of this process but little is known about the procedures that are performed in practice.

PURPOSE

To examine and summarise published on-site monitoring methods for health-care clinical trials, including evaluations of their benefits and costs to trials.

METHODS

A systematic literature review identified all articles reporting the methods and practices of on-site monitoring of randomised controlled trials (RCTs). Articles were categorised into (1) reports from research groups and organisations, (2) reports from individual RCTs, (3) randomised trials of on-site monitoring interventions, (4) cost simulations, or (5) surveys of trial staff and monitors. Data were extracted on the characteristics of the trials and groups reporting on-site monitoring (e.g., geographical origin, sponsor, and trial focus). Information from articles in categories (1)-(3) was summarised on the frequency and scope of site monitoring visits, monitoring team size and composition, activities during site visits, and reporting structures. Evaluations of the benefits and disadvantages of on-site monitoring were examined for all included articles.

RESULTS

In total, 57 articles were identified, comprising 21 articles about the on-site monitoring practices of 16 research groups, 30 articles from 26 RCTs, 1 on-site monitoring intervention RCT, 2 cost simulations, and 3 surveys. Publications in categories (1)-(3), mostly originated from the United States (33/52, 63%) or Europe (15/52, 29%), were predominantly describing non-commercial organisations or trials (45/52, 87%), with heart disease (9/26, 35%) or cancer (5/26, 19%) the commonest focus of individual RCTs. The frequency of visits ranged from every 6-8 weeks up to once every 3 years, with mostly all trial sites visited. The number of monitors visiting a site varied between 1 and 8. The most common on-site monitoring activity was verifying source data and consent forms, with a focus on data accuracy. Only six articles evaluated their on-site monitoring process, with improvements observed in recruitment rates and protocol adherence but with direct costs and staff time viewed as the major disadvantages. The on-site monitoring RCT ended prematurely so preventing full assessment.

LIMITATIONS

Trialists and organisations may utilise additional unpublished on-site monitoring systems. The varied terminology used to describe monitoring may have limited identification of some relevant articles.

CONCLUSIONS

This review demonstrated that on-site monitoring is utilised in trials worldwide but systems vary considerably with little evidence to support practice. These on-site monitoring practices need to be evaluated empirically, including costs, to provide robust evidence for the contribution of site visits to trial performance and quality.

[Public clinical trials: which kind of monitoring should be used?]

OBJECTIVE

Sponsors must take responsibility for the quality of trials at the best possible cost. Our objective was to describe the most frequent quality failures, how they impact trial results, and identify the most efficient monitoring strategies using published articles and reports.

RESULTS

Errors affecting clinical trials include conception, procedures, data management, and data analysis. The consequences are usually an overestimation of the treatment effect. No study shows that monitoring reduces the risk of errors, and there is no comparison of monitoring methods. Many research organisations advocate for monitoring based on risk analysis and recommend an extensive use of centralised monitoring.

CONCLUSIONS

Trial quality depends on trial conception and design. Study conduct should guarantee a maximum level of quality level. This should be done using risk management and extensive centralised monitoring.

Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis

This review is an abridged version of a Cochrane Review previously published in the Cochrane Database of Systematic Reviews 2010, Issue 4, Art. No.: MR000013 DOI: 10.1002/14651858.MR000013.pub5 (see www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as new evidence emerges and in response to feedback, and Cochrane Database of Systematic Reviews should be consulted for the most recent version of the review.

OBJECTIVE

To identify interventions designed to improve recruitment to randomised controlled trials, and to quantify their effect on trial participation.

DESIGN

Systematic review.

DATA SOURCES

The Cochrane Methodology Review Group Specialised Register in the Cochrane Library, MEDLINE, EMBASE, ERIC, Science Citation Index, Social Sciences Citation Index, C2-SPECTR, the National Research Register and PubMed. Most searches were undertaken up to 2010; no language restrictions were applied.

STUDY SELECTION

Randomised and quasi-randomised controlled trials, including those recruiting to hypothetical studies. Studies on retention strategies, examining ways to increase questionnaire response or evaluating the use of incentives for clinicians were excluded. The study population included any potential trial participant (eg, patient, clinician and member of the public), or individual or group of individuals responsible for trial recruitment (eg, clinicians, researchers and recruitment sites). Two authors independently screened identified studies for eligibility.

RESULTS

45 trials with over 43 000 participants were included. Some interventions were effective in increasing recruitment: telephone reminders to non-respondents (risk ratio (RR) 1.66, 95% CI 1.03 to 2.46; two studies, 1058 participants), use of opt-out rather than opt-in procedures for contacting potential participants (RR 1.39, 95% CI 1.06 to 1.84; one study, 152 participants) and open designs where participants know which treatment they are receiving in the trial (RR 1.22, 95% CI 1.09 to 1.36; two studies, 4833 participants). However, the effect of many other strategies is less clear, including the use of video to provide trial information and interventions aimed at recruiters.

CONCLUSIONS

There are promising strategies for increasing recruitment to trials, but some methods, such as open-trial designs and opt-out strategies, must be considered carefully as their use may also present methodological or ethical challenges. Questions remain as to the applicability of results originating from hypothetical trials, including those relating to the use of monetary incentives, and there is a clear knowledge gap with regard to effective strategies aimed at recruiters.

The value of source data verification in a cancer clinical trial

BACKGROUND

Source data verification (SDV) is a resource intensive method of quality assurance frequently used in clinical trials. There is no empirical evidence to suggest that SDV would impact on comparative treatment effect results from a clinical trial.

METHODS

Data discrepancies and comparative treatment effects obtained following 100% SDV were compared to those based on data without SDV. Overall survival (OS) and Progression-free survival (PFS) were compared using Kaplan-Meier curves, log-rank tests and Cox models. Tumour response classifications and comparative treatment Odds Ratios (ORs) for the outcome objective response rate, and number of Serious Adverse Events (SAEs) were compared. OS estimates based on SDV data were compared against estimates obtained from centrally monitored data.

FINDINGS

Data discrepancies were identified between different monitoring procedures for the majority of variables examined, with some variation in discrepancy rates. There were no systematic patterns to discrepancies and their impact was negligible on OS, the primary outcome of the trial (HR (95% CI): 1.18(0.99 to 1.41), p = 0.064 with 100% SDV; 1.18(0.99 to 1.42), p = 0.068 without SDV; 1.18(0.99 to 1.40), p = 0.073 with central monitoring). Results were similar for PFS. More extreme discrepancies were found for the subjective outcome overall objective response (OR (95% CI): 1.67(1.04 to 2.68), p = 0.03 with 100% SDV; 2.45(1.49 to 4.04), p = 0.0003 without any SDV) which was mostly due to differing CT scans.

INTERPRETATION

Quality assurance methods used in clinical trials should be informed by empirical evidence. In this empirical comparison, SDV was expensive and identified random errors that made little impact on results and clinical conclusions of the trial. Central monitoring using an external data source was a more efficient approach for the primary outcome of OS. For the subjective outcome objective response, an independent blinded review committee and tracking system to monitor missing scan data could be more efficient than SDV.

A systematic search for reports of site monitoring technique comparisons in clinical trials

BACKGROUND

As part of a broader methodological programme of work around clinical trial monitoring, we wanted to evaluate the existing evidence for the effectiveness of different monitoring techniques.

PURPOSE

To identify and evaluate prospective studies of the effectiveness of different monitoring strategies.

METHODS

A systematic search of MEDLINE from 1950 onwards, using free-text terms to identify relevant published studies. We intended to extract data on details of comparative techniques, monitoring findings identified by different techniques, and recommendations or identification of areas in need of further research made by authors.

RESULTS

A total of 1222 published abstracts were identified and reviewed. Of these, nine articles described methods for quality control (QC) of clinical trial activities, and one article was identified that compared the same monitoring technique at two timepoints. None included a direct comparison of different monitoring techniques and findings.

LIMITATIONS

The search strategy was limited to MEDLINE. However, MEDLINE includes all the journals that tend to report trial methodological research.

CONCLUSIONS

There is a lack of published empirical data that compare monitoring strategies prospectively. Assessment of the usefulness and cost-effectiveness of monitoring techniques in a variety of clinical trial settings and indications is needed.

A statistical approach to central monitoring of data quality in clinical trials

Background

Classical monitoring approaches rely on extensive on-site visits and source data verification. These activities are associated with high cost and a limited contribution to data quality. Central statistical monitoring is of particular interest to address these shortcomings.

Purpose

This article outlines the principles of central statistical monitoring and the challenges of implementing it in actual trials.

Methods

A statistical approach to central monitoring is based on a large number of statistical tests performed on all variables collected in the database, in order to identify centers that differ from the others. The tests generate a high-dimensional matrix of p-values, which can be analyzed by statistical methods and bioinformatic tools to identify extreme centers.

Results

Results from actual trials are provided to illustrate typical findings that can be expected from a central statistical monitoring approach, which detects abnormal patterns that were not (or could not have been) detected by on-site monitoring.

Limitations

Central statistical monitoring can only address problems present in the data. Important aspects of trial conduct such as a lack of informed consent documentation, for instance, require other approaches. The results provided here are empirical examples from a limited number of studies.

Conclusion

Central statistical monitoring can both optimize on-site monitoring and improve data quality and as such provides a cost-effective way of meeting regulatory requirements for clinical trials.

Improving the recruitment activity of clinicians in randomised controlled trials: a systematic review

Background Poor recruitment to randomised controlled trials (RCTs) is a widespread problem. Provision of interventions aimed at supporting or incentivising clinicians may improve recruitment to RCTs. Objectives To quantify the effects of strategies aimed at improving the recruitment activity of clinicians in RCTs, complemented with a synthesis of qualitative evidence related to clinicians' attitudes towards recruiting to RCTs. Data sources A systematic review of English and non-English articles identified from: The Cochrane Library, Ovid MEDLINE, Ovid EMBASE, Ovid PsycINFO, Ebsco CINAHL, Index to Theses and Open SIGLE from 2001 to March 2011. Additional reports were identified through citation searches of included articles. Study eligibility criteria Quantitative studies were included if they evaluated interventions aimed at improving the recruitment activity of clinicians or compared recruitment by different groups of clinicians. Information about host trial, study design, participants, interventions, outcomes and host RCT was extracted by one researcher and checked by another. Studies that met the inclusion criteria were assessed for quality using a standardised tool, the Effective Public Health Practice Project tool. Qualitative studies were included if they investigated clinicians' attitudes to recruiting patients to RCTs. All results/findings were extracted, and content analysis was carried out. Overarching themes were abstracted, followed by a metasummary analysis. Studies that met the inclusion criteria were assessed for quality using the Critical Appraisal Skills Programme qualitative checklist. Data extraction Data extraction was carried out by one researcher using predefined data fields, including study quality indicators, and verified by another. Results Eight quantitative studies were included describing four interventions and a comparison of recruiting clinicians. One study was rated as strong, one as moderate and the remaining six as weak when assessed for quality using the Effective Public Health Practice Project tool. Effective interventions included the use of qualitative research to identify and overcome barriers to recruitment, reduction of the clinical workload associated with participation in RCTs and the provision of extra training and protected research time. Eleven qualitative studies were identified, and eight themes were abstracted from the data: understanding of research, communication, perceived patient barriers, patient-clinician relationship, effect on patients, effect on clinical practice, individual benefits for clinicians and methods associated with successful recruitment. Metasummary analysis identified the most frequently reported subthemes to be: difficulty communicating trial methods, poor understanding of research and priority given to patient well-being. Overall, the qualitative studies were found to be of good quality when assessed using the Critical Appraisal Skills Programme checklist. Conclusions There were few high-quality trials that tested interventions to improve clinicians' recruitment activity in RCTs. The most promising intervention was the use of qualitative methods to identify and overcome barriers to clinician recruitment activity. More good quality studies of interventions are needed to add to the evidence base. The metasummary of qualitative findings identified understanding and communicating RCT methods as a key target for future interventions to improve recruitment. Reinforcement of the potential benefits, both for clinicians and for their patients, could also be a successful factor in improving recruitment. A bias was found towards investigating barriers to recruitment, so future work should also encompass a focus on successfully recruiting trials.

The potential for central monitoring techniques to replace on-site monitoring: findings from an international multi-centre clinical trial

Background Compliance with Good Clinical Practice (GCP) guidelines should ensure the safety of trial participants and the reliability of trial results. Over the last decade, increasing emphasis has been placed on the role of costly on-site monitoring and source data verification as processes to demonstrate that GCP is being followed, despite a lack of empirical evidence that these are effective.

Purpose To assess whether findings from on-site monitoring of a recent international multi-centre clinical trial could have been identified using central data review and other centralised monitoring techniques.

Methods Findings documented in a sample of site monitoring reports, and Programme Management Board Executive (PMBe) reports, from the Microbicides Development Programme (MDP) 301 trial – a randomised placebo-controlled trial of a microbicide gel to prevent vaginally acquired HIV infection conducted in four countries in East and Southern Africa – were extracted and individually assessed to determine whether they could have been detected in the trial database or through other central means.

Results Four site visit reports contained 268 monitoring findings from a review of 104 participant files covering 324 study visits. Of the 268 findings, 76 (28.4%) were also identified in the study database. Central checks, had these been in place (such as central receipt and review of back-translated documents, enrolment and testing logs, informed consent, and more complex database queries), could have identified a further 179 (66.8%); 13 (4.9%) other findings (all minor) could have been identified through a review of the participant folder at site. The four PMBe reports reviewed included six major and three critical findings from a review of over 1000 participant files: only two of these (both major) were assessed as unlikely to be identified using central monitoring techniques.

Limitations The study data used were not collected with this retrospective review in mind. It suggests that prospective work is needed to compare monitoring practices in real time.

Conclusions While there may be some categories of findings that it is not possible to identify centrally, the very large majority of findings reviewed in this analysis could be identified using central monitoring strategies. These data suggest that with better central and targeted on-site monitoring, it should be possible to identify and address most protocol and procedural compliance issues without performing intensive and costly routine on-site data monitoring.

A Peer Review Intervention for Monitoring and Evaluating sites (PRIME) that improved randomized controlled trial conduct and performance

OBJECTIVE

Good clinical practice (GCP) guidelines emphasize trial site monitoring, although the implementation is unspecified and evidence for benefit is sparse. We aimed to develop a site monitoring process using peer reviewers to improve staff training, site performance, data collection, and GCP compliance.

STUDY DESIGN AND SETTING

The Peer Review Intervention for Monitoring and Evaluating sites (PRIME) team observed and gave feedback on trial recruitment and follow-up appointments, held staff meetings, and examined documentation during annual 2-day site visits. The intervention was evaluated in the ProtecT trial, a UK randomized controlled trial of localized prostate cancer treatments (ISRCTN20141297). The ProtecT coordinator and senior nurses conducted three monitoring rounds at eight sites (2004-2007). The process evaluation used PRIME report findings, trial databases, resource use, and a site nurse survey.

RESULTS

Adverse findings decreased across all sites from 44 in round 1 to 19 in round 3. Most findings related to protocol adherence or site organizational issues, including improvements in eligibility criteria application and data collection. Staff found site monitoring acceptable and made changes after reviews.

CONCLUSION

The PRIME process used observation by peer reviewers to improve protocol adherence and train site staff, which increased trial performance and consistency.

Validation of a risk-assessment scale and a risk-adapted monitoring plan for academic clinical research studies--the Pre-Optimon study

CONTEXT

Good Clinical Practice regulates monitoring activities in clinical research. Due to question and design diversity, and limited resources, on-site monitoring is often less intensive in the academic context, and variable. Standardization is needed, and relies on definition and validation of tools accounting for risk.

OBJECTIVE

To define, and validate tools, to implement a risk-based monitoring strategy for academic clinical research.

METHODS

Working groups of experienced professionals searched the literature, and built a consensus risk-assessment scale (RAS), and a risk-adapted monitoring plan (RAMP). We allocated 200 protocols to 49 assessors. We assessed the RAS relevance vs. a visual analogue scale (VAS), and its reproducibility through Kraemer's kappa, and intraclass correlation coefficient (ICC) from a random proportional odds model. We identified sources of disagreement through a logistic regression. We described assessors' difficulties during assessment. We applied the RAMP to 10 protocols per risk level, and rated its feasibility (0 = easy to 4 = impossible).

RESULTS

RAS and RAMP were defined in 4 levels. RAS relevance was good: RAS-risk levels were evenly distributed on VAS-risk (0.6, 2.6, 5.6, and 7.9). Reproducibility was moderate to good: kappa=0.48, ICC=0.70. Major disagreements (36%) arose from decision-makers, rather than hands-on managers. Most difficulties occurred in ill-written protocols (17%). RAMP was easily feasible for most protocols (mean score: 0.2 to 0.9). We proposed a standard synopsis for evaluation purpose.

CONCLUSION

We defined, and validated risk-based tools. This risk-adapted strategy will be compared to an intensive one in a randomized trial, Optimon, to define a standard of practice for academic clinical research.

Strategies to improve recruitment to randomised controlled trials

BACKGROUND

Recruiting participants to trials can be extremely difficult. Identifying strategies that improve trial recruitment would benefit both trialists and health research.

OBJECTIVES

To quantify the effects of strategies to improve recruitment of participants to randomised controlled trials.

SEARCH STRATEGY

We searched the Cochrane Methodology Review Group Specialised Register - CMR (The Cochrane Library (online) Issue 1 2008) (searched 20 February 2008); MEDLINE, Ovid (1950 to date of search) (searched 06 May 2008); EMBASE, Ovid (1980 to date of search) (searched 16 May 2008); ERIC, CSA (1966 to date of search) (searched 19 March 2008); Science Citation Index Expanded, ISI Web of Science (1975 to date of search) (searched 19 March 2008); Social Sciences Citation Index, ISI Web of Science (1975 to date of search) (searched 19 March 2008); and National Research Register (online) (Issue 3 2007) (searched 03 September 2007); C2-SPECTR (searched 09 April 2008). We also searched PubMed (25 March 2008) to retrieve "related articles" for 15 studies included in a previous version of this review.

SELECTION CRITERIA

Randomised and quasi-randomised controlled trials of methods to increase recruitment to randomised controlled trials. This includes non-healthcare studies and studies recruiting to hypothetical trials. Studies aiming to increase response rates to questionnaires or trial retention, or which evaluated incentives and disincentives for clinicians to recruit patients were excluded.

DATA COLLECTION AND ANALYSIS

Data were extracted on the method evaluated; country in which the study was carried out; nature of the population; nature of the study setting; nature of the study to be recruited into; randomisation or quasi-randomisation method; and numbers and proportions in each intervention group. We used risk ratios and their 95% confidence intervals to describe the effects in individual trials, and assessed heterogeneity of these ratios between trials.

MAIN RESULTS

We identified 27 eligible trials with more than 26,604 participants. There were 24 studies involving interventions aimed directly at trial participants, while three evaluated interventions aimed at people recruiting participants. All studies were in health care. Some interventions were effective in increasing recruitment: telephone reminders to non-respondents (RR 2.66, 95% CI 1.37 to 5.18), use of opt-out, rather than opt-in, procedures for contacting potential trial participants (RR 1.39, 95% CI 1.06 to 1.84) and open designs where participants know which treatment they are receiving in the trial (RR 1.25, 95% CI 1.18 to 1.34). However, some of these strategies have disadvantages, which may limit their widespread use. For example, opt-out procedures are controversial and open designs are by definition unblinded. The effects of many other recruitment strategies are unclear; examples include the use of video to provide trial information to potential participants and modifying the training of recruiters. Many studies looked at recruitment to hypothetical trials and it is unclear how applicable these results are to real trials.

AUTHORS' CONCLUSIONS

Trialists can increase recruitment to their trials by using the strategies shown to be effective in this review: telephone reminders; use of opt-out, rather than opt-in; procedures for contacting potential trial participants and open designs. Some strategies (e.g. open trial designs) need to be considered carefully before use because they also have disadvantages. For example, opt-out procedures are controversial and open designs are by definition unblinded.

Strategies to improve recruitment to randomised controlled trials

BACKGROUND

Recruiting participants to trials can be extremely difficult. Identifying strategies that improve trial recruitment would benefit both trialists and health research.

OBJECTIVES

To quantify the effects of strategies to improve recruitment of participants to randomised controlled trials.

SEARCH STRATEGY

We searched the Cochrane Methodology Review Group Specialised Register - CMR (The Cochrane Library (online) Issue 1 2008) (searched 20 February 2008); MEDLINE, Ovid (1950 to date of search) (searched 06 May 2008); EMBASE, Ovid (1980 to date of search) (searched 16 May 2008); ERIC, CSA (1966 to date of search) (searched 19 March 2008); Science Citation Index Expanded, ISI Web of Science (1975 to date of search) (searched 19 March 2008); Social Sciences Citation Index, ISI Web of Science (1975 to date of search) (searched 19 March 2008); and National Research Register (online) (Issue 3 2007) (searched 03 September 2007); C2-SPECTR (searched 09 April 2008). We also searched PubMed (25 March 2008) to retrieve "related articles" for 15 studies included in a previous version of this review.

SELECTION CRITERIA

Randomised and quasi-randomised controlled trials of methods to increase recruitment to randomised controlled trials. This includes non-healthcare studies and studies recruiting to hypothetical trials. Studies aiming to increase response rates to questionnaires or trial retention, or which evaluated incentives and disincentives for clinicians to recruit patients were excluded.

DATA COLLECTION AND ANALYSIS

Data were extracted on the method evaluated; country in which the study was carried out; nature of the population; nature of the study setting; nature of the study to be recruited into; randomisation or quasi-randomisation method; and numbers and proportions in each intervention group. We used risk ratios and their 95% confidence intervals to describe the effects in individual trials, and assessed heterogeneity of these ratios between trials.

MAIN RESULTS

We identified 27 eligible trials with more than 26,604 participants. There were 24 studies involving interventions aimed directly at trial participants, while three evaluated interventions aimed at people recruiting participants. All studies were in health care. Some interventions were effective in increasing recruitment: telephone reminders to non-respondents (RR 2.66, 95% CI 1.37 to 5.18), use of opt-out, rather than opt-in, procedures for contacting potential trial participants (RR 1.39, 95% CI 1.06 to 1.84) and open designs where participants know which treatment they are receiving in the trial (RR 1.25, 95% CI 1.18 to 1.34). However, some of these strategies have disadvantages, which may limit their widespread use. For example, opt-out procedures are controversial and open designs are by definition unblinded. The effects of many other recruitment strategies are unclear; examples include the use of video to provide trial information to potential participants and modifying the training of recruiters. Many studies looked at recruitment to hypothetical trials and it is unclear how applicable these results are to real trials.

AUTHORS' CONCLUSIONS

Trialists can increase recruitment to their trials by using the strategies shown to be effective in this review: telephone reminders; use of opt-out, rather than opt-in; procedures for contacting potential trial participants and open designs. Some strategies (e.g. open trial designs) need to be considered carefully before use because they also have disadvantages. For example, opt-out procedures are controversial and open designs are by definition unblinded.

Risk analysis and risk adapted on-site monitoring in noncommercial clinical trials

BACKGROUND

The concept of risk assessment for clinical trials has been discussed before, but no comprehensive structured procedure leading to risk-adapted quality management has been published so far. Such a procedure is of particular interest for noncommercial trials in order to optimally use the sparse resources.

PURPOSE

To provide a structured procedure for risk analysis in clinical trials. To propose strategies for on-site monitoring adapted to the risks identified.

RESULTS

The risk analysis refers to the risk of noncompliance with the main objectives of Good Clinical Practice. It takes into account risks of the study intervention compared to the risks a patient would run if treated outside a protocol as well as further potential risks regarding patient safety, patient rights, or the credibility of results. The risk analysis is based on detailed questionnaires, which are used to draw up (a) an on-site monitoring strategy recommendation, (b) a list of trial-specific tasks to be covered by on-site monitoring, and (c) a specification of further quality management measures e.g., central monitoring measures. The resulting risk-adapted monitoring strategies focus on the trial's critical aspects, and differ in terms of the recommended extent of on-site activities.

LIMITATIONS

The effectiveness of the proposed risk analysis and risk-adapted monitoring has not yet been confirmed. However, the ADAMON project (prospective cluster-randomised study of trial-specific adapted strategies for on-site monitoring in combination with additional quality management measures) has been started in Germany to investigate whether a trial-specific, risk-adapted, reduced on-site monitoring strategy is as effective as an intensive monitoring strategy with regard to the occurrence of serious or critical audit findings. Twelve clinical trials planning to recruit more than 3200 patients participate in this investigation.

CONCLUSIONS

Our proposal will provide sponsor-investigators and other noncommercial sponsors with an instrument that may facilitate risk analysis and the implementation of targeted quality management measures.

Development of a complex intervention improved randomization and informed consent in a randomized controlled trial

OBJECTIVE

Multicenter randomized trials are required for pragmatic evaluations of health care interventions, but recruitment is difficult. Systematic reviews failed to identify robust strategies to improve recruitment. We developed and evaluated a complex intervention to increase levels of randomization and informed consent.

STUDY DESIGN AND SETTING

The ProtecT (Prostate testing for cancer and Treatment) trial compares radical surgery, radical conformal radiotherapy, and active monitoring for men aged 50-69 years with localized prostate cancer. The intervention was developed using qualitative research methods (content, thematic and conversation analysis). Rates of randomization and immediate acceptance of allocation were measured every 6 months to evaluate the impact of the intervention.

RESULTS

The complex intervention comprised reviews of centers falling below study targets, training programmes, documents and individually tailored feedback. Over 65% of eligible participants consented to randomization. Trial participants became increasingly well informed as immediate acceptance of allocation rose from 65% to 81% between 2001 and 2005.

CONCLUSION

This complex intervention resulted in high levels of randomization and informed consent in a difficult trial. The generic aspects of the intervention could be applied to other trials to maximize randomization and informed consent, and allow the mounting of trials previously considered too difficult.