A randomized evaluation of on-site monitoring nested in a multinational randomized trial

Approaches and experiences implementing remote, electronic consent at the Leeds Clinical Trials Research Unit

BACKGROUND

Use of electronic methods to support informed consent ('eConsent') is increasingly popular in clinical research. This commentary reports the approach taken to implement electronic consent methods and subsequent experiences from a range of studies at the Leeds Clinical Trials Research Unit (CTRU), a large clinical trials unit in the UK.

MAIN TEXT

We implemented a remote eConsent process using the REDCap platform. The process can be used in trials of investigational medicinal products and other intervention types or research designs. Our standard eConsent system focuses on documenting informed consent, with other aspects of consent (e.g. providing information to potential participants and a recruiter discussing the study with each potential participant) occurring outside the system, though trial teams can use electronic methods for these activities where they have ethical approval. Our overall process includes a verbal consent step prior to confidential information being entered onto REDCap and an identity verification step in line with regulator guidance. We considered the regulatory requirements around the system's generation of source documents, how to ensure data protection standards were upheld and how to monitor informed consent within the system. We present four eConsent case studies from the CTRU: two randomised clinical trials and two other health research studies. These illustrate the ways eConsent can be implemented, and lessons learned, including about differences in uptake.

CONCLUSIONS

We successfully implemented a remote eConsent process at the CTRU across multiple studies. Our case studies highlight benefits of study participants being able to give consent without having to be present at the study site. This may better align with patient preferences and trial site needs and therefore improve recruitment and resilience against external shocks (such as pandemics). Variation in uptake of eConsent may be influenced more by site-level factors than patient preferences, which may not align well with the aspiration towards patient-centred research. Our current process has some limitations, including the provision of all consent-related text in more than one language, and scalability of implementing more than one consent form version at a time. We consider how enhancements in CTRU processes, or external developments, might affect our approach.

Monitoring metrics over time: Why clinical trialists need to systematically collect site performance metrics

Background

Over the last decade, there has been an increasing interest in risk-based monitoring (RBM) in clinical trials, resulting in a number of guidelines from regulators and its inclusion in ICH GCP. However, there is a lack of detail on how to approach RBM from a practical perspective, and insufficient understanding of best practice.

Purpose

We present a method for clinical trials units to track their metrics within clinical trials using descriptive statistics and visualisations.

Research Design

We suggest descriptive statistics and visualisations within a SWAT methodology.

Study Sample

We illustrate this method using the metrics from TEMPER, a monitoring study carried out in three trials at the MRC Clinical Trials Unit at UCL.

Data Collection

The data collection for TEMPER is described in DOI: 10.1177/1740774518793379.

Results

We show the results and discuss a protocol for a Study-Within-A-Trial (SWAT 167) for those wishing to use the method.

Conclusions

The potential benefits metric tracking brings to clinical trials include enhanced assessment of sites for potential corrective action, improved evaluation and contextualisation of the influence of metrics and their thresholds, and the establishment of best practice in RBM. The standardisation of the collection of such monitoring data would benefit both individual trials and the clinical trials community.

Monitoring and auditing protocol adherence, data integrity and ethical conduct of a randomized clinical trial: A case study

PURPOSE

To categorize, quantify and interpret findings documented in feedback letters of monitoring or auditing visits for an investigator-initiated, peer-review funded multicenter randomized trial testing probiotics for critically ill patients.

MATERIALS & METHODS

In 37 Canadian centers, monitoring and auditing visits were performed by 3 trained individuals; findings were reported in feedback letters. At trial termination, we performed duplicate content analysis on letters, categorizing observations first into unique findings, followed by 10 pre-determined trial quality management domains. We further classified each observation into a) missing operational records, b) errors in process, and potential threats to c) data integrity, d) patient privacy or e) safety.

RESULTS

Across 37 monitoring or auditing visits, 75 unique findings were categorized into 10 domains. Most frequently, observations were in domains of training documentation (180/566 [32%]) and the informed consent process (133/566 [23%]). Most observations were missing operational records (438/566 [77%]) rather than errors in process (128/566 [23%]). Of 75 findings, 13 (62/566 observations [11%]) posed a potential threat to data integrity, 1 (1/566 observation [0.18%]) to patient privacy, and 9 (49/566 observations [8.7%]) to patient safety.

CONCLUSIONS

Monitoring and auditing findings predominantly concerned missing documentation with minimal threats to data integrity, patient privacy or safety.

TRIAL REGISTRATION

PROSPECT (Probiotics: Prevention of Severe Pneumonia and Endotracheal Colonization Trial): NCT02462590.

Effect of intensive versus limited monitoring on clinical trial conduct and outcomes: A randomized trial

BACKGROUND

Regulatory agencies have endorsed more limited approaches to clinical trial site monitoring. However, the impact of different monitoring strategies on trial conduct and outcomes is unclear.

METHODS

We conducted a patient-level block-randomized controlled trial evaluating the effect of intensive versus limited monitoring on cardiovascular clinical trial conduct and outcomes nested within the CoreValve Continued Access and Expanded Use Studies. Intensive monitoring included complete source data verification of all critical datapoints whereas limited monitoring included automated data checks only. This study's endpoints included clinical trial outcome ascertainment as well as monitoring action items, protocol deviations, and adverse event ascertainment.

RESULTS

A total of 2,708 patients underwent transcatheter aortic valve replacement (TAVR) and were randomized to either intensive monitoring (n = 1,354) or limited monitoring (n = 1,354). Monitoring action items were more common with intensive monitoring (52% vs 15%; P < .001), but there was no difference in the percentage of patients with any protocol deviation (91.6% vs 90.4%; P = .314). The reported incidence of trial outcomes between intensive and limited monitoring was similar for mortality (30 days: 4.8% vs 5.5%, P = .442; 1 year: 20.3% vs 21.3%, P = .473) and stroke (30 days: 2.8% vs 2.4%, P = .458), as well as most secondary trial outcomes with the exception of bleeding (intensive: 36.3% vs limited: 32.0% at 30 days, P = .019). There was a higher reported incidence of cardiac adverse events reported in the intensive monitoring group at 1 year (76.7% vs 72.4%; P = .019).

CONCLUSIONS

Tailored limited monitoring strategies can be implemented without influencing the integrity of TAVR trial outcomes.

Monitoring strategies for clinical intervention studies

BACKGROUND

Trial monitoring is an important component of good clinical practice to ensure the safety and rights of study participants, confidentiality of personal information, and quality of data. However, the effectiveness of various existing monitoring approaches is unclear. Information to guide the choice of monitoring methods in clinical intervention studies may help trialists, support units, and monitors to effectively adjust their approaches to current knowledge and evidence.

OBJECTIVES

To evaluate the advantages and disadvantages of different monitoring strategies (including risk-based strategies and others) for clinical intervention studies examined in prospective comparative studies of monitoring interventions.

SEARCH METHODS

We systematically searched CENTRAL, PubMed, and Embase via Ovid for relevant published literature up to March 2021. We searched the online 'Studies within A Trial' (SWAT) repository, grey literature, and trial registries for ongoing or unpublished studies.

SELECTION CRITERIA

We included randomized or non-randomized prospective, empirical evaluation studies of different monitoring strategies in one or more clinical intervention studies. We applied no restrictions for language or date of publication.

DATA COLLECTION AND ANALYSIS

We extracted data on the evaluated monitoring methods, countries involved, study population, study setting, randomization method, and numbers and proportions in each intervention group. Our primary outcome was critical and major monitoring findings in prospective intervention studies. Monitoring findings were classified according to different error domains (e.g. major eligibility violations) and the primary outcome measure was a composite of these domains. Secondary outcomes were individual error domains, participant recruitment and follow-up, and resource use. If we identified more than one study for a comparison and outcome definitions were similar across identified studies, we quantitatively summarized effects in a meta-analysis using a random-effects model. Otherwise, we qualitatively summarized the results of eligible studies stratified by different comparisons of monitoring strategies. We used the GRADE approach to assess the certainty of the evidence for different groups of comparisons.

MAIN RESULTS

We identified eight eligible studies, which we grouped into five comparisons. 1. Risk-based versus extensive on-site monitoring: based on two large studies, we found moderate certainty of evidence for the combined primary outcome of major or critical findings that risk-based monitoring is not inferior to extensive on-site monitoring. Although the risk ratio was close to 'no difference' (1.03 with a 95% confidence interval [CI] of 0.81 to 1.33, below 1.0 in favor of the risk-based strategy), the high imprecision in one study and the small number of eligible studies resulted in a wide CI of the summary estimate. Low certainty of evidence suggested that monitoring strategies with extensive on-site monitoring were associated with considerably higher resource use and costs (up to a factor of 3.4). Data on recruitment or retention of trial participants were not available. 2. Central monitoring with triggered on-site visits versus regular on-site visits: combining the results of two eligible studies yielded low certainty of evidence with a risk ratio of 1.83 (95% CI 0.51 to 6.55) in favor of triggered monitoring intervention. Data on recruitment, retention, and resource use were not available. 3. Central statistical monitoring and local monitoring performed by site staff with annual on-site visits versus central statistical monitoring and local monitoring only: based on one study, there was moderate certainty of evidence that a small number of major and critical findings were missed with the central monitoring approach without on-site visits: 3.8% of participants in the group without on-site visits and 6.4% in the group with on-site visits had a major or critical monitoring finding (odds ratio 1.7, 95% CI 1.1 to 2.7; P = 0.03). The absolute number of monitoring findings was very low, probably because defined major and critical findings were very study specific and central monitoring was present in both intervention groups. Very low certainty of evidence did not suggest a relevant effect on participant retention, and very low certainty evidence indicated an extra cost for on-site visits of USD 2,035,392. There were no data on recruitment. 4. Traditional 100% source data verification (SDV) versus targeted or remote SDV: the two studies assessing targeted and remote SDV reported findings only related to source documents. Compared to the final database obtained using the full SDV monitoring process, only a small proportion of remaining errors on overall data were identified using the targeted SDV process in the MONITORING study (absolute difference 1.47%, 95% CI 1.41% to 1.53%). Targeted SDV was effective in the verification of source documents, but increased the workload on data management. The other included study was a pilot study, which compared traditional on-site SDV versus remote SDV and found little difference in monitoring findings and the ability to locate data values despite marked differences in remote access in two clinical trial networks. There were no data on recruitment or retention. 5. Systematic on-site initiation visit versus on-site initiation visit upon request: very low certainty of evidence suggested no difference in retention and recruitment between the two approaches. There were no data on critical and major findings or on resource use.

AUTHORS' CONCLUSIONS

The evidence base is limited in terms of quantity and quality. Ideally, for each of the five identified comparisons, more prospective, comparative monitoring studies nested in clinical trials and measuring effects on all outcomes specified in this review are necessary to draw more reliable conclusions. However, the results suggesting risk-based, targeted, and mainly central monitoring as an efficient strategy are promising. The development of reliable triggers for on-site visits is ongoing; different triggers might be used in different settings. More evidence on risk indicators that identify sites with problems or the prognostic value of triggers is needed to further optimize central monitoring strategies. In particular, approaches with an initial assessment of trial-specific risks that need to be closely monitored centrally during trial conduct with triggered on-site visits should be evaluated in future research.

Clinical trial recruiters' experiences working with trial eligibility criteria: results of an exploratory, cross-sectional, online survey in the UK

BACKGROUND

Eligibility criteria are a fundamental element of clinical trial design, defining who can and who should not participate in a trial. Problems with the design or application of criteria are known to occur and pose risks to participants' safety and trial integrity, sometimes also negatively impacting on trial recruitment and generalisability. We conducted a short, exploratory survey to gather evidence on UK recruiters' experiences interpreting and applying eligibility criteria and their views on how criteria are communicated and developed.

METHODS

Our survey included topics informed by a wider programme of work at the Clinical Trials Research Unit, University of Leeds, on assuring eligibility criteria quality. Respondents were asked to answer based on all their trial experience, not only on experiences with our trials. The survey was disseminated to recruiters collaborating on trials run at our trials unit, and via other mailing lists and social media. The quantitative responses were descriptively analysed, with inductive analysis of free-text responses to identify themes.

RESULTS

A total of 823 eligible respondents participated. In total, 79% of respondents reported finding problems with eligibility criteria in some trials, and 9% in most trials. The main themes in the types of problems experienced were criteria clarity (67% of comments), feasibility (34%), and suitability (14%). In total, 27% of those reporting some level of problem said these problems had led to patients being incorrectly included in trials; 40% said they had led to incorrect exclusions. Most respondents (56%) reported accessing eligibility criteria mainly in the trial protocol. Most respondents (74%) supported the idea of recruiter review of eligibility criteria earlier in the protocol development process.

CONCLUSIONS

Our survey corroborates other evidence about the existence of suboptimal trial eligibility criteria. Problems with clarity were the most often reported, but the number of comments on feasibility and suitability suggest some recruiters feel eligibility criteria and associated assessments can hinder recruitment to trials. Our proposal for more recruiter involvement in protocol development has strong support and some potential benefits, but questions remain about how best to implement this. We invite other trialists to consider our other suggestions for how to assure quality in trial eligibility criteria.

Central monitoring in a randomized, open-label, controlled phase 3 clinical trial for a treatment-shortening regimen for pulmonary tuberculosis

INTRODUCTION

With the growing use of online study management systems and rapid availability of data, timely data review and quality assessments are necessary to ensure proper clinical trial implementation. In this report we describe central monitoring used to ensure protocol compliance and accurate data reporting, implemented during a large phase 3 clinical trial.

MATERIAL AND METHODS

The Tuberculosis Trials Consortium (TBTC) Study 31/AIDS Clinical Trials Group (ACTG) study A5349 (S31) is an international, multi-site, randomized, open-label, controlled, non-inferiority phase 3 clinical trial comparing two 4-month regimens to a standard 6 month regimen for treatment of drug-susceptible tuberculosis (TB) among adolescents and adults with a sample size of 2500 participants.

RESULTS

Central monitoring utilized primary study data in a five-tiered approach, including (1) real-time data checks & topic-specific intervention reports, (2) missing forms reports, (3) quality assurance metrics, (4) critical data reports and (5) protocol deviation identification, aimed to detect and resolve quality challenges. Over the course of the study, 240 data checks and reports were programed across the five tiers used.

DISCUSSION

This use of primary study data to identify issues rapidly allowed the study sponsor to focus quality assurance and data cleaning activities on prioritized data, related to protocol compliance and accurate reporting of study results. Our approach enabled us to become more efficient and effective as we informed sites about deviations, resolved missing or inconsistent data, provided targeted guidance, and gained a deeper understanding of challenges experienced at clinical trial sites.

TRIAL REGISTRATION

This trial was registered with ClinicalTrials.gov (Identifier: NCT02410772) on April 8, 2015.

Assessing the potential for prevention or earlier detection of on-site monitoring findings from randomised controlled trials: Further analyses of findings from the prospective TEMPER triggered monitoring study

BACKGROUND/AIMS

Clinical trials should be designed and managed to minimise important errors with potential to compromise patient safety or data integrity, employ monitoring practices that detect and correct important errors quickly, and take robust action to prevent repetition. Regulators highlight the use of risk-based monitoring, making greater use of centralised monitoring and reducing reliance on centre visits. The TEMPER study was a prospective evaluation of triggered monitoring (a risk-based monitoring method), whereby centres are prioritised for visits based on central monitoring results. Conducted in three UK-based randomised cancer treatment trials of investigational medicine products with time-to-event outcomes, it found high levels of serious findings at triggered centre visits but also at visits to matched control centres that, based on central monitoring, were not of concern. Here, we report a detailed review of the serious findings from TEMPER centre visits. We sought to identify feasible, centralised processes which might detect or prevent these findings without a centre visit.

METHODS

The primary outcome of this study was the proportion of all 'major' and 'critical' TEMPER centre visit findings theoretically detectable or preventable through a feasible, centralised process. To devise processes, we considered a representative example of each finding type through an internal consensus exercise. This involved (a) agreeing the potential, by some described process, for each finding type to be centrally detected or prevented and (b) agreeing a proposed feasibility score for each proposed process. To further assess feasibility, we ran a consultation exercise, whereby the proposed processes were reviewed and rated for feasibility by invited external trialists.

RESULTS

In TEMPER, 312 major or critical findings were identified at 94 visits. These findings comprised 120 distinct issues, for which we proposed 56 different centralised processes. Following independent review of the feasibility of the proposed processes by 87 consultation respondents across eight different trial stakeholder groups, we conclude that 306/312 (98%) findings could theoretically be prevented or identified centrally. Of the processes deemed feasible, those relating to informed consent could have the most impact. Of processes not currently deemed feasible, those involving use of electronic health records are among those with the largest potential benefit.

CONCLUSIONS

This work presents a best-case scenario, where a large majority of monitoring findings were deemed theoretically preventable or detectable by central processes. Caveats include the cost of applying all necessary methods, and the resource implications of enhanced central monitoring for both centre and trials unit staff. Our results will inform future monitoring plans and emphasise the importance of continued critical review of monitoring processes and outcomes to ensure they remain appropriate.

Time-consuming and expensive data quality monitoring procedures persist in clinical trials: A national survey

INTRODUCTION

The Good Clinical Practice guideline identifies that data monitoring is an essential research activity. However, limited evidence exists on how to perform monitoring including the amount or frequency that is needed to ensure data quality. This study aims to explore the monitoring procedures that are implemented to ensure data quality in Australian clinical research studies.

MATERIAL AND METHODS

Clinical studies listed on the Australian and New Zealand Clinical Trials Registry were invited to participate in a national cross-sectional, mixed-mode, multi-contact (postal letter and e-mail) web-based survey. Information was gathered about the types of data quality monitoring procedures being implemented.

RESULTS

Of the 3689 clinical studies contacted, 589 (16.0%) responded, of which 441 (77.4%) completed the survey. Over half (55%) of the studies applied source data verification (SDV) compared to risk-based targeted and triggered monitoring (10-11%). Conducting 100% on-site monitoring was most common for those who implemented the traditional approach. Respondents who did not conduct 100% monitoring, included 1-25% of data points for SDV, centralized or on-site monitoring. The incidence of adverse events and protocol deviations were the most likely factors to trigger a site visit for risk-based triggered (63% and 44%) and centralized monitoring (48% and 44%), respectively.

CONCLUSION

Instead of using more optimal risk-based approaches, small single-site clinical studies are conducting traditional monitoring procedures which are time consuming and expensive. Formal guidelines need to be improved and provided to all researchers for 'new' risk-based monitoring approaches.

Statistical Considerations for Trials in Adjuvant Treatment of Colorectal Cancer

The design of the best possible clinical trials of adjuvant interventions in colorectal cancer will entail the use of both time-tested and novel methods that allow efficient, reliable and patient-relevant therapeutic development. The ultimate goal of this endeavor is to safely and expeditiously bring to clinical practice novel interventions that impact patient lives. In this paper, we discuss statistical aspects and provide suggestions to optimize trial design, data collection, study implementation, and the use of predictive biomarkers and endpoints in phase 3 trials of systemic adjuvant therapy. We also discuss the issues of collaboration and patient centricity, expecting that several novel agents with activity in the (neo)adjuvant therapy of colon and rectal cancers will become available in the near future.

Central statistical monitoring of investigator-led clinical trials in oncology

Investigator-led clinical trials are pragmatic trials that aim to investigate the benefits and harms of treatments in routine clinical practice. These much-needed trials represent the majority of all trials currently conducted. They are however threatened by the rising costs of clinical research, which are in part due to extensive trial monitoring processes that focus on unimportant details. Risk-based quality management focuses, instead, on “things that really matter”. We discuss the role of central statistical monitoring as part of risk-based quality management. We describe the principles of central statistical monitoring, provide examples of its use, and argue that it could help drive down the cost of randomized clinical trials, especially investigator-led trials, whilst improving their quality.

Monitoring in practice - How are UK academic clinical trials monitored? A survey

Background

Despite the US Food and Drug Administration (FDA) and European Medicines Agency (EMA) encouraging the use of risk-based monitoring for trials in 2013, there remains a lack of evidence-based guidelines on how to monitor. We surveyed the academic United Kingdom Clinical Research Collaboration (UKCRC) registered clinical trials units (CTUs) to find out their policy on monitoring of phase III randomised clinical trials of an investigational medicinal product (CTIMPs).

Methods

An online survey of monitoring policy with sections on the CTU, central monitoring and on-site monitoring was sent to all 50 UKCRC registered CTUs in November 2018.

Descriptive data analysis and tabulations are reported using the total number answering each question.

Results

A total of 43/50 (86%) of CTUs responded with 38 conducting phase III randomised CTIMP trials. Of these 38 CTUs, 34 finished the survey. Most CTUs (36/37, 97%) use a central monitoring process to guide, target or supplement site visits. More than half (19/36, 53%) of CTUs do not use an automated monitoring report when centrally monitoring trials and all units use trial team knowledge to make a final decision on whether an on-site visit is required.

A total of 31/34 (91%) CTUs used triggers to decide whether or not to conduct an on-site monitoring visit. On-site, a mixture of source data verification and checking of processes was carried out.

The CTUs overwhelmingly (27/34, 79%) selected optimising central monitoring as their most pressing concern.

Conclusion

The survey showed a wide variation in phase III randomised CTIMP trial monitoring practices by academic clinical trials units within a single research-active country. We urgently need to develop evidence-based regulator-agreed guidance for CTUs on best practice for both central and on-site monitoring and to develop tools for all CTUs to use.