A Data-Driven Approach to Risk-Based Source Data Verification

Impact of monitoring approaches on data quality in clinical trials

AIMS

Source data verification (SDV) has been reported to account for up to 25% of the budget in clinical trials (CT) and cost-benefit of SDV has been questioned. Guidelines for risk-based monitoring (RBM) were published in 2013 by agencies and in 2016, ICH-GCP-E6-(R2) added a requirement for risk-based approaches. This report will perform a comparison of the impact of RBM vs classic monitoring (CM) on data quality (defined as accuracy of data reporting from source data to final trial data) and expected impact on costs of CTs.

METHODS

Data on residual errors from four, large comparable randomised CTs were examined by post-trial SDV. Observed discrepancies were analysed in the categories of "overall" data, "major efficacy" and "major safety". In each category, the residual error rate was calculated as the number of discrepancies divided by the number of data-fields verified.

RESULTS

A total of 1 716 087 data points were verified using CM and 323 174 using RBM. The overall error rate was 0.40% for RBM and 0.37% for CM (P < 0.01). For major efficacy, defined by risk assessment, the error rate was 0.15% and 0.28% (P < 0.0001); in major safety, defined by risk assessment, the error rate was 0.49% and 0.67% (P = 0.15), both in favour of the RBM approach.

CONCLUSION

These empirical data, directly comparing RBM with CM, suggest that RBM improves data quality regarding data-points of major importance to trial outcomes, efficacy and major safety. Overall, the RBM approach showed a correlation to reduced amount of data collection errors with major relevance for interpretation of study results and subject safety as well as reducing on-site monitoring and data cleaning resources.

Clinical trial monitoring effectiveness: Remote risk-based monitoring versus on-site monitoring with 100% source data verification

BACKGROUND/AIMS

Traditional on-site monitoring of clinical trials via frequent site visits and 100% source data verification is cost-consuming, and it still cannot guarantee data quality effectively. Depending on the types and designs of clinical trials, an alternative would be combining several monitoring methods, such as risk-based monitoring and remote monitoring. However, there is insufficient evidence of its effectiveness. This research compared the effectiveness of risk-based monitoring with a remote monitoring system with that of traditional on-site monitoring.

METHODS

With a cloud-based remote monitoring system called beagle View^®, we created a remote risk-based monitoring methodology that focused only on critical data and processes. We selected a randomized controlled trial conducted at Tohoku University Hospital and randomly sampled 11 subjects whose case report forms had already been reviewed by data managers. Critical data and processes were verified retrospectively by remote risk-based monitoring; later, all data and processes were confirmed by on-site monitoring. We compared the ability of remote risk-based monitoring to detect critical data and process errors with that of on-site monitoring with 100% source data verification, including an examination of clinical trial staff workload and potential cost savings.

RESULTS

Of the total data points (n = 5617), 19.7% (n = 1105, 95% confidence interval = 18.7-20.7) were identified as critical. The error rates of critical data detected by on-site monitoring, remote risk-based monitoring, and data review by data managers were 7.6% (n = 84, 95% CI = 6.2-9.3), 7.6% (n = 84, 95% confidence interval = 6.2-9.3), and 3.9% (n = 43, 95% confidence interval = 2.9-5.2), respectively. The total number of critical process errors detected by on-site monitoring was 14. Of these 14, 92.9% (n = 13, 95% confidence interval = 68.5-98.7) and 42.9% (n = 6, 95% confidence interval = 21.4-67.4) of critical process errors were detected by remote risk-based monitoring and data review by data managers, respectively. The mean time clinical trial staff spent dealing with remote risk-based monitoring was 9.9 ± 5.3 (mean ± SD) min per visit per subject. Our calculations show that remote risk-based monitoring saved between 9 and 41 on-site monitoring visits, corresponding to a cost of between US$13,500 and US$61,500 per trial site.

CONCLUSION

Remote risk-based monitoring was able to detect critical data and process errors as well as on-site monitoring with 100% source data verification, saving travel time and monitoring costs. Remote risk-based monitoring offers an effective alternative to traditional on-site monitoring of clinical trials.

Investigation of the Factors Affecting Risk-Based Quality Management of Investigator-Initiated Investigational New-Drug Trials for Unapproved Anticancer Drugs in Japan

BACKGROUND

With an increase in the complexity and cost of clinical trials and the advances in information technology, monitoring guidance issued by regulatory authorities recommends risk-adapted monitoring. To introduce the monitoring method for investigator-initiated investigational new drug (IND) trials using unapproved anticancer drugs, we performed exploratory retrospective analyses to identify risk factors for data quality.

METHODS

To select investigator-initiated IND trials using unapproved anticancer drugs, we set the trial selection criteria. Data collection was performed by using audit trails and monitoring reports. Collected data were analyzed by univariate and multivariate analyses to identify the independent risk factors related to error.

RESULTS

By trial selection criteria, 5 investigator-initiated IND trials using unapproved anticancer drugs were selected. The error rates of the total data, critical data, and noncritical data were 7.4%, 9.7%, and 5.9%, respectively. There was no difference between clinical research core hospitals certified by the Ministry of Health, Labour and Welfare and other hospitals in univariate analysis (odds ratio [OR], 1.00; 99% confidence interval [CI], 0.96-1.05; P = .9179). As the main independent risk factors related to error, critical data in the importance of data (OR, 1.28; 99% CI, 1.24-1.33; P < .0001) and groups with ≤3 patients after registration (OR, 1.12; 99% CI, 1.10-1.15; P < .0001) were significantly related to errors in multivariate analysis.

CONCLUSIONS

The results of this research suggest that the feasibility of risk-based monitoring and sampling source data verification was indicated for noncritical data and patients after the third case.

The impact of clinical trial monitoring approaches on data integrity and cost--a review of current literature

PURPOSE

Monitoring is a costly requirement when conducting clinical trials. New regulatory guidance encourages the industry to consider alternative monitoring methods to the traditional 100 % source data verification (SDV) approach. The purpose of this literature review is to provide an overview of publications on different monitoring methods and their impact on subject safety data, data integrity, and monitoring cost.

METHODS

The literature search was performed by keyword searches in MEDLINE and hand search of key journals. All publications were reviewed for details on how a monitoring approach impacted subject safety data, data integrity, or monitoring costs.

RESULTS

Twenty-two publications were identified. Three publications showed that SDV has some value for detection of not initially reported adverse events and centralized statistical monitoring (CSM) captures atypical trends. Fourteen publications showed little objective evidence of improved data integrity with traditional monitoring such as 100 % SDV and sponsor queries as compared to reduced SDV, CSM, and remote monitoring. Eight publications proposed a potential for significant cost reductions of monitoring by reducing SDV without compromising the validity of the trial results.

CONCLUSIONS

One hundred percent SDV is not a rational method of ensuring data integrity and subject safety based on the high cost, and this literature review indicates that reduced SDV is a viable monitoring method. Alternative methods of monitoring such as centralized monitoring utilizing statistical tests are promising alternatives but have limitations as stand-alone tools. Reduced SDV combined with a centralized, risk-based approach may be the ideal solution to reduce monitoring costs while improving essential data quality.

Efficient Source Data Verification Using Statistical Acceptance Sampling: A Simulation Study

BACKGROUND

One approach to increase the efficiency of clinical trial monitoring is to replace 100% source data verification (SDV) by verification of samples of source data. An intuitive strategy for determining appropriate sampling plans (ie, sample sizes and the maximum tolerable number of transcription errors in the samples) is to use acceptance sampling methodology. Expanding upon earlier work in which the use of acceptance sampling strategies for sampling-based SDV was proposed, we describe an alternative acceptance sampling strategy that, instead of relying on sampling standards, evaluates all possible sampling plans algorithmically, thereby ensuring that selected sampling plans conform to prespecified criteria.

METHODS

Empirical trial data guided the design of the proposed strategy. In addition, extensive simulations, also based on the empirical data, were performed to assess the performance in terms of workload reductions and the post-SDV error proportion of applying the proposed strategy.

RESULTS

13 different scenarios were simulated, but results of the default scenario show that the average pre-SDV error proportion per trial of .056 was reduced to .023 by inspecting only 40.5% of the case report form entries. Of the inspected data entries, almost half (18.0/40.5) was, on average, SDV-ed as part of the sampling process; remaining entries were inspected during full inspections after too many errors were observed in the samples.

CONCLUSION

Our results suggest that major reductions in workload can be achieved, while maintaining acceptable data quality levels. However, the results also indicate that the proposed strategy is conservative and further improvement is possible.

Risk-Based Monitoring: A Closer Statistical Look at Source Document Verification, Queries, Study Size Effects, and Data Quality

BACKGROUND

Data quality within the clinical research enterprise can be defined as the absence of errors that matter and whether the data are fit for purpose. This concept, proposed by the Clinical Trials Transformation Initiative, resulted from a culmination of collaboration with industry, academia, patient advocates, and regulators, and it emphasizes the presence of a hierarchy of error types, resulting in a more efficient and modern data-cleaning paradigm. While source document verification (SDV) is commonly used as a quality control method in clinical research, it is disproportionately expensive and often leads to questionable benefits. Although the current literature suggests that there is a need to reduce the burden of SDV, there is no consensus on how to replace this "tried and true" practice.

METHODS

This article proposes a practical risk-based monitoring approach based on published statistical evidence addressing the impact of database changes subsequent to SDV.

RESULTS

The analysis clearly demonstrates minimal effects of errors and error corrections on study results and study conclusions, with diminishing effect as the study size increases, and it suggests that, on average, <8% SDV is adequate to ensure data quality, with perhaps higher SDV rates for smaller studies and virtually 0% SDV for large studies.

CONCLUSIONS

It is recommended that SDV, rather than just focusing on key primary efficacy and safety outcomes, focus on data clarification queries as highly discrepant (and the riskiest) data.

Evaluating Source Data Verification as a Quality Control Measure in Clinical Trials

TransCelerate has developed a risk-based monitoring methodology that transforms clinical trial monitoring from a model rooted in source data verification (SDV) to a comprehensive approach leveraging cross-functional risk assessment, technology, and adaptive on-site, off-site, and central monitoring activities to ensure data quality and subject safety. Evidence suggests that monitoring methods that concentrate on what is critical for a study and a site may produce better outcomes than do conventional SDV-driven models. This article assesses the value of SDV in clinical trial monitoring via a literature review, a retrospective analysis of data from clinical trials, and an assessment of major and critical findings from TransCelerate member company internal audits. The results support the hypothesis that generalized SDV has limited value as a quality control measure and reinforce the value of other risk-based monitoring activities.