Clinical trial monitoring: hit or miss?

Impact of on-site initiation visits on patient recruitment and data quality in a randomized trial of adjuvant chemotherapy for breast cancer

Purpose To provide empirical evidence on the impact of on-site initiation visits on the following outcomes: patient recruitment, quantity and quality of data submitted to the trial coordinating office, and patients' follow-up time.

Patients and methods This methodological study was performed as part of a randomized trial comparing two combination chemotherapies for adjuvant treatment of breast cancer. Centers participating to the trial were randomized to either receive systematic on-site visits (Visited group), or not (Non-visited group).

Results The study was terminated after two years, while the main randomized trial continued. Of the 135 centers that had expressed an interest in the trial, only 69 randomized at least one patient (35/68 in the Visited group, 34/67 in the Nonvisited group). Almost two-thirds of the patients were entered by 17 centers (10 in the Visited group, seven in the Non-visited group) that accrued more than 10 patients each. None of the prespecified outcomes favored the group of centers submitted to on-site initiation visits (ie, mean number of queries par patient: 6.1 ± 9.7 versus 5.4 ± 6.4, respectively for the Visited and Non-visited groups). Spontaneous transmittal of case report forms, although required by protocol, was low in both randomized groups (mean number of pages per patient: 1.5 ± 2.0 versus 2.1 ± 2.3, respectively), with investigators submitting about one-third of the expected forms on time (29% and 39%, respectively).

Limitations This study could not evaluate the impact of repeated on-site visits on clinical outcomes.

Conclusion Systematic on-site initiation visits did not contribute significantly to this clinical trial.

A systematic review of on-site monitoring methods for health-care randomised controlled trials

BACKGROUND

Monitoring the conduct of clinical trials is recommended by International Conference of Harmonisation Good Clinical Practice (ICH GCP) guidelines and is integral to trial quality assurance. On-site monitoring, that is, visiting trial sites, is one part of this process but little is known about the procedures that are performed in practice.

PURPOSE

To examine and summarise published on-site monitoring methods for health-care clinical trials, including evaluations of their benefits and costs to trials.

METHODS

A systematic literature review identified all articles reporting the methods and practices of on-site monitoring of randomised controlled trials (RCTs). Articles were categorised into (1) reports from research groups and organisations, (2) reports from individual RCTs, (3) randomised trials of on-site monitoring interventions, (4) cost simulations, or (5) surveys of trial staff and monitors. Data were extracted on the characteristics of the trials and groups reporting on-site monitoring (e.g., geographical origin, sponsor, and trial focus). Information from articles in categories (1)-(3) was summarised on the frequency and scope of site monitoring visits, monitoring team size and composition, activities during site visits, and reporting structures. Evaluations of the benefits and disadvantages of on-site monitoring were examined for all included articles.

RESULTS

In total, 57 articles were identified, comprising 21 articles about the on-site monitoring practices of 16 research groups, 30 articles from 26 RCTs, 1 on-site monitoring intervention RCT, 2 cost simulations, and 3 surveys. Publications in categories (1)-(3), mostly originated from the United States (33/52, 63%) or Europe (15/52, 29%), were predominantly describing non-commercial organisations or trials (45/52, 87%), with heart disease (9/26, 35%) or cancer (5/26, 19%) the commonest focus of individual RCTs. The frequency of visits ranged from every 6-8 weeks up to once every 3 years, with mostly all trial sites visited. The number of monitors visiting a site varied between 1 and 8. The most common on-site monitoring activity was verifying source data and consent forms, with a focus on data accuracy. Only six articles evaluated their on-site monitoring process, with improvements observed in recruitment rates and protocol adherence but with direct costs and staff time viewed as the major disadvantages. The on-site monitoring RCT ended prematurely so preventing full assessment.

LIMITATIONS

Trialists and organisations may utilise additional unpublished on-site monitoring systems. The varied terminology used to describe monitoring may have limited identification of some relevant articles.

CONCLUSIONS

This review demonstrated that on-site monitoring is utilised in trials worldwide but systems vary considerably with little evidence to support practice. These on-site monitoring practices need to be evaluated empirically, including costs, to provide robust evidence for the contribution of site visits to trial performance and quality.

Dermatologic surgical research and the Institutional Review Board

BACKGROUND

Dermatologic surgical techniques have transformed cosmetic surgery over the last two decades. Many of the advances in cutaneous laser surgery, botulinum toxin injections, tumescent liposuction, and the latest filler agents have been evaluated on willing volunteers as part of organized trials under the auspices of an Institutional Review Board (IRB).

OBJECTIVES

To review the developmental history of today's IRB and to provide relevance to cosmetic dermatologic research.

RESULTS

Although an IRB must evaluate the "voluntariness" of a proposed research protocol, the risk-benefit ratio, particularly of elective cosmetic dermatologic surgical human studies, is of even greater importance.

CONCLUSION

As more dermatologic surgeons become involved in clinical research, an understanding of the function and purpose of the IRB has become increasingly important. Research relating to cosmetic dermatologic procedures raises challenging and perplexing problems for an IRB.

The classic caries clinical trial: constraints and opportunities

The history of clinical trials would include events in 1747 on board the Salisbury, a British Navy vessel at sea with 12 seamen critically ill with scurvy. Involving these 12 sailors in a study, an officer on board by the name of Lind evaluated six potential treatments for scurvy, and rapidly reached the conclusion that daily consumption of citrus fruits returned the men fit for duty in approximately six days (Bull, 1959). The concept of experimental randomization was first developed by Sir R.A. Fisher (1925, 1926), and the method was introduced to medical research via a study of tuberculosis treatment by Amberson and co-workers (1931), who randomized 24 TB patients into two groups, one to receive the experimental therapy, the other serving as the control. Amberson et al. also incorporated the concept of blinding into their study. Sir Austin Bradford Hill codified and built on the principles of scientific experimentation developed by Fisher, and introduced the use of random numbers in the allocation of patients in the British Medical Research Council (1948) study of the effect of streptomycin in the treatment of tuberculosis (Daniels and Hill, 1952; Hill, 1952). The first applications of clinical trial methodology for testing interventions on dental, oral, and maxillofacial diseases and conditions are more difficult to determine. For dental caries prevention, however, Chilton and Fertig (1958) and Slack and Martin (1964) were certainly among the early caries clinical trial pioneers. As clinical trials have come into the mainstream of clinical research in medicine and dentistry, a great deal of developmental work has focused on their methodological enhancement. The most successful of these efforts have come from fruitful, ongoing collaborations among clinician investigators, biostatisticians, data management specialists, biomedical ethicists, and others with an academic interest in clinical trial design and utilization. During the past 25 years, the emergence of systematic reviews and the evidence-based medicine (EBM) movement have also contributed significantly to the increasing reliance on randomized clinical trial outcomes for the advancement of better clinical practice (Richards et al., 1997; Straus and Sackett, 1998; www.cochrane.org/cochrane/ccbroch.htm#BDL, 2002).

The cystic fibrosis therapeutics development network (CF TDN): a paradigm of a clinical trials network for genetic and orphan diseases

Clinical trials have become critical to the advancement of medical science and to the evolution of patient care in medicine. The science of clinical research has advanced from early studies in which treatment was assessed without controls to sophisticated multinational collaborative randomized, double-blind, placebo controlled trials of therapeutic interventions. To facilitate the advancement of clinical research, clinical trials networks have been developed to conduct multicenter studies. This review describes the history of clinical trials, clinical trials networks, and the goals of such networks in the United States. The Cystic Fibrosis Therapeutics Development Network, a network that represents the paradigm for genetic and orphan diseases, is described in detail. This network has been extremely successful in its first 3.5 years of existence conducting 18 different clinical trials in patients with Cystic Fibrosis. Unique aspects of the network include the use of internet applications for study conduct and communication, the development of statistical methodology to enhance the efficiency of clinical trial design, the development of outcome measures specific to Cystic Fibrosis, and the development of infrastructure necessary for expediting protocol development. In the current environment, clinical research faces significant challenges related to ensuring the safe and ethical conduct of clinical research while promoting fast and efficient clinical trials. To succeed and move forward to provide treatments and find cures for diseases, clinical trials networks must continue to evolve. The Cystic Fibrosis Therapeutics Development Network represents a network that has met this challenge and will continue to provide a venue for the safe and efficient conduct of clinical trials in Cystic Fibrosis.

Measurement of endometrial thickness at US in multicenter drug trials: value of central quality assurance reading

PURPOSE

To assess the value of central quality assurance (QA) reading of transvaginal ultrasonographic (US) images obtained to measure endometrial thickness and to assess image quality.

MATERIALS AND METHODS

Results of 2,000 US examinations performed in 1,000 subjects during one of two multicenter drug trials were evaluated. Endometrial thickness was measured at the study site; images were then sent to the QA center, where an experienced sonologist evaluated endometrial thickness and image quality.

RESULTS

In 360 (18%) of the 2,000 examinations, image quality was insufficient for central QA reading. Repeat examinations were requested, and suggestions for improvement in technique were provided. In 349 (97%) of the 360 examinations, repeat US images were of acceptable quality. In 99 (5%) of the 1,989 examinations in which endometrial thickness was measured, central measurement of thickness differed by more than 2 mm from that of the site. In a group (n = 300) that was followed up for 1 year, requests for repeat US examinations decreased from 24% at baseline to 11% at 1 year.

CONCLUSION

Central QA reading provides a consistent evaluation of endometrial thickness on US images obtained in multicenter drug trials and helps to ensure the acquisition of high-quality transvaginal US images. It further leads to demonstrable improvement in site performance.

The role of biostatistics in the prevention, detection and treatment of fraud in clinical trials

Recent cases of fraud in clinical trials have attracted considerable media attention, but relatively little reaction from the biostatistical community. In this paper we argue that biostatisticians should be involved in preventing fraud (as well as unintentional errors), detecting it, and quantifying its impact on the outcome of clinical trials. We use the term 'fraud' specifically to refer to data fabrication (making up data values) and falsification (changing data values). Reported cases of such fraud involve cheating on inclusion criteria so that ineligible patients can enter the trial, and fabricating data so that no requested data are missing. Such types of fraud are partially preventable through a simplification of the eligibility criteria and through a reduction in the amount of data requested. These two measures are feasible and desirable in a surprisingly large number of clinical trials, and neither of them in any way jeopardizes the validity of the trial results. With regards to detection of fraud, a brute force approach has traditionally been used, whereby the participating centres undergo extensive monitoring involving up to 100 per cent verification of their case records. The cost-effectiveness of this approach seems highly debatable, since one could implement quality control through random sampling schemes, as is done in fields other than clinical medicine. Moreover, there are statistical techniques available (but insufficiently used) to detect 'strange' patterns in the data including, but no limited to, techniques for studying outliers, inliers, overdispersion, underdispersion and correlations or lack thereof. These techniques all rest upon the premise that it is quite difficult to invent plausible data, particularly highly dimensional multivariate data. The multicentric nature of clinical trials also offers an opportunity to check the plausibility of the data submitted by one centre by comparing them with the data from all other centres. Finally, with fraud detected, it is essential to quantify its likely impact upon the outcome of the clinical trial. Many instances of fraud in clinical trials, although morally reprehensible, have a negligible impact on the trial's scientific conclusions.

Artificial neural networks: a study in clinical psychopharmacology

Controlled trials in clinical psychopharmacology may fail to provide reliable information about the benefit of treatment when the patient is viewed in a real-life setting rather than as part of a well-defined sampling procedure. A viewpoint, rooted in systems theory, is proposed based on the identification of complex relationships among such dimensions as clinician's reasoning, drug properties, and patient's condition. Artificial Neural Network (ANN) technology provides efficient tools for data analysis within a systems-oriented approach. This study proposes a way to predict the outcome of psychopharmacological treatment. Analysis was conducted on retrospective data from clinical records of psychiatric patients treated with moclobemide. Twelve pharmacological, diagnostic, and topological variables were identified as the decisional items considered by six clinicians: age at onset, sex, previous treatment, duration and dose of moclobemide treatment, other drugs, psychiatric diagnosis and other clinical features. Data were binarily coded and transformed into observed frequencies in the sampling space; treatment outcome was binarily scored as the model's target. A Back-Propagation ANN based on the Delta rule with logistic transfer function was used. ANN correctly classified all cases of successful treatment (n = 51, 100%) but only half of the unsuccessful cases (n = 14, 52%). Patterns of response and areas of uncertainty were analyzed in a topological approach.

Guidelines for quality assurance in multicenter trials: a position paper

In the wake of reports of falsified data in one of the trials of the National Surgical Adjuvant Project for Breast and Bowel Cancer supported by the National Cancer Institute, clinical trials came under close scrutiny by the public, the press, and Congress. Questions were asked about the quality and integrity of the collected data and the analyses and conclusions of trials. In 1995, the leaders of the Society for Clinical Trials (the Chair of the Policy Committee, Dr. David DeMets, and the President of the Society, Dr. Sylvan Green) asked two members of the Society (Dr. Genell Knatterud and Dr. Frank Rockhold) to act as co-chairs of a newly formed subcommittee to discuss the issues of data integrity and auditing. In consultation with Drs. DeMets and Green, the co-chairs selected other members (Ms. Franca Barton, Dr. C.E. Davis, Dr. Bill Fairweather, Dr. Stephen George, Mr. Tom Honohan, Dr. Richard Mowery, and Dr. Robert O'Neill) to serve on the subcommittee. The subcommittee considered "how clean clinical trial data should be, to what extent auditing procedures are required, and who should conduct audits and how often." During the initial discussions, the subcommittee concluded that data auditing was insufficient to achieve data integrity. Accordingly, the subcommittee prepared this set of guidelines for standards of quality assurance for multicenter clinical trials. We include recommendations for appropriate action if problems are detected.

Selecting and validating biologic markers for drug development

Biochemical and clinical markers are critical for efficient development of new molecular entities. Biologic markers of drug effect, sometimes referred to as "surrogate" markers, are used when such clinical outcome measures as survival are substantially delayed relative to predictive biochemical changes or clinical effects of the new molecular entity. Biologic markers have generally been used for early-phase decision-making studies and accelerated regulatory approvals for much-needed drugs to treat cancer and acquired immune deficiency syndrome (AIDS). The rationale for these two uses of biologic markers is different and therefore the foundation required for establishing and validating each may be different. The theoretical foundation required for a marker that will be used to justify the regulatory approval of a treatment for a life-threatening disease should be greater than the required for an early decision-making study with an angiotensin II antagonist that will be used to treat mild to moderate hypertension. Use of CD4 counts as "surrogate markers" for prolonged survival was inappropriate. In contrast, changes in angiotensin-II concentrations and other renin-angiotensin system biochemical markers, observed for the first time in a study in humans, with a purported angiotensin-II receptor antagonist indicate that the new molecular entity is working as hoped. This is a good decision-making tool, because theory indicates that these changes should lead to reduced blood pressure, which is a predictive "surrogate" for reduction in subsequent cardiovascular events. Surrogate, biologic markers should be used only if they have a rational theoretical basis, are proven in preclinical or clinical experience, and are measured with validated methods. Different validation-acceptance criteria for decision-making markers compared with markers used for regulatory approval must be prospectively acknowledged and delineated.