Performance evaluation in multicenter clinical trials: development of a model by the AIDS Clinical Trials Group

Monitoring performance of sites within multicentre randomised trials: a systematic review of performance metrics

Background

Large multicentre trials are complex and expensive projects. A key factor for their successful planning and delivery is how well sites meet their targets in recruiting and retaining participants, and in collecting high-quality, complete data in a timely manner. Collecting and monitoring easily accessible data relevant to performance of sites has the potential to improve trial management efficiency. The aim of this systematic review was to identify metrics that have either been proposed or used for monitoring site performance in multicentre trials.

Methods

We searched the Cochrane Library, five biomedical bibliographic databases (CINAHL, EMBASE, Medline, PsychINFO and SCOPUS) and Google Scholar for studies describing ways of monitoring or measuring individual site performance in multicentre randomised trials. Records identified were screened for eligibility. For included studies, data on study content were extracted independently by two reviewers, and disagreements resolved by discussion.

Results

After removing duplicate citations, we identified 3188 records. Of these, 21 were eligible for inclusion and yielded 117 performance metrics. The median number of metrics reported per paper was 8, range 1–16. Metrics broadly fell into six categories: site potential; recruitment; retention; data collection; trial conduct and trial safety.

Conclusions

This review identifies a list of metrics to monitor site performance within multicentre randomised trials. Those that would be easy to collect, and for which monitoring might trigger actions to mitigate problems at site level, merit further evaluation.

Adherence, Quality of Life, and General Satisfaction with Co-formulated Zidovudine, Lamivudine, and Abacavir on Antiretroviral-Experienced Patients

PURPOSE

The main objective of the study was to evaluate patients' adherence to a fixed dose combination containing abacavir, zidovudine, and lamivudine (TZV) and to evaluate whether TZV might improve general satisfaction and quality of life (QOL) with anti-HIV treatment.

METHOD

Patients with viral load (VL) <400 copies switched their antiretroviral therapy to TZV. Adherence was assessed by patient self-report and medication accountability at week 24. General satisfaction was evaluated with 10-point Visual Analog Scale (VAS) at baseline and week 24. QOL was assessed with MOS-HIV questionnaire at baseline and weeks 12 and 24. Biochemical and hematological parameters were evaluated for safety reasons.

RESULTS

A total of 224 patients were included in the study. Overall, 81% (intent to treat [ITT]) and 98% (per protocol [PP]) of patients reported an adherence >95%, reaching 75% and 100% among the intravenous drug user population by ITT and PP analysis, respectively. General satisfaction was significantly higher at week 24 (90 +/- 14 vs. 53 +/- 26) in both ITT and PP populations (p<.0001). There were no statistically significant differences in QOL. Total cholesterol and triglycerides levels decreased significantly at week 24 compared to baseline (p<.05).

CONCLUSION

Patients who switched to TZV showed an increase in general satisfaction and high levels of adherence.

Clinical trial performance of community- vs university-based practices in the submacular surgery trials (SST): SST report no. 2

OBJECTIVE

To compare the performance of community- vs university-based clinical centers in 3 multicenter randomized clinical trials of intraocular surgery.

METHODS

Each Submacular Surgery Trials clinical center was classified as a university-based center, if the contract to perform as a center was signed by a university official, or as a community-based center. The 2 groups of centers were compared on performance, assessed cumulatively by the Submacular Surgery Trials Quality Assurance and Monitoring Subcommittee.

OUTCOME MEASURES

Patient accrual, completion of scheduled examinations, completion of masked vision examinations 2 years after enrollment (the designated primary study end point evaluation time), timeliness of submission of retinal photographs required by protocol to the Photograph Reading Center, completion of health- and vision-related quality-of-life interviews, and timeliness of submission of the primary outcome data to the Coordinating Center after completion of the examination.

RESULTS

Almost all centers performed at a very high (good) level, although there was a trend for some community-based centers to be at the lower end of most distributions.

CONCLUSIONS

Most community- and university-based centers performed well in these multicenter clinical trials. Monitoring performance and periodically providing feedback to clinical center investigators may encourage excellent performance in areas critical to the success of clinical trials, regardless of whether the center is community or university based.

Late-starter sites in randomized controlled trials

In a cohort of 14 randomized controlled trials conducted by the Adult AIDS Clinical Trials Group between 1986 and 1999 with a target sample size of >400 (total enrollment 15,531 patients), we evaluated whether "late-starter" sites can make a meaningful contribution to eventual trial accrual. The sites that started recruiting within 5 months from the time the first patient entered the trial were eventually responsible for over 90% of the total enrollment in 11 of the 14 trials. Across the 14 trials, some sites were consistently among the first to start enrollment, whereas others were routinely among the last. The late-starter sites are unlikely to make important contributions to eventual trial enrollment in large clinical trials conducted by groups with a fixed number of sites. Protracting administrative efforts to add more sites many months after a multicenter trial has started may not be useful to trial accrual.

Recruiting patients for female sexual arousal disorder studies

This study described the efficacy of various strategies for the recruitment of women with female sexual arousal disorder (FSAD) and matched controls in an obstetrics and gynecology clinic. During both phases of the study, actual recruitment time exceeded expectations. Clinical screening was the most effective recruitment strategy, accounting for one half of the enrolled subjects. Mass media advertising proved effective in recruiting up to one third of subjects. Control subjects were easily enrolled using all recruitment methods. Future reports should compare the efficiency and cost-benefit strategies for the recruitment of women with FSAD.

Determinants of patient recruitment in a multicenter clinical trials group: trends, seasonality and the effect of large studies

BACKGROUND

We examined whether quarterly patient enrollment in a large multicenter clinical trials group could be modeled in terms of predictors including time parameters (such as long-term trends and seasonality), the effect of large trials and the number of new studies launched each quarter. We used the database of all clinical studies launched by the AIDS Clinical Trials Group (ACTG) between October 1986 and November 1999. Analyses were performed in two datasets: one included all studies and substudies (n = 475, total enrollment 69,992 patients) and the other included only main studies (n = 352, total enrollment 57,563 patients).

RESULTS

Enrollment differed across different months of the year with peaks in spring and late fall. Enrollment accelerated over time (+27 patients per quarter for all studies and +16 patients per quarter for the main studies, p < 0.001) and was affected by the performance of large studies with target sample size > 1,000 (p < 0.001). These relationships remained significant in multivariate autoregressive modeling. A time series based on enrollment during the first 32 quarters could forecast adequately the remaining 21 quarters.

CONCLUSIONS

The fate and popularity of large trials may determine the overall recruitment of multicenter groups. Modeling of enrollment rates may be used to comprehend long-term patterns and to perform future strategic planning.

Predictors and impact of patients lost to follow-up in a long-term randomized trial of immediate versus deferred antiretroviral treatment

We studied predictors for losses to follow-up and the impact of such losses in the AIDS Clinical Trials Group 019 protocol, a long-term randomized trial of immediate versus deferred antiretroviral therapy in asymptomatic HIV-1-infected patients with >500 CD4 cells/mm3. The trial was selected because of its key importance in determining guidelines for antiretroviral therapy, and because it had the longest follow-up among all antiretroviral trials and the largest percentage of patients whose vital status was unknown at study end. Younger age, a history of parenteral drug use, and nonwhite race were associated with higher rates of loss to follow-up, but race was not an important predictor after adjusting for clinical site. There was large and statistically significant variability in the rates of losses among different clinical sites (p < 0.001). Patient retention was significantly better in clinical sites that enrolled many participants, with 25% of enrollees lost to follow-up in sites enrolling >100 patients and 44% in sites enrolling <33 patients each. As a group, patients lost to follow-up after the 2nd year had steeper declines of CD4 cell counts, and a significantly larger proportion had reached a CD4 cell count <300/mm3 in the year before being lost, compared with patients remaining in the study. Losses to follow-up probably decreased substantially the observed number of primary endpoints, curtailed the power of the trial to demonstrate any difference between immediate and deferred initiation of antiretroviral therapy, and may have introduced large bias in the estimated hazard ratio for the primary endpoint and its statistical significance.

Uses of clinical databases

Clinical databases consist of observational data collected on patients who meet specific criteria. The uses of these databases depend on whether the observations are drawn from a single institution, multiple clinical centers, or are population-based. Single institution databases frequently are used to profile patient accrual. In cases of rare diseases or unusual procedures, multicenter databases are required to amass sufficient information for study. Multicenter databases can be used to address issues related to intercenter variation and to develop statistical models to predict outcome based on prognostic factors. Population-based databases are required to assess incidence, prevalence, and mortality rates of disease. Although the inferences that can be drawn from observational data are limited by selection bias, clinical databases are valuable tools in planning clinical research. Clearly, however, resources are required to develop and maintain clinical databases. In an era when much research funding is directed at hypothesis-driven research, the importance of these clinical databases in developing clinical research hypotheses should not be overlooked.