Challenges to accrual predictions to phase III cancer clinical trials: a survey of study chairs and lead statisticians of 248 NCI-sponsored trials

Enrollment Success, Factors, and Prediction Models in Cancer Trials (2008-2019)

PURPOSE

To investigate the enrollment success rate of cancer clinical trials conducted in 2008-2019 and various factors lowering the enrollment success rate.

METHODS

This is a cross-sectional study with clinical trial information from the largest registration database ClinicalTrials.gov. Enrollment success rate was defined as actual enrollment greater or equal to 85% of the estimated enrollment goal. The association between trial characteristics and enrollment success was evaluated using the multivariable logistic regression.

RESULTS

A total of 4,004 trials in breast, lung, and colorectal cancers were included. The overall enrollment success rate was 49.1%. Compared with 2008-2010 (51.5%) and 2011-2013 (52.1%), the enrollment success rate is lower in 2014-2016 (46.5%) and 2017-2019 (36.4%). Regression analyses found trial activation year, phase I, phase I/phase II, and phase II (v phase III), sponsor agency of government (v industry), not requiring healthy volunteers, and estimated enrollment of 50-100, 100-200, 200, and >500 (v 0-50) were associated with a lower enrollment success rate (P < .05). However, trials with placebo comparator, ≥5 locations (v 1 location), and a higher number of secondary end points (eg, ≥5 v 0) were associated with a higher enrollment success rate (P < .05). The AUC for prediction of the final logistic regression models for all trials and specific trial groups ranged from 0.69 to 0.76.

CONCLUSION

This large-scale study supports a lower enrollment success rate over years in cancer clinical trials. Identified factors for enrollment success can be used to develop and improve recruitment strategies for future cancer trials.

Age Differences in Clinical Trial Understanding in Non-Hodgkin Lymphoma Patients

BACKGROUND

Clinical trials are often an important component of cancer care but are misunderstood by many patients. Few studies have examined age differences in clinical trial understanding in older versus younger adults, especially among patients with indolent non-Hodgkin lymphoma (NHL), a slowly progressive and not typically curable cancer diagnosed primarily in older adults.

PATIENTS AND METHODS

Participants aged ≥21 years with a diagnosis of NHL were recruited from a single academic medical center in an urban setting. Age was dichotomized as <65 and ≥65 years. Clinical trial understanding was assessed using a four-item survey of potential goals of a clinical trial, with responses including "yes," "no," and "I don't know." Survey responses were examined by age using Chi-square tests.

RESULTS

The sample was comprised of 74 patients who were predominantly non-Latino White, with a mean age of 60.4 years (SD = 12.27). Compared to younger patients, older patients were more likely to respond "I don't know" to the clinical trial goals of reducing the lymphoma (41.4% vs. 13.3%; P = .023) and keeping the lymphoma from worsening (41.4% vs. 13.3%; P = .017). Age differences for the remaining goals were not statistically significant. Similar findings emerged when the sample was restricted to patients under active surveillance.

CONCLUSION

Relative to younger adults, older adults may have a less nuanced understanding of clinical trial goals. Therefore, older adults may benefit from developmentally-tailored interventions to improve clinical trial understanding. Future research should examine the relationship between clinical trial understanding and enrollment by age using validated measures in diverse samples.

Center-Specific Modeling Predicts Cancer Trial Accrual More Accurately Than Investigators and Random Effects Modeling at 16 Cancer Centers

PURPOSE

Clinical trials often exceed their anticipated enrollment periods, and study sites often do not meet accrual goals. We previously reported the development and validation of a single-site accrual prediction model. Here, we describe the expansion of this methodology at 16 cancer centers (CCs) and compare an overall model versus site-specific models.

METHODS

This retrospective cohort study used data from treatment and supportive care intervention studies permanently closed to accrual between 2009 and 2015 at 16 United States-based CCs. Center and ClinicalTrials.gov data were used to generate both site-specific and random effects mixed models (random effect: institution). Accrual predictions were generated from each model and compared with the accrual prediction of the disease team (DT).

RESULTS

Sixteen institutions submitted 5,787 eligible trials (range, 93 to 697 trials per institution). Local accrual ranged from 363 to 6,716 participants; 1,053 studies (18%) accrued no participants. Actual average accrual was 8.5 participants (median, four participants). Site-specific models predicted accrual at 99% of actual and correctly predicted whether a study would accrue four or more participants 73% of the time versus DT prediction of 58%. Correlation at the category level was 30%; model sensitivity and specificity were 83% and 62%, respectively. The overall model predicted accrual 93% of actual and correctly predicted accrual of four or more participants 66% of the time, with a correlation at the category level of 28%.

CONCLUSION

Both regression models predicted clinical trial accrual at least as or more accurately than DT at all but one center. Site-specific models generally performed slightly better than the random effects model. This study confirms the previous finding that this method is an accurate and objective metric that can be easily implemented to improve clinical research resource allocation across multiple centers.

Conducting clinical trials-costs, impacts, and the value of clinical trials networks: A scoping review

BACKGROUND

A significant barrier to conducting clinical trials is their high cost, which is driven primarily by the time and resources required to activate trials and reach accrual targets. The high cost of running trials has a substantial impact on their long-term feasibility and the type of clinical research undertaken.

METHODS

A scoping review of the empirical literature on the costs associated with conducting clinical trials was undertaken for the years 2001-2015. Five reference databases were consulted to elicit how trials costs are presented in the literature. A review instrument was developed to extract the content of in-scope papers. Findings were characterized by date and place of publication, clinical disease area, and network/cooperative group designation, when specified. Costs were captured and grouped by patient accrual and management, infrastructure, and the opportunity costs associated with industry funding for trials research. Cost impacts on translational research and health systems were also captured, as were recommendations to reduce trial expenditures. Since articles often cited multiple costs, multiple cost coding was used during data extraction to capture the range and frequency of costs.

RESULTS

A total of 288 empirical articles were included. The distribution of reported costs was: patient management and accrual costs (132 articles), infrastructure costs (118 articles) and the opportunity costs of industry sponsorship (72 articles). 221 articles reported on the impact of undertaking costly trials on translational research and health systems; of these, the most frequently reported consequences were to research integrity (52% of articles), research capacity (36% of articles) and running low-value trials (34% of articles). 254 articles provided recommendations to reduce trial costs; of these, the most frequently reported recommendations related to improvements in: operational efficiencies (33% of articles); patient accrual (24% of articles); funding for trials and transparency in trials reporting (18% of articles, each).

CONCLUSION

Key findings from the review are: 1) delayed trial activation has costs to budgets and research; 2) poor accrual leads to low-value trials and wasted resources; 3) the pharmaceutical industry can be a pragmatic, if problematic, partner in clinical research; 4) organizational know-how and successful research collaboration are benefits of network/cooperative groups; and 5) there are spillover benefits of clinical trials to healthcare systems, including better health outcomes, enhanced research capacity, and drug cost avoidance. There is a need for more economic evaluations of the benefits of clinical research, such as health system use (or avoidance) and health outcomes in cities and health authorities with institutions that conduct clinical research, to demonstrate the affordability of clinical trials, despite their high cost.

Predicting Low Accrual in the National Cancer Institute's Cooperative Group Clinical Trials

BACKGROUND

The extent to which trial-level factors differentially influence accrual to trials has not been comprehensively studied. Our objective was to evaluate the empirical relationship and predictive properties of putative risk factors for low accrual in the National Cancer Institute's (NCI's) Cooperative Group Program, now the National Clinical Trials Network (NCTN).

METHODS

Data from 787 phase II/III adult NCTN-sponsored trials launched between 2000 and 2011 were used to develop a logistic regression model to predict low accrual, defined as trials that closed with or were accruing at less than 50% of target; 46 trials opened between 2012 and 2013 were used for prospective validation. Candidate predictors were identified from a literature review and expert interviews; final predictors were selected using stepwise regression. Model performance was evaluated by calibration and discrimination via the area under the curve (AUC). All statistical tests were two-sided.

RESULTS

Eighteen percent (n = 145) of NCTN-sponsored trials closed with low accrual or were accruing at less than 50% of target three years or more after initiation. A multivariable model of twelve trial-level risk factors had good calibration and discrimination for predicting trials with low accrual (AUC in trials launched 2000-2011 = 0.739, 95% confidence interval [CI] = 0.696 to 0.783]; 2012-2013: AUC = 0.732, 95% CI = 0.547 to 0.917). Results were robust to different definitions of low accrual and predictor selection strategies.

CONCLUSIONS

We identified multiple characteristics of NCTN-sponsored trials associated with low accrual, several of which have not been previously empirically described, and developed a prediction model that can provide a useful estimate of accrual risk based on these factors. Future work should assess the role of such prediction tools in trial design and prioritization decisions.

Effect of medical oncologists' attitudes on accrual to clinical trials in a community setting

PURPOSE

Oncology clinical trials (OCTs) are crucial in evaluating new cancer treatments, but only 2% to 3% of US adult patients with cancer enter OCTs. This study assessed barriers to participation in clinical trials among oncologists in a large integrated health care delivery system with an active clinical trials program. Although many studies have identified major physician barriers to enrollment, few have examined how these barriers affect actual trial accrual.

METHODS

Using information from a mailed survey, we examined the effect of oncologists' attitudes, beliefs, experiences, sociodemographic factors, and practice characteristics on clinical trial accrual in the 2 years following the survey. We identified relationships between these variables and subsequent clinical trial accrual using correlations and mixed effects models.

RESULTS

A construct combining questions that assessed oncologist attitudes, beliefs, and experiences substantially influenced OCT enrollment (r = .51; P < .0001). This construct included awareness of open clinical trials and specific eligible patients, as well as the practice of initiating a discussion about OCTs with most eligible patients. This broad concept of awareness had the greatest correlation with enrollment and mediated the effect on enrollment of other values and beliefs, such as welcoming a patient's initiation of a trial discussion and valuing the support of research nurses and coordinators.

CONCLUSION

Even in a health care setting with an active clinical trials program, substantial research personnel, infrastructure support, and widespread access to trials among oncologists and patients, oncologists' participation remains quite variable. Oncologist values, beliefs, and awareness of clinical trials play an important role in OCT accrual.

The National Cancer Institute-American Society of Clinical Oncology Cancer Trial Accrual Symposium: summary and recommendations

INTRODUCTION

Many challenges to clinical trial accrual exist, resulting in studies with inadequate enrollment and potentially delaying answers to important scientific and clinical questions.

METHODS

The National Cancer Institute (NCI) and the American Society of Clinical Oncology (ASCO) cosponsored the Cancer Trial Accrual Symposium: Science and Solutions on April 29-30, 2010 to examine the state of accrual science related to patient/community, physician/provider, and site/organizational influences, and identify new interventions to facilitate clinical trial enrollment. The symposium featured breakout sessions, plenary sessions, and a poster session including 100 abstracts. Among the 358 attendees were clinical investigators, researchers of accrual strategies, research administrators, nurses, research coordinators, patient advocates, and educators. A bibliography of the accrual literature in these three major areas was provided to participants in advance of the meeting. After the symposium, the literature in these areas was revisited to determine if the symposium recommendations remained relevant within the context of the current literature.

RESULTS

Few rigorously conducted studies have tested interventions to address challenges to clinical trials accrual. Attendees developed recommendations for improving accrual and identified priority areas for future accrual research at the patient/community, physician/provider, and site/organizational levels. Current literature continues to support the symposium recommendations.

CONCLUSIONS

A combination of approaches addressing both the multifactorial nature of accrual challenges and the characteristics of the target population may be needed to improve accrual to cancer clinical trials. Recommendations for best practices and for future research developed from the symposium are provided.

Design-phase prediction of potential cancer clinical trial accrual success using a research data mart

BACKGROUND

Many cancer interventional clinical trials are not completed because the required number of eligible patients are not enrolled.

OBJECTIVE

To assess the value of using a research data mart (RDM) during the design of cancer clinical trials as a predictor of potential patient accrual, so that less trials fail to meet enrollment requirements.

MATERIALS AND METHODS

The eligibility criteria for 90 interventional cancer trials were translated into i2b2 RDM queries and cohort sizes obtained for the 2 years prior to the trial initiation. These RDM cohort numbers were compared to the trial accrual requirements, generating predictions of accrual success. These predictions were then compared to the actual accrual performance to evaluate the ability of this methodology to predict the trials' likelihood of enrolling sufficient patients.

RESULTS

Our methodology predicted successful accrual (specificity) with 0.969 (=31/32 trials) accuracy (95% CI 0.908 to 1) and predicted failed accrual (sensitivity) with 0.397 (=23/58 trials) accuracy (95% CI 0.271 to 0.522). The positive predictive value, or precision rate, is 0.958 (=23/24) (95% CI 0.878 to 1).

DISCUSSION

A prediction of 'failed accrual' by this methodology is very reliable, whereas a prediction of accrual success is less so, as causes of accrual failure other than an insufficient eligible patient pool are not considered.

CONCLUSIONS

The application of this methodology to cancer clinical design would significantly improve cancer clinical research by reducing the costly efforts expended initiating trials that predictably will fail to meet accrual requirements.

Usefulness of prestudy assessment of patient willingness to undergo tissue biopsy for correlative studies in a melanoma vaccine trial

BACKGROUND

Performing biopsies for correlative studies in cancer trials raises ethical and regulatory concerns and may impact trial accrual negatively. However, strategies to address these concerns remain largely unexplored.

PURPOSE

We sought to assess the perceived risk of mandatory tissue biopsies to be performed for research purposes as part of a clinical trial of a melanoma vaccine by administering a pretrial accrual assessment questionnaire in the population of interest. Furthermore, we explored how such survey data may be used to address potential concerns of regulatory and funding organizations that may not be able to assess the risks of those biopsies. Method A total of 91 melanoma patients, similar to potential participants in a melanoma vaccine pilot study, scored their willingness, on a 9-point Likert scale, to participate in vaccine trials involving no skin biopsy versus a skin biopsy resulting in a 3-, 6-, or 12-cm scar. The vaccine trial was performed with skin biopsies leaving a 6-cm scar. Accrual rate was assessed and that accrual was compared to the accrual of two similar vaccine trials without biopsy requirements.

RESULTS

A total of 95% of the participants expressed willingness to enter a vaccine trial (likely to highly likely). This proportion decreased to 74%, 63%, and 59%, respectively, for vaccine trials requiring skin biopsy leaving a 3-, 6-, or 12-cm scar. The trial was designed with an estimated 40% decrease in accrual rate compared to prior studies (2 participants expected/month). The resulting trial with a 6-cm biopsy exceeded that accrual rate estimate and had a similar accrual rate to vaccine trials without a biopsy (4.1 vs 2.7-4.6 participants/month).

LIMITATIONS

Potential limitations of this study include the exclusion of some questionnaire responses and the post hoc nature of the analysis.

CONCLUSION

Willingness to participate in vaccine trials was decreased by the requirement for skin biopsy, but the size of the biopsy was less of a deterrent than expected. Findings from brief surveys may aid in risk assessment during regulatory review, predict acceptability of tissue collection for correlative studies, and support regulatory approval and meeting accrual goals of the study.

Achieving sufficient accrual to address the primary endpoint in phase III clinical trials from U.S. Cooperative Oncology Groups

PURPOSE

Assessing impact of poor accrual on premature trial closure requires a relevant metric. We propose defining accrual sufficiency on apparent ability to address primary endpoints (PE) rather than attaining accrual targets.

EXPERIMENTAL DESIGN

All phase III trials open January 1, 1993, to December 31, 2002, by five U.S. oncology Clinical Trials Cooperative Groups (CTCG) were evaluated for accrual sufficiency and scientific results. Sufficient accrual included meeting accrual target, CTCGs documentation attesting adequate accrual, or conclusive results at interim analysis; insufficient accrual included poor accrual as cited closure reason or other reasons rendering a trial unable to address its primary endpoints. Closure rates based on our accrual sufficiency definition are compared with rates of meeting accrual targets and addressing the primary endpoints. A percentage of target accrual above which trials commonly answer the intended scientific question was identified to serve as an alternative to meeting full target accrual in designating accrual success.

RESULTS

Of 238 eligible trials, 158 (66%) closed with sufficient accrual. Among 80 trials with insufficient accrual, 70 (29%) closed specifically because of poor accrual. Inadequate accrual rates are overemphasized when defining accrual success solely by meeting accrual targets. Nearly 75% of trials conclusively addressed the primary endpoints with positive results in 39% of trials. Exceeding 80% of target accrual serves as a reliable proxy for answering the intended scientific question.

CONCLUSIONS

Approximately one third of phase III trials closed with insufficient accrual to address the primary endpoints, primarily due to poor accrual. Defining accrual sufficiency broader than meeting accrual targets represents a fairer account of trial closures.

Clinical trial implementation and recruitment: lessons learned from the early closure of a randomized clinical trial

BACKGROUND

The NHLBI-sponsored Sickle Cell Disease Clinical Research Network (SCDCRN) conducted a multi-center, acute intervention randomized clinical trial of two methods of Patient Controlled Analgesia for acute pain. This trial was terminated early due to low enrollment. We analyzed the perceived barriers and recruitment difficulties as reported by the coordinators and principal investigators.

METHODS

Participating sites completed a missed eligibility log of subjects admitted in pain crisis throughout the study and a survey at the end of the trial. The survey covered site-specific factors, policies, and procedures in study implementation, recruitment strategies, and eligibility factors. The New England Research Institutes (NERI) collected de-identified surveys from 31 respondents at 29 of 31 participating sites.

RESULTS

From December 2009 to June 2010, 1116 patient encounters for SCD and pain occurred at participating institutions: 38 subjects were enrolled (14 pediatric and 24 adults) and 34 completed the trial, below the projected 278 subjects. Fourteen sites enrolled subjects and seventeen did not. Recruitment barriers included insufficient staff, subject ineligibility or in too much pain to consent, competing protocols, and concerns regarding pain control. Recruitment methods were referrals from urgent care, SCD clinics and in house databases. No use of media or outside physicians was reported.

CONCLUSION

We identified multiple barriers to patient accrual including short duration of enrollment period, protocol design, complex dosing schedule, requirement for staff availability during week-end and after hours, multiple departments' involvement, protocol acceptance, eligibility criteria, competing protocols, and limited staff. Each of these areas should be targeted for intervention in order to plan and conduct successful future clinical trials.

AccrualNet: Addressing Low Accrual Via a Knowledge-Based, Community of Practice Platform

PURPOSE

Present the design and initial evaluation of a unique, Web-enabled platform for the development of a community of practice around issues of oncology clinical trial accrual.

METHODS

The National Cancer Institute (NCI) conducted research with oncology professionals to identify unmet clinical trial accrual needs in the field. In response, a comprehensive platform for accrual resources, AccrualNet, was created by using an agile development process, storyboarding, and user testing. Literature and resource searches identified relevant content to populate the site. Descriptive statistics were tracked for resource and site usage. Use cases were defined to support implementation.

RESULTS

ACCRUALNET HAS FIVE LEVELS: (1) clinical trial macrostages (prestudy, active study, and poststudy); (2) substages (developing a protocol, selecting a trial, preparing to open, enrolling patients, managing the trial, retaining participants, and lessons learned); (3) strategies for each substage; (4) multiple activities for each strategy; and (5) multiple resources for each activity. Since its launch, AccrualNet has had more than 45,000 page views, with the Tools & Resources, Conversations, and Training sections being the most viewed. Total resources have increased 69%, to 496 items. Analysis of articles in the site reveals that 22% are from two journals and 46% of the journals supplied a single article. To date, there are 29 conversations with 43 posts. Four use cases are discussed.

CONCLUSION

AccrualNet represents a unique, centralized comprehensive-solution platform to systematically capture accrual knowledge for all stages of a clinical trial. It is designed to foster a community of practice by encouraging users to share additional strategies, resources, and ideas.

Phase III clinical trial development: a process of chutes and ladders

PURPOSE

The Institute of Medicine report on cooperative groups and the National Cancer Institute (NCI) report from the Operational Efficiency Working Group both recommend changes to the processes for opening a clinical trial. This article provides evidence for the need for such changes by completing the first comprehensive review of all the time and steps required to open a phase III oncology clinical trial and discusses the effect of time to protocol activation on subject accrual.

METHODS

The Dilts and Sandler method was used at four cancer centers, two cooperative groups, and the NCI Cancer Therapy Evaluation Program. Accrual data were also collected.

RESULTS

Opening a phase III cooperative group therapeutic trial requires 769 steps, 36 approvals, and a median of approximately 2.5 years from formal concept review to study opening. Time to activation at one group ranged from 435 to 1,604 days, and time to open at one cancer center ranged from 21 to 836 days. At centers, group trials are significantly more likely to have zero accruals (38.8%) than nongroup trials (20.6%; P < 0.0001). Of the closed NCI Cancer Therapy Evaluation Program-approved phase III clinical trials from 2000 to 2007, 39.1% resulted in <21 accruals.

CONCLUSIONS

The length, variability, and low accrual results demonstrate the need for the NCI clinical trials system to be reengineered. Improvements will be of only limited effectiveness if done in isolation; there is a need to return to the collaborative spirit with all parties creating an efficient and effective system. Recommendations put forth by the Institute of Medicine and Operational Efficiency Working Group reports, if implemented, will aid this renewal.