Single-arm oncology trials and the nature of external controls arms

Supplementing Single-Arm Trials with External Control Arms-Evaluation of German Real-World Data

As single-arm trials (SATs) are increasingly used in pharmaceutical research, the validity of such study designs needs to be critically assessed. We characterize the feasibility of supplementing SATs with real-world data (RWD)-derived external control arms by determining the proportion of SATs on breast cancer and amyotrophic lateral sclerosis (ALS) for which an external control arm based on RWD can be constructed. The main outcome measure is the number and percentage of trials for which all important eligibility criteria and at least one primary endpoint could be identified in one of five German RWD sources. We surveyed all SATs concerning breast cancer or ALS treatment registered in the European Union's clinical trial registers between 2004 and 2023. Ten out of 379 breast cancer SATs and 2 of 11 ALS SATs could feasibly be supplemented with RWD-derived external control arms, if all important eligibility criteria and a primary endpoint have to be identifiable in the RWD source. Ninety-three breast cancer trials had at least one outcome ascertainable in a RWD source, and 35 trials had all important eligibility criteria recorded in a RWD source. Nine ALS trials had at least one primary endpoint ascertainable in RWD sources, and 2 had all important eligibility criteria recorded in a RWD source. Our study shows that SATs with RWD-derived external control arms will rarely be suitable to establish treatment effects of medicines in the current setting for the investigated phenotypes and that SATs should be designed with limitations of the source of external controls in mind.

Enhancing External Control Arm Analyses through Data Calibration and Hybrid Designs

Almost all external control arm analyses to contextualize findings of a single arm trial struggle with two key issues: the lack of baseline randomization, and equally important, the difference in data collection between the experimental arm with its primary data collection, and the external control arm using secondary data. We illustrate the data calibration design to remedy issues arising from differential measurements in the two arms, and discuss the hybrid design that expands an underpowered randomized internal control arm with real-world data to mitigate the lack of randomization of the external control arm. We show how the two approaches fit into an evidence-development strategy that naturally builds on the incremental insights gained.

A bootstrapping method to optimize go/no-go decisions from single-arm, signal-finding studies in oncology

Phase Ib trials are common in oncology development but often are not powered for statistical significance. Go/no-go decisions are largely driven by observed trends in response data. We applied a bootstrapping method to systematically compare tumor dynamic end points to historical control data to identify drugs with clinically meaningful efficacy. A proprietary mathematical model calibrated to phase Ib anti-PD-1 therapy trial data (KEYNOTE-001) was used to simulate thousands of phase Ib trials (n = 30) with a combination of anti-PD-1 therapy and four novel agents with varying efficacy. A redacted bootstrapping method compared these results to a simulated phase III control arm (N = 511) while adjusting for differences in trial duration and cohort size to determine the probability that the novel agent provides clinically meaningful efficacy. Receiver operating characteristic (ROC) analysis showed strong ability to separate drugs with modest (area under ROC [AUROC] = 83%), moderate (AUROC = 96%), and considerable efficacy (AUROC = 99%) from placebo in early-phase trials (n = 30). The method was shown to effectively move drugs with a range of efficacy through an in silico pipeline with an overall success rate of 93% and false-positive rate of 7.5% from phase I to phase III. This model allows for effective comparisons of tumor dynamics from early clinical trials with more mature historical control data and provides a framework to predict drug efficacy in early-phase trials. We suggest this method should be employed to improve decision making in early oncology trials.

Examining external control arms in oncology: A scoping review of applications to date

OBJECTIVES

Randomized controlled trials (RCTs) are the gold standard for evaluating the comparative efficacy and safety of new cancer therapies. However, enrolling patients in control arms of clinical trials can be challenging for rare cancers, particularly in the context of precision oncology and targeted therapies. External Control Arms (ECAs) are a potential solution to address these challenges in clinical research design. We conducted a scoping review to explore the use of ECAs in oncology.

METHODS

We systematically searched four databases, namely MEDLINE, EMBASE, Web of Science, and Scopus. We screened titles, abstracts, and full texts for eligible articles focusing on patients undergoing therapy for cancer, employing ECAs, and reporting clinical outcomes.

RESULTS

Of the 629 articles screened, 23 were included in this review. The earliest included studies were published in 1996, while most studies were published in the past 5 years. 44% (10/23) of ECAs were employed in blood-related cancer studies. Geographically, 30% (7/23) of studies were conducted in the United States, 22% (5/23) in Japan, and 9% (2/23) in South Korea. The primary data sources used to construct the ECAs involved pooled data from previous trials (35%, 8/23), administrative health databases (17%, 4/23) and electronic medical records (17%, 4/23). While 52% (12/23) of the studies employed methods to align treatment and ECAs characteristics, 48% (11/23) lacked explicit strategies.

CONCLUSION

ECAs offer a valuable approach in oncology research, particularly when alternative designs are not feasible. However, careful methodological planning and detailed reporting are essential for meaningful and reliable results.

Justifying the source of external comparators in single-arm oncology health technology submissions: a review of NICE and PBAC assessments

Background: The drive to expedite patient access for diseases with high unmet treatment needs has come with an increasing use of single-arm trials (SATs), especially in oncology. However, the lack of control arms in such trials creates challenges to assess and demonstrate comparative efficacy. External control (EC) arms can be used to bridge this gap, with various types of sources available to obtain relevant data. Objective: To examine the source of ECs in single-arm oncology health technology assessment (HTA) submissions to the National Institute for Health and Care Excellence (NICE) and the Pharmaceutical Benefits Advisory Committee (PBAC) and how this selection was justified by manufacturers and assessed by the respective HTA body. Methods: Single-arm oncology HTA submission reports published by NICE (England) and PBAC (Australia) from January 2011 to August 2021 were reviewed, with data qualitatively synthesized to identify themes. Results: Forty-eight oncology submissions using EC arms between 2011 and 2021 were identified, with most submissions encompassing blood and bone marrow cancers (52%). In HTA submissions to NICE and PBAC, the EC arm was typically constructed from a combination of data sources, with the company's justification in data source selection infrequently provided (PBAC [2 out of 19]; NICE [6 out of 29]), although this lack of justification was not heavily criticized by either HTA body. Conclusion: Although HTA bodies such as NICE and PBAC encourage that EC source justification should be provided in submissions, this review found that this is not typically implemented in practice. Guidance is needed to establish best practices as to how EC selection should be documented in HTA submissions.

Can a propensity score matching method be applied to assessing efficacy from single-arm proof-of-concept trials in oncology?

As a result of the escalating number of new cancer treatments being developed and competition among pharmaceutical companies, decisions regarding how to proceed with phase III trials are frequently based on findings from either single-arm phase I expansion cohorts or phase II studies that compare the efficacy of the study drug to a standard-of-care benchmark derived from historical data. However, even when eligibility criteria are matched, differences in the distribution of baseline patient features may influence the outcome of single-arm trials in real-world scenarios. Therefore, novel methods are needed to enhance the accuracy of efficacy prediction from current cohorts relative to historical data. In this study, we demonstrated the feasibility of using the propensity score matching (PSM) method to improve decision making by matching relevant baseline features between current and historical cohorts. According to our findings, utilizing the PSM method may provide a less biased means of comparing outcomes between current and historical cohorts relative to a naïve approach, which relies solely on differences in average outcomes between the cohorts.

Historic Clinical Trial External Control Arm Provides Actionable GEN-1 Efficacy Estimate Before a Randomized Trial

PURPOSE

To inform continued development of the novel immune agent GEN-1, we compared ovarian cancer patients' end points from a neoadjuvant single-arm phase IB study with those of similar historic clinical trial (HCT) patients who received standard neoadjuvant chemotherapy.

METHODS

Applying OVATION-1 trial (ClinicalTrials.gov identifier: NCT02480374) inclusion and exclusion criteria to Medidata HCT data, we identified historical trial patients for comparison. Integrating patient-level Medidata historic trial data (N = 41) from distinct neoadjuvant ovarian phase I-III trials with patient-level OVATION-1 data (N = 18), we selected Medidata patients with similar baseline characteristics as OVATION-1 patients using propensity score methods to create an external control arm (ECA).

RESULTS

Fifteen OVATION-1 patients (15 of 18, 83%) were matched to 15 (37%, 15 of 41) Medidata historical trial control patients. Matching attenuated preexisting differences in attributes between the groups. The median progression-free survival time was not reached by the OVATION-1 group and was 15.8 months (interquartile range, 11.40 months to nonestimable) for the ECA. The hazard of progression was 0.53 (95% CI, 0.16 to 1.73), favoring GEN-1 patients. Compared with ECA patients, OVATION-1 patients had more nausea, fatigue, chills, and infusion-related reactions.

CONCLUSION

Comparing results of a single-arm early-phase trial to those of a rigorously matched HCT ECA yielded insights regarding comparative efficacy prior to a randomized controlled trial. The effect size estimate itself informed both the decision to continue development and the randomized phase II trial (ClinicalTrials.gov identifier: NCT03393884) sample size. The work illustrates the potential of HCT data to inform drug development.

Utilization of real-world data in assessing treatment effectiveness for diffuse large B-cell lymphoma

Direct comparisons of the effectiveness of the numerous novel therapies in the diffuse large B-cell lymphoma (DLBCL) treatment landscape in a range of head-to-head randomized phase 3 trials would be time-consuming and costly. Comparative effectiveness studies using real-world data (RWD) represent a complementary approach. Recently, several studies of relapsed/refractory (R/R) DLBCL have used RWD to create observational cohorts to compare patient outcomes with cohorts derived from single-arm phase 2 trials. Using propensity score methods to balance clinically and prognostically relevant baseline covariates, closely matched patient-level cohorts can be generated. By incorporating appropriate measures to assess covariate balance and address potential bias in comparative effectiveness study designs, robust comparative analyses can be performed. Results from such studies have been used to supplement regulatory approval of therapies assessed in single-arm trials. While RWD studies have a greater susceptibility to bias compared to randomized controlled trials, well-designed and appropriately analyzed studies can provide complementary real-world evidence on treatment effectiveness.

Dynamic incorporation of real world evidence within the framework of adaptive design

For the clinical studies in rare diseases or small patient populations, having an adequately powered randomized controlled trial is further complicated by variability. As such, sample size re-estimation can be a useful tool if at an interim look the trial sample size needs to be increased to achieve adequate power to reject the null hypothesis. Meanwhile, borrowing or extrapolating information from real-world data or real-world evidence has gained increasing use in trial design and analysis since 2014. Combining these two strategies, high-quality real-world data, if leveraged properly, has the potential to generate real-world evidence that can assist interim decision-making, lower enrollment burden, and reduce study timeline and costs. With proper borrowing from historical control, some of the challenges in these high unmet medical need studies could be resolved considerably. We examine the incorporation of real-world evidence within the framework of adaptive design strategy in pediatric type II diabetes trials where recruitment has been challenging and the completion is hardly on time. Simulations under various scenarios are conducted to assess the borrowing strategy, i.e., the matching method in combination of sample size re-estimation. Comparisons of performance metrics are presented to showcase the advantages of proposed method.