Evaluating the effects of design parameters on the performances of phase I trial designs

The 3 + 3 design in dose-finding studies with small sample sizes: Pitfalls and possible remedies

In the last few years, numerous novel designs have been proposed to improve the efficiency and accuracy of phase I trials to identify the maximum-tolerated dose (MTD) or the optimal biological dose (OBD) for noncytotoxic agents. However, the conventional 3+3 approach, known for its and poor performance, continues to be an attractive choice for many trials despite these alternative suggestions. The article seeks to underscore the importance of moving beyond the 3+3 design by highlighting a different key element in trial design: the estimation of sample size and its crucial role in predicting toxicity and determining the MTD. We use simulation studies to compare the performance of the most used phase I approaches: 3+3, Continual Reassessment Method (CRM), Keyboard and Bayesian Optimal Interval (BOIN) designs regarding three key operating characteristics: the percentage of correct selection of the true MTD, the average number of patients allocated per dose level, and the average total sample size. The simulation results consistently show that the 3+3 algorithm underperforms in comparison to model-based and model-assisted designs across all scenarios and metrics. The 3+3 method yields significantly lower (up to three times) probabilities in identifying the correct MTD, often selecting doses one or even two levels below the actual MTD. The 3+3 design allocates significantly fewer patients at the true MTD, assigns higher numbers to lower dose levels, and rarely explores doses above the target dose-limiting toxicity (DLT) rate. The overall performance of the 3+3 method is suboptimal, with a high level of unexplained uncertainty and significant implications for accurately determining the MTD. While the primary focus of the article is to demonstrate the limitations of the 3+3 algorithm, the question remains about the preferred alternative approach. The intention is not to definitively recommend one model-based or model-assisted method over others, as their performance can vary based on parameters and model specifications. However, the presented results indicate that the CRM, Keyboard, and BOIN designs consistently outperform the 3+3 and offer improved efficiency and precision in determining the MTD, which is crucial in early-phase clinical trials.

Performance of phase-I dose finding designs with and without a run-in intra-patient dose escalation stage

Dose-finding designs for phase-I trials aim to determine the recommended phase-II dose (RP2D) for further phase-II drug development. If the trial includes patients for whom several lines of standard therapy failed or if the toxicity of the investigated agent does not necessarily increase with dose, optimal dose-finding designs should limit the frequency of treatment with suboptimal doses. We propose a two-stage design strategy with a run-in intra-patient dose escalation part followed by a more traditional dose-finding design. We conduct simulation studies to compare the 3 + 3 design, the Bayesian Optimal Interval Design (BOIN) and the Continual Reassessment Method (CRM) with and without intra-patient dose escalation. The endpoints are accuracy, sample size, safety, and therapeutic efficiency. For scenarios where the correct RP2D is the highest dose, inclusion of an intra-patient dose escalation stage generally increases accuracy and therapeutic efficiency. However, for scenarios where the correct RP2D is below the highest dose, intra-patient dose escalation designs lead to increased risk of overdosing and an overestimation of RP2D. The magnitude of the change in operating characteristics after including an intra-patient stage is largest for the 3 + 3 design, decreases for the BOIN and is smallest for the CRM.

A Road Map for Designing Phase I Clinical Trials of Radiotherapy-Novel Agent Combinations

Radiotherapy has proven efficacy in a wide range of cancers. There is growing interest in evaluating radiotherapy-novel agent combinations and a drive to initiate this earlier in the clinical development of the novel agent, where the scientific rationale and preclinical evidence for a radiotherapy combination approach are high. Optimal design, delivery, and interpretation of studies are essential. In particular, the design of phase I studies to determine safety and dosing is critical to an efficient development strategy. There is significant interest in early-phase research among scientific and clinical communities over recent years, at a time when the scrutiny of the trial methodology has significantly increased. To enhance trial design, optimize safety, and promote efficient trial conduct, this position paper reviews the current phase I trial design landscape. Key design characteristics extracted from 37 methodology papers were used to define a road map and a design selection process for phase I radiotherapy-novel agent trials. Design selection is based on single- or dual-therapy dose escalation, dose-limiting toxicity categorization, maximum tolerated dose determination, subgroup evaluation, software availability, and design performance. Fifteen of the 37 designs were identified as being immediately accessible and relevant to radiotherapy-novel agent phase I trials. Applied examples of using the road map are presented. Developing these studies is intensive, highlighting the need for funding and statistical input early in the trial development to ensure appropriate design and implementation from the outset. The application of this road map will improve the design of phase I radiotherapy-novel agent combination trials, enabling a more efficient development pathway.

An overview of the BOIN design and its current extensions for novel early-phase oncology trials

Bayesian Optimal Interval (BOIN) designs are a class of model-assisted dose-finding designs that can be used in oncology trials to determine the maximum tolerated dose (MTD) of a study drug based on safety or the optimal biological dose (OBD) based on safety and efficacy. BOIN designs provide a complete suite for dose finding in early phase trials, as well as a consistent way to explore different scenarios such as toxicity, efficacy, continuous outcomes, delayed toxicity or efficacy and drug combinations in a unified manner with easy access to software to implement most of these designs. Although built upon Bayesian probability models, BOIN designs are operationally simple in general and have good statistical operating characteristics compared to other dose-finding designs. This review paper describes the original BOIN design and its many extensions, their advantages and limitations, the software used to implement them, and the most suitable situation for use of each of these designs. Published examples of the implementation of BOIN designs are provided in the Appendix.

Contemporary dose-escalation methods for early phase studies in the immunotherapeutics era

Phase 1 dose-escalation trials are crucial to drug development by providing a framework to assess the toxicity of novel agents in a stepwise and monitored fashion. Despite widely adopted, rule-based dose-escalation methods (such as 3 + 3) are limited in finding the maximum tolerated dose (MTD) and tend to treat a significant number of patients at subtherapeutic doses. Newer methods of dose escalation, such as model-based and model-assisted designs, have emerged and are more accurate in finding MTD. However, these designs have not yet been broadly embraced by investigators. In this review, we summarise the advantages and disadvantages of contemporary dose-escalation methods, with emphasis on model-assisted designs, including time-to-event designs and hybrid methods involving optimal biological dose (OBD). The methods reviewed include mTPI, keyboard, BOIN, and their variations. In addition, the challenges of drug development (and dose-escalation) in the era of immunotherapeutics are discussed, where many of these agents typically have a wide therapeutic window. Fictional examples of how the dose-escalation method chosen can alter the outcomes of a phase 1 study are described, including the number of patients enrolled, the trial's timeframe, and the dose level chosen as MTD. Finally, the recent trends in dose-escalation methods applied in phase 1 trials in the immunotherapeutics era are reviewed. Among 856 phase I trials from 2014 to 2019, a trend towards the increased use of model-based and model-assisted designs over time (OR = 1.24) was detected. However, only 8% of the studies used non-rule-based dose-escalation methods. Increasing familiarity with such dose-escalation methods will likely facilitate their uptake in clinical trials.

Would the Recommended Dose Have Been Different Using Novel Dose-Finding Designs? Comparing Dose-Finding Designs in Published Trials

Simulation studies have shown that novel designs such as the continual reassessment method and the Bayesian optimal interval (BOIN) design outperform the 3 + 3 design by recommending the maximum tolerated dose (MTD) more often, using less patients, and allotting more patients to the MTD. However, it is not clear whether these novel designs would have yielded different results in the context of real-world dose-finding trials. This is a commonly mentioned reason for the continuous use of 3 + 3 designs for oncology trials, with investigators considering simulation studies not sufficiently convincing to warrant the additional design complexity of novel designs.

METHODS

We randomly sampled 60 published dose-finding trials to obtain 22 that used the 3 + 3 design, identified an MTD, published toxicity data, and had more than two dose levels. We compared the published MTD with the estimated MTD using the continual reassessment method and BOIN using target toxicity rates of 25% and 30% and toxicity data from the trial. Moreover, we compared patient allocation and sample size assuming that these novel designs had been implemented.

RESULTS

Model-based designs chose dose levels higher than the published MTD in about 40% of the trials, with estimated and observed toxicity rates closer to the target toxicity rates of 25% and 30%. They also assigned less patients to suboptimal doses and permitted faster dose escalation.

CONCLUSION

This study using published dose-finding trials shows that novel designs would recommend different MTDs and confirms the advantages of these designs compared with the 3 + 3 design, which were demonstrated by simulation studies.

Bayesian optimization for estimating the maximum tolerated dose in Phase I clinical trials

We introduce a Bayesian optimization method for estimating the maximum tolerated dose in this article. A number of parametric model-based methods have been proposed to estimate the maximum tolerated dose; however, parametric model-based methods need an assumption that dose-toxicity relationships follow specific theoretical models. This assumption potentially leads to suboptimal dose selections if the dose-toxicity curve is misspecified. Our proposed method is based on a Bayesian optimization framework for finding a global optimizer of unknown functions that are expensive to evaluate while using very few function evaluations. It models dose-toxicity relationships with a nonparametric model; therefore, a more flexible estimation can be realized compared with existing parametric model-based methods. Also, most existing methods rely on point estimates of dose-toxicity curves in their dose selections. In contrast, our proposed method exploits a probabilistic model for an unknown function to determine the next dose candidate without ignoring the uncertainty of posterior while imposing some dose-escalation limitations. We investigate the operating characteristics of our proposed method by comparing them with those of the Bayesian-based continual reassessment method and two different nonparametric methods. Simulation results suggest that our proposed method works successfully in terms of selections of the maximum tolerated dose correctly and safe dose allocations.

The importance of loss functions: A note on the evolution of the Toxicity Probability Interval design

The Toxicity Probability Interval Design by Ji et al. (2007), which was subsequently modified by the mTPI (Ji et al., 2010), proposed a more efficient approach to early-phase dose-finding than conventional designs like 3 + 3. Subsequent authors reported issues with the method, finding that it tends to stay at a dose level when clinical intuition would suggest the toxicity level warrants decrease. Several iterations of refinement proceeded in an effort to address these issues, including the mTPI-2 and the keyboard method, as well as alternative approaches such as the BOIN. This author suggests the reason for these safety issues involves the underlying loss function. The TPI and mTPI used the identify function defined over wide intervals. As explained in this paper, this function and its domain can be problematic as a model of patients' loss experience. Later refinements moved the loss function closer to one more consistent with clinical intuition, and this explains their improved safety performance. Greater attention to quality as defined by fitness for use, including early evaluation of patient-experience and clinical-intuition implications of proposed loss functions, may improve future design efforts.