Designing exploratory cancer trials using change in tumour size as primary endpoint

Modeling tumor measurement data to predict overall survival (OS) in cancer clinical trials

Introduction

Longitudinal tumor measurements (TM) are commonly recorded in cancer clinical trials of solid tumors. To define patient response to treatment, the Response Evaluation Criteria in Solid Tumors (RECIST) categorizes the otherwise continuous measurements, which results in substantial information loss. We investigated two modeling approaches to incorporate all available cycle-by-cycle (continuous) TM to predict overall survival (OS) and compare the predictive accuracy of these two approaches to RECIST.

Material and methods

Joint modeling (JM) for longitudinal TM and OS and two-stage modeling with potential time-varying coefficients were utilized to predict OS using data from three trials with cycle-by-cycle TM. The JM approach incorporates TM data collected throughout the course of the clinical trial. The two-stage modeling approach incorporates information from early assessments (before 12 weeks) to predict subsequent OS outcome. The predictive accuracy was quantified by c-indices.

Results

Data from 577, 337, and 126 patients were included for the analysis (from two stage IV colorectal cancer trials (N9741, N9841) and an advanced non-small cell lung cancer trial (N0026), respectively). Both the JM and two-stage modeling reached a similar conclusion, i.e. the baseline covariates (age, gender, and race) were mostly not predictive of OS (p-value > 0.05). Quantities derived from TM were strong predictors of OS in the two colorectal cancer trials (p < 0.001 for both association in JM and two-stage modeling parameters); but less so in the lung cancer trial (p = 0.053 for association in JM and p = 0.024 and 0.160 for two-stage modeling parameters). The c-indices from the two-stage modeling were higher than those from a model using RECIST (range: 0.611–0.633 versus 0.586–0.590). The dynamic c-indices from the JM were in the range of 0.627–0.683 indicating good predictive accuracy.

Conclusion

Both modeling approaches provide highly interpretable and clinical meaningful results; the improved predictive performance compared with RECIST indicates the possibility of deriving better trial endpoints from these approaches.

•

Two-stage modeling incorporating time-varying coefficients achieves better predictive accuracy than RECIST-alone.

•

Two–stage modeling offers the possibility of alternative endpoint definition.

•

Serial tumor measurements can be incorporated in OS prediction using joint modeling.

•

Joint modeling can potentially guide individualized medicine.

Efficient analysis of time-to-event endpoints when the event involves a continuous variable crossing a threshold

In many trials, the duration between patient enrolment and an event occurring is used as the efficacy endpoint. Common endpoints of this type include the time until relapse, progression to the next stage of a disease, or time until remission. The criteria of an event may be defined by multiple components, one or more of which may be a continuous measurement being above or below a threshold. Typical analyses consider all components as binary variables and record the first time at which the patient has an event. This is analysed through constructing and testing survival functions using Kaplan-Meier, parametric models or Cox models. This approach ignores information contained in the continuous components. We propose a method that makes use of this information to improve the precision of analyses using these types of endpoints. We use joint modelling of the continuous and binary components to construct survival curves. We show how to compute confidence intervals for quantities of interest, such as the median or mean event time. We assess the properties of the proposed method using simulations and data from a phase II cancer trial and an observational study in renal disease.

Missing tumor measurement (TM) data in the search for alternative TM-based endpoints in cancer clinical trials

PURPOSE

Missing data commonly occur in cancer clinical trials (CCT) and may hinder the search for alternative trial endpoints. We consider reasons for missing tumor measurement (TM) data in CCT and how missing TM data are typically handled. We explore the potential impact of missing TM data on predictive ability of a set of TM-based endpoints.

METHODS

Literature review identifies reasons for and approaches to handling missing TM data. Data from 3 actual clinical trials were used for illustration. A sensitivity analysis of the potential impact of missing TM data was performed by comparing overall survival (OS) predictive ability of alternative endpoints using observed and imputed data.

RESULTS

Reasons for missing TM data in CCT are presented, based on the literature review and the three trials. Although missing TM data impacted individual objective status (e.g. 12-week status changed for 53% of patients in one imputation set), it surprisingly only minimally impacted endpoint predictive ability (e.g. median c-indices of 500 imputed datasets ranged from 0.566 to 0.570 for N9741, 0.592-0.616 for N9841, and 0.542-0.624 for N0026).

CONCLUSION

By understanding the reasons for missingness, we can better anticipate them and minimize their occurrence. Our preliminary analysis suggests missing TM data may not impact endpoint predictive ability, but could impact objective response status classification; however these findings require further validation. With response status accepted as an important phase II endpoint in the development of new cancer therapies (including immunotherapy), we urge that in CCT complete TM data collection and adherence to protocol-defined disease evaluation as closely as possible be a priority.

A response-adaptive randomization procedure for multi-armed clinical trials with normally distributed outcomes

We propose a novel response‐adaptive randomization procedure for multi‐armed trials with continuous outcomes that are assumed to be normally distributed. Our proposed rule is non‐myopic, and oriented toward a patient benefit objective, yet maintains computational feasibility. We derive our response‐adaptive algorithm based on the Gittins index for the multi‐armed bandit problem, as a modification of the method first introduced in Villar et al. (Biometrics, 71, pp. 969‐978). The resulting procedure can be implemented under the assumption of both known or unknown variance. We illustrate the proposed procedure by simulations in the context of phase II cancer trials. Our results show that, in a multi‐armed setting, there are efficiency and patient benefit gains of using a response‐adaptive allocation procedure with a continuous endpoint instead of a binary one. These gains persist even if an anticipated low rate of missing data due to deaths, dropouts, or complete responses is imputed online through a procedure first introduced in this paper. Additionally, we discuss how there are response‐adaptive designs that outperform the traditional equal randomized design both in terms of efficiency and patient benefit measures in the multi‐armed trial context.

An adaptive design for the identification of the optimal dose using joint modeling of continuous repeated biomarker measurements and time-to-toxicity in phase I/II clinical trials in oncology

We present a new adaptive dose-finding method, based on a joint modeling of longitudinal continuous biomarker activity measurements and time to first dose limiting toxicity, with a shared random effect. Estimation relies on likelihood that does not require approximation, an important property in the context of small sample sizes, typical of phase I/II trials. We address the important case of missing at random data that stem from unacceptable toxicity, lack of activity and rapid deterioration of phase I patients. The objective is to determine the lowest dose within a range of highly active doses, under the constraint of not exceeding the maximum tolerated dose. The maximum tolerated dose is associated to some cumulative risk of dose limiting toxicity over a predefined number of treatment cycles. Operating characteristics are explored via simulations in various scenarios.

Are tumor size changes predictive of survival for checkpoint blockade based immunotherapy in metastatic melanoma?

BACKGROUND

In oncology clinical development, objective response rate, disease control rate and early tumor size changes are commonly used as efficacy metrics for early decision-making. However, for immunotherapy trials, it is unclear whether these early efficacy metrics are still predictive of long-term clinical benefit such as overall survival. The goal of this paper is to identify appropriate early efficacy metrics predictive of overall survival for immunotherapy trials.

METHODS

Based on several checkpoint blockade based immunotherapy studies in metastatic melanoma, we evaluated the predictive value of early tumor size changes and RECIST-based efficacy metrics at various time points on overall survival. The cut-off values for tumor size changes to predict survival were explored via tree based recursive partitioning and validated by external data. Sensitivity analyses were performed for the cut-offs.

RESULTS

The continuous tumor size change metric and RECIST-based trichotomized response metric at different landmark time points were found to be statistically significantly associated with overall survival. The predictive values were higher at Week 12 and 18 than those at Week 24. The percentage of tumor size changes appeared to have comparable or lower predictive values than the RECIST-based trichotomized metric, and a cut-off of approximately 10% tumor reduction appeared to be reasonable for predicting survival.

CONCLUSIONS

An approximate 10% tumor reduction may be a reasonable cut-off for early decision-making while the RECIST-based efficacy metric remains the primary tool. Early landmark analysis is especially useful for decision making when accrual is fast. Composite response rate (utilizing different weights for PR/CR and SD) may be worth further investigation.

TRIAL REGISTRATION

Clinical trials gov, NCT01295827 , Registered February 15, 2011; NCT01704287 , Registered October 11, 2012; NCT01866319 , Registered May 31, 2013.

Improving phase II oncology trials using best observed RECIST response as an endpoint by modelling continuous tumour measurements

In many phase II trials in solid tumours, patients are assessed using endpoints based on the Response Evaluation Criteria in Solid Tumours (RECIST) scale. Often, analyses are based on the response rate. This is the proportion of patients who have an observed tumour shrinkage above a predefined level and no new tumour lesions. The augmented binary method has been proposed to improve the precision of the estimator of the response rate. The method involves modelling the tumour shrinkage to avoid dichotomising it. However, in many trials the best observed response is used as the primary outcome. In such trials, patients are followed until progression, and their best observed RECIST outcome is used as the primary endpoint. In this paper, we propose a method that extends the augmented binary method so that it can be used when the outcome is best observed response. We show through simulated data and data from a real phase II cancer trial that this method improves power in both single-arm and randomised trials. The average gain in power compared to the traditional analysis is equivalent to approximately a 35% increase in sample size. A modified version of the method is proposed to reduce the computational effort required. We show this modified method maintains much of the efficiency advantages.

A review of statistical designs for improving the efficiency of phase II studies in oncology

Phase II oncology trials are carried out to assess whether an experimental anti-cancer treatment shows sufficient signs of effectiveness to justify being tested in a phase III trial. Traditionally such trials are conducted as single-arm studies using a binary response rate as the primary endpoint. In this article, we review and contrast alternative approaches for such studies. Each approach uses only data that are necessary for the traditional analysis. We consider two broad classes of methods: ones that aim to improve the efficiency using novel design ideas, such as multi-stage and multi-arm multi-stage designs; and ones that aim to improve the analysis, by making better use of the richness of the data that is ignored in the traditional analysis. The former class of methods provides considerable gains in efficiency but also increases the administrative and logistical issues in running the trial. The second class consists of viable alternatives to the standard analysis that come with little additional requirements and provide considerable gains in efficiency.

Clinical Utility of Metrics Based on Tumor Measurements in Phase II Trials to Predict Overall Survival Outcomes in Phase III Trials by Using Resampling Methods

PURPOSE

Phase II clinical trials inform go/no-go decisions for proceeding to phase III trials, and appropriate end points in phase II trials are critical for facilitating this decision. Phase II solid tumor trials have traditionally used end points such as tumor response defined by Response Evaluation Criteria for Solid Tumors (RECIST). We previously reported that absolute and relative changes in tumor measurements demonstrated potential, but not convincing, improvement over RECIST to predict overall survival (OS). We have evaluated the metrics by using additional measures of clinical utility and data from phase III trials.

METHODS

Resampling methods were used to assess the clinical utility of metrics to predict phase III outcomes from simulated phase II trials. In all, 2,000 phase II trials were simulated from four actual phase III trials (two positive for OS and two negative for OS). Cox models for three metrics landmarked at 12 weeks and adjusted for baseline tumor burden were fit for each phase II trial: absolute changes, relative changes, and RECIST. Clinical utility was assessed by positive predictive value and negative predictive value, that is, the probability of a positive or negative phase II trial predicting an effective or ineffective phase III conclusion, by prediction error, and by concordance index (c-index).

RESULTS

Absolute and relative change metrics had higher positive predictive value and negative predictive value than RECIST in five of six treatment comparisons and lower prediction error curves in all six. However, differences were negligible. No statistically significant difference in c-index across metrics was found.

CONCLUSION

The absolute and relative change metrics are not meaningfully better than RECIST in predicting OS.

Evaluating Continuous Tumor Measurement-Based Metrics as Phase II Endpoints for Predicting Overall Survival

BACKGROUND

We sought to develop and validate clinically relevant, early assessment continuous tumor measurement-based metrics for predicting overall survival (OS) using the Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 data warehouse.

METHODS

Data from 13 trials representing 2096 patients with breast cancer, non-small cell lung cancer (NSCLC), or colorectal cancer were used in a complete case analysis. Tumor measurements from weeks 0-6-12 assessments were used to evaluate the ability of slope (absolute change in tumor size from 0-6 and 6-12 weeks) and percent change (relative change in tumor size from 0-6 and 6-12 weeks) metrics to predict OS using Cox models, adjusted for average baseline tumor size. Metrics were evaluated by discrimination (via concordance or c-index), calibration (goodness-of-fit type statistics), association (hazard ratios), and likelihood (Bayesian Information Criteria), with primary focus on the c-index. All statistical tests were two-sided.

RESULTS

Comparison of c-indices suggests slight improvement in predictive ability for the continuous tumor measurement-based metrics vs categorical RECIST response metrics, with slope metrics performing better than percent change metrics for breast cancer and NSCLC. However, these differences were not statistically significant. The goodness-of-fit statistics for the RECIST metrics were as good as or better than those for the continuous metrics. In general, all the metrics performed poorly in breast cancer, compared with NSCLC and colorectal cancer.

CONCLUSION

Absolute and relative change in tumor measurements do not demonstrate convincingly improved overall survival predictive ability over the RECIST model. Continued work is necessary to address issues of missing tumor measurements and model selection in identifying improved tumor measurement-based metrics.

The power of phase II end-points for different possible mechanisms of action of an experimental treatment

BACKGROUND

The high failure rate in phase III oncology trials is partly because the signal obtained from phase II trials is often weak. Several papers have considered the appropriateness of various phase II end-points for individual trials, but there has not been a systematic comparison using simulated data to determine which end-point should be used in which situation.

METHODS

In this paper we carry out simulation studies to compare the power of several Response Evaluation Criteria in Solid Tumours (RECIST) response-based end-points for one-arm and two-arm trials, together with progression-free survival (PFS) and testing the tumour-shrinkage directly for two-arm trials. We consider six scenarios: (1) short-term cytotoxic therapy; (2) continuous cytotoxic therapy; (3+4) cytostatic therapy; (5+6) delayed tumour-shrinkage effect (seen in some immunotherapies). We also consider measurement error in the assessment of tumour size.

RESULTS

Measurement error affects the type-I error rate and power of single-arm trials, and the power of two-arm trials. Generally no single end-point performed well in all scenarios. Best observed response rate, PFS and directly testing the tumour-shrinkages performed best for a number of scenarios. PFS performed very poorly when the effect of the treatment was short-lived. In scenario 6, where the delay in effect was long, no end-point performed well.

CONCLUSIONS

A clinician setting up a phase II trial should consider the likely mechanism of action the drug will have and choose an end-point that provides high power for that scenario. Testing the difference in tumour-shrinkage is often powerful. Alternative end-points are required for therapies with a long delayed effect.

Resampling the N9741 trial to compare tumor dynamic versus conventional end points in randomized phase II trials

PURPOSE

The optimal end point for randomized phase II trials of anticancer therapies remains controversial. We simulated phase II trials by resampling patients from N9741, a randomized phase III trial of chemotherapy regimens for metastatic colorectal cancer, and compared the power of various end points to detect the superior therapy (FOLFOX [infusional fluorouracil, leucovorin, and oxaliplatin] had longer overall survival than both IROX [irinotecan plus oxaliplatin] and IFL [irinotecan and bolus fluorouracil plus leucovorin]).

METHODS

Tumor measurements and progression-free survival (PFS) data were obtained for 1,471 patients; 1,002 had consistently measured tumors and were resampled (5,000 replicates) to simulate two-arm, randomized phase II trials with α = 0.10 (one sided) and 20 to 80 patients per arm. End points included log ratio of tumor size at 6, 12, and 18 weeks relative to baseline; time to tumor growth (TTG), estimated using a nonlinear mixed-effects model; and PFS. Arms were compared using rank sum tests for log ratio and TTG and a log-rank test for PFS.

RESULTS

For FOLFOX versus IFL, TTG and PFS had similar power, with both exceeding the power of log ratio at 18 weeks; for FOLFOX versus IROX, TTG and log ratio at 18 weeks had similar power, with both exceeding the power of PFS. The best end points exhibited > 80% power with 60 to 80 patients per arm.

CONCLUSION

TTG is a powerful end point for randomized phase II trials of cytotoxic therapies in metastatic colorectal cancer; it was either comparable or superior to PFS and log ratio at 18 weeks. Additional studies will be needed to clarify the potential of TTG as a phase II end point.

Using continuous data on tumour measurements to improve inference in phase II cancer studies

In phase II cancer trials, tumour response is either the primary or an important secondary endpoint. Tumour response is a binary composite endpoint determined, according to the Response Evaluation Criteria in Solid Tumors, by (1) whether the percentage change in tumour size is greater than a prescribed threshold and (2) (binary) criteria such as whether a patient develops new lesions. Further binary criteria, such as death or serious toxicity, may be added to these criteria. The probability of tumour response (i.e. 'success' on the composite endpoint) would usually be estimated simply as the proportion of successes among patients. This approach uses the tumour size variable only through a discretised form, namely whether or not it is above the threshold. In this article, we propose a method that also estimates the probability of success but that gains precision by using the information on the undiscretised (i.e. continuous) tumour size variable. This approach can also be used to increase the power to detect a difference between the probabilities of success under two different treatments in a comparative trial. We demonstrate these increases in precision and power using simulated data. We also apply the method to real data from a phase II cancer trial and show that it results in a considerably narrower confidence interval for the probability of tumour response.