Extrapolation of Survival Curves Using Standard Parametric Models and Flexible Parametric Spline Models: Comparisons in Large Registry Cohorts with Advanced Cancer

Comparing Survival Extrapolation within All-Cause and Relative Survival Frameworks by Standard Parametric Models and Flexible Parametric Spline Models Using the Swedish Cancer Registry

BACKGROUND

In health technology assessment, restricted mean survival time and life expectancy are commonly evaluated. Parametric models are typically used for extrapolation. Spline models using a relative survival framework have been shown to estimate life expectancy of cancer patients more reliably; however, more research is needed to assess spline models using an all-cause survival framework and standard parametric models using a relative survival framework.

AIM

To assess survival extrapolation using standard parametric models and spline models within relative survival and all-cause survival frameworks.

METHODS

From the Swedish Cancer Registry, we identified patients diagnosed with 5 types of cancer (colon, breast, melanoma, prostate, and chronic myeloid leukemia) between 1981 and 1990 with follow-up until 2020. Patients were categorized into 15 cancer cohorts by cancer and age group (18-59, 60-69, and 70-99 y). We right-censored the follow-up at 2, 3, 5, and 10 y and fitted the parametric models within an all-cause and a relative survival framework to extrapolate to 10 y and lifetime in comparison with the observed Kaplan-Meier survival estimates. All cohorts were modeled with 6 standard parametric models (exponential, Weibull, Gompertz, log-logistic, log-normal, and generalized gamma) and 3 spline models (on hazard, odds, and normal scales).

RESULTS

For predicting 10-y survival, spline models generally performed better than standard parametric models. However, using an all-cause or a relative survival framework did not show any distinct difference. For lifetime survival, extrapolating from a relative survival framework agreed better with the observed survival, particularly using spline models.

CONCLUSIONS

For extrapolation to 10 y, we recommend spline models. For extrapolation to lifetime, we suggest extrapolating in a relative survival framework, especially using spline models.

HIGHLIGHTS

For survival extrapolation to 10 y, spline models generally performed better than standard parametric models did. However, using an all-cause or a relative survival framework showed no distinct difference under the same parametric model.Survival extrapolation to lifetime within a relative survival framework agreed well with the observed data, especially using spline models.Extrapolating parametric models within an all-cause survival framework may overestimate survival proportions at lifetime; models for the relative survival approach may underestimate instead.

Assessing the Performance of Alternative Methods for Estimating Long-Term Survival Benefit of Immuno-oncology Therapies

OBJECTIVES

This study aimed to determine the accuracy and consistency of established methods of extrapolating mean survival for immuno-oncology (IO) therapies, the extent of any systematic biases in estimating long-term clinical benefit, what influences the magnitude of any bias, and the potential implications for health technology assessment.

METHODS

A targeted literature search was conducted to identify published long-term follow-up from clinical trials of immune-checkpoint inhibitors. Earlier published results were identified and Kaplan-Meier estimates for short- and long-term follow-up were digitized and converted to pseudo-individual patient data using an established algorithm. Six standard parametric, 5 flexible parametric, and 2 mixture-cure models (MCMs) were used to extrapolate long-term survival. Mean and restricted mean survival time (RMST) were estimated and compared between short- and long-term follow-up.

RESULTS

Predicted RMST from extrapolation of early data underestimated observed RMST in long-term follow-up for 184 of 271 extrapolations. All models except the MCMs frequently underestimated observed RMST. Mean survival estimates increased with longer follow-up in 196 of 270 extrapolations. The increase exceeded 20% in 122 extrapolations. Log-logistic and log-normal models showed the smallest change with additional follow-up. MCM performance varied substantially with functional form.

CONCLUSIONS

Standard and flexible parametric models frequently underestimate mean survival for IO treatments. Log-logistic and log-normal models may be the most pragmatic and parsimonious solutions for estimating IO mean survival from immature data. Flexible parametric models may be preferred when the data used in health technology assessment are more mature. MCMs fitted to immature data produce unreliable results and are not recommended.

Quantifying the impact of novel metastatic cancer therapies on health inequalities in survival outcomes

Background: Novel therapies in metastatic cancers have contributed to improvements in survival outcomes, yet real-world data suggest that improvements may be mainly driven by those patient groups who already had the highest survival outcomes. This study aimed to develop and apply a framework for quantifying the impact of novel metastatic cancer therapies on health inequalities in survival outcomes based on published aggregate data. Methods: Nine (N = 9) novel therapies for metastatic breast cancer (mBC), metastatic colorectal cancer (mCRC), and metastatic non-small cell lung cancer (mNSCLC) were identified, 3 for each cancer type. Individual patient data (IPD) for overall survival (OS) and progression-free survival (PFS) were replicated from published Kaplan-Meier (KM) curves. For each cancer type, data were pooled for the novel therapies and comparators separately and weighted based on sample size to ensure equal contribution of each therapy in the analyses. Parametric (mixture) distributions were fitted to the weighted data to model and extrapolate survival. The inequality in survival was defined by the absolute difference between groups with the highest and lowest survival for 2 stratifications: one for which survival was stratified into 2 groups and one using 5 groups. Additionally, a linear regression model was fitted to survival estimates for the 5 groups, with the regression coefficient or slope considered as the inequality gradient (IG). The impact of the pooled novel therapies was subsequently defined as the change in survival inequality relative to the pooled comparator therapies. A probabilistic analysis was performed to quantify parameter uncertainty. Results: The analyses found that novel therapies were associated with significant increases in inequalities in survival outcomes relative to their comparators, except in terms of OS for mNSCLC. For mBC, the inequalities in OS increased by 13.9 (95% CI: 1.4; 26.6) months, or 25.0%, if OS was stratified in 5 groups. The IG for mBC increased by 3.2 (0.3; 6.1) months, or 24.7%. For mCRC, inequalities increased by 6.7 (3.0; 10.5) months, or 40.4%, for stratification based on 5 groups; the IG increased by 1.6 (0.7; 2.4) months, or 40.2%. For mNSCLC, inequalities decreased by 14.9 (-84.5; 19.0) months, or 12.2%, for the 5-group stratification; the IG decreased by 2.0 (-16.1; 5.1) months, or 5.5%. Results for the stratification based on 2 groups demonstrated significant increases in OS inequality for all cancer types. In terms of PFS, the increases in survival inequalities were larger in a relative sense compared with OS. For mBC, PFS inequalities increased by 8.7 (5.9; 11.6) months, or 71.7%, for stratification based on 5 groups; the IG increased by 2.0 (1.3; 2.6) months, or 67.6%. For mCRC, PFS inequalities increased by 5.4 (4.2; 6.6) months, or 147.6%, for the same stratification. The IG increased by 1.3 (1.1; 1.6) months, or 172.7%. For mNSCLC, inequalities increased by 18.2 (12.5; 24.4) months, or 93.8%, for the 5-group stratification; the IG increased by 4.0 (2.8; 5.4) months, or 88.1%. Results from the stratification based on 2 groups were similar. Conclusion: Novel therapies for mBC, mCRC, and mNSCLC are generally associated with significant increases in survival inequalities relative to their comparators in randomized controlled trials, though inequalities in OS for mNSCLC decreased nonsignificantly when stratified based on 5 groups. Although further research using real-world IPD is warranted to assess how, for example, social determinants of health affect the impact of therapies on health inequalities among patient groups, the proposed framework can provide important insights in the absence of such data.

Extrapolating empirical long-term survival data: the impact of updated follow-up data and parametric extrapolation methods on survival estimates in multiple myeloma

BACKGROUND

In economic evaluations, survival is often extrapolated to smooth out the Kaplan-Meier estimate and because the available data (e.g., from randomized controlled trials) are often right censored. Validation of the accuracy of extrapolated results can depend on the length of follow-up and the assumptions made about the survival hazard. Here, we analyze the accuracy of different extrapolation techniques while varying the data cut-off to estimate long-term survival in newly diagnosed multiple myeloma (MM) patients.

METHODS

Empirical data were available from a randomized controlled trial and a registry for MM patients treated with melphalan + prednisone, thalidomide, and bortezomib- based regimens. Standard parametric and spline models were fitted while artificially reducing follow-up by introducing database locks. The maximum follow-up for these locks varied from 3 to 13 years. Extrapolated (conditional) restricted mean survival time (RMST) was compared to the Kaplan-Meier RMST and models were selected according to statistical tests, and visual fit.

RESULTS

For all treatments, the RMST error decreased when follow-up and the absolute number of events increased, and censoring decreased. The decline in RMST error was highest when maximum follow-up exceeded six years. However, even when censoring is low there can still be considerable deviations in the extrapolated RMST conditional on survival until extrapolation when compared to the KM-estimate.

CONCLUSIONS

We demonstrate that both standard parametric and spline models could be worthy candidates when extrapolating survival for the populations examined. Nevertheless, researchers and decision makers should be wary of uncertainty in results even when censoring has decreased, and the number of events has increased.

Impact of Extrapolation Model Choices on the Structural Uncertainty in Economic Evaluations for Cancer Immunotherapy: A Case Study of Checkmate 067

OBJECTIVES

The aim of this study was to compare the performance of different extrapolation modeling techniques and analyze their impact on structural uncertainties in the economic evaluations of cancer immunotherapy.

METHODS

The individual patient data was reconstructed through published Checkmate 067 Kaplan Meier curves. Standard parametric models and six flexible techniques were tested, including fractional polynomial, restricted cubic splines, Royston-Parmar models, generalized additive models, parametric mixture models, and mixture cure models. Mean square errors (MSE) and bias from raw survival plots were used to test the model fitness and extrapolation performance. Variability of estimated incremental cost-effectiveness ratios (ICERs) from different models was used to inform the structural uncertainty in economic evaluations. All indicators were analyzed and compared under cut-offs of 3 years and 6.5 years, respectively, to further discuss model impact under different data maturity. R Codes for reproducing this study can be found on GitHub.

RESULTS

The flexible techniques in general performed better than standard parametric models with smaller MSE irrespective of the data maturity. Survival outcomes projected by long-term extrapolation using immature data differed from those with mature data. Although a best-performing model was not found because several models had very similar MSE in this case, the variability of modeled ICERs significantly increased when prolonging simulation cycles.

CONCLUSIONS

Flexible techniques show better performance in the case of Checkmate 067, regardless of data maturity. Model choices affect ICERs of cancer immunotherapy, especially when dealing with immature survival data. When researchers lack evidence to identify the 'right' model, we recommend identifying and revealing the model impacts on structural uncertainty.

A Guide to Selecting Flexible Survival Models to Inform Economic Evaluations of Cancer Immunotherapies

OBJECTIVES

Parametric models are routinely used to estimate the benefit of cancer drugs beyond trial follow-up. The advent of immune checkpoint inhibitors has challenged this paradigm, and emerging evidence suggests that more flexible survival models, which can better capture the shapes of complex hazard functions, might be needed for these interventions. Nevertheless, there is a need for an algorithm to help analysts decide whether flexible models are required and, if so, which should be chosen for testing. This position article has been produced to bridge this gap.

METHODS

A virtual advisory board comprising 7 international experts with in-depth knowledge of survival analysis and health technology assessment was held in summer 2021. The experts discussed 24 questions across 6 topics: the current survival model selection procedure, data maturity, heterogeneity of treatment effect, cure and mortality, external evidence, and additions to existing guidelines. Their responses culminated in an algorithm to inform selection of flexible survival models.

RESULTS

The algorithm consists of 8 steps and 4 questions. Key elements include the systematic identification of relevant external data, using clinical expert input at multiple points in the selection process, considering the future and the observed hazard functions, assessing the potential for long-term survivorship, and presenting results from all plausible models.

CONCLUSIONS

This algorithm provides a systematic, evidence-based approach to justify the selection of survival extrapolation models for cancer immunotherapies. If followed, it should reduce the risk of selecting inappropriate models, partially addressing a key area of uncertainty in the economic evaluation of these agents.

Comparison of statistical methods for extrapolating survival in previously untreated diffuse large B-cell lymphoma: results based on the POLARIX study

OBJECTIVE

The ongoing Phase III randomized POLARIX study (GO39942; NCT03274492) demonstrated significantly improved progression-free survival (PFS) with polatuzumab vedotin plus rituximab, cyclophosphamide, doxorubicin and prednisone (Pola-R-CHP) versus rituximab plus cyclophosphamide, doxorubicin, vincristine, and prednisone (R-CHOP) in patients with previously untreated diffuse large B-cell lymphoma (DLBCL). We compared statistical methodologies to extrapolate long-term PFS data from POLARIX.

MATERIALS AND METHODS

This analysis explored four different approaches to extrapolate the POLARIX data: standard parametric survival, mixture-cure, landmark, and spline models. The resulting extrapolation curves were validated via comparison with the corresponding Kaplan-Meier (KM) curves from POLARIX and the POLARIX-like population of the Phase III GOYA study (NCT01287741; R-CHOP arm).

RESULTS

The R-CHOP PFS KM curve from the GOYA validation set was well aligned with the POLARIX KM curve. As we anticipated that PFS in POLARIX would evolve similarly to that of GOYA, the data from GOYA were used to externally validate the extrapolated modelling results. While all four statistical methods were able to fit the data to the POLARIX KM curve, the mixture-cure model was the most accurate in predicting long-term PFS in the GOYA external validation set. In the mixture-cure model, generalized gamma distribution estimated 64% (95% confidence intervals [CI]: 56-71%) of patients to have long-term remission in the R-CHOP arm of POLARIX and GOYA, and 75% (95% CI: 70-79%) in the Pola-R-CHP arm of POLARIX. A limitation of this study was the comparison of the statistical models only in the PFS KM curves, since it was not possible to determine which statistical method was more appropriate to extrapolate the overall survival KM curves.

CONCLUSIONS

Within this analysis, the mixture-cure model provided the best prediction of long-term outcomes from the primary PFS analysis of the POLARIX study.

Use of Advanced Flexible Modeling Approaches for Survival Extrapolation from Early Follow-up Data in two Nivolumab Trials in Advanced NSCLC with Extended Follow-up

OBJECTIVES

Immuno-oncology (IO) therapies are often associated with delayed responses that are deep and durable, manifesting as long-term survival benefits in patients with metastatic cancer. Complex hazard functions arising from IO treatments may limit the accuracy of extrapolations from standard parametric models (SPMs). We evaluated the ability of flexible parametric models (FPMs) to improve survival extrapolations using data from 2 trials involving patients with non-small-cell lung cancer (NSCLC).

METHODS

Our analyses used consecutive database locks (DBLs) at 2-, 3-, and 5-y minimum follow-up from trials evaluating nivolumab versus docetaxel in patients with pretreated metastatic squamous (CheckMate-017) and nonsquamous (CheckMate-057) NSCLC. For each DBL, SPMs, as well as 3 FPMs-landmark response models (LRMs), mixture cure models (MCMs), and Bayesian multiparameter evidence synthesis (B-MPES)-were estimated on nivolumab overall survival (OS). The performance of each parametric model was assessed by comparing milestone restricted mean survival times (RMSTs) and survival probabilities with results obtained from externally validated SPMs.

RESULTS

For the 2- and 3-y DBLs of both trials, all models tended to underestimate 5-y OS. Predictions from nonvalidated SPMs fitted to the 2-y DBLs were highly unreliable, whereas extrapolations from FPMs were much more consistent between models fitted to successive DBLs. For CheckMate-017, in which an apparent survival plateau emerges in the 3-y DBL, MCMs fitted to this DBL estimated 5-y OS most accurately (11.6% v. 12.3% observed), and long-term predictions were similar to those from the 5-y validated SPM (20-y RMST: 30.2 v. 30.5 mo). For CheckMate-057, where there is no clear evidence of a survival plateau in the early DBLs, only B-MPES was able to accurately predict 5-y OS (14.1% v. 14.0% observed [3-y DBL]).

CONCLUSIONS

We demonstrate that the use of FPMs for modeling OS in NSCLC patients from early follow-up data can yield accurate estimates for RMST observed with longer follow-up and provide similar long-term extrapolations to externally validated SPMs based on later data cuts. B-MPES generated reasonable predictions even when fitted to the 2-y DBLs of the studies, whereas MCMs were more reliant on longer-term data to estimate a plateau and therefore performed better from 3 y. Generally, LRM extrapolations were less reliable than those from alternative FPMs and validated SPMs but remained superior to nonvalidated SPMs. Our work demonstrates the potential benefits of using advanced parametric models that incorporate external data sources, such as B-MPES and MCMs, to allow for accurate evaluation of treatment clinical and cost-effectiveness from trial data with limited follow-up.

HIGHLIGHTS

Flexible advanced parametric modeling methods can provide improved survival extrapolations for immuno-oncology cost-effectiveness in health technology assessments from early clinical trial data that better anticipate extended follow-up.Advantages include leveraging additional observable trial data, the systematic integration of external data, and more detailed modeling of underlying processes.Bayesian multiparameter evidence synthesis performed particularly well, with well-matched external data.Mixture cure models also performed well but may require relatively longer follow-up to identify an emergent plateau, depending on the specific setting.Landmark response models offered marginal benefits in this scenario and may require greater numbers in each response group and/or increased follow-up to support improved extrapolation within each subgroup.

How accurate are the longer-term projections of overall survival for cancer immunotherapy for standard versus more flexible parametric extrapolation methods?

AIMS

To assess the accuracy of standard parametric survival models, spline models, and mixture cure models (MCMs) fitted to overall survival (OS) data available at the time of submission in the NICE HTA process compared with data subsequently made available.

METHODS

Standard parametric distributions, spline models, and MCMs were fitted to OS data presented in single technology appraisals (TAs) for immune-checkpoint inhibitors (ICIs) in cancer. For each TA, the estimated survival from the fitted models was compared with Kaplan-Meier (KM) data that were made available following the HTA submission using differences between point estimates and restricted area under the curve (AUC) at both the midpoint and the end of additional follow-up. Differences in interval AUC values (calculated for each 6-month period) were also assessed.

RESULTS

Standard parametric survival models and spline models were more likely to underestimate longer-term survival, irrespective of the measure used to assess model accuracy. MCMs were more likely to overestimate survival; however, this was improved in some cases by applying an additional hazard of mortality for "statistically cured" patients.

LIMITATIONS

The accuracy of the models was assessed based on much shorter OS data than the period for which extrapolation is needed, which may impact conclusions regarding the most accurate models. The most recent TAs for ICIs have not been captured.

CONCLUSIONS

There are no definitive findings that unquestionably support the use of one specific extrapolation technique. Rather, each has the potential to provide accurate or inaccurate extrapolation to longer-term data in certain circumstances, but the added flexibility of more complex models can be justified for treatments, like ICIs, that have extended survival for patients across disease areas. The use of mortality adjustments for "statistically cured" patients allows decision-makers to explore more conservative scenarios in the face of high decision uncertainty.

Biased Survival Predictions When Appraising Health Technologies in Heterogeneous Populations

INTRODUCTION

Time-to-event data from clinical trials are routinely extrapolated using parametric models to estimate the cost effectiveness of novel therapies, but how this approach performs in the presence of heterogeneous populations remains unknown.

METHODS

We performed a simulation study of seven scenarios with varying exponential distributions modelling treatment and prognostic effects across subgroup and complement populations, with follow-up typical of clinical trials used to appraise the cost effectiveness of therapies by agencies such as the UK National Institute for Health and Care Excellence (NICE). We compared established and emerging methods of estimating population life-years (LYs) using parametric models. We also proved analytically that an exponential model fitted to censored heterogeneous survival times sampled from two distinct exponential distributions will produce a biased estimate of the hazard rate and LYs.

RESULTS

LYs are underestimated by the methods in the presence of heterogeneity, resulting in either under- or overestimation of the incremental benefit. In scenarios where the overestimation of benefit is likely, which is of interest to the healthcare provider, the method of taking the average LYs from all plausible models has the least bias. LY estimates from complete Kaplan-Meier curves have high variation, suggesting mature data may not be a reliable solution. We explore the effect of increasing trial sample size and accounting for detected treatment-subgroup interactions.

CONCLUSIONS

The bias associated with heterogeneous populations suggests that NICE may need to be more cautious when appraising therapies and to consider model averaging or the separate modelling of subgroups when heterogeneity is suspected or detected.

Finite Mixture Models, a Flexible Alternative to Standard Modeling Techniques for Extrapolated Mean Survival Times Needed for Cost-Effectiveness Analyses

OBJECTIVES

To compare finite mixture models with common survival models with respect to how well they fit heterogenous data used to estimate mean survival times required for cost-effectiveness analysis.

METHODS

Publicly available overall survival (OS) and progression-free survival (PFS) curves were digitized to produce nonproprietary data. Regression models based on the following distributions were fit to the data: Weibull, lognormal, log-logistic, generalized F, generalized gamma, Gompertz, mixture of 2 Weibulls, and mixture of 3 Weibulls. A second set of analyses was performed based on data in which patients who had not experienced an event by 30 months were censored. Model performance was compared based on the Akaike information criterion (AIC).

RESULTS

For PFS, the 3-Weibull mixture (AIC = 479.94) and 2-Weibull mixture (AIC = 488.24) models outperformed other models by more than 40 points and produced the most accurate estimates of mean survival times. For OS, the AIC values for all models were similar (all within 4 points). The means for the mixture 3-Weibulls mixture model (17.60 months) and the 2-Weibull mixture model (17.59 months) were the closest to the Kaplan-Meier mean estimate of (17.58 months). The results and conclusions from the censored analysis of PFS were similar to the uncensored PFS analysis. On the basis of extrapolated mean OS, all models produced estimates within 10% of the Kaplan-Meier mean survival time.

CONCLUSIONS

Finite mixture models offer a flexible modeling approach that has benefits over standard parametric models when analyzing heterogenous data for estimating survival times needed for cost-effectiveness analysis.

Applying State-of-the-Art Survival Extrapolation Techniques to the Evaluation of CAR-T Therapies: Evidence from a Systematic Literature Review

INTRODUCTION

Traditional statistical techniques for extrapolating short-term survival data for anticancer therapies assume the same mortality rate for noncured and "cured" patients, which is appropriate for projecting survival of non-curative therapies but may lead to an underestimation of the treatment effectiveness for potentially curative therapies. Our objective was to ascertain research trends in survival extrapolation techniques used to project the survival benefits of chimeric antigen receptor T cell (CAR-T) therapies.

METHODS

A global systematic literature search produced a review of survival analyses of CAR-T therapies, published between January 1, 2015 and December 14, 2020, based on publications sourced from MEDLINE, scientific conferences, and health technology assessment agencies. Trends in survival extrapolation techniques used, and the rationale for selecting advanced techniques, are discussed.

RESULTS

Twenty publications were included, the majority of which (65%, N = 13) accounted for curative intent of CAR-T therapies through the use of advanced extrapolation techniques, i.e., mixture cure models [MCMs] (N = 10) or spline-based models (N = 3). The authors' rationale for using the MCM approach included (a) better statistical fits to the observed Kaplan-Meier curves (KMs) and (b) visual inspection of the KMs indicated that a proportion of patients experienced long-term remission and survival which is not inherently captured in standard parametric distributions.

DISCUSSION

Our findings suggest that an advanced extrapolation technique should be considered in base case survival analyses of CAR-T therapies when extrapolating short-term survival data to long-term horizons extending beyond the clinical trial duration.

CONCLUSION

Advanced extrapolation techniques allow researchers to account for the proportion of patients with an observed plateau in survival from clinical trial data; by only using standard-partitioned modeling, researchers may risk underestimating the survival benefits for the subset of patients with long-term remission. Sensitivity analysis with an alternative advanced extrapolation technique should be implemented and re-assessment using clinical trial extension data and/or real-world data should be conducted as longer-term data become available.