A Case Study Examining the Usefulness of Cure Modelling for the Prediction of Survival Based on Data Maturity

Assessing the Performance of Alternative Methods for Estimating Long-Term Survival Benefit of Immuno-oncology Therapies

OBJECTIVES

This study aimed to determine the accuracy and consistency of established methods of extrapolating mean survival for immuno-oncology (IO) therapies, the extent of any systematic biases in estimating long-term clinical benefit, what influences the magnitude of any bias, and the potential implications for health technology assessment.

METHODS

A targeted literature search was conducted to identify published long-term follow-up from clinical trials of immune-checkpoint inhibitors. Earlier published results were identified and Kaplan-Meier estimates for short- and long-term follow-up were digitized and converted to pseudo-individual patient data using an established algorithm. Six standard parametric, 5 flexible parametric, and 2 mixture-cure models (MCMs) were used to extrapolate long-term survival. Mean and restricted mean survival time (RMST) were estimated and compared between short- and long-term follow-up.

RESULTS

Predicted RMST from extrapolation of early data underestimated observed RMST in long-term follow-up for 184 of 271 extrapolations. All models except the MCMs frequently underestimated observed RMST. Mean survival estimates increased with longer follow-up in 196 of 270 extrapolations. The increase exceeded 20% in 122 extrapolations. Log-logistic and log-normal models showed the smallest change with additional follow-up. MCM performance varied substantially with functional form.

CONCLUSIONS

Standard and flexible parametric models frequently underestimate mean survival for IO treatments. Log-logistic and log-normal models may be the most pragmatic and parsimonious solutions for estimating IO mean survival from immature data. Flexible parametric models may be preferred when the data used in health technology assessment are more mature. MCMs fitted to immature data produce unreliable results and are not recommended.

Impact of Extrapolation Model Choices on the Structural Uncertainty in Economic Evaluations for Cancer Immunotherapy: A Case Study of Checkmate 067

OBJECTIVES

The aim of this study was to compare the performance of different extrapolation modeling techniques and analyze their impact on structural uncertainties in the economic evaluations of cancer immunotherapy.

METHODS

The individual patient data was reconstructed through published Checkmate 067 Kaplan Meier curves. Standard parametric models and six flexible techniques were tested, including fractional polynomial, restricted cubic splines, Royston-Parmar models, generalized additive models, parametric mixture models, and mixture cure models. Mean square errors (MSE) and bias from raw survival plots were used to test the model fitness and extrapolation performance. Variability of estimated incremental cost-effectiveness ratios (ICERs) from different models was used to inform the structural uncertainty in economic evaluations. All indicators were analyzed and compared under cut-offs of 3 years and 6.5 years, respectively, to further discuss model impact under different data maturity. R Codes for reproducing this study can be found on GitHub.

RESULTS

The flexible techniques in general performed better than standard parametric models with smaller MSE irrespective of the data maturity. Survival outcomes projected by long-term extrapolation using immature data differed from those with mature data. Although a best-performing model was not found because several models had very similar MSE in this case, the variability of modeled ICERs significantly increased when prolonging simulation cycles.

CONCLUSIONS

Flexible techniques show better performance in the case of Checkmate 067, regardless of data maturity. Model choices affect ICERs of cancer immunotherapy, especially when dealing with immature survival data. When researchers lack evidence to identify the 'right' model, we recommend identifying and revealing the model impacts on structural uncertainty.

A Guide to Selecting Flexible Survival Models to Inform Economic Evaluations of Cancer Immunotherapies

OBJECTIVES

Parametric models are routinely used to estimate the benefit of cancer drugs beyond trial follow-up. The advent of immune checkpoint inhibitors has challenged this paradigm, and emerging evidence suggests that more flexible survival models, which can better capture the shapes of complex hazard functions, might be needed for these interventions. Nevertheless, there is a need for an algorithm to help analysts decide whether flexible models are required and, if so, which should be chosen for testing. This position article has been produced to bridge this gap.

METHODS

A virtual advisory board comprising 7 international experts with in-depth knowledge of survival analysis and health technology assessment was held in summer 2021. The experts discussed 24 questions across 6 topics: the current survival model selection procedure, data maturity, heterogeneity of treatment effect, cure and mortality, external evidence, and additions to existing guidelines. Their responses culminated in an algorithm to inform selection of flexible survival models.

RESULTS

The algorithm consists of 8 steps and 4 questions. Key elements include the systematic identification of relevant external data, using clinical expert input at multiple points in the selection process, considering the future and the observed hazard functions, assessing the potential for long-term survivorship, and presenting results from all plausible models.

CONCLUSIONS

This algorithm provides a systematic, evidence-based approach to justify the selection of survival extrapolation models for cancer immunotherapies. If followed, it should reduce the risk of selecting inappropriate models, partially addressing a key area of uncertainty in the economic evaluation of these agents.

Predicting Survival for Chimeric Antigen Receptor T-Cell Therapy: A Validation of Survival Models Using Follow-Up Data From ZUMA-1

OBJECTIVES

Survival extrapolation for chimeric antigen receptor T-cell therapies is challenging, owing to their unique mechanistic properties that translate to complex hazard functions. Axicabtagene ciloleucel is indicated for the treatment of relapse or refractory diffuse large B-cell lymphoma after 2 or more lines of therapy based on the ZUMA-1 trial. Four data snapshots are available, with minimum follow-up of 12, 24, 36, and 48 months. This analysis explores how survival extrapolations for axicabtagene ciloleucel using ZUMA-1 data can be validated and compared.

METHODS

Three different parametric modeling approaches were applied: standard parametric, spline-based, and cure-based models. Models were compared using a range of metrics, across the 4 data snapshot, including visual fit, plausibility of long-term estimates, statistical goodness of fit, inspection of hazard plots, point-estimate accuracy, and conditional survival estimates.

RESULTS

Standard and spline-based parametric extrapolations were generally incapable of fitting the ZUMA-1 data well. Cure-based models provided the best fit based on the earliest data snapshot, with extrapolations remaining consistent as data matured. At 48 months, the maximum survival overestimate was 8.3% (Gompertz mixture-cure model) versus the maximum underestimate of 33.5% (Weibull standard parametric model).

CONCLUSIONS

Where a plateau in the survival curve is clinically plausible, cure-based models may be helpful in making accurate predictions based on immature data. The ability to reliably extrapolate from maturing data may reduce delays in patient access to potentially lifesaving treatments. Additional research is required to understand how models compare in broader contexts, including different treatments and therapeutic areas.

Methods for Extrapolating Survival Analyses for the Economic Evaluation of Advanced Therapy Medicinal Products

There are two significant challenges for analysts conducting economic evaluations of advanced therapy medicinal products (ATMPs): (i) estimating long-term treatment effects in the absence of mature clinical data, and (ii) capturing potentially complex hazard functions. This review identifies and critiques a variety of methods that can be used to overcome these challenges. The narrative review is informed by a rapid literature review of methods used for the extrapolation of survival analyses in the economic evaluation of ATMPs. There are several methods that are more suitable than traditional parametric survival modelling approaches for capturing complex hazard functions, including, cure-mixture models and restricted cubic spline models. In the absence of mature clinical data, analysts may augment clinical trial data with data from other sources to aid extrapolation, however, the relative merits of employing methods for including data from different sources is not well understood. Given the high and potentially irrecoverable costs of making incorrect decisions concerning the reimbursement or commissioning of ATMPs, it is important that economic evaluations are correctly specified, and that both parameter and structural uncertainty associated with survival extrapolations are considered. Value of information analyses allow for this uncertainty to be expressed explicitly, and in monetary terms.

How accurate are the longer-term projections of overall survival for cancer immunotherapy for standard versus more flexible parametric extrapolation methods?

AIMS

To assess the accuracy of standard parametric survival models, spline models, and mixture cure models (MCMs) fitted to overall survival (OS) data available at the time of submission in the NICE HTA process compared with data subsequently made available.

METHODS

Standard parametric distributions, spline models, and MCMs were fitted to OS data presented in single technology appraisals (TAs) for immune-checkpoint inhibitors (ICIs) in cancer. For each TA, the estimated survival from the fitted models was compared with Kaplan-Meier (KM) data that were made available following the HTA submission using differences between point estimates and restricted area under the curve (AUC) at both the midpoint and the end of additional follow-up. Differences in interval AUC values (calculated for each 6-month period) were also assessed.

RESULTS

Standard parametric survival models and spline models were more likely to underestimate longer-term survival, irrespective of the measure used to assess model accuracy. MCMs were more likely to overestimate survival; however, this was improved in some cases by applying an additional hazard of mortality for "statistically cured" patients.

LIMITATIONS

The accuracy of the models was assessed based on much shorter OS data than the period for which extrapolation is needed, which may impact conclusions regarding the most accurate models. The most recent TAs for ICIs have not been captured.

CONCLUSIONS

There are no definitive findings that unquestionably support the use of one specific extrapolation technique. Rather, each has the potential to provide accurate or inaccurate extrapolation to longer-term data in certain circumstances, but the added flexibility of more complex models can be justified for treatments, like ICIs, that have extended survival for patients across disease areas. The use of mortality adjustments for "statistically cured" patients allows decision-makers to explore more conservative scenarios in the face of high decision uncertainty.

Can the Risk of Postoperative Cerebrospinal Fluid Leakage Be Predicted for Patients Undergoing Cervical Spine Surgery? Development and Evaluation of a New Predictive Nomogram

OBJECTIVE

Previous studies have retrospectively analyzed the likely causes of cerebrospinal fluid leakage (CSFL) during cervical spine surgery and the management of CSFL after its occurrence. In the present study, we aimed to develop and validate a nomogram for the risk of CSFL in Chinese patients who had undergone cervical decompression and internal fixation (CDIF) surgery.

METHODS

We performed a retrospective analysis of patients who had undergone CDIF surgery. Of the 1286 included patients, 54 were in the CSFL group and 1232 were in the normal group. The patients were randomly divided into training and validation tests. The risk assessment for CSFL included 21 characteristics. The feature selection for the CSFL model was optimized using the least absolute shrinkage and selection operator regression model in the training test. Multivariate logistic regression analysis was performed to construct the model according to the selected characteristics. The clinical usefulness of the predictive model was assessed using the C-index, calibration curve, and decision curve analysis with identification and calibration.

RESULTS

The risk prediction nomogram included the diagnosis, revision surgery, ossification of the posterior longitudinal ligament, cervical instability, and a history of malignancy in the training test. The model demonstrated high predictive power, with a C-index of 0.914 (95% confidence interval, 0.876-0.951) and an area under the curve of 0.914. The results of the decision curve analysis demonstrated the clinical usefulness of the CSFL risk nomogram when the probability threshold for CSFL was 1%-62%.

CONCLUSIONS

Our proposed nomogram for CSFL risk includes the diagnosis, revision surgery, ossification of the posterior longitudinal ligament, cervical instability, and a history of malignancy. The nomogram can be used to evaluate the risk of CSFL for patients undergoing CDIF surgery.

The Extrapolation Performance of Survival Models for Data With a Cure Fraction: A Simulation Study

OBJECTIVES

Curative treatments can result in complex hazard functions. The use of standard survival models may result in poor extrapolations. Several models for data which may have a cure fraction are available, but comparisons of their extrapolation performance are lacking. A simulation study was performed to assess the performance of models with and without a cure fraction when fit to data with a cure fraction.

METHODS

Data were simulated from a Weibull cure model, with 9 scenarios corresponding to different lengths of follow-up and sample sizes. Cure and noncure versions of standard parametric, Royston-Parmar, and dynamic survival models were considered along with noncure fractional polynomial and generalized additive models. The mean-squared error and bias in estimates of the hazard function were estimated.

RESULTS

With the shortest follow-up, none of the cure models provided good extrapolations. Performance improved with increasing follow-up, except for the misspecified standard parametric cure model (lognormal). The performance of the flexible cure models was similar to that of the correctly specified cure model. Accurate estimates of the cured fraction were not necessary for accurate hazard estimates. Models without a cure fraction provided markedly worse extrapolations.

CONCLUSIONS

For curative treatments, failure to model the cured fraction can lead to very poor extrapolations. Cure models provide improved extrapolations, but with immature data there may be insufficient evidence to choose between cure and noncure models, emphasizing the importance of clinical knowledge for model choice. Dynamic cure fraction models were robust to model misspecification, but standard parametric cure models were not.

Applying State-of-the-Art Survival Extrapolation Techniques to the Evaluation of CAR-T Therapies: Evidence from a Systematic Literature Review

INTRODUCTION

Traditional statistical techniques for extrapolating short-term survival data for anticancer therapies assume the same mortality rate for noncured and "cured" patients, which is appropriate for projecting survival of non-curative therapies but may lead to an underestimation of the treatment effectiveness for potentially curative therapies. Our objective was to ascertain research trends in survival extrapolation techniques used to project the survival benefits of chimeric antigen receptor T cell (CAR-T) therapies.

METHODS

A global systematic literature search produced a review of survival analyses of CAR-T therapies, published between January 1, 2015 and December 14, 2020, based on publications sourced from MEDLINE, scientific conferences, and health technology assessment agencies. Trends in survival extrapolation techniques used, and the rationale for selecting advanced techniques, are discussed.

RESULTS

Twenty publications were included, the majority of which (65%, N = 13) accounted for curative intent of CAR-T therapies through the use of advanced extrapolation techniques, i.e., mixture cure models [MCMs] (N = 10) or spline-based models (N = 3). The authors' rationale for using the MCM approach included (a) better statistical fits to the observed Kaplan-Meier curves (KMs) and (b) visual inspection of the KMs indicated that a proportion of patients experienced long-term remission and survival which is not inherently captured in standard parametric distributions.

DISCUSSION

Our findings suggest that an advanced extrapolation technique should be considered in base case survival analyses of CAR-T therapies when extrapolating short-term survival data to long-term horizons extending beyond the clinical trial duration.

CONCLUSION

Advanced extrapolation techniques allow researchers to account for the proportion of patients with an observed plateau in survival from clinical trial data; by only using standard-partitioned modeling, researchers may risk underestimating the survival benefits for the subset of patients with long-term remission. Sensitivity analysis with an alternative advanced extrapolation technique should be implemented and re-assessment using clinical trial extension data and/or real-world data should be conducted as longer-term data become available.

Mixture Cure Models in Oncology: A Tutorial and Practical Guidance

Novel cancer therapies are associated with survival patterns that differ from established therapies, which may include survival curves that plateau after a certain follow-up time point. A fraction of the patient population is then considered statistically cured and subject to the same mortality experience as the cancer-free general population. Mixture cure models have been developed to account for this characteristic. As compared to standard survival analysis, mixture cure models can often lead to profoundly different estimates of long-term survival, required for health economic evaluations. This tutorial is designed as a practical introduction to mixture cure models. Step-by-step instructions are provided for the entire implementation workflow, i.e., from gathering and combining data from different sources to fitting models using maximum likelihood estimation and model results interpretation. Two mixture cure models were developed to illustrate (1) an "uninformed" approach where the cure fraction is estimated from trial data and (2) an "informed" approach where the cure fraction is obtained from an external source (e.g., real-world data) used as an input to the model. These models were implemented in the statistical software R, with the freely available code on GitHub. The cure fraction can be estimated as an output from ("uninformed" approach) or used as an input to ("informed" approach) a mixture cure model. Mixture cure models suggest presumed estimates of long-term survival proportions, especially in instances where some fraction of patients is expected to be statistically cured. While this type of model may initially seem complex, it is straightforward to use and interpret. Mixture cure models have the potential to improve the accuracy of survival estimates for treatments associated with statistical cure, and the present tutorial outlines the interpretation and implementation of mixture cure models in R. This type of model will likely become more widely used in health economic analyses as novel cancer therapies enter the market.

A Systematic Review of the Efforts and Hindrances of Modeling and Simulation of CAR T-cell Therapy

Chimeric antigen receptor (CAR) T-cell therapy is an immunotherapy that has recently become highly instrumental in the fight against life-threatening diseases. A variety of modeling and computational simulation efforts have addressed different aspects of CAR T-cell therapy, including T-cell activation, T- and malignant cell population dynamics, therapeutic cost-effectiveness strategies, and patient survival. In this article, we present a systematic review of those efforts, including mathematical, statistical, and stochastic models employing a wide range of algorithms, from differential equations to machine learning. To the best of our knowledge, this is the first review of all such models studying CAR T-cell therapy. In this review, we provide a detailed summary of the strengths, limitations, methodology, data used, and data gap in currently published models. This information may help in designing and building better models for enhanced prediction and assessment of the benefit-risk balance associated with novel CAR T-cell therapies, as well as with the data need for building such models.

R and Shiny for Cost-Effectiveness Analyses: Why and When? A Hypothetical Case Study

INTRODUCTION

Health economics models are typically built in Microsoft Excel^® owing to its wide familiarity, accessibility and perceived transparency. However, given the increasingly rapid and analytically complex decision-making needs of both the pharmaceutical industry and the field of health economics and outcomes research (HEOR), the demands of cost-effectiveness analyses may be better met by the programming language R.

OBJECTIVE

This case study provides an explicit comparison between Excel and R for contemporary cost-effectiveness analysis.

METHODS

We constructed duplicate cost-effectiveness models using Excel and R (with a user interface built using the Shiny package) to address a hypothetical case study typical of contemporary health technology assessment.

RESULTS

We compared R and Excel versions of the same model design to determine the advantages and limitations of the modelling platforms in terms of (i) analytical capability, (ii) data safety, (iii) building considerations, (iv) usability for technical and non-technical users and (v) model adaptability.

CONCLUSIONS

The findings of this explicit comparison are used to produce recommendations for when R might be more suitable than Excel in contemporary cost-effectiveness analyses. We conclude that selection of appropriate modelling software needs to consider case-by-case modelling requirements, particularly (i) intended audience, (ii) complexity of analysis, (iii) nature and frequency of updates and (iv) anticipated model run time.