1
|
Gallacher D. SurvInt: a simple tool to obtain precise parametric survival extrapolations. BMC Med Inform Decis Mak 2024; 24:76. [PMID: 38486175 PMCID: PMC10938652 DOI: 10.1186/s12911-024-02475-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Accepted: 03/04/2024] [Indexed: 03/18/2024] Open
Abstract
BACKGROUND Economic evaluation of emerging health technologies is mandated by agencies such as the National Institute for Health and Care Excellence (NICE) to ensure their cost is proportional to their benefit. To avoid bias, NICE stipulate that the benefit of a treatment is assessed across the lifetime of the patient population, which can be many decades. Unfortunately, follow-up from a clinical trial will not usually cover the required period and the observed follow-up will require extrapolation. For survival data this is often done by selecting a preferred model from a set of candidate parametric models. This approach is limited in that the choice of model is restricted to those originally fitted. What if none of the models are consistent with clinical prediction or external data? METHOD/RESULTS This paper introduces SurvInt, a tool that estimates the parameters of common parametric survival models which interpolate key survival time co-ordinates specified by the user, which could come from external trials, real world data or expert clinical opinion. This is achieved by solving simultaneous equations based on the survival functions of the parametric models. The application of SurvInt is shown through two examples where traditional parametric modelling did not produce models that were consistent with external data or clinical opinion. Additional features include model averaging, mixture cure models, background mortality, piecewise modelling, restricted mean survival time estimation and probabilistic sensitivity analysis. CONCLUSIONS SurvInt allows precise parametric survival models to be estimated and carried forward into economic models. It provides access to extrapolations that are consistent with multiple data sources such as observed data and clinical predictions, opening the door to precise exploration of regions of uncertainty/disagreement. SurvInt could avoid the need for post-hoc adjustments for complications such as treatment switching, which are often applied to obtain a plausible survival model but at the cost of introducing additional uncertainty. Phase III clinical trials are not designed with extrapolation in mind, and so it is sensible to consider alternative approaches to predict future survival that incorporate external information.
Collapse
Affiliation(s)
- Daniel Gallacher
- Warwick Medical School, University of Warwick, CV4 7HL, Coventry, UK.
| |
Collapse
|
2
|
Jackson CH. survextrap: a package for flexible and transparent survival extrapolation. BMC Med Res Methodol 2023; 23:282. [PMID: 38030986 PMCID: PMC10685663 DOI: 10.1186/s12874-023-02094-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Accepted: 11/03/2023] [Indexed: 12/01/2023] Open
Abstract
BACKGROUND Health policy decisions are often informed by estimates of long-term survival based primarily on short-term data. A range of methods are available to include longer-term information, but there has previously been no comprehensive and accessible tool for implementing these. RESULTS This paper introduces a novel model and software package for parametric survival modelling of individual-level, right-censored data, optionally combined with summary survival data on one or more time periods. It could be used to estimate long-term survival based on short-term data from a clinical trial, combined with longer-term disease registry or population data, or elicited judgements. All data sources are represented jointly in a Bayesian model. The hazard is modelled as an M-spline function, which can represent potential changes in the hazard trajectory at any time. Through Bayesian estimation, the model automatically adapts to fit the available data, and acknowledges uncertainty where the data are weak. Therefore long-term estimates are only confident if there are strong long-term data, and inferences do not rely on extrapolating parametric functions learned from short-term data. The effects of treatment or other explanatory variables can be estimated through proportional hazards or with a flexible non-proportional hazards model. Some commonly-used mechanisms for survival can also be assumed: cure models, additive hazards models with known background mortality, and models where the effect of a treatment wanes over time. All of these features are provided for the first time in an R package, survextrap, in which models can be fitted using standard R survival modelling syntax. This paper explains the model, and demonstrates the use of the package to fit a range of models to common forms of survival data used in health technology assessments. CONCLUSIONS This paper has provided a tool that makes comprehensive and principled methods for survival extrapolation easily usable.
Collapse
|
3
|
Peterse EFP, Verburg-Baltussen EJM, Stewart A, Liu FF, Parker C, Treur M, Malcolm B, Klijn SL. Retrospective Comparison of Survival Projections for CAR T-Cell Therapies in Large B-Cell Lymphoma. PHARMACOECONOMICS - OPEN 2023; 7:941-950. [PMID: 37651087 PMCID: PMC10721757 DOI: 10.1007/s41669-023-00435-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 08/06/2023] [Indexed: 09/01/2023]
Abstract
BACKGROUND Durable remission has been observed in patients with relapsed or refractory (R/R) large B-cell lymphoma (LBCL) treated with chimeric antigen receptor (CAR) T-cell therapy. Consequently, hazard functions for overall survival (OS) are often complex, requiring the use of flexible methods for extrapolations. OBJECTIVES We aimed to retrospectively compare the predictive accuracy of different survival extrapolation methods and evaluate the validity of goodness-of-fit (GOF) criteria-based model selection for CAR T-cell therapies in R/R LBCL. METHODS OS data were sourced from JULIET, ZUMA-1, and TRANSCEND NHL 001. Standard parametric, mixture cure, cubic spline, and mixture models were fit to multiple database locks (DBLs), with varying follow-up durations. GOF was assessed using the Akaike information criterion and Bayesian information criterion. Predictive accuracy was calculated as the mean absolute error (MAE) relative to OS observed in the most mature DBL. RESULTS For all studies, mixture cure and cubic spline models provided the best predictive accuracy for the least mature DBL (MAE 0.013‒0.085 and 0.014‒0.128, respectively). The predictive accuracy of the standard parametric and mixture models showed larger variation (MAE 0.024‒0.162 and 0.013‒0.176, respectively). With increasing data maturity, the predictive accuracy of standard parametric models remained poor. Correlation between GOF criteria and predictive accuracy was low, particularly for the least mature DBL. CONCLUSIONS Our analyses demonstrated that mixture cure and cubic spline models provide the most accurate survival extrapolations of CAR T-cell therapies in LBCL. Furthermore, GOF should not be the only criteria used when selecting the optimal survival model.
Collapse
|
4
|
Bakker LJ, Thielen FW, Redekop WK, Groot CUD, Blommestein HM. Extrapolating empirical long-term survival data: the impact of updated follow-up data and parametric extrapolation methods on survival estimates in multiple myeloma. BMC Med Res Methodol 2023; 23:132. [PMID: 37248477 DOI: 10.1186/s12874-023-01952-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 05/16/2023] [Indexed: 05/31/2023] Open
Abstract
BACKGROUND In economic evaluations, survival is often extrapolated to smooth out the Kaplan-Meier estimate and because the available data (e.g., from randomized controlled trials) are often right censored. Validation of the accuracy of extrapolated results can depend on the length of follow-up and the assumptions made about the survival hazard. Here, we analyze the accuracy of different extrapolation techniques while varying the data cut-off to estimate long-term survival in newly diagnosed multiple myeloma (MM) patients. METHODS Empirical data were available from a randomized controlled trial and a registry for MM patients treated with melphalan + prednisone, thalidomide, and bortezomib- based regimens. Standard parametric and spline models were fitted while artificially reducing follow-up by introducing database locks. The maximum follow-up for these locks varied from 3 to 13 years. Extrapolated (conditional) restricted mean survival time (RMST) was compared to the Kaplan-Meier RMST and models were selected according to statistical tests, and visual fit. RESULTS For all treatments, the RMST error decreased when follow-up and the absolute number of events increased, and censoring decreased. The decline in RMST error was highest when maximum follow-up exceeded six years. However, even when censoring is low there can still be considerable deviations in the extrapolated RMST conditional on survival until extrapolation when compared to the KM-estimate. CONCLUSIONS We demonstrate that both standard parametric and spline models could be worthy candidates when extrapolating survival for the populations examined. Nevertheless, researchers and decision makers should be wary of uncertainty in results even when censoring has decreased, and the number of events has increased.
Collapse
Affiliation(s)
- L J Bakker
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, P.O. Box 1738, Rotterdam, 3000 DR, The Netherlands.
- Erasmus Centre for Health Economics Rotterdam, Erasmus University, Rotterdam, The Netherlands.
| | - F W Thielen
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, P.O. Box 1738, Rotterdam, 3000 DR, The Netherlands
- Erasmus Centre for Health Economics Rotterdam, Erasmus University, Rotterdam, The Netherlands
| | - W K Redekop
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, P.O. Box 1738, Rotterdam, 3000 DR, The Netherlands
- Erasmus Centre for Health Economics Rotterdam, Erasmus University, Rotterdam, The Netherlands
| | - Ca Uyl-de Groot
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, P.O. Box 1738, Rotterdam, 3000 DR, The Netherlands
- Erasmus Centre for Health Economics Rotterdam, Erasmus University, Rotterdam, The Netherlands
| | - H M Blommestein
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, P.O. Box 1738, Rotterdam, 3000 DR, The Netherlands
- Erasmus Centre for Health Economics Rotterdam, Erasmus University, Rotterdam, The Netherlands
| |
Collapse
|
5
|
Shao T, Zhao M, Liang L, Shi L, Tang W. Impact of Extrapolation Model Choices on the Structural Uncertainty in Economic Evaluations for Cancer Immunotherapy: A Case Study of Checkmate 067. PHARMACOECONOMICS - OPEN 2023; 7:383-392. [PMID: 36757569 PMCID: PMC10169997 DOI: 10.1007/s41669-023-00391-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 01/16/2023] [Indexed: 05/11/2023]
Abstract
OBJECTIVES The aim of this study was to compare the performance of different extrapolation modeling techniques and analyze their impact on structural uncertainties in the economic evaluations of cancer immunotherapy. METHODS The individual patient data was reconstructed through published Checkmate 067 Kaplan Meier curves. Standard parametric models and six flexible techniques were tested, including fractional polynomial, restricted cubic splines, Royston-Parmar models, generalized additive models, parametric mixture models, and mixture cure models. Mean square errors (MSE) and bias from raw survival plots were used to test the model fitness and extrapolation performance. Variability of estimated incremental cost-effectiveness ratios (ICERs) from different models was used to inform the structural uncertainty in economic evaluations. All indicators were analyzed and compared under cut-offs of 3 years and 6.5 years, respectively, to further discuss model impact under different data maturity. R Codes for reproducing this study can be found on GitHub. RESULTS The flexible techniques in general performed better than standard parametric models with smaller MSE irrespective of the data maturity. Survival outcomes projected by long-term extrapolation using immature data differed from those with mature data. Although a best-performing model was not found because several models had very similar MSE in this case, the variability of modeled ICERs significantly increased when prolonging simulation cycles. CONCLUSIONS Flexible techniques show better performance in the case of Checkmate 067, regardless of data maturity. Model choices affect ICERs of cancer immunotherapy, especially when dealing with immature survival data. When researchers lack evidence to identify the 'right' model, we recommend identifying and revealing the model impacts on structural uncertainty.
Collapse
Affiliation(s)
- Taihang Shao
- School of International Pharmaceutical Business, China Pharmaceutical University, Nanjing, 211198, China
- Center for Pharmacoeconomics and Outcomes Research, China Pharmaceutical University, Nanjing, 211198, China
| | - Mingye Zhao
- School of International Pharmaceutical Business, China Pharmaceutical University, Nanjing, 211198, China
- Center for Pharmacoeconomics and Outcomes Research, China Pharmaceutical University, Nanjing, 211198, China
| | - Leyi Liang
- School of International Pharmaceutical Business, China Pharmaceutical University, Nanjing, 211198, China
- Center for Pharmacoeconomics and Outcomes Research, China Pharmaceutical University, Nanjing, 211198, China
| | - Lizheng Shi
- Department of Global Health Management and Policy, School of Public Health and Tropical Medicine, Tulane University, New Orleans, LA, 70118, USA.
| | - Wenxi Tang
- School of International Pharmaceutical Business, China Pharmaceutical University, Nanjing, 211198, China.
- Department of Public Affairs Management, School of International Pharmaceutical Business, China Pharmaceutical University, Nanjing, 211198, China.
| |
Collapse
|
6
|
Che Z, Green N, Baio G. Blended Survival Curves: A New Approach to Extrapolation for Time-to-Event Outcomes from Clinical Trials in Health Technology Assessment. Med Decis Making 2023; 43:299-310. [PMID: 36314662 PMCID: PMC10026162 DOI: 10.1177/0272989x221134545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
BACKGROUND Survival extrapolation is essential in cost-effectiveness analysis to quantify the lifetime survival benefit associated with a new intervention, due to the restricted duration of randomized controlled trials (RCTs). Current approaches of extrapolation often assume that the treatment effect observed in the trial can continue indefinitely, which is unrealistic and may have a huge impact on decisions for resource allocation. OBJECTIVE We introduce a novel methodology as a possible solution to alleviate the problem of survival extrapolation with heavily censored data from clinical trials. METHOD The main idea is to mix a flexible model (e.g., Cox semiparametric) to fit as well as possible the observed data and a parametric model encoding assumptions on the expected behavior of underlying long-term survival. The two are "blended" into a single survival curve that is identical with the Cox model over the range of observed times and gradually approaching the parametric model over the extrapolation period based on a weight function. The weight function regulates the way two survival curves are blended, determining how the internal and external sources contribute to the estimated survival over time. RESULTS A 4-y follow-up RCT of rituximab in combination with fludarabine and cyclophosphamide versus fludarabine and cyclophosphamide alone for the first-line treatment of chronic lymphocytic leukemia is used to illustrate the method. CONCLUSION Long-term extrapolation from immature trial data may lead to significantly different estimates with various modelling assumptions. The blending approach provides sufficient flexibility, allowing a wide range of plausible scenarios to be considered as well as the inclusion of external information, based, for example, on hard data or expert opinion. Both internal and external validity can be carefully examined. HIGHLIGHTS Interim analyses of trials with limited follow-up are often subject to high degrees of administrative censoring, which may result in implausible long-term extrapolations using standard approaches.In this article, we present an innovative methodology based on "blending" survival curves to relax the traditional proportional hazard assumption and simultaneously incorporate external information to guide the extrapolation.The blended method provides a simple and powerful framework to allow a careful consideration of a wide range of plausible scenarios, accounting for model fit to the short-term data as well as the plausibility of long-term extrapolations.
Collapse
Affiliation(s)
- Zhaojing Che
- Department of Statistical Science, University College London, Gower Street, London UK
| | - Nathan Green
- Department of Statistical Science, University College London, Gower Street, London UK
| | - Gianluca Baio
- Department of Statistical Science, University College London, Gower Street, London UK
| |
Collapse
|
7
|
Palmer S, Borget I, Friede T, Husereau D, Karnon J, Kearns B, Medin E, Peterse EFP, Klijn SL, Verburg-Baltussen EJM, Fenwick E, Borrill J. A Guide to Selecting Flexible Survival Models to Inform Economic Evaluations of Cancer Immunotherapies. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2023; 26:185-192. [PMID: 35970706 DOI: 10.1016/j.jval.2022.07.009] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Revised: 06/10/2022] [Accepted: 07/06/2022] [Indexed: 06/15/2023]
Abstract
OBJECTIVES Parametric models are routinely used to estimate the benefit of cancer drugs beyond trial follow-up. The advent of immune checkpoint inhibitors has challenged this paradigm, and emerging evidence suggests that more flexible survival models, which can better capture the shapes of complex hazard functions, might be needed for these interventions. Nevertheless, there is a need for an algorithm to help analysts decide whether flexible models are required and, if so, which should be chosen for testing. This position article has been produced to bridge this gap. METHODS A virtual advisory board comprising 7 international experts with in-depth knowledge of survival analysis and health technology assessment was held in summer 2021. The experts discussed 24 questions across 6 topics: the current survival model selection procedure, data maturity, heterogeneity of treatment effect, cure and mortality, external evidence, and additions to existing guidelines. Their responses culminated in an algorithm to inform selection of flexible survival models. RESULTS The algorithm consists of 8 steps and 4 questions. Key elements include the systematic identification of relevant external data, using clinical expert input at multiple points in the selection process, considering the future and the observed hazard functions, assessing the potential for long-term survivorship, and presenting results from all plausible models. CONCLUSIONS This algorithm provides a systematic, evidence-based approach to justify the selection of survival extrapolation models for cancer immunotherapies. If followed, it should reduce the risk of selecting inappropriate models, partially addressing a key area of uncertainty in the economic evaluation of these agents.
Collapse
Affiliation(s)
- Stephen Palmer
- Centre for Health Economics, University of York, York, England, UK
| | - Isabelle Borget
- Biostatistics and Epidemiology office, Gustave Roussy, Paris-Saclay University, Villejuif, France; Oncostat, Paris-Saclay University U1018, Inserm, Paris-Saclay University, "Ligue Contre le Cancer" labeled team, Villejuif, France
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| | - Don Husereau
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
| | - Jonathan Karnon
- Flinders Health and Medical Research Institute, Flinders University, Adelaide, SA, Australia
| | - Ben Kearns
- School of Health and Related Research, University of Sheffield, Sheffield, England, UK
| | - Emma Medin
- Parexel International, Stockholm, Sweden; Department of Learning, Infomatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden
| | | | - Sven L Klijn
- Worldwide Health Economics and Outcomes Research - Economic and Predictive Modeling, Bristol Myers Squibb, Utrecht, The Netherlands
| | | | | | - John Borrill
- Worldwide Health Economics and Outcomes Research, Bristol Myers Squibb, Uxbridge, Greater London, England, UK.
| |
Collapse
|
8
|
Kearns B, Stevenson MD, Triantafyllopoulos K, Manca A. Dynamic and Flexible Survival Models for Extrapolation of Relative Survival: A Case Study and Simulation Study. Med Decis Making 2022; 42:945-955. [PMID: 35769004 PMCID: PMC9459356 DOI: 10.1177/0272989x221107649] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND Extrapolation of survival data is a key task in health technology assessments (HTAs), which may be improved by incorporating general population mortality data via relative survival models. Dynamic survival models are a promising method for extrapolation that may be expanded to dynamic relative survival models (DRSMs), a novel development presented here. There are currently neither examples of dynamic models in HTA nor comparisons of DRSMs with other relative survival models when used for survival extrapolation. METHODS An existing appraisal, for which there had been disagreement over the approach to survival extrapolation, was chosen and the health economic model recreated. The sensitivity of estimates of cost-effectiveness to different model choices (standard survival models, DSMs, and DRSMs) and specifications was examined. The appraisal informed a simulation study to evaluate DRSMs with relative survival models based on both standard and spline-based (flexible) models. RESULTS Dynamic models provided insight into the behavior of the trend in the hazard function and how it may vary during the extrapolated phase. DRSMs led to extrapolations with improved plausibility for which model choice may be based on clinical input. In the simulation study, the flexible and dynamic relative survival models performed similarly and provided highly variable extrapolations. LIMITATIONS Further experience with these models is required to identify settings when they are most useful, and they provide sufficiently accurate extrapolations. CONCLUSIONS Dynamic models provide a flexible and attractive method for extrapolating survival data and facilitate the use of clinical input for model choice. Flexible and dynamic relative survival models make few structural assumptions and can improve extrapolation plausibility, but further research is required into methods for reducing the variability in extrapolations.
Collapse
|
9
|
A Review of Overall Survival Extrapolations of Immune-Checkpoint Inhibitors Used in Health Technology Assessments by the French Health Authorities. Int J Technol Assess Health Care 2022; 38:e28. [PMID: 35331347 DOI: 10.1017/s0266462322000125] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
OBJECTIVES Extrapolation is often required to inform cost-effectiveness (CE) evaluations of immune-checkpoint inhibitors (ICIs) since survival data from pivotal clinical trials are seldom complete. The objectives of this study were to evaluate the accuracy of estimates of long-term overall survival (OS) predicted in French CE assessment reports of ICIs, and to identify models presenting the best fit to the observed long-term survival data. METHODS A systematic review of French assessment reports of ICIs in the metastatic setting since inception until May 2020 was performed. A targeted literature review was conducted to collect associated extended follow-up of randomized controlled trials (RCTs) used in the CE assessment reports. Difference between projected and observed OS was calculated. A range of standard parametric and spline-based models were applied to the extended follow-up data from the RCT to determine the best-fitting survival models. RESULTS Of the 121 CE assessment reports published, 11 reports met the inclusion criteria. OS was underestimated in 73 percent of the CE assessment reports. The mean relative difference between each source was -13 percent (median: -15 percent; IQR: -0.4 to 26 percent). Models providing the best fit were those that could reflect nonmonotonic hazards. CONCLUSIONS Based on the available data at the time of submission, longer-term survival of ICIs was not fully captured by the extrapolation models used in CE assessments. Standard and flexible parametric models which can capture nonmonotonic hazard functions provided the best fit to the extended follow-up data. However, these models may still have performed poorly if fitted to survival data available at the time of submission to the French National Authority for Health.
Collapse
|
10
|
Latimer NR, Adler AI. Extrapolation beyond the end of trials to estimate long term survival and cost effectiveness. BMJ MEDICINE 2022; 1:e000094. [PMID: 36936578 PMCID: PMC9951371 DOI: 10.1136/bmjmed-2021-000094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Accepted: 01/14/2022] [Indexed: 11/03/2022]
Affiliation(s)
- Nicholas R Latimer
- School of Health and Related Research, University of Sheffield, Sheffield, UK
| | | |
Collapse
|