Using Evidence from Randomised Controlled Trials in Economic Models: What Information is Relevant and is There a Minimum Amount of Sample Data Required to Make Decisions?

How Uncertain is the Survival Extrapolation? A Study of the Impact of Different Parametric Survival Models on Extrapolated Uncertainty About Hazard Functions, Lifetime Mean Survival and Cost Effectiveness

BACKGROUND AND OBJECTIVE

The extrapolation of estimated hazard functions can be an important part of cost-effectiveness analyses. Given limited follow-up time in the sample data, it may be expected that the uncertainty in estimates of hazards increases the further into the future they are extrapolated. The objective of this study was to illustrate how the choice of parametric survival model impacts on estimates of uncertainty about extrapolated hazard functions and lifetime mean survival.

METHODS

We examined seven commonly used parametric survival models and described analytical expressions and approximation methods (delta and multivariate normal) for estimating uncertainty. We illustrate the multivariate normal method using case studies based on four representative hypothetical datasets reflecting hazard functions commonly encountered in clinical practice (constant, increasing, decreasing, or unimodal), along with a hypothetical cost-effectiveness analysis.

RESULTS

Depending on the survival model chosen, the uncertainty in extrapolated hazard functions could be constant, increasing or decreasing over time for the case studies. Estimates of uncertainty in mean survival showed a large variation (up to sevenfold) for each case study. The magnitude of uncertainty in estimates of cost effectiveness, as measured using the incremental cost per quality-adjusted life-year gained, varied threefold across plausible models. Differences in estimates of uncertainty were observed even when models provided near-identical point estimates.

CONCLUSIONS

Survival model choice can have a significant impact on estimates of uncertainty of extrapolated hazard functions, mean survival and cost effectiveness, even when point estimates were similar. We provide good practice recommendations for analysts and decision makers, emphasizing the importance of considering the plausibility of estimates of uncertainty in the extrapolated period as a complementary part of the model selection process.

A Review of Survival Analysis Methods Used in NICE Technology Appraisals of Cancer Treatments: Consistency, Limitations, and Areas for Improvement

Objectives. In June 2011, the National Institute for Health and Care Excellence (NICE) Decision Support Unit published a Technical Support Document (TSD) providing recommendations on survival analysis for NICE technology appraisals (TAs). Survival analysis outputs are influential inputs into economic models estimating the cost-effectiveness of new cancer treatments. Hence, it is important that systematic and justifiable model selection approaches are used. This study investigates the extent to which the TSD recommendations have been followed since its publication. Methods. We reviewed NICE cancer TAs completed between July 2011 and July 2017. Information on survival analyses undertaken and associated critiques for overall survival (OS) and progression-free survival were extracted from the company submissions, Evidence Review Group (ERG) reports, and final appraisal determination documents. Results. Information was extracted from 58 TAs. Only 4 (7%) followed all TSD recommendations for OS outcomes. The vast majority (91%) compared a range of common parametric models and assessed their fit to the data (86%). Only a minority of TAs included an assessment of the shape of the hazard function (38%) or proportional hazards assumption (40%). Validation of the extrapolated portion of the survival function using external data was attempted in a minority of TAs (40%). Extrapolated survival functions were frequently criticized by ERGs (71%). Conclusions. Survival analysis within NICE TAs remains suboptimal, despite publication of the TSD. Model selection is not undertaken in a systematic way, resulting in inconsistencies between TAs. More attention needs to be given to assessing hazard functions and validation of extrapolated survival functions. Novel methods not described in the TSD have been used, particularly in the context of immuno-oncology, suggesting that an updated TSD may be of value.