Economic evaluations of implemented vaccination programmes: key methodological challenges in retrospective analyses

How to Prevent Vaccines Falling Victim to Their Own Success: Intertemporal Dependency of Incidence Levels on Indirect Effects in Economic Reevaluations

OBJECTIVES

Incremental cost-effectiveness analyses may inform the optimal choice of healthcare interventions. Nevertheless, for many vaccines, benefits fluctuate with incidence levels over time. Reevaluating a vaccine after it has successfully decreased incidences may eventually cause a disease resurgence if switching to a vaccine with lower indirect benefits. Decisions may successively alternate between vaccines alongside repeated rises and falls in incidence and when indirect effects from historic use are ignored. Our suggested proposal aims to prevent suboptimal decision making.

METHODS

We used a conceptual model of demand to illustrate alternating decisions between vaccines because of time-varying levels of indirect effects. Similar to the concept of subsidies, we propose internalizing the indirect effects achievable with vaccines. In a case study over 60 years, we simulated a hypothetical 10-year reevaluation of 2 oncogenic human papillomavirus vaccines, of which only 1 protects additionally against anogenital warts.

RESULTS

Our case study showed that the vaccine with additional warts protection is initially valued higher than the vaccine without additional warts protection. After 10 years, this differential decreases because of declines in warts incidence, which supports switching to the nonwarts vaccine that causes a warts resurgence eventually. Instead, pricing the indirect effects separately supports continuing with the warts vaccine.

CONCLUSIONS

Ignoring how the observed incidences depend on the indirect effects achieved with a particular vaccine may lead to repeated changes in vaccines at successive reevaluations, with unintended resurgences, economic inefficiencies, and eroding vaccine confidence. We propose internalizing indirect effects to prevent vaccines falling victim to their own success.

Rotavirus vaccination likely to be cost saving to society in the United States

BACKGROUND

Following the introduction of rotavirus immunization in 2006 in the United States (US) there were substantial declines in the domestic rotavirus disease burden. In this study we assess the value for money achieved by the program in the decade following vaccine introduction.

METHODS

We applied an age-specific static multi-cohort compartmental model to examine the impact and cost-effectiveness of the US rotavirus immunization program in children <5 years of age using healthcare utilization data from 2001-2015 inclusive. We calculated the incremental cost-effectiveness ratio (ICER) per quality-adjusted life year (QALY) gained from both a healthcare system and societal perspective.

RESULTS

Declines in healthcare utilization associated with the rotavirus and acute gastroenteritis occurred from 2006 and continued to grow before stabilizing from 2010-2011. From 2011-2015, an estimated annual average of approximately 118,000 hospitalizations, 86,000 emergency department presentations and 460,000 outpatient and physician office visits were prevented. From a societal perspective during this same period the program was estimated to be cost saving in the base case model and in >90% of probabilistic sensitivity analysis simulations and from a healthcare system perspective >98% of simulations found an ICER below $100,000 per QALY gained.

CONCLUSIONS

After the program stabilized, we found the rotavirus immunization in the US was likely to have been cost saving to society. The greater than expected healthcare and productivity savings reflect the success of the rotavirus immunization program in the US.

Infant Hospitalizations and Fatalities Averted by the Maternal Pertussis Vaccination Program in England, 2012-2017: Post-implementation Economic Evaluation

In October 2012, a maternal pertussis vaccination program was implemented in England following an increased incidence and mortality in infants. We evaluated the cost-effectiveness of the program by comparing pertussis-related infant hospitalizations and deaths in 2012-2017 with nonvaccination scenarios. Despite considerable uncertainties, findings support the cost-effectiveness of the program.

Evaluation of the Impact of a Rotavirus Vaccine Program on Pediatric Acute Gastroenteritis Hospitalizations: Estimating the Overall Effect Attributable to the Program as a Whole and as a Per-Unit Change in Rotavirus Vaccine Coverage

Estimation of the overall effect of a vaccine program is essential, but the effect is typically estimated for a whole program. We estimated the overall effect of the Quebec rotavirus vaccine program, launched in November 2011, and the effect for each 10% increase in rotavirus vaccine coverage on pediatric hospitalizations for all-cause acute gastroenteritis. We implemented negative binomial regressions adjusted for seasonality, long-term trends, and infection dynamics, to estimate the effect of the vaccine program as: 1) a dichotomous variable, representing program presence/absence, and linear term to account for changes in trend in the period after the program began; and 2) a continuous variable, representing rotavirus vaccine coverage. Using exposure 1, the vaccine program was associated with a 51.2% (95% confidence interval (CI): 28.5, 66.7) relative decline in adjusted weekly hospitalization rates for all-cause acute gastroenteritis as of December 28, 2014. Using exposure 2, a 10% increase in rotavirus ≥1-dose coverage was associated with a 7.1% (95% CI: 3.5, 10.5) relative decline in adjusted weekly rates, with maximum coverage of 87.0% associated with a 47.2% (95% CI: 26.9, 61.9) relative decline. Estimation of the overall effect attributable to a change in vaccine coverage might be a useful addition to standard measurement of the overall effect.

Beyond expectations: Post-implementation data shows rotavirus vaccination is likely cost-saving in Australia

BACKGROUND

Universal vaccination against rotavirus was included in the funded Australian National Immunisation Program in July 2007. Predictive cost-effectiveness models assessed the program before introduction.

METHODS

We conducted a retrospective economic evaluation of the Australian rotavirus program using national level post-implementation data on vaccine uptake, before-after measures of program impact and published estimates of excess intussusception cases. These data were used as inputs into a multi-cohort compartmental model which assigned cost and quality of life estimates to relevant health states, adopting a healthcare payer perspective. The primary outcome was discounted cost per quality adjusted life year gained, including or excluding unspecified acute gastroenteritis (AGE) hospitalisations.

RESULTS

Relative to the baseline period (1997-2006), over the 6years (2007-2012) after implementation of the rotavirus program, we estimated that ∼77,000 hospitalisations (17,000 coded rotavirus and 60,000 unspecified AGE) and ∼3 deaths were prevented, compared with an estimated excess of 78 cases of intussusception. Approximately 90% of hospitalisations prevented were in children <5years, with evidence of herd protection in older age groups. The program was cost-saving when observed changes (declines) in both hospitalisations coded as rotavirus and as unspecified AGE were attributed to the rotavirus vaccine program. The adverse impact of estimated excess cases of intussusception was far outweighed by the benefits of the program.

CONCLUSION

The inclusion of herd impact and declines in unspecified AGE hospitalisations resulted in the value for money achieved by the Australian rotavirus immunisation program being substantially greater than predicted bypre-implementation models, despite the potential increased cases of intussusception. This Australian experience is likely to be relevant to high-income countries yet to implement rotavirus vaccination programs.

Economic evaluations of vaccines in Canada: a scoping review

BACKGROUND

This study aims to summarise and describe the evolution of published economic evaluations of vaccines in Canada, thereby outlining the current state of this expanding and meaningful research.

METHODS

Using Arksey and O'Malley's scoping review framework we assembled relevant research from both academic and grey literature. Following abstract and full-text review we identified 60 articles to be included in the final analysis.

RESULTS

We found that since 1988 there has been a steady increase in the number of economic evaluations on vaccines in Canada. Many of these studies focus on the more recently licensed vaccines, such as influenza (16.7%), human papillomavirus (15.0%) and pneumococcal disease (15.0%). Since 2010 economic evaluations of vaccines have shown increased adherence to economic evaluation guidelines (OR = 4.6, CI 1.33, 18.7), suggesting there has been improvement in the consistency and transparency of these studies. However, there remains room for improvement, for instance, we found evidence that studies who stated a conflict of interest are more likely to assert the vaccine of interest was cost-effective (OR = 7.4; CI 1.04, 17.8). Furthermore, most reports use static models that do not consider herd immunity, and only a few evaluate vaccines post-implementation (ex-post) and traveller's vaccinations.

CONCLUSION

Researchers should examine identified research gaps and continue to improve standardization and transparency when reporting to ensure economic evaluations of vaccines best meet the needs of policy-makers, other researchers and the public.

Economic Evaluation of Vaccination Programmes in Older Adults and the Elderly: Important Issues and Challenges

High-income countries are undergoing demographic transitions towards populations with substantial larger proportions of older adults. Due to the increased susceptibility of older adults to infectious diseases and their consequences, vaccination programmes are an important health intervention to help maintain healthy ageing. While much of the existing literature suggests that current vaccination programmes targeted at older adults and the elderly are likely to be cost effective in high-income countries, we argue that it is important to more fully consider some important issues and challenges. Since the majority of vaccines have been developed for children, economic evaluations of vaccination programmes have consequentially tended to focus on this age group and on how to incorporate herd-immunity effects. While programmes targeted at older adults and the elderly may also induce some herd effects, there are other important challenges to consider in these economic evaluations. For example, age and time effects in relation to vaccine efficacy and duration of immunity, as well as heterogeneity between targeted individuals in terms of risk of infection, severity of disease and response to vaccination. For some pathogens, there is also the potential for interactions with childhood programmes in the form of herd-immunity effects.

What do we know about the cost-effectiveness of pneumococcal conjugate vaccination in older adults?

The cost-effectiveness of 13-type pneumococcal conjugate vaccine (PCV13) use in older adults, and the relative merits when compared to the 23-type polysaccharide pneumococcal vaccine (PPV23), has been a topic of much debate. Although a number of economics evaluations have been conducted many of these were completed before the availability of critical data on PCV13 efficacy in older adults. Recent studies using this data have found conflicting results. This may in part reflect differences in the level of herd protection from infant pneumococcal vaccination programs in different countries. The costs and benefits of pneumococcal vaccination in adults are likely to rest on several critical parameters: the magnitude pneumococcal disease in older adults and the serotypes responsible for it, the efficacy of each vaccine against invasive and non-invasive pneumonia, the duration of vaccine protection, and differences in vaccine price. The ongoing changes in pneumococcal disease patterns highlight the need for economic evaluations to use recent serotype-specific disease estimates from the setting under consideration. In countries that do recommend PCV13 use in adults, post-implementation economic evaluation (using data from after a program is implemented) may be useful to help inform potential future changes to vaccine recommendations as well as the maximum price that should be paid for the vaccines in future negotiations.

Methodological Challenges to Economic Evaluations of Vaccines: Is a Common Approach Still Possible?

Economic evaluation of vaccination is a key tool to inform effective spending on vaccines. However, many evaluations have been criticised for failing to capture features of vaccines which are relevant to decision makers. These include broader societal benefits (such as improved educational achievement, economic growth and political stability), reduced health disparities, medical innovation, reduced hospital beds pressures, greater peace of mind and synergies in economic benefits with non-vaccine interventions. Also, the fiscal implications of vaccination programmes are not always made explicit. Alternative methodological frameworks have been proposed to better capture these benefits. However, any broadening of the methodology for economic evaluation must also involve evaluations of non-vaccine interventions, and hence may not always benefit vaccines given a fixed health-care budget. The scope of an economic evaluation must consider the budget from which vaccines are funded, and the decision-maker's stated aims for that spending to achieve.

Assessing the impact of vaccination programmes on burden of disease: Underlying complexities and statistical methods

It is important to assess the impact a vaccination programme has on the burden of disease after it is implemented. For example, this may reveal herd immunity effects or vaccine-induced shifts in the incidence of disease or in circulating strains or serotypes of the pathogen. In this article we summarise the key features of infectious diseases that need to be considered when trying to detect any changes in the burden of diseases at a population level as a result of vaccination efforts. We outline the challenges of using routine surveillance databases to monitor infectious diseases, such as the identification of diseased cases and the availability of vaccination status for cases. We highlight the complexities in modelling the underlying patterns in infectious disease rates (e.g. presence of autocorrelation) and discuss the main statistical methods that can be used to control for periodicity (e.g. seasonality) and autocorrelation when assessing the impact of vaccination programmes on burden of disease (e.g. cosinor terms, generalised additive models, autoregressive processes and moving averages). For some analyses, there may be multiple methods that can be used, but it is important for authors to justify the method chosen and discuss any limitations. We present a case study review of the statistical methods used in the literature to assess the rotavirus vaccination programme impact in Australia. The methods used varied and included generalised linear models and descriptive statistics. Not all studies accounted for autocorrelation and seasonality, which can have a major influence on results. We recommend that future analyses consider the strength and weakness of alternative statistical methods and justify their choice.

Methods for Health Economic Evaluation of Vaccines and Immunization Decision Frameworks: A Consensus Framework from a European Vaccine Economics Community

BACKGROUND

Incremental cost-effectiveness and cost-utility analyses [health economic evaluations (HEEs)] of vaccines are routinely considered in decision making on immunization in various industrialized countries. While guidelines advocating more standardization of such HEEs (mainly for curative drugs) exist, several immunization-specific aspects (e.g. indirect effects or discounting approach) are still a subject of debate within the scientific community.

OBJECTIVE

The objective of this study was to develop a consensus framework for HEEs of vaccines to support the development of national guidelines in Europe.

METHODS

A systematic literature review was conducted to identify prevailing issues related to HEEs of vaccines. Furthermore, European experts in the field of health economics and immunization decision making were nominated and asked to select relevant aspects for discussion. Based on this, a workshop was held with these experts. Aspects on 'mathematical modelling', 'health economics' and 'decision making' were debated in group-work sessions (GWS) to formulate recommendations and/or--if applicable--to state 'pros' and 'contras'.

RESULTS

A total of 13 different aspects were identified for modelling and HEE: model selection, time horizon of models, natural disease history, measures of vaccine-induced protection, duration of vaccine-induced protection, indirect effects apart from herd protection, target population, model calibration and validation, handling uncertainty, discounting, health-related quality of life, cost components, and perspectives. For decision making, there were four aspects regarding the purpose and the integration of HEEs of vaccines in decision making as well as the variation of parameters within uncertainty analyses and the reporting of results from HEEs. For each aspect, background information and an expert consensus were formulated.

CONCLUSIONS

There was consensus that when HEEs are used to prioritize healthcare funding, this should be done in a consistent way across all interventions, including vaccines. However, proper evaluation of vaccines implies using tools that are not commonly used for therapeutic drugs. Due to the complexity of and uncertainties around vaccination, transparency in the documentation of HEEs and during subsequent decision making is essential.

Retrospective economic evaluation of childhood 7-valent pneumococcal conjugate vaccination in Australia: Uncertain herd impact on pneumonia critical

BACKGROUND

Retrospective cost-effectiveness analyses of vaccination programs using routinely collected post-implementation data are sparse by comparison with pre-program analyses. We performed a retrospective economic evaluation of the childhood 7-valent pneumococcal conjugate vaccine (PCV7) program in Australia.

METHODS

We developed a deterministic multi-compartment model that describes health states related to invasive and non-invasive pneumococcal disease. Costs (Australian dollars, A$) and health effects (quality-adjusted life years, QALYs) were attached to model states. The perspective for costs was that of the healthcare system and government. Where possible, we used observed changes in the disease rates from national surveillance and healthcare databases to estimate the impact of the PCV7 program (2005-2010). We stratified our cost-effectiveness results into alternative scenarios which differed by the outcome states included. Parameter uncertainty was explored using probabilistic sensitivity analysis.

RESULTS

The PCV7 program was estimated to have prevented ∼5900 hospitalisations and ∼160 deaths from invasive pneumococcal disease (IPD). Approximately half of these were prevented in adults via herd protection. The incremental cost-effectiveness ratio was ∼A$161,000 per QALY gained when including only IPD-related outcomes. The cost-effectiveness of PCV7 remained in the range A$88,000-$122,000 when changes in various non-invasive disease states were included. The inclusion of observed changes in adult non-invasive pneumonia deaths substantially improved cost-effectiveness (∼A$9000 per QALY gained).

CONCLUSION

Using the initial vaccine price negotiated for Australia, the PCV7 program was unlikely to have been cost-effective (at conventional thresholds) unless observed reductions in non-invasive pneumonia deaths in the elderly are attributed to it. Further analyses are required to explore this finding, which has significant implications for the incremental benefit achievable by adult PCV programs.

Dealing with Time in Health Economic Evaluation: Methodological Issues and Recommendations for Practice

Time is an important aspect of health economic evaluation, as the timing and duration of clinical events, healthcare interventions and their consequences all affect estimated costs and effects. These issues should be reflected in the design of health economic models. This article considers three important aspects of time in modelling: (1) which cohorts to simulate and how far into the future to extend the analysis; (2) the simulation of time, including the difference between discrete-time and continuous-time models, cycle lengths, and converting rates and probabilities; and (3) discounting future costs and effects to their present values. We provide a methodological overview of these issues and make recommendations to help inform both the conduct of cost-effectiveness analyses and the interpretation of their results. For choosing which cohorts to simulate and how many, we suggest analysts carefully assess potential reasons for variation in cost effectiveness between cohorts and the feasibility of subgroup-specific recommendations. For the simulation of time, we recommend using short cycles or continuous-time models to avoid biases and the need for half-cycle corrections, and provide advice on the correct conversion of transition probabilities in state transition models. Finally, for discounting, analysts should not only follow current guidance and report how discounting was conducted, especially in the case of differential discounting, but also seek to develop an understanding of its rationale. Our overall recommendations are that analysts explicitly state and justify their modelling choices regarding time and consider how alternative choices may impact on results.

Analytics for vaccine economics and pricing: insights and observations

Pediatric immunization programs in the USA are a successful and cost-effective public health endeavor, profoundly reducing mortalities caused by infectious diseases. Two important issues relate to the success of the immunization programs, the selection of cost-effective vaccines and the appropriate pricing of vaccines. The recommended childhood immunization schedule, published annually by the CDC, continues to expand with respect to the number of injections required and the number of vaccines available for selection. The advent of new vaccines to meet the growing requirements of the schedule results: in a large, combinatorial number of possible vaccine formularies. The expansion of the schedule and the increase in the number of available vaccines constitutes a challenge for state health departments, large city immunization programs, private practices and other vaccine purchasers, as a cost-effective vaccine formulary must be selected from an increasingly large set of possible vaccine combinations to satisfy the schedule. The pediatric vaccine industry consists of a relatively small number of pharmaceutical firms engaged in the research, development, manufacture and distribution of pediatric vaccines. The number of vaccine manufacturers has dramatically decreased in the past few decades for a myriad of reasons, most notably due to low profitability. The contraction of the industry negatively impacts the reliable provision of pediatric vaccines. The determination of appropriate vaccine prices is an important issue and influences a vaccine manufacturer's decision to remain in the market. Operations research is a discipline that applies advanced analytical methods to improve decision making; analytics is the application of operations research to a particular problem using pertinent data to provide a practical result. Analytics provides a mechanism to resolve the challenges facing stakeholders in the vaccine development and delivery system, in particular, the selection of cost-effective vaccines and the appropriate pricing of vaccines. A review of applicable analytics papers is provided.