ADVANCE system testing: Benefit-risk analysis of a marketed vaccine using multi-criteria decision analysis and individual-level state transition modelling

The Brighton collaboration standardized module for vaccine benefit-risk assessment

Vaccine Benefit-Risk (B-R) assessment consists of evaluating the benefits and risks of a vaccine and making a judgment whether the expected key benefits outweigh the potential key risks associated with its expected use. B-R supports regulatory and public health decision-making throughout the vaccine's lifecycle. In August 2021, the Brighton Collaboration's Benefit-Risk Assessment of VAccines by TechnolOgy (BRAVATO) Benefit-Risk Assessment Module working group was established to develop a standard module to support the planning, conduct and evaluation of structured B-R assessments for vaccines from different platforms, based on data from clinical trials, post-marketing studies and real-world evidence. It enables sharing of relevant information via value trees, effects tables and graphical depictions of B-R trade-offs. It is intended to support vaccine developers, funders, regulators and policy makers in high-, middle- or low-income countries to help inform decision-making and facilitate transparent communication concerning development, licensure, deployment and other lifecycle decisions.

Benefit-Risk Assessment of Vaccines. Part I: A Systematic Review to Identify and Describe Studies About Quantitative Benefit-Risk Models Applied to Vaccines

Introduction

Understanding the balance between the benefits and risks of vaccination is essential to ensure informed and adequate public health decision making. Quantitative benefit–risk models (qBRm) represent useful tools to help decision makers with supporting benefit–risk assessment throughout the lifecycle of a medical product. However, few initiatives have been launched to harmonise qBRm approaches, specifically for vaccines.

Objectives

The aim of this paper was to identify publications about qBRm applied to vaccines through a systematic literature review, and to describe their characteristics.

Methods

Medline, Scopus and Institute for Scientific Information Web of Knowledge databases were searched to identify articles in English, published from database inceptions up to December 2019. The search strategy included the combination of three key concepts: ‘benefit–risk’, ‘modelling’ and ‘vaccines’. Data extracted included the modelling context and the methodological approaches used.

Results

Of 3172 publications screened, 48 original publications were included. Most of the selected studies were published over the past decade and focused on rotavirus (15), dengue (10) and influenza (6) vaccines. The majority (30) of studies reported analyses related to high-income countries. The methodology of the studies differed, particularly in modelling techniques, benefit–risk measures, and sensitivity analyses. The present work also pointed out a high level of variability in the quality of reporting across studies, with particular regard to input parameters and methodological approaches.

Conclusions

This review provides an extensive list of qBRm applied to vaccines. Discrepancies across studies were identified during our review. While the number of published qBRm studies is increasing, no reporting guidance for qBRm applied to vaccines is currently available. This may affect decision makers’ confidence in the results and their benefit–risk assessment(s); therefore, the development of such reporting guidance is highly needed.

Electronic supplementary material

The online version of this article (10.1007/s40264-020-00984-7) contains supplementary material, which is available to authorized users.

Methodological frontiers in vaccine safety: qualifying available evidence for rare events, use of distributed data networks to monitor vaccine safety issues, and monitoring the safety of pregnancy interventions

While vaccines are rigorously tested for safety and efficacy in clinical trials, these trials do not include enough subjects to detect rare adverse events, and they generally exclude special populations such as pregnant women. It is therefore necessary to conduct postmarketing vaccine safety assessments using observational data sources. The study of rare events has been enabled in through large linked databases and distributed data networks, in combination with development of case-centred methods. Distributed data networks necessitate common protocols, definitions, data models and analytics and the processes of developing and employing these tools are rapidly evolving. Assessment of vaccine safety in pregnancy is complicated by physiological changes, the challenges of mother-child linkage and the need for long-term infant follow-up. Potential sources of bias including differential access to and utilisation of antenatal care, immortal time bias, seasonal timing of pregnancy and unmeasured determinants of pregnancy outcomes have yet to be fully explored. Available tools for assessment of evidence generated in postmarketing studies may downgrade evidence from observational data and prioritise evidence from randomised controlled trials. However, real-world evidence based on real-world data is increasingly being used for safety assessments, and new tools for evaluating real-world evidence have been developed. The future of vaccine safety surveillance, particularly for rare events and in special populations, comprises the use of big data in single countries as well as in collaborative networks. This move towards the use of real-world data requires continued development of methodologies to generate and assess real world evidence.

ADVANCE database characterisation and fit for purpose assessment for multi-country studies on the coverage, benefits and risks of pertussis vaccinations

INTRODUCTION

The public-private ADVANCE consortium (Accelerated development of vaccine benefit-risk collaboration in Europe) aimed to assess if electronic healthcare databases can provide fit-for purpose data for collaborative, distributed studies and monitoring of vaccine coverage, benefits and risks of vaccines.

OBJECTIVE

To evaluate if European healthcare databases can be used to estimate vaccine coverage, benefit and/or risk using pertussis-containing vaccines as an example.

METHODS

Characterisation was conducted using open-source Java-based (Jerboa) software and R scripts. We obtained: (i) The general characteristics of the database and data source (meta-data) and (ii) a detailed description of the database population (size, representatively of age/sex of national population, rounding of birth dates, delay between birth and database entry), vaccinations (number of vaccine doses, recording of doses, pattern of doses by age and coverage) and events of interest (diagnosis codes, incidence rates). A total of nine databases (primary care, regional/national record linkage) provided data on events (pertussis, pneumonia, death, fever, convulsions, injection site reactions, hypotonic hypo-responsive episode, persistent crying) and vaccines (acellular pertussis and whole cell pertussis) related to the pertussis proof of concept studies.

RESULTS

The databases contained data for a total population of 44 million individuals. Seven databases had recorded doses of vaccines. The pertussis coverage estimates were similar to those reported by the World Health Organisation (WHO). Incidence rates of events were comparable in magnitude and age-distribution between databases with the same characteristics. Several conditions (persistent crying and somnolence) were not captured by the databases for which outcomes were restricted to hospital discharge diagnoses.

CONCLUSION

The database characterisation programs and workflows allowed for an efficient, transparent and standardised description and verification of electronic healthcare databases which may participate in pertussis vaccine coverage, benefit and risk studies. This approach is ready to be used for other vaccines/events to create readiness for participation in other vaccine related studies.

Advance system testing: Vaccine benefit studies using multi-country electronic health data - The example of pertussis vaccination

The Accelerated Development of VAccine benefit-risk Collaboration in Europe (ADVANCE), a public-private consortium, implemented and tested a distributed network system for the generation of evidence on the benefits-risks of marketed vaccines in Europe. We tested the system by estimating the incidence rate (IR) of pertussis and pertussis-related complications in children vaccinated with acellular (aP) and whole-cell (wP) pertussis vaccine. Data from seven electronic databases from four countries (Denmark: AUH and SSI, Spain: SIDIAP and BIFAP, UK: THIN and RCGP RSC and Italy: Pedianet) were included in a retrospective cohort analysis. Exposure was defined as any pertussis vaccination (aP or wP). The follow-up time started 14 days after the first dose. Children who had received any pertussis vaccine from January 1990 to December 2015 were included (those who switched type, or had unknown type were excluded). The outcomes of interest were confirmed or suspected pertussis and pertussis-related pneumonia and generalised convulsions within one month of pertussis diagnosis and death within three months of pertussis diagnosis. The cohort comprised 2,886,367 children ≤5 years of age. Data on wP and aP vaccination were available in three and seven databases, respectively. The IRs (per 100,000 person-years) for pertussis varied largely and ranged between 0.15 (95% CI: 0.12; 0.19) and 1.15 (95% CI: 1.07; 1.23), and the trends over time was consistent with those observed from national surveillance databases for confirmed pertussis. The pertussis IRs decreased as the number of wP and aP vaccine doses increased. Pertussis-related complications were rare (89 pneumonia, 7 generalised convulsions and no deaths) and their relative risk (vs. non-pertussis) could not be reliably estimated. The study demonstrated the feasibility of the ADVANCE system to estimate the change in pertussis IRs following pertussis vaccination. Larger sample sizes would provide additional power to compare the risk for complications between children with and without pertussis. The feasibility of vaccine-type specific effectiveness studies may be considered in the future.

ADVANCE system testing: Can safety studies be conducted using electronic healthcare data? An example using pertussis vaccination

INTRODUCTION

The Accelerated Development of Vaccine benefit-risk Collaboration in Europe (ADVANCE) public-private collaboration, aimed to develop and test a system for rapid benefit-risk monitoring of vaccines using healthcare databases in Europe. The objective of this proof-of-concept (POC) study was to test the feasibility of the ADVANCE system to generate incidence rates (IRs) per 1000 person-years and incidence rate ratios (IRRs) for risks associated with whole cell- (wP) and acellular- (aP) pertussis vaccines, occurring in event-specific risk windows in children prior to their pre-school-entry booster.

METHODS

The study population comprised almost 5.1 million children aged 1 month to <6 years vaccinated with wP or aP vaccines during the study period from 1 January 1990 to 31 December 2015. Data from two Danish hospital (H) databases (AUH and SSI) and five primary care (PC) databases from, UK (THIN and RCGP RSC), Spain (SIDIAP and BIFAP) and Italy (Pedianet) were analysed. Database-specific IRRs between risk vs. non-risk periods were estimated in a self-controlled case series study and pooled using random-effects meta-analyses.

RESULTS

The overall IRs were: fever, 58.2 (95% CI: 58.1; 58.3), 96.9 (96.7; 97.1) for PC DBs and 8.56 (8.5; 8.6) for H DBs; convulsions, 7.6 (95% CI: 7.6; 7.7), 3.55 (3.5; 3.6) for PC and 12.87 (12.8; 13) for H; persistent crying, 3.9 (95% CI: 3.8; 3.9) for PC, injection-site reactions, 2.2 (95% CI 2.1; 2.2) for PC, hypotonic hypo-responsive episode (HHE), 0.4 (95% CI: 0.4; 0.4), 0.6 (0.6; 0.6) for PC and 0.2 (0.2; 0.3) for H; and somnolence: 0.3 (95% CI: 0.3; 0.3) for PC. The pooled IRRs for persistent crying, fever, and ISR, adjusted for age and healthy vaccinee period were higher after wP vs. aP vaccination, and lower for convulsions, for all doses. The IRR for HHE was slightly lower for wP than aP, while wP was associated with somnolence only for dose 1 and dose 3 compared with aP.

CONCLUSIONS

The estimated IRs and IRRs were comparable with published data, therefore demonstrating that the ADVANCE system was able to combine several European healthcare databases to assess vaccine safety data for wP and aP vaccination.