Post-licensure rapid immunization safety monitoring program (PRISM) data characterization

Establishment of Safety Monitoring System for Vaccines Not Included in the National Immunization Program in Korea

BACKGROUND

In Korea, there are no surveillance programs for vaccines that are not included in the national immunization program (NIP), and vaccine safety monitoring in the adult population is inadequate. This study aimed to establish a safety monitoring system for non-NIP vaccines in adults.

METHODS

Frequently administered non-NIP vaccines were selected. Individuals were included if they received at least one of the selected vaccines at a participating institution and provided informed consent. Solicited and unsolicited adverse events were monitored using questionnaires sent through text messages on days 1, 3, 7, 28, and 90 post-vaccination. Selected adverse events of special interest (AESIs) were monitored monthly by retrospective review of electronic medical records. Causality was assessed according to the Korea Disease Control and Prevention Agency guidelines.

RESULTS

Four vaccines (tetanus-diphtheria-pertussis [Tdap], pneumococcal conjugate 13-valent [PCV13], live zoster vaccine [ZVL], and recombinant zoster vaccine [RZV]) were selected, and their safety profiles were monitored at four tertiary hospitals and 10 primary care clinics. The response rates of the questionnaires on post-vaccination days 1, 7, 28, and 90 were 99.2%, 93.6%, 81.0%, and 48.7%, respectively. Of 555 AESI identified over 10 months, 10 cases received one of the selected non-NIP vaccines within 90 days of the event.

CONCLUSION

We are establishing the first safety monitoring system for selected non-NIP vaccines in Korea since September 2022 and report its progress as of July 2023. However, continuous government support is essential for its maintenance and improvement.

Bayesian safety surveillance with adaptive bias correction

Postmarket safety surveillance is an integral part of mass vaccination programs. Typically relying on sequential analysis of real-world health data as they accrue, safety surveillance is challenged by sequential multiple testing and by biases induced by residual confounding in observational data. The current standard approach based on the maximized sequential probability ratio test (MaxSPRT) fails to satisfactorily address these practical challenges and it remains a rigid framework that requires prespecification of the surveillance schedule. We develop an alternative Bayesian surveillance procedure that addresses both aforementioned challenges using a more flexible framework. To mitigate bias, we jointly analyze a large set of negative control outcomes that are adverse events with no known association with the vaccines in order to inform an empirical bias distribution, which we then incorporate into estimating the effect of vaccine exposure on the adverse event of interest through a Bayesian hierarchical model. To address multiple testing and improve on flexibility, at each analysis timepoint, we update a posterior probability in favor of the alternative hypothesis that vaccination induces higher risks of adverse events, and then use it for sequential detection of safety signals. Through an empirical evaluation using six US observational healthcare databases covering more than 360 million patients, we benchmark the proposed procedure against MaxSPRT on testing errors and estimation accuracy, under two epidemiological designs, the historical comparator and the self-controlled case series. We demonstrate that our procedure substantially reduces Type 1 error rates, maintains high statistical power and fast signal detection, and provides considerably more accurate estimation than MaxSPRT. Given the extensiveness of the empirical study which yields more than 7 million sets of results, we present all results in a public R ShinyApp. As an effort to promote open science, we provide full implementation of our method in the open-source R package EvidenceSynthesis.

Current Approaches to Vaccine Safety Using Observational Data: A Rationale for the EUMAEUS (Evaluating Use of Methods for Adverse Events Under Surveillance-for Vaccines) Study Design

Post-marketing vaccine safety surveillance aims to detect adverse events following immunization in a population. Whether certain methods of surveillance are more precise and unbiased in generating safety signals is unclear. Here, we synthesized information from existing literature to provide an overview of the strengths, weaknesses, and clinical applications of epidemiologic and analytical methods used in vaccine monitoring, focusing on cohort, case-control and self-controlled designs. These designs are proposed to be evaluated in the EUMAEUS (Evaluating Use of Methods for Adverse Event Under Surveillance-for vaccines) study because of their widespread use and potential utility. Over the past decades, there have been an increasing number of epidemiological study designs used for vaccine safety surveillance. While traditional cohort and case-control study designs remain widely used, newer, novel designs such as the self-controlled case series and self-controlled risk intervals have been developed. Each study design comes with its strengths and limitations, and the most appropriate study design will depend on availability of resources, access to records, number and distribution of cases, and availability of population coverage data. Several assumptions have to be made while using the various study designs, and while the goal is to mitigate any biases, violations of these assumptions are often still present to varying degrees. In our review, we discussed some of the potential biases (i.e., selection bias, misclassification bias and confounding bias), and ways to mitigate them. While the types of epidemiological study designs are well established, a comprehensive comparison of the analytical aspects (including method evaluation and performance metrics) of these study designs are relatively less well studied. We summarized the literature, reporting on two simulation studies, which compared the detection time, empirical power, error rate and risk estimate bias across the above-mentioned study designs. While these simulation studies provided insights on the analytic performance of each of the study designs, its applicability to real-world data remains unclear. To bridge that gap, we provided the rationale of the EUMAEUS study, with a brief description of the study design; and how the use of real-world multi-database networks can provide insights into better methods evaluation and vaccine safety surveillance.

Predicted and Observed Incidence of Thromboembolic Events among Koreans Vaccinated with ChAdOx1 nCoV-19 Vaccine

We used the nationwide claims database to calculate the incidence of thrombotic events and predict their overall 2-week incidence. From 2006 to 2020, the incidence of deep vein thrombosis (DVT), pulmonary embolism (PE), and disseminated intravascular coagulation (DIC) tended to increase. Unlike intracranial venous thrombosis (ICVT) and intracranial thrombophlebitis (ICTP), which showed no age difference, other venous embolism, and thrombosis (OVET), DIC, DVT, and PE were significantly more common in over 65 years. The overall 2-week incidence of ICVT was 0.21/1,000,000 (95% confidence interval [CI], 0.11-0.32). ICTP, OVET, DIC, DVT and PE were expected to occur in 0.08 (95% CI, 0.02-0.14), 7.66 (95% CI, 6.08-9.23), 5.95 (95% CI, 4.88-7.03), 13.28 (95% CI, 11.92-14.64), 14.09 (95% CI, 12.80-15.37) per 1,000,000, respectively. To date, of 8,548,231 patients vaccinated with ChAdOx1 nCoV-19 in Korea, two had confirmed thrombosis with thrombocytopenia syndrome within 2 weeks. The observed incidence of ICVT after vaccination was 0.23/1,000,000.

Characterising the background incidence rates of adverse events of special interest for covid-19 vaccines in eight countries: multinational network cohort study

OBJECTIVE

To quantify the background incidence rates of 15 prespecified adverse events of special interest (AESIs) associated with covid-19 vaccines.

DESIGN

Multinational network cohort study.

SETTING

Electronic health records and health claims data from eight countries: Australia, France, Germany, Japan, the Netherlands, Spain, the United Kingdom, and the United States, mapped to a common data model.

PARTICIPANTS

126 661 070 people observed for at least 365 days before 1 January 2017, 2018, or 2019 from 13 databases.

MAIN OUTCOME MEASURES

Events of interests were 15 prespecified AESIs (non-haemorrhagic and haemorrhagic stroke, acute myocardial infarction, deep vein thrombosis, pulmonary embolism, anaphylaxis, Bell's palsy, myocarditis or pericarditis, narcolepsy, appendicitis, immune thrombocytopenia, disseminated intravascular coagulation, encephalomyelitis (including acute disseminated encephalomyelitis), Guillain-Barré syndrome, and transverse myelitis). Incidence rates of AESIs were stratified by age, sex, and database. Rates were pooled across databases using random effects meta-analyses and classified according to the frequency categories of the Council for International Organizations of Medical Sciences.

RESULTS

Background rates varied greatly between databases. Deep vein thrombosis ranged from 387 (95% confidence interval 370 to 404) per 100 000 person years in UK CPRD GOLD data to 1443 (1416 to 1470) per 100 000 person years in US IBM MarketScan Multi-State Medicaid data among women aged 65 to 74 years. Some AESIs increased with age. For example, myocardial infarction rates in men increased from 28 (27 to 29) per 100 000 person years among those aged 18-34 years to 1400 (1374 to 1427) per 100 000 person years in those older than 85 years in US Optum electronic health record data. Other AESIs were more common in young people. For example, rates of anaphylaxis among boys and men were 78 (75 to 80) per 100 000 person years in those aged 6-17 years and 8 (6 to 10) per 100 000 person years in those older than 85 years in Optum electronic health record data. Meta-analytic estimates of AESI rates were classified according to age and sex.

CONCLUSION

This study found large variations in the observed rates of AESIs by age group and sex, showing the need for stratification or standardisation before using background rates for safety surveillance. Considerable population level heterogeneity in AESI rates was found between databases.

Estimating Baseline Incidence of Conditions Potentially Associated with Vaccine Adverse Events: a Call for Surveillance System Using the Korean National Health Insurance Claims Data

BACKGROUND

Vaccines against coronavirus disease 2019 (COVID-19) are raising concerns about vaccine safety, particularly in the context of large-scale immunization. To address public concerns, we measured the baseline incidence rates of major conditions potentially related to vaccine-related adverse events (VAEs). We aimed to provide a basis for evaluating VAEs and verifying causality.

METHODS

Conditions of interest were selected from the US Vaccine Adverse Event Reporting System Table of Reportable Events and a recent report from a European consortium on vaccine surveillance. We used the National Health Insurance Service database in Korea to identify the monthly numbers of cases with these conditions. Data from January 2006 to June 2020 were included. Prediction models were constructed from the observed incidences using an autoregressive integrated moving average. We predicted the incidences of the conditions and their respective 95% confidence intervals (CIs) for January through December 2021. In addition, subgroup analysis for the expected vaccination population was conducted.

RESULTS

Mean values (95% CIs) of the predicted monthly incidence of vasovagal syncope, anaphylaxis, brachial neuritis, acute disseminated encephalomyelitis, Bell's palsy, Guillain-Barré syndrome, encephalopathy, optic neuritis, transverse myelitis, immune thrombocytopenic purpura, and systemic lupus erythematosus in 2021 were 23.89 (19.81-27.98), 4.72 (3.83-5.61), 57.62 (51.37-63.88), 0.03 (0.01-0.04), 8.58 (7.90-9.26), 0.26 (0.18-0.34), 2.13 (1.42-2.83), 1.65 (1.17-2.13), 0.19 (0.14-0.25), 0.75 (0.61-0.90), and 3.40 (2.79-4.01) cases per 100,000 respectively. The majority of the conditions showed an increasing trend with seasonal variations in their incidences.

CONCLUSION

We measured the incidence of a total of 11 conditions that could potentially be associated with VAEs to predict the monthly incidence in 2021. In Korea, conditions that could potentially be related to VAEs occur on a regular basis, and an increasing trend is observed with seasonality.

Incidence Rates of Autoimmune Diseases in European Healthcare Databases: A Contribution of the ADVANCE Project

INTRODUCTION

The public-private ADVANCE collaboration developed and tested a system to generate evidence on vaccine benefits and risks using European electronic healthcare databases. In the safety of vaccines, background incidence rates are key to allow proper monitoring and assessment. The goals of this study were to compute age-, sex-, and calendar-year stratified incidence rates of nine autoimmune diseases in seven European healthcare databases from four countries and to assess validity by comparing with published data.

METHODS

Event rates were calculated for the following outcomes: acute disseminated encephalomyelitis, Bell's palsy, Guillain-Barré syndrome, immune thrombocytopenia purpura, Kawasaki disease, optic neuritis, narcolepsy, systemic lupus erythematosus, and transverse myelitis. Cases were identified by diagnosis codes. Participating organizations/databases originated from Denmark, Italy, Spain, and the UK. The source population comprised all persons registered, with at least 1 year of data prior to the study start, or follow-up from birth. Stratified incidence rates were computed per database over the period 2003 to 2014.

RESULTS

Between 2003 and 2014, 148,947 incident cases of nine autoimmune diseases were identified. Crude incidence rates were highest for Bell's palsy [23.8/100,000 person-years (PYs), 95% confidence interval (CI) 23.6-24.1] and lowest for Kawasaki disease (0.7/100,000 PYs, 95% CI 0.6-0.7). Specific patterns were observed by sex, age, calendar time, and data sources. Rates were comparable with published estimates.

CONCLUSION

A range of autoimmune events could be identified in the ADVANCE system. Estimation of rates indicated consistency across selected European healthcare databases, as well as consistency with US published data.

Real-World Evidence for Assessing Pharmaceutical Treatments in the Context of COVID-19

The emergence and global spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has resulted in an urgent need for evidence on medical interventions and outcomes of the resulting disease, coronavirus disease 2019 (COVID-19). Although many randomized controlled trials (RCTs) evaluating treatments and vaccines for COVID-19 are already in progress, the number of clinical questions of interest greatly outpaces the available resources to conduct RCTs. Therefore, there is growing interest in whether nonrandomized real-world evidence (RWE) can be used to supplement RCT evidence and aid in clinical decision making, but concerns about nonrandomized RWE have been highlighted by a proliferation of RWE studies on medications and COVID-19 outcomes with widely varying conclusions. The objective of this paper is to review some clinical questions of interest, potential data types, challenges, and merits of RWE in COVID-19, resulting in recommendations for nonrandomized RWE designs and analyses based on established RWE principles.

ADVANCE database characterisation and fit for purpose assessment for multi-country studies on the coverage, benefits and risks of pertussis vaccinations

INTRODUCTION

The public-private ADVANCE consortium (Accelerated development of vaccine benefit-risk collaboration in Europe) aimed to assess if electronic healthcare databases can provide fit-for purpose data for collaborative, distributed studies and monitoring of vaccine coverage, benefits and risks of vaccines.

OBJECTIVE

To evaluate if European healthcare databases can be used to estimate vaccine coverage, benefit and/or risk using pertussis-containing vaccines as an example.

METHODS

Characterisation was conducted using open-source Java-based (Jerboa) software and R scripts. We obtained: (i) The general characteristics of the database and data source (meta-data) and (ii) a detailed description of the database population (size, representatively of age/sex of national population, rounding of birth dates, delay between birth and database entry), vaccinations (number of vaccine doses, recording of doses, pattern of doses by age and coverage) and events of interest (diagnosis codes, incidence rates). A total of nine databases (primary care, regional/national record linkage) provided data on events (pertussis, pneumonia, death, fever, convulsions, injection site reactions, hypotonic hypo-responsive episode, persistent crying) and vaccines (acellular pertussis and whole cell pertussis) related to the pertussis proof of concept studies.

RESULTS

The databases contained data for a total population of 44 million individuals. Seven databases had recorded doses of vaccines. The pertussis coverage estimates were similar to those reported by the World Health Organisation (WHO). Incidence rates of events were comparable in magnitude and age-distribution between databases with the same characteristics. Several conditions (persistent crying and somnolence) were not captured by the databases for which outcomes were restricted to hospital discharge diagnoses.

CONCLUSION

The database characterisation programs and workflows allowed for an efficient, transparent and standardised description and verification of electronic healthcare databases which may participate in pertussis vaccine coverage, benefit and risk studies. This approach is ready to be used for other vaccines/events to create readiness for participation in other vaccine related studies.

Big Data in the Assessment of Pediatric Medication Safety

Big data (BD) in pediatric medication safety research provides many opportunities to improve the safety and health of children. The number of pediatric medication and device trials has increased in part because of the past 20 years of US legislation requiring and incentivizing study of the effects of medical products in children (Food and Drug Administration Modernization Act of 1997, Pediatric Rule in 1998, Best Pharmaceuticals for Children Act of 2002, and Pediatric Research Equity Act of 2003). There are some limitations of traditional approaches to studying medication safety in children. Randomized clinical trials within the regulatory context may not enroll patients who are representative of the general pediatric population, provide the power to detect rare safety signals, or provide long-term safety data. BD sources may have these capabilities. In recent years, medical records have become digitized, and cell phones and personal devices have proliferated. In this process, the field of biomedical science has progressively used BD from those records coupled with other data sources, both digital and traditional. Additionally, large distributed databases that include pediatric-specific outcome variables are available. A workshop entitled "Advancing the Development of Pediatric Therapeutics: Application of 'Big Data' to Pediatric Safety Studies" held September 18 to 19, 2017, in Silver Spring, Maryland, formed the basis of many of the ideas outlined in this article, which are intended to identify key examples, critical issues, and future directions in this early phase of an anticipated dramatic change in the availability and use of BD.

ADVANCE system testing: Can coverage of pertussis vaccination be estimated in European countries using electronic healthcare databases: An example

INTRODUCTION

The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines, using existing healthcare databases in Europe. The objective of this paper was to assess the feasibility of using electronic healthcare databases to estimate dose-specific acellular pertussis (aP) and whole cell pertussis (wP) vaccine coverage.

METHODS

Seven electronic healthcare databases in four European countries (Denmark (n = 2), UK (n = 2), Spain (n = 2) and Italy (n = 1)) participated in this study. Children were included from birth and followed up to age six years. Vaccination exposure was obtained from the databases and classified by type (aP or wP), and dose 1, 2 or 3. Coverage was estimated using period prevalence. For the 2006 birth cohort, two estimation methods for pertussis vaccine coverage, period prevalence and cumulative incidence were compared for each database.

RESULTS

The majority of the 2,575,576 children included had been vaccinated at the country-specific recommended ages. Overall, the estimated dose 3 coverage was 88-97% in Denmark (birth cohorts from 2003 to 2014), 96-100% in the UK (2003-2014), 95-98% in Spain (2004-2014) and 94% in Italy (2006-2007). The estimated dose 3 coverage per birth cohort in Denmark and the UK differed by 1-6% compared with national estimates, with our estimates mostly higher. The estimated dose 3 coverage in Spain differed by 0-2% with no consistent over- or underestimation. In Italy, the estimates were 3% lower compared with the national estimates. Except for Italy, for which the two coverage estimation methods generated the same results, the estimated cumulative incidence coverages were consistently 1-10% lower than period prevalence estimates.

CONCLUSION

This study showed that it was possible to provide consistent estimates of pertussis immunisation coverage from the electronic healthcare databases included, and that the estimates were comparable with the national estimates.

Estimation of vaccination coverage from electronic healthcare records; methods performance evaluation - A contribution of the ADVANCE-project

INTRODUCTION

The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines, using existing electronic healthcare record (eHR) databases in Europe. Part of the data in such sources is missing due to incomplete follow-up hampering the accurate estimation of vaccination coverage. We compared different methods for coverage estimation from eHR databases; naïve period prevalence, complete case period prevalence, period prevalence adjusted for follow-up time, Kaplan-Meier (KM) analysis and (adjusted) inverse probability weighing (IPW).

METHODS

We created simulation scenarios with different proportions of completeness of follow-up. Both completeness independent and dependent from vaccination date and status were considered. The root mean squared error (RMSE) and relative difference between the estimated and true coverage were used to assess the performance of the different methods for each of the scenarios. We included data examples on the vaccination coverage of human papilloma virus and pertussis component containing vaccines from the Spanish BIFAP database.

RESULTS

Under completeness independent from vaccination date or status, several methods provided estimates with bias close to zero. However, when dependence between completeness of follow-up and vaccination date or status was present, all methods generated biased estimates. The IPW/CDF methods were generally the least biased. Preference for a specific method should be based on the type of censoring and type of dependence between completeness of follow-up and vaccination. Additional insights into these aspects, might be gained by applying several methods.

Kawasaki disease and 13-valent pneumococcal conjugate vaccination among young children: A self-controlled risk interval and cohort study with null results

BACKGROUND

Kawasaki disease is an acute vasculitis that primarily affects children younger than 5 years of age. Its etiology is unknown. The United States Vaccine Safety Datalink conducted postlicensure safety surveillance for 13-valent pneumococcal conjugate vaccine (PCV13), comparing the risk of Kawasaki disease within 28 days of PCV13 vaccination with the historical risk after 7-valent PCV (PCV7) vaccination and using chart-validation. A relative risk (RR) of 2.38 (95% CI 0.92-6.38) was found. Concurrently, the Food and Drug Administration (FDA) conducted a postlicensure safety review that identified cases of Kawasaki disease through adverse event reporting. The FDA decided to initiate a larger study of Kawasaki disease risk following PCV13 vaccination in the claims-based Sentinel/Postlicensure Rapid Immunization Safety Monitoring (PRISM) surveillance system. The objective of this study was to determine the existence and magnitude of any increased risk of Kawasaki disease in the 28 days following PCV13 vaccination.

METHODS AND FINDINGS

The study population included mostly commercially insured children from birth to <24 months of age in 2010 to 2015 from across the US. Using claims data of participating Sentinel/PRISM data-providing organizations, PCV13 vaccinations were identified by means of current procedural terminology (CPT), Healthcare Common Procedure Coding System (HCPCS), and National Drug Code (NDC) codes. Potential cases of Kawasaki disease were identified by first-in-365-days International Classification of Diseases 9th revision (ICD-9) code 446.1 or International Classification of Diseases 10th revision (ICD-10) code M30.3 in the inpatient setting. Medical records were sought for potential cases and adjudicated by board-certified pediatricians. The primary analysis used chart-confirmed cases with adjudicated symptom onset in a self-controlled risk interval (SCRI) design, which controls for time-invariant potential confounders. The prespecified risk interval was Days 1-28 after vaccination; a 28-day-long control interval followed this risk interval. A secondary analytic approach used a cohort design, with alternative potential risk intervals of Days 1-28 and Days 1-42. The varying background risk of Kawasaki disease by age was adjusted for in both designs. In the primary analysis, there were 43 confirmed cases of Kawasaki disease in the risk interval and 44 in the control interval. The age-adjusted risk estimate was 1.07 (95% CI 0.70-1.63; p = 0.76). In the secondary, cohort analyses, which included roughly 700 potential cases and more than 3 million person-years, the risk estimates of potential Kawasaki disease in the risk interval versus in unexposed person-time were 0.84 (95% CI 0.65-1.08; p = 0.18) for the Days 1-28 risk interval and 0.97 (95% CI 0.79-1.19; p = 0.80) for the Days 1-42 risk interval. The main limitation of the study was that we lacked the resources to conduct medical record review for all the potential cases of Kawasaki disease. As a result, potential cases rather than chart-confirmed cases were used in the cohort analyses.

CONCLUSIONS

With more than 6 million doses of PCV13 administered, no evidence was found of an association between PCV13 vaccination and Kawasaki disease onset in the 4 weeks after vaccination nor of an elevated risk extending or concentrated somewhat beyond 4 weeks. These null results were consistent across alternative designs, age-adjustment methods, control intervals, and categories of Kawasaki disease case included.

Countering vaccine hesitancy through immunization information systems, a narrative review

Immunization is one of the most important public health interventions to contrast infectious disease; however, many people nowadays refuse vaccination. Vaccine hesitancy (VH) is due to several factors that influence the complex decision-making process. Information technology tools might play an important role in vaccination programs. In particular, immunization information systems (IISs) have the potential to improve performance of vaccination programs and to increase vaccine uptake. This review aimed to present IIS functionalities in order to counter VH. In detail, we analyzed the automatic reminder/recall system, the interoperability of the system, the decision support system, the web page interface and the possibility to record adverse events following immunization. IIS could concretely represent a valid instrument to increase vaccine confidence, especially trust in both health-care workers and decision makers. There are not enough trials aimed to evaluate the efficacy of IIS to counter VH. Further researches might focalize on this aspect.

Defining the interval for monitoring potential adverse events following immunization (AEFIs) after receipt of live viral vectored vaccines

Live viral vectors that express heterologous antigens of the target pathogen are being investigated in the development of novel vaccines against serious infectious agents like HIV and Ebola. As some live recombinant vectored vaccines may be replication-competent, a key challenge is defining the length of time for monitoring potential adverse events following immunization (AEFI) in clinical trials and epidemiologic studies. This time period must be chosen with care and based on considerations of pre-clinical and clinical trials data, biological plausibility and practical feasibility. The available options include: (1) adapting from the current relevant regulatory guidelines; (2) convening a panel of experts to review the evidence from a systematic literature search to narrow down a list of likely potential or known AEFI and establish the optimal risk window(s); and (3) conducting "near real-time" prospective monitoring for unknown clustering's of AEFI in validated large linked vaccine safety databases using Rapid Cycle Analysis for pre-specified adverse events of special interest (AESI) and Treescan to identify previously unsuspected outcomes. The risk window established by any of these options could be used along with (4) establishing a registry of clinically validated pre-specified AESI to include in case-control studies. Depending on the infrastructure, human resources and databases available in different countries, the appropriate option or combination of options can be determined by regulatory agencies and investigators.

Incidence of outcomes relevant to vaccine safety monitoring in a US commercially-insured population

BACKGROUND

Background incidence rates (IRs) of potential safety outcomes among vaccine eligible individuals can inform assessment of vaccine safety. Vaccine safety surveillance often uses claims databases, but the impact of outcome definitions on background IR estimates is largely unexplored. Using two definitions for each outcome, we estimated background IRs of 32 cardiac, metabolic, allergic, autoimmune, neurologic, hematologic and nephrologic outcomes among individuals eligible to receive pneumococcal vaccination.

METHODS

We defined a cohort of individuals aged 6-100 years in US commercial health plans who had ≥12 months of health plan enrollment between January 2007 and August 2014 and no previous record of conjugate or simple polysaccharide pneumococcal vaccination. We developed a sensitive and a specific definition for each outcome, with the specific definition requiring evidence of additional care consistent with the outcome. IRs per 100,000 person-years for each outcome were presented overall and stratified by age, gender, and invasive pneumococcal disease (IPD) risk category.

RESULTS

We followed 19.9 million individuals for a median of 2.5 years. Wide variation was seen in IRs across different definitions of the 32 outcomes, with 19 (59%) outcomes having a specific definition IR less than half of the sensitive definition IR. IRs were particularly variable by definition for outcomes categorized as either hematologic/nephrologic or neurologic (mean ratio of specific IR to sensitive IR = 0.26 and 0.30, respectively). Across definitions, the IRs of the 32 outcomes were often highest in females, adults ≥65, and those at higher IPD risk.

CONCLUSIONS

Background IRs of safety outcomes relevant to populations indicated for pneumococcal vaccine varied by outcome definitions and population subgroups in this large US commercially-insured population. Given large differences in estimated IRs using sensitive versus specific case definitions, neurologic, and hematologic/nephrologic safety outcomes as compared to allergic and autoimmune outcomes may warrant more refined definitions and medical record validation.

Bias from outcome misclassification in immunization schedule safety research

PURPOSE

The Institute of Medicine recommended conducting observational studies of childhood immunization schedule safety. Such studies could be biased by outcome misclassification, leading to incorrect inferences. Using simulations, we evaluated (1) outcome positive predictive values (PPVs) as indicators of bias of an exposure-outcome association, and (2) quantitative bias analyses (QBA) for bias correction.

METHODS

Simulations were conducted based on proposed or ongoing Vaccine Safety Datalink studies. We simulated 4 studies of 2 exposure groups (children with no vaccines or on alternative schedules) and 2 baseline outcome levels (100 and 1000/100 000 person-years), with 3 relative risk (RR) levels (RR = 0.50, 1.00, and 2.00), across 1000 replications using probabilistic modeling. We quantified bias from non-differential and differential outcome misclassification, based on levels previously measured in database research (sensitivity > 95%; specificity > 99%). We calculated median outcome PPVs, median observed RRs, Type 1 error, and bias-corrected RRs following QBA.

RESULTS

We observed PPVs from 34% to 98%. With non-differential misclassification and true RR = 2.00, median bias was toward the null, with severe bias (median observed RR = 1.33) with PPV = 34% and modest bias (median observed RR = 1.83) with PPV = 83%. With differential misclassification, PPVs did not reflect median bias, and there was Type 1 error of 100% with PPV = 90%. QBA was generally effective in correcting misclassification bias.

CONCLUSIONS

In immunization schedule studies, outcome misclassification may be non-differential or differential to exposure. Overall outcome PPVs do not reflect the distribution of false positives by exposure and are poor indicators of bias in individual studies. Our results support QBA for immunization schedule safety research.

Innovative Digital Tools and Surveillance Systems for the Timely Detection of Adverse Events at the Point of Care: A Proof-of-Concept Study

INTRODUCTION AND OBJECTIVE

Regulatory authorities often receive poorly structured safety reports requiring considerable effort to investigate potential adverse events post hoc. Automated question-and-answer systems may help to improve the overall quality of safety information transmitted to pharmacovigilance agencies. This paper explores the use of the VACC-Tool (ViVI Automated Case Classification Tool) 2.0, a mobile application enabling physicians to classify clinical cases according to 14 pre-defined case definitions for neuroinflammatory adverse events (NIAE) and in full compliance with data standards issued by the Clinical Data Interchange Standards Consortium.

METHODS

The validation of the VACC-Tool 2.0 (beta-version) was conducted in the context of a unique quality management program for children with suspected NIAE in collaboration with the Robert Koch Institute in Berlin, Germany. The VACC-Tool was used for instant case classification and for longitudinal follow-up throughout the course of hospitalization. Results were compared to International Classification of Diseases , Tenth Revision (ICD-10) codes assigned in the emergency department (ED).

RESULTS

From 07/2013 to 10/2014, a total of 34,368 patients were seen in the ED, and 5243 patients were hospitalized; 243 of these were admitted for suspected NIAE (mean age: 8.5 years), thus participating in the quality management program. Using the VACC-Tool in the ED, 209 cases were classified successfully, 69 % of which had been missed or miscoded in the ED reports. Longitudinal follow-up with the VACC-Tool identified additional NIAE.

CONCLUSION

Mobile applications are taking data standards to the point of care, enabling clinicians to ascertain potential adverse events in the ED setting and during inpatient follow-up. Compliance with Clinical Data Interchange Standards Consortium (CDISC) data standards facilitates data interoperability according to regulatory requirements.

CodeMapper: semiautomatic coding of case definitions. A contribution from the ADVANCE project

BACKGROUND

Assessment of drug and vaccine effects by combining information from different healthcare databases in the European Union requires extensive efforts in the harmonization of codes as different vocabularies are being used across countries. In this paper, we present a web application called CodeMapper, which assists in the mapping of case definitions to codes from different vocabularies, while keeping a transparent record of the complete mapping process.

METHODS

CodeMapper builds upon coding vocabularies contained in the Metathesaurus of the Unified Medical Language System. The mapping approach consists of three phases. First, medical concepts are automatically identified in a free-text case definition. Second, the user revises the set of medical concepts by adding or removing concepts, or expanding them to related concepts that are more general or more specific. Finally, the selected concepts are projected to codes from the targeted coding vocabularies. We evaluated the application by comparing codes that were automatically generated from case definitions by applying CodeMapper's concept identification and successive concept expansion, with reference codes that were manually created in a previous epidemiological study.

RESULTS

Automated concept identification alone had a sensitivity of 0.246 and positive predictive value (PPV) of 0.420 for reproducing the reference codes. Three successive steps of concept expansion increased sensitivity to 0.953 and PPV to 0.616.

CONCLUSIONS

Automatic concept identification in the case definition alone was insufficient to reproduce the reference codes, but CodeMapper's operations for concept expansion provide an effective, efficient, and transparent way for reproducing the reference codes.

Public Health Surveillance Systems: Recent Advances in Their Use and Evaluation

Surveillance is critical for improving population health. Public health surveillance systems generate information that drives action, and the data must be of sufficient quality and with a resolution and timeliness that matches objectives. In the context of scientific advances in public health surveillance, changing health care and public health environments, and rapidly evolving technologies, the aim of this article is to review public health surveillance systems. We consider their current use to increase the efficiency and effectiveness of the public health system, the role of system stakeholders, the analysis and interpretation of surveillance data, approaches to system monitoring and evaluation, and opportunities for future advances in terms of increased scientific rigor, outcomes-focused research, and health informatics.

Countering Vaccine Hesitancy

Immunizations have led to a significant decrease in rates of vaccine-preventable diseases and have made a significant impact on the health of children. However, some parents express concerns about vaccine safety and the necessity of vaccines. The concerns of parents range from hesitancy about some immunizations to refusal of all vaccines. This clinical report provides information about addressing parental concerns about vaccination.

Vaccine monitoring systems: A potential model for medications in pregnancy

Multiple vaccine safety systems contribute to monitor and assess the safety of vaccines given to pregnant women and their offspring. This article presents a review of the strengths and limitations of several national vaccine safety systems. The review concludes that the present framework of vaccine safety systems offers lessons to be learned toward the design of a system for monitoring and assessing the safety of medications administered to pregnant women in clinical practice and research.

Febrile Seizures After 2010-2011 Trivalent Inactivated Influenza Vaccine

OBJECTIVES

In the Post-Licensure Rapid Immunization Safety Monitoring Program, we examined risk of febrile seizures (FS) after trivalent inactivated influenza vaccine (TIV) and 13-valent pneumococcal conjugate vaccine (PCV13) during the 2010-2011 influenza season, adjusted for concomitant diphtheria tetanus acellular pertussis-containing vaccines (DTaP). Assuming children would receive both vaccines, we examined whether same-day TIV and PCV13 vaccination was associated with greater FS risk when compared with separate-day vaccination.

METHODS

We used a self-controlled risk interval design, comparing the FS rate in a risk interval (0-1 days) versus control interval (14-20 days). Vaccinations were identified in claims and immunization registry data. FS were confirmed with medical records.

RESULTS

No statistically significant TIV-FS associations were found in unadjusted or adjusted models (incidence rate ratio [IRR] adjusted for age, seasonality, and concomitant PCV13 and DTaP: 1.36, 95% confidence interval [CI] 0.78 to 2.39). Adjusted for age and seasonality, PCV13 was significantly associated with FS (IRR 1.74, 95% CI 1.06 to 2.86), but not when further adjusting for concomitant TIV and DTaP (IRR 1.61, 95% CI 0.91 to 2.82). Same-day TIV and PCV13 vaccination was not associated with excess risk of FS when compared with separate-day vaccination (1.08 fewer FS per 100 000 with same day administration, 95% CI -5.68 to 6.09).

CONCLUSIONS

No statistically significant increased risk of FS was found for 2010-2011 TIV or PCV13, when adjusting for concomitant vaccines. Same-day TIV and PCV13 vaccination was not associated with more FS compared with separate-day vaccination.

Big Data Usage Patterns in the Health Care Domain: A Use Case Driven Approach Applied to the Assessment of Vaccination Benefits and Risks. Contribution of the IMIA Primary Healthcare Working Group

BACKGROUND

Generally benefits and risks of vaccines can be determined from studies carried out as part of regulatory compliance, followed by surveillance of routine data; however there are some rarer and more long term events that require new methods. Big data generated by increasingly affordable personalised computing, and from pervasive computing devices is rapidly growing and low cost, high volume, cloud computing makes the processing of these data inexpensive.

OBJECTIVE

To describe how big data and related analytical methods might be applied to assess the benefits and risks of vaccines.

METHOD

We reviewed the literature on the use of big data to improve health, applied to generic vaccine use cases, that illustrate benefits and risks of vaccination. We defined a use case as the interaction between a user and an information system to achieve a goal. We used flu vaccination and pre-school childhood immunisation as exemplars.

RESULTS

We reviewed three big data use cases relevant to assessing vaccine benefits and risks: (i) Big data processing using crowdsourcing, distributed big data processing, and predictive analytics, (ii) Data integration from heterogeneous big data sources, e.g. the increasing range of devices in the "internet of things", and (iii) Real-time monitoring for the direct monitoring of epidemics as well as vaccine effects via social media and other data sources.

CONCLUSIONS

Big data raises new ethical dilemmas, though its analysis methods can bring complementary real-time capabilities for monitoring epidemics and assessing vaccine benefit-risk balance.