Quantifying the impact of time-varying baseline risk adjustment in the self-controlled risk interval design

A comparison of four self-controlled study designs in an analysis of COVID-19 vaccines and myocarditis using five European databases

INTRODUCTION

The aim of this study was to assess the possible extent of bias due to violation of a core assumption (event-dependent exposures) when using self-controlled designs to analyse the association between COVID-19 vaccines and myocarditis.

METHODS

We used data from five European databases (Spain: BIFAP, FISABIO VID, and SIDIAP; Italy: ARS-Tuscany; England: CPRD Aurum) converted to the ConcePTION Common Data Model. Individuals who experienced both myocarditis and were vaccinated against COVID-19 between 1 September 2020 and the end of data availability in each country were included. We compared a self-controlled risk interval study (SCRI) using a pre-vaccination control window, an SCRI using a post-vaccination control window, a standard SCCS and an extension of the SCCS designed to handle violations of the assumption of event-dependent exposures.

RESULTS

We included 1,757 cases of myocarditis. For analyses of the first dose of the Pfizer vaccine, to which all databases contributed information, we found results consistent with a null effect in both of the SCRI and extended SCCS, but some indication of a harmful effect in a standard SCCS. For the second dose, we found evidence of a harmful association for all study designs, with relatively similar effect sizes (SCRI pre = 1.99, 1.40 - 2.82; SCRI post 2.13, 95 %CI - 1.43, 3.18; standard SCCS 1.79, 95 %CI 1.31 - 2.44, extended SCCS 1.52, 95 %CI = 1.08 - 2.15). Adjustment for calendar time did not change these conclusions. Findings using all designs were also consistent with a harmful effect following a second dose of the Moderna vaccine.

CONCLUSIONS

In the context of the known association between COVID-19 vaccines and myocarditis, we have demonstrated that two forms of SCRI and two forms of SCCS led to largely comparable results, possibly because of limited violation of the assumption of event-dependent exposures.

Methodological frontiers in vaccine safety: qualifying available evidence for rare events, use of distributed data networks to monitor vaccine safety issues, and monitoring the safety of pregnancy interventions

While vaccines are rigorously tested for safety and efficacy in clinical trials, these trials do not include enough subjects to detect rare adverse events, and they generally exclude special populations such as pregnant women. It is therefore necessary to conduct postmarketing vaccine safety assessments using observational data sources. The study of rare events has been enabled in through large linked databases and distributed data networks, in combination with development of case-centred methods. Distributed data networks necessitate common protocols, definitions, data models and analytics and the processes of developing and employing these tools are rapidly evolving. Assessment of vaccine safety in pregnancy is complicated by physiological changes, the challenges of mother-child linkage and the need for long-term infant follow-up. Potential sources of bias including differential access to and utilisation of antenatal care, immortal time bias, seasonal timing of pregnancy and unmeasured determinants of pregnancy outcomes have yet to be fully explored. Available tools for assessment of evidence generated in postmarketing studies may downgrade evidence from observational data and prioritise evidence from randomised controlled trials. However, real-world evidence based on real-world data is increasingly being used for safety assessments, and new tools for evaluating real-world evidence have been developed. The future of vaccine safety surveillance, particularly for rare events and in special populations, comprises the use of big data in single countries as well as in collaborative networks. This move towards the use of real-world data requires continued development of methodologies to generate and assess real world evidence.

Control yourself: ISPE-endorsed guidance in the application of self-controlled study designs in pharmacoepidemiology

PURPOSE

Consensus is needed on conceptual foundations, terminology and relationships among the various self-controlled "trigger" study designs that control for time-invariant confounding factors and target the association between transient exposures (potential triggers) and abrupt outcomes. The International Society for Pharmacoepidemiology (ISPE) funded a working group of ISPE members to develop guidance material for the application and reporting of self-controlled study designs, similar to Standards of Reporting Observational Epidemiology (STROBE). This first paper focuses on navigation between the types of self-controlled designs to permit a foundational understanding with guiding principles.

METHODS

We leveraged a systematic review of applications of these designs, that we term Self-controlled Crossover Observational PharmacoEpidemiologic (SCOPE) studies. Starting from first principles and using case examples, we reviewed outcome-anchored (case-crossover [CCO], case-time control [CTC], case-case-time control [CCTC]) and exposure-anchored (self-controlled case-series [SCCS]) study designs.

RESULTS

Key methodological features related to exposure, outcome and time-related concerns were clarified, and a common language and worksheet to facilitate the design of SCOPE studies is introduced.

CONCLUSIONS

Consensus on conceptual foundations, terminology and relationships among SCOPE designs will facilitate understanding and critical appraisal of published studies, as well as help in the design, analysis and review of new SCOPE studies. This manuscript is endorsed by ISPE.

Self-controlled case series studies: Just how rare does a rare non-recurrent outcome need to be?

The self-controlled case series method assumes that adverse outcomes arise according to a non-homogeneous Poisson process. This implies that it is applicable to independent recurrent outcomes. However, the self-controlled case series method may also be applied to unique, non-recurrent outcomes or first outcomes only, in the limit where these become rare. We investigate this rare outcome assumption when the self-controlled case series method is applied to non-recurrent outcomes. We study this requirement analytically and by simulation, and quantify what is meant by 'rare' in this context. In simulations we also apply the self-controlled risk interval design, a special case of the self-controlled case series design. To illustrate, we extract data on the incidence rate of some recurrent and non-recurrent outcomes within a defined study population to check whether outcomes are sufficiently rare for the rare outcome assumption to hold when applying the self-controlled case series method to first or unique outcomes. The main findings are that the relative bias should be no more than 5% when the cumulative incidence over total time observed is less than 0.1 per individual. Inclusion of age (or calendar time) effects will further reduce bias. Designs that begin observation with exposure maximise bias, whereas little or no bias will be apparent when there is no time trend in the distribution of exposures, or when exposure is central within time observed.

Development and application of two semi-automated tools for targeted medical product surveillance in a distributed data network

Purpose of Review

An important component of the Food and Drug Administration's Sentinel Initiative is the active post-market risk identification and analysis (ARIA) system, which utilizes semi-automated, parameterized computer programs to implement propensity-score adjusted and self-controlled risk interval designs to conduct targeted surveillance of medical products in the Sentinel Distributed Database. In this manuscript, we review literature relevant to the development of these programs and describe their application within the Sentinel Initiative.

Recent Findings

These quality-checked and publicly available tools have been successfully used to conduct rapid, replicable, and targeted safety analyses of several medical products. In addition to speed and reproducibility, use of semi-automated tools allows investigators to focus on decisions regarding key methodological parameters. We also identified challenges associated with the use of these methods in distributed and prospective datasets like the Sentinel Distributed Database, namely uncertainty regarding the optimal approach to estimating propensity scores in dynamic data among data partners of heterogeneous size.

Summary

Future research should focus on the methodological challenges raised by these applications as well as developing new modular programs for targeted surveillance of medical products.