Mini-Sentinel methods: framework for assessment of positive results from signal refinement

Novel methods for pregnancy drug safety surveillance in the FDA Sentinel System

PURPOSE

It is a priority of the US Food and Drug Administration (FDA) to monitor the safety of medications used during pregnancy. Pregnancy exposure registries and cohort studies utilizing electronic health record data are primary sources of information but are limited by small sample sizes and limited outcome assessment. TreeScan™, a statistical data mining tool, can be applied within the FDA Sentinel System to simultaneously identify multiple potential adverse neonatal and infant outcomes after maternal medication exposure.

METHODS

We implemented TreeScan using the Sentinel analytic tools in a cohort of linked live birth deliveries and infants nested in the IBM MarketScan® Research Database. As a case study, we compared first trimester fluoroquinolone use and cephalosporin use. We used the Bernoulli and Poisson TreeScan statistics with compatible propensity score-based study designs for confounding control (matching and stratification) and used multiple propensity score models with various strategies for confounding control to inform best practices. We developed a hierarchical outcome tree including major congenital malformations and outcomes of gestational length and birth weight.

RESULTS

A total of 1791 fluoroquinolone-exposed and 8739 cephalosporin-exposed mother-infant pairs were eligible for analysis. Both TreeScan analysis methods resulted in single alerts that were deemed to be due to uncontrolled confounding or otherwise not warranting follow-up.

CONCLUSIONS

In this implementation of TreeScan using Sentinel analytic tools, we did not observe any new safety signals for fluoroquinolone use in the first trimester. TreeScan, with tailored or high-dimensional propensity scores for confounding control, is a valuable tool in addition to current safety surveillance methods for medications used during pregnancy.

Monitoring Drug Safety in Pregnancy with Scan Statistics: A Comparison of Two Study Designs

BACKGROUND

Traditional surveillance of adverse infant outcomes following maternal medication exposures relies on pregnancy exposure registries, which are often underpowered. We characterize the statistical power of TreeScan, a data mining tool, to identify potential signals in the setting of perinatal medication exposures and infant outcomes.

METHODS

We used empirical data to inform background incidence of major congenital malformations and other birth conditions. Statistical power was calculated using two probability models compatible with TreeScan, Bernoulli and Poisson, while varying the sample size, magnitude of the risk increase, and incidence of a specified outcome. We also simulated larger referent to exposure matching ratios when using the Bernoulli model in the setting of fixed N:1 propensity score matching. Finally, we assessed the impact of outcome misclassification on power.

RESULTS

The Poisson model demonstrated greater power to detect signals than the Bernoulli model across all scenarios and suggested a sample size of 4,000 exposed pregnancies is needed to detect a twofold increase in risk of a common outcome (approximately 8 per 1,000) with 85% power. Increasing the fixed matching ratio with the Bernoulli model did not reliably increase power. An outcome definition with high sensitivity is expected to have somewhat greater power to detect signals than an outcome definition with high positive predictive value.

CONCLUSIONS

Use of the Poisson model with an outcome definition that prioritizes sensitivity may be optimal for signal detection. TreeScan is a viable method for surveillance of adverse infant outcomes following maternal medication use.

Active Vaccine Safety Surveillance: Global Trends and Challenges in China

Importance. The great success in vaccine-preventable diseases has been accompanied by vaccine safety concerns. This has caused vaccine hesitancy to be the top 10 in threats to global health. The comprehensive understanding of adverse events following immunization should be entirely based on clinical trials and postapproval surveillance. It has increasingly been recognized worldwide that the active surveillance of vaccine safety should be an essential part of immunization programs due to its complementary advantages to passive surveillance and clinical trials.Highlights. In the present study, the framework of vaccine safety surveillance was summarized to illustrate the importance of active surveillance and address vaccine hesitancy or safety concerns. Then, the global progress of active surveillance systems was reviewed, mainly focusing on population-based or hospital-based active surveillance. With these successful paradigms, the practical and reliable ways to create robust and similar systems in China were discussed and presented from the perspective of available databases, methodology challenges, policy supports, and ethical considerations.Conclusion. In the inevitable trend of the global vaccine safety ecosystem, the establishment of an active surveillance system for vaccine safety in China is urgent and feasible. This process can be accelerated with the consensus and cooperation of regulatory departments, research institutions, and data owners.

A Survey of Adverse Event Reporting Practices Among US Healthcare Professionals

Introduction

The under-reporting of adverse drug events (ADEs) is an international health concern. A number of studies have assessed the root causes but, to our knowledge, little information exists relating under-reporting to practices and systems used for the recording and tracking of drug‐related adverse event observations in ambulatory settings, institutional settings, and retail pharmacies.

Objectives

Our objective was to explore the process for reporting ADEs in US hospitals, ambulatory settings, and retail pharmacies; to explore gaps and inconsistencies in the reporting process; and to identify the causes of under-reporting ADEs in these settings.

Methods

The Tufts Center for the Study of Drug Development (Tufts CSDD) interviewed 11 thought leaders and conducted a survey between May and August 2014 among US-based healthcare providers (HCPs) in diverse settings to assess their experiences with, and processes for, reporting ADEs.

Results

A total of 123 individuals completed the survey (42 % were pharmacists; 27 % were nurses; 15 % were physicians; and 16 % were classified as ‘other’). HCPs indicated that the main reasons for under-reporting were difficulty in determining the cause of the ADE, given that most patients receive multiple therapies simultaneously (66 % of respondents); that HCPs lack sufficient time to report ADEs (63 % of respondents); poor integration of ADE-reporting systems (53 % of respondents); and uncertainty about reporting procedures (52 % of respondents).

Discussion

The results of this pilot study identify that key factors contributing to the under-reporting of ADEs relate to a lack of standardized process, a lack of training and education, and a lack of integrated health information technologies.

Electronic supplementary material

The online version of this article (doi:10.1007/s40264-016-0455-4) contains supplementary material, which is available to authorized users.

Social Media Listening for Routine Post-Marketing Safety Surveillance

INTRODUCTION

Post-marketing safety surveillance primarily relies on data from spontaneous adverse event reports, medical literature, and observational databases. Limitations of these data sources include potential under-reporting, lack of geographic diversity, and time lag between event occurrence and discovery. There is growing interest in exploring the use of social media ('social listening') to supplement established approaches for pharmacovigilance. Although social listening is commonly used for commercial purposes, there are only anecdotal reports of its use in pharmacovigilance. Health information posted online by patients is often publicly available, representing an untapped source of post-marketing safety data that could supplement data from existing sources.

OBJECTIVES

The objective of this paper is to describe one methodology that could help unlock the potential of social media for safety surveillance.

METHODS

A third-party vendor acquired 24 months of publicly available Facebook and Twitter data, then processed the data by standardizing drug names and vernacular symptoms, removing duplicates and noise, masking personally identifiable information, and adding supplemental data to facilitate the review process. The resulting dataset was analyzed for safety and benefit information.

RESULTS

In Twitter, a total of 6,441,679 Medical Dictionary for Regulatory Activities (MedDRA(®)) Preferred Terms (PTs) representing 702 individual PTs were discussed in the same post as a drug compared with 15,650,108 total PTs representing 946 individual PTs in Facebook. Further analysis revealed that 26 % of posts also contained benefit information.

CONCLUSION

Social media listening is an important tool to augment post-marketing safety surveillance. Much work remains to determine best practices for using this rapidly evolving data source.

Clinical phenotyping in selected national networks: demonstrating the need for high-throughput, portable, and computational methods

OBJECTIVE

The combination of phenomic data from electronic health records (EHR) and clinical data repositories with dense biological data has enabled genomic and pharmacogenomic discovery, a first step toward precision medicine. Computational methods for the identification of clinical phenotypes from EHR data will advance our understanding of disease risk and drug response, and support the practice of precision medicine on a national scale.

METHODS

Based on our experience within three national research networks, we summarize the broad approaches to clinical phenotyping and highlight the important role of these networks in the progression of high-throughput phenotyping and precision medicine. We provide supporting literature in the form of a non-systematic review.

RESULTS

The practice of clinical phenotyping is evolving to meet the growing demand for scalable, portable, and data driven methods and tools. The resources required for traditional phenotyping algorithms from expert defined rules are significant. In contrast, machine learning approaches that rely on data patterns will require fewer clinical domain experts and resources.

CONCLUSIONS

Machine learning approaches that generate phenotype definitions from patient features and clinical profiles will result in truly computational phenotypes, derived from data rather than experts. Research networks and phenotype developers should cooperate to develop methods, collaboration platforms, and data standards that will enable computational phenotyping and truly modernize biomedical research and precision medicine.

Quantitative Bias Analysis in Regulatory Settings

Nonrandomized studies are essential in the postmarket activities of the US Food and Drug Administration, which, however, must often act on the basis of imperfect data. Systematic errors can lead to inaccurate inferences, so it is critical to develop analytic methods that quantify uncertainty and bias and ensure that these methods are implemented when needed. "Quantitative bias analysis" is an overarching term for methods that estimate quantitatively the direction, magnitude, and uncertainty associated with systematic errors influencing measures of associations. The Food and Drug Administration sponsored a collaborative project to develop tools to better quantify the uncertainties associated with postmarket surveillance studies used in regulatory decision making. We have described the rationale, progress, and future directions of this project.

Data Extraction and Management in Networks of Observational Health Care Databases for Scientific Research: A Comparison of EU-ADR, OMOP, Mini-Sentinel and MATRICE Strategies

Introduction:

We see increased use of existing observational data in order to achieve fast and transparent production of empirical evidence in health care research. Multiple databases are often used to increase power, to assess rare exposures or outcomes, or to study diverse populations. For privacy and sociological reasons, original data on individual subjects can’t be shared, requiring a distributed network approach where data processing is performed prior to data sharing.

Case Descriptions and Variation Among Sites:

We created a conceptual framework distinguishing three steps in local data processing: (1) data reorganization into a data structure common across the network; (2) derivation of study variables not present in original data; and (3) application of study design to transform longitudinal data into aggregated data sets for statistical analysis. We applied this framework to four case studies to identify similarities and differences in the United States and Europe: Exploring and Understanding Adverse Drug Reactions by Integrative Mining of Clinical Records and Biomedical Knowledge (EU-ADR), Observational Medical Outcomes Partnership (OMOP), the Food and Drug Administration’s (FDA’s) Mini-Sentinel, and the Italian network—the Integration of Content Management Information on the Territory of Patients with Complex Diseases or with Chronic Conditions (MATRICE).

Findings:

National networks (OMOP, Mini-Sentinel, MATRICE) all adopted shared procedures for local data reorganization. The multinational EU-ADR network needed locally defined procedures to reorganize its heterogeneous data into a common structure. Derivation of new data elements was centrally defined in all networks but the procedure was not shared in EU-ADR. Application of study design was a common and shared procedure in all the case studies. Computer procedures were embodied in different programming languages, including SAS, R, SQL, Java, and C++.

Conclusion:

Using our conceptual framework we found several areas that would benefit from research to identify optimal standards for production of empirical knowledge from existing databases.an opportunity to advance evidence-based care management. In addition, formalized CM outcomes assessment methodologies will enable us to compare CM effectiveness across health delivery settings.