Useful Interplay Between Spontaneous ADR Reports and Electronic Healthcare Records in Signal Detection

Interplay of Spontaneous Reporting and Longitudinal Healthcare Databases for Signal Management: Position Statement from the Real-World Evidence and Big Data Special Interest Group of the International Society of Pharmacovigilance

Signal management, defined as the set of activities from signal detection to recommendations for action, is conducted using different data sources and leveraging data from spontaneous reporting databases (SRDs), which represent the cornerstone of pharmacovigilance. However, the exponentially increasing generation and availability of real-world data collected in longitudinal healthcare databases (LHDs), along with the rapid evolution of artificial intelligence-based algorithms and other advanced analytical methods, offers a wide range of opportunities to complement SRDs throughout all stages of signal management, especially signal detection. Integrating information derived from SRDs and LHDs may reduce their respective limitations, thus potentially enhancing post-marketing surveillance. The aim of this position statement is to critically evaluate the complementary role of SRDs and LHDs in signal management, exploring the potential benefits and challenges in integrating information coming from these two data sources. Furthermore, we presented successful cases of the interplay between SRDs and LHDs for signal management, along with future opportunities and directions to improve such interplay.

A pharmacovigilance analysis of abrocitinib-related skin adverse events based on the FDA Adverse Event Reporting System (FAERS)

In January 2022, the U.S. Food and Drug Administration (FDA) approved the marketing of abrocitinib, an oral small molecule selectively inhibiting Janus kinase, for the treatment of patients with recurrent moderate to severe atopic dermatitis. Despite the lack of long-term post-marketing safety studies, this drug was cited with a black box warning by the FDA for potentially increasing the risk of several severe adverse events (AEs). This retrospective pharmacovigilance disproportionality analysis study used data from the FDA Adverse Event Reporting System (FAERS) from the first quarter of 2022 to the first quarter of 2024, aiming to analyze the potential association between abrocitinib and skin AEs. Three disproportionality measurement were performed for data mining, including the reporting odd ratio method, the proportional reporting ratio method and the Medicines and Healthcare products Regulatory Agency method. Out of 3,269,835 AE reports extracted from the database, 699 cases of skin-related AEs were identified, where abrocitinib was implicated as the primary suspect drug. The patient demographic information, outcomes and report sources were analyzed. 34 kinds of positive risk signals were detected, many of which were unexpected safety signals. This study provided a scientific reference to understand the safety of abrocitinib in practical applications.

Disproportionality analysis of Raynaud's phenomenon associated with calcitonin gene-related peptide inhibitors using the Food and Drug Administration adverse event reporting system

Raynaud's phenomenon is a vascular condition characterized by episodic vasoconstriction, and recent reports suggest a potential link between calcitonin gene-related peptide (CGRP) inhibitors, used for migraine treatment, and the onset of this condition. This study evaluated the association between CGRP inhibitors and Raynaud's phenomenon using data from the FDA Adverse Event Reporting System (FAERS). A retrospective analysis of adverse events from the approval year of each drug through August 2023 was conducted. Disproportionality was assessed using Reporting Odds Ratios (ROR) and Information Components (IC), with significant signals of disproportionate reporting (SDR) identified by a lower 95% confidence interval (CI) for ROR > 1.0 and IC > 0. Intra-class and inter-class analyses were conducted to compare SDRs among CGRP inhibitors and other migraine therapies, including triptans, beta-blockers, and anticonvulsants. CGRP inhibitors demonstrated significant SDRs for Raynaud's phenomenon (ROR 19.12; 95% CI 15.44-23.69), with rimegepant, ubrogepant, and atogepant showing particularly strong signals. Intra-class analysis revealed a significant SDR only for galcanezumab (ROR 2.01; 95% CI 1.28-3.17). Inter-class analysis indicated significant SDRs for CGRP inhibitors compared to beta-blockers, anticonvulsants, and celecoxib, but not triptans. These findings underscore the importance of ongoing pharmacovigilance and further research to validate these associations and ensure patient safety.

Advanced AI-Driven Prediction of Pregnancy-Related Adverse Drug Reactions

Ensuring drug safety during pregnancy is critical due to the potential risks to both the mother and fetus. However, the exclusion of pregnant women from clinical trials complicates the assessment of adverse drug reactions (ADRs) in this population. This study aimed to develop and validate risk prediction models for pregnancy-related ADRs of drugs using advanced Machine Learning (ML) and Deep Learning (DL) techniques, leveraging real-world data from the FDA Adverse Event Reporting System. We explored three methods─Information Component, Reporting Odds Ratio, and 95% confidence interval of ROR─for classifying drugs into high-risk and low-risk categories. DL models, including Directed Message Passing Neural Networks (DMPNN), Graph Neural Networks, and Graph Convolutional Networks, were developed and compared to traditional ML models like Random Forest, Support Vector Machines, and XGBoost. Among these, the DMPNN model, which integrated molecular graph information and molecular descriptors, exhibited the highest predictive performance, particularly at the preferred term level. The model was validated against external data sets from SIDER and DailyMed, demonstrating strong generalizability. Additionally, the model was applied to assess the risk of 22 oral hypoglycemic drugs, and potential substructure alerts for pregnancy-related ADRs were identified. These findings suggest that the DMPNN model is a valuable tool for predicting ADRs in pregnant women, offering significant advancement in drug safety assessment and providing crucial insights for safer medication use during pregnancy.

A discovery and verification approach to pharmacovigilance using electronic healthcare data

Introduction

Pharmacovigilance is vital for drug safety. The process typically involves two key steps: initial signal generation from spontaneous reporting systems (SRSs) and subsequent expert review to assess the signals' (potential) causality and decide on the appropriate action.

Methods

We propose a novel discovery and verification approach to pharmacovigilance based on electronic healthcare data. We enhance the signal detection phase by introducing an ensemble of methods which generated signals are combined using Borda count ranking; a method designed to emphasize consensus. Ensemble methods tend to perform better when data is noisy and leverage the strengths of individual classifiers, while trying to mitigate some of their limitations. Additionally, we offer the committee of medical experts with the option to perform an in-depth investigation of selected signals through tailored pharmacoepidemiological studies to evaluate their plausibility or spuriousness. To illustrate our approach, we utilize data from the German Pharmacoepidemiological Research Database, focusing on drug reactions to the direct oral anticoagulant rivaroxaban.

Results

In this example, the ensemble method is built upon the Bayesian confidence propagation neural network, longitudinal Gamma Poisson shrinker, penalized regression and random forests. We also conduct a pharmacoepidemiological verification study in the form of a nested active comparator case-control study, involving patients diagnosed with atrial fibrillation who initiated anticoagulant treatment between 2011 and 2017.

Discussion

The case study reveals our ability to detect known adverse drug reactions and discover new signals. Importantly, the ensemble method is computationally efficient. Hasty false conclusions can be avoided by a verification study, which is, however, time-consuming to carry out. We provide an online tool for easy application: https://borda.bips.eu.

An innovative method to strengthen evidence for potential drug safety signals using Electronic Health Records

Reports from spontaneous reporting systems (SRS) are hypothesis generating. Additional evidence such as more reports is required to determine whether the generated drug-event associations are in fact safety signals. However, underreporting of adverse drug reactions (ADRs) delays signal detection. Through the use of natural language processing, different sources of real-world data can be used to proactively collect additional evidence for potential safety signals. This study aims to explore the feasibility of using Electronic Health Records (EHRs) to identify additional cases based on initial indications from spontaneous ADR reports, with the goal of strengthening the evidence base for potential safety signals. For two confirmed and two potential signals generated by the SRS of the Netherlands Pharmacovigilance Centre Lareb, targeted searches in the EHR of the Leiden University Medical Centre were performed using a text-mining based tool, CTcue. The search for additional cases was done by constructing and running queries in the structured and free-text fields of the EHRs. We identified at least five additional cases for the confirmed signals and one additional case for each potential safety signal. The majority of the identified cases for the confirmed signals were documented in the EHRs before signal detection by the Dutch Medicines Evaluation Board. The identified cases for the potential signals were reported to Lareb as further evidence for signal detection. Our findings highlight the feasibility of performing targeted searches in the EHR based on an underlying hypothesis to provide further evidence for signal generation.

Active surveillance pharmacovigilance for Clostridioides difficile infection and gastrointestinal bleeding: an analytic framework based on case-control studies

BACKGROUND

Active surveillance pharmacovigilance is an emerging approach to identify medications with unanticipated effects. We previously developed a framework called pharmacopeia-wide association studies (PharmWAS) that limits false positive medication associations through high-dimensional confounding adjustment and set enrichment. We aimed to assess the transportability and generalizability of the PharmWAS framework by using medical claims data to reproduce known medication associations with Clostridioides difficile infection (CDI) or gastrointestinal bleeding (GIB).

METHODS

We conducted case-control studies using Optum's de-identified Clinformatics Data Mart Database of individuals enrolled in large commercial and Medicare Advantage health plans in the United States. Individuals with CDI (from 2010 to 2015) or GIB (from 2010 to 2021) were matched to controls by age and sex. We identified all medications utilized prior to diagnosis and analysed the association of each with CDI or GIB using conditional logistic regression adjusted for risk factors for the outcome and a high-dimensional propensity score.

FINDINGS

For the CDI study, we identified 55,137 cases, 220,543 controls, and 290 medications to analyse. Antibiotics with Gram-negative spectrum, including ciprofloxacin (aOR 2.83), ceftriaxone (aOR 2.65), and levofloxacin (aOR 1.60), were strongly associated. For the GIB study, we identified 450,315 cases, 1,801,260 controls, and 354 medications to analyse. Antiplatelets, anticoagulants, and non-steroidal anti-inflammatory drugs, including ticagrelor (aOR 2.81), naproxen (aOR 1.87), and rivaroxaban (aOR 1.31), were strongly associated.

INTERPRETATION

These studies demonstrate the generalizability and transportability of the PharmWAS pharmacovigilance framework. With additional validation, PharmWAS could complement traditional passive surveillance systems to identify medications that unexpectedly provoke or prevent high-impact conditions.

FUNDING

U.S. National Institute of Diabetes and Digestive and Kidney Diseases.

A Survey of Pharmacists and Other Healthcare Professionals in Vietnam: Factors Influencing Knowledge and Attitudes Toward Reporting Adverse Drug Reactions

Background

Knowledge and attitudes of healthcare professionals are significant factors that affect the reporting of adverse drug reactions (ADRs). No previous research has examined the predictors of knowledge and attitudes toward ADR reporting in Vietnam.

Objectives

The aim of this study was to examine the factors (ie, demographic and job-related characteristics) associated with inadequate knowledge and negative attitudes toward ADR reporting in a Vietnamese public hospital.

Methods

A survey recruited a cross-sectional sample of 511 healthcare professionals (with a response rate of 92.9%) at a public hospital in Vinh Long province, Vietnam, from December 2022 to February 2023, using a self-administered questionnaire. Factors related to knowledge and attitudes toward ADR reporting were identified using univariate and multivariate logistic regression.

Results

Pharmacists had significantly lower knowledge scores (mean = 5.86) than medical practitioners (7.24) and nurses (6.72). Additionally, pharmacists' attitudes scored significantly lower (34.61) than those of medical practitioners (37.21) and nurses (36.86). Multivariate logistic regression showed that educational level, healthcare profession, monthly on-call shifts, and number of direct patient interactions were factors associated with a lower level of knowledge regarding ADR reporting. Additionally, age group and healthcare profession were identified as factors associated with negative attitudes toward ADR reporting among healthcare workers.

Conclusions

Our study identified several factors associated with lower levels of knowledge and negative attitudes toward ADR reporting among healthcare workers in Vietnam. These findings highlight the need for targeted interventions and education programs to improve healthcare workers' knowledge and attitudes toward ADR reporting.

An Awareness of Pharmacovigilance Among Healthcare Professionals Due to an Underreporting of Adverse Drug Reactions Issue: A Systematic Review of the Current State, Obstacles, and Strategy

BACKGROUND

Healthcare professionals play an essential role in reporting adverse drug reactions as part of pharmacovigilance activities. However, adverse drug reactions reported by healthcare professionals remain low.

OBJECTIVE

The aim of this systematic review was to investigate healthcare professionals' knowledge, awareness, attitude, and practice on pharmacovigilance and adverse drug reaction reporting, explore the causes of the underreporting issue, and provide improvement strategies.

METHODS

This systematic review was conducted using four electronic databases for original papers, including PubMed, Scopus, Google Scholar, and Scholar ID. Recent publications from 1st January 2012 to 31st December 2022 were selected. The following terms were used in the search: "awareness", "knowledge", "adverse drug reaction", "pharmacovigilance", "healthcare professional", and "underreporting factor". Articles were chosen, extracted, and reviewed by the two authors.

RESULTS

Twenty-five studies were selected for systematic review. This review found that 24.8%-73.33% of healthcare professionals were unaware of the National Pharmacovigilance Center. Around 20%-95.7% of healthcare professionals have a positive attitude toward pharmacovigilance and adverse drug reaction reporting, while 12%-60.8% of healthcare professionals have experience reporting any adverse drug reaction in their practice. The most frequently highlighted barriers to pharmacovigilance were a lack of awareness and knowledge regarding what, when, and to whom to report.

CONCLUSION

Underreporting issues require immediate attention among healthcare professionals due to a lack of awareness and knowledge of pharmacovigilance and adverse drug reaction reporting. Educational and training program interventions have been suggested by most studies to address these issues.

Supporting Pharmacovigilance Signal Validation and Prioritization with Analyses of Routinely Collected Health Data: Lessons Learned from an EHDEN Network Study

INTRODUCTION

Individual case reports are the main asset in pharmacovigilance signal management. Signal validation is the first stage after signal detection and aims to determine if there is sufficient evidence to justify further assessment. Throughout signal management, a prioritization of signals is continually made. Routinely collected health data can provide relevant contextual information but are primarily used at a later stage in pharmacoepidemiological studies to assess communicated signals.

OBJECTIVE

The aim of this study was to examine the feasibility and utility of analysing routine health data from a multinational distributed network to support signal validation and prioritization and to reflect on key user requirements for these analyses to become an integral part of this process.

METHODS

Statistical signal detection was performed in VigiBase, the WHO global database of individual case safety reports, targeting generic manufacturer drugs and 16 prespecified adverse events. During a 5-day study-a-thon, signal validation and prioritization were performed using information from VigiBase, regulatory documents and the scientific literature alongside descriptive analyses of routine health data from 10 partners of the European Health Data and Evidence Network (EHDEN). Databases included in the study were from the UK, Spain, Norway, the Netherlands and Serbia, capturing records from primary care and/or hospitals.

RESULTS

Ninety-five statistical signals were subjected to signal validation, of which eight were considered for descriptive analyses in the routine health data. Design, execution and interpretation of results from these analyses took up to a few hours for each signal (of which 15-60 minutes were for execution) and informed decisions for five out of eight signals. The impact of insights from the routine health data varied and included possible alternative explanations, potential public health and clinical impact and feasibility of follow-up pharmacoepidemiological studies. Three signals were selected for signal assessment, two of these decisions were supported by insights from the routine health data. Standardization of analytical code, availability of adverse event phenotypes including bridges between different source vocabularies, and governance around the access and use of routine health data were identified as important aspects for future development.

CONCLUSIONS

Analyses of routine health data from a distributed network to support signal validation and prioritization are feasible in the given time limits and can inform decision making. The cost-benefit of integrating these analyses at this stage of signal management requires further research.

Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review

INTRODUCTION

Pharmacovigilance programs protect patient health and safety by identifying adverse event signals through postmarketing surveillance of claims data and spontaneous reports. Electronic health records (EHRs) provide new opportunities to address limitations of traditional approaches and promote discovery-oriented pharmacovigilance.

METHODS

To evaluate the current state of EHR-based medication safety signal identification, we conducted a scoping literature review of studies aimed at identifying safety signals from routinely collected patient-level EHR data. We extracted information on study design, EHR data elements utilized, analytic methods employed, drugs and outcomes evaluated, and key statistical and data analysis choices.

RESULTS

We identified 81 eligible studies. Disproportionality methods were the predominant analytic approach, followed by data mining and regression. Variability in study design makes direct comparisons difficult. Studies varied widely in terms of data, confounding adjustment, and statistical considerations.

CONCLUSION

Despite broad interest in utilizing EHRs for safety signal identification, current efforts fail to leverage the full breadth and depth of available data or to rigorously control for confounding. The development of best practices and application of common data models would promote the expansion of EHR-based pharmacovigilance.

Methods for drug safety signal detection using routinely collected observational electronic health care data: A systematic review

PURPOSE

Signal detection is a crucial step in the discovery of post-marketing adverse drug reactions. There is a growing interest in using routinely collected data to complement established spontaneous report analyses. This work aims to systematically review the methods for drug safety signal detection using routinely collected healthcare data and their performance, both in general and for specific types of drugs and outcomes.

METHODS

We conducted a systematic review following the PRISMA guidelines, and registered a protocol in PROSPERO. MEDLINE, EMBASE, PubMed, Web of Science, Scopus, and the Cochrane Library were searched until July 13, 2021.

RESULTS

The review included 101 articles, among which there were 39 methodological works, 25 performance assessment papers, and 24 observational studies. Methods included adaptations from those used with spontaneous reports, traditional epidemiological designs, methods specific to signal detection with real-world data. More recently, implementations of machine learning have been studied in the literature. Twenty-five studies evaluated method performances, 16 of them using the area under the curve (AUC) for a range of positive and negative controls as their main measure. Despite the likelihood that performance measurement could vary by drug-event pair, only 10 studies reported performance stratified by drugs and outcomes, in a heterogeneous manner. The replicability of the performance assessment results was limited due to lack of transparency in reporting and the lack of a gold standard reference set.

CONCLUSIONS

A variety of methods have been described in the literature for signal detection with routinely collected data. No method showed superior performance in all papers and across all drugs and outcomes, performance assessment and reporting were heterogeneous. However, there is limited evidence that self-controlled designs, high dimensional propensity scores, and machine learning can achieve higher performances than other methods.

Effectiveness of a structured stimulated spontaneous safety monitoring of medicines reporting program in strengthening pharmacovigilance system in Tanzania

Under-reporting of adverse drug events (ADEs) is a challenge facing developing countries including Tanzania. Given the high magnitude of under-reporting, it was necessary to develop and assess the effectiveness of a ‘structured stimulated spontaneous safety monitoring’ (SSSSM) reporting program of ADEs which aimed at strengthening pharmacovigilance system in Tanzania. A quasi-experimental design and data mining technique were used to assess the effect of intervention after the introduction of program in seven tertiary hospitals. ADEs reports were collected from a single group and compared for 18 months before (July 2017 to December, 2018) and after the program (January 2019 to June 2020). Out of 16,557 ADEs reports, 98.6% (16,332) were reported after intervention and 0.1% (23) death related to adverse drug reactions (ADRs) were reported. Reports increased from 20 to 11,637 after intervention in Dar es salaam, 49 to 316 in Kilimanjaro and 17 to 77 in Mbeya. The population-based reporting ratio per 1,000,000 inhabitants increased from 2 reports per million inhabitants in 2018 to 85 reports in 2019. The SSSSM program can increase the reporting rate of ADEs and was useful in detecting signals from all types of medicines. This was first effective developed spontaneous program to monitor medicine safety in Tanzania.

Detection of Drug-Induced Thrombocytopenia Signals in Children Using Routine Electronic Medical Records

Background: Drug-induced thrombocytopenia (DITP) is a severe adverse reaction and a significantly under-recognized clinical problem in children. However, for post-marketing pharmacovigilance purposes, detection of DITP signals is crucial. This study aimed to develop a signal detection model for DITP using the pediatric electronic medical records (EMR) data.

Methods: This study used the electronic medical records collected at Beijing Children’s Hospital between 2009 and 2020. A two-stage modeling method was developed to detect the signal of DITP. In the first stage, we calculated the crude incidence by mining cases of thrombocytopenia to select the potential suspected drugs. In the second stage, we constructed propensity score–matched retrospective cohorts of specific screened drugs from the first stage and estimated the odds ratio (OR) and 95% confidence interval (CI) using conditional logistic regression models. The novelty of the signal was assessed by current evidence.

Results: In the study, from a total of 839 drugs, 21 drugs were initially screened as potentially inducing thrombocytopenia. In total, we identified 18 positive DITP associations. Of these, potential DITP risk of nystatin (OR: 1.75, 95% CI: 1.37–2.22) and latamoxef sodium (OR: 1.61, 95% CI: 1.38–1.88) were two new DITP signals in both children and adults. Six associations between thrombocytopenia and drugs including imipenem (OR: 1.69, 95% CI: 1.16–2.45), teicoplanin (OR: 4.75, 95% CI: 3.33–6.78), fusidic acid (OR: 2.81, 95% CI: 2.06–3.86), ceftizoxime sodium (OR: 1.83, 95% CI: 1.36–2.45), ceftazidime (OR: 2.16, 95% CI: 1.58–2.95), and cefepime (OR: 5.06, 95% CI: 3.77–6.78) were considered as new signals in children.

Conclusion: This study developed a two-stage algorithm to detect safety signals of DITP and found eighteen positive signals of DITP, including six new signals in a pediatric population. This method is a promising tool for pharmacovigilance based on EMR data.

Identifying Drugs Inducing Prematurity by Mining Claims Data with High-Dimensional Confounder Score Strategies

BACKGROUND

Pregnant women are largely exposed to medications. However, knowledge is lacking about their effects on pregnancy and the fetus.

OBJECTIVE

This study sought to evaluate the potential of high-dimensional propensity scores and high-dimensional disease risk scores for automated signal detection in pregnant women from medico-administrative databases in the context of drug-induced prematurity.

METHODS

We used healthcare claims and hospitalization discharges of a 1/97th representative sample of the French population. We tested the association between prematurity and drug exposure during the trimester before delivery, for all drugs prescribed to at least five pregnancies. We compared different strategies (1) for building the two scores, including two machine-learning methods and (2) to account for these scores in the final logistic regression models: adjustment, weighting, and matching. We also proposed a new signal detection criterion derived from these scores: the p value relative decrease. Evaluation was performed by assessing the relevance of the signals using a literature review and clinical expertise.

RESULTS

Screening 400 drugs from a cohort of 57,407 pregnancies, we observed that choosing between the two machine-learning methods had little impact on the generated signals. Score adjustment performed better than weighting and matching. Using the p value relative decrease efficiently filtered out spurious signals while maintaining a number of relevant signals similar to score adjustment. Most of the relevant signals belonged to the psychotropic class with benzodiazepines, antidepressants, and antipsychotics.

CONCLUSIONS

Mining complex healthcare databases with statistical methods from the high-dimensional inference field may improve signal detection in pregnant women.

Evaluation of Covid-19 vaccines: Pharmacoepidemiological aspects

The marketing authorization granted to SARS-Cov-2 vaccines was accompanied by reinforced safety monitoring plans. These plans' implementation was part of the usual logic of post-marketing surveillance of new and innovative health products. It was especially adapted to the context of post-marketing monitoring of drugs developed according to the usual scientific quality standards but in an accelerated schedule. In Europe, the reinforced surveillance system relies on the complementary strengths of pharmacovigilance and pharmacoepidemiology. If the performances of pharmacovigilance monitoring are incomparable for the detection of safety signals relating to rare events of atypical presentation, it needs to be completed with pharmacoepidemiology activities for more common events, either multifactorial or frequently classified as idiopathic. The pharmacoepidemiological monitoring developed in Europe was elaborated before the first SARS-Cov-2 vaccines where marketed, taking into account the lessons learned from the vaccination campaign against 2009 A (H1N1) influenza. It includes numerous academic studies as well as studies performed within vaccines risk management plans. In terms of safety, those defined a priori mostly concerns a list of pre-established health events of specific interest. Aside of these planned activities, ad-hoc studies will be latter developed on purpose to investigate safety signals or potential signals that could be identified as the result of pharmacovigilance activities. Aside of these regulated activities, as for today, very few studies have been published regarding SARS-Cov-2 vaccines; most of the existing consist in preprints that should be considered with caution. Pharmacoepidemiology of vaccines is thought to allow near-real time monitoring that needs sufficient time to provide with valid results. In the constant urge for information that accompanies COVID-related science, it is important not to make haste the enemy of speed and to let pharmacoepidemiology provides with what it is expected to do: rock-solid scientific information contributing to evidence-based decision-making.

The 2011–2020 Trends of Data-Driven Approaches in Medical Informatics for Active Pharmacovigilance

Pharmacovigilance, the scientific discipline pertaining to drug safety, has been studied extensively and is progressing continuously. In this field, medical informatics techniques and interpretation play important roles, and appropriate approaches are required. In this study, we investigated and analyzed the trends of pharmacovigilance systems, especially the data collection, detection, assessment, and monitoring processes. We used PubMed to collect papers on pharmacovigilance published over the past 10 years, and analyzed a total of 40 significant papers to determine the characteristics of the databases and data analysis methods used to identify drug safety indicators. Through systematic reviews, we identified the difficulty of standardizing data and terminology and establishing an adverse drug reactions (ADR) evaluation system in pharmacovigilance, and their corresponding implications. We found that appropriate methods and guidelines for active pharmacovigilance using medical big data are still required and should continue to be developed.

Automation in signal management in pharmacovigilance-an insight

Drugs are the imperial part of modern society, but along with their therapeutic effects, drugs can also cause adverse effects, which can be mild to morbid. Pharmacovigilance is the process of collection, detection, assessment, monitoring and prevention of adverse drug events in both clinical trials as well as in the post-marketing phase. The recent trends in increasing unknown adverse events, known as signals, have raised the need to develop an ideal system for monitoring and detecting the potential signals timely. The process of signal management comprises of techniques to identify individual case safety reports systematically. Automated signal detection is highly based upon the data mining of the spontaneous reporting system such as reports from health care professional, observational studies, medical literature or from social media. If a signal is not managed properly, it can become an identical risk associated with the drug which can be hazardous for the patient safety and may have fatal outcomes which may impact health care system adversely. Once a signal is detected quantitatively, it can be further processed by the signal management team for the qualitative analysis and further evaluations. The main components of automated signal detection are data extraction, data acquisition, data selection, and data analysis and data evaluation. This system must be developed in the correct format and context, which eventually emphasizes the quality of data collected and leads to the optimal decision-making based upon the scientific evaluation.

Empirical assessment of case-based methods for identification of drugs associated with acute liver injury in the French National Healthcare System database (SNDS)

PURPOSES

Drug induced acute liver injury (ALI) is a frequent cause of liver failure. Case-based designs were empirically assessed and calibrated in the French National claims database (SNDS), aiming to identify the optimum design for drug safety alert generation associated with ALI.

METHODS

All cases of ALI were extracted from SNDS (2009-2014) using specific and sensitive definitions. Positive and negative drug controls were used to compare 196 self-controlled case series (SCCS), case-control (CC), and case-population (CP) design variants, using area under the receiver operating curve (AUC), mean square error (MSE) and coverage probability. Parameters that had major impacts on results were identified through logistic regression.

RESULTS

Using a specific ALI definition, AUCs ranged from 0.78 to 0.94, 0.64 to 0.92 and 0.48 to 0.85, for SCCS, CC and CP, respectively. MSE ranged from 0.12 to 0.40, 0.22 to 0.39 and 1.03 to 5.29, respectively. Variants adjusting for multiple drug use had higher coverage probabilities. Univariate regressions showed that high AUCs were achieved with SCCS using exposed time as the risk window. The top SCCS variant yielded an AUC = 0.93 and MSE = 0.22 and coverage = 86%, with 1/7 negative and 13/18 positive controls presenting significant estimates.

CONCLUSIONS

SCCS adjusting for multiple drugs and using exposed time as the risk window performed best in generating ALI-related drug safety alert and providing estimates of the magnitude of the risk. This approach may be useful for ad-hoc pharmacoepidemiology studies to support regulatory actions.

Exploratory Study of Signals for Asthma Drugs in Children, Using the EudraVigilance Database of Spontaneous Reports

Introduction

As asthma medications are frequently prescribed for children, knowledge of the safety of these drugs in the paediatric population is important. Although spontaneous reports cannot be used to prove causality of adverse events, they are important in the detection of safety signals.

Objective

Our objective was to provide an overview of adverse drug events associated with asthma medications in children from a spontaneous reports database and to identify new signals.

Methods

Spontaneous reports concerning asthma drugs were obtained from EudraVigilance, the European Medicine Agency’s database for suspected adverse drug reactions. For each drug–event combination, we calculated the proportional reporting ratio (PRR) in the study period 2011–2017. Signals in children (aged 0–17 years) were compared with signals in the whole population. Analyses were repeated for different age categories, by sex and by therapeutic area.

Results

In total, 372,345 reports in children resulted in 385 different signals concerning asthma therapy. The largest group consisted of psychiatric events (65 signals). Only 30 signals were new, with seven, including herpes viral infections, associated with omalizumab. Stratification by age, sex and therapeutic area provided additional new signals, such as hypertrichoses with budesonide and encephalopathies with theophylline. Of all signals in children, 60 (16%) did not appear in the whole population.

Conclusions

The majority of signals regarding asthma therapy in children were already known, but we also identified new signals. We showed that signals can be masked if age stratification is not conducted. Further exploration is needed to investigate the risk and causality of the newly found signals.

Electronic supplementary material

The online version of this article (10.1007/s40264-019-00870-x) contains supplementary material, which is available to authorized users.

Adverse drug event presentation and tracking (ADEPT): semiautomated, high throughput pharmacovigilance using real-world data

Objective

To advance use of real-world data (RWD) for pharmacovigilance, we sought to integrate a high-sensitivity natural language processing (NLP) pipeline for detecting potential adverse drug events (ADEs) with easily interpretable output for high-efficiency human review and adjudication of true ADEs.

Materials and methods

The adverse drug event presentation and tracking (ADEPT) system employs an open source NLP pipeline to identify in clinical notes mentions of medications and signs and symptoms potentially indicative of ADEs. ADEPT presents the output to human reviewers by highlighting these drug-event pairs within the context of the clinical note. To measure incidence of seizures associated with sildenafil, we applied ADEPT to 149 029 notes for 982 patients with pediatric pulmonary hypertension.

Results

Of 416 patients identified as taking sildenafil, NLP found 72 [17%, 95% confidence interval (CI) 14–21] with seizures as a potential ADE. Upon human review and adjudication, only 4 (0.96%, 95% CI 0.37–2.4) patients with seizures were determined to have true ADEs. Reviewers using ADEPT required a median of 89 s (interquartile range 57–142 s) per patient to review potential ADEs.

Discussion

ADEPT combines high throughput NLP to increase sensitivity of ADE detection and human review, to increase specificity by differentiating true ADEs from signs and symptoms related to comorbidities, effects of other medications, or other confounders.

Conclusion

ADEPT is a promising tool for creating gold standard, patient-level labels for advancing NLP-based pharmacovigilance. ADEPT is a potentially time savings platform for computer-assisted pharmacovigilance based on RWD.

Knowledge, attitudes and practices of health care professionals towards adverse drug reaction reporting in public sector primary health care facilities in a South African district

PURPOSE

Adverse drug reactions (ADRs) have an appreciable impact on patients' health. Little is known however about ADR reporting in ambulatory care environments especially in low- and middle-income countries. Consequently, our aim was to determine knowledge, attitudes and practices (KAP) among health care professionals (HCPs) towards ADR reporting in primary health care (PHC) facilities in South Africa. The findings will be used to direct future activities.

METHODS

Descriptive, cross-sectional design using quantitative methodology among 8 public sector community health care centres and 40 PHC clinics in the Tshwane Health District, Gauteng Province. A self-administered questionnaire was distributed to 218 HCPs, including all key groups.

RESULTS

A total of 200 responses were received (91.7%). Although an appropriate attitude towards ADR reporting existed, the actual frequency of ADR reporting was low (16.0%). Of the respondents, 60.5% did not know how to report, where to report or when to report an ADR and 51.5% said the level of their clinical knowledge made it difficult to decide whether or not an ADR had occurred. Over 97.5% stated they should be reporting ADRs with 89% feeling that ADR reporting is a professional obligation and over 70% that ADR reporting should be compulsory. When results were combined, the overall mean score in terms of positive or preferred practices for ADR reporting was 24.6% with pharmacists having the highest scores.

CONCLUSION

Under-reporting of ADRs with gaps in KAP was evident. There is a serious and urgent need for education and training of HCPs on ADR reporting in South Africa.

Factors influencing harmonized health data collection, sharing and linkage in Denmark and Switzerland: A systematic review

INTRODUCTION

The digitalization of medicine has led to a considerable growth of heterogeneous health datasets, which could improve healthcare research if integrated into the clinical life cycle. This process requires, amongst other things, the harmonization of these datasets, which is a prerequisite to improve their quality, re-usability and interoperability. However, there is a wide range of factors that either hinder or favor the harmonized collection, sharing and linkage of health data.

OBJECTIVE

This systematic review aims to identify barriers and facilitators to health data harmonization-including data sharing and linkage-by a comparative analysis of studies from Denmark and Switzerland.

METHODS

Publications from PubMed, Web of Science, EMBASE and CINAHL involving cross-institutional or cross-border collection, sharing or linkage of health data from Denmark or Switzerland were searched to identify the reported barriers and facilitators to data harmonization.

RESULTS

Of the 345 projects included, 240 were single-country and 105 were multinational studies. Regarding national projects, a Swiss study reported on average more barriers and facilitators than a Danish study. Barriers and facilitators of a technical nature were most frequently reported.

CONCLUSION

This systematic review gathered evidence from Denmark and Switzerland on barriers and facilitators concerning data harmonization, sharing and linkage. Barriers and facilitators were strictly interrelated with the national context where projects were carried out. Structural changes, such as legislation implemented at the national level, were mirrored in the projects. This underlines the impact of national strategies in the field of health data. Our findings also suggest that more openness and clarity in the reporting of both barriers and facilitators to data harmonization constitute a key element to promote the successful management of new projects using health data and the implementation of proper policies in this field. Our study findings are thus meaningful beyond these two countries.

Deconstructing the Pharmacovigilance Hype Cycle

Data science is making increasing contributions to pharmacovigilance. Although the technical innovation of these works are indisputable, efficient progress in real-world pharmacovigilance signal detection may be hampered by corresponding technology life cycle effects, with a resulting tendency to conclude that, with large enough datasets and intricate algorithms, "the numbers speak for themselves," discounting the importance of clinical and scientific judgment. A practical consequence is overzealous declarations regarding the safety or lack of safety of drugs. We describe these concerns through a critical discussion of key results and conclusions from case studies selected to illustrate these points.

Developing a Standardization Algorithm for Categorical Laboratory Tests for Clinical Big Data Research: Retrospective Study

Background

Data standardization is essential in electronic health records (EHRs) for both clinical practice and retrospective research. However, it is still not easy to standardize EHR data because of nonidentical duplicates, typographical errors, or inconsistencies. To overcome this drawback, standardization efforts have been undertaken for collecting data in a standardized format as well as for curating the stored data in EHRs. To perform clinical big data research, the stored data in EHR should be standardized, starting from laboratory results, given their importance. However, most of the previous efforts have been based on labor-intensive manual methods.

Objective

We aimed to develop an automatic standardization method for eliminating the noises of categorical laboratory data, grouping, and mapping of cleaned data using standard terminology.

Methods

We developed a method called standardization algorithm for laboratory test–categorical result (SALT-C) that can process categorical laboratory data, such as pos +, 250 4+ (urinalysis results), and reddish (urinalysis color results). SALT-C consists of five steps. First, it applies data cleaning rules to categorical laboratory data. Second, it categorizes the cleaned data into 5 predefined groups (urine color, urine dipstick, blood type, presence-finding, and pathogenesis tests). Third, all data in each group are vectorized. Fourth, similarity is calculated between the vectors of data and those of each value in the predefined value sets. Finally, the value closest to the data is assigned.

Results

The performance of SALT-C was validated using 59,213,696 data points (167,938 unique values) generated over 23 years from a tertiary hospital. Apart from the data whose original meaning could not be interpreted correctly (eg, ** and _^), SALT-C mapped unique raw data to the correct reference value for each group with accuracy of 97.6% (123/126; urine color tests), 97.5% (198/203; (urine dipstick tests), 95% (53/56; blood type tests), 99.68% (162,291/162,805; presence-finding tests), and 99.61% (4643/4661; pathogenesis tests).

Conclusions

The proposed SALT-C successfully standardized the categorical laboratory test results with high reliability. SALT-C can be beneficial for clinical big data research by reducing laborious manual standardization efforts.

Authenticity and credibility aware detection of adverse drug events from social media

OBJECTIVES

Adverse drug events (ADEs) are among the top causes of hospitalization and death. Social media is a promising open data source for the timely detection of potential ADEs. In this paper, we study the problem of detecting signals of ADEs from social media.

METHODS

Detecting ADEs whose drug and AE may be reported in different posts of a user leads to major concerns regarding the content authenticity and user credibility, which have not been addressed in previous studies. Content authenticity concerns whether a post mentions drugs or adverse events that are actually consumed or experienced by the writer. User credibility indicates the degree to which chronological evidence from a user's sequence of posts should be trusted in the ADE detection. We propose AC-SPASM, a Bayesian model for the authenticity and credibility aware detection of ADEs from social media. The model exploits the interaction between content authenticity, user credibility and ADE signal quality. In particular, we argue that the credibility of a user correlates with the user's consistency in reporting authentic content.

RESULTS

We conduct experiments on a real-world Twitter dataset containing 1.2 million posts from 13,178 users. Our benchmark set contains 22 drugs and 8089 AEs. AC-SPASM recognizes authentic posts with F₁ - the harmonic mean of precision and recall of 80%, and estimates user credibility with precision@10 = 90% and NDCG@10 - a measure for top-10 ranking quality of 96%. Upon validation against known ADEs, AC-SPASM achieves F₁ = 91%, outperforming state-of-the-art baseline models by 32% (p < 0.05). Also, AC-SPASM obtains precision@456 = 73% and NDCG@456 = 94% in detecting and prioritizing unknown potential ADE signals for further investigation. Furthermore, the results show that AC-SPASM is scalable to large datasets.

CONCLUSIONS

Our study demonstrates that taking into account the content authenticity and user credibility improves the detection of ADEs from social media. Our work generates hypotheses to reduce experts' guesswork in identifying unknown potential ADEs.

Authenticity and credibility aware detection of adverse drug events from social media

OBJECTIVES

Adverse drug events (ADEs) are among the top causes of hospitalization and death. Social media is a promising open data source for the timely detection of potential ADEs. In this paper, we study the problem of detecting signals of ADEs from social media.

METHODS

Detecting ADEs whose drug and AE may be reported in different posts of a user leads to major concerns regarding the content authenticity and user credibility, which have not been addressed in previous studies. Content authenticity concerns whether a post mentions drugs or adverse events that are actually consumed or experienced by the writer. User credibility indicates the degree to which chronological evidence from a user's sequence of posts should be trusted in the ADE detection. We propose AC-SPASM, a Bayesian model for the authenticity and credibility aware detection of ADEs from social media. The model exploits the interaction between content authenticity, user credibility and ADE signal quality. In particular, we argue that the credibility of a user correlates with the user's consistency in reporting authentic content.

RESULTS

We conduct experiments on a real-world Twitter dataset containing 1.2 million posts from 13,178 users. Our benchmark set contains 22 drugs and 8089 AEs. AC-SPASM recognizes authentic posts with F₁ - the harmonic mean of precision and recall of 80%, and estimates user credibility with precision@10 = 90% and NDCG@10 - a measure for top-10 ranking quality of 96%. Upon validation against known ADEs, AC-SPASM achieves F₁ = 91%, outperforming state-of-the-art baseline models by 32% (p < 0.05). Also, AC-SPASM obtains precision@456 = 73% and NDCG@456 = 94% in detecting and prioritizing unknown potential ADE signals for further investigation. Furthermore, the results show that AC-SPASM is scalable to large datasets.

CONCLUSIONS

Our study demonstrates that taking into account the content authenticity and user credibility improves the detection of ADEs from social media. Our work generates hypotheses to reduce experts' guesswork in identifying unknown potential ADEs.

Signal Detection for Recently Approved Products: Adapting and Evaluating Self-Controlled Case Series Method Using a US Claims and UK Electronic Medical Records Database

INTRODUCTION

The Self-Controlled Case Series (SCCS) method has been widely used for hypothesis testing, but there is limited evidence of its performance for safety signal detection.

OBJECTIVE

The objective of this study was to evaluate SCCS for signal detection on recently approved products.

METHODS

A retrospective study covered the period after three recently marketed drugs were launched through to 31 December 2010 using The Health Improvement Network, a UK primary care database, and Optum, a US claims database. The SCCS method was applied to examine five heterogenous outcomes with desvenlafaxine and escitalopram and six outcomes with adalimumab for Signals of Disproportional Recording (SDRs); a positive finding was determined to be when the lower bound of 95% Confidence Interval of the incidence rate ratio (IRR) estimate was >  1. Multiple design choices were tested and the trend in IRR estimates over calendar time for one drug event pair was examined.

RESULTS

All six outcomes with adalimumab, three of five outcomes with desvenlafaxine, and four of five outcomes with escitalopram had SDRs. SCCS highlighted all acute events in the primary analysis but was less successful with slower-onset outcomes. Performance varied by risk period definition. Changes in IRR estimates over quarterly intervals for adalimumab with herpes zoster showed marked higher SDR within 9 months of drug launch.

CONCLUSION

SCCS shows promise for signal detection: it may highlight known associations for recent marketed products and has potential for early signal identification. SCCS performance varied by design choice and the nature of both exposure and event pair. Future work is needed to determine how effective the approach is in prospective testing and determining the performance characteristics of the approach.

Can Electronic Health Records Databases Complement Spontaneous Reporting System Databases? A Historical-Reconstruction of the Association of Rofecoxib and Acute Myocardial Infarction

Background: Several initiatives have assessed if mining electronic health records (EHRs) may accelerate the process of drug safety signal detection. In Europe, Exploring and Understanding Adverse Drug Reactions (EU-ADR) Project Focused on utilizing clinical data from EHRs of over 30 million patients from several European countries. Rofecoxib is a prescription COX-2 selective Non-Steroidal Anti-Inflammatory Drugs (NSAID) approved in 1999. In September 2004, the manufacturer withdrew rofecoxib from the market because of safety concerns. In this study, we investigated if the signal concerning rofecoxib and acute myocardial infarction (AMI) could have been identified in EHR database (EU-ADR project) earlier than spontaneous reporting system (SRS), and in advance of rofecoxib withdrawal.

Methods: Data from the EU-ADR project and WHO-VigiBase (for SRS) were used for the analysis. Signals were identified when respective statistics exceeded defined thresholds. The SRS analyses was conducted two ways- based on the date the AMI events with rofecoxib as a suspect medication were entered into the database and also the date that the AMI event occurred with exposure to rofecoxib.

Results: Within the databases participating in EU-ADR it was possible to identify a strong signal concerning rofecoxib and AMI since Q3 2000 [RR LGPS = 4.5 (95% CI: 2.84–6.72)] and peaked to 4.8 in Q4 2000. In WHO-VigiBase, for AMI term grouping, the EB05 threshold of 2 was crossed in the Q4 2004 (EB05 = 2.94). Since then, the EB05 value increased consistently and peaked in Q3 2006 (EB05 = 48.3) and then again in Q2 2008 (EB05 = 48.5). About 93% (2260 out of 2422) of AMIs reported in WHO-VigiBase database actually occurred prior to the product withdrawal, however, they were reported after the risk minimization/risk communication efforts.

Conclusion: In this study, EU-EHR databases were able to detect the AMI signal 4 years prior to the SRS database. We believe that for events that are consistently documented in EHR databases, such as serious events or events requiring in-patient medical intervention or hospitalization, the signal detection exercise in EHR would be beneficial for newly introduced medicinal products on the market, in addition to the SRS data.

Frequent Adverse Drug Reactions, and Medication Groups under Suspicion

BACKGROUND

The adverse drug reaction database of the German Federal Institute for Drugs and Medical Devices (Bundesinstitut für Arzneimittel und Medizinprodukte, BfArM) contains reports of suspected adverse drug reactions (ADRs) that are spon- taneously submitted by physicians, pharmacists, or patients. The aim of the present study was a descriptive analysis of all of these spontaneous reports.

METHODS

345 662 spontaneously submitted reports were analyzed with respect to the number of reports per year, the sources of the reports, demographic variables, the most commonly reported ADRs, and the drug classes most commonly suspected.

RESULTS

The number of reports submitted spontaneously each year has grown steadily since 1978. At the least detailed level of analysis, "drugs for the treatment of nervous system disorders" were the most common class of drugs under suspicion of causing the reported adverse drug reactions (23.1%). In a more detailed analysis by therapeutic subgroup, the three subgroups most commonly reported as suspected of causing side effects were antithrombotic agents, systemic antibiotics, and psycholeptics-causing thrombocytopenia, diarrhea, and drug dependency as the most frequently reported ADRs, respectively. The order of drug classes most commonly causing ADRs differed markedly between the physicians' reports (diazepines, fluoroquinolones, heparins) and the patients' reports (interferons, anti- thrombotic drugs, selective immunosuppressant drugs). Patients more commonly reported subjectively perceived ADRs, while physicians more commonly reported findings or diagnoses that require medical expertise.

CONCLUSION

The increasing number of spontaneous reports is mainly due to reports forwarded from pharmaceutical companies to the BfArM. This, in turn, is probably a result of increasingly strict legal reporting requirements in Germany. The detected differences between physicians' and patients' ADR reports can be taken to indicate that patients should be more specifically informed and questioned about potential ADRs. By reporting adverse drug reactions, physicians may improve drug safety.

Pharmacovigilance of sodium-glucose co-transporter-2 inhibitors: What a clinician should know on disproportionality analysis of spontaneous reporting systems

Sodium-glucose co-transporter-2 inhibitors (SGLT2-Is) have consistently demonstrated a clinically significant reduction of cardiovascular mortality. However, their safety in clinical practice is still incompletely characterized, and post-marketing monitoring is required considering the expected increase in clinical use. Different analyses of international spontaneous reporting systems, known as disproportionality analyses (DAs), have highlighted the occurrence of ketoacidosis, amputations, acute renal failure and skin toxicity. In this viewpoint, we critically appraise these pharmacovigilance data on SGLT2-Is, with the aim of supporting clinicians in proper interpretation of these studies, and discussing their risk-benefit profile. To this aim, we offer a broad perspective on basic technical aspects subtending DAs of spontaneous reporting databases (describing peculiarities of the Food and Drug Administration Adverse Event Reporting System), their common and evolving uses, key pitfalls in presenting study results (in terms of "risk" or "association") and relevant strategies to account for major confounders. This will also facilitate reviewers and editors in proper evaluation of DAs, and prompt pharmacovigilance experts in converging towards a set of minimum requirements in standardization of design, performance and reporting of DAs. A consensus on quality assessment of DAs will finally establish their transferability to clinical practice. It is anticipated that DAs cannot be used per se as a standalone approach to assess a drug-related risk and cannot replace clinical judgment in the individual patient.

Effectiveness of pharmacovigilance: multifaceted educational intervention related to the knowledge, skills and attitudes of multidisciplinary hospital staff

OBJECTIVES:

Most educational interventions in pharmacovigilance are designed to encourage physicians to report adverse drug reactions. However, multidisciplinary teams may play an important role in reporting drug-related problems. This study assessed the impact of a multifaceted educational intervention in pharmacovigilance on the knowledge, skills and attitudes of hospital professionals.

METHOD:

This prospective, open-label, non-randomized study was performed in a medium-complexity hospital in São Paulo, Brazil. The intervention involved four activities: 1) an interactive lecture, 2) a practical class, 3) a pre-post questionnaire administered to professionals on a multidisciplinary team, and 4) educational material. The intervention's impact on the professionals' knowledge and skills was assessed using the World Health Organization's definitions. The intervention's effect on the professionals' attitudes was analysed by the prevalence of adverse drug event reports (adverse drug reactions, medication errors, therapeutic failure and drug quality deviations) and the relevance (seriousness and expectancy) of the events.

RESULTS:

One hundred seventy-three professionals were enrolled. A 70-fold increase in the number of adverse drug event reports was observed during the 12 months post-intervention. The intervention improved the professionals' form-completion skills (p<0.0001) and their knowledge of pharmacovigilance (p<0.0001). The intervention also contributed to detecting serious drug-induced events. The nursing staff reported medication errors, and pharmacists and physiotherapists recognized serious adverse drug reactions. Physicians communicated suspicions of therapeutic failure.

CONCLUSIONS:

A multidisciplinary approach to drug-safety assessments contributes to identifying new, relevant drug-related problems and improving the rate of adverse drug event reporting. This strategy may therefore be applied to improve risk communication in hospitals.

Exploiting heterogeneous publicly available data sources for drug safety surveillance: computational framework and case studies

OBJECTIVE

Driven by the need of pharmacovigilance centres and companies to routinely collect and review all available data about adverse drug reactions (ADRs) and adverse events of interest, we introduce and validate a computational framework exploiting dominant as well as emerging publicly available data sources for drug safety surveillance.

METHODS

Our approach relies on appropriate query formulation for data acquisition and subsequent filtering, transformation and joint visualization of the obtained data. We acquired data from the FDA Adverse Event Reporting System (FAERS), PubMed and Twitter. In order to assess the validity and the robustness of the approach, we elaborated on two important case studies, namely, clozapine-induced cardiomyopathy/myocarditis versus haloperidol-induced cardiomyopathy/myocarditis, and apixaban-induced cerebral hemorrhage.

RESULTS

The analysis of the obtained data provided interesting insights (identification of potential patient and health-care professional experiences regarding ADRs in Twitter, information/arguments against an ADR existence across all sources), while illustrating the benefits (complementing data from multiple sources to strengthen/confirm evidence) and the underlying challenges (selecting search terms, data presentation) of exploiting heterogeneous information sources, thereby advocating the need for the proposed framework.

CONCLUSIONS

This work contributes in establishing a continuous learning system for drug safety surveillance by exploiting heterogeneous publicly available data sources via appropriate support tools.