Adverse Drug Reaction Identification and Extraction in Social Media: A Scoping Review

Medication safety perceptions in China: Media exposure, healthcare experiences, and trusted information sources

OBJECTIVE

Amid ongoing medication safety concerns in China and limited research on public perceptions, this study investigates the correlations between media exposure, healthcare experiences, and individuals' perceptions of medication safety. It also examines individuals' reliance on information sources during safety crises.

METHODS

A multistage stratified random sampling was employed with the gross sample containing 3090 Chinese adults aged 18-60 years. Data were analyzed using multiple linear regression.

RESULTS

Social media exposure was found to negatively correlate with perceptions of current medication safety and its perceived improvement, while exposure to television and print media showed positive correlations. Positive healthcare experiences were associated with improved medication safety perceptions. Among various information sources, healthcare professionals were deemed most trustworthy during medication safety incidents.

CONCLUSIONS

Media exposure and personal healthcare experiences significantly shape individuals' perceptions of medication safety in China, with healthcare professionals playing a crucial role in this context. Practiceimplications: Effective health crisis communication in China needs to be multifaceted, integrating traditional media and social media platforms to disseminate accurate information broadly. Additionally, healthcare professionals should be actively involved in crisis communication. Their role as trusted sources can be leveraged to clarify misconceptions, and reassure the public during medication safety incidents.

Using Social Media as a Source of Real-World Data for Pharmaceutical Drug Development and Regulatory Decision Making

INTRODUCTION

While pharmaceutical companies aim to leverage real-world data (RWD) to bridge the gap between clinical drug development and real-world patient outcomes, extant research has mainly focused on the use of social media in a post-approval safety-surveillance setting. Recent regulatory and technological developments indicate that social media may serve as a rich source to expand the evidence base to pre-approval and drug development activities. However, use cases related to drug development have been largely omitted, thereby missing some of the benefits of RWD. In addition, an applied end-to-end understanding of RWD rooted in both industry and regulations is lacking.

OBJECTIVE

We aimed to investigate how social media can be used as a source of RWD to support regulatory decision making and drug development in the pharmaceutical industry. We aimed to specifically explore the data pipeline and examine how social-media derived RWD can align with regulatory guidance from the US Food and Drug Administration and industry needs.

METHODS

A machine learning pipeline was developed to extract patient insights related to anticoagulants from X (Twitter) data. These findings were then analysed from an industry perspective, and complemented by interviews with professionals from a pharmaceutical company.

RESULTS

The analysis reveals several use cases where RWD derived from social media can be beneficial, particularly in generating hypotheses around patient and therapeutic area needs. We also note certain limitations of social media data, particularly around inferring causality.

CONCLUSIONS

Social media display considerable potential as a source of RWD for guiding efforts in pharmaceutical drug development and pre-approval settings. Although further regulatory guidance on the use of social media for RWD is needed to encourage its use, regulatory and technological developments are suggested to warrant at least exploratory uses for drug development.

A holistic AI-based approach for pharmacovigilance optimization from patients behavior on social media

In this paper, we propose a holistic AI-based pharmacovigilance optimization approach using patient's social media data. Instead of focusing on the detection and identification of Adverse Drug Events (ADE) in social media posts in single time points, we propose a holistic approach that looks at the evolution of different user behavior indicators in time. We examine various NLP-based indicators such as word frequency, semantic similarity, Adverse Drug Reactions mentions, and sentiment analysis. We introduce a classification approach to identify normal vs. abnormal time periods based on patient comments. This approach, along with user behavior indicators, can optimize the pharmacovigilance process by flagging the need for immediate attention and further investigation. We specifically focus on the Levothyrox® case in France, which sparked media attention due to changes in the medication formula and affected patient behavior on medical forums. For classification, we propose a deep learning architecture called Word Cloud Convolutional Neural Network (WC-CNN), trained on word clouds from patient comments. We evaluate different temporal resolutions and NLP pre-processing techniques, finding that monthly resolution and the proposed indicators can effectively detect new safety signals, with an accuracy of 75%. We have made the code open source, available via github.

The Role of Social Media for Identifying Adverse Drug Events Data in Pharmacovigilance: Protocol for a Scoping Review

BACKGROUND

Adverse drug events (ADEs) are a considerable public health burden resulting in disability, hospitalization, and death. Even those ADEs deemed nonserious can severely impact a patient's quality of life and adherence to intervention. Monitoring medication safety, however, is challenging. Social media may be a useful adjunct for obtaining real-world data on ADEs. While many studies have been undertaken to detect adverse events on social media, a consensus has not yet been reached as to the value of social media in pharmacovigilance or its role in pharmacovigilance in relation to more traditional data sources.

OBJECTIVE

The aim of the study is to evaluate and characterize the use of social media in ADE detection and pharmacovigilance as compared to other data sources.

METHODS

A scoping review will be undertaken. We will search 11 bibliographical databases as well as Google Scholar, hand-searching, and forward and backward citation searching. Records will be screened in Covidence by 2 independent reviewers at both title and abstract stage as well as full text. Studies will be included if they used any type of social media (such as Twitter or patient forums) to detect any type of adverse event associated with any type of medication and then compared the results from social media to any other data source (such as spontaneous reporting systems or clinical literature). Data will be extracted using a data extraction sheet piloted by the authors. Important data on the types of methods used (such as machine learning), any limitations of the methods used, types of adverse events and drugs searched for and included, availability of data and code, details of the comparison data source, and the results and conclusions will be extracted.

RESULTS

We will present descriptive summary statistics as well as identify any patterns in the types and timing of ADEs detected, including but not limited to the similarities and differences in what is reported, gaps in the evidence, and the methods used to extract ADEs from social media data. We will also summarize how the data from social media compares to conventional data sources. The literature will be organized by the data source for comparison. Where possible, we will analyze the impact of the types of adverse events, the social media platform used, and the methods used.

CONCLUSIONS

This scoping review will provide a valuable summary of a large body of research and important information for pharmacovigilance as well as suggest future directions of further research in this area. Through the comparisons with other data sources, we will be able to conclude the added value of social media in monitoring adverse events of medications, in terms of type of adverse events and timing.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/47068.

Computing Drug-Drug Similarity from Patient-Centric Data

In modern biology and medicine, drug-drug similarity is a major task with various applications in pharmaceutical drug development. Various direct and indirect sources of evidence obtained from drug-centric data such as side effects, drug interactions, biological targets, and chemical structures are used in the current methods to measure the level of drug-drug similarity. This paper proposes a computational method to measure drug-drug similarity using a novel source of evidence that is obtained from patient-centric data. More specifically, patients' narration of their thoughts, opinions, and experience with drugs in social media are explored as a potential source to compute drug-drug similarity. Online healthcare communities were used to extract a dataset of patients' reviews on anti-epileptic drugs. The collected dataset is preprocessed through Natural Language Processing (NLP) techniques and four text similarity methods are applied to measure the similarities among them. The obtained similarities are then used to generate drug-drug similarity-based ranking matrices which are analyzed through Pearson correlation, to answer questions related to the overall drug-drug similarity and the accuracy of the four similarity measures. To evaluate the obtained drug-drug similarities, they are compared with the corresponding ground-truth similarities obtained from DrugSimDB, a well-known drug-drug similarity tool that is based on drug-centric data. The results provide evidence on the feasibility of patient-centric data from social media as a novel source for computing drug-drug similarity.

Can Twitter posts serve as early indicators for potential safety signals? A retrospective analysis

BACKGROUND

As Twitter has gained significant popularity, tweets can serve as large pool of readily available data to estimate the adverse events (AEs) of medications.

OBJECTIVE

This study evaluated whether tweets were an early indicator for potential safety warnings. Additionally, the trend of AEs posted on Twitter was compared with AEs from the Yellow Card system in the United Kingdom.

METHODS

English Tweets for 35 drug-event pairs for the period 2017-2019, two years prior to the date of EMA Pharmacovigilance Risk Assessment Committee (PRAC) meeting, were collected. Both signal and non-signal AEs were manually identified and encoded using the MedDRA dictionary. AEs from Yellow Card were also gathered for the same period. Descriptive and inferential statistical analysis was conducted using Fisher's exact test to assess the distribution and proportion of AEs from the two data sources.

RESULTS

Of the total 61,661 English tweets, 1,411 had negative or neutral sentiment and mention of at least one AE. Tweets for 15 out of the 35 drugs (42.9%) contained AEs associated with the signals. On pooling data from Twitter and Yellow Card, 24 out of 35 drug-event pairs (68.6%) were identified prior to the respective PRAC meetings. Both data sources showed similar distribution of AEs based on seriousness, however, the distribution based on labelling was divergent.

CONCLUSION

Twitter cannot be used in isolation for signal detection in current pharmacovigilance (PV) systems. However, it can be used in combination with traditional PV systems for early signal detection, as it can provide a holistic drug safety profile.

Australasian Institute of Digital Health Summit 2022-Automated Social Media Surveillance for Detection of Vaccine Safety Signals: A Validation Study

BACKGROUND

Social media platforms have emerged as a valuable data source for public health research and surveillance. Monitoring of social media and user-generated data on the Web enables timely and inexpensive collection of information, overcoming time lag and cost of traditional health reporting systems.

OBJECTIVES

This article identifies personally experienced coronavirus disease 2019 (COVID-19) vaccine reactions expressed on Twitter and validate the findings against an established vaccine reactions reporting system.

METHODS

We collected around 3 million tweets from 1.4 million users between February 1, 2021, to January 31, 2022, using COVID-19 vaccines and vaccine reactions keyword lists. We performed topic modeling on a sample of the data and applied a modified F1 scoring technique to identify a topic that best differentiated vaccine-related personal health mentions. We then manually annotated 4,000 of the records from this topic, which were used to train a transformer-based classifier to identify likely personally experienced vaccine reactions. Applying the trained classifier to the entire data set allowed us to select records we could use to quantify potential vaccine side effects. Adverse events following immunization (AEFI) referred to in these records were compared with those reported to the state of Victoria's spontaneous vaccine safety surveillance system, SAEFVIC (Surveillance of Adverse Events Following Vaccination In the Community).

RESULTS

The most frequently mentioned potential vaccine reactions generally aligned with SAEFVIC data. Notable exceptions were increased Twitter reporting of bleeding-related AEFI and allergic reactions, and more frequent SAEFVIC reporting of cardiac AEFI.

CONCLUSION

Social media conversations are a potentially valuable supplementary data source for detecting vaccine adverse event mentions. Monitoring of online observations about new vaccine-related personal health experiences has the capacity to provide early warnings about emerging vaccine safety issues.

Social media and internet search data to inform drug utilization: A systematic scoping review

Introduction

Drug utilization is currently assessed through traditional data sources such as big electronic medical records (EMRs) databases, surveys, and medication sales. Social media and internet data have been reported to provide more accessible and more timely access to medications' utilization.

Objective

This review aims at providing evidence comparing web data on drug utilization to other sources before the COVID-19 pandemic.

Methods

We searched Medline, EMBASE, Web of Science, and Scopus until November 25th, 2019, using a predefined search strategy. Two independent reviewers conducted screening and data extraction.

Results

Of 6,563 (64%) deduplicated publications retrieved, 14 (0.2%) were included. All studies showed positive associations between drug utilization information from web and comparison data using very different methods. A total of nine (64%) studies found positive linear correlations in drug utilization between web and comparison data. Five studies reported association using other methods: One study reported similar drug popularity rankings using both data sources. Two studies developed prediction models for future drug consumption, including both web and comparison data, and two studies conducted ecological analyses but did not quantitatively compare data sources. According to the STROBE, RECORD, and RECORD-PE checklists, overall reporting quality was mediocre. Many items were left blank as they were out of scope for the type of study investigated.

Conclusion

Our results demonstrate the potential of web data for assessing drug utilization, although the field is still in a nascent period of investigation. Ultimately, social media and internet search data could be used to get a quick preliminary quantification of drug use in real time. Additional studies on the topic should use more standardized methodologies on different sets of drugs in order to confirm these findings. In addition, currently available checklists for study quality of reporting would need to be adapted to these new sources of scientific information.

Sample Bias in Web-Based Patient-Generated Health Data of Dutch Patients With Gastrointestinal Stromal Tumor: Survey Study

BACKGROUND

Increasingly, social media is being recognized as a potential resource for patient-generated health data, for example, for pharmacovigilance. Although the representativeness of the web-based patient population is often noted as a concern, studies in this field are limited.

OBJECTIVE

This study aimed to investigate the sample bias of patient-centered social media in Dutch patients with gastrointestinal stromal tumor (GIST).

METHODS

A population-based survey was conducted in the Netherlands among 328 patients with GIST diagnosed 2-13 years ago to investigate their digital communication use with fellow patients. A logistic regression analysis was used to analyze clinical and demographic differences between forum users and nonusers.

RESULTS

Overall, 17.9% (59/328) of survey respondents reported having contact with fellow patients via social media. Moreover, 78% (46/59) of forum users made use of GIST patient forums. We found no statistically significant differences for age, sex, socioeconomic status, and time since diagnosis between forum users (n=46) and nonusers (n=273). Patient forum users did differ significantly in (self-reported) treatment phase from nonusers (P=.001). Of the 46 forum users, only 2 (4%) were cured and not being monitored; 3 (7%) were on adjuvant, curative treatment; 19 (41%) were being monitored after adjuvant treatment; and 22 (48%) were on palliative treatment. In contrast, of the 273 patients who did not use disease-specific forums to communicate with fellow patients, 56 (20.5%) were cured and not being monitored, 31 (11.3%) were on curative treatment, 139 (50.9%) were being monitored after treatment, and 42 (15.3%) were on palliative treatment. The odds of being on a patient forum were 2.8 times as high for a patient who is being monitored compared with a patient that is considered cured. The odds of being on a patient forum were 1.9 times as high for patients who were on curative (adjuvant) treatment and 10 times as high for patients who were in the palliative phase compared with patients who were considered cured. Forum users also reported a lower level of social functioning (84.8 out of 100) than nonusers (93.8 out of 100; P=.008).

CONCLUSIONS

Forum users showed no particular bias on the most important demographic variables of age, sex, socioeconomic status, and time since diagnosis. This may reflect the narrowing digital divide. Overrepresentation and underrepresentation of patients with GIST in different treatment phases on social media should be taken into account when sourcing patient forums for patient-generated health data. A further investigation of the sample bias in other web-based patient populations is warranted.

Patient reporting of sexual adverse events on an online platform for medication experiences

AIMS

For >300 drugs, sexual side effects are included in the drug information leaflet. As sexual adverse events (sAEs) may be more easily shared at online medication platforms, patient-reported drug experiences may add to the current knowledge on sAE experiences. This study evaluated patient reports from the online platform mijnmedicijn.nl for the frequency of sAE reporting, sex differences concerning sAEs and to assess drugs with disproportional sAE reporting.

METHODS

On the online platform, terms for sAEs as used by patients were collected with a poll. Subsequently, drug reports posted between 2008 and 2020 were searched for sAEs with the identified terms. From the retrieved reports, the sAE frequencies and complaints and reporting odds ratios (ROR) were calculated, stratified for sex and drug (class). sAE reporting was considered disproportional frequent if the lower 95% confidence interval bound of the ROR >2.0.

RESULTS

For 189 drugs, sAEs were identified in 2408 reports (3.9%). Women posted 1383 reports (3.5% of all female reports) and men 1025 (4.7%). Almost half of the sAE reports addressed antidepressants: 586 reports of women (ROR 4.2; 95%CI 3.8-4.7) and 510 reports of men (ROR 7.5; 95%CI 6.6-8.5). Disproportional high numbers of sAE reports were found for 27 drugs, mostly antidepressants, hormonal contraceptives and drugs used in benign prostatic hyperplasia. Of these drugs with frequent sAEs, 7 had low sAE risks in their professional drug information.

CONCLUSION

One in 25 drug reports on mijnmedicijn.nl included sAEs. The sAEs were reported frequently for antidepressants, contraceptives and drugs used in benign prostatic hyperplasia.

Socio-historical analysis of the social importance of pharmacovigilance

Pharmacovigilance is a scientific discipline that has changed a lot in recent years and is of great social importance. The case of the so-called sulfonamide elixir showed society the importance of this discipline. Since then, pharmacovigilance has evolved into a scientific discipline with a strong social character. In this paper, a historical review is made of several paradigmatic examples of this discipline to reflect on what pharmacovigilance could be like finally. We conclude that this discipline could be more closely related to other areas of the social sciences, which would help to promote a more democratic social environment taking into account the needs of individuals and social groups.

Automated gathering of real-world data from online patient forums can complement pharmacovigilance for rare cancers

Current methods of pharmacovigilance result in severe under-reporting of adverse drug events (ADEs). Patient forums have the potential to complement current pharmacovigilance practices by providing real-time uncensored and unsolicited information. We are the first to explore the value of patient forums for rare cancers. To this end, we conduct a case study on a patient forum for Gastrointestinal Stromal Tumor patients. We have developed machine learning algorithms to automatically extract and aggregate side effects from messages on open online discussion forums. We show that patient forum data can provide suggestions for which ADEs impact quality of life the most: For many side effects the relative reporting rate differs decidedly from that of the registration trials, including for example cognitive impairment and alopecia as side effects of avapritinib. We also show that our methods can provide real-world data for long-term ADEs, such as osteoporosis and tremors for imatinib, and novel ADEs not found in registration trials, such as dry eyes and muscle cramping for imatinib. We thus posit that automated pharmacovigilance from patient forums can provide real-world data for ADEs and should be employed as input for medical hypotheses for rare cancers.

Spontaneously generated online patient experience data - how and why is it being used in health research: an umbrella scoping review

PURPOSE

Social media has led to fundamental changes in the way that people look for and share health related information. There is increasing interest in using this spontaneously generated patient experience data as a data source for health research. The aim was to summarise the state of the art regarding how and why SGOPE data has been used in health research. We determined the sites and platforms used as data sources, the purposes of the studies, the tools and methods being used, and any identified research gaps.

METHODS

A scoping umbrella review was conducted looking at review papers from 2015 to Jan 2021 that studied the use of SGOPE data for health research. Using keyword searches we identified 1759 papers from which we included 58 relevant studies in our review.

RESULTS

Data was used from many individual general or health specific platforms, although Twitter was the most widely used data source. The most frequent purposes were surveillance based, tracking infectious disease, adverse event identification and mental health triaging. Despite the developments in machine learning the reviews included lots of small qualitative studies. Most NLP used supervised methods for sentiment analysis and classification. Very early days, methods need development. Methods not being explained. Disciplinary differences - accuracy tweaks vs application. There is little evidence of any work that either compares the results in both methods on the same data set or brings the ideas together.

CONCLUSION

Tools, methods, and techniques are still at an early stage of development, but strong consensus exists that this data source will become very important to patient centred health research.

Crowdsourcing for Machine Learning in Public Health Surveillance: Lessons Learned From Amazon Mechanical Turk

Background

Crowdsourcing services, such as Amazon Mechanical Turk (AMT), allow researchers to use the collective intelligence of a wide range of web users for labor-intensive tasks. As the manual verification of the quality of the collected results is difficult because of the large volume of data and the quick turnaround time of the process, many questions remain to be explored regarding the reliability of these resources for developing digital public health systems.

Objective

This study aims to explore and evaluate the application of crowdsourcing, generally, and AMT, specifically, for developing digital public health surveillance systems.

Methods

We collected 296,166 crowd-generated labels for 98,722 tweets, labeled by 610 AMT workers, to develop machine learning (ML) models for detecting behaviors related to physical activity, sedentary behavior, and sleep quality among Twitter users. To infer the ground truth labels and explore the quality of these labels, we studied 4 statistical consensus methods that are agnostic of task features and only focus on worker labeling behavior. Moreover, to model the meta-information associated with each labeling task and leverage the potential of context-sensitive data in the truth inference process, we developed 7 ML models, including traditional classifiers (offline and active), a deep learning–based classification model, and a hybrid convolutional neural network model.

Results

Although most crowdsourcing-based studies in public health have often equated majority vote with quality, the results of our study using a truth set of 9000 manually labeled tweets showed that consensus-based inference models mask underlying uncertainty in data and overlook the importance of task meta-information. Our evaluations across 3 physical activity, sedentary behavior, and sleep quality data sets showed that truth inference is a context-sensitive process, and none of the methods studied in this paper were consistently superior to others in predicting the truth label. We also found that the performance of the ML models trained on crowd-labeled data was sensitive to the quality of these labels, and poor-quality labels led to incorrect assessment of these models. Finally, we have provided a set of practical recommendations to improve the quality and reliability of crowdsourced data.

Conclusions

Our findings indicate the importance of the quality of crowd-generated labels in developing ML models designed for decision-making purposes, such as public health surveillance decisions. A combination of inference models outlined and analyzed in this study could be used to quantitatively measure and improve the quality of crowd-generated labels for training ML models.

Social Media Analytics for Pharmacovigilance of Antiepileptic Drugs

Epilepsy is a common neurological disorder worldwide and antiepileptic drug (AED) therapy is the cornerstone of its treatment. It has a laudable aim of achieving seizure freedom with minimal, if any, adverse drug reactions (ADRs). Too often, AED treatment is a long-lasting journey, in which ADRs have a crucial role in its administration. Therefore, from a pharmacovigilance perspective, detecting the ADRs of AEDs is a task of utmost importance. Typically, this task is accomplished by analyzing relevant data from spontaneous reporting systems. Despite their wide adoption for pharmacovigilance activities, the passiveness and high underreporting ratio associated with spontaneous reporting systems have encouraged the consideration of other data sources such as electronic health databases and pharmaceutical databases. Social media is the most recent alternative data source with many promising potentials to overcome the shortcomings of traditional data sources. Although in the literature some attempts have investigated the validity and utility of social media for ADR detection of different groups of drugs, none of them was dedicated to the ADRs of AEDs. Hence, this paper presents a novel investigation of the validity and utility of social media as an alternative data source for the detection of AED ADRs. To this end, a dataset of consumer reviews from two online health communities has been collected. The dataset is preprocessed; the unigram, bigram, and trigram are generated; and the ADRs of each AED are extracted with the aid of consumer health vocabulary and ADR lexicon. Three widely used measures, namely, proportional reporting ratio, reporting odds ratio, and information component, are used to measure the association between each ADR and AED. The resulting list of signaled ADRs for each AED is validated against a widely used ADR database, called Side Effect Resource, in terms of the precision of ADR detection. The validation results indicate the validity of online health community data for the detection of AED ADRs. Furthermore, the lists of signaled AED ADRs are analyzed to answer questions related to the common ADRs of AEDs and the similarities between AEDs in terms of their signaled ADRs. The consistency of the drawn answers with the existing pharmaceutical knowledge suggests the utility of the data from online health communities for AED-related knowledge discovery tasks.

Comparison of Online Patient Reviews and National Pharmacovigilance Data for Tramadol-Related Adverse Events: Comparative Observational Study

BACKGROUND

Tramadol is known to cause fewer adverse events (AEs) than other opioids. However, recent research has raised concerns about various safety issues.

OBJECTIVE

We aimed to explore these new AEs related to tramadol using social media and conventional pharmacovigilance data.

METHODS

This study used 2 data sets, 1 from patients' drug reviews on WebMD (January 2007 to January 2021) and 1 from the US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS; January 2016 to December 2020). We analyzed 2062 and 29,350 patient reports from WebMD and FAERS, respectively. Patient posts on WebMD were manually assigned the preferred terms of the Medical Dictionary for Regulatory Activities. To analyze AEs from FAERS, a disproportionality analysis was performed with 3 measures: proportional reporting ratio, reporting odds ratio, and information component.

RESULTS

From the 869 AEs reported, we identified 125 new signals related to tramadol use not listed on the drug label that satisfied all 3 signal detection criteria. In addition, 20 serious AEs were selected from new signals. Among new serious AEs, vascular disorders had the largest signal detection criteria value. Based on the disproportionality analysis and patients' symptom descriptions, tramadol-induced pain might also be an unexpected AE.

CONCLUSIONS

This study detected several novel signals related to tramadol use, suggesting newly identified possible AEs. Additionally, this study indicates that unexpected AEs can be detected using social media analysis alongside traditional pharmacovigilance data.

Vaccine adverse event mentions in social media: Mining the language of Twitter conversations (Preprint)

Background

Traditional monitoring for adverse events following immunization (AEFI) relies on various established reporting systems, where there is inevitable lag between an AEFI occurring and its potential reporting and subsequent processing of reports. AEFI safety signal detection strives to detect AEFI as early as possible, ideally close to real time. Monitoring social media data holds promise as a resource for this.

Objective

The primary aim of this study is to investigate the utility of monitoring social media for gaining early insights into vaccine safety issues, by extracting vaccine adverse event mentions (VAEMs) from Twitter, using natural language processing techniques. The secondary aims are to document the natural language processing techniques used and identify the most effective of them for identifying tweets that contain VAEM, with a view to define an approach that might be applicable to other similar social media surveillance tasks.

Methods

A VAEM-Mine method was developed that combines topic modeling with classification techniques to extract maximal VAEM posts from a vaccine-related Twitter stream, with high degree of confidence. The approach does not require a targeted search for specific vaccine reaction–indicative words, but instead, identifies VAEM posts according to their language structure.

Results

The VAEM-Mine method isolated 8992 VAEMs from 811,010 vaccine-related Twitter posts and achieved an F₁ score of 0.91 in the classification phase.

Conclusions

Social media can assist with the detection of vaccine safety signals as a valuable complementary source for monitoring mentions of vaccine adverse events. A social media–based VAEM data stream can be assessed for changes to detect possible emerging vaccine safety signals, helping to address the well-recognized limitations of passive reporting systems, including lack of timeliness and underreporting.

Adverse drug event detection using reason assignments in FDA drug labels

Adverse drug events (ADEs) are unintended incidents that involve the taking of a medication. ADEs pose significant health and financial problems worldwide. Information about ADEs can inform health care and improve patient safety. However, much of this information is buried in narrative texts and needs to be extracted with Natural Language Processing techniques, in order to be useful to computerized methods. ADEs can be found on drug labels, contained in the different sections such as descriptions of the drug's active components or more prominently in descriptions of studied side-effects. Extracting these automatically could be useful in triaging and processing drug reports. In this paper, we present three base methods consisting of a Conditional Random Field (CRF), a bi-directional Long Short Term Memory unit with a CRF layer (biLSTM+CRF), and a pre-trained Bi-directional Encoder Representations from Transformers (BERT) model. We also present several ensembles of the CRF and biLSTM+CRF methods for extracting ADEs and their Reason from FDA drug labels. We show that all three methods perform well on our task, and that combining the models through different ensemble methods can improve results, providing increases in recall for the majority class and improving precision for all other classes. We also show the potential of framing ADE extraction from drug labels as a multi-class classification task on the Reason, or type, of ADE.

Identifying Influential Factors in the Discussion Dynamics of Emerging Health Issues on Social Media: Computational Study

Background

Social media has become a major resource for observing and understanding public opinions using infodemiology and infoveillance methods, especially during emergencies such as disease outbreaks. For public health agencies, understanding the driving forces of web-based discussions will help deliver more effective and efficient information to general users on social media and the web.

Objective

The study aimed to identify the major contributors that drove overall Zika-related tweeting dynamics during the 2016 epidemic. In total, 3 hypothetical drivers were proposed: (1) the underlying Zika epidemic quantified as a time series of case counts; (2) sporadic but critical real-world events such as the 2016 Rio Olympics and World Health Organization’s Public Health Emergency of International Concern (PHEIC) announcement, and (3) a few influential users’ tweeting activities.

Methods

All tweets and retweets (RTs) containing the keyword Zika posted in 2016 were collected via the Gnip application programming interface (API). We developed an analytical pipeline, EventPeriscope, to identify co-occurring trending events with Zika and quantify the strength of these events. We also retrieved Zika case data and identified the top influencers of the Zika discussion on Twitter. The influence of 3 potential drivers was examined via a multivariate time series analysis, signal processing, a content analysis, and text mining techniques.

Results

Zika-related tweeting dynamics were not significantly correlated with the underlying Zika epidemic in the United States in any of the four quarters in 2016 nor in the entire year. Instead, peaks of Zika-related tweeting activity were strongly associated with a few critical real-world events, both planned, such as the Rio Olympics, and unplanned, such as the PHEIC announcement. The Rio Olympics was mentioned in >15% of all Zika-related tweets and PHEIC occurred in 27% of Zika-related tweets around their respective peaks. In addition, the overall tweeting dynamics of the top 100 most actively tweeting users on the Zika topic, the top 100 users receiving most RTs, and the top 100 users mentioned were the most highly correlated to and preceded the overall tweeting dynamics, making these groups of users the potential drivers of tweeting dynamics. The top 100 users who retweeted the most were not critical in driving the overall tweeting dynamics. There were very few overlaps among these different groups of potentially influential users.

Conclusions

Using our proposed analytical workflow, EventPeriscope, we identified that Zika discussion dynamics on Twitter were decoupled from the actual disease epidemic in the United States but were closely related to and highly influenced by certain sporadic real-world events as well as by a few influential users. This study provided a methodology framework and insights to better understand the driving forces of web-based public discourse during health emergencies. Therefore, health agencies could deliver more effective and efficient web-based communications in emerging crises.

Ontological Organization and Bioinformatic Analysis of Adverse Drug Reactions From Package Inserts: Development and Usability Study

BACKGROUND

Licensed drugs may cause unexpected adverse reactions in patients, resulting in morbidity, risk of mortality, therapy disruptions, and prolonged hospital stays. Officially approved drug package inserts list the adverse reactions identified from randomized controlled clinical trials with high evidence levels and worldwide postmarketing surveillance. Formal representation of the adverse drug reaction (ADR) enclosed in semistructured package inserts will enable deep recognition of side effects and rational drug use, substantially reduce morbidity, and decrease societal costs.

OBJECTIVE

This paper aims to present an ontological organization of traceable ADR information extracted from licensed package inserts. In addition, it will provide machine-understandable knowledge for bioinformatics analysis, semantic retrieval, and intelligent clinical applications.

METHODS

Based on the essential content of package inserts, a generic ADR ontology model is proposed from two dimensions (and nine subdimensions), covering the ADR information and medication instructions. This is followed by a customized natural language processing method programmed with Python to retrieve the relevant information enclosed in package inserts. After the biocuration and identification of retrieved data from the package insert, an ADR ontology is automatically built for further bioinformatic analysis.

RESULTS

We collected 165 package inserts of quinolone drugs from the National Medical Products Administration and other drug databases in China, and built a specialized ADR ontology containing 2879 classes and 15,711 semantic relations. For each quinolone drug, the reported ADR information and medication instructions have been logically represented and formally organized in an ADR ontology. To demonstrate its usage, the source data were further bioinformatically analyzed. For example, the number of drug-ADR triples and major ADRs associated with each active ingredient were recorded. The 10 ADRs most frequently observed among quinolones were identified and categorized based on the 18 categories defined in the proposal. The occurrence frequency, severity, and ADR mitigation method explicitly stated in package inserts were also analyzed, as well as the top 5 specific populations with contraindications for quinolone drugs.

CONCLUSIONS

Ontological representation and organization using officially approved information from drug package inserts enables the identification and bioinformatic analysis of adverse reactions caused by a specific drug with regard to predefined ADR ontology classes and semantic relations. The resulting ontology-based ADR knowledge source classifies drug-specific adverse reactions, and supports a better understanding of ADRs and safer prescription of medications.

Mining Social Media Data for Biomedical Signals and Health-Related Behavior

Social media data have been increasingly used to study biomedical and health-related phenomena. From cohort-level discussions of a condition to population-level analyses of sentiment, social media have provided scientists with unprecedented amounts of data to study human behavior associated with a variety of health conditions and medical treatments. Here we review recent work in mining social media for biomedical, epidemiological, and social phenomena information relevant to the multilevel complexity of human health. We pay particular attention to topics where social media data analysis has shown the most progress, including pharmacovigilance and sentiment analysis, especially for mental health. We also discuss a variety of innovative uses of social media data for health-related applications as well as important limitations of social media data access and use.

Use of Social Media for Pharmacovigilance Activities: Key Findings and Recommendations from the Vigi4Med Project

The large-scale use of social media by the population has gained the attention of stakeholders and researchers in various fields. In the domain of pharmacovigilance, this new resource was initially considered as an opportunity to overcome underreporting and monitor the safety of drugs in real time in close connection with patients. Research is still required to overcome technical challenges related to data extraction, annotation, and filtering, and there is not yet a clear consensus concerning the systematic exploration and use of social media in pharmacovigilance. Although the literature has mainly considered signal detection, the potential value of social media to support other pharmacovigilance activities should also be explored. The objective of this paper is to present the main findings and subsequent recommendations from the French research project Vigi4Med, which evaluated the use of social media, mainly web forums, for pharmacovigilance activities. This project included an analysis of the existing literature, which contributed to the recommendations presented herein. The recommendations are categorized into three categories: ethical (related to privacy, confidentiality, and follow-up), qualitative (related to the quality of the information), and quantitative (related to statistical analysis). We argue that the progress in information technology and the societal need to consider patients' experiences should motivate future research on social media surveillance for the reinforcement of classical pharmacovigilance.

Integrating Electronic Health Record Data into the ADEpedia-on-OHDSI Platform for Improved Signal Detection: A Case Study of Immune-related Adverse Events

With widespread adoption of electronic health records (EHRs), Real World Data and Real World Evidence (RWE) have been increasingly used by FDA for evaluating drug safety and effectiveness. However, integration of heterogeneous drug safety data sources and systems remains an impediment for effective pharmacovigilance studies. In an ongoing project, we have developed a next generation pharmacovigilance signal detection framework known as ADEpedia-on-OHDSI using the OMOP common data model (CDM). The objective of the study is to demonstrate the feasibility of the framework for integrating both spontaneous reporting data and EHR data for improved signal detection with a case study of immune-related adverse events. We first loaded the OMOP CDM with both recent and legacy FAERS (FDA Adverse Event Reporting System) data (from the time period between Jan. 2004 and Dec. 2018). We also integrated the clinical data from the Mayo Clinic EHR system for six oncological immunotherapy drugs. We implemented a signal detection algorithm and compared the timelines of positive signals detected from both FAERS and EHR data. We found that the signals detected from EHRs are 4 months earlier than signals detected from FAERS database (depending on the signal detection methods used) for the ipilimumab-induced hypopituitarism. Our CDM-based approach would be useful to provide a scalable solution to integrate both drug safety data and EHR data to generate RWE for improved signal detection.

Adverse Events Due to Insomnia Drugs Reported in a Regulatory Database and Online Patient Reviews: Comparative Study

BACKGROUND

Patient online drug reviews are a resource for other patients seeking information about the practical benefits and drawbacks of drug therapies. Patient reviews may also serve as a source of postmarketing safety data that are more user-friendly than regulatory databases. However, the reliability of online reviews has been questioned, because they do not undergo professional review and lack means of verification.

OBJECTIVE

We evaluated online reviews of hypnotic medications, because they are commonly used and their therapeutic efficacy is particularly amenable to patient self-evaluation. Our primary objective was to compare the types and frequencies of adverse events reported to the Food and Drug Administration Adverse Event Reporting System (FAERS) with analogous information in patient reviews on the consumer health website Drugs.com. The secondary objectives were to describe patient reports of efficacy and adverse events and assess the influence of medication cost, effectiveness, and adverse events on user ratings of hypnotic medications.

METHODS

Patient ratings and narratives were retrieved from 1407 reviews on Drugs.com between February 2007 and March 2018 for eszopiclone, ramelteon, suvorexant, zaleplon, and zolpidem. Reviews were coded to preferred terms in the Medical Dictionary for Regulatory Activities. These reviews were compared to 5916 cases in the FAERS database from January 2015 to September 2017.

RESULTS

Similar adverse events were reported to both Drugs.com and FAERS. Both resources identified a lack of efficacy as a common complaint for all five drugs. Both resources revealed that amnesia commonly occurs with eszopiclone, zaleplon, and zolpidem, while nightmares commonly occur with suvorexant. Compared to FAERS, online reviews of zolpidem reported a much higher frequency of amnesia and partial sleep activities. User ratings were highest for zolpidem and lowest for suvorexant. Statistical analyses showed that patient ratings are influenced by considerations of efficacy and adverse events, while drug cost is unimportant.

CONCLUSIONS

For hypnotic medications, online patient reviews and FAERS emphasized similar adverse events. Online reviewers rated drugs based on perception of efficacy and adverse events. We conclude that online patient reviews of hypnotics are a valid source that can supplement traditional adverse event reporting systems.

Exploring Big Data Analytic Approaches to Cancer Blog Text Analysis

In this article, the authors explore the potential of a big data analytics approach to unstructured text analytics of cancer blogs. The application is developed using Cloudera platform's Hadoop MapReduce framework. It uses several text analytics algorithms, including word count, word association, clustering, and classification, to identify and analyze the patterns and keywords in cancer blog postings. This article establishes an exploratory approach to involving big data analytics methods in developing text analytics applications for the analysis of cancer blogs. Additional insights are extracted through various means, including the development of categories or keywords contained in the blogs, the development of a taxonomy, and the examination of relationships among the categories. The application has the potential for generalizability and implementation with health content in other blogs and social media. It can provide insight and decision support for cancer management and facilitate efficient and relevant searches for information related to cancer.

Pharmacology and social media: Potentials and biases of web forums for drug mention analysis-case study of France

The aim of this study is to analyze drug mentions in web forums to evaluate the utility of this data source for drug post-marketing studies. We automatically annotated over 60 million posts extracted from 21 French web forums. Drug mentions detected in this corpus were matched to drug names in a French drug database (Theriaque^®). Our analysis showed that a high proportion of the most frequent drug mentions in the selected web forums correspond to drugs that are usually prescribed to young women, such as combined oral contraceptives. The most mentioned drugs in our corpus correlated weakly to the most prescribed drugs in France but seemed to be influenced by events widely reported in traditional media. In this article, we conclude that web forums have high potential for post-marketing drug-related studies, such as pharmacovigilance, and observation of drug utilization. However, the bias related to forum selection and the corresponding population representativeness should always be taken into account.

A Social Media Study on the Effects of Psychiatric Medication Use

Understanding the effects of psychiatric medications during mental health treatment constitutes an active area of inquiry. While clinical trials help evaluate the effects of these medications, many trials suffer from a lack of generalizability to broader populations. We leverage social media data to examine psychopathological effects subject to self-reported usage of psychiatric medication. Using a list of common approved and regulated psychiatric drugs and a Twitter dataset of 300M posts from 30K individuals, we develop machine learning models to first assess effects relating to mood, cognition, depression, anxiety, psychosis, and suicidal ideation. Then, based on a stratified propensity score based causal analysis, we observe that use of specific drugs are associated with characteristic changes in an individual's psychopathology. We situate these observations in the psychiatry literature, with a deeper analysis of pre-treatment cues that predict treatment outcomes. Our work bears potential to inspire novel clinical investigations and to build tools for digital therapeutics.

A review of drug knowledge discovery using BioNLP and tensor or matrix decomposition

Prediction of the relations among drug and other molecular or social entities is the main knowledge discovery pattern for the purpose of drug-related knowledge discovery. Computational approaches have combined the information from different sources and levels for drug-related knowledge discovery, which provides a sophisticated comprehension of the relationship among drugs, targets, diseases, and targeted genes, at the molecular level, or relationships among drugs, usage, side effect, safety, and user preference, at a social level. In this research, previous work from the BioNLP community and matrix or matrix decomposition was reviewed, compared, and concluded, and eventually, the BioNLP open-shared task was introduced as a promising case study representing this area.

Overview of the First Natural Language Processing Challenge for Extracting Medication, Indication, and Adverse Drug Events from Electronic Health Record Notes (MADE 1.0)

INTRODUCTION

This work describes the Medication and Adverse Drug Events from Electronic Health Records (MADE 1.0) corpus and provides an overview of the MADE 1.0 2018 challenge for extracting medication, indication, and adverse drug events (ADEs) from electronic health record (EHR) notes.

OBJECTIVE

The goal of MADE is to provide a set of common evaluation tasks to assess the state of the art for natural language processing (NLP) systems applied to EHRs supporting drug safety surveillance and pharmacovigilance. We also provide benchmarks on the MADE dataset using the system submissions received in the MADE 2018 challenge.

METHODS

The MADE 1.0 challenge has released an expert-annotated cohort of medication and ADE information comprising 1089 fully de-identified longitudinal EHR notes from 21 randomly selected patients with cancer at the University of Massachusetts Memorial Hospital. Using this cohort as a benchmark, the MADE 1.0 challenge designed three shared NLP tasks. The named entity recognition (NER) task identifies medications and their attributes (dosage, route, duration, and frequency), indications, ADEs, and severity. The relation identification (RI) task identifies relations between the named entities: medication-indication, medication-ADE, and attribute relations. The third shared task (NER-RI) evaluates NLP models that perform the NER and RI tasks jointly. In total, 11 teams from four countries participated in at least one of the three shared tasks, and 41 system submissions were received in total.

RESULTS

The best systems F1 scores for NER, RI, and NER-RI were 0.82, 0.86, and 0.61, respectively. Ensemble classifiers using the team submissions improved the performance further, with an F1 score of 0.85, 0.87, and 0.66 for the three tasks, respectively.

CONCLUSION

MADE results show that recent progress in NLP has led to remarkable improvements in NER and RI tasks for the clinical domain. However, some room for improvement remains, particularly in the NER-RI task.

Methods to Compare Adverse Events in Twitter to FAERS, Drug Information Databases, and Systematic Reviews: Proof of Concept with Adalimumab

INTRODUCTION

Adverse drug reactions (ADRs) are associated with significant health-related and financial burden, and multiple sources are currently utilized to actively discover them. Social media has been proposed as a potential resource for monitoring ADRs, but drug-specific analytical studies comparing social media with other sources are scarce.

OBJECTIVES

Our objective was to develop methods to compare ADRs mentioned in social media with those in traditional sources: the US FDA Adverse Event Reporting System (FAERS), drug information databases (DIDs), and systematic reviews.

METHODS

A total of 10,188 tweets mentioning adalimumab collected between June 2014 and August 2016 were included. ADRs in the corpus were extracted semi-automatically and manually mapped to standardized concepts in the Unified Medical Language System. ADRs were grouped into 16 biologic categories for comparisons. Frequencies, relative frequencies, disproportionality analyses, and rank ordering were used as metrics.

RESULTS

There was moderate agreement between ADRs in social media and traditional sources. "Local and injection site reactions" was the top ADR in Twitter, DIDs, and systematic reviews by frequency, ranked frequency, and index ranking. The next highest ADR in Twitter-fatigue-ranked fifth and seventh in FAERS and DIDs.

CONCLUSION

Social media posts often express mild and symptomatic ADRs, but rates are measured differently in scientific sources. ADRs in FAERS are reported as absolute numbers, in DIDs as percentages, and in systematic reviews as percentages, risk ratios, or other metrics, which makes comparisons challenging; however, overlap is substantial. Social media analysis facilitates open-ended investigation of patient perspectives and may reveal concepts (e.g. anxiety) not available in traditional sources.

Extraction of Information Related to Adverse Drug Events from Electronic Health Record Notes: Design of an End-to-End Model Based on Deep Learning

Background

Pharmacovigilance and drug-safety surveillance are crucial for monitoring adverse drug events (ADEs), but the main ADE-reporting systems such as Food and Drug Administration Adverse Event Reporting System face challenges such as underreporting. Therefore, as complementary surveillance, data on ADEs are extracted from electronic health record (EHR) notes via natural language processing (NLP). As NLP develops, many up-to-date machine-learning techniques are introduced in this field, such as deep learning and multi-task learning (MTL). However, only a few studies have focused on employing such techniques to extract ADEs.

Objective

We aimed to design a deep learning model for extracting ADEs and related information such as medications and indications. Since extraction of ADE-related information includes two steps—named entity recognition and relation extraction—our second objective was to improve the deep learning model using multi-task learning between the two steps.

Methods

We employed the dataset from the Medication, Indication and Adverse Drug Events (MADE) 1.0 challenge to train and test our models. This dataset consists of 1089 EHR notes of cancer patients and includes 9 entity types such as Medication, Indication, and ADE and 7 types of relations between these entities. To extract information from the dataset, we proposed a deep-learning model that uses a bidirectional long short-term memory (BiLSTM) conditional random field network to recognize entities and a BiLSTM-Attention network to extract relations. To further improve the deep-learning model, we employed three typical MTL methods, namely, hard parameter sharing, parameter regularization, and task relation learning, to build three MTL models, called HardMTL, RegMTL, and LearnMTL, respectively.

Results

Since extraction of ADE-related information is a two-step task, the result of the second step (ie, relation extraction) was used to compare all models. We used microaveraged precision, recall, and F1 as evaluation metrics. Our deep learning model achieved state-of-the-art results (F1=65.9%), which is significantly higher than that (F1=61.7%) of the best system in the MADE1.0 challenge. HardMTL further improved the F1 by 0.8%, boosting the F1 to 66.7%, whereas RegMTL and LearnMTL failed to boost the performance.

Conclusions

Deep learning models can significantly improve the performance of ADE-related information extraction. MTL may be effective for named entity recognition and relation extraction, but it depends on the methods, data, and other factors. Our results can facilitate research on ADE detection, NLP, and machine learning.

Utility of social media and crowd-intelligence data for pharmacovigilance: a scoping review

BACKGROUND

A scoping review to characterize the literature on the use of conversations in social media as a potential source of data for detecting adverse events (AEs) related to health products.

METHODS

Our specific research questions were (1) What social media listening platforms exist to detect adverse events related to health products, and what are their capabilities and characteristics? (2) What is the validity and reliability of data from social media for detecting these adverse events? MEDLINE, EMBASE, Cochrane Library, and relevant websites were searched from inception to May 2016. Any type of document (e.g., manuscripts, reports) that described the use of social media data for detecting health product AEs was included. Two reviewers independently screened citations and full-texts, and one reviewer and one verifier performed data abstraction. Descriptive synthesis was conducted.

RESULTS

After screening 3631 citations and 321 full-texts, 70 unique documents with 7 companion reports available from 2001 to 2016 were included. Forty-six documents (66%) described an automated or semi-automated information extraction system to detect health product AEs from social media conversations (in the developmental phase). Seven pre-existing information extraction systems to mine social media data were identified in eight documents. Nineteen documents compared AEs reported in social media data with validated data and found consistent AE discovery in all except two documents. None of the documents reported the validity and reliability of the overall system, but some reported on the performance of individual steps in processing the data. The validity and reliability results were found for the following steps in the data processing pipeline: data de-identification (n = 1), concept identification (n = 3), concept normalization (n = 2), and relation extraction (n = 8). The methods varied widely, and some approaches yielded better results than others.

CONCLUSIONS

Our results suggest that the use of social media conversations for pharmacovigilance is in its infancy. Although social media data has the potential to supplement data from regulatory agency databases; is able to capture less frequently reported AEs; and can identify AEs earlier than official alerts or regulatory changes, the utility and validity of the data source remains under-studied.

TRIAL REGISTRATION

Open Science Framework ( https://osf.io/kv9hu/ ).

Use of Social Media in the Assessment of Relative Effectiveness: Explorative Review With Examples From Oncology

Background

An element of health technology assessment constitutes assessing the clinical effectiveness of drugs, generally called relative effectiveness assessment. Little real-world evidence is available directly after market access, therefore randomized controlled trials are used to obtain information for relative effectiveness assessment. However, there is growing interest in using real-world data for relative effectiveness assessment. Social media may provide a source of real-world data.

Objective

We assessed the extent to which social media-generated health data has provided insights for relative effectiveness assessment.

Methods

An explorative literature review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to identify examples in oncology where health data were collected using social media. Scientific and grey literature published between January 2010 and June 2016 was identified by four reviewers, who independently screened studies for eligibility and extracted data. A descriptive qualitative analysis was performed.

Results

Of 1032 articles identified, eight were included: four articles identified adverse events in response to cancer treatment, three articles disseminated quality of life surveys, and one study assessed the occurrence of disease-specific symptoms. Several strengths of social media-generated health data were highlighted in the articles, such as efficient collection of patient experiences and recruiting patients with rare diseases. Conversely, limitations included validation of authenticity and presence of information and selection bias.

Conclusions

Social media may provide a potential source of real-world data for relative effectiveness assessment, particularly on aspects such as adverse events, symptom occurrence, quality of life, and adherence behavior. This potential has not yet been fully realized and the degree of usefulness for relative effectiveness assessment should be further explored.

Mining Patients' Narratives in Social Media for Pharmacovigilance: Adverse Effects and Misuse of Methylphenidate

Background: The Food and Drug Administration (FDA) in the United States and the European Medicines Agency (EMA) have recognized social media as a new data source to strengthen their activities regarding drug safety.

Objective: Our objective in the ADR-PRISM project was to provide text mining and visualization tools to explore a corpus of posts extracted from social media. We evaluated this approach on a corpus of 21 million posts from five patient forums, and conducted a qualitative analysis of the data available on methylphenidate in this corpus.

Methods: We applied text mining methods based on named entity recognition and relation extraction in the corpus, followed by signal detection using proportional reporting ratio (PRR). We also used topic modeling based on the Correlated Topic Model to obtain the list of the matics in the corpus and classify the messages based on their topics.

Results: We automatically identified 3443 posts about methylphenidate published between 2007 and 2016, among which 61 adverse drug reactions (ADR) were automatically detected. Two pharmacovigilance experts evaluated manually the quality of automatic identification, and a f-measure of 0.57 was reached. Patient's reports were mainly neuro-psychiatric effects. Applying PRR, 67% of the ADRs were signals, including most of the neuro-psychiatric symptoms but also palpitations. Topic modeling showed that the most represented topics were related to Childhood and Treatment initiation, but also Side effects. Cases of misuse were also identified in this corpus, including recreational use and abuse.

Conclusion: Named entity recognition combined with signal detection and topic modeling have demonstrated their complementarity in mining social media data. An in-depth analysis focused on methylphenidate showed that this approach was able to detect potential signals and to provide better understanding of patients' behaviors regarding drugs, including misuse.

Causality Patterns for Detecting Adverse Drug Reactions From Social Media: Text Mining Approach

Background

Detecting adverse drug reactions (ADRs) is an important task that has direct implications for the use of that drug. If we can detect previously unknown ADRs as quickly as possible, then this information can be provided to the regulators, pharmaceutical companies, and health care organizations, thereby potentially reducing drug-related morbidity and saving lives of many patients. A promising approach for detecting ADRs is to use social media platforms such as Twitter and Facebook. A high level of correlation between a drug name and an event may be an indication of a potential adverse reaction associated with that drug. Although numerous association measures have been proposed by the signal detection community for identifying ADRs, these measures are limited in that they detect correlations but often ignore causality.

Objective

This study aimed to propose a causality measure that can detect an adverse reaction that is caused by a drug rather than merely being a correlated signal.

Methods

To the best of our knowledge, this was the first causality-sensitive approach for detecting ADRs from social media. Specifically, the relationship between a drug and an event was represented using a set of automatically extracted lexical patterns. We then learned the weights for the extracted lexical patterns that indicate their reliability for expressing an adverse reaction of a given drug.

Results

Our proposed method obtains an ADR detection accuracy of 74% on a large-scale manually annotated dataset of tweets, covering a standard set of drugs and adverse reactions.

Conclusions

By using lexical patterns, we can accurately detect the causality between drugs and adverse reaction–related events.

Descriptions of Adverse Drug Reactions Are Less Informative in Forums Than in the French Pharmacovigilance Database but Provide More Unexpected Reactions

Background: Social media have drawn attention for their potential use in Pharmacovigilance. Recent work showed that it is possible to extract information concerning adverse drug reactions (ADRs) from posts in social media. The main objective of the Vigi4MED project was to evaluate the relevance and quality of the information shared by patients on web forums about drug safety and its potential utility for pharmacovigilance.

Methods: After selecting websites of interest, we manually evaluated the relevance of the content of posts for pharmacovigilance related to six drugs (agomelatine, baclofen, duloxetine, exenatide, strontium ranelate, and tetrazepam). We compared forums to the French Pharmacovigilance Database (FPVD) to (1) evaluate whether they contained relevant information to characterize a pharmacovigilance case report (patient’s age and sex; treatment indication, dose and duration; time-to-onset (TTO) and outcome of the ADR, and drug dechallenge and rechallenge) and (2) perform impact analysis (nature, seriousness, unexpectedness, and outcome of the ADR).

Results: The cases in the FPVD were significantly more informative than posts in forums for patient description (age, sex), treatment description (dose, duration, TTO), and outcome of the ADR, but the indication for the treatment was more often found in forums. Cases were more often serious in the FPVD than in forums (46% vs. 4%), but forums more often contained an unexpected ADR than the FPVD (24% vs. 17%). Moreover, 197 unexpected ADRs identified in forums were absent from the FPVD and the distribution of the MedDRA System Organ Classes (SOCs) was different between the two data sources.

Discussion: This study is the first to evaluate if patients’ posts may qualify as potential and informative case reports that should be stored in a pharmacovigilance database in the same way as case reports submitted by health professionals. The posts were less informative (except for the indication) and focused on less serious ADRs than the FPVD cases, but more unexpected ADRs were presented in forums than in the FPVD and their SOCs were different. Thus, web forums should be considered as a secondary, but complementary source for pharmacovigilance.

Clinical Relation Extraction Toward Drug Safety Surveillance Using Electronic Health Record Narratives: Classical Learning Versus Deep Learning

BACKGROUND

Medication and adverse drug event (ADE) information extracted from electronic health record (EHR) notes can be a rich resource for drug safety surveillance. Existing observational studies have mainly relied on structured EHR data to obtain ADE information; however, ADEs are often buried in the EHR narratives and not recorded in structured data.

OBJECTIVE

To unlock ADE-related information from EHR narratives, there is a need to extract relevant entities and identify relations among them. In this study, we focus on relation identification. This study aimed to evaluate natural language processing and machine learning approaches using the expert-annotated medical entities and relations in the context of drug safety surveillance, and investigate how different learning approaches perform under different configurations.

METHODS

We have manually annotated 791 EHR notes with 9 named entities (eg, medication, indication, severity, and ADEs) and 7 different types of relations (eg, medication-dosage, medication-ADE, and severity-ADE). Then, we explored 3 supervised machine learning systems for relation identification: (1) a support vector machines (SVM) system, (2) an end-to-end deep neural network system, and (3) a supervised descriptive rule induction baseline system. For the neural network system, we exploited the state-of-the-art recurrent neural network (RNN) and attention models. We report the performance by macro-averaged precision, recall, and F1-score across the relation types.

RESULTS

Our results show that the SVM model achieved the best average F1-score of 89.1% on test data, outperforming the long short-term memory (LSTM) model with attention (F1-score of 65.72%) as well as the rule induction baseline system (F1-score of 7.47%) by a large margin. The bidirectional LSTM model with attention achieved the best performance among different RNN models. With the inclusion of additional features in the LSTM model, its performance can be boosted to an average F1-score of 77.35%.

CONCLUSIONS

It shows that classical learning models (SVM) remains advantageous over deep learning models (RNN variants) for clinical relation identification, especially for long-distance intersentential relations. However, RNNs demonstrate a great potential of significant improvement if more training data become available. Our work is an important step toward mining EHRs to improve the efficacy of drug safety surveillance. Most importantly, the annotated data used in this study will be made publicly available, which will further promote drug safety research in the community.

Hybrid Semantic Analysis for Mapping Adverse Drug Reaction Mentions in Tweets to Medical Terminology

Social networks, such as Twitter, have become important sources for active monitoring of user-reported adverse drug reactions (ADRs). Automatic extraction of ADR information can be crucial for healthcare providers, drug manufacturers, and consumers. However, because of the non-standard nature of social media language, automatically extracted ADR mentions need to be mapped to standard forms before they can be used by operational pharmacovigilance systems. We propose a modular natural language processing pipeline for mapping (normalizing) colloquial mentions of ADRs to their corresponding standardized identifiers. We seek to accomplish this task and enable customization of the pipeline so that distinct unlabeled free text resources can be incorporated to use the system for other normalization tasks. Our approach, which we call Hybrid Semantic Analysis (HSA), sequentially employs rule-based and semantic matching algorithms for mapping user-generated mentions to concept IDs in the Unified Medical Language System vocabulary. The semantic matching component of HSA is adaptive in nature and uses a regression model to combine various measures of semantic relatedness and resources to optimize normalization performance on the selected data source. On a publicly available corpus, our normalization method achieves 0.502 recall and 0.823 precision (F-measure: 0.624). Our proposed method outperforms a baseline based on latent semantic analysis and another that uses MetaMap.

Detection of Adverse Drug Reactions using Medical Named Entities on Twitter

Adverse Drug Reactions (ADRs) are unintentional reactions caused by a drug or combination of drugs taken by a patient. The current ADR reporting systems inevitably have delays in reporting such events. The broad scope of social media conversations on sites such as Twitter means that inevitably health-related topics will be covered. This means that these sites could then be used to detect potentially novel ADRs with less latency for subsequent further investigation. In this work, we investigate ADR surveillance using a large corpus of Twitter data, containing around 50 billion tweets spanning 3 years (2012-2014), and evaluate against over 3000 drugs reported in the FAERS database. This is both a larger corpus and broader selection of drugs than previous work in the domain. We compare the ADRs identified using our method to the FDA Adverse Event Reporting System (FAERS) database of ADRs reported using more traditional techniques, and find that Twitter is a useful resource for ADR detection up to 72% micro-averaged precision. Micro-averaged recall of 6% is achievable using only 10% of Twitter, indicating that with a higher-volume or targeted feed it would be possible to detect a large percentage of ADRs.

Detection of Cases of Noncompliance to Drug Treatment in Patient Forum Posts: Topic Model Approach

Background

Medication nonadherence is a major impediment to the management of many health conditions. A better understanding of the factors underlying noncompliance to treatment may help health professionals to address it. Patients use peer-to-peer virtual communities and social media to share their experiences regarding their treatments and diseases. Using topic models makes it possible to model themes present in a collection of posts, thus to identify cases of noncompliance.

Objective

The aim of this study was to detect messages describing patients’ noncompliant behaviors associated with a drug of interest. Thus, the objective was the clustering of posts featuring a homogeneous vocabulary related to nonadherent attitudes.

Methods

We focused on escitalopram and aripiprazole used to treat depression and psychotic conditions, respectively. We implemented a probabilistic topic model to identify the topics that occurred in a corpus of messages mentioning these drugs, posted from 2004 to 2013 on three of the most popular French forums. Data were collected using a Web crawler designed by Kappa Santé as part of the Detec’t project to analyze social media for drug safety. Several topics were related to noncompliance to treatment.

Results

Starting from a corpus of 3650 posts related to an antidepressant drug (escitalopram) and 2164 posts related to an antipsychotic drug (aripiprazole), the use of latent Dirichlet allocation allowed us to model several themes, including interruptions of treatment and changes in dosage.

The topic model approach detected cases of noncompliance behaviors with a recall of 98.5% (272/276) and a precision of 32.6% (272/844).

Conclusions

Topic models enabled us to explore patients’ discussions on community websites and to identify posts related with noncompliant behaviors. After a manual review of the messages in the noncompliance topics, we found that noncompliance to treatment was present in 6.17% (276/4469) of the posts.

Big Data's Role in Precision Public Health

Precision public health is an emerging practice to more granularly predict and understand public health risks and customize treatments for more specific and homogeneous subpopulations, often using new data, technologies, and methods. Big data is one element that has consistently helped to achieve these goals, through its ability to deliver to practitioners a volume and variety of structured or unstructured data not previously possible. Big data has enabled more widespread and specific research and trials of stratifying and segmenting populations at risk for a variety of health problems. Examples of success using big data are surveyed in surveillance and signal detection, predicting future risk, targeted interventions, and understanding disease. Using novel big data or big data approaches has risks that remain to be resolved. The continued growth in volume and variety of available data, decreased costs of data capture, and emerging computational methods mean big data success will likely be a required pillar of precision public health into the future. This review article aims to identify the precision public health use cases where big data has added value, identify classes of value that big data may bring, and outline the risks inherent in using big data in precision public health efforts.

Social Media Impact of the Food and Drug Administration's Drug Safety Communication Messaging About Zolpidem: Mixed-Methods Analysis

Background

The Food and Drug Administration (FDA) issues drug safety communications (DSCs) to health care professionals, patients, and the public when safety issues emerge related to FDA-approved drug products. These safety messages are disseminated through social media to ensure broad uptake.

Objective

The objective of this study was to assess the social media dissemination of 2 DSCs released in 2013 for the sleep aid zolpidem.

Methods

We used the MedWatcher Social program and the DataSift historic query tool to aggregate Twitter and Facebook posts from October 1, 2012 through August 31, 2013, a period beginning approximately 3 months before the first DSC and ending 3 months after the second. Posts were categorized as (1) junk, (2) mention, and (3) adverse event (AE) based on a score between –0.2 (completely unrelated) to 1 (perfectly related). We also looked at Google Trends data and Wikipedia edits for the same time period. Google Trends search volume is scaled on a range of 0 to 100 and includes “Related queries” during the relevant time periods. An interrupted time series (ITS) analysis assessed the impact of DSCs on the counts of posts with specific mention of zolpidem-containing products. Chow tests for known structural breaks were conducted on data from Twitter, Facebook, and Google Trends. Finally, Wikipedia edits were pulled from the website’s editorial history, which lists all revisions to a given page and the editor’s identity.

Results

In total, 174,286 Twitter posts and 59,641 Facebook posts met entry criteria. Of those, 16.63% (28,989/174,286) of Twitter posts and 25.91% (15,453/59,641) of Facebook posts were labeled as junk and excluded. AEs and mentions represented 9.21% (16,051/174,286) and 74.16% (129,246/174,286) of Twitter posts and 5.11% (3,050/59,641) and 68.98% (41,138/59,641) of Facebook posts, respectively. Total daily counts of posts about zolpidem-containing products increased on Twitter and Facebook on the day of the first DSC; Google searches increased on the week of the first DSC. ITS analyses demonstrated variability but pointed to an increase in interest around the first DSC. Chow tests were significant (P<.0001) for both DSCs on Facebook and Twitter, but only the first DSC on Google Trends. Wikipedia edits occurred soon after each DSC release, citing news articles rather than the DSC itself and presenting content that needed subsequent revisions for accuracy.

Conclusions

Social media offers challenges and opportunities for dissemination of the DSC messages. The FDA could consider strategies for more actively disseminating DSC safety information through social media platforms, particularly when announcements require updating. The FDA may also benefit from directly contributing content to websites like Wikipedia that are frequently accessed for drug-related information.

HARNESSING SOCIAL MEDIA FOR HEALTH INFORMATION MANAGEMENT

The remarkable upsurge of social media has dramatic impacts on health care research and practice in the past decade. Social media are reshaping health information management in a variety of ways, ranging from providing cost-effective ways to improve clinician-patient communication and exchange health-related information and experience, to enabling the discovery of new medical knowledge and information. Despite some demonstrated initial success, social media use and analytics for improving health as a research field is still at its infancy. Information systems researchers can potentially play a key role in advancing the field. This study proposes a conceptual framework for social media-based health information management by drawing on multi-disciplinary research. With the guidance of the framework, this research presents related research challenges, identifies important yet under-explored research issues, and discusses promising directions for future research.