Biases in using social media data for public health surveillance: A scoping review

Epidemiological methods in transition: Minimizing biases in classical and digital approaches

Epidemiology and Public Health have increasingly relied on structured and unstructured data, collected inside and outside of typical health systems, to study, identify, and mitigate diseases at the population level. Focusing on infectious diseases, we review the state of Digital Epidemiology at the beginning of 2020 and how it changed after the COVID-19 pandemic, in both nature and breadth. We argue that Epidemiology's progressive use of data generated outside of clinical and public health systems creates several technical challenges, particularly in carrying specific biases that are almost impossible to correct for a priori. Using a statistical perspective, we discuss how a definition of Digital Epidemiology that emphasizes "data-type" instead of "data-source," may be more operationally useful, by clarifying key methodological differences and gaps. Therefore, we briefly describe some of the possible biases arising from varied collection methods and sources, and offer some recommendations to better explore the potential of Digital Epidemiology, particularly on how to help reduce inequity.

Targeting COVID-19 and Human Resources for Health News Information Extraction: Algorithm Development and Validation

BACKGROUND

Global pandemics like COVID-19 put a high amount of strain on health care systems and health workers worldwide. These crises generate a vast amount of news information published online across the globe. This extensive corpus of articles has the potential to provide valuable insights into the nature of ongoing events and guide interventions and policies. However, the sheer volume of information is beyond the capacity of human experts to process and analyze effectively.

OBJECTIVE

The aim of this study was to explore how natural language processing (NLP) can be leveraged to build a system that allows for quick analysis of a high volume of news articles. Along with this, the objective was to create a workflow comprising human-computer symbiosis to derive valuable insights to support health workforce strategic policy dialogue, advocacy, and decision-making.

METHODS

We conducted a review of open-source news coverage from January 2020 to June 2022 on COVID-19 and its impacts on the health workforce from the World Health Organization (WHO) Epidemic Intelligence from Open Sources (EIOS) by synergizing NLP models, including classification and extractive summarization, and human-generated analyses. Our DeepCovid system was trained on 2.8 million news articles in English from more than 3000 internet sources across hundreds of jurisdictions.

RESULTS

Rules-based classification with hand-designed rules narrowed the data set to 8508 articles with high relevancy confirmed in the human-led evaluation. DeepCovid's automated information targeting component reached a very strong binary classification performance of 98.98 for the area under the receiver operating characteristic curve (ROC-AUC) and 47.21 for the area under the precision recall curve (PR-AUC). Its information extraction component attained good performance in automatic extractive summarization with a mean Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score of 47.76. DeepCovid's final summaries were used by human experts to write reports on the COVID-19 pandemic.

CONCLUSIONS

It is feasible to synergize high-performing NLP models and human-generated analyses to benefit open-source health workforce intelligence. The DeepCovid approach can contribute to an agile and timely global view, providing complementary information to scientific literature.

Tracking the Spread of Pollen on Social Media Using Pollen-Related Messages From Twitter: Retrospective Analysis

BACKGROUND

Allergy disorders caused by biological particles, such as the proteins in some airborne pollen grains, are currently considered one of the most common chronic diseases, and European Academy of Allergy and Clinical Immunology forecasts indicate that within 15 years 50% of Europeans will have some kind of allergy as a consequence of urbanization, industrialization, pollution, and climate change.

OBJECTIVE

The aim of this study was to monitor and analyze the dissemination of information about pollen symptoms from December 2006 to January 2022. By conducting a comprehensive evaluation of public comments and trends on Twitter, the research sought to provide valuable insights into the impact of pollen on sensitive individuals, ultimately enhancing our understanding of how pollen-related information spreads and its implications for public health awareness.

METHODS

Using a blend of large language models, dimensionality reduction, unsupervised clustering, and term frequency-inverse document frequency, alongside visual representations such as word clouds and semantic interaction graphs, our study analyzed Twitter data to uncover insights on respiratory allergies. This concise methodology enabled the extraction of significant themes and patterns, offering a deep dive into public knowledge and discussions surrounding respiratory allergies on Twitter.

RESULTS

The months between March and August had the highest volume of messages. The percentage of patient tweets appeared to increase notably during the later years, and there was also a potential increase in the prevalence of symptoms, mainly in the morning hours, indicating a potential rise in pollen allergies and related discussions on social media. While pollen allergy is a global issue, specific sociocultural, political, and economic contexts mean that patients experience symptomatology at a localized level, needing appropriate localized responses.

CONCLUSIONS

The interpretation of tweet information represents a valuable tool to take preventive measures to mitigate the impact of pollen allergy on sensitive patients to achieve equity in living conditions and enhance access to health information and services.

Identifying Medicine Shortages With the Twitter Social Network: Retrospective Observational Study

BACKGROUND

Early identification is critical for mitigating the impact of medicine shortages on patients. The internet, specifically social media, is an emerging source of health data.

OBJECTIVE

This study aimed to explore whether a routine analysis of data from the Twitter social network can detect signals of a medicine shortage and serve as an early warning system and, if so, for which medicines or patient groups.

METHODS

Medicine shortages between January 31 and December 1, 2019, were collected from the Dutch pharmacists' society's national catalog Royal Dutch Pharmacists Association (KNMP) Farmanco. Posts on these shortages were collected by searching for the name, the active pharmaceutical ingredient, or the first word of the brand name of the medicines in shortage. Posts were then selected based on relevant keywords that potentially indicated a shortage and the percentage of shortages with at least 1 post was calculated. The first posts per shortage were analyzed for their timing (median number of days, including the IQR) versus the national catalog, also stratified by disease and medicine characteristics. The content of the first post per shortage was analyzed descriptively for its reporting stakeholder and the nature of the post.

RESULTS

Of the 341 medicine shortages, 102 (29.9%) were mentioned on Twitter. Of these 102 shortages, 18 (5.3% of the total) were mentioned prior to or simultaneous to publication by KNMP Farmanco. Only 4 (1.2%) of these were mentioned on Twitter more than 14 days before. On average, posts were published with a median delay of 37 (IQR 7-81) days to publication by KNMP Farmanco. Shortages mentioned on Twitter affected a greater number of patients and lasted longer than those that were not mentioned. We could not conclusively relate either the presence or absence on Twitter to a disease area or route of administration of the medicine in shortage. The first posts on the 102 shortages were mainly published by patients (n=51, 50.0%) and health care professionals (n=46, 45.1%). We identified 8 categories of nature of content. Sharing personal experience (n=44, 43.1%) was the most common category.

CONCLUSIONS

The Twitter social network is not a suitable early warning system for medicine shortages. Twitter primarily echoes already-known information rather than spreads new information. However, Twitter or potentially any other social media platform provides the opportunity for future qualitative research in the increasingly important field of medicine shortages that investigates how a larger population of patients is affected by shortages.

Natural Language Processing-Powered Real-Time Monitoring Solution for Vaccine Sentiments and Hesitancy on Social Media: System Development and Validation

BACKGROUND

Vaccines serve as a crucial public health tool, although vaccine hesitancy continues to pose a significant threat to full vaccine uptake and, consequently, community health. Understanding and tracking vaccine hesitancy is essential for effective public health interventions; however, traditional survey methods present various limitations.

OBJECTIVE

This study aimed to create a real-time, natural language processing (NLP)-based tool to assess vaccine sentiment and hesitancy across 3 prominent social media platforms.

METHODS

We mined and curated discussions in English from Twitter (subsequently rebranded as X), Reddit, and YouTube social media platforms posted between January 1, 2011, and October 31, 2021, concerning human papillomavirus; measles, mumps, and rubella; and unspecified vaccines. We tested multiple NLP algorithms to classify vaccine sentiment into positive, neutral, or negative and to classify vaccine hesitancy using the World Health Organization's (WHO) 3Cs (confidence, complacency, and convenience) hesitancy model, conceptualizing an online dashboard to illustrate and contextualize trends.

RESULTS

We compiled over 86 million discussions. Our top-performing NLP models displayed accuracies ranging from 0.51 to 0.78 for sentiment classification and from 0.69 to 0.91 for hesitancy classification. Explorative analysis on our platform highlighted variations in online activity about vaccine sentiment and hesitancy, suggesting unique patterns for different vaccines.

CONCLUSIONS

Our innovative system performs real-time analysis of sentiment and hesitancy on 3 vaccine topics across major social networks, providing crucial trend insights to assist campaigns aimed at enhancing vaccine uptake and public health.

Social Acceptability of Health Behavior Posts on Social Media: An Experiment

INTRODUCTION

Social media sites like Twitter (now X) are increasingly used to create health behavior metrics for public health surveillance. Yet little is known about social norms that may bias the content of posts about health behaviors. Social norms for posts about four health behaviors (smoking tobacco, drinking alcohol, physical activity, eating food) on Twitter/X were evaluated.

METHODS

This was a randomized experiment delivered via web-based survey to adult, English-speaking Twitter/X users in three Michigan, USA, counties from 2020 to 2022 (n=559). Each participant viewed 24 posts presenting experimental manipulations regarding four health behaviors and answered questions about each post's social acceptability. Principal component analysis was used to combine survey responses into one perceived social acceptability measure. Linear mixed models with the Benjamini-Hochberg correction were implemented to test seven study hypotheses in 2023.

RESULTS

Supporting six hypotheses, posts presenting healthier (CI: 0.028, 0.454), less stigmatized behaviors (CI: 0.552, 0.157) were more socially acceptable than posts regarding unhealthier, stigmatized behaviors. Unhealthy (CI: -0.268, -0.109) and stigmatized behavior (CI: -0.261, -0.103) posts were less acceptable for more educated participants. Posts about collocated activities (CI: 0.410, 0.573) and accompanied by expressions of liking (CI: 0.906, 1.11) were more acceptable than activities undertaken alone or disliked. Contrary to one hypothesis, posts reporting unusual activities were less acceptable than usual ones (CI: -0.472, 0.312).

CONCLUSIONS

Perceived social acceptability may be associated with the frequency and content of health behavior posts. Users of Twitter/X and other social media platform posts to estimate health behavior prevalence should account for potential estimation biases from perceived social acceptability of posts.

BERT-based language model for accurate drug adverse event extraction from social media: implementation, evaluation, and contributions to pharmacovigilance practices

Introduction

Social media platforms serve as a valuable resource for users to share health-related information, aiding in the monitoring of adverse events linked to medications and treatments in drug safety surveillance. However, extracting drug-related adverse events accurately and efficiently from social media poses challenges in both natural language processing research and the pharmacovigilance domain.

Method

Recognizing the lack of detailed implementation and evaluation of Bidirectional Encoder Representations from Transformers (BERT)-based models for drug adverse event extraction on social media, we developed a BERT-based language model tailored to identifying drug adverse events in this context. Our model utilized publicly available labeled adverse event data from the ADE-Corpus-V2. Constructing the BERT-based model involved optimizing key hyperparameters, such as the number of training epochs, batch size, and learning rate. Through ten hold-out evaluations on ADE-Corpus-V2 data and external social media datasets, our model consistently demonstrated high accuracy in drug adverse event detection.

Result

The hold-out evaluations resulted in average F1 scores of 0.8575, 0.9049, and 0.9813 for detecting words of adverse events, words in adverse events, and words not in adverse events, respectively. External validation using human-labeled adverse event tweets data from SMM4H further substantiated the effectiveness of our model, yielding F1 scores 0.8127, 0.8068, and 0.9790 for detecting words of adverse events, words in adverse events, and words not in adverse events, respectively.

Discussion

This study not only showcases the effectiveness of BERT-based language models in accurately identifying drug-related adverse events in the dynamic landscape of social media data, but also addresses the need for the implementation of a comprehensive study design and evaluation. By doing so, we contribute to the advancement of pharmacovigilance practices and methodologies in the context of emerging information sources like social media.

Patient Perception and Self-Reported Outcomes with Presbyopia-Correcting Intraocular Lenses (PCIOLs): A Social Media Listening Study

INTRODUCTION

Presbyopia-correcting intraocular lens (PCIOL) implantation is a popular treatment option for cataract surgery patients who desire spectacle independence. This study aimed to understand patient perception and outcomes with PCIOLs by analyzing patient social media posts.

METHODS

This was a non-interventional retrospective study that used predefined search strings to identify publicly available social media data discussing patient perceptions and outcomes with seven PCIOLs (three trifocal, one multifocal with continuous range of vision, and three extended depth-of-focus [EDOF] PCIOLs). Relevant posts were searched from Reddit, YouTube, and Facebook and patient forums Patient.info, Medicine.net, Optiker-Forum, and Medizin Forum from September 2020 to October 2022 in four languages (English, German, French, and Spanish).

RESULTS

A total of 2237 posts were included, all in English, with 68% of posts identified on Patient.info. The themes most discussed by patients were quality of vision (69% of total posts), patient experience after PCIOL implantation (30%), patient perception before PCIOL implantation (26%), and visual disturbances (24%). Most discussed PCIOLs were Vivity® (58% of total posts), PanOptix® (38%), Synergy® (26%), and Symfony® (13%). Patient perception of PCIOLs was most frequently influenced by healthcare professionals, online reading, and online videos (31%, 18%, and 15% of posts, respectively). A total of 215 posts (10% of total) discussed glasses use after PCIOL surgery: for EDOF and trifocal/multifocal PCIOLs, 37% and 56% of posts discussing glasses use stated being glasses free, respectively. A total of 537 posts discussed visual disturbances: halos/rings (66%) and starbursts (36%) were the most discussed visual disturbances for all lens types. Being glasses free after PCIOL implantation appeared to be a key driver of patient satisfaction.

CONCLUSION

Social media provides a rich source of information on patient perception, experience, and overall satisfaction of PCIOLs that can be used to complement and guide the collection of further evidence generated through controlled trials.

Self-reported Xylazine Experiences: A Mixed-methods Study of Reddit Subscribers

OBJECTIVES

Xylazine is an α 2 -agonist increasingly prevalent in the illicit drug supply. Our objectives were to curate information about xylazine through social media from people who use drugs (PWUDs). Specifically, we sought to answer the following: (1) What are the demographics of Reddit subscribers reporting exposure to xylazine? (2) Is xylazine a desired additive? And (3) what adverse effects of xylazine are PWUDs experiencing?

METHODS

Natural language processing (NLP) was used to identify mentions of "xylazine" from posts by Reddit subscribers who also posted on drug-related subreddits. Posts were qualitatively evaluated for xylazine-related themes. A survey was developed to gather additional information about the Reddit subscribers. This survey was posted on subreddits that were identified by NLP to contain xylazine-related discussions from March 2022 to October 2022.

RESULTS

Seventy-six posts were extracted via NLP from 765,616 posts by 16,131 Reddit subscribers (January 2018 to August 2021). People on Reddit described xylazine as an unwanted adulterant in their opioid supply. Sixty-one participants completed the survey. Of those who disclosed their location, 25 of 50 participants (50%) reported locations in the Northeastern United States. The most common route of xylazine use was intranasal use (57%). Thirty-one of 59 (53%) reported experiencing xylazine withdrawal. Frequent adverse events reported were prolonged sedation (81%) and increased skin wounds (43%).

CONCLUSIONS

Among respondents on these Reddit forums, xylazine seems to be an unwanted adulterant. People who use drugs may be experiencing adverse effects such as prolonged sedation and xylazine withdrawal. This seemed to be more common in the Northeast.

Evaluating the perceptions of pesticide use, safety, and regulation and identifying common pesticide-related topics on Twitter

Synthetic pesticides are important agricultural tools that increase crop yield and help feed the world's growing population. These products are also highly regulated to balance benefits and potential environmental and human risks. Public perception of pesticide use, safety, and regulation is an important topic necessitating discussion across a variety of stakeholders from lay consumers to regulatory agencies since attitudes toward this subject could differ markedly. Individuals and organizations can perceive the same message(s) about pesticides differently due to prior differences in technical knowledge, perceptions, attitudes, and individual or group circumstances. Social media platforms, like Twitter, include both individuals and organizations and function as a townhall where each group promotes their topics of interest, shares their perspectives, and engages in both well-informed and misinformed discussions. We analyzed public Twitter posts about pesticides by user group, time, and location to understand their communication behaviors, including their sentiments and discussion topics, using machine learning-based text analysis methods. We extracted tweets related to pesticides between 2013 and 2021 based on relevant keywords developed through a "snowball" sampling process. Each tweet was grouped into individual versus organizational groups, then further categorized into media, government, industry, academia, and three types of nongovernmental organizations. We compared topic distributions within and between those groups using topic modeling and then applied sentiment analysis to understand the public's attitudes toward pesticide safety and regulation. Individual accounts expressed concerns about health and environmental risks, while industry and government accounts focused on agricultural usage and regulations. Public perceptions are heavily skewed toward negative sentiments, although this varies geographically. Our findings can help managers and decision-makers understand public sentiments, priorities, and perceptions and provide insights into public discourse on pesticides. Integr Environ Assess Manag 2023;19:1581-1599. © 2023 The Authors. Integrated Environmental Assessment and Management published by Wiley Periodicals LLC on behalf of Society of Environmental Toxicology & Chemistry (SETAC).

Sentiment analysis of epidemiological surveillance reports on COVID-19 in Greece using machine learning models

The present research deals with sentiment analysis performed with Microsoft Azure Machine Learning Studio to classify Facebook posts on the Greek National Public Health Organization (EODY) from November 2021 to January 2022 during the pandemic. Positive, negative and neutral sentiments were included after processing 300 reviews. This approach involved analyzing the words appearing in the comments and exploring the sentiments related to daily surveillance reports of COVID-19 published on the EODY Facebook page. Moreover, machine learning algorithms were implemented to predict the classification of sentiments. This research assesses the efficiency of a few popular machine learning models, which is one of the initial efforts in Greece in this domain. People have negative sentiments toward COVID surveillance reports. Words with the highest frequency of occurrence include government, vaccinated people, unvaccinated, telephone communication, health measures, virus, COVID-19 rapid/molecular tests, and of course, COVID-19. The experimental results disclose additionally that two classifiers, namely two class Neural Network and two class Bayes Point Machine, achieved high sentiment analysis accuracy and F1 score, particularly 87% and over 35%. A significant limitation of this study may be the need for more comparison with other research attempts that identified the sentiments of the EODY surveillance reports of COVID in Greece. Machine learning models can provide critical information combating public health hazards and enrich communication strategies and proactive actions in public health issues and opinion management during the COVID-19 pandemic.

#ChronicPain: Automated Building of a Chronic Pain Cohort from Twitter Using Machine Learning

Background

Due to the high burden of chronic pain, and the detrimental public health consequences of its treatment with opioids, there is a high-priority need to identify effective alternative therapies. Social media is a potentially valuable resource for knowledge about self-reported therapies by chronic pain sufferers.

Methods

We attempted to (a) verify the presence of large-scale chronic pain-related chatter on Twitter, (b) develop natural language processing and machine learning methods for automatically detecting self-disclosures, (c) collect longitudinal data posted by them, and (d) semiautomatically analyze the types of chronic pain-related information reported by them. We collected data using chronic pain-related hashtags and keywords and manually annotated 4,998 posts to indicate if they were self-reports of chronic pain experiences. We trained and evaluated several state-of-the-art supervised text classification models and deployed the best-performing classifier. We collected all publicly available posts from detected cohort members and conducted manual and natural language processing-driven descriptive analyses.

Results

Interannotator agreement for the binary annotation was 0.82 (Cohen's kappa). The RoBERTa model performed best (F1 score: 0.84; 95% confidence interval: 0.80 to 0.89), and we used this model to classify all collected unlabeled posts. We discovered 22,795 self-reported chronic pain sufferers and collected over 3 million of their past posts. Further analyses revealed information about, but not limited to, alternative treatments, patient sentiments about treatments, side effects, and self-management strategies.

Conclusion

Our social media based approach will result in an automatically growing large cohort over time, and the data can be leveraged to identify effective opioid-alternative therapies for diverse chronic pain types.