Content and Dynamics of Websites Shared Over Vaccine-Related Tweets in COVID-19 Conversations: Computational Analysis

Accuracy of a large language model in distinguishing anti- and pro-vaccination messages on social media: The case of human papillomavirus vaccination

Objective

Vaccination has engendered a spectrum of public opinions, with social media acting as a crucial platform for health-related discussions. The emergence of artificial intelligence technologies, such as large language models (LLMs), offers a novel opportunity to efficiently investigate public discourses. This research assesses the accuracy of ChatGPT, a widely used and freely available service built upon an LLM, for sentiment analysis to discern different stances toward Human Papillomavirus (HPV) vaccination.

Methods

Messages related to HPV vaccination were collected from social media supporting different message formats: Facebook (long format) and Twitter (short format). A selection of 1,000 human-evaluated messages was input into the LLM, which generated multiple response instances containing its classification results. Accuracy was measured for each message as the level of concurrence between human and machine decisions, ranging between 0 and 1.

Results

Average accuracy was notably high when 20 response instances were used to determine the machine decision of each message: .882 (SE = .021) and .750 (SE = .029) for anti- and pro-vaccination long-form; .773 (SE = .027) and .723 (SE = .029) for anti- and pro-vaccination short-form, respectively. Using only three or even one instance did not lead to a severe decrease in accuracy. However, for long-form messages, the language model exhibited significantly lower accuracy in categorizing pro-vaccination messages than anti-vaccination ones.

Conclusions

ChatGPT shows potential in analyzing public opinions on HPV vaccination using social media content. However, understanding the characteristics and limitations of a language model within specific public health contexts remains imperative.

Network embedding aided vaccine skepticism detection

We investigate automatic methods to assess COVID vaccination views in Twitter content. Vaccine skepticism has been a controversial topic of long history that has become more important than ever with the COVID-19 pandemic. Our main goal is to demonstrate the importance of network effects in detecting vaccination skeptic content. Towards this end, we collected and manually labeled vaccination-related Twitter content in the first half of 2021. Our experiments confirm that the network carries information that can be exploited to improve the accuracy of classifying attitudes towards vaccination over content classification as baseline. We evaluate a variety of network embedding algorithms, which we combine with text embedding to obtain classifiers for vaccination skeptic content. In our experiments, by using Walklets, we improve the AUC of the best classifier with no network information by. We publicly release our labels, Tweet IDs and source codes on GitHub.

Cross-platform spread: vaccine-related content, sources, and conspiracy theories in YouTube videos shared in early Twitter COVID-19 conversations

High uptake of vaccinations is essential in fighting infectious diseases, including severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that causes the ongoing coronavirus disease 2019 (COVID-19) pandemic. Social media play a crucial role in propagating misinformation about vaccination, including through conspiracy theories and can negatively impact trust in vaccination. Users typically engage with multiple social media platforms; however, little is known about the role and content of cross-platform use in spreading vaccination-related information. This study examined the content and dynamics of YouTube videos shared in vaccine-related tweets posted to COVID-19 conversations before the COVID-19 vaccine rollout. We screened approximately 144 million tweets posted to COVID-19 conversations and identified 930,539 unique tweets in English that discussed vaccinations posted between 1 February and 23 June 2020. We then identified links to 2,097 unique YouTube videos that were tweeted. Analysis of the video transcripts using Latent Dirichlet Allocation topic modeling and independent coders indicate the dominance of conspiracy theories. Following the World Health Organization's declaration of the COVID-19 outbreak as a public health emergency of international concern, anti-vaccination frames rapidly transitioned from claiming that vaccines cause autism to pandemic conspiracy theories, often featuring Bill Gates. Content analysis of the 20 most tweeted videos revealed that the majority (n = 15) opposed vaccination and included conspiracy theories. Their spread on Twitter was consistent with spamming and coordinated efforts. These findings show the role of cross-platform sharing of YouTube videos over Twitter as a strategy to propagate primarily anti-vaccination messages. Future policies and interventions should consider how to counteract misinformation spread via such cross-platform activities.

Cross-platform information spread during the January 6th capitol riots

Social media has become an integral component of the modern information system. An average person typically has multiple accounts across different platforms. At the same time, the rise of social media facilitates the spread of online mis/disinformation narratives within and across these platforms. In this study, we characterize the coordinated information dissemination of information laden with mis- and disinformation narratives within and across two platforms, Parler and Twitter, during the online discourse surrounding the January 6th 2021 Capitol Riots event. Through the use of username similarity, we discover joint theme endorsements between both platforms. Using anomalously high volume of shared-link matches of external websites and YouTube videos, we discover separate information consumption habits between both platforms, with very few common sources of information between users of the different platforms. However, through analyzing the similarity of the texts with Locality Sensitive Hashing of constructed text vectors, we identify similar narratives between the platforms despite separate consumption of external websites, highlighting the similarities and differences of information spread within and between the two social media environments.

Revealing Public Opinion towards the COVID-19 Vaccine with Weibo Data in China: BertFDA-Based Model

The COVID-19 pandemic has created unprecedented burdens on people's health and subjective well-being. While countries around the world have established models to track and predict the affective states of COVID-19, identifying the topics of public discussion and sentiment evolution of the vaccine, particularly the differences in topics of concern between vaccine-support and vaccine-hesitant groups, remains scarce. Using social media data from the two years following the outbreak of COVID-19 (23 January 2020 to 23 January 2022), coupled with state-of-the-art natural language processing (NLP) techniques, we developed a public opinion analysis framework (BertFDA). First, using dynamic topic clustering on Weibo through the latent Dirichlet allocation (LDA) model, a total of 118 topics were generated in 24 months using 2,211,806 microblog posts. Second, by building an improved Bert pre-training model for sentiment classification, we provide evidence that public negative sentiment continued to decline in the early stages of COVID-19 vaccination. Third, by modeling and analyzing the microblog posts from the vaccine-support group and the vaccine-hesitant group, we discover that the vaccine-support group was more concerned about vaccine effectiveness and the reporting of news, reflecting greater group cohesion, whereas the vaccine-hesitant group was particularly concerned about the spread of coronavirus variants and vaccine side effects. Finally, we deployed different machine learning models to predict public opinion. Moreover, functional data analysis (FDA) is developed to build the functional sentiment curve, which can effectively capture the dynamic changes with the explicit function. This study can aid governments in developing effective interventions and education campaigns to boost vaccination rates.

COVID-19 Misinformation and Social Network Crowdfunding: Cross-sectional Study of Alternative Treatments and Antivaccine Mandates

BACKGROUND

Crowdfunding is increasingly used to offset the financial burdens of illness and health care. In the era of the COVID-19 pandemic and associated infodemic, the role of crowdfunding to support controversial COVID-19 stances is unknown.

OBJECTIVE

We sought to examine COVID-19-related crowdfunding focusing on the funding of alternative treatments not endorsed by major medical entities, including campaigns with an explicit antivaccine, antimask, or antihealth care stances.

METHODS

We performed a cross-sectional analysis of GoFundMe campaigns for individuals requesting donations for COVID-19 relief. Campaigns were identified by key word and manual review to categorize campaigns into "Traditional treatments," "Alternative treatments," "Business-related," "Mandate," "First Response," and "General." For each campaign, we extracted basic narrative, engagement, and financial variables. Among those that were manually reviewed, the additional variables of "mandate type," "mandate stance," and presence of COVID-19 misinformation within the campaign narrative were also included. COVID-19 misinformation was defined as "false or misleading statements," where cited evidence could be provided to refute the claim. Descriptive statistics were used to characterize the study cohort.

RESULTS

A total of 30,368 campaigns met the criteria for final analysis. After manual review, we identified 53 campaigns (0.17%) seeking funding for alternative medical treatment for COVID-19, including popularized treatments such as ivermectin (n=14, 26%), hydroxychloroquine (n=6, 11%), and vitamin D (n=4, 7.5%). Moreover, 23 (43%) of the 53 campaigns seeking support for alternative treatments contained COVID-19 misinformation. There were 80 campaigns that opposed mandating masks or vaccination, 48 (60%) of which contained COVID-19 misinformation. Alternative treatment campaigns had a lower median amount raised (US $1135) compared to traditional (US $2828) treatments (P<.001) and a lower median percentile of target achieved (11.9% vs 31.1%; P=.003). Campaigns for alternative treatments raised substantially lower amounts (US $115,000 vs US $52,715,000, respectively) and lower proportions of fundraising goals (2.1% vs 12.5%) for alternative versus conventional campaigns. The median goal for campaigns was significantly higher (US $25,000 vs US $10,000) for campaigns opposing mask or vaccine mandates relative to those in support of upholding mandates (P=.04). Campaigns seeking funding to lift mandates on health care workers reached US $622 (0.15%) out of a US $410,000 goal.

CONCLUSIONS

A small minority of web-based crowdfunding campaigns for COVID-19 were directed at unproven COVID-19 treatments and support for campaigns aimed against masking or vaccine mandates. Approximately half (71/133, 53%) of these campaigns contained verifiably false or misleading information and had limited fundraising success.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1001/jamainternmed.2019.3330.

Exploring Online Twitter Conversations surrounding National Healthcare Decisions Day and Advance Care Planning from a Socio-Cultural Perspective: A Computational Mixed-Methods Analysis (Preprint)

Background

Within the cultures and societies of the United States, topics related to death and dying continue to be taboo, and as a result, opportunities for presence and engagement during the end of life, which could lead to a good death, are avoided. Several efforts have been made to help people engage in advance care planning (ACP) conversations, including completing advance care directives so that they may express their goals of care if they become too sick to communicate their wishes. A major effort in the United States toward encouraging such challenging discussions is the annual celebration of the National Healthcare Decisions Day.

Objective

This study aimed to explore ACP from a sociocultural perspective by using Twitter as a communication tool.

Methods

All publicly available tweets published between August 1, 2020, and July 30, 2021 (N=9713) were collected and analyzed using the computational mixed methods Analysis of Topic Model Network approach.

Results

The results revealed that conversations driven primarily by laypersons (7107/7410, 95.91% of tweets originated from unverified accounts) surrounded the following three major themes: importance and promotion, surrounding language, and systemic issues.

Conclusions

On the basis of the results, we argue that there is a need for awareness of the barriers that people may face when engaging in ACP conversations, including systemic barriers, literacy levels, misinformation, policies (including Medicare reimbursements), and trust among health care professionals, in the United States. This is incredibly important for clinicians and scholars worldwide to be aware of as we strive to re-envision ACP, so that people are more comfortable engaging in ACP conversations. In terms of the content of tweets, we argue that there is a chasm between the biomedical and biopsychosocial elements of ACP, including patient narratives. If used properly, Twitter conversations and National Health Care Decision Day hashtags could be harnessed to serve as a connecting point among organizations, physicians, patients, and family members to lay the groundwork for the trajectory toward a good death.