Estimating time-series changes in social sentiment @Twitter in U.S. metropolises during the COVID-19 pandemic

Understanding Citizens' Response to Social Activities on Twitter in US Metropolises During the COVID-19 Recovery Phase Using a Fine-Tuned Large Language Model: Application of AI

BACKGROUND

The COVID-19 pandemic continues to hold an important place in the collective memory as of 2024. As of March 2024, >676 million cases, 6 million deaths, and 13 billion vaccine doses have been reported. It is crucial to evaluate sociopsychological impacts as well as public health indicators such as these to understand the effects of the COVID-19 pandemic.

OBJECTIVE

This study aimed to explore the sentiments of residents of major US cities toward restrictions on social activities in 2022 during the transitional phase of the COVID-19 pandemic, from the peak of the pandemic to its gradual decline. By illuminating people's susceptibility to COVID-19, we provide insights into the general sentiment trends during the recovery phase of the pandemic.

METHODS

To analyze these trends, we collected posts (N=119,437) on the social media platform Twitter (now X) created by people living in New York City, Los Angeles, and Chicago from December 2021 to December 2022, which were impacted by the COVID-19 pandemic in similar ways. A total of 47,111 unique users authored these posts. In addition, for privacy considerations, any identifiable information, such as author IDs and usernames, was excluded, retaining only the text for analysis. Then, we developed a sentiment estimation model by fine-tuning a large language model on the collected data and used it to analyze how citizens' sentiments evolved throughout the pandemic.

RESULTS

In the evaluation of models, GPT-3.5 Turbo with fine-tuning outperformed GPT-3.5 Turbo without fine-tuning and Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach (RoBERTa)-large with fine-tuning, demonstrating significant accuracy (0.80), recall (0.79), precision (0.79), and F1-score (0.79). The findings using GPT-3.5 Turbo with fine-tuning reveal a significant relationship between sentiment levels and actual cases in all 3 cities. Specifically, the correlation coefficient for New York City is 0.89 (95% CI 0.81-0.93), for Los Angeles is 0.39 (95% CI 0.14-0.60), and for Chicago is 0.65 (95% CI 0.47-0.78). Furthermore, feature words analysis showed that COVID-19-related keywords were replaced with non-COVID-19-related keywords in New York City and Los Angeles from January 2022 onward and Chicago from March 2022 onward.

CONCLUSIONS

The results show a gradual decline in sentiment and interest in restrictions across all 3 cities as the pandemic approached its conclusion. These results are also ensured by a sentiment estimation model fine-tuned on actual Twitter posts. This study represents the first attempt from a macro perspective to depict sentiment using a classification model created with actual data from the period when COVID-19 was prevalent. This approach can be applied to the spread of other infectious diseases by adjusting search keywords for observational data.

Sentiment analysis of social media responses to the approval of lecanemab for the treatment of Alzheimer's disease in Japan

Lecanemab, an anti-amyloid therapy for early Alzheimer's disease, received approval by the FDA and Japan in 2023. Public response on social media was scrutinized, aiming to obtain insights into communication and treatment development. For 478 posts from X and Facebook, their sentiments on efficacy, safety, societal significance, and overall lecanemab impression were assessed by GPT-4 and the authors. Results indicated impressions were 43.7% negative, 26.6% neutral, and 29.7% positive. Social significance concerns dominated negative views. Specific attitude patterns were observed in the overall impression to lecanemab's approval. These insights highlight the need for targeted communication and research on anti-amyloid therapies.

Use of Large Language Models to Assess the Likelihood of Epidemics From the Content of Tweets: Infodemiology Study

BACKGROUND

Previous work suggests that Google searches could be useful in identifying conjunctivitis epidemics. Content-based assessment of social media content may provide additional value in serving as early indicators of conjunctivitis and other systemic infectious diseases.

OBJECTIVE

We investigated whether large language models, specifically GPT-3.5 and GPT-4 (OpenAI), can provide probabilistic assessments of whether social media posts about conjunctivitis could indicate a regional outbreak.

METHODS

A total of 12,194 conjunctivitis-related tweets were obtained using a targeted Boolean search in multiple languages from India, Guam (United States), Martinique (France), the Philippines, American Samoa (United States), Fiji, Costa Rica, Haiti, and the Bahamas, covering the time frame from January 1, 2012, to March 13, 2023. By providing these tweets via prompts to GPT-3.5 and GPT-4, we obtained probabilistic assessments that were validated by 2 human raters. We then calculated Pearson correlations of these time series with tweet volume and the occurrence of known outbreaks in these 9 locations, with time series bootstrap used to compute CIs.

RESULTS

Probabilistic assessments derived from GPT-3.5 showed correlations of 0.60 (95% CI 0.47-0.70) and 0.53 (95% CI 0.40-0.65) with the 2 human raters, with higher results for GPT-4. The weekly averages of GPT-3.5 probabilities showed substantial correlations with weekly tweet volume for 44% (4/9) of the countries, with correlations ranging from 0.10 (95% CI 0.0-0.29) to 0.53 (95% CI 0.39-0.89), with larger correlations for GPT-4. More modest correlations were found for correlation with known epidemics, with substantial correlation only in American Samoa (0.40, 95% CI 0.16-0.81).

CONCLUSIONS

These findings suggest that GPT prompting can efficiently assess the content of social media posts and indicate possible disease outbreaks to a degree of accuracy comparable to that of humans. Furthermore, we found that automated content analysis of tweets is related to tweet volume for conjunctivitis-related posts in some locations and to the occurrence of actual epidemics. Future work may improve the sensitivity and specificity of these methods for disease outbreak detection.

Nursing Home Residents' COVID-19 Infections in the United States: A Systematic Review of Personal and Contextual Factors

Background: This mixed methods systemic review synthesizes the evidence about nursing home risks for COVID-19 infections. Methods: Four electronic databases (PubMed, Web of Science, Scopus, and Sage Journals Online) were searched between January 2020 and October 2022. Inclusion criteria were studies reported on nursing home COVID-19 infection risks by geography, demography, type of nursing home, staffing and resident's health, and COVID-19 vaccination status. The Mixed Methods Appraisal Tool (MMAT) was used to assess the levels of evidence for quality, and a narrative synthesis for reporting the findings by theme. Results: Of 579 initial articles, 48 were included in the review. Findings suggest that highly populated counties and urban locations had a higher likelihood of COVID-19 infections. Larger nursing homes with a low percentage of fully vaccinated residents also had increased risks for COVID-19 infections than smaller nursing homes. Residents with advanced age, of racial minority, and those with chronic illnesses were at higher risk for COVID-19 infections. Discussion and implications: Findings suggest that along with known risk factors for COVID-19 infections, geographic and resident demographics are also important preventive care considerations. Access to COVID-19 vaccinations for vulnerable residents should be a priority.

Analysing sentiment change detection of Covid-19 tweets

The Covid-19 pandemic made a significant impact on society, including the widespread implementation of lockdowns to prevent the spread of the virus. This measure led to a decrease in face-to-face social interactions and, as an equivalent, an increase in the use of social media platforms, such as Twitter. As part of Industry 4.0, sentiment analysis can be exploited to study public attitudes toward future pandemics and sociopolitical situations in general. This work presents an analysis framework by applying a combination of natural language processing techniques and machine learning algorithms to classify the sentiment of each tweet as positive, or negative. Through extensive experimentation, we expose the ideal model for this task and, subsequently, utilize sentiment predictions to perform time series analysis over the course of the pandemic. In addition, a change point detection algorithm was applied in order to identify the turning points in public attitudes toward the pandemic, which were validated by cross-referencing the news report at that particular period of time. Finally, we study the relationship between sentiment trends on social media and, news coverage of the pandemic, providing insights into the public's perception of the pandemic and its influence on the news.