1
|
Carollo A, Stella M, Lim M, Bizzego A, Esposito G. Emotional content and semantic structure of dialogues are associated with Interpersonal Neural Synchrony in the Prefrontal Cortex. Neuroimage 2025; 309:121087. [PMID: 39993613 DOI: 10.1016/j.neuroimage.2025.121087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2024] [Revised: 11/29/2024] [Accepted: 02/11/2025] [Indexed: 02/26/2025] Open
Abstract
A fundamental characteristic of social exchanges is the synchronization of individuals' behaviors, physiological responses, and neural activity. However, the association between how individuals communicate in terms of emotional content and expressed associative knowledge and interpersonal synchrony has been scarcely investigated so far. This study addresses this research gap by bridging recent advances in cognitive neuroscience data, affective computing, and cognitive data science frameworks. Using functional near-infrared spectroscopy (fNIRS) hyperscanning, prefrontal neural data were collected during social interactions involving 84 participants (i.e., 42 dyads) aged 18-35 years. Wavelet transform coherence was used to assess interpersonal neural synchrony between participants. We used manual transcription of dialogues and automated methods to codify transcriptions as emotional levels and syntactic/semantic networks. Our quantitative findings reveal higher than random expectations levels of interpersonal neural synchrony in the superior frontal gyrus (q = .038) and the bilateral middle frontal gyri (q< .001, q< .001). Linear mixed models based on dialogues' emotional content only significantly predicted interpersonal neural synchrony across the prefrontal cortex (Rmarginal2=3.62%). Conversely, models relying on syntactic/semantic features were more effective at the local level, for predicting brain synchrony in the right middle frontal gyrus (Rmarginal2=9.97%). Generally, models based on the emotional content of dialogues were not effective when limited to data from one region of interest at a time, whereas models based on syntactic/semantic features show the opposite trend, losing predictive power when incorporating data from all regions of interest. Moreover, we found an interplay between emotions and associative knowledge in predicting brain synchrony, providing quantitative support to the major role played by these linguistic components in social interactions and in prefrontal processes. Our study identifies a mind-brain duality in emotions and associative knowledge reflecting neural synchrony levels, opening new ways for investigating human interactions.
Collapse
Affiliation(s)
- Alessandro Carollo
- Department of Psychology and Cognitive Science, University of Trento, 38068 Rovereto, Italy.
| | - Massimo Stella
- CogNosco Lab, Department of Psychology and Cognitive Science, University of Trento, 38068 Rovereto, Italy
| | - Mengyu Lim
- Psychology Program, Nanyang Technological University, Singapore 639818, Singapore
| | - Andrea Bizzego
- Department of Psychology and Cognitive Science, University of Trento, 38068 Rovereto, Italy
| | - Gianluca Esposito
- Department of Psychology and Cognitive Science, University of Trento, 38068 Rovereto, Italy.
| |
Collapse
|
2
|
Semeraro A, Vilella S, Improta R, De Duro ES, Mohammad SM, Ruffo G, Stella M. EmoAtlas: An emotional network analyzer of texts that merges psychological lexicons, artificial intelligence, and network science. Behav Res Methods 2025; 57:77. [PMID: 39871025 DOI: 10.3758/s13428-024-02553-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/29/2024] [Indexed: 01/29/2025]
Abstract
We introduce EmoAtlas, a computational library/framework extracting emotions and syntactic/semantic word associations from texts. EmoAtlas combines interpretable artificial intelligence (AI) for syntactic parsing in 18 languages and psychologically validated lexicons for detecting the eight emotions in Plutchik's theory. We show that EmoAtlas can match or surpass transformer-based natural language processing techniques, BERT or large language models like ChatGPT 3.5 or LLaMAntino, in detecting emotions from Italian and English online posts and news articles (e.g., achieving 85.6 % accuracy in detecting anger in posts vs the 68.8 % value of ChatGPT and 89.9% value for BERT). EmoAtlas presents important advantages in terms of speed and absence of fine-tuning, e.g., it runs 12x faster than BERT on the same data. Testing EmoAtlas' and easily trainable transformers' relevance in a psychometric task like reproducing human creativity ratings for 1071 short texts, we find that EmoAtlas and BERT obtain equivalent predictive power (fourfold cross-validation, ρ ≈ 0.495 , p < 10 - 4 ). Combining BERT's semantic features with EmoAtlas' emotional/syntactic networks of words gets substantially better at estimating creativity rates of stories ( ρ = 0.628 , p < 10 - 4 ). This indicates an interplay between the creativity of narratives and their semantic, emotional, and syntactic structure. Via interpretable emotional profiles and syntactic networks, EmoAtlas can also quantify how emotions are channeled through specific words in texts, e.g., how did customers frame their ideas and emotions towards "beds" in hotel reviews? We release EmoAtlas as a standalone "text as data" computational tool and discuss its impact in extracting interpretable and reproducible insights from texts.
Collapse
Affiliation(s)
- Alfonso Semeraro
- Department of Computer Science, University of Turin, Turin, Italy
| | - Salvatore Vilella
- Dipartimento di Scienze e Innovazione Tecnologica, University of Eastern Piedmont, Alessandria, Italy
| | - Riccardo Improta
- CogNosco Lab, Department of Psychology and Cognitive Sciences, University of Trento, Trento, Italy
| | | | | | - Giancarlo Ruffo
- Dipartimento di Scienze e Innovazione Tecnologica, University of Eastern Piedmont, Alessandria, Italy
| | - Massimo Stella
- CogNosco Lab, Department of Psychology and Cognitive Sciences, University of Trento, Trento, Italy.
| |
Collapse
|
3
|
Pfeifer VA, Chilton TD, Grilli MD, Mehl MR. How ready is speech-to-text for psychological language research? Evaluating the validity of AI-generated English transcripts for analyzing free-spoken responses in younger and older adults. Behav Res Methods 2024; 56:7621-7631. [PMID: 38773028 PMCID: PMC11365748 DOI: 10.3758/s13428-024-02440-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/30/2024] [Indexed: 05/23/2024]
Abstract
For the longest time, the gold standard in preparing spoken language corpora for text analysis in psychology was using human transcription. However, such standard comes at extensive cost, and creates barriers to quantitative spoken language analysis that recent advances in speech-to-text technology could address. The current study quantifies the accuracy of AI-generated transcripts compared to human-corrected transcripts across younger (n = 100) and older (n = 92) adults and two spoken language tasks. Further, it evaluates the validity of Linguistic Inquiry and Word Count (LIWC)-features extracted from these two kinds of transcripts, as well as transcripts specifically prepared for LIWC analyses via tagging. We find that overall, AI-generated transcripts are highly accurate with a word error rate of 2.50% to 3.36%, albeit being slightly less accurate for younger compared to older adults. LIWC features extracted from either transcripts are highly correlated, while the tagging procedure significantly alters filler word categories. Based on these results, automatic speech-to-text appears to be ready for psychological language research when using spoken language tasks in relatively quiet environments, unless filler words are of interest to researchers.
Collapse
Affiliation(s)
- Valeria A Pfeifer
- Department of Psychology, University of Arizona, 1503 E University Bldv, Tucson, AZ, 85721, USA.
| | - Trish D Chilton
- Department of Psychology, University of Arizona, 1503 E University Bldv, Tucson, AZ, 85721, USA
| | - Matthew D Grilli
- Department of Psychology, University of Arizona, 1503 E University Bldv, Tucson, AZ, 85721, USA
| | - Matthias R Mehl
- Department of Psychology, University of Arizona, 1503 E University Bldv, Tucson, AZ, 85721, USA
| |
Collapse
|
4
|
Stella M, Citraro S, Rossetti G, Marinazzo D, Kenett YN, Vitevitch MS. Cognitive modelling of concepts in the mental lexicon with multilayer networks: Insights, advancements, and future challenges. Psychon Bull Rev 2024; 31:1981-2004. [PMID: 38438713 PMCID: PMC11543778 DOI: 10.3758/s13423-024-02473-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/28/2024] [Indexed: 03/06/2024]
Abstract
The mental lexicon is a complex cognitive system representing information about the words/concepts that one knows. Over decades psychological experiments have shown that conceptual associations across multiple, interactive cognitive levels can greatly influence word acquisition, storage, and processing. How can semantic, phonological, syntactic, and other types of conceptual associations be mapped within a coherent mathematical framework to study how the mental lexicon works? Here we review cognitive multilayer networks as a promising quantitative and interpretative framework for investigating the mental lexicon. Cognitive multilayer networks can map multiple types of information at once, thus capturing how different layers of associations might co-exist within the mental lexicon and influence cognitive processing. This review starts with a gentle introduction to the structure and formalism of multilayer networks. We then discuss quantitative mechanisms of psychological phenomena that could not be observed in single-layer networks and were only unveiled by combining multiple layers of the lexicon: (i) multiplex viability highlights language kernels and facilitative effects of knowledge processing in healthy and clinical populations; (ii) multilayer community detection enables contextual meaning reconstruction depending on psycholinguistic features; (iii) layer analysis can mediate latent interactions of mediation, suppression, and facilitation for lexical access. By outlining novel quantitative perspectives where multilayer networks can shed light on cognitive knowledge representations, including in next-generation brain/mind models, we discuss key limitations and promising directions for cutting-edge future research.
Collapse
Affiliation(s)
- Massimo Stella
- CogNosco Lab, Department of Psychology and Cognitive Science, University of Trento, Trento, Italy.
| | - Salvatore Citraro
- Institute of Information Science and Technologies, National Research Council, Pisa, Italy
| | - Giulio Rossetti
- Institute of Information Science and Technologies, National Research Council, Pisa, Italy
| | - Daniele Marinazzo
- Faculty of Psychology and Educational Sciences, Department of Data Analysis, University of Ghent, Ghent, Belgium
| | - Yoed N Kenett
- Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology, Haifa, Israel
| | - Michael S Vitevitch
- Department of Speech Language Hearing, University of Kansas, Lawrence, KS, USA
| |
Collapse
|
5
|
Watson J, van der Linden S, Watson M, Stillwell D. Negative online news articles are shared more to social media. Sci Rep 2024; 14:21592. [PMID: 39285221 PMCID: PMC11405697 DOI: 10.1038/s41598-024-71263-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2024] [Accepted: 08/26/2024] [Indexed: 09/20/2024] Open
Abstract
Prior research demonstrates that news-related social media posts using negative language are re-posted more, rewarding users who produce negative content. We investigate whether negative material from external news sites is also introduced to social media through more user posts, offering comparable incentives for journalists to adopt a negative tone. Data from four US and UK news sites (95,282 articles) and two social media platforms (579,182,075 posts on Facebook and Twitter, now X) show social media users are 1.91 times more likely to share links to negative news articles. The impact of negativity varies by news site and social media platform and, for political articles, is moderated by topic focus, with users showing a greater inclination to share negative articles referring to opposing political groups. Additionally, negativity amplifies news dissemination on social media to a greater extent when accounting for the re-sharing of user posts containing article links. These findings suggest a higher prevalence of negatively toned articles on Facebook and Twitter compared to online news sites. Further, should journalists respond to the incentives created by the heightened sharing of negative articles to social media platforms, this could even increase negative news exposure for those who do not use social media.
Collapse
Affiliation(s)
- Joe Watson
- Psychometrics Centre, Judge Business School, University of Cambridge, Cambridge, UK.
| | | | - Michael Watson
- Department of Informatics, King's College London, London, UK
| | - David Stillwell
- Psychometrics Centre, Judge Business School, University of Cambridge, Cambridge, UK
- Organisational Behaviour Group, Judge Business School, University of Cambridge, Cambridge, UK
| |
Collapse
|
6
|
Amoretti MC, Lalumera E. Unveiling the interplay between evidence, values and cognitive biases. The case of the failure of the AstraZeneca COVID-19 vaccine. J Eval Clin Pract 2023; 29:1294-1301. [PMID: 37515407 DOI: 10.1111/jep.13903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Revised: 07/06/2023] [Accepted: 07/11/2023] [Indexed: 07/30/2023]
Abstract
This paper depicts a Covid science case, that of the AstraZeneca Vaxzevria vaccine, with specific focus on what happened in Italy. Given that we believe acknowledging the role of non-evidential factors in medicine is an important insight into the recent philosophy of science, we illustrate how in the case of Vaxzevria, the interplay between facts, values (both epistemic and non-epistemic) and cognitive biases may have possibly led to different institutional decisions based on the same evidence. The structure of the paper is as follows. First, we provide a glossary of the relevant terms involved, that is to say, epistemic values, non-epistemic values and cognitive biases. Second, we sketch a timeline of Vaxzevria's approvals and suspensions by relevant institutional healthcare authorities with special focus on Italy and the Italian Medicines Agency. Then we show the interplay between the evidence base, epistemic as well as non-epistemic values and cognitive biases using a narrative review of political decisions along with newspaper and social media content pertaining to Vaxzevria. We briefly compare Italy with other European countries to show that different political decisions were made on the basis of the same evidence.
Collapse
|
7
|
Grouin C, Grabar N. Year 2022 in Medical Natural Language Processing: Availability of Language Models as a Step in the Democratization of NLP in the Biomedical Area. Yearb Med Inform 2023; 32:244-252. [PMID: 38147866 PMCID: PMC10751107 DOI: 10.1055/s-0043-1768752] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2023] Open
Abstract
OBJECTIVES To analyse the content of publications within the medical Natural Language Processing (NLP) domain in 2022. METHODS Automatic and manual preselection of publications to be reviewed, and selection of the best NLP papers of the year. Analysis of the important issues. RESULTS Three best papers have been selected. We also propose an analysis of the content of the NLP publications in 2022, stressing on some of the topics. CONCLUSION The main trend in 2022 is certainly related to the availability of large language models, especially those based on Transformers, and to their use by non-NLP researchers. This leads to the democratization of the NLP methods. We also observe the renewal of interest to languages other than English, the continuation of research on information extraction and prediction, the massive use of data from social media, and the consideration of needs and interests of patients.
Collapse
Affiliation(s)
- Cyril Grouin
- Université Paris Saclay, CNRS, Laboratoire Interdisciplinaire des Sciences du Numérique, 91400 Orsay, France
| | - Natalia Grabar
- UMR8163 STL, CNRS, Université de Lille, Domaine du Pont-de-bois, 59653 Villeneuve-d'Ascq cedex, France
| | | |
Collapse
|
8
|
Improving Public Health Policy by Comparing the Public Response during the Start of COVID-19 and Monkeypox on Twitter in Germany: A Mixed Methods Study. Vaccines (Basel) 2022; 10:vaccines10121985. [PMID: 36560395 PMCID: PMC9787903 DOI: 10.3390/vaccines10121985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Revised: 11/06/2022] [Accepted: 11/17/2022] [Indexed: 11/24/2022] Open
Abstract
Little is known about monkeypox public concerns since its widespread emergence in many countries. Tweets in Germany were examined in the first three months of COVID-19 and monkeypox to examine concerns and issues raised by the public. Understanding views and positions of the public could help to shape future public health campaigns. Few qualitative studies reviewed large datasets, and the results provide the first instance of the public thinking comparing COVID-19 and monkeypox. We retrieved 15,936 tweets from Germany using query words related to both epidemics in the first three months of each one. A sequential explanatory mixed methods research joined a machine learning approach with thematic analysis using a novel rapid tweet analysis protocol. In COVID-19 tweets, there was the selfing construct or feeling part of the emerging narrative of the spread and response. In contrast, during monkeypox, the public considered othering after the fatigue of the COVID-19 response, or an impersonal feeling toward the disease. During monkeypox, coherence and reconceptualization of new and competing information produced a customer rather than a consumer/producer model. Public healthcare policy should reconsider a one-size-fits-all model during information campaigns and produce a strategic approach embedded within a customer model to educate the public about preventative measures and updates. A multidisciplinary approach could prevent and minimize mis/disinformation.
Collapse
|