Forma mentis networks quantify crucial differences in STEM perception between students and experts

The "LLM World of Words" English free association norms generated by large language models

Free associations have been extensively used in psychology and linguistics for studying how conceptual knowledge is organized. Recently, the potential of applying a similar approach for investigating the knowledge encoded in LLMs has emerged, specifically as a method for investigating LLM biases. However, the absence of large-scale LLM-generated free association norms that are comparable with human-generated norms is an obstacle to this research direction. To address this, we create a new dataset of LLM-generated free association norms modeled after the "Small World of Words"(SWOW) human-generated norms with nearly 12,000 cue words. We prompt three LLMs (Mistral, Llama3, and Haiku) with the same cues as those in SWOW to generate three novel comparable datasets, the "LLM World of Words" (LWOW). From the datasets, we construct network models of semantic memory that represent the conceptual knowledge possessed by humans and LLMs. We validate the datasets by simulating semantic priming within the network models, and we briefly discuss how the datasets can be used for investigating implicit biases in humans and LLMs.

Emotional content and semantic structure of dialogues are associated with Interpersonal Neural Synchrony in the Prefrontal Cortex

A fundamental characteristic of social exchanges is the synchronization of individuals' behaviors, physiological responses, and neural activity. However, the association between how individuals communicate in terms of emotional content and expressed associative knowledge and interpersonal synchrony has been scarcely investigated so far. This study addresses this research gap by bridging recent advances in cognitive neuroscience data, affective computing, and cognitive data science frameworks. Using functional near-infrared spectroscopy (fNIRS) hyperscanning, prefrontal neural data were collected during social interactions involving 84 participants (i.e., 42 dyads) aged 18-35 years. Wavelet transform coherence was used to assess interpersonal neural synchrony between participants. We used manual transcription of dialogues and automated methods to codify transcriptions as emotional levels and syntactic/semantic networks. Our quantitative findings reveal higher than random expectations levels of interpersonal neural synchrony in the superior frontal gyrus (q = .038) and the bilateral middle frontal gyri (q< .001, q< .001). Linear mixed models based on dialogues' emotional content only significantly predicted interpersonal neural synchrony across the prefrontal cortex (Rmarginal2=3.62%). Conversely, models relying on syntactic/semantic features were more effective at the local level, for predicting brain synchrony in the right middle frontal gyrus (Rmarginal2=9.97%). Generally, models based on the emotional content of dialogues were not effective when limited to data from one region of interest at a time, whereas models based on syntactic/semantic features show the opposite trend, losing predictive power when incorporating data from all regions of interest. Moreover, we found an interplay between emotions and associative knowledge in predicting brain synchrony, providing quantitative support to the major role played by these linguistic components in social interactions and in prefrontal processes. Our study identifies a mind-brain duality in emotions and associative knowledge reflecting neural synchrony levels, opening new ways for investigating human interactions.

Mapping Mental Representations With Free Associations: A Tutorial Using the R Package associatoR

People's understanding of topics and concepts such as risk, sustainability, and intelligence can be important for psychological researchers and policymakers alike. One underexplored way of accessing this information is to use free associations to map people's mental representations. In this tutorial, we describe how free association responses can be collected, processed, mapped, and compared across groups using the R package associatoR. We discuss study design choices and different approaches to uncovering the structure of mental representations using natural language processing, including the use of embeddings from large language models. We posit that free association analysis presents a powerful approach to revealing how people and machines represent key social and technological issues.

The "Small World of Words" free association norms for Rioplatense Spanish

Large-scale word association datasets are both important tools used in psycholinguistics and used as models that capture meaning when considered as semantic networks. Here, we present word association norms for Rioplatense Spanish, a variant spoken in Argentina and Uruguay. The norms were derived through a large-scale crowd-sourced continued word association task in which participants give three associations to a list of cue words. Covering over 13,000 words and +3.6 M responses, it is currently the most extensive dataset available for Spanish. We compare the obtained dataset with previous studies in Dutch and English to investigate the role of grammatical gender and studies that used Iberian Spanish to test generalizability to other Spanish variants. Finally, we evaluated the validity of our data in word processing (lexical decision reaction times) and semantic (similarity judgment) tasks. Our results demonstrate that network measures such as in-degree provide a good prediction of lexical decision response times. Analyzing semantic similarity judgments showed that results replicate and extend previous findings demonstrating that semantic similarity derived using spreading activation or spectral methods outperform word embeddings trained on text corpora.

Cognitive networks detect structural patterns and emotional complexity in suicide notes

Communicating one's mindset means transmitting complex relationships between concepts and emotions. Using network science and word co-occurrences, we reconstruct conceptual associations as communicated in 139 genuine suicide notes, i.e., notes left by individuals who took their lives. We find that, despite their negative context, suicide notes are surprisingly positively valenced. Through emotional profiling, their ending statements are found to be markedly more emotional than their main body: The ending sentences in suicide notes elicit deeper fear/sadness but also stronger joy/trust and anticipation than the main body. Furthermore, by using data from the Emotional Recall Task, we model emotional transitions within these notes as co-occurrence networks and compare their structure against emotional recalls from mentally healthy individuals. Supported by psychological literature, we introduce emotional complexity as an affective analog of structural balance theory, measuring how elementary cycles (closed triads) of emotion co-occurrences mix positive, negative and neutral states in narratives and recollections. At the group level, authors of suicide narratives display a higher complexity than healthy individuals, i.e., lower levels of coherently valenced emotional states in triads. An entropy measure identified a similar tendency for suicide notes to shift more frequently between contrasting emotional states. Both the groups of authors of suicide notes and healthy individuals exhibit less complexity than random expectation. Our results demonstrate that suicide notes possess highly structured and contrastive narratives of emotions, more complex than expected by null models and healthy populations.

Cognitive Networks Extract Insights on COVID-19 Vaccines from English and Italian Popular Tweets: Anticipation, Logistics, Conspiracy and Loss of Trust

Monitoring social discourse about COVID-19 vaccines is key to understanding how large populations perceive vaccination campaigns. This work reconstructs how popular and trending posts framed semantically and emotionally COVID-19 vaccines on Twitter. We achieve this by merging natural language processing, cognitive network science and AI-based image analysis. We focus on 4765 unique popular tweets in English or Italian about COVID-19 vaccines between December 2020 and March 2021. One popular English tweet contained in our data set was liked around 495,000 times, highlighting how popular tweets could cognitively affect large parts of the population. We investigate both text and multimedia content in tweets and build a cognitive network of syntactic/semantic associations in messages, including emotional cues and pictures. This network representation indicates how online users linked ideas in social discourse and framed vaccines along specific semantic/emotional content. The English semantic frame of “vaccine” was highly polarised between trust/anticipation (towards the vaccine as a scientific asset saving lives) and anger/sadness (mentioning critical issues with dose administering). Semantic associations with “vaccine,” “hoax” and conspiratorial jargon indicated the persistence of conspiracy theories and vaccines in extremely popular English posts. Interestingly, these were absent in Italian messages. Popular tweets with images of people wearing face masks used language that lacked the trust and joy found in tweets showing people with no masks. This difference indicates a negative effect attributed to face-covering in social discourse. Behavioural analysis revealed a tendency for users to share content eliciting joy, sadness and disgust and to like sad messages less. Both patterns indicate an interplay between emotions and content diffusion beyond sentiment. After its suspension in mid-March 2021, “AstraZeneca” was associated with trustful language driven by experts. After the deaths of a small number of vaccinated people in mid-March, popular Italian tweets framed “vaccine” by crucially replacing earlier levels of trust with deep sadness. Our results stress how cognitive networks and innovative multimedia processing open new ways for reconstructing online perceptions about vaccines and trust.

Academic performance of K-12 students in an online-learning environment for mathematics increased during the shutdown of schools in wake of the COVID-19 pandemic

The shutdown of schools in response to the rapid spread of COVID-19 poses risks to the education of young children, including a widening education gap. In the present article, we investigate how school closures in 2020 influenced the performance of German students in a curriculum-based online learning software for mathematics. We analyzed data from more than 2,500 K-12 students who computed over 124,000 mathematical problem sets before and during the shutdown, and found that students' performance increased during the shutdown of schools in 2020 relative to the year before. Our analyses also revealed that low-achieving students showed greater improvements in performance than high-achieving students, suggesting a narrowing gap in performance between low- and high-achieving students. We conclude that online learning environments may be effective in preventing educational losses associated with current and future shutdowns of schools.

Text-mining forma mentis networks reconstruct public perception of the STEM gender gap in social media

Mindset reconstruction maps how individuals structure and perceive knowledge, a map unfolded here by investigating language and its cognitive reflection in the human mind, i.e., the mental lexicon. Textual forma mentis networks (TFMN) are glass boxes introduced for extracting and understanding mindsets' structure (in Latin forma mentis) from textual data. Combining network science, psycholinguistics and Big Data, TFMNs successfully identified relevant concepts in benchmark texts, without supervision. Once validated, TFMNs were applied to the case study of distorted mindsets about the gender gap in science. Focusing on social media, this work analysed 10,000 tweets mostly representing individuals' opinions at the beginning of posts. "Gender" and "gap" elicited a mostly positive, trustful and joyous perception, with semantic associates that: celebrated successful female scientists, related gender gap to wage differences, and hoped for a future resolution. The perception of "woman" highlighted jargon of sexual harassment and stereotype threat (a form of implicit cognitive bias) about women in science "sacrificing personal skills for success". The semantic frame of "man" highlighted awareness of the myth of male superiority in science. No anger was detected around "person", suggesting that tweets got less tense around genderless terms. No stereotypical perception of "scientist" was identified online, differently from real-world surveys. This analysis thus identified that Twitter discourse mostly starting conversations promoted a majorly stereotype-free, positive/trustful perception of gender disparity, aimed at closing the gap. Hence, future monitoring against discriminating language should focus on other parts of conversations like users' replies. TFMNs enable new ways for monitoring collective online mindsets, offering data-informed ground for policy making.

#lockdown: Network-Enhanced Emotional Profiling in the Time of COVID-19

The COVID-19 pandemic forced countries all over the world to take unprecedented measures, like nationwide lockdowns. To adequately understand the emotional and social repercussions, a large-scale reconstruction of how people perceived these unexpected events is necessary but currently missing. We address this gap through social media by introducing MERCURIAL (Multi-layer Co-occurrence Networks for Emotional Profiling), a framework which exploits linguistic networks of words and hashtags to reconstruct social discourse describing real-world events. We use MERCURIAL to analyse 101,767 tweets from Italy, the first country to react to the COVID-19 threat with a nationwide lockdown. The data were collected between the 11th and 17th March, immediately after the announcement of the Italian lockdown and the WHO declaring COVID-19 a pandemic. Our analysis provides unique insights into the psychological burden of this crisis, focussing on—(i) the Italian official campaign for self-quarantine (#iorestoacasa), (ii) national lockdown (#italylockdown), and (iii) social denounce (#sciacalli). Our exploration unveils the emergence of complex emotional profiles, where anger and fear (towards political debates and socio-economic repercussions) coexisted with trust, solidarity, and hope (related to the institutions and local communities). We discuss our findings in relation to mental well-being issues and coping mechanisms, like instigation to violence, grieving, and solidarity. We argue that our framework represents an innovative thermometer of emotional status, a powerful tool for policy makers to quickly gauge feelings in massive audiences and devise appropriate responses based on cognitive data.

Editorial: Networks Applied in Science Education Research

Science education research is, in many ways, involved with exploring relational aspects of diverse elements that affect students’ learning outcomes; at one end, the elements may be concepts to be learned, and at the other end, the relations between students in different types of learning settings and environments and, ultimately, how such elements may interact [...]

Applications of Network Science to Education Research: Quantifying Knowledge and the Development of Expertise through Network Analysis

A fundamental goal of education is to inspire and instill deep, meaningful, and long-lasting conceptual change within the knowledge landscapes of students. This commentary posits that the tools of network science could be useful in helping educators achieve this goal in two ways. First, methods from cognitive psychology and network science could be helpful in quantifying and analyzing the structure of students’ knowledge of a given discipline as a knowledge network of interconnected concepts. Second, network science methods could be relevant for investigating the developmental trajectories of knowledge structures by quantifying structural change in knowledge networks, and potentially inform instructional design in order to optimize the acquisition of meaningful knowledge as the student progresses from being a novice to an expert in the subject. This commentary provides a brief introduction to common network science measures and suggests how they might be relevant for shedding light on the cognitive processes that underlie learning and retrieval, and discusses ways in which generative network growth models could inform pedagogical strategies to enable meaningful long-term conceptual change and knowledge development among students.

Between Social and Semantic Networks: A Case Study on Classroom Complexity

Classrooms are complex in their real sets. To understand such sets and their emergent patterns, network approach provides useful theoretical and methodological tools. In this work, we used network approach to explore two domains of complexity in a classroom: the interpersonal domain, via social networks; and the representational domain, through collective semantic networks. This work is grounded in both Social Network Analyses and Social Representation Theory for gathering information from interpersonal and representational domains. We investigated a physics high school classroom by proceeding sociometric tests and by using words freely evoked by students to explore relations between students’ dyad’s weights, in social networks, and emerging consensus in semantic networks. Our findings showed closer relations between social ties’ weight and consensus formed on intra-school representational objects, while consensus on extra-school representational objects is less dependent on the classroom interpersonal ties’ strength.

Transferring Knowledge in a Knowledge-in-Use Task—Investigating the Role of Knowledge Organization

Knowledge-in-Use, i.e., the ability to apply what one has learned, is a major goal of education and involves the ability to transfer one’s knowledge. While some general principles of knowledge transfer have been revealed, the literature is full of inconclusive results and it remains hard to predict successful transfer. However, research into expertise suggests that how one organizes one’s knowledge is critical for successful transfer. Drawing on data from a larger study on the learning of energy, we employed network analysis to investigate how the organization of students’ knowledge about energy influenced their ability to transfer and what role achievement goal orientation may have played in this. We found that students that had more coherently organized knowledge networks were more successful in transfer. Furthermore, we also found a connection between mastery goal orientation and the organization of students’ knowledge networks. Our results extend the literature by providing evidence for a direct connection between the organization of students’ knowledge networks, their success in transfer, and their goal orientation and hint at the complexities in the relationship between mastery approach goal orientation and successful transfer beyond what is reported in the literature.

Forma Mentis Networks Reconstruct How Italian High Schoolers and International STEM Experts Perceive Teachers, Students, Scientists, and School

This study investigates how students and researchers shape their knowledge and perception of educational topics. The mindset or forma mentis of 159 Italian high school students and of 59 international researchers in science, technology, engineering and maths (STEM) are reconstructed through forma mentis networks, i.e., cognitive networks of concepts connected by free associations and enriched with sentiment labels. The layout of conceptual associations between positively/negatively/neutrally perceived concepts is informative on how people build their own mental constructs or beliefs about specific topics. Researchers displayed mixed positive/neutral mental representations of “teacher”, “student” and, “scientist”. Students’ conceptual associations of “scientist” were highly positive and largely non-stereotypical, although links about the “mad scientist” stereotype persisted. Students perceived “teacher” as a complex figure, associated with positive aspects like mentoring/knowledge transmission but also to negative sides revolving around testing and grading. “School” elicited stronger differences between the two groups. In the students’ mindset, “school” was surrounded by a negative emotional aura or set of associations, indicating an anxious perception of the school setting, mixing scholastic concepts, anxiety-eliciting words, STEM disciplines like maths and physics, and exam-related notions. Researchers’ positive stance of “school” included concepts of fun, friendship, and personal growth instead. Along the perspective of Education Research, the above results are discussed as quantitative evidence for test- and STEM anxiety co-occurring in the way Italian students perceive education places and their actors. Detecting these patterns in student populations through forma mentis networks offers new, simple to gather yet detailed knowledge for future data-informed intervention policies and action research.