Now you see it, now you don't: on emotion, context, and the algorithmic prediction of human imageability judgments

Extrapolation of Human Estimates of the Concreteness/ Abstractness of Words by Neural Networks of Various Architectures

In a great deal of theoretical and applied cognitive and neurophysiological research, it is essential to have more vocabularies with concreteness/abstractness ratings. Since creating such dictionaries by interviewing informants is labor-intensive, considerable effort has been made to machine-extrapolate human rankings. The purpose of the article is to study the possibility of the fast construction of high-quality machine dictionaries. In this paper, state-of-the-art deep learning neural networks are involved for the first time to solve this problem. For the English language, the BERT model has achieved a record result for the quality of a machine-generated dictionary. It is known that the use of multilingual models makes it possible to transfer ratings from one language to another. However, this approach is understudied so far and the results achieved so far are rather weak. Microsoft’s Multilingual-MiniLM-L12-H384 model also obtained the best result to date in transferring ratings from one language to another. Thus, the article demonstrates the advantages of transformer-type neural networks in this task. Their use will allow the generation of good-quality dictionaries in low-resource languages. Additionally, we study the dependence of the result on the amount of initial data and the number of languages in the multilingual case. The possibilities of transferring into a certain language from one language and from several languages together are compared. The influence of the volume of training and test data has been studied. It has been found that an increase in the amount of training data in a multilingual case does not improve the result.

subs2vec: Word embeddings from subtitles in 55 languages

This paper introduces a novel collection of word embeddings, numerical representations of lexical semantics, in 55 languages, trained on a large corpus of pseudo-conversational speech transcriptions from television shows and movies. The embeddings were trained on the OpenSubtitles corpus using the fastText implementation of the skipgram algorithm. Performance comparable with (and in some cases exceeding) embeddings trained on non-conversational (Wikipedia) text is reported on standard benchmark evaluation datasets. A novel evaluation method of particular relevance to psycholinguists is also introduced: prediction of experimental lexical norms in multiple languages. The models, as well as code for reproducing the models and all analyses reported in this paper (implemented as a user-friendly Python package), are freely available at: https://github.com/jvparidon/subs2vec.

Representation of associative and affective semantic similarity of abstract words in the lateral temporal perisylvian language regions

The examination of semantic cognition has traditionally identified word concreteness as well as valence as two of the principal dimensions in the representation of conceptual knowledge. More recently, corpus-based vector space models as well as graph-theoretical analysis of large-scale task-related behavioural responses have revolutionized our insight into how the meaning of words is structured. In this fMRI study, we apply representational similarity analysis to investigate the conceptual representation of abstract words. Brain activity patterns were related to a cued-association based graph as well as to a vector-based co-occurrence model of word meaning. Twenty-six subjects (19 females and 7 males) performed an overt repetition task during fMRI. First, we performed a searchlight classification procedure to identify regions where activity is discriminable between abstract and concrete words. These regions were left inferior frontal gyrus, the upper and lower bank of the superior temporal sulcus bilaterally, posterior middle temporal gyrus and left fusiform gyrus. Representational Similarity Analysis demonstrated that for abstract words, the similarity of activity patterns in the cortex surrounding the superior temporal sulcus bilaterally and in the left anterior superior temporal gyrus reflects the similarity in word meaning. These effects were strongest for semantic similarity derived from the cued association-based graph and for affective similarity derived from either of the two models. The latter effect was mainly driven by positive valence words. This research highlights the close neurobiological link between the information structure of abstract and affective word content and the similarity in activity pattern in the lateral and anterior temporal language system.

Eye movements and mental imagery during reading of literary texts with different narrative styles

Based on Kuzmičová's [1] phenomenological typology of narrative styles, we studied the specific contributions of mental imagery to literary reading experience and to reading behavior by combining questionnaires with eye-tracking methodology. Specifically, we focused on the two main categories in Kuzmičová's [1] typology, i.e., texts dominated by an "enactive" style, and texts dominated by a "descriptive" style. "Enactive" style texts render characters interacting with their environment, and "descriptive" style texts render environments dissociated from human action. The quantitative analyses of word category distributions of two dominantly enactive and two dominantly descriptive texts indicated significant differences especially in the number of verbs, with more verbs in enactment compared to descriptive texts. In a second study, participants read two texts (one theoretically cueing descriptive imagery, the other cueing enactment imagery) while their eye movements were recorded. After reading, participants completed questionnaires assessing aspects of the reading experience generally, as well as their text-elicited mental imagery specifically. Results show that readers experienced more difficulties conjuring up mental images during reading descriptive style texts and that longer fixation duration on words were associated with enactive style text. We propose that enactive style involves more imagery processes which can be reflected in eye movement behavior.

Estimating the prevalence and diversity of words in written language

Recently, a new crowd-sourced language metric has been introduced, entitled word prevalence, which estimates the proportion of the population that knows a given word. This measure has been shown to account for unique variance in large sets of lexical performance. This article aims to build on the work of Brysbaert et al. and Keuleers et al. by introducing new corpus-based metrics that estimate how likely a word is to be an active member of the natural language environment, and hence known by a larger subset of the general population. This metric is derived from an analysis of a newly collected corpus of over 25,000 fiction and non-fiction books and will be shown that it is capable of accounting for significantly more variance than past corpus-based measures.

Duality of Function: Activation for Meaningless Nonwords and Semantic Codes in the Same Brain Areas

Studies of the neural substrates of semantic (word meaning) processing have typically focused on semantic manipulations, with less consideration for potential differences in difficulty across conditions. While the idea that particular brain regions can support multiple functions is widely accepted, studies of specific cognitive domains rarely test for co-location with other functions. Here we start with standard univariate analyses comparing words to meaningless nonwords, replicating our recent finding that this contrast can activate task-positive regions for words, and default-mode regions in the putative semantic network for nonwords, pointing to difficulty effects. Critically, this was followed up with a multivariate analysis to test whether the same areas activated for meaningless nonwords contained semantic information sufficient to distinguish high- from low-imageability words. Indeed, this classification was performed reliably better than chance at 75% accuracy. This is compatible with two non-exclusive interpretations. Numerous areas in the default-mode network are task-negative in the sense of activating for less demanding conditions, and the same areas contain information supporting semantic cognition. Therefore, while areas of the default mode network have been hypothesized to support semantic cognition, we offer evidence that these areas can respond to both domain-general difficulty effects, and to specific aspects of semantics.

The Impact of Context on Affective Norms: A Case of Study With Suspense

The emotional response to a stimulus is typically measured in three variables called valence, arousal and dominance. Based on such dimensions, Bradley and Lang (1999) published the Affective Norms for English Words (ANEW), a corpus of affective ratings for 1,034 non-contextualized words. Expanded and adapted to many languages, ANEW provides a corpus to evaluate and to predict human responses to different stimuli, and it has been used in a number of studies involving analysis of emotions. However, ANEW seems not to appropriately predict affective responses to concepts when these are contextualized in certain situational backgrounds, in which words can have different connotations from those in non-contextualized scenarios. These contextualized affective norms have not been sufficiently contrasted yet because the literature does not provide a corpus of the ANEW list in specific contexts. On this basis, this paper reports on the creation of a new corpus of affective norms for the original 1,034 ANEW words in a particular context (a fictional scene of suspense). An extensive quantitative data analysis comparing both corpora was carried out, confirming that the affective ratings are highly influenced by the context. The corpus can be downloaded as Supplementary Material.

Sentiment Analysis for Words and Fiction Characters From the Perspective of Computational (Neuro-)Poetics

Two computational studies provide different sentiment analyses for text segments (e.g., "fearful" passages) and figures (e.g., "Voldemort") from the Harry Potter books (Rowling, 1997, 1998, 1999, 2000, 2003, 2005, 2007) based on a novel simple tool called SentiArt. The tool uses vector space models together with theory-guided, empirically validated label lists to compute the valence of each word in a text by locating its position in a 2d emotion potential space spanned by the words of the vector space model. After testing the tool's accuracy with empirical data from a neurocognitive poetics study, it was applied to compute emotional figure and personality profiles (inspired by the so-called "big five" personality theory) for main characters from the book series. The results of comparative analyses using different machine-learning classifiers (e.g., AdaBoost, Neural Net) show that SentiArt performs very well in predicting the emotion potential of text passages. It also produces plausible predictions regarding the emotional and personality profile of fiction characters which are correctly identified on the basis of eight character features, and it achieves a good cross-validation accuracy in classifying 100 figures into "good" vs. "bad" ones. The results are discussed with regard to potential applications of SentiArt in digital literary, applied reading and neurocognitive poetics studies such as the quantification of the hybrid hero potential of figures.

Linking somatic and symbolic representation in semantic memory: the dynamic multilevel reactivation framework

Biological plausibility is an essential constraint for any viable model of semantic memory. Yet, we have only the most rudimentary understanding of how the human brain conducts abstract symbolic transformations that underlie word and object meaning. Neuroscience has evolved a sophisticated arsenal of techniques for elucidating the architecture of conceptual representation. Nevertheless, theoretical convergence remains elusive. Here we describe several contrastive approaches to the organization of semantic knowledge, and in turn we offer our own perspective on two recurring questions in semantic memory research: (1) to what extent are conceptual representations mediated by sensorimotor knowledge (i.e., to what degree is semantic memory embodied)? (2) How might an embodied semantic system represent abstract concepts such as modularity, symbol, or proposition? To address these questions, we review the merits of sensorimotor (i.e., embodied) and amodal (i.e., disembodied) semantic theories and address the neurobiological constraints underlying each. We conclude that the shortcomings of both perspectives in their extreme forms necessitate a hybrid middle ground. We accordingly propose the Dynamic Multilevel Reactivation Framework-an integrative model predicated upon flexible interplay between sensorimotor and amodal symbolic representations mediated by multiple cortical hubs. We discuss applications of the dynamic multilevel reactivation framework to abstract and concrete concept representation and describe how a multidimensional conceptual topography based on emotion, sensation, and magnitude can successfully frame a semantic space containing meanings for both abstract and concrete words. The consideration of 'abstract conceptual features' does not diminish the role of logical and/or executive processing in activating, manipulating and using information stored in conceptual representations. Rather, it proposes that the materials upon which these processes operate necessarily combine pure sensorimotor information and higher-order cognitive dimensions involved in symbolic representation.

Extrapolating human judgments from skip-gram vector representations of word meaning

There is a growing body of research in psychology that attempts to extrapolate human lexical judgments from computational models of semantics. This research can be used to help develop comprehensive norm sets for experimental research, it has applications to large-scale statistical modelling of lexical access and has broad value within natural language processing and sentiment analysis. However, the value of extrapolated human judgments has recently been questioned within psychological research. Of primary concern is the fact that extrapolated judgments may not share the same pattern of statistical relationship with lexical and semantic variables as do actual human judgments; often the error component in extrapolated judgments is not psychologically inert, making such judgments problematic to use for psychological research. We present a new methodology for extrapolating human judgments that partially addresses prior concerns of validity. We use this methodology to extrapolate human judgments of valence, arousal, dominance, and concreteness for 78,286 words. We also provide resources for users to extrapolate these human judgments for three million English words and short phrases. Applications for large sets of extrapolated human judgments are demonstrated and discussed.

Imaging Imageability: Behavioral Effects and Neural Correlates of Its Interaction with Affect and Context

The construct of imageability refers to the extent to which a word evokes a tangible sensation. Previous research (Westbury et al., 2013) suggests that the behavioral effects attributed to a word's imageability can be largely or wholly explained by two objective constructs, contextual density and estimated affect. Here, we extend these previous findings in two ways. First, we show that closely matched stimuli on the three measures of contextual density, estimated affect, and human-judged imageability show a three-way interaction in explaining variance in LD RTs, but that imagebility accounts for no additional variance after contextual density and estimated affect are entered first. Secondly, we demonstrate that the loci and functional connectivity (via graphical models) of the brain regions implicated in processing the three variables during that task are largely over-lapping and similar. These two lines of evidence support the conclusion that the effect usually attributed to human-judged imageability is largely or entirely due to the effects of other correlated measures that are directly computable.

10 years of BAWLing into affective and aesthetic processes in reading: what are the echoes?

Reading is not only "cold" information processing, but involves affective and aesthetic processes that go far beyond what current models of word recognition, sentence processing, or text comprehension can explain. To investigate such "hot" reading processes, standardized instruments that quantify both psycholinguistic and emotional variables at the sublexical, lexical, inter-, and supralexical levels (e.g., phonological iconicity, word valence, arousal-span, or passage suspense) are necessary. One such instrument, the Berlin Affective Word List (BAWL) has been used in over 50 published studies demonstrating effects of lexical emotional variables on all relevant processing levels (experiential, behavioral, neuronal). In this paper, we first present new data from several BAWL studies. Together, these studies examine various views on affective effects in reading arising from dimensional (e.g., valence) and discrete emotion features (e.g., happiness), or embodied cognition features like smelling. Second, we extend our investigation of the complex issue of affective word processing to words characterized by a mixture of affects. These words entail positive and negative valence, and/or features making them beautiful or ugly. Finally, we discuss tentative neurocognitive models of affective word processing in the light of the present results, raising new issues for future studies.

How useful are corpus-based methods for extrapolating psycholinguistic variables?

Subjective ratings for age of acquisition, concreteness, affective valence, and many other variables are an important element of psycholinguistic research. However, even for well-studied languages, ratings usually cover just a small part of the vocabulary. A possible solution involves using corpora to build a semantic similarity space and to apply machine learning techniques to extrapolate existing ratings to previously unrated words. We conduct a systematic comparison of two extrapolation techniques: k-nearest neighbours, and random forest, in combination with semantic spaces built using latent semantic analysis, topic model, a hyperspace analogue to language (HAL)-like model, and a skip-gram model. A variant of the k-nearest neighbours method used with skip-gram word vectors gives the most accurate predictions but the random forest method has an advantage of being able to easily incorporate additional predictors. We evaluate the usefulness of the methods by exploring how much of the human performance in a lexical decision task can be explained by extrapolated ratings for age of acquisition and how precisely we can assign words to discrete categories based on extrapolated ratings. We find that at least some of the extrapolation methods may introduce artefacts to the data and produce results that could lead to different conclusions that would be reached based on the human ratings. From a practical point of view, the usefulness of ratings extrapolated with the described methods may be limited.

Avoid violence, rioting, and outrage; approach celebration, delight, and strength: Using large text corpora to compute valence, arousal, and the basic emotions

Ever since Aristotle discussed the issue in Book II of his Rhetoric, humans have attempted to identify a set of "basic emotion labels". In this paper we propose an algorithmic method for evaluating sets of basic emotion labels that relies upon computed co-occurrence distances between words in a 12.7-billion-word corpus of unselected text from USENET discussion groups. Our method uses the relationship between human arousal and valence ratings collected for a large list of words, and the co-occurrence similarity between each word and emotion labels. We assess how well the words in each of 12 emotion label sets-proposed by various researchers over the past 118 years-predict the arousal and valence ratings on a test and validation dataset, each consisting of over 5970 items. We also assess how well these emotion labels predict lexical decision residuals (LDRTs), after co-varying out the effects attributable to basic lexical predictors. We then demonstrate a generalization of our method to determine the most predictive "basic" emotion labels from among all of the putative models of basic emotion that we considered. As well as contributing empirical data towards the development of a more rigorous definition of basic emotions, our method makes it possible to derive principled computational estimates of emotionality-specifically, of arousal and valence-for all words in the language.

Measuring inconsistencies can lead you forward: Imageability and the x-ception theory

According to the traditional view, both imageability and concreteness ratings reflect the way word meanings rely on information mediated by the senses. As a consequence, the two measures should and do correlate. The link between these two indexes was already hypothesized and demonstrated by Paivio et al. (1968) in a seminal article, where they introduced the idea of imageability ratings for the first time. However, in this first study, they also noted a contrasting pattern in the ratings for imageability and concreteness with some words that refer to affective attitudes or emotional states receiving high imageability but low concreteness ratings. Recent studies confirm this inconsistency (e.g., Altarriba and Bauer, 2004) leading to the claim that emotion words form a particular class of terms different from both concrete and abstract words. Here we use the MRC psycholinguistic database to show that the there are other classes of terms for which imageability and concreteness are uncorrelated. We show that the common feature of these word classes is that they directly or indirectly refer to proprioceptive, interoceptive, or affective states, i.e., to internal, body-related, sensory experiences. Thus, imageability and concreteness can no longer be considered interchangeable constructs; rather, imageability is a different, and perhaps more interesting, measure: it not only reflects the ease with which memories of external events come to mind, as previously hypothesized, but also reflects the ease with which memories of internal events come to mind.