Neurocognitive poetics: methods and models for investigating the neuronal and cognitive-affective bases of literature reception

Idiomatic expressions evoke stronger emotional responses in the brain than literal sentences

Recent neuroscientific research shows that metaphors engage readers at the emotional level more strongly than literal expressions. What still remains unclear is what makes metaphors more engaging, and whether this generalises to all figurative expressions, no matter how conventionalised they are. This fMRI study aimed to investigate whether idiomatic expressions - the least creative part of figurative language - indeed trigger a higher affective resonance than literal expressions, and to explore possible interactions between activation in emotion-relevant neural structures and regions associated with figurative language processing. Participants silently read for comprehension a set of emotionally positive, negative and neutral idioms embedded in short sentences, and similarly valenced literal sentences. As in studies on metaphors, we found enhanced activation of the left inferior frontal gyrus and left amygdala in response to idioms, indexing stronger recruitment of executive control functions and enhanced emotional engagement, respectively. This suggests that the comprehension of even highly conventionalised and familiar figurative expressions, namely idioms, recruits regions involved in emotional processing. Furthermore, increased activation of the IFG interacted positively with activation in the amygdala, suggesting that the stronger cognitive engagement driven by idioms may in turn be coupled with stronger involvement at the emotional level.

Inferior parietal lobule and early visual areas support elicitation of individualized meanings during narrative listening

INTRODUCTION

When listening to a narrative, the verbal expressions translate into meanings and flow of mental imagery. However, the same narrative can be heard quite differently based on differences in listeners' previous experiences and knowledge. We capitalized on such differences to disclose brain regions that support transformation of narrative into individualized propositional meanings and associated mental imagery by analyzing brain activity associated with behaviorally assessed individual meanings elicited by a narrative.

METHODS

Sixteen right-handed female subjects were instructed to list words that best described what had come to their minds while listening to an eight-minute narrative during functional magnetic resonance imaging (fMRI). The fMRI data were analyzed by calculating voxel-wise intersubject correlation (ISC) values. We used latent semantic analysis (LSA) enhanced with Wordnet knowledge to measure semantic similarity of the produced words between subjects. Finally, we predicted the ISC with the semantic similarity using representational similarity analysis.

RESULTS

We found that semantic similarity in these word listings between subjects, estimated using LSA combined with WordNet knowledge, predicting similarities in brain hemodynamic activity. Subject pairs whose individual semantics were similar also exhibited similar brain activity in the bilateral supramarginal and angular gyrus of the inferior parietal lobe, and in the occipital pole.

CONCLUSIONS

Our results demonstrate, using a novel method to measure interindividual differences in semantics, brain mechanisms giving rise to semantics and associated imagery during narrative listening. During listening to a captivating narrative, the inferior parietal lobe and early visual cortical areas seem, thus, to support elicitation of individual meanings and flow of mental imagery.

Reading Shakespeare Sonnets: Combining Quantitative Narrative Analysis and Predictive Modeling -an Eye Tracking Study

As a part of a larger interdisciplinary project on Shakespeare sonnets' reception (1, 2), the present study analyzed the eye movement behavior of participants reading three of the 154 sonnets as a function of seven lexical features extracted via Quantitative Narrative Analysis (QNA). Using a machine learning-based predictive modeling approach five 'surface' features (word length, orthographic neighborhood density, word frequency, orthographic dissimilarity and sonority score) were detected as important predictors of total reading time and fixation probability in poetry reading. The fact that one phonological feature, i.e., sonority score, also played a role is in line with current theorizing on poetry reading. Our approach opens new ways for future eye movement research on reading poetic texts and other complex literary materials(3).

Comparing Comprehension of a Long Text Read in Print Book and on Kindle: Where in the Text and When in the Story?

Digital reading devices such as Kindle differ from paper books with respect to the kinesthetic and tactile feedback provided to the reader, but the role of these features in reading is rarely studied empirically. This experiment compares reading of a long text on Kindle DX and in print. Fifty participants (24 years old) read a 28 page (∼1 h reading time) long mystery story on Kindle or in a print pocket book and completed several tests measuring various levels of reading comprehension: engagement, recall, capacities to locate events in the text and reconstructing the plot of the story. Results showed that on most tests subjects performed identically whatever the reading medium. However, on measures related to chronology and temporality, those who had read in the print pocket book, performed better than those who had read on a Kindle. It is concluded that, basically comprehension was similar with both media, but, because kinesthetic feedback is less informative with a Kindle, readers were not as efficient to locate events in the space of the text and hence in the temporality of the story. We suggest that, to get a correct spatial representation of the text and consequently a coherent temporal organization of the story, readers would be reliant on the sensorimotor cues which are afforded by the manipulation of the book.

Cognitive engagement in emotional text reading: concurrent recordings of eye movements and head motion

The present study examined the effects of emotions on eye movements, head motion, and iPad motion during reading. Thirty-one participants read neutral, emotionally negative texts and emotionally positive texts on a digital tablet and both participants' eye movements and body movements were recorded using respectively eye-tracking glasses and a motion capture system. The results showed that emotionally positive texts were read faster than neutral texts, and that readers' movements decreased when reading emotional texts regardless of valence polarity. Recent studies suggested that postural movements may reflect cognitive engagement and especially the engagement in the task to be done. Our findings seem to validate this hypothesis of a bodily engagement in reading emotional contents. The present results suggest that the novel methodology of eye and postural movement recordings is informative in studying the readers' embodied engagement during reading emotional materials.

Effects of Valence and Emotional Intensity on the Comprehension and Memorization of Texts

In the present study, we independently manipulated valence (positive, negative, or neutral) and emotional intensity (low, medium, or high), asking what impact they have on text comprehension (via surface, paraphrase, and inference questions) and memorization (via Remember/Know test) in adults. Results show that emotional contents, including valence and intensity affects comprehension. Emotional valence had a significant effect on text comprehension, with higher scores for positive and neutral texts than for negative ones. Participants scored higher on the surface questions for positive texts and on the inference questions for negative texts, with equivalent scores for paraphrase questions. Regarding emotional intensity, medium intensity generally fostered better comprehension of both positive and negative texts. High emotional intensity is beneficial for positively valenced texts, but hinders the understanding of negatively valenced ones. Regarding memorization, participants recalled more emotional words than neutral ones, and more words for positive texts than for either negative or neutral ones. In conclusion, our results show that emotions play an important role and improve the processing of information.

The Experience of Beauty of Chinese Poetry and Its Neural Substrates

Chinese poetry has a long history and high esthetic value. People who engage esthetically with Chinese poetry would feel the sense of beauty naturally. However, there is little information regarding what happens in the brain when an individual appreciates Chinese poetry, and how the brain processes the subject’s appreciation of beauty. Herein, we used functional magnetic resonance imaging (fMRI) to investigate the neural substrates of experiencing beauty by appreciating Chinese poetry. The participants in our study were 28 college students and the stimuli consisted of 25 Chinese poetry and 25 prose selections. Based on an event-related paradigm, the findings of this study suggested that different areas scattered in both the left and right cerebral hemispheres are activated when an individual appreciates Chinese poetry. Compared to reading prose, appreciating Chinese poetry heightens the activation of the left inferior orbitofrontal cortex (OFC), the bilateral insula, the left fusiform, the left supplementary motor area (SMA), and the left precentral gyrus. In these areas, the left inferior OFC and the bilateral insula are considered closely related to experiencing beauty of Chinese poetry, which have been demonstrated that it is an important neural basis of esthetic beauty when using other types of materials. The findings of this study shed new light on the complex but ordinary processes of experiencing beauty when appreciating Chinese poetry and show that some key processes underlying the feeling of esthetic beauty are shared across different esthetic domains.

Why 'piss' is ruder than 'pee'? The role of sound in affective meaning making

Most language users agree that some words sound harsh (e.g. grotesque) whereas others sound soft and pleasing (e.g. lagoon). While this prominent feature of human language has always been creatively deployed in art and poetry, it is still largely unknown whether the sound of a word in itself makes any contribution to the word's meaning as perceived and interpreted by the listener. In a large-scale lexicon analysis, we focused on the affective substrates of words' meaning (i.e. affective meaning) and words' sound (i.e. affective sound); both being measured on a two-dimensional space of valence (ranging from pleasant to unpleasant) and arousal (ranging from calm to excited). We tested the hypothesis that the sound of a word possesses affective iconic characteristics that can implicitly influence listeners when evaluating the affective meaning of that word. The results show that a significant portion of the variance in affective meaning ratings of printed words depends on a number of spectral and temporal acoustic features extracted from these words after converting them to their spoken form (study1). In order to test the affective nature of this effect, we independently assessed the affective sound of these words using two different methods: through direct rating (study2a), and through acoustic models that we implemented based on pseudoword materials (study2b). In line with our hypothesis, the estimated contribution of words' sound to ratings of words' affective meaning was indeed associated with the affective sound of these words; with a stronger effect for arousal than for valence. Further analyses revealed crucial phonetic features potentially causing the effect of sound on meaning: For instance, words with short vowels, voiceless consonants, and hissing sibilants (as in 'piss') feel more arousing and negative. Our findings suggest that the process of meaning making is not solely determined by arbitrary mappings between formal aspects of words and concepts they refer to. Rather, even in silent reading, words' acoustic profiles provide affective perceptual cues that language users may implicitly use to construct words' overall meaning.

The Fictive Brain: Neurocognitive Correlates of Engagement in Literature

Fiction is vital to our being. Many people enjoy engaging with fiction every day. Here we focus on literary reading as 1 instance of fiction consumption from a cognitive neuroscience perspective. The brain processes which play a role in the mental construction of fiction worlds and the related engagement with fictional characters, remain largely unknown. The authors discuss the neurocognitive poetics model ( Jacobs, 2015a ) of literary reading specifying the likely neuronal correlates of several key processes in literary reading, namely inference and situation model building, immersion, mental simulation and imagery, figurative language and style, and the issue of distinguishing fact from fiction. An overview of recent work on these key processes is followed by a discussion of methodological challenges in studying the brain bases of fiction processing.

Affective Congruence between Sound and Meaning of Words Facilitates Semantic Decision

A similarity between the form and meaning of a word (i.e., iconicity) may help language users to more readily access its meaning through direct form-meaning mapping. Previous work has supported this view by providing empirical evidence for this facilitatory effect in sign language, as well as for onomatopoetic words (e.g., cuckoo) and ideophones (e.g., zigzag). Thus, it remains largely unknown whether the beneficial role of iconicity in making semantic decisions can be considered a general feature in spoken language applying also to “ordinary” words in the lexicon. By capitalizing on the affective domain, and in particular arousal, we organized words in two distinctive groups of iconic vs. non-iconic based on the congruence vs. incongruence of their lexical (meaning) and sublexical (sound) arousal. In a two-alternative forced choice task, we asked participants to evaluate the arousal of printed words that were lexically either high or low arousing. In line with our hypothesis, iconic words were evaluated more quickly and more accurately than their non-iconic counterparts. These results indicate a processing advantage for iconic words, suggesting that language users are sensitive to sound-meaning mappings even when words are presented visually and read silently.

The Sound of Words Evokes Affective Brain Responses

The long history of poetry and the arts, as well as recent empirical results suggest that the way a word sounds (e.g., soft vs. harsh) can convey affective information related to emotional responses (e.g., pleasantness vs. harshness). However, the neural correlates of the affective potential of the sound of words remain unknown. In an fMRI study involving passive listening, we focused on the affective dimension of arousal and presented words organized in two discrete groups of sublexical (i.e., sound) arousal (high vs. low), while controlling for lexical (i.e., semantic) arousal. Words sounding high arousing, compared to their low arousing counterparts, resulted in an enhanced BOLD signal in bilateral posterior insula, the right auditory and premotor cortex, and the right supramarginal gyrus. This finding provides first evidence on the neural correlates of affectivity in the sound of words. Given the similarity of this neural network to that of nonverbal emotional expressions and affective prosody, our results support a unifying view that suggests a core neural network underlying any type of affective sound processing.

The emotional power of poetry: neural circuitry, psychophysiology and compositional principles

It is a common experience-and well established experimentally-that music can engage us emotionally in a compelling manner. The mechanisms underlying these experiences are receiving increasing scrutiny. However, the extent to which other domains of aesthetic experience can similarly elicit strong emotions is unknown. Using psychophysiology, neuroimaging and behavioral responses, we show that recited poetry can act as a powerful stimulus for eliciting peak emotional responses, including chills and objectively measurable goosebumps that engage the primary reward circuitry. Importantly, while these responses to poetry are largely analogous to those found for music, their neural underpinnings show important differences, specifically with regard to the crucial role of the nucleus accumbens. We also go beyond replicating previous music-related studies by showing that peak aesthetic pleasure can co-occur with physiological markers of negative affect. Finally, the distribution of chills across the trajectory of poems provides insight into compositional principles of poetry.

Readers’ Insensitivity to Tense Revealed: No Differences in Mental Simulation During Reading of Present and Past Tense Stories

While the importance of mental simulation during literary reading has long been recognized, we know little about the factors that determine when, what, and how much readers mentally simulate. Here we investigate the influence of a specific text characteristic, namely verb tense (present vs. past), on mental simulation during literary reading. Verbs usually denote the actions and events that take place in narratives and hence it is hypothesized that verb tense will influence the amount of mental simulation elicited in readers. Although the present tense is traditionally considered to be more “vivid”, this study is one of the first to experimentally assess this claim. We recorded eye-movements while subjects read stories in the past or present tense and collected data regarding self-reported levels of mental simulation, transportation and appreciation. We found no influence of tense on any of the offline measures. The eye-tracking data showed a slightly more complex pattern. Although we did not find a main effect of sensorimotor simulation content on reading times, we were able to link the degree to which subjects slowed down when reading simulation eliciting content to offline measures of attention and transportation, but this effect did not interact with the tense of the story. Unexpectedly, we found a main effect of tense on reading times per word, with past tense stories eliciting longer first fixation durations and gaze durations. However, we were unable to link this effect to any of the offline measures. In sum, this study suggests that tense does not play a substantial role in the process of mental simulation elicited by literary stories.

Men Who Compliment a Woman's Appearance Using Metaphorical Language: Associations with Creativity, Masculinity, Intelligence and Attractiveness

Language may have evolved as a signal of mental fitness. However, it remains unclear what language form and topic men use to covertly signal mate quality. In this study 69 men created compliments to impress unfamiliar women they chose to either date or work with and provided hand scans to compute 2D4D ratio as a proxy for prenatal testosterone exposure and masculinity indicator. Compliments were coded in terms of form (literal vs. metaphorical) and topic (women's appearance vs. non-appearance), with metaphorical ones being subsequently rated by 114 women for psycholinguistic features, indices of intelligence and willingness to have a romantic relationship with the author. Results showed that in a dating context, men produced more metaphorical form compliments targeting appearance compared to the working context and they were associated with men's art creativity and negatively with 2D4D ratio (i.e., positively with masculinity). Women preferred establishing a romantic relationship with a higher proportion of the men producing metaphorical compliments in a dating than a working context. Furthermore, in the dating but not the working context, women perceived men producing such compliments as being more intelligent, and importantly this correlated with the men's actual verbal intelligence. Overall, findings suggest that men may use metaphorical language compliments targeting women's appearance in a dating context to signal covertly their mate quality.

Quantifying the Beauty of Words: A Neurocognitive Poetics Perspective

In this paper I would like to pave the ground for future studies in Computational Stylistics and (Neuro-)Cognitive Poetics by describing procedures for predicting the subjective beauty of words. A set of eight tentative word features is computed via Quantitative Narrative Analysis (QNA) and a novel metric for quantifying word beauty, the aesthetic potential is proposed. Application of machine learning algorithms fed with this QNA data shows that a classifier of the decision tree family excellently learns to split words into beautiful vs. ugly ones. The results shed light on surface and semantic features theoretically relevant for affective-aesthetic processes in literary reading and generate quantitative predictions for neuroaesthetic studies of verbal materials.

Sound iconicity of abstract concepts: Place of articulation is implicitly associated with abstract concepts of size and social dominance

The concept of sound iconicity implies that phonemes are intrinsically associated with non-acoustic phenomena, such as emotional expression, object size or shape, or other perceptual features. In this respect, sound iconicity is related to other forms of cross-modal associations in which stimuli from different sensory modalities are associated with each other due to the implicitly perceived correspondence of their primal features. One prominent example is the association between vowels, categorized according to their place of articulation, and size, with back vowels being associated with bigness and front vowels with smallness. However, to date the relative influence of perceptual and conceptual cognitive processing on this association is not clear. To bridge this gap, three experiments were conducted in which associations between nonsense words and pictures of animals or emotional body postures were tested. In these experiments participants had to infer the relation between visual stimuli and the notion of size from the content of the pictures, while directly perceivable features did not support–or even contradicted–the predicted association. Results show that implicit associations between articulatory-acoustic characteristics of phonemes and pictures are mainly influenced by semantic features, i.e., the content of a picture, whereas the influence of perceivable features, i.e., size or shape, is overridden. This suggests that abstract semantic concepts can function as an interface between different sensory modalities, facilitating cross-modal associations.

When Fiction Is Just as Real as Fact: No Differences in Reading Behavior between Stories Believed to be Based on True or Fictional Events

Experiments have shown that compared to fictional texts, readers read factual texts faster and have better memory for described situations. Reading fictional texts on the other hand seems to improve memory for exact wordings and expressions. Most of these studies used a “newspaper” vs. “literature” comparison. In the present study, we investigated the effect of reader's expectation to whether information is true or fictional with a subtler manipulation by labeling short stories as either based on true or fictional events. In addition, we tested whether narrative perspective or individual preference in perspective taking affects reading true or fictional stories differently. In an online experiment, participants (final N = 1,742) read one story which was introduced as based on true events or as fictional (factor fictionality). The story could be narrated in either 1st or 3rd person perspective (factor perspective). We measured immersion in and appreciation of the story, perspective taking, as well as memory for events. We found no evidence that knowing a story is fictional or based on true events influences reading behavior or experiential aspects of reading. We suggest that it is not whether a story is true or fictional, but rather expectations toward certain reading situations (e.g., reading newspaper or literature) which affect behavior by activating appropriate reading goals. Results further confirm that narrative perspective partially influences perspective taking and experiential aspects of reading.

Neurofunctionally dissecting the reading system in children

The reading system can be broken down into four basic subcomponents in charge of prelexical, orthographic, phonological, and lexico-semantic processes. These processes need to jointly work together to become a fluent and efficient reader. Using functional magnetic resonance imaging (fMRI), we systematically analyzed differences in neural activation patterns of these four basic subcomponents in children (N=41, 9-13 years) using tasks specifically tapping each component (letter identification, orthographic decision, phonological decision, and semantic categorization). Regions of interest (ROI) were selected based on a meta-analysis of child reading and included the left ventral occipito-temporal cortex (vOT), left posterior parietal cortex (PPC), left inferior frontal gyrus (IFG), and bilateral supplementary motor area (SMA). Compared to a visual baseline task, enhanced activation in vOT and IFG was observed for all tasks with very little differences between tasks. Activity in the dorsal PPC system was confined to prelexical and phonological processing. Activity in the SMA was found in orthographic, phonological, and lexico-semantic tasks. Our results are consistent with the idea of an early engagement of the vOT accompanied by executive control functions in the frontal system, including the bilateral SMA.

Evoking and Measuring Identification with Narrative Characters - A Linguistic Cues Framework

Current research on identification with narrative characters poses two problems. First, although identification is seen as a dynamic process of which the intensity varies during reading, it is usually measured by means of post-reading questionnaires containing self-report items. Second, it is not clear which linguistic characteristics evoke identification. The present paper proposes that an interdisciplinary framework allows for more precise manipulations and measurements of identification, which will ultimately advance our understanding of the antecedents and nature of this process. The central hypothesis of our Linguistic Cues Framework is that identification with a narrative character is a multidimensional experience for which different dimensions are evoked by different linguistic cues. The first part of the paper presents a literature review on identification, resulting in a renewed conceptualization of identification which distinguishes six dimensions: a spatiotemporal, a perceptual, a cognitive, a moral, an emotional, and an embodied dimension. The second part argues that each of these dimensions is influenced by specific linguistic cues which represent various aspects of the narrative character’s perspective. The proposed relations between linguistic cues and identification dimensions are specified in six propositions. The third part discusses what psychological and neurocognitive methods enable the measurement of the various identification dimensions in order to test the propositions. By establishing explicit connections between the linguistic characteristics of narratives and readers’ physical, psychological, and neurocognitive responses to narratives, this paper develops a research agenda for future empirical research on identification with narrative characters.

Complementarity As Generative Principle: A Thought Pattern for Aesthetic Appreciations and Cognitive Appraisals in General

In experimental aesthetics the relationship between the arts and cognitive neuroscience has gained particular interest in recent years. But has cognitive neuroscience indeed something to offer when studying the arts? Here we present a theoretical frame within which the concept of complementarity as a generative or creative principle is proposed; neurocognitive processes are characterized by the duality of complementary activities like bottom-up and top-down control, or logistical functions like temporal control and content functions like perceptions in the neural machinery. On that basis a thought pattern is suggested for aesthetic appreciations and cognitive appraisals in general. This thought pattern is deeply rooted in the history of philosophy and art theory since antiquity; and complementarity also characterizes neural operations as basis for cognitive processes. We then discuss some challenges one is confronted with in experimental aesthetics; in our opinion, one serious problem is the lack of a taxonomy of functions in psychology and neuroscience which is generally accepted. This deficit makes it next to impossible to develop acceptable models which are similar to what has to be modeled. Another problem is the severe language bias in this field of research as knowledge gained in many languages over the ages remains inaccessible to most scientists. Thus, an inspection of research results or theoretical concepts is necessarily too narrow. In spite of these limitations we provide a selective summary of some results and viewpoints with a focus on visual art and its appreciation. It is described how questions of art and aesthetic appreciations using behavioral methods and in particular brain-imaging techniques are analyzed and evaluated focusing on such issues like the representation of artwork or affective experiences. Finally, we emphasize complementarity as a generative principle on a practical level when artists and scientists work directly together which can lead to new insights and broader perspectives on both sides.

Reading English-Language Haiku: Processes of Meaning Construction Revealed by Eye Movements

In the present study, poets and cognitive scientists came together to investigate the construction of meaning in the process of reading normative, 3-line English-language haiku (ELH), as found in leading ELH journals. The particular haiku which we presented to our readers consisted of two semantically separable parts, or images, that were set in a 'tense' relationship by the poet. In our sample of poems, the division, or cut, between the two parts was positioned either after line 1 or after line 2; and the images related to each other in terms of either a context-action association (context-action haiku) or a conceptually more abstract association (juxtaposition haiku). From a constructivist perspective, understanding such haiku would require the reader to integrate these parts into a coherent 'meaning Gestalt', mentally (re-)creating the pattern intended by the poet (or one from within the poem's meaning potential). To examine this process, we recorded readers' eye movements, and we obtained measures of memory for the read poems as well as subjective ratings of comprehension difficulty and understanding achieved. The results indicate that processes of meaning construction are reflected in patterns of eye movements during reading (1st-pass) and re-reading (2nd- and 3rd-pass). From those, the position of the cut (after line 1 vs. after line 2) and, to some extent, the type of haiku (context-action vs. juxtaposition) can be 'recovered'. Moreover, post-reading, readers tended to explicitly recognize a particular haiku they had read if they had been able to understand the poem, pointing to a role of actually resolving the haiku's meaning (rather than just attempting to resolve it) for memory consolidation and subsequent retrieval. Taken together, these first findings are promising, suggesting that haiku can be a paradigmatic material for studying meaning construction during poetry reading.

On the Relation between the General Affective Meaning and the Basic Sublexical, Lexical, and Inter-lexical Features of Poetic Texts-A Case Study Using 57 Poems of H. M. Enzensberger

The literary genre of poetry is inherently related to the expression and elicitation of emotion via both content and form. To explore the nature of this affective impact at an extremely basic textual level, we collected ratings on eight different general affective meaning scales—valence, arousal, friendliness, sadness, spitefulness, poeticity, onomatopoeia, and liking—for 57 German poems (“die verteidigung der wölfe”) which the contemporary author H. M. Enzensberger had labeled as either “friendly,” “sad,” or “spiteful.” Following Jakobson's (1960) view on the vivid interplay of hierarchical text levels, we used multiple regression analyses to explore the specific influences of affective features from three different text levels (sublexical, lexical, and inter-lexical) on the perceived general affective meaning of the poems using three types of predictors: (1) Lexical predictor variables capturing the mean valence and arousal potential of words; (2) Inter-lexical predictors quantifying peaks, ranges, and dynamic changes within the lexical affective content; (3) Sublexical measures of basic affective tone according to sound-meaning correspondences at the sublexical level (see Aryani et al., 2016). We find the lexical predictors to account for a major amount of up to 50% of the variance in affective ratings. Moreover, inter-lexical and sublexical predictors account for a large portion of additional variance in the perceived general affective meaning. Together, the affective properties of all used textual features account for 43–70% of the variance in the affective ratings and still for 23–48% of the variance in the more abstract aesthetic ratings. In sum, our approach represents a novel method that successfully relates a prominent part of variance in perceived general affective meaning in this corpus of German poems to quantitative estimates of affective properties of textual components at the sublexical, lexical, and inter-lexical level.

Context matters: Anterior and posterior cortical midline responses to sad movie scenes

Narrative movies can create powerful emotional responses. While recent research has advanced the understanding of neural networks involved in immersive movie viewing, their modulation within a movie's dynamic context remains inconclusive. In this study, 24 healthy participants passively watched sad scene climaxes taken from 24 romantic comedies, while brain activity was measured using functional magnetic resonance (fMRI). To study effects of context, the sad scene climaxes were presented with either coherent scene context, replaced non-coherent context or without context. In a second viewing, the same clips were rated continuously for sadness. The ratings varied over time with peaks of experienced sadness within the assumed climax intervals. Activations in anterior and posterior cortical midline regions increased if presented with both coherent and replaced context, while activation in the temporal gyri decreased. This difference was more pronounced for the coherent context condition. Psycho-Physiological interactions (PPI) analyses showed a context-dependent coupling of midline regions with occipital visual and sub-cortical reward regions. Our results demonstrate the pivotal role of midline structures and their interaction with perceptual and reward areas in processing contextually embedded socio-emotional information in movies.

Drifting through Basic Subprocesses of Reading: A Hierarchical Diffusion Model Analysis of Age Effects on Visual Word Recognition

Reading is one of the most popular leisure activities and it is routinely performed by most individuals even in old age. Successful reading enables older people to master and actively participate in everyday life and maintain functional independence. Yet, reading comprises a multitude of subprocesses and it is undoubtedly one of the most complex accomplishments of the human brain. Not surprisingly, findings of age-related effects on word recognition and reading have been partly contradictory and are often confined to only one of four central reading subprocesses, i.e., sublexical, orthographic, phonological and lexico-semantic processing. The aim of the present study was therefore to systematically investigate the impact of age on each of these subprocesses. A total of 1,807 participants (young, N = 384; old, N = 1,423) performed four decision tasks specifically designed to tap one of the subprocesses. To account for the behavioral heterogeneity in older adults, this subsample was split into high and low performing readers. Data were analyzed using a hierarchical diffusion modeling approach, which provides more information than standard response time/accuracy analyses. Taking into account incorrect and correct response times, their distributions and accuracy data, hierarchical diffusion modeling allowed us to differentiate between age-related changes in decision threshold, non-decision time and the speed of information uptake. We observed longer non-decision times for older adults and a more conservative decision threshold. More importantly, high-performing older readers outperformed younger adults at the speed of information uptake in orthographic and lexico-semantic processing, whereas a general age-disadvantage was observed at the sublexical and phonological levels. Low-performing older readers were slowest in information uptake in all four subprocesses. Discussing these results in terms of computational models of word recognition, we propose age-related disadvantages for older readers to be caused by inefficiencies in temporal sampling and activation and/or inhibition processes.

Implicit Detection of Poetic Harmony by the Naïve Brain

The power of poetry is universally acknowledged, but it is debatable whether its appreciation is reserved for experts. Here, we show that readers with no particular knowledge of a traditional form of Welsh poetry unconsciously distinguish phrases conforming to its complex poetic construction rules from those that violate them. We studied the brain response of native speakers of Welsh as they read meaningful sentences ending in a word that either complied with strict poetic construction rules, violated rules of consonantal repetition, violated stress pattern, or violated both these constraints. Upon reading the last word of each sentence, participants indicated sentence acceptability. As expected, our inexperienced participants did not explicitly distinguish between sentences that conformed to the poetic rules from those that violated them. However, in the case of orthodox sentences, the critical word elicited a distinctive brain response characteristic of target detection –the P3b– as compared to the other conditions, showing that speakers of Welsh with no expertise of this particular form of poetry implicitly detect poetic harmony. These results show for the first time that before we even consider literal meaning, the musical properties of poetry speak to the human mind in ways that escape consciousness.

On Elementary Affective Decisions: To Like Or Not to Like, That Is the Question

Perhaps the most ubiquitous and basic affective decision of daily life is deciding whether we like or dislike something/somebody, or, in terms of psychological emotion theories, whether the object/subject has positive or negative valence. Indeed, people constantly make such liking decisions within a glimpse and, importantly, often without expecting any obvious benefit or knowing the exact reasons for their judgment. In this paper, we review research on such elementary affective decisions (EADs) that entail no direct overt reward with a special focus on Neurocognitive Poetics and discuss methods and models for investigating the neuronal and cognitive-affective bases of EADs to verbal materials with differing degrees of complexity. In line with evolutionary and appraisal theories of (aesthetic) emotions and data from recent neurocognitive studies, the results of a decision tree modeling approach simulating EADs to single words suggest that a main driving force behind EADs is the extent to which such high-dimensional stimuli are associated with the “basic” emotions joy/happiness and disgust.

Non-conventional figurative language as aesthetics of everyday communication

This study focuses on the emotional aesthetic appreciation of figurative language, a dimension which has often been neglected in experimental psycholinguistics. Our goal was to demonstrate that non-conventional figurative utterances are evaluated as more aesthetically pleasing although they are cognitively more demanding than conventional rhetorical figures. This hypothesis was tested for three main types of figurative language (metaphors, irony and idioms) in three separate surveys. Participants assessed utterances by means of a questionnaire which comprised several semantic differential items. The postulated covariation of non-conventionality and cognitive effort as well as of non-conventionality and aesthetics could be clearly established for metaphors and for irony. For idioms we could only partially provide this evidence. However, in a combined sample for all figurative language forms (compiled from the three studies) the main hypothesis was again confirmed. Thus, the results demonstrate that non-conventional variants of figurative language must be considered as the core of figurative aesthetics. Furthermore, our exploratory data gave evidence of an aesthetic paradox: the cognitive costs of understanding conventional figurative language reduce aesthetic pleasure, while in the case of non-conventional rhetoric figures the enhanced cognitive effort is accompanied by an increase in aesthetic pleasure.

Fiction: Simulation of Social Worlds

Fiction is the simulation of selves in interaction. People who read it improve their understanding of others. This effect is especially marked with literary fiction, which also enables people to change themselves. These effects are due partly to the process of engagement in stories, which includes making inferences and becoming emotionally involved, and partly to the contents of fiction, which include complex characters and circumstances that we might not encounter in daily life. Fiction can be thought of as a form of consciousness of selves and others that can be passed from an author to a reader or spectator, and can be internalized to augment everyday cognition.

The Berlin Affective Word List for Children (kidBAWL): Exploring Processing of Affective Lexical Semantics in the Visual and Auditory Modalities

While research on affective word processing in adults witnesses increasing interest, the present paper looks at another group of participants that have been neglected so far: pupils (age range: 6-12 years). Introducing a variant of the Berlin Affective Wordlist (BAWL) especially adapted for children of that age group, the "kidBAWL," we examined to what extent pupils process affective lexical semantics similarly to adults. In three experiments using rating and valence decision tasks in both the visual and auditory modality, it was established that children show the two ubiquitous phenomena observed in adults with emotional word material: the asymmetric U-shaped function relating valence to arousal ratings, and the inversely U-shaped function relating response times to valence decision latencies. The results for both modalities show large structural similarities between pupil and adult data (taken from previous studies) indicating that in the present age range, the affective lexicon and the dynamic interplay between language and emotion is already well-developed. Differential effects show that younger children tend to choose less extreme ratings than older children and that rating latencies decrease with age. Overall, our study should help to develop more realistic models of word recognition and reading that include affective processes and offer a methodology for exploring the roots of pleasant literary experiences and ludic reading.

Taking Perspective: Personal Pronouns Affect Experiential Aspects of Literary Reading

Personal pronouns have been shown to influence cognitive perspective taking during comprehension. Studies using single sentences found that 3^rd person pronouns facilitate the construction of a mental model from an observer’s perspective, whereas 2^nd person pronouns support an actor’s perspective. The direction of the effect for 1^st person pronouns seems to depend on the situational context. In the present study, we investigated how personal pronouns influence discourse comprehension when people read fiction stories and if this has consequences for affective components like emotion during reading or appreciation of the story. We wanted to find out if personal pronouns affect immersion and arousal, as well as appreciation of fiction. In a natural reading paradigm, we measured electrodermal activity and story immersion, while participants read literary stories with 1^st and 3^rd person pronouns referring to the protagonist. In addition, participants rated and ranked the stories for appreciation. Our results show that stories with 1^st person pronouns lead to higher immersion. Two factors—transportation into the story world and mental imagery during reading—in particular showed higher scores for 1^st person as compared to 3^rd person pronoun stories. In contrast, arousal as measured by electrodermal activity seemed tentatively higher for 3^rd person pronoun stories. The two measures of appreciation were not affected by the pronoun manipulation. Our findings underscore the importance of perspective for language processing, and additionally show which aspects of the narrative experience are influenced by a change in perspective.

Caring About Dostoyevsky: The Untapped Potential of Studying Literature

Should cognitive scientists and neuroscientists care about Dostoyevsky? Engaging with fiction is a natural and rich behavior, providing a unique window onto the mind and brain, particularly for mental simulation, emotion, empathy, and immersion. With advances in analysis techniques, it is time that cognitive scientists and neuroscientists embrace literature and fiction.

Human, Nature, Dynamism: The Effects of Content and Movement Perception on Brain Activations during the Aesthetic Judgment of Representational Paintings

Movement perception and its role in aesthetic experience have been often studied, within empirical aesthetics, in relation to the human body. No such specificity has been defined in neuroimaging studies with respect to contents lacking a human form. The aim of this work was to explore, through functional magnetic imaging (f MRI), how perceived movement is processed during the aesthetic judgment of paintings using two types of content: human subjects and scenes of nature. Participants, untutored in the arts, were shown the stimuli and asked to make aesthetic judgments. Additionally, they were instructed to observe the paintings and to rate their perceived movement in separate blocks. Observation highlighted spontaneous processes associated with aesthetic experience, whereas movement judgment outlined activations specifically related to movement processing. The ratings recorded during aesthetic judgment revealed that nature scenes received higher scored than human content paintings. The imaging data showed similar activation, relative to baseline, for all stimuli in the three tasks, including activation of occipito-temporal areas, posterior parietal, and premotor cortices. Contrast analyses within aesthetic judgment task showed that human content activated, relative to nature, precuneus, fusiform gyrus, and posterior temporal areas, whose activation was prominent for dynamic human paintings. In contrast, nature scenes activated, relative to human stimuli, occipital and posterior parietal cortex/precuneus, involved in visuospatial exploration and pragmatic coding of movement, as well as central insula. Static nature paintings further activated, relative to dynamic nature stimuli, central and posterior insula. Besides insular activation, which was specific for aesthetic judgment, we found a large overlap in the activation pattern characterizing each stimulus dimension (content and dynamism) across observation, aesthetic judgment, and movement judgment tasks. These findings support the idea that the aesthetic evaluation of artworks depicting both human subjects and nature scenes involves a motor component, and that the associated neural processes occur quite spontaneously in the viewer. Furthermore, considering the functional roles of posterior and central insula, we suggest that nature paintings may evoke aesthetic processes requiring an additional proprioceptive and sensori-motor component implemented by “motor accessibility” to the represented scenario, which is needed to judge the aesthetic value of the observed painting.

The emotion potential of simple sentences: additive or interactive effects of nouns and adjectives?

The vast majority of studies on affective processes in reading focus on single words. The most robust finding is a processing advantage for positively valenced words, which has been replicated in the rare studies investigating effects of affective features of words during sentence or story comprehension. Here we were interested in how the different valences of words in a sentence influence its processing and supralexical affective evaluation. Using a sentence verification task we investigated how comprehension of simple declarative sentences containing a noun and an adjective depends on the valences of both words. The results are in line with the assumed general processing advantage for positive words. We also observed a clear interaction effect, as can be expected from the affective priming literature: sentences with emotionally congruent words (e.g., The grandpa is clever) were verified faster than sentences containing emotionally incongruent words (e.g., The grandpa is lonely). The priming effect was most prominent for sentences with positive words suggesting that both, early processing as well as later meaning integration and situation model construction, is modulated by affective processing. In a second rating task we investigated how the emotion potential of supralexical units depends on word valence. The simplest hypothesis predicts that the supralexical affective structure is a linear combination of the valences of the nouns and adjectives (Bestgen, 1994). Overall, our results do not support this: The observed clear interaction effect on ratings indicate that especially negative adjectives dominated supralexical evaluation, i.e., a sort of negativity bias in sentence evaluation. Future models of sentence processing thus should take interactive affective effects into account.

The neuroaesthetics of prose fiction: pitfalls, parameters and prospects

There is a paucity of neuroaesthetic studies on prose fiction. This is in contrast to the very many impressive studies that have been conducted in recent times on the neuroaesthetics of sister arts such as painting, music and dance. Why might this be the case, what are its causes and, of greatest importance, how can it best be resolved? In this article, the pitfalls, parameters and prospects of a neuroaesthetics of prose fiction will be explored. The article itself is part critical review, part methodological proposal and part opinion paper. Its aim is simple: to stimulate, excite and energize thinking in the discipline as to how prose fiction might be fully integrated in the canon of neuroaesthetics and to point to opportunities where neuroimaging studies on literary discourse processing might be conducted in collaborative work bringing humanists and scientists together.

Rhetorical features facilitate prosodic processing while handicapping ease of semantic comprehension

Studies on rhetorical features of language have reported both enhancing and adverse effects on ease of processing. We hypothesized that two explanations may account for these inconclusive findings. First, the respective gains and losses in ease of processing may apply to different dimensions of language processing (specifically, prosodic and semantic processing) and different types of fluency (perceptual vs. conceptual) and may well allow for an integration into a more comprehensive framework. Second, the effects of rhetorical features may be sensitive to interactions with other rhetorical features; employing a feature separately or in combination with others may then predict starkly different effects. We designed a series of experiments in which we expected the same rhetorical features of the very same sentences to exert adverse effects on semantic (conceptual) fluency and enhancing effects on prosodic (perceptual) fluency. We focused on proverbs that each employ three rhetorical features: rhyme, meter, and brevitas (i.e., artful shortness). The presence of these target features decreased ease of conceptual fluency (semantic comprehension) while enhancing perceptual fluency as reflected in beauty and succinctness ratings that were mainly driven by prosodic features. The rhetorical features also predicted choices for persuasive purposes, yet only for the sentence versions featuring all three rhetorical features; the presence of only one or two rhetorical features had an adverse effect on the choices made. We suggest that the facilitating effects of a combination of rhyme, meter, and rhetorical brevitas on perceptual (prosodic) fluency overcompensated for their adverse effects on conceptual (semantic) fluency, thus resulting in a total net gain both in processing ease and in choices for persuasive purposes.

10 years of BAWLing into affective and aesthetic processes in reading: what are the echoes?

Reading is not only "cold" information processing, but involves affective and aesthetic processes that go far beyond what current models of word recognition, sentence processing, or text comprehension can explain. To investigate such "hot" reading processes, standardized instruments that quantify both psycholinguistic and emotional variables at the sublexical, lexical, inter-, and supralexical levels (e.g., phonological iconicity, word valence, arousal-span, or passage suspense) are necessary. One such instrument, the Berlin Affective Word List (BAWL) has been used in over 50 published studies demonstrating effects of lexical emotional variables on all relevant processing levels (experiential, behavioral, neuronal). In this paper, we first present new data from several BAWL studies. Together, these studies examine various views on affective effects in reading arising from dimensional (e.g., valence) and discrete emotion features (e.g., happiness), or embodied cognition features like smelling. Second, we extend our investigation of the complex issue of affective word processing to words characterized by a mixture of affects. These words entail positive and negative valence, and/or features making them beautiful or ugly. Finally, we discuss tentative neurocognitive models of affective word processing in the light of the present results, raising new issues for future studies.