Neural correlates of combinatorial semantic processing of literal and figurative noun noun compound words

Neural network of metaphor comprehension: an ALE meta-analysis and MACM analysis

The comprehension of metaphor, a vivid and figurative language, is a complex endeavor requiring cooperation among multiple cognitive systems. There are still many important questions regarding neural mechanisms implicated in specific types of metaphor. To address these questions, we conducted activation likelihood estimation meta-analyses on 30 studies (containing data of 480 participants) and meta-analytic connectivity modeling analyses. First, the results showed that metaphor comprehension engaged the inferior frontal gyrus, middle temporal gyrus, fusiform gyrus, lingual gyrus, and middle occipital gyrus-all in the left hemisphere. In addition to the commonly reported networks of language and attention, metaphor comprehension engaged networks of visual. Second, sub-analysis showed that the contextual complexity can modulate figurativeness, with the convergence on the left fusiform gyrus during metaphor comprehension at discourse-level. Especially, right hemisphere only showed convergence in studies of novel metaphors, suggesting that the right hemisphere is more associated with difficulty than metaphorical. The work here extends knowledge of the neural mechanisms underlying metaphor comprehension in individual brain regions and neural networks.

Shared and distinct neural correlates of first and second language morphological processing in bilingual brain

While morphology constitutes a crucial component of the human language system, the neural bases of morphological processing in the human brain remains to be elucidated. The current study aims at exploring the extent to which the second language (L2) morphological processing would resemble or differ from that of their first language (L1) in adult Chinese-English bilinguals. Bilingual participants were asked to complete a morphological priming lexical decision task drawing on derivational morphology, which is present for both Chinese and English, when their electrophysiological and optical responses were recorded concurrently. Functional near-infrared spectroscopy (fNIRS) revealed a neural dissociation between morphological and semantic priming effects in the left fronto-temporal network, while L1 Chinese engaged enhanced activation in the left prefrontal cortex for morphological parsing relative to L2 English. In the early stage of lexical processing, cross-language morphological processing manifested a difference in degree, not in kind, as revealed by the early left anterior negativity (ELAN) effect. In addition, L1 and L2 shared both early and late structural parsing processes (P250 and 300 ~ 500 ms negativity, respectively). Therefore, the current results support a unified competition model for bilingual development, where bilinguals would primarily employ L1 neural resources for L2 morphological representation and processing.

Lesion-symptom mapping of language impairments in people with brain tumours: The influence of linguistic stimuli

People with tumours in specific brain sites might face difficulties in tasks with different linguistic material. Previous lesion-symptom mapping studies (VLSM) demonstrated that people with tumours in posterior temporal regions have more severe linguistic impairments. However, to the best of our knowledge, preoperative performance and lesion location on tasks with different linguistic stimuli have not been examined. In the present study, we performed VLSM on 52 people with left gliomas to examine whether tumour distribution differs depending on the tasks of the Aachen Aphasia Test. The VLSM analysis revealed that single-word production (e.g. object naming) was associated with the inferior parietal lobe and that compound and sentence production were additionally associated with posterior temporal gyri. Word repetition was affected in people with tumours in inferior parietal areas, whereas sentence repetition was the only task to be associated with frontal regions. Subcortically, word and sentence production were found to be affected in people with tumours reaching the arcuate fasciculus, and compound production was primarily associated with tumours affecting the inferior longitudinal and inferior fronto-occipital fasciculus. Our work shows that tasks with linguistic stimuli other than single-word naming (e.g. compound and sentence production) relate to additional cortical and subcortical brain areas. At a clinical level, we show that tasks that target the same processes (e.g. repetition) can have different neural correlates depending on the linguistic stimuli used. Also, we highlight the importance of left temporoparietal areas.

Visual event-related potentials reveal the early whole-word lexical processing of Chinese two-character words

Morphologically complex words are common across different languages, especially in Chinese, because more than 90% of common modern Chinese words are complex words. Many behavioral studies have suggested the whole-word processing of Chinese complex words, but the neural correlates of whole-word processing remain unclear. Previous electrophysiological studies revealed automatic and early (∼250 ms) access to the orthographic forms of monomorphic words in the ventral occipitotemporal area. In this study, we investigated whether there is also automatic and early orthographic recognition of Chinese complex words (as whole units) by recording event-related potentials (ERPs). A total of 150 two-character words and 150 two-character pseudowords composed of the same 300 characters (morphemes) were pseudorandomly presented to proficient Chinese readers. Participants were required to determine the color of each stimulus in the color decision task and to determine whether each stimulus was a word in the lexical decision task. The two constituent characters of each stimulus were horizontally arranged in Experiment 1 and vertically arranged in Experiment 2. The results revealed a significant early ERP difference between words and pseudowords approximately 250-300 ms after stimulus onset in the parieto-occipital scalp region. The early ERP difference was more prominent in the color decision task than in the lexical decision task, more prominent in Experiment 1 than in Experiment 2, and more prominent in the left parieto-occipital scalp region than in the right. Source analysis results showed that the early ERP difference originated from the left ventral occipitotemporal cortex. These results reflected early and automatic access to whole-word orthographic representations of Chinese complex words in the left ventral occipitotemporal cortex.

Left and right temporal-parietal junctions (TPJs) as "match/mismatch" hedonic machines: A unifying account of TPJ function

Experimental and theoretical studies have tried to gain insights into the involvement of the Temporal Parietal Junction (TPJ) in a broad range of cognitive functions like memory, attention, language, self-agency and theory of mind. Recent investigations have demonstrated the partition of the TPJ in discrete subsectors. Nonetheless, whether these subsectors play different roles or implement an overarching function remains debated. Here, based on a review of available evidence, we propose that the left TPJ codes both matches and mismatches between expected and actual sensory, motor, or cognitive events while the right TPJ codes mismatches. These operations help keeping track of statistical contingencies in personal, environmental, and conceptual space. We show that this hypothesis can account for the participation of the TPJ in disparate cognitive functions, including "humour", and explain: a) the higher incidence of spatial neglect in right brain damage; b) the different emotional reactions that follow left and right brain damage; c) the hemispheric lateralisation of optimistic bias mechanisms; d) the lateralisation of mechanisms that regulate routine and novelty behaviours. We propose that match and mismatch operations are aimed at approximating "free energy", in terms of the free energy principle of decision-making. By approximating "free energy", the match/mismatch TPJ system supports both information seeking to update one's own beliefs and the pleasure of being right in one's own' current choices. This renewed view of the TPJ has relevant clinical implications because the misfunctioning of TPJ-related "match" and "mismatch" circuits in unilateral brain damage can produce low-dimensional deficits of active-inference and predictive coding that can be associated with different neuropsychological disorders.

Deep neural networks reveal topic-level representations of sentences in medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus

When reading a sentence, individual words can be combined to create more complex meaning. In this study, we sought to uncover brain regions that reflect the representation of the meaning of sentences at the topic level, as opposed to the meaning of their individual constituent words when considered irrespective of their context. Using fMRI, we recorded the neural activity of participants while reading sentences. We constructed a topic-level sentence representations using the final layer of a convolutional neural network (CNN) trained to classify Wikipedia sentences into broad semantic categories. This model was contrasted with word-level sentence representations constructed using the average of the word embeddings constituting the sentence. Using representational similarity analysis, we found that the medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus more strongly represent sentence topic-level, compared to word-level, meaning, uncovering the important role of these semantic system regions in the representation of topic-level meaning. Results were comparable when sentence meaning was modelled with a multilayer perceptron that was not sensitive to word order within a sentence, suggesting that the learning objective, in the terms of the topic being modelled, is the critical factor in capturing these neural representational spaces.

From words to phrases: neural basis of social event semantic composition

Events are typically composed of at least actions and entities. Both actions and entities have been shown to be represented by neural structures respecting domain organizations in the brain, including those of social/animate (face and body; person-directed action) versus inanimate (man-made object or tool; object-directed action) concepts. It is unclear whether the brain combines actions and entities into events in a (relative) domain-specific fashion or via domain-general mechanisms in regions that have been shown to support semantic and syntactic composition. We tested these hypotheses in a functional magnetic resonance imaging experiment where two domains of verb-noun event phrases (social-person versus manipulation-artifact, e.g., “hug mother” versus “fold napkin”) and their component words were contrasted. We found a series of brain region supporting social-composition effects more strongly than the manipulation phrase composition—the bilateral inferior occipital gyrus (IOG), inferior temporal gyrus (ITG) and anterior temporal lobe (ATL)—which either showed stronger activation strength tested by univariate contrast, stronger content representation tested by representation similarity analysis, or stronger relationship between the neural activation patterns of phrases and synthesis (additive and multiplication) of the neural activity patterns of the word constituents. No regions were observed showing evidence of phrase composition for both domains or stronger effects of manipulation phrases. These findings highlight the roles of the visual cortex and ATL in social event compositions, suggesting a domain-preferring, rather than domain-general, mechanisms of verbal event composition.

The Elephant in the Room: A Systematic Review of Stimulus Control in Neuro-Measurement Studies on Figurative Language Processing

The processing of metaphors and idioms has been the subject of neuroscientific research for several decades. However, results are often contradictory, which can be traced back to inconsistent terminology and stimulus control. In this systematic review of research methods, we analyse linguistic aspects of 116 research papers which used EEG, fMRI, PET, MEG, or NIRS to investigate the neural processing of the two figurative subtypes metaphor and idiom. We critically examine the theoretical foundations as well as stimulus control by performing a systematic literature synthesis according to the PRISMA guidelines. We explicitly do not analyse the findings of the studies but instead focus on four primary aspects: definitions of figurative language and its subtypes, linguistic theory behind the studies, control for factors influencing figurative language processing, and the relationship between theoretical and operational definitions. We found both a lack and a broad variety in existing definitions and operationalisation, especially in regard to familiarity and conventionality. We identify severe obstacles in the comparability and validation potential of the results of the papers in our review corpus. We propose the development of a consensus in fundamental terminology and more transparency in the reporting of stimulus design in the research on figurative language processing.

Examining the transition of novel information toward familiarity

Throughout their lives, humans encounter multiple instances of new information that can be inconsistent with prior knowledge (novel). Over time, the once-novel information becomes integrated into their established knowledge base, shifting from novelty to familiarity. In this study, we investigated the processes by which the first steps of this transition take place. We hypothesized that the neural representations of initially novel items gradually change over the course of repeated presentations, expressing a shift toward familiarity. We further assumed that this shift could be traced by examining neural patterns using fMRI. In two experiments, while being scanned, participants read noun-adjective word pairs that were either consistent or inconsistent with their prior knowledge. Stimuli were repeated 3-6 times within the scans. Employing mass univariate and multivariate similarity analyses, we showed that the neural representations associated with the initial presentation of familiar versus novel objects differed in lateral frontal and temporal regions, the medial prefrontal cortex, and the medial temporal lobe. Importantly, the neural representations of novel stimuli gradually changed throughout repetitions until they became indistinguishable from their respective familiar items. We interpret these findings as indicating that an early phase of familiarization can be completed within a few repetitions. This initial familiarization can then serve as the prerequisite to the integration of novel items into existing knowledge. Future empirical and theoretical works can build on the current findings to develop a comprehensive model of the transition from novelty to familiarity.

The neural basis of compound word processing revealed by varying semantic transparency and morphemic neighborhood size

This study investigated the neural basis of compound word processing by using fMRI and Chinese two-character compounds for lexical decision. Semantic transparency and morphemic neighborhood size were manipulated to augment the processing profile for measurement. The behavioral results disclosed a semantic transparency effect and its interaction with the neighborhood size, which supported existence of a mechanism for compound processing. The fMRI results located a neural substrate in the left inferior prefrontal cortex (BA 45) which reacted in an interactive manner to the two variables. While its activities were lower when their neighborhood size was larger for processing transparent compounds, its activities became higher when their neighborhood size was larger for processing opaque compounds. When scaling to a larger scope, the function of this mechanism fitted well with the theoretical account of unification function of the left inferior frontal cortex for language processing.

Psycholinguistic norms for a set of 506 French compound words

Compounds are morphologically complex words made of different linguistic parts. They are very prevalent in a number of languages such as French. Different psycholinguistic characteristics of compounds have been used in certain studies to investigate the mechanisms involved in compound processing (see Table 7). We provide psycholinguistic norms for a set of 506 French compound words. The words were normed on seven characteristics: lexeme meaning dominance, semantic transparency, sensory experience, conceptual familiarity, imageability, age of acquisition (AoA) and subjective frequency. Reliability measures were computed for the collected norms. Descriptive statistical analyses, and correlational and multiple regression analyses were performed. We also report some comparisons made between our normative data and certain norms obtained in other similar studies. The entire set of norms, which will be very useful to researchers investigating the processing of compounds, is available as Supplemental Material.

Which is better for concept visualization? Shape or spirit similarity: Evidence from event-related potentials

Shape and spirit similarity are two kinds of common artistic modes in concept visualization. The adoption depends on the designers' subjective preference and judgment, which may cause potential risks for semantic communication. This article used pairs of real image-concrete word as the roots, and contrasted four kinds of multimodal mappings such as shape similarity-concrete concept, shape similarity-abstract concept, spirit similarity-concrete concept, and spirit similarity-abstract concept to compare the matching difference through the S1(picture)-S2(word) paradigm. The behavioral results showed that shape similarity had advantages in both matching rate and reaction time over spirit similarity, but the difference was more significant to the concrete word than to the abstract word. The ERPs showed that the N1, P2, and N400 components had alike effects with the behavioral results, but the mappings of spirit similarity-concrete concept elicited the largest positivity of P600, suggesting the complicated mechanisms of semantic integration and concreteness effect in the multimodal mappings. This study proves that the concrete concept should be visualized according to its appearance, not the most striking feature or function; but the visulization of abstract concept shows less difference after a concreteness transition.

Affective and psycholinguistic norms for German conceptual metaphors (COMETA)

Figurative expressions have been shown to play a special role in evoking affective responses, as compared to their literal counterparts. This study provides the first database of conceptual metaphors that includes ratings of affective properties beyond psycholinguistic properties. To allow for the investigation of natural reading processes, 64 natural stories were created, half of which contained two or three conceptual metaphors that relied on the same mapping, whereas the other half contained the metaphors' literal counterparts. To allow for tighter control and manipulation of the different properties, 120 isolated sentences were also created, half of which contained one metaphorical word, which was replaced by its literal rendering in the other half. All stimuli were rated for emotional valence, arousal, imageability, and metaphoricity, and the pairs of metaphorical and literal stimuli were rated for their similarity in meaning. A measure of complexity was determined and computed. The stories were also rated for naturalness and understandability, and the sentences for familiarity. Differences between the metaphorical and literal stimuli and relationships between the affective and psycholinguistic variables were explored and are discussed in light of extant empirical research. In a nutshell, the metaphorical stimuli were rated as being higher in emotional arousal and easier to imagine than their literal counterparts, thus confirming a role of metaphor in evoking emotion and in activating sensorimotor representations. Affective variables showed the typical U-shaped relationship consistently found in word databases, whereby increasingly positive and negative valence is associated with higher arousal. Finally, interesting differences between the stories and sentences were observed.

An Electrophysiological Abstractness Effect for Metaphorical Meaning Making

Neuroimaging studies show that metaphors activate sensorimotor areas. These findings were interpreted as metaphors contributing to conceptual thought by mapping concrete, somatosensory information onto abstract ideas. But is sensorimotor information a necessary constituent of figurative meaning? The present study employed event-related potentials (ERPs) in a divided visual field paradigm with healthy adults to explore the role of sensorimotor feature processing in the comprehension of novel metaphors via the electrophysiological concreteness effect. Participants read French, novel adjective-noun expressions that were either metaphorical (“fat sentence”) or literal (“fat hip”). While literal expressions evoked a typical concreteness effect, an enhanced frontal negativity during right hemisphere (RH) as opposed to left hemisphere (LH) presentation, metaphors showed no such sign of sensorimotor feature processing. Relative to literals, they evoked a sustained frontal negativity during LH presentation and similar amplitudes during RH presentation, but both of these effects were the greater the more abstract the metaphors were. It is the first time such an electrophysiological abstractness effect is reported, just the opposite of a concreteness effect. It is particularly noteworthy that ERPs evoked by metaphors were not contingent on figurativeness, novelty, meaningfulness, imageability, emotional valence, or arousal, only on abstractness. When compared with similarly novel literal expressions, metaphors did not evoke a typical N400 and did not activate the RH either. The findings shed new light on the neurocognitive machinery of figurative meaning construction, pervasive in everyday communication. Contrary to embodied cognition, the conceptual system might be organized around abstract representations and not sensorimotor information, even for lush, metaphorical language.

Metaphor processing is supramodal semantic processing: The role of the bilateral lateral temporal regions in multimodal communication

This paper presents an fMRI study on healthy adult understanding of metaphors in multimodal communication. We investigated metaphors expressed either only in coverbal gestures ("monomodal metaphors") or in speech with accompanying gestures ("multimodal metaphors"). Monomodal metaphoric gestures convey metaphoric information not expressed in the accompanying speech (e.g. saying the non-metaphoric utterance, "She felt bad" while dropping down the hand with palm facing up; here, the gesture alone indicates metaphoricity), whereas coverbal gestures in multimodal metaphors indicate metaphoricity redundant to the speech (e.g. saying the metaphoric utterance, "Her spirits fell" while dropping the hand with palm facing up). In other words, in monomodal metaphors, gestures add information not spoken, whereas the gestures in multimodal metaphors can be redundant to the spoken content. Understanding and integrating the information in each modality, here spoken and visual, is important in multimodal communication, but most prior studies have only considered multimodal metaphors where the gesture is redundant to what is spoken. Our participants watched audiovisual clips of an actor speaking while gesturing. We found that abstract metaphor comprehension recruited the lateral superior/middle temporal cortices, regardless of the modality in which the conceptual metaphor is expressed. These results suggest that abstract metaphors, regardless of modality, involve resources implicated in general semantic processing and are consistent with the role of these areas in supramodal semantic processing as well as the theory of embodied cognition.

Dissociating the neural correlates of the sociality and plausibility effects in simple conceptual combination

Neuroimaging studies have indicated that a brain network distributed in the supramodal cortical regions of the frontal, temporal, and parietal lobes plays a central role in conceptual processing. The activation of this network is modulated by two orthogonal dimensions in conceptual processing-the semantic features of individual concepts and the meaningfulness of conceptual combinations-but it remains unclear how the network is functionally organized along these two dimensions. In this fMRI study, we focused on two specific factors, i.e. the social semantic richness of words and the semantic plausibility of word combinations, along the two dimensions. In literature, the distributions of the effects of the two factors are very similar, but have not been rigorously compared in one study. We orthogonally manipulated the two factors in a phrase comprehension task and found a clear dissociation between their effects. The combination of these results with our previous findings reveals three adjacently distributed subnetworks of the supramodal semantic network, associated with the sociality effect, imageability effect, and semantic plausibility effect, respectively. Further analysis of the resting-state functional connectivity data indicated that the functional dissociation among the three subnetworks is associated with their underlying intrinsic connectivity structures.

Concepts and Compositionality: In Search of the Brain's Language of Thought

Imagine Genghis Khan, Aretha Franklin, and the Cleveland Cavaliers performing an opera on Maui. This silly sentence makes a serious point: As humans, we can flexibly generate and comprehend an unbounded number of complex ideas. Little is known, however, about how our brains accomplish this. Here we assemble clues from disparate areas of cognitive neuroscience, integrating recent research on language, memory, episodic simulation, and computational models of high-level cognition. Our review is framed by Fodor's classic language of thought hypothesis, according to which our minds employ an amodal, language-like system for combining and recombining simple concepts to form more complex thoughts. Here, we highlight emerging work on combinatorial processes in the brain and consider this work's relation to the language of thought. We review evidence for distinct, but complementary, contributions of map-like representations in subregions of the default mode network and sentence-like representations of conceptual relations in regions of the temporal and prefrontal cortex.

Affective iconic words benefit from additional sound-meaning integration in the left amygdala

Recent studies have shown that a similarity between sound and meaning of a word (i.e., iconicity) can help more readily access the meaning of that word, but the neural mechanisms underlying this beneficial role of iconicity in semantic processing remain largely unknown. In an fMRI study, we focused on the affective domain and examined whether affective iconic words (e.g., high arousal in both sound and meaning) activate additional brain regions that integrate emotional information from different domains (i.e., sound and meaning). In line with our hypothesis, affective iconic words, compared to their non‐iconic counterparts, elicited additional BOLD responses in the left amygdala known for its role in multimodal representation of emotions. Functional connectivity analyses revealed that the observed amygdalar activity was modulated by an interaction of iconic condition and activations in two hubs representative for processing sound (left superior temporal gyrus) and meaning (left inferior frontal gyrus) of words. These results provide a neural explanation for the facilitative role of iconicity in language processing and indicate that language users are sensitive to the interaction between sound and meaning aspect of words, suggesting the existence of iconicity as a general property of human language.

Idiomatic expressions evoke stronger emotional responses in the brain than literal sentences

Recent neuroscientific research shows that metaphors engage readers at the emotional level more strongly than literal expressions. What still remains unclear is what makes metaphors more engaging, and whether this generalises to all figurative expressions, no matter how conventionalised they are. This fMRI study aimed to investigate whether idiomatic expressions - the least creative part of figurative language - indeed trigger a higher affective resonance than literal expressions, and to explore possible interactions between activation in emotion-relevant neural structures and regions associated with figurative language processing. Participants silently read for comprehension a set of emotionally positive, negative and neutral idioms embedded in short sentences, and similarly valenced literal sentences. As in studies on metaphors, we found enhanced activation of the left inferior frontal gyrus and left amygdala in response to idioms, indexing stronger recruitment of executive control functions and enhanced emotional engagement, respectively. This suggests that the comprehension of even highly conventionalised and familiar figurative expressions, namely idioms, recruits regions involved in emotional processing. Furthermore, increased activation of the IFG interacted positively with activation in the amygdala, suggesting that the stronger cognitive engagement driven by idioms may in turn be coupled with stronger involvement at the emotional level.

Morphological and Whole-Word Semantic Processing Are Distinct: Event Related Potentials Evidence From Spoken Word Recognition in Chinese

Behavioral and imaging studies in alphabetic languages have shown that morphological processing is a discrete and independent element of lexical access. However, there is no explicit marker of morphological structure in Chinese complex words, such that the extent to which morpheme meaning is related to word meaning is unknown. Event-related potentials (ERPs) were used in the present study to investigate the dissociation of morphemic and whole-word meaning in an auditory-auditory primed lexical decision task. All the prime and target words are compounds consisting of two Chinese morphemes. The relationship between morpheme and whole-word meaning was manipulated while controlling the phonology and orthography of the first syllable in each prime-target pair. A clear dissociation was found between morphemic and whole-word meaning on N400 amplitude and topography. Specifically, sharing a morpheme produced a larger N400 in the anterior-central electrode sites, while sharing whole-word meaning produced a smaller N400 in central-posterior electrode sites. In addition, the morphological N400 effect was negatively correlated with the participants' reading ability, with better readers needing less orthographic information to distinguish different morphemes in compound words. These findings indicate that morphological and whole-word meaning are dissociated in spoken Chinese compound word recognition and that even in the spoken language modality, good readers are better able to access the meaning of individual morphemes in Chinese compound word processing.

A Quantitative Meta-analysis of Neuroimaging Studies of Pragmatic Language Comprehension: In Search of a Universal Neural Substrate

Pragmatics may be defined as the ability to communicate by expressing and recognizing intentions. The objective of this meta-analysis was to identify neural substrates for comprehension of pragmatic content in general, as well as the differences between pragmatic forms, and to describe if there is differential recruitment of brain areas according to natural language. This meta-analysis included 48 functional magnetic resonance imaging studies that reported pragmatic versus literal language contrasts. The pragmatic forms were speech acts, metaphors, idioms, and irony. Effect Size-Signed Differential Mapping software was used to calculate the mean for all contrasts as well as for each pragmatic form, and make comparisons among all forms. Due to variations in pragmatic content configuration such as natural language, stimulus modality, and writing systems, these variations were also analyzed with subgroups' analyses. The analyses found a highly reproducible bilateral fronto-temporal and medial prefrontal cortex network for pragmatic comprehension. Each pragmatic form showed a specific convergence pattern within this bilateral network. Natural language analyses showed that fronto-temporal regions were recruited by Germanic languages, while only left frontal areas were recruited by Romance languages, and right medial prefrontal cortex by Japanese. In conclusion, pragmatic language comprehension involves classical language areas in bilateral perisylvian regions, along with the medial prefrontal cortex, an area involved in social cognition. Together, these areas could represent the "pragmatic language network". Nonetheless, when proposing a universal neural substrate for all forms of pragmatic language, the diversity among studies in terms of pragmatic form, and configuration, must be taken into consideration.

Morphological processing in the brain: The good (inflection), the bad (derivation) and the ugly (compounding)

There is considerable behavioral evidence that morphologically complex words such as 'tax-able' and 'kiss-es' are processed and represented combinatorially. In other words, they are decomposed into their constituents 'tax' and '-able' during comprehension (reading or listening), and producing them might also involve on-the-spot combination of these constituents (especially for inflections). However, despite increasing amount of neurocognitive research, the neural mechanisms underlying these processes are still not fully understood. The purpose of this critical review is to offer a comprehensive overview on the state-of-the-art of the research on the neural mechanisms of morphological processing. In order to take into account all types of complex words, we include findings on inflected, derived, and compound words presented both visually and aurally. More specifically, we cover a wide range of electro- and magnetoencephalography (EEG and MEG, respectively) as well as structural/functional magnetic resonance imaging (s/fMRI) studies that focus on morphological processing. We present the findings with respect to the temporal course and localization of morphologically complex word processing. We summarize the observed findings, their interpretations with respect to current psycholinguistic models, and discuss methodological approaches as well as their possible limitations.

The Fictive Brain: Neurocognitive Correlates of Engagement in Literature

Fiction is vital to our being. Many people enjoy engaging with fiction every day. Here we focus on literary reading as 1 instance of fiction consumption from a cognitive neuroscience perspective. The brain processes which play a role in the mental construction of fiction worlds and the related engagement with fictional characters, remain largely unknown. The authors discuss the neurocognitive poetics model ( Jacobs, 2015a ) of literary reading specifying the likely neuronal correlates of several key processes in literary reading, namely inference and situation model building, immersion, mental simulation and imagery, figurative language and style, and the issue of distinguishing fact from fiction. An overview of recent work on these key processes is followed by a discussion of methodological challenges in studying the brain bases of fiction processing.

When emotions are expressed figuratively: Psycholinguistic and Affective Norms of 619 Idioms for German (PANIG)

Despite flourishing research on the relationship between emotion and literal language, and despite the pervasiveness of figurative expressions in communication, the role of figurative language in conveying affect has been underinvestigated. This study provides affective and psycholinguistic norms for 619 German idiomatic expressions and explores the relationships between affective and psycholinguistic idiom properties. German native speakers rated each idiom for emotional valence, arousal, familiarity, semantic transparency, figurativeness, and concreteness. They also described the figurative meaning of each idiom and rated how confident they were about the attributed meaning. The results showed that idioms rated high in valence were also rated high in arousal. Negative idioms were rated as more arousing than positive ones, in line with results from single words. Furthermore, arousal correlated positively with figurativeness (supporting the idea that figurative expressions are more emotionally engaging than literal expressions) and with concreteness and semantic transparency. This suggests that idioms may convey a more direct reference to sensory representations, mediated by the meanings of their constituting words. Arousal correlated positively with familiarity. In addition, positive idioms were rated as more familiar than negative idioms. Finally, idioms without a literal counterpart were rated as more emotionally valenced and arousing than idioms with a literal counterpart. Although the meanings of ambiguous idioms were less correctly defined than those of unambiguous idioms, ambiguous idioms were rated as more concrete than unambiguous ones. We also discuss the relationships between the various psycholinguistic variables characterizing idioms, with reference to the literature on idiom structure and processing.

On Elementary Affective Decisions: To Like Or Not to Like, That Is the Question

Perhaps the most ubiquitous and basic affective decision of daily life is deciding whether we like or dislike something/somebody, or, in terms of psychological emotion theories, whether the object/subject has positive or negative valence. Indeed, people constantly make such liking decisions within a glimpse and, importantly, often without expecting any obvious benefit or knowing the exact reasons for their judgment. In this paper, we review research on such elementary affective decisions (EADs) that entail no direct overt reward with a special focus on Neurocognitive Poetics and discuss methods and models for investigating the neuronal and cognitive-affective bases of EADs to verbal materials with differing degrees of complexity. In line with evolutionary and appraisal theories of (aesthetic) emotions and data from recent neurocognitive studies, the results of a decision tree modeling approach simulating EADs to single words suggest that a main driving force behind EADs is the extent to which such high-dimensional stimuli are associated with the “basic” emotions joy/happiness and disgust.

Mixing positive and negative valence: Affective-semantic integration of bivalent words

Single words have affective and aesthetic properties that influence their processing. Here we investigated the processing of a special case of word stimuli that are extremely difficult to evaluate, bivalent noun-noun-compounds (NNCs), i.e. novel words that mix a positive and negative noun, e.g. ‘Bombensex’ (bomb-sex). In a functional magnetic resonance imaging (fMRI) experiment we compared their processing with easier-to-evaluate non-bivalent NNCs in a valence decision task (VDT). Bivalent NNCs produced longer reaction times and elicited greater activation in the left inferior frontal gyrus (LIFG) than non-bivalent words, especially in contrast to words of negative valence. We attribute this effect to a LIFG-grounded process of semantic integration that requires greater effort for processing converse information, supporting the notion of a valence representation based on associations in semantic networks.

Disentangling Metaphor from Context: An ERP Study

A large body of electrophysiological literature showed that metaphor comprehension elicits two different event-related brain potential responses, namely the so-called N400 and P600 components. Yet most of these studies test metaphor in isolation while in natural conversation metaphors do not come out of the blue but embedded in linguistic and extra-linguistic context. This study aimed at assessing the role of context in the metaphor comprehension process. We recorded EEG activity while participants were presented with metaphors and equivalent literal expressions in a minimal context (Experiment 1) and in a supportive context where the word expressing the ground between the metaphor's topic and vehicle was made explicit (Experiment 2). The N400 effect was visible only in minimal context, whereas the P600 was visible both in the absence and in the presence of contextual cues. These findings suggest that the N400 observed for metaphor is related to contextual aspects, possibly indexing contextual expectations on upcoming words that guide lexical access and retrieval, while the P600 seems to reflect truly pragmatic interpretative processes needed to make sense of a metaphor and derive the speaker's meaning, also in the presence of contextual cues. In sum, previous information in the linguistic context biases toward a metaphorical interpretation but does not suppress interpretative pragmatic mechanisms to establish the intended meaning.

Neuronal bases of structural coherence in contemporary dance observation

The neuronal processes underlying dance observation have been the focus of an increasing number of brain imaging studies over the past decade. However, the existing literature mainly dealt with effects of motor and visual expertise, whereas the neural and cognitive mechanisms that underlie the interpretation of dance choreographies remained unexplored. Hence, much attention has been given to the action observation network (AON) whereas the role of other potentially relevant neuro-cognitive mechanisms such as mentalizing (theory of mind) or language (narrative comprehension) in dance understanding is yet to be elucidated. We report the results of an fMRI study where the structural coherence of short contemporary dance choreographies was manipulated parametrically using the same taped movement material. Our participants were all trained dancers. The whole-brain analysis argues that the interpretation of structurally coherent dance phrases involves a subpart (superior parietal) of the AON as well as mentalizing regions in the dorsomedial prefrontal cortex. An ROI analysis based on a similar study using linguistic materials (Pallier et al., 2011) suggests that structural processing in language and dance might share certain neural mechanisms.

Rhetorical features facilitate prosodic processing while handicapping ease of semantic comprehension

Studies on rhetorical features of language have reported both enhancing and adverse effects on ease of processing. We hypothesized that two explanations may account for these inconclusive findings. First, the respective gains and losses in ease of processing may apply to different dimensions of language processing (specifically, prosodic and semantic processing) and different types of fluency (perceptual vs. conceptual) and may well allow for an integration into a more comprehensive framework. Second, the effects of rhetorical features may be sensitive to interactions with other rhetorical features; employing a feature separately or in combination with others may then predict starkly different effects. We designed a series of experiments in which we expected the same rhetorical features of the very same sentences to exert adverse effects on semantic (conceptual) fluency and enhancing effects on prosodic (perceptual) fluency. We focused on proverbs that each employ three rhetorical features: rhyme, meter, and brevitas (i.e., artful shortness). The presence of these target features decreased ease of conceptual fluency (semantic comprehension) while enhancing perceptual fluency as reflected in beauty and succinctness ratings that were mainly driven by prosodic features. The rhetorical features also predicted choices for persuasive purposes, yet only for the sentence versions featuring all three rhetorical features; the presence of only one or two rhetorical features had an adverse effect on the choices made. We suggest that the facilitating effects of a combination of rhyme, meter, and rhetorical brevitas on perceptual (prosodic) fluency overcompensated for their adverse effects on conceptual (semantic) fluency, thus resulting in a total net gain both in processing ease and in choices for persuasive purposes.

10 years of BAWLing into affective and aesthetic processes in reading: what are the echoes?

Reading is not only "cold" information processing, but involves affective and aesthetic processes that go far beyond what current models of word recognition, sentence processing, or text comprehension can explain. To investigate such "hot" reading processes, standardized instruments that quantify both psycholinguistic and emotional variables at the sublexical, lexical, inter-, and supralexical levels (e.g., phonological iconicity, word valence, arousal-span, or passage suspense) are necessary. One such instrument, the Berlin Affective Word List (BAWL) has been used in over 50 published studies demonstrating effects of lexical emotional variables on all relevant processing levels (experiential, behavioral, neuronal). In this paper, we first present new data from several BAWL studies. Together, these studies examine various views on affective effects in reading arising from dimensional (e.g., valence) and discrete emotion features (e.g., happiness), or embodied cognition features like smelling. Second, we extend our investigation of the complex issue of affective word processing to words characterized by a mixture of affects. These words entail positive and negative valence, and/or features making them beautiful or ugly. Finally, we discuss tentative neurocognitive models of affective word processing in the light of the present results, raising new issues for future studies.

Neurocognitive poetics: methods and models for investigating the neuronal and cognitive-affective bases of literature reception

A long tradition of research including classical rhetoric, esthetics and poetics theory, formalism and structuralism, as well as current perspectives in (neuro)cognitive poetics has investigated structural and functional aspects of literature reception. Despite a wealth of literature published in specialized journals like Poetics, however, still little is known about how the brain processes and creates literary and poetic texts. Still, such stimulus material might be suited better than other genres for demonstrating the complexities with which our brain constructs the world in and around us, because it unifies thought and language, music and imagery in a clear, manageable way, most often with play, pleasure, and emotion (Schrott and Jacobs, 2011). In this paper, I discuss methods and models for investigating the neuronal and cognitive-affective bases of literary reading together with pertinent results from studies on poetics, text processing, emotion, or neuroaesthetics, and outline current challenges and future perspectives.

The magical activation of left amygdala when reading Harry Potter: an fMRI study on how descriptions of supra-natural events entertain and enchant

Literature containing supra-natural, or magical events has enchanted generations of readers. When reading narratives describing such events, readers mentally simulate a text world different from the real one. The corresponding violation of world-knowledge during this simulation likely increases cognitive processing demands for ongoing discourse integration, catches readers’ attention, and might thus contribute to the pleasure and deep emotional experience associated with ludic immersive reading. In the present study, we presented participants in an MR scanner with passages selected from the Harry Potter book series, half of which described magical events, while the other half served as control condition. Passages in both conditions were closely matched for relevant psycholinguistic variables including, e.g., emotional valence and arousal, passage-wise mean word imageability and frequency, and syntactic complexity. Post-hoc ratings showed that readers considered supra-natural contents more surprising and more strongly associated with reading pleasure than control passages. In the fMRI data, we found stronger neural activation for the supra-natural than the control condition in bilateral inferior frontal gyri, bilateral inferior parietal lobules, left fusiform gyrus, and left amygdala. The increased activation in the amygdala (part of the salience and emotion processing network) appears to be associated with feelings of surprise and the reading pleasure, which supra-natural events, full of novelty and unexpectedness, brought about. The involvement of bilateral inferior frontal gyri likely reflects higher cognitive processing demand due to world knowledge violations, whereas increased attention to supra-natural events is reflected in inferior frontal gyri and inferior parietal lobules that are part of the fronto-parietal attention network.

Metaphors are physical and abstract: ERPs to metaphorically modified nouns resemble ERPs to abstract language

Metaphorical expressions very often involve words referring to physical entities and experiences. Yet, figures of speech such as metaphors are not intended to be understood literally, word-by-word. We used event-related brain potentials (ERPs) to determine whether metaphorical expressions are processed more like physical or more like abstract expressions. To this end, novel adjective-noun word pairs were presented visually in three conditions: (1) Physical, easy to experience with the senses (e.g., “printed schedule”); (2) Abstract, difficult to experience with the senses (e.g., “conditional schedule”); and (3) novel Metaphorical, expressions with a physical adjective, but a figurative meaning (e.g., “thin schedule”). We replicated the N400 lexical concreteness effect for concrete vs. abstract adjectives. In order to increase the sensitivity of the concreteness manipulation on the expressions, we divided each condition into high and low groups according to rated concreteness. Mirroring the adjective result, we observed a N400 concreteness effect at the noun for physical expressions with high concreteness ratings vs. abstract expressions with low concreteness ratings, even though the nouns per se did not differ in lexical concreteness. Paradoxically, the N400 to nouns in the metaphorical expressions was indistinguishable from that to nouns in the literal abstract expressions, but only for the more concrete subgroup of metaphors; the N400 to the less concrete subgroup of metaphors patterned with that to nouns in the literal concrete expressions. In sum, we not only find evidence for conceptual concreteness separable from lexical concreteness but also that the processing of metaphorical expressions is not driven strictly by either lexical or conceptual concreteness.

Familiarity differentially affects right hemisphere contributions to processing metaphors and literals

The role of the two hemispheres in processing metaphoric language is controversial. While some studies have reported a special role of the right hemisphere (RH) in processing metaphors, others indicate no difference in laterality relative to literal language. Some studies have found a role of the RH for novel/unfamiliar metaphors, but not conventional/familiar metaphors. It is not clear, however, whether the role of the RH is specific to metaphor novelty, or whether it reflects processing, reinterpretation or reanalysis of novel/unfamiliar language in general. Here we used functional magnetic resonance imaging (fMRI) to examine the effects of familiarity in both metaphoric and non-metaphoric sentences. A left lateralized network containing the middle and inferior frontal gyri, posterior temporal regions in the left hemisphere (LH), and inferior frontal regions in the RH, was engaged across both metaphoric and non-metaphoric sentences; engagement of this network decreased as familiarity decreased. No region was engaged selectively for greater metaphoric unfamiliarity. An analysis of laterality, however, showed that the contribution of the RH relative to that of LH does increase in a metaphor-specific manner as familiarity decreases. These results show that RH regions, taken by themselves, including commonly reported regions such as the right inferior frontal gyrus (IFG), are responsive to increased cognitive demands of processing unfamiliar stimuli, rather than being metaphor-selective. The division of labor between the two hemispheres, however, does shift towards the right for metaphoric processing. The shift results not because the RH contributes more to metaphoric processing. Rather, relative to its contribution for processing literals, the LH contributes less.

Flying under the radar: figurative language impairments in focal lesion patients

Despite the prevalent and natural use of metaphor in everyday language, the neural basis of this powerful communication device remains poorly understood. Early studies of brain-injured patients suggested the right hemisphere plays a critical role in metaphor comprehension, but more recent patient and neuroimaging studies do not consistently support this hypothesis. One explanation for this discrepancy is the challenge in designing optimal tasks for brain-injured populations. As traditional aphasia assessments do not assess figurative language comprehension, we designed a new metaphor comprehension task to consider whether impaired metaphor processing is missed by standard clinical assessments. Stimuli consisted of 60 pairs of moderately familiar metaphors and closely matched literal sentences. Sentences were presented visually in a randomized order, followed by four adjective-noun answer choices (target + three foil types). Participants were instructed to select the phrase that best matched the meaning of the sentence. We report the performance of three focal lesion patients and a group of 12 healthy, older controls. Controls performed near ceiling in both conditions, with slightly more accurate performance on literal than metaphoric sentences. While the Western Aphasia Battery (Kertesz, 1982) and the objects and actions naming battery (Druks and Masterson, 2000) indicated minimal to no language difficulty, our metaphor comprehension task indicated three different profiles of metaphor comprehension impairment in the patients’ performance. Single case statistics revealed comparable impairment on metaphoric and literal sentences, disproportionately greater impairment on metaphors than literal sentences, and selective impairment on metaphors. We conclude our task reveals that patients can have selective metaphor comprehension deficits. These deficits are not captured by traditional neuropsychological language assessments, suggesting overlooked communication difficulties.

The role of literal meaning in figurative language comprehension: evidence from masked priming ERP

The role of literal meaning during the construction of meaning that goes beyond pure literal composition was investigated by combining cross-modal masked priming and ERPs. This experimental design was chosen to compare two conflicting theoretical positions on this topic. The indirect access account claims that literal aspects are processed first, and additional meaning components are computed only if no satisfactory interpretation is reached. In contrast, the direct access approach argues that figurative aspects can be accessed immediately. We presented metaphors (These lawyers are hyenas, Experiment 1a and 1b) and producer-for-product metonymies (The boy read Böll, Experiment 2a and 2b) with and without a prime word that was semantically relevant to the literal meaning of the target word (furry and talented, respectively). In the presentation without priming, metaphors revealed a biphasic N400-Late Positivity pattern, while metonymies showed an N400 only. We interpret the findings within a two-phase language architecture where contextual expectations guide initial access (N400) and precede pragmatic adjustment resulting in reconceptualization (Late Positivity). With masked priming, the N400-difference was reduced for metaphors and vanished for metonymies. This speaks against the direct access view that predicts a facilitating effect for the literal condition only and hence would predict the N400-difference to increase. The results are more consistent with indirect access accounts that argue for facilitation effects for both conditions and consequently for consistent or even smaller N400-amplitude differences. This combined masked priming ERP paradigm therefore yields new insights into the role of literal meaning in the online composition of figurative language.

Compound words prompt arbitrary semantic associations in conceptual memory

Linguistic relativity theory has received empirical support in domains such as color perception and object categorization. It is unknown, however, whether relations between words idiosyncratic to language impact non-verbal representations and conceptualizations. For instance, would one consider the concepts of horse and sea as related were it not for the existence of the compound seahorse? Here, we investigated such arbitrary conceptual relationships using a non-linguistic picture relatedness task in participants undergoing event-related brain potential recordings. Picture pairs arbitrarily related because of a compound and presented in the compound order elicited N400 amplitudes similar to unrelated pairs. Surprisingly, however, pictures presented in the reverse order (as in the sequence horse–sea) reduced N400 amplitudes significantly, demonstrating the existence of a link in memory between these two concepts otherwise unrelated. These results break new ground in the domain of linguistic relativity by revealing predicted semantic associations driven by lexical relations intrinsic to language.

Beyond the N400: complementary access to early neural correlates of novel metaphor comprehension using combined electrophysiological and haemodynamic measurements

The simultaneous application of different neuroimaging methods combining high temporal and spatial resolution can uniquely contribute to current issues and open questions in the field of pragmatic language perception. In the present study, comprehension of novel metaphors was investigated using near-infrared spectroscopy (NIRS) combined with the simultaneous acquisition of electroencephalography (EEG)/event-related potentials (ERPs). For the first time, we investigated the effects of figurative language on early electrophysiological markers (P200, N400) and their functional relationship to cortical haemodynamic responses within the language network (Broca's area, Wernicke's area). To this end, 20 healthy subjects judged 120 sentences with respect to their meaningfulness, whereby phrases were either literal, metaphoric, or meaningless. Our results indicated a metaphor-specific P200 reduction and a linear increase of N400 amplitudes from literal over metaphoric to meaningless sentences. Moreover, there were metaphor related effects on haemodynamic responses accessed with NIRS, especially within the left lateral frontal cortex (Broca's area). Significant correlations between electrophysiological and haemodynamic responses indicated that P200 reductions during metaphor comprehension were associated with an increased recruitment of neural activity within left Wernicke's area, indicating a link between variations in neural activity and haemodynamic changes within Wernicke's area. This link may reflect processes related to interindividual differences regarding the ability to classify novel metaphors. The present study underlines the usefulness of simultaneous NIRS measurements in language paradigms - especially for investigating the functional significance of neurophysiological markers that have so far been rarely examined - as these measurements are easily and efficiently realizable and allow for a complementary examination of neural activity and associated metabolic changes in cortical areas.

Lateralized processing of novel metaphors: disentangling figurativeness and novelty

One of the intriguing and sometimes controversial findings in figurative language research is a right-hemisphere processing advantage for novel metaphors. The current divided visual field study introduced novel literal expressions as a control condition to assess processing novelty independent of figurativeness. Participants evaluated word pairs belonging to one of the five categories: (1) conventional metaphorical, (2) conventional literal, (3) novel metaphorical, (4) novel literal, and (5) unrelated expressions in a semantic decision task. We presented expressions without sentence context and controlled for additional factors including emotional valence, arousal, and imageability that could potentially influence hemispheric processing. We also utilized an eye-tracker to ensure lateralized presentation. We did not find the previously reported right-hemispherical processing advantage for novel metaphors. Processing was faster in the left hemisphere for all types of word pairs, and accuracy was also higher in the right visual field - left hemisphere. Novel metaphors were processed just as fast as novel literal expressions, suggesting that the primary challenge during the comprehension of novel expressions is not a serial processing of salience, but perhaps a more left hemisphere weighted semantic integration. Our results cast doubt on the right-hemisphere theory of metaphors, and raise the possibility that other uncontrolled variables were responsible for previous results. The lateralization of processing of two word expressions seems to be more contingent on the specific task at hand than their figurativeness or saliency.