The balance between memory and unification in semantics: A dynamic account of the N400

Frequency-based Segregation of Syntactic and Semantic Unification during Online Sentence Level Language Comprehension

During sentence level language comprehension, semantic and syntactic unification are functionally distinct operations. Nevertheless, both recruit roughly the same brain areas (spatially overlapping networks in the left frontotemporal cortex) and happen at the same time (in the first few hundred milliseconds after word onset). We tested the hypothesis that semantic and syntactic unification are segregated by means of neuronal synchronization of the functionally relevant networks in different frequency ranges: gamma (40 Hz and up) for semantic unification and lower beta (10-20 Hz) for syntactic unification. EEG power changes were quantified as participants read either correct sentences, syntactically correct though meaningless sentences (syntactic prose), or sentences that did not contain any syntactic structure (random word lists). Other sentences contained either a semantic anomaly or a syntactic violation at a critical word in the sentence. Larger EEG gamma-band power was observed for semantically coherent than for semantically anomalous sentences. Similarly, beta-band power was larger for syntactically correct sentences than for incorrect ones. These results confirm the existence of a functional dissociation in EEG oscillatory dynamics during sentence level language comprehension that is compatible with the notion of a frequency-based segregation of syntactic and semantic unification.

Disrupting morphosyntactic and lexical semantic processing has opposite effects on the sample entropy of neural signals

Converging evidence in neuroscience suggests that syntax and semantics are dissociable in brain space and time. However, it is possible that partly disjoint cortical networks, operating in successive time frames, still perform similar types of neural computations. To test the alternative hypothesis, we collected EEG data while participants read sentences containing lexical semantic or morphosyntactic anomalies, resulting in N400 and P600 effects, respectively. Next, we reconstructed phase space trajectories from EEG time series, and we measured the complexity of the resulting dynamical orbits using sample entropy - an index of the rate at which the system generates or loses information over time. Disrupting morphosyntactic or lexical semantic processing had opposite effects on sample entropy: it increased in the N400 window for semantic anomalies, and it decreased in the P600 window for morphosyntactic anomalies. These findings point to a fundamental divergence in the neural computations supporting meaning and grammar in language.

Gamma oscillations as a neural signature of shifting times in narrative language

Verbs and other temporal expressions allow speakers to specify the location of events in time, as well as to move back and forth in time, shifting in a narrative between past, present and future. The referential flexibility of temporal expressions is well understood in linguistics but its neurocognitive bases remain unknown. We aimed at obtaining a neural signature of shifting times in narrative language. We recorded and analyzed event-related brain potentials (ERPs) and oscillatory responses to the adverb 'now' and to the second main verb in Punctual ('An hour ago the boy stole a candy and now he peeled the fruit') and Iterative ('The entire afternoon the boy stole candy and now he peeled the fruit') contexts. 'An hour ago' introduces a time frame that lies entirely in the past, 'now' shifts the narrative to the present, and 'peeled' shifts it back to the past. These two referential shifts in Punctual contexts are expected to leave very similar traces on neural responses. In contrast, 'The entire afternoon' specifies a time frame that may encompass past, present and future, such that both 'now' and 'peeled' are consistent with it. Here, no time shift is required. We found no difference in ERPs between Punctual and Iterative contexts either at 'now' or at the second verb. However, reference shifts modulated oscillatory signals. 'Now' and the second verb in Punctual contexts resulted in similar responses: an increase in gamma power with a left-anterior distribution. Gamma bursts were absent in Iterative contexts. We propose that gamma oscillations here reflect the binding of temporal variables to the values allowed by constraints introduced by temporal expressions in discourse.

The LATL as locus of composition: MEG evidence from English and Arabic

Neurolinguistic investigations into the processing of structured sentences as well as simple adjective-noun phrases point to the left anterior temporal lobe (LATL) as a leading candidate for basic linguistic composition. Here, we characterized the combinatory profile of the LATL over a variety of syntactic and semantic environments, and across two languages, English and Arabic. The contribution of the LATL was investigated across two types of composition: the optional modification of a predicate (modification) and the satisfaction of a predicate's argument position (argument saturation). Target words were presented during MEG recordings, either in combinatory contexts (e.g. "eats meat") or in non-combinatory contexts (preceded by an unpronounceable consonant string, e.g. "xqkr meat"). Across both languages, the LATL showed increased responses to words in combinatory contexts, an effect that was robust to composition type and word order. Together with related findings, these results solidify the role of the LATL in basic semantic composition.

Two routes to actorhood: lexicalized potency to act and identification of the actor role

The inference of causality is a crucial cognitive ability and language processing is no exception: recent research suggests that, across different languages, the human language comprehension system attempts to identify the primary causer of the state of affairs described (the "actor") quickly and unambiguously (Bornkessel-Schlesewsky and Schlesewsky, 2009). This identification can take place verb-independently based on certain prominence cues (e.g., case, word order, animacy). Here, we present two experiments demonstrating that actor potential is also encoded at the level of individual nouns (a king is a better actor than a beggar). Experiment 1 collected ratings for 180 German nouns on 12 scales defined by adjective oppositions and deemed relevant for actorhood potential. By means of structural equation modeling, an actor potential (ACT) value was calculated for each noun. Experiment 2, an event-related potential study, embedded nouns from Experiment 1 in verb-final sentences, in which they were either actors or non-actors. N400 amplitude increased with decreasing ACT values and this modulation was larger for highly frequent nouns and for actor versus non-actor nouns. We argue that potency to act is lexically encoded for individual nouns and, since it modulates the N400 even for non-actor participants, it should be viewed as a property that modulates ease of lexical access (akin, for example, to lexical frequency). We conclude that two separate dimensions of actorhood computation are crucial to language comprehension: an experience-based, lexically encoded (bottom-up) representation of actorhood potential, and a prominence-based, computational mechanism for calculating goodness-of-fit to the actor role in a particular (top-down) sentence context.

The processing difference between person names and common nouns in sentence contexts: an ERP study

Person names and common nouns differ in how they are stored in the mental lexicon. Using event-related potentials, this study compared the integration of names and nouns into sentence contexts. Both person names and common nouns were highly related in meaning and either congruent or incongruent within the previous contexts. Name incongruence elicited an N400 effect, suggesting that people were able to rapidly retrieve the semantic meaning of names from long-term memory even when this process was mediated by person identification. Conversely, participants showed a "good enough" processing of the nouns due to their low specificity level and, thus, rich semantic associations, leading to a P600 effect. These distinctive ERP effects provide clear evidence for the distinctive semantic representations of these word categories by showing that the activation of a name's meaning is mediated by a single connection between identity-specific information and person identity, whereas multiple connections exist between nouns and their meanings.

Word-to-text integration: Message level and lexical level influences in ERPs

Although the reading of connected text proceeds in a largely incremental fashion, the relative degree to which message level and lexical level factors contribute to integration processes across sentences remains an open question. We examined the influence of both factors on single words using event-related potentials (ERPs). Word pairs with either strong or weak forward association strength were critical items: embedded as coreferential words within two-sentence passages in a text comprehension task, and as isolated word pairs in a word meaning judgment task. While the N400 ERP component reflected an effect of forward association strength on lexico-semantic processing in the word task (i.e., reduced N400 amplitudes were seen for strongly associated pairs relative to weakly associated pairs), in the comprehension task, passages embedded with any associated word pairs elicited reduced N400 amplitudes relative to coherent baseline passages lacking one of the critical words. These comprehension effects reflect responses from the highest skilled comprehenders. The results demonstrate the effects of message level factors, and reading abilities, on the processing of single words.

Prolegomena to a neurocomputational architecture for human grammatical encoding and decoding

This study develops a neurocomputational architecture for grammatical processing in language production and language comprehension (grammatical encoding and decoding, respectively). It seeks to answer two questions. First, how is online syntactic structure formation of the complexity required by natural-language grammars possible in a fixed, preexisting neural network without the need for online creation of new connections or associations? Second, is it realistic to assume that the seemingly disparate instantiations of syntactic structure formation in grammatical encoding and grammatical decoding can run on the same neural infrastructure? This issue is prompted by accumulating experimental evidence for the hypothesis that the mechanisms for grammatical decoding overlap with those for grammatical encoding to a considerable extent, thus inviting the hypothesis of a single "grammatical coder." The paper answers both questions by providing the blueprint for a syntactic structure formation mechanism that is entirely based on prewired circuitry (except for referential processing, which relies on the rapid learning capacity of the hippocampal complex), and can subserve decoding as well as encoding tasks. The model builds on the "Unification Space" model of syntactic parsing developed by Vosse and Kempen (Cognition 75:105-143, 2000; Cognitive Neurodynamics 3:331-346, 2009a). The design includes a neurocomputational mechanism for the treatment of an important class of grammatical movement phenomena.

Integrating the meaning of person names into discourse context: an event-related potential study

The meaning of person names is determined by their associated information. This study used event related potentials to investigate the time course of integrating the newly constructed meaning of person names into discourse context. The meaning of person names was built by two-sentence descriptions of the names. Then we manipulated the congruence of person names relative to discourse context in a way that the meaning of person names either matched or did not match the previous context. ERPs elicited by the names were compared between the congruent and the incongruent conditions. We found that the incongruent names elicited a larger N400 as well as a larger P600 compared to the congruent names. The results suggest that the meaning of unknown names can be effectively constructed from short linguistic descriptions and that the established meaning can be rapidly retrieved and integrated into contexts.

ERP evidence for conceptual mappings and comparison processes during the comprehension of conventional and novel metaphors

Cognitive linguists suggest that understanding metaphors requires activation of conceptual mappings between the involved concepts. We tested whether mappings are indeed in use during metaphor comprehension, and what mapping means as a cognitive process with Event-Related Potentials. Participants read literal, conventional metaphorical, novel metaphorical, and anomalous target sentences preceded by primes with related or unrelated mappings. Experiment 1 used sentence-primes to activate related mappings, and Experiment 2 used simile-primes to induce comparison thinking. In the unprimed conditions of both experiments, metaphors elicited N400s more negative than the literals. In Experiment 1, related sentence-primes reduced the metaphor-literal N400 difference in conventional, but not in novel metaphors. In Experiment 2, related simile-primes reduced the metaphor-literal N400 difference in novel, but not clearly in conventional metaphors. We suggest that mapping as a process occurs in metaphors, and the ways in which it can be facilitated by comparison differ between conventional and novel metaphors.

Sit down and read on: working memory and long-term memory in particle-verb processing

Particle verbs (e.g., look up) are lexical items for which particle and verb share a single lexical entry. Using event-related brain potentials, we examined working memory and long-term memory involvement in particle-verb processing. Dutch participants read sentences with head verbs that allow zero, two, or more than five particles to occur downstream. Additionally, sentences were presented for which the encountered particle was semantically plausible, semantically implausible, or forming a non-existing particle verb. An anterior negativity was observed at the verbs that potentially allow for a particle downstream relative to verbs that do not, possibly indexing storage of the verb until the dependency with its particle can be closed. Moreover, a graded N400 was found at the particle (smallest amplitude for plausible particles and largest for particles forming non-existing particle verbs), suggesting that lexical access to a shared lexical entry occurred at two separate time points.

Cortical dynamics of semantic processing during sentence comprehension: evidence from event-related optical signals

Using the event-related optical signal (EROS) technique, this study investigated the dynamics of semantic brain activation during sentence comprehension. Participants read sentences constituent-by-constituent and made a semantic judgment at the end of each sentence. The EROSs were recorded simultaneously with ERPs and time-locked to expected or unexpected sentence-final target words. The unexpected words evoked a larger N400 and a late positivity than the expected ones. Critically, the EROS results revealed activations first in the left posterior middle temporal gyrus (LpMTG) between 128 and 192 ms, then in the left anterior inferior frontal gyrus (LaIFG), the left middle frontal gyrus (LMFG), and the LpMTG in the N400 time window, and finally in the left posterior inferior frontal gyrus (LpIFG) between 832 and 864 ms. Also, expected words elicited greater activation than unexpected words in the left anterior temporal lobe (LATL) between 192 and 256 ms. These results suggest that the early lexical-semantic retrieval reflected by the LpMTG activation is followed by two different semantic integration processes: a relatively rapid and transient integration in the LATL and a relatively slow but enduring integration in the LaIFG/LMFG and the LpMTG. The late activation in the LpIFG, however, may reflect cognitive control.

Context-dependent Semantic Processing in the Human Brain: Evidence from Idiom Comprehension

Language comprehension involves activating word meanings and integrating them with the sentence context. This study examined whether these routines are carried out even when they are theoretically unnecessary, namely, in the case of opaque idiomatic expressions, for which the literal word meanings are unrelated to the overall meaning of the expression. Predictable words in sentences were replaced by a semantically related or unrelated word. In literal sentences, this yielded previously established behavioral and electrophysiological signatures of semantic processing: semantic facilitation in lexical decision, a reduced N400 for semantically related relative to unrelated words, and a power increase in the gamma frequency band that was disrupted by semantic violations. However, the same manipulations in idioms yielded none of these effects. Instead, semantic violations elicited a late positivity in idioms. Moreover, gamma band power was lower in correct idioms than in correct literal sentences. It is argued that the brain's semantic expectancy and literal word meaning integration operations can, to some extent, be “switched off” when the context renders them unnecessary. Furthermore, the results lend support to models of idiom comprehension that involve unitary idiom representations.

Reconciling time, space and function: a new dorsal-ventral stream model of sentence comprehension

We present a new dorsal-ventral stream framework for language comprehension which unifies basic neurobiological assumptions (Rauschecker & Scott, 2009) with a cross-linguistic neurocognitive sentence comprehension model (eADM; Bornkessel & Schlesewsky, 2006). The dissociation between (time-dependent) syntactic structure-building and (time-independent) sentence interpretation assumed within the eADM provides a basis for the division of labour between the dorsal and ventral streams in comprehension. We posit that the ventral stream performs time-independent unifications of conceptual schemata, serving to create auditory objects of increasing complexity. The dorsal stream engages in the time-dependent combination of elements, subserving both syntactic structuring and a linkage to action. Furthermore, frontal regions accomplish general aspects of cognitive control in the service of action planning and execution rather than linguistic processing. This architecture is supported by a range of existing empirical findings and helps to resolve a number of theoretical and empirical puzzles within the existing dorsal-ventral streams literature.

How long-term memory and accentuation interact during spoken language comprehension

Spoken language comprehension requires immediate integration of different information types, such as semantics, syntax, and prosody. Meanwhile, both the information derived from speech signals and the information retrieved from long-term memory exert their influence on language comprehension immediately. Using EEG (electroencephalogram), the present study investigated how the information retrieved from long-term memory interacts with accentuation during spoken language comprehension. Mini Chinese discourses were used as stimuli, with an interrogative or assertive context sentence preceding the target sentence. The target sentence included one critical word conveying new information. The critical word was either highly expected or lowly expected given the information retrieved from long-term memory. Moreover, the critical word was either consistently accented or inconsistently de-accented. The results revealed that for lowly expected new information, inconsistently de-accented words elicited a larger N400 and larger theta power increases (4-6 Hz) than consistently accented words. In contrast, for the highly expected new information, consistently accented words elicited a larger N400 and larger alpha power decreases (8-14 Hz) than inconsistently de-accented words. The results suggest that, during spoken language comprehension, the effect of accentuation interacted with the information retrieved from long-term memory immediately. Moreover, our results also have important consequences for our understanding of the processing nature of the N400. The N400 amplitude is not only enhanced for incorrect information (new and de-accented word) but also enhanced for correct information (new and accented words).

Levels of processing and language modality specificity in working memory

Neural networks underpinning working memory demonstrate sign language specific components possibly related to differences in temporary storage mechanisms. A processing approach to memory systems suggests that the organisation of memory storage is related to type of memory processing as well. In the present study, we investigated for the first time semantic, phonological and orthographic processing in working memory for sign- and speech-based language. During fMRI we administered a picture-based 2-back working memory task with Semantic, Phonological, Orthographic and Baseline conditions to 11 deaf signers and 20 hearing non-signers. Behavioural data showed poorer and slower performance for both groups in Phonological and Orthographic conditions than in the Semantic condition, in line with depth-of-processing theory. An exclusive masking procedure revealed distinct sign-specific neural networks supporting working memory components at all three levels of processing. The overall pattern of sign-specific activations may reflect a relative intermodality difference in the relationship between phonology and semantics influencing working memory storage and processing.

CNTNAP2 and language processing in healthy individuals as measured with ERPs

The genetic FOXP2-CNTNAP2 pathway has been shown to be involved in the language capacity. We investigated whether a common variant of CNTNAP2 (rs7794745) is relevant for syntactic and semantic processing in the general population by using a visual sentence processing paradigm while recording ERPs in 49 healthy adults. While both AA homozygotes and T-carriers showed a standard N400 effect to semantic anomalies, the response to subject-verb agreement violations differed across genotype groups. T-carriers displayed an anterior negativity preceding the P600 effect, whereas for the AA group only a P600 effect was observed. These results provide another piece of evidence that the neuronal architecture of the human faculty of language is shaped differently by effects that are genetically determined.

Reduced N400 semantic priming effects in adult survivors of paediatric and adolescent traumatic brain injury

The immediate and long-term neural correlates of linguistic processing deficits reported following paediatric and adolescent traumatic brain injury (TBI) are poorly understood. Therefore, the current research investigated event-related potentials (ERPs) elicited during a semantic picture-word priming experiment in two groups of highly functioning individuals matched for various demographic variables and behavioural language performance. Participants in the TBI group had a recorded history of paediatric or adolescent TBI involving injury mechanisms associated with diffuse white matter pathology, while participants in the control group never sustained any insult to the brain. A comparison of N400 Mean Amplitudes elicited during three experimental conditions with varying semantic relatedness between the prime and target stimuli (congruent, semantically related, unrelated) revealed a significantly smaller N400 response in the unrelated condition in the TBI group, indicating residual linguistic processing deviations when processing demands required the quick detection of a between-category (unrelated) violation of semantic expectancy.

Brain network of semantic integration in sentence reading: insights from independent component analysis and graph theoretical analysis

A set of cortical and sub-cortical brain structures has been linked with sentence-level semantic processes. However, it remains unclear how these brain regions are organized to support the semantic integration of a word into sentential context. To look into this issue, we conducted a functional magnetic resonance imaging (fMRI) study that required participants to silently read sentences with semantically congruent or incongruent endings and analyzed the network properties of the brain with two approaches, independent component analysis (ICA) and graph theoretical analysis (GTA). The GTA suggested that the whole-brain network is topologically stable across conditions. The ICA revealed a network comprising the supplementary motor area (SMA), left inferior frontal gyrus, left middle temporal gyrus, left caudate nucleus, and left angular gyrus, which was modulated by the incongruity of sentence ending. Furthermore, the GTA specified that the connections between the left SMA and left caudate nucleus as well as that between the left caudate nucleus and right thalamus were stronger in response to incongruent vs. congruent endings.

So that's what you meant! Event-related potentials reveal multiple aspects of context use during construction of message-level meaning

Factors that modulate the influence of contextual information on semantic processing in language comprehension have been thoroughly investigated with the N400 component of the event-related potential (ERP), a direct measure of initial contact with semantic memory. Although context has a strong and immediate impact on processing, multiple mechanisms contribute to the construction of message-level representations during normal comprehension. Some of these may be engaged after or concurrent with the formation of an initial meaning representation, and can then serve to revise or reshape meaning. In this study, ERPs were recorded while participants read plausible sentences that continuously varied in the amount of contextual constraint for the sentence-final word, defined via extensive norming data including the range of possible alternative completions for the contexts. Consistent with numerous past studies, the amplitude of the N400 was graded with expectancy, as amplitudes decreased with increasing constraint. Additionally, a left-lateralized, broad, slow negativity onsetting around 400-500 ms was largest for sentences with moderately strong constraint. Within this range of constraint, the negativity was larger for sentences with fewer alternative completions compared to those with many different ones. The timing and scalp distribution of the effect resemble brain responses linked to engagement of working memory resources, ambiguity resolution, and comprehension of jokes. Similar to cases of "frame-shifting" in non-literal language, this effect may reflect processing associated with reinterpretation or reconsideration of contextual material when multiple interpretations of a sentence were likely.

Beta oscillations relate to the N400m during language comprehension

The relationship between the evoked responses (ERPs/ERFs) and the event-related changes in EEG/MEG power that can be observed during sentence-level language comprehension is as yet unclear. This study addresses a possible relationship between MEG power changes and the N400m component of the event-related field. Whole-head MEG was recorded while subjects listened to spoken sentences with incongruent (IC) or congruent (C) sentence endings. A clear N400m was observed over the left hemisphere, and was larger for the IC sentences than for the C sentences. A time-frequency analysis of power revealed a decrease in alpha and beta power over the left hemisphere in roughly the same time range as the N400m for the IC relative to the C condition. A linear regression analysis revealed a positive linear relationship between N400m and beta power for the IC condition, not for the C condition. No such linear relation was found between N400m and alpha power for either condition. The sources of the beta decrease were estimated in the LIFG, a region known to be involved in semantic unification operations. One source of the N400m was estimated in the left superior temporal region, which has been related to lexical retrieval. We interpret our data within a framework in which beta oscillations are inversely related to the engagement of task-relevant brain networks. The source reconstructions of the beta power suppression and the N400m effect support the notion of a dynamic communication between the LIFG and the left superior temporal region during language comprehension.

Differences in word recognition between early bilinguals and monolinguals: behavioral and ERP evidence

We investigated the behavioral and brain responses (ERPs) of bilingual word recognition to three fundamental psycholinguistic factors, frequency, morphology, and lexicality, in early bilinguals vs. monolinguals. Earlier behavioral studies have reported larger frequency effects in bilinguals' nondominant vs. dominant language and in some studies also when compared to corresponding monolinguals. In ERPs, language processing differences between bilinguals vs. monolinguals have typically been found in the N400 component. In the present study, highly proficient Finnish-Swedish bilinguals who had acquired both languages during childhood were compared to Finnish monolinguals during a visual lexical decision task and simultaneous ERP recordings. Behaviorally, we found that the response latencies were overall longer in bilinguals than monolinguals, and that the effects for all three factors, frequency, morphology, and lexicality were also larger in bilinguals even though they had acquired both languages early and were highly proficient in them. In line with this, the N400 effects induced by frequency, morphology, and lexicality were larger for bilinguals than monolinguals. Furthermore, the ERP results also suggest that while most inflected Finnish words are decomposed into stem and suffix, only monolinguals have encountered high frequency inflected word forms often enough to develop full-form representations for them. Larger behavioral and neural effects in bilinguals in these factors likely reflect lower amount of exposure to words compared to monolinguals, as the language input of bilinguals is divided between two languages.

Predictability and novelty in literal language comprehension: an ERP study

Linguists have suggested that one mechanism for the creative extension of meaning in language involves mapping, or constructing correspondences between conceptual domains. For example, the sentence, "The clever boys used a cardboard box as a boat," sets up a novel mapping between the concepts cardboard box and boat, while "His main method of transportation is a boat," relies on a more conventional mapping between method of transportation and boat. To examine the electrophysiological signature of this mapping process, electroencephalogram (EEG) was recorded from the scalp as healthy adults read three sorts of sentences: low-cloze (unpredictable) conventional ("His main method of transportation is a boat,"), low-cloze novel mapp'ing ("The clever boys used a cardboard box as a boat,"), and high-cloze (predictable) conventional ("The only way to get around Venice is to navigate the canals in a boat,"). Event-related brain potentials (ERPs) were time-locked to sentence final words. The novel and conventional conditions were matched for cloze probability (a measure of predictability based on the sentence context), lexical association between the sentence frame and the final word (using latent semantic analysis), and other factors known to influence ERPs to language stimuli. The high-cloze conventional control condition was included to compare the effects of mapping conventionality to those of predictability. The N400 component of the ERPs was affected by predictability but not by conventionality. By contrast, a late positivity was affected both by the predictability of sentence final words, being larger for words in low-cloze contexts that made target words difficult to predict, and by novelty, as words in the novel condition elicited a larger positivity 700-900ms than the same words in the (cloze-matched) conventional condition.