Syntactically Based Sentence Processing Classes: Evidence from Event-Related Brain Potentials

Morphosyntactic prediction in automatic neural processing of spoken language: EEG evidence

Automatic parsing of syntactic information by the human brain is a well-established phenomenon, but its mechanisms remain poorly understood. Its best-known neurophysiological reflection is the so-called early left-anterior negativity (ELAN) component of event-related potentials (ERPs), with two alternative hypotheses for its origin: (1) error detection, or (2) morphosyntactic prediction/priming. To test these alternatives, we conducted two experiments using a non-attend passive design with visual distraction and recorded ERPs to spoken pronoun-verb phrases with/without agreement violations and to the same critical verbs presented in isolation without preceding pronouns. The results revealed an ELAN at ∼130-220 ms for pronoun-verb gender agreement violations, confirming a high degree of automaticity in early morphosyntactic parsing. Critically, the strongest ELAN was elicited by verbs outside phrasal context, which suggests that the typical ELAN pattern is underpinned by a reduction of ERP amplitudes for felicitous combinations, reflecting syntactic priming/predictability between related words/morphemes (potentially mediated by associative links formed during previous linguistic experience) rather than specialised error-detection processes.

Cortical networks responsive to phrase structure and subject island violations

In principle, functional neuroimaging provides uniquely informative data in addressing linguistic questions, because it can indicate distinct processes that are not apparent from behavioral data alone. This could involve adjudicating the source of unacceptability via the different patterns of elicited brain responses to different ungrammatical sentence types. However, it is difficult to interpret brain activations to syntactic violations. Such responses could reflect processes that have nothing intrinsically related to linguistic representations, such as domain-general executive function abilities. In order to facilitate the potential use of functional neuroimaging methods to identify the source of different syntactic violations, we conducted an fMRI experiment to identify the brain activation maps associated with two distinct syntactic violation types: phrase structure (created by inverting the order of two adjacent words within a sentence) and subject islands (created by extracting a wh-phrase out of an embedded subject). The comparison of these violations to control sentences surprisingly showed no indication of a generalized violation response, with almost completely divergent activation patterns. Phrase structure violations seemingly activated regions previously implicated in verbal working memory and structural complexity in sentence processing, whereas the subject islands appeared to activate regions previously implicated in conceptual-semantic processing, broadly defined. We review our findings in the context of previous research on syntactic and semantic violations using event-related potentials. We suggest that functional neuroimaging is a potentially fruitful technique in unpacking the distinct sets of cognitive processes elicited by theoretically-relevant syntactic violations, when interpreted with care and paired with appropriate control conditions.

Neurophysiological signatures of prediction in language: A critical review of anticipatory negativities

Recent event-related potential (ERP) studies in language comprehension converge in finding anticipatory negativities preceding words or word segments that can be pre-activated based on either sentence contexts or phonological cues. We review these findings from different paradigms in the light of evidence from other cognitive domains in which slow negative potentials have long been associated with anticipatory processes and discuss their potential underlying mechanisms. We propose that this family of anticipatory negativities captures common mechanisms associated with the pre-activation of linguistic information both within words and within sentences. Future studies could utilize these anticipatory negativities in combination with other, well-established ERPs, to simultaneously track prediction-related processes emerging at different time intervals (before and after the perception of pre-activated input) and with distinct time courses (shorter-lived and longer-lived cognitive operations).

Anaphoric distance dependencies in visual narrative structure and processing

Linguistic syntax has often been claimed as uniquely complex due to features like anaphoric relations and distance dependencies. However, visual narratives of sequential images, like those in comics, have been argued to use sequencing mechanisms analogous to those in language. These narrative structures include "refiner" panels that "zoom in" on the contents of another panel. Similar to anaphora in language, refiners indexically connect inexplicit referential information in one unit (refiner, pronoun) to a more informative "antecedent" elsewhere in the discourse. Also like in language, refiners can follow their antecedents (anaphoric) or precede them (cataphoric), along with having either proximal or distant connections. We here explore the constraints on visual narrative refiners created by modulating these features of order and distance. Experiment 1 examined participants' preferences for where refiners are placed in a sequence using a force-choice test, which revealed that refiners are preferred to follow their antecedents and have proximal distances from them. Experiment 2 then showed that distance dependencies lead to slower self-paced viewing times. Finally, measurements of event-related brain potentials (ERPs) in Experiment 3 revealed that these patterns evoke similar brain responses as referential dependencies in language (i.e., N400, LAN, Nref). Across all three studies, the constraints and (neuro)cognitive responses to refiners parallel those shown to anaphora in language, suggesting domain-general constraints on the sequencing of referential dependencies.

Individual differences in the neural dynamics of visual narrative comprehension: The effects of proficiency and age of acquisition

Understanding visual narrative sequences, as found in comics, is known to recruit similar cognitive mechanisms to verbal language. As measured by event-related potentials (ERPs), these manifest as initial negativities (N400, LAN) and subsequent positivities (P600). While these components are thought to index discrete processing stages, they differentially arise across participants for any given stimulus. In language contexts, proficiency modulates brain responses, with smaller N400 effects and larger P600 effects appearing with increasing proficiency. In visual narratives, recent work has also emphasized the role of proficiency in neural response patterns. We thus explored whether individual differences in proficiency modulate neural responses to visual narrative sequencing in similar ways as in language. We combined ERP data from 12 studies examining semantic and/or grammatical processing of visual narrative sequences. Using linear mixed effects modeling, we demonstrate differential effects of visual language proficiency and "age of acquisition" on N400 and P600 responses. Our results align with those reported in language contexts, providing further evidence for the similarity of linguistic and visual narrative processing, and emphasize the role of both proficiency and age of acquisition in visual narrative comprehension.

Syntax matters in shaping sensorimotor activation driven by nouns

Existing evidence has shown that adjectives modulate the grasp-compatibility effect elicited by object nouns. The aim of the present study was to investigate the role of syntax on the sensorimotor activation elicited by nouns in a grasp-compatibility task. We assessed two languages with different syntactic rules, Italian in Experiment 1 and English in Experiment 2. In both experiments, an adjective-noun pair was shown on the screen. The adjective was always in a pre-nominal position and denoted either a disadvantageous quality of the object graspability (e.g., sharp) or the object colour (e.g., reddish). Participants had to categorize the object nouns as natural or artifact, performing a precision or a power reach-to-grasp movement. On different trials, the grasp response was compatible or incompatible with the grip typically used to manipulate the object indicated by the noun. In Experiment 1 (Italian language) the adjective-noun order violated the syntactic order and no difference emerged between reaction times on compatible and incompatible trials (no grasp compatibility effect). In Experiment 2 (English language), the adjective-noun order followed the syntactic rule. Results showed a grasp-compatibility effect when a colour adjective was presented before a natural object noun. When a disadvantageous adjective preceded an artifact or a natural object noun, an inverted grasp-compatibility effect emerged with slower responses on compatible than incompatible trials. Taken together, these findings suggest that adjectives can shape the sensorimotor activation elicited by nouns of graspable objects only when the syntax is correct. Results are discussed with respect to embodied cognition theories.

Parafoveal syntactic processing from word N + 2 during reading: the case of gender-specific German articles

Previous research has suggested that some syntactic information such as word class can be processed parafoveally during reading. However, it is still unclear to what extent early syntactic cueing within noun phrases can facilitate word processing during dynamic reading. Two experiments (total N = 72) were designed to address this question using a gaze-contingent boundary change paradigm to manipulate the syntactic fit within a nominal phrase. Either the article (Experiment 1) or the noun (Experiment 2) was manipulated in the parafovea, resulting in a syntactic mismatch, depending on the condition. Results indicated a substantial elevation of viewing times on both parts of the noun phrase when conflicting syntactic information had been present in the parafovea. In Experiment 1, the article was also fixated more often in the syntactic mismatch condition. These results provide direct evidence of parafoveal syntactic processing. Based on the early time-course of this effect, it can be concluded that grammatical gender is used to generate constraints for the processing of upcoming nouns. To our knowledge, these results also provide the first evidence that syntactic information can be extracted from a parafoveal word N + 2.

Syntactic prediction during self-paced reading is age invariant

Controversy exists as to whether, compared to young adults, older adults are more, equally or less likely to make linguistic predictions while reading. While previous studies have examined age effects on the prediction of upcoming words, the prediction of upcoming syntactic structures has been largely unexplored. We compared the benefit that young and older readers gain when the syntactic structure is made predictable, as well as potential age differences in the costs involved in making predictions. In a self-paced reading study, 60 young and 60 older adults read sentences in which noun-phrase coordination (e.g. large pizza or tasty calzone) is made predictable through the inclusion of the word either earlier in the sentence. Results showed a benefit of the presence of either in the second half of the coordination phrase, and a cost of the presence of either in the first half. We observed no age differences in the benefit or costs of making these predictions; Bayes factor analyses offered strong evidence that these effects are age invariant. Together, these findings suggest that both older and younger adults make similar strength syntactic predictions with a similar level of difficulty. We relate this age invariance in syntactic prediction to specific aspects of the ageing process.

Mandarin Chinese L1 and L2 complex sentence reading reveals a consistent electrophysiological pattern of highly interactive syntactic and semantic processing: An ERP study

Introduction

A hallmark of the human language faculty is processing complex hierarchical syntactic structures across languages. However, for Mandarin Chinese, a language typically dependent on semantic combinations and free of morphosyntactic information, the relationship between syntactic and semantic processing during Chinese complex sentence reading is unclear. From the neuropsychological perspective of bilingual studies, whether second language (L2) learners can develop a consistent pattern of target language (i.e., L2) comprehension regarding the interplay of syntactic and semantic processing, especially when their first language (L1) and L2 are typologically distinct, remains to be determined. In this study, Chinese complex sentences with center-embedded relative clauses were generated. By utilizing the high-time-resolution technique of event-related potentials (ERPs), this study aimed to investigate the processing relationships between syntactic and semantic information during Chinese complex sentence reading in both Chinese L1 speakers and highly proficient L2 learners from South Korea.

Methods

Normal, semantically violated (SEM), and double-violated (containing both semantic and syntactic violations, SEM + SYN) conditions were set with regard to the nonadjacent dependencies of the Chinese complex sentence, and participants were required to judge whether the sentences they read were acceptable.

Results

The ERP results showed that sentences with "SEM + SYN" did not elicit early left anterior negativity (ELAN), a component assumed to signal initial syntactic processing, but evoked larger components in the N400 and P600 windows than those of the "SEM" condition, thus exhibiting a biphasic waveform pattern consistent for both groups and in line with previous studies using simpler Chinese syntactic structures. The only difference between the L1 and L2 groups was that L2 learners presented later latencies of the corresponding ERP components.

Discussion

Taken together, these results do not support the temporal and functional priorities of syntactic processing as identified in morphologically rich languages (e.g., German) and converge on the notion that even for Chinese complex sentence reading, syntactic and semantic processing are highly interactive. This is consistent across L1 speakers and high-proficiency L2 learners with typologically different language backgrounds.

Taxonomically-related Word Pairs Evoke both N400 and LPC at Long SOA in Turkish

Semantic priming in Turkish was examined in 36 right-handed healthy participants in a delayed lexical decision task via taxonomic relations using EEG. Prime-target relations included related- unrelated- and pseudo-words. Taxonomically related words at long stimulus onset asynchrony (SOA) were shown to modulate N400 and late positive component (LPC) amplitudes. N400 semantic priming effect in the time window of 300-500 ms was the largest for pseudo-words, intermediate for semantically-unrelated targets, and smallest for semantically-related targets as a reflection of lexical-semantic retrieval. This finding contributes to the ERP literature showing how remarkably universal the N400 brain potential is, with similar effects across languages and orthography. The ERP data also revealed different influences of related, unrelated, and pseudo-word conditions on the amplitude of the LPC. Attention scores and mean LPC amplitudes of related words in parietal region showed a moderate correlation, indicating LPC may be related to "relationship-detection process".

Online neurostimulation of Broca's area does not interfere with syntactic predictions: A combined TMS-EEG approach to basic linguistic combination

Categorical predictions have been proposed as the key mechanism supporting the fast pace of syntactic composition in language. Accordingly, grammar-based expectations are formed-e.g., the determiner "a" triggers the prediction for a noun-and facilitate the analysis of incoming syntactic information, which is then checked against a single or few other word categories. Previous functional neuroimaging studies point towards Broca's area in the left inferior frontal gyrus (IFG) as one fundamental cortical region involved in categorical prediction during incremental language processing. Causal evidence for this hypothesis is however still missing. In this study, we combined Electroencephalography (EEG) and Transcranial Magnetic Stimulation (TMS) to test whether Broca's area is functionally relevant in predictive mechanisms for language. We transiently perturbed Broca's area during the first word in a two-word construction, while simultaneously measuring the Event-Related Potential (ERP) correlates of syntactic composition. We reasoned that if Broca's area is involved in predictive mechanisms for syntax, disruptive TMS during the first word would mitigate the difference in the ERP responses for predicted and unpredicted categories in basic two-word constructions. Contrary to this hypothesis, perturbation of Broca's area at the predictive stage did not affect the ERP correlates of basic composition. The correlation strength between the electrical field induced by TMS and the ERP responses further confirmed this pattern. We discuss the present results considering an alternative account of the role of Broca's area in syntactic composition, namely the bottom-up integration of words into constituents, and of compensatory mechanisms within the language predictive network.

African American English speaking 2nd graders, verbal-s, and educational achievement: Event related potential and math study findings

A number of influential linguistic analyses hold that African American English (AAE) has no verbal-s, the-s that, for example, turns drink into drinks in more mainstream English varieties.On such accounts, sentences like Mary drinks coffee are ungrammatical in AAE. Previous behavioral studies suggest that in addition to being ungrammatical, AAE speaking children find these sentences cognitively demanding, and that their presence in mathematical reasoning tests can depress scores. Until now, however, no online sentence processing study nor investigation of neurophysiological markers has been done to support these findings. Aimed at addressing this gap in the literature, the auditory ERP experiment described herein revealed two different processes associated with AAE speaking 2nd graders listening to this type of sentence: a morphosyntactic structure building problem, reflected in a bilateral early anterior-central negativity; and an increase in working memory load, indicated by a bilateral late long-lasting anterior-central negativity. Study participants also took an orally administered test of math word problems. Consistent with previous findings, results showed they answered fewer questions correctly when those questions contained verbal-s than when they did not.

Modulation of Spectral Representation and Connectivity Patterns in Response to Visual Narrative in the Human Brain

The regional brain networks and the underlying neurophysiological mechanisms subserving the cognition of visual narrative in humans have largely been studied with non-invasive brain recording. In this study, we specifically investigated how regional and cross-regional cortical activities support visual narrative interpretation using intracranial stereotactic electroencephalograms recordings from thirteen human subjects (6 females, and 7 males). Widely distributed recording sites across the brain were sampled while subjects were explicitly instructed to observe images from fables presented in “sequential” order, and a set of images drawn from multiple fables presented in “scrambled” order. Broadband activity mainly within the frontal and temporal lobes were found to encode if a presented image is part of a visual narrative (sequential) or random image set (scrambled). Moreover, the temporal lobe exhibits strong activation in response to visual narratives while the frontal lobe is more engaged when contextually novel stimuli are presented. We also investigated the dynamics of interregional interactions between visual narratives and contextually novel series of images. Interestingly, the interregional connectivity is also altered between sequential and scrambled sequences. Together, these results suggest that both changes in regional neuronal activity and cross-regional interactions subserve visual narrative and contextual novelty processing.

Neurophysiological correlates of automatic integration of voice and gender information during grammatical processing

During verbal communication, interlocutors rely on both linguistic (e.g., words, syntax) and extralinguistic (e.g., voice quality) information. The neural mechanisms of extralinguistic information processing are particularly poorly understood. To address this, we used EEG and recorded event-related brain potentials while participants listened to Russian pronoun-verb phrases presented in either male or female voice. Crucially, we manipulated congruency between the grammatical gender signaled by the verbs' ending and the speakers' apparent gender. To focus on putative automatic integration of extralinguistic information into syntactic processing and avoid confounds arising from secondary top-down processes, we used passive non-attend auditory presentation with visual distraction and no stimulus-related task. Most expressed neural responses were found at both early (150 ms, ELAN-like) and late (400 ms, N400-like) phrase processing stages. Crucially, both of these brain responses exhibited sensitivity to extralinguistic information and were significantly enhanced for phrases whose voice and grammatical gender were incongruent, similar to what is known for ERPs effects related to overt grammatical violations. Our data suggest a high degree of automaticity in processing extralinguistic information during spoken language comprehension which indicates existence of a rapid automatic syntactic integration mechanism sensitive to both linguistic and extralinguistic information.

Children with autism spectrum disorder show atypical electroencephalographic response to processing contextual incongruencies

Children with autism spectrum disorder (ASD) experience difficulties with social communication, making it challenging to interpret contextual information that aids in accurately interpreting language. To investigate how the brain processes the contextual information and how this is different in ASD, we compared event-related potentials (ERPs) in response to processing visual and auditory congruent and incongruent information. Two groups of children participated in the study: 37 typically developing children and 15 children with ASD (age range = 6 to 12). We applied a language task involving auditory sentences describing congruent or incongruent images. We investigated two ERP components associated with language processing: the N400 and P600. Our results showed how children with ASD present significant differences in their neural responses in comparison with the TD group, even when their reaction times and correct trials are not significantly different from the TD group.

Processing Aspectual Agreement in a Language with Limited Morphological Inflection by Second Language Learners: An ERP Study of Mandarin Chinese

Previous studies on the neural cognitive mechanisms of aspectual processing in second language (L2) learners have focused on Indo-European languages with rich inflectional morphology. These languages have aspects which are equipped with inflected verb forms combined with auxiliary or modal verbs. Meanwhile, little attention has been paid to Mandarin Chinese, which has limited morphological inflection, and its aspect is equipped with aspectual particles (e.g., le, zhe, guo). The present study explores the neurocognitive mechanism of Mandarin Chinese aspect processing among two groups of late Mandarin Chinese proficient learners with Thai (with Mandarin Chinese-like aspect markers) and Indonesian (lack of Mandarin Chinese-like aspect markers) as their first language (L1). We measured event-related potentials (ERPs) time locked to the aspect marker le in two different conditions (the aspect violation sentences and the correct sentences). A triphasic ELAN-LAN-P600 effect was produced by the Mandarin Chinese native speakers. However, there was no ELAN and LAN in Indonesian native speakers and Thai native speakers, except a 300-500 ms negativity widely distributed in the right hemisphere and P600-like effect. This suggests that both groups of Mandarin Chinese learners cannot reach the same level as Mandarin Chinese native speakers to process Mandarin Chinese aspect information, probably due to the complex feature of Mandarin Chinese aspect maker, the participants' L2 proficiency and age of L2 acquisition.

Native Word Order Processing Is Not Uniform: An ERP Study of Verb-Second Word Order

Studies of native syntactic processing often target phrase structure violations that do not occur in natural production. In contrast, this study examines how variation in basic word order is processed, looking specifically at structures traditionally labelled as violations but that do occur naturally. We examined Swedish verb-second (V2) and verb-third (V3) word order processing in adult native Swedish speakers, manipulating sentence-initial adverbials (temporal idag ‘today’, spatial hemma ‘at home’ and sentential kanske ‘maybe’) in acceptability judgements, in simultaneously recorded event-related potentials (ERP) to visually presented sentences and in a written sentence completion task. An initial corpus study showed that the adverbials differ in frequency in fronted position (idag > kanske > hemma), and although all occur mainly with V2 word order, kanske occurs more frequently with V3 in natural production than both idag and hemma. The experimental results reflected these patterns such that V2 sentences were overall more frequently produced and were deemed more acceptable than V3 sentences. The ERP results consisted of a biphasic N400/P600 response to V3 word order that indicated effects of word retrieval and sentence reanalysis. We also found consistent effects of adverbials. As predicted, V3 was produced more frequently and judged as more acceptable in Kanske sentences than in sentences with the other two adverbials. The ERP analyses showed stronger effects for idag and hemma with V3, especially regarding the P600. The results suggest that the naturally occurring word order ‘violation’, V3 with kanske, is processed differently than V3 with other adverbials where the V2 norm is stronger. Moreover, these patterns are related to individuals’ own production patterns. Overall, the results suggest a more varied native word order processing than previously reported.

Developmental Language Disorder as Syntactic Prediction Impairment

We provide evidence that children with Developmental Language Disorder (DLD) are impaired in predictive syntactic processing. In the current study, children listened passively to auditorily-presented sentences, where the critical condition included an unexpected "filled gap" in the direct object position of the relative clause verb. A filled gap is illustrated by the underlined phrase in "The zebra that the hippo kissed the camel on the nose…", rather than the expected "the zebra that the hippo kissed [e] on the nose", where [e] denotes the gap. Brain responses to the filled gap were compared to a control condition using adverb-relative clauses with identical substrings: "The weekend that the hippo kissed the camel on the nose [e]…". Here, the same noun phrase is not unexpected because the adverb gap occurs later in the structure. We hypothesized that a filled gap would elicit a prediction error brain signal in the form of an early anterior negativity, as we have previously observed in adults. We found an early (bilateral) anterior negativity to the filled gap in a control group of children with Typical Development (TD), but the children with DLD exhibited no brain response to the filled gap during the same early time window. This suggests that children with DLD fail to predict that a relativized object should correspond to an empty position after the relative clause verb, suggesting an impairment in predictive processing. We discuss how this lack of a prediction error signal can interact with language acquisition and result in DLD.

Is semantic processing impaired near the hands?

In a sentence decision task, Davoli et al. (2010) found that the semantic analysis of sentences differed depending on whether the participants' hands were close to or far from the computer screen. According to the authors, the findings reflected an impoverishment of semantic processing near the hands. In the current study, we examine this explanation by asking whether hand position affects 1) other aspects of sentence processing, such as syntactic analysis, 2) semantic processing at the individual word level, and 3) performance in a picture naming task that requires access to meaning. In Experiment 1, participants judged the acceptability of sentences, half of which included semantic or syntactic violations. In Experiment 2, only semantically acceptable or nonacceptable sentences were presented. In Experiment 3, participants performed a go/nogo semantic categorization task on individual words. In Experiment 4, participants performed a picture naming task. Participants performed these tasks both with their hands near to and far from the computer screen. Regardless of the task, we found no evidence of impoverished semantic processing near the hands.

Processing Aspectual Agreement in an Inflexionless Language: An ERP Study of Mandarin Chinese

This is a study of the collocation of Chinese verbs with different lexical aspects and aspect markers. Using event-related potentials (ERPs), we explored the processing of aspect violation sentences. In the experiment, we combined verbs of various lexical aspect types with the progressive aspect marker zhe, and the combination of the achievement verbs and the progressive aspect marker zhe constituted the sentence's aspect violation. The participants needed to judge whether a sentence was correct after it was presented. Finally, we observed and analyzed the components of ERPs. The results suggest that when the collocation of aspect markers and lexical aspect is ungrammatical, the N400-like and P600 are elicited on aspect markers, while the late AN is elicited by the word after the aspect marker. P600 and N400-like show that the collocation of Chinese verbs with various lexical aspects and aspect markers involve not only syntactic processing, but also the semantic processing; and the late AN may have been due to the syntax revision and the conclusion at the end of sentences.

Common ERP responses to narrative incoherence in sentence and picture pair comprehension

Understanding the neural processes underlying the comprehension of visual images and sentences remains a major open challenge in cognitive neuroscience. We previously demonstrated with fMRI and DTI that comprehension of visual images and sentences describing human activities recruits a common extended parietal-temporal-frontal semantic system. The current research tests the hypothesis that this common semantic system will display similar ERP profiles during processing in these two modalities, providing further support for the common comprehension system. We recorded EEG from naïve subjects as they saw simple narratives made up of a first visual image depicting a human event, followed by a second image that was either a sequentially coherent narrative follow-up, or not, of the first. Incoherent second stimuli depict the same agents but shifted into a different situation. In separate blocks of trials the same protocol was presented using narrative sentence stimuli. Part of the novelty is the comparison of sentence and visual narrative responses. ERPs revealed common neural profiles for narrative processing across image and sentence modalities in the form of early and late central and frontal positivities in response to narrative incoherence. There was an additional posterior positivity only for sentences in a very late window. These results are discussed in the context of ERP signatures of narrative processing and meaning, and a current model of narrative comprehension.

Unexpected words or unexpected languages? Two ERP effects of code-switching in naturalistic discourse

Bilingual speakers often switch between languages in conversation without any advance notice. Psycholinguistic research has found that these language shifts (or code-switches) can be costly for comprehenders in certain situations. The present study explores the nature of these costs by comparing code-switches to other types of unexpected linguistic material. To do this, we used a novel EEG paradigm, the Storytime task, in which we record readings of natural texts, and then experimentally manipulate their properties by splicing in words. In this study, we manipulated the language of our target words (English, Spanish) and their fit with the preceding context (strong-fit, weak-fit). If code-switching incurs a unique cost beyond that incurred by an unexpected word, then we should see an additive pattern in our ERP indices. If an effect is driven by lexical expectation alone, then there should be a non-additive interaction such that all unexpected forms incur a similar cost. We found three effects: a general prediction effect (a non-additive N400), a post-lexical recognition of the switch in languages (an LPC for code-switched words), and a prolonged integration difficulty associated with weak-fitting words regardless of language (a sustained negativity). We interpret these findings as suggesting that the processing difficulties experienced by bilinguals can largely be understood within more general frameworks for understanding language comprehension. Our findings are consistent with the broader literature demonstrating that bilinguals do not have two wholly separate language systems but rather a single language system capable of using two coding systems.

Prediction of Second Language Proficiency Based on Electroencephalographic Signals Measured While Listening to Natural Speech

This study had two goals: to clarify the relationship between electroencephalographic (EEG) features estimated while non-native speakers listened to a second language (L2) and their proficiency in L2 determined by a conventional paper test and to provide a predictive model for L2 proficiency based on EEG features. We measured EEG signals from 205 native Japanese speakers, who varied widely in English proficiency while they listened to natural speech in English. Following the EEG measurement, they completed a conventional English listening test for Japanese speakers. We estimated multivariate temporal response functions separately for word class, speech rate, word position, and parts of speech. We found significant negative correlations between listening score and 17 EEG features, which included peak latency of early components (corresponding to N1 and P2) for both open and closed class words and peak latency and amplitude of a late component (corresponding to N400) for open class words. On the basis of the EEG features, we generated a predictive model for Japanese speakers’ English listening proficiency. The correlation coefficient between the true and predicted listening scores was 0.51. Our results suggest that L2 or foreign language ability can be assessed using neural signatures measured while listening to natural speech, without the need of a conventional paper test.

Lexical-semantic processing costs are not inherent to intra-sentential code-switching: The role of switching habits

A bank of past electrophysiological research suggests that code-switching (i.e., the use of multiple languages during a single conversational event) is characterized by increased processing costs. However, the majority of these studies tested bilinguals who did not necessarily code-switch on a regular basis. In order to investigate the impact of switching habits on language processing, two groups of French-English bilinguals (habitual code-switchers and non-habitual code-switchers) read sentences containing unilingual and switched determiner phrases while their EEG was recorded. The findings indicate important processing differences between the groups. Notably, switched trials (particularly those with a French determiner) generated an increased negativity from 300 to 500 ms for non-habitual code-switchers, but not for habitual code-switchers. The veritable impact of switching habits were also supported by a correlation analysis. This finding suggests that code-switching is not inherently effortful to process, but rather, that comprehension costs are related to idiosyncratic factors, such as the extent of prior exposure to code-switched input. An LPC was also observed from 500 to 900 for habitual code-switchers; for non-habitual switchers, this effect may have been attenuated by the long-lasting N400 effect. We advance several possible interpretations for the late positivity. Altogether, the results from the current study support the claim that the code-switching cost observed in previous literature may be linked to the bilingual samples tested in those studies: habitual code-switchers may not find switches effortful to process, but not all bilinguals code-switch. We highlight the importance of experience and individual differences in the study of bilingual cognition.

A starring role for inference in the neurocognition of visual narratives

Research in verbal and visual narratives has often emphasized backward-looking inferences, where absent information is subsequently inferred. However, comics use conventions like star-shaped “action stars” where a reader knows events are undepicted at that moment, rather than omitted entirely. We contrasted the event-related brain potentials (ERPs) to visual narratives depicting an explicit event, an action star, or a “noise” panel of scrambled lines. Both action stars and noise panels evoked large N400s compared to explicit-events (300–500 ms), but action stars and noise panels then differed in their later effects (500–900 ms). Action stars elicited sustained negativities and P600s, which could indicate further interpretive processes and integration of meaning into a mental model, while noise panels evoked late frontal positivities possibly indexing that they were improbable narrative units. Nevertheless, panels following action stars and noise panels both evoked late sustained negativities, implying further inferential processing. Inference in visual narratives thus uses cascading mechanisms resembling those in language processing that differ based on the inferential techniques.

Head errors of syntactic dependency increase neuromagnetic mismatch intensities

Mismatch-related brain activation in healthy individuals is an important area of neural investigation. Previously, we evaluated sentence-level syntactic dependencies, composed of a head and a dependent between two syntactically related words in head-initial English structures. We demonstrated that prominent mismatch effects were induced by within-category dependent errors when semantic interpretation was preserved. However, the following issues were not addressed: (1) whether head errors of syntactic dependency in head-final structures would elicit large mismatch field (MMF) intensities, and (2) whether an MMF effect of syntactic errors would be seen in the left superior temporal cortex alone. In this study, auditory MMFs were obtained by magnetocephalography (MEG) from healthy Japanese adults (n = 8) who were subjected to a passive auditory oddball paradigm with syntactically legal or illegal utterances and single words in Japanese. The results demonstrate that the source waveforms had significantly higher MMF cortical activation in response to the head error, which involved altered polarity of the predicate. This resulted in a syntactically incorrect and semantically incomprehensible expression, when compared to the syntactically correct expression and the non-structural lexical item. This mismatch effect, with a peak latency of 164 ms, was confined to the anterior region of the left superior temporal cortex. The current results clearly indicate that the representation of syntactic dependency is stored in long-term memory and tends to be activated in automatic auditory processing.

Reduced right-hemisphere ERP P600 grammaticality effect is associated with greater right-hemisphere inhibition: Evidence from right-handers with familial sinistrality

The present study examined the hypothesis that left hemisphere (LH) equivalent language capabilities in the right hemisphere (RH) are inhibited in neurologically intact individuals by testing healthy young right-handers with a history of familial sinistrality (FS+, i.e. with at least one left handed biological relative), a population documented to show greater variability for RH language processing. Event-Related Potential (ERP) and split visual field presentation techniques were combined to assess LH- and RH- biased responses to syntactic category violations. In addition, a bilateral flanker task was used to measure inter-hemispheric inhibition ability in the same set of participants. Replicating prior findings, in addition to the LH-biased P600 grammaticality effect previously seen for right-handers in general, a fair amount, though not all, of FS + right-handers showed RH-biased P600 responses, leading to a reliable RH P600 grammaticality effect at the group level. Capitalizing on the variability of RH P600 responses, our results further revealed that reduced RH-biased P600 effects were reliably correlated with more effective RH inhibition (indexed by smaller reaction time differences between incongruent and neutral flankers presented to the RH via the left visual field). These results corroborated previous findings that the RH is capable of processing syntactic information in a manner qualitatively similar to that in the LH and further demonstrated that LH-equivalent processing in the RH as indexed by the P600 responses is modulated by RH inhibition, contributing to inter-individual variability in syntactic lateralization.

Verbing nouns and nouning verbs: Using a balanced design provides ERP evidence against "syntax-first" approaches to sentence processing

In this event-related potential (ERP) study we reevaluate syntax-first approaches to sentence processing by implementing a novel paradigm in French that includes correct sentences, pure syntactic category violations, lexical-semantic anomalies, and combined anomalies. Our balanced design systematically controlled for target word (noun vs. verb) and the context immediately preceding it. Group results from 36 native speakers of Quebec French revealed that, up to 300 ms, ERPs elicited by syntactic category violations were comparable with ERP responses to correct sentences, showing that there is no early activation reflecting syntactic category identification. Instead, in response to all anomalous conditions, we observed an N400 followed by a P600. Combined anomalies yielded additive effects of syntactic category and lexical-semantic anomalies on the N400, and a large P600 effect similar to the one observed in the pure syntactic condition. These results provide strong evidence against the hypothesis that (i) syntactic categories are processed first, and (ii) that syntactic category errors "block" lexical-semantic processing. Further, the N400 effect in response to pure syntactic category violations reflects a mismatch detection between a predicted word-stem and the actual target. This mechanism takes place simultaneously (and potentially in parallel) with lexical-semantic processing. In contrast, an interaction of syntax and semantics for the P600 reveals that the same neurocognitive resources are recruited for syntactic and semantic integration, both promoted by the implementation of an acceptability judgement task in our design. Additional analyses of individual data complemented these observations: during sentence processing, participants did not rely on one single cognitive mechanism reflected by either the N400 or the P600 effect but on both, suggesting that the biphasic N400-P600 ERP wave can indeed be considered to be an index of phrase-structure violations in most individuals, at least if they are realized on content words.

Functional Linking Between Negative and Positive ERPs for Syntactic Processing in Japanese: Mutual Enhancement, Syntactic Prediction, and Working Memory Constraints

This study attempts to detect the differences in event-related potentials (ERPs) associated with two syntactic processes: the syntactic integration of discontinuous dependency and the detection of a violation of the syntactic island constraint. We recorded the electroencephalogram elicited by complex sentences in Japanese that included a dependency between a quantifier and its head noun, in which we changed the word order of the two words to manipulate the presence and absence of a syntactic integration and a syntactic island violation while keeping the lexical items and construction unchanged. We found significant negative and positive deflections for the syntactic integration only when a quantifier preceded its head noun. We also observed significant negative and positive deflections for the syntactic island violation, for which the negativity was more salient when a quantifier preceded its head noun. This study is the first to report a late positive ERP for a violation of the syntactic island constraints in Japanese, and the results showed that the ERP elicited by syntactic integration and that by syntactic island violation were different in terms of their latency, topography, and duration. More importantly, the ERPs elicited by the two syntactic processes were biphasic, and the amplitudes of the negative ERP and of the positive ERP were positively correlated. This positive correlation could be a characteristic of syntactic processing because it contrasted with the negative correlation reported for the ERP elicited by semantic anomalies in English. Furthermore, the amplitude of the ERP for syntactic integration was negatively correlated with the individual capacity of working memory (WM). That is, a reader with greater WM capacity showed smaller negativity and positivity for the syntactic integration, whereas the amplitude for the syntactic island violation showed no significant correlation with the individual capacity of WM. Our results suggested that linguistic ERPs functionally interacted with each other and that the ERP involving the retention and the retrieval of a distant word could be constrained by the individual differences in WM capacity. We discuss the possible reasons for the contrast between English and Japanese on the basis of the cross-linguistic differences in the two languages.

Processing syntax: perspectives on language specificity

Objective: The past few decades of research in language processing provides empirical data on the dimensions of the brain-language relationship. The methodologies used to study language processing have evidenced an immense advancement over the years, tracking real-time processing events with millisecond precision. Event-related potentials is one such method which assists to visualize the neural mechanisms that underlie language processing. Different electrophysiological components mark different components of language depending on their structural and functional aspects. Since research on language processing is expanding its boundaries, the neural mechanisms for processing syntax components have been the focus of recent investigations across the languages of the world. The present review article aims to discuss the findings of studies on syntax processing besides highlighting the functional significance of P600, the electrophysiological marker of syntax processing.Methods: Electronic databases such as Pubmed, Science Direct, Research gate, PLOS, Directory of Open Access Journals were searched for relavant articles. The review process followed PRISMA guidelines for screening, identification, and selection of articles.Results: The results of the review elucidate the need for evaluating the finer details of syntax, including morpho-syntax concerning specific language structures. Studies across the languages of the world exemplify the uniqueness in the structure of different languages that may provide varied perspectives on the universality in syntax processing.Conclusion: The present review contributes a new dimension towards understanding the nature of syntax processing with respect to language specificity.

Attention Modulates Electrophysiological Responses to Simultaneous Music and Language Syntax Processing

Music and language are hypothesized to engage the same neural resources, particularly at the level of syntax processing. Recent reports suggest that attention modulates the shared processing of music and language, but the time-course of the effects of attention on music and language syntax processing are yet unclear. In this EEG study we vary top-down attention to language and music, while manipulating the syntactic structure of simultaneously presented musical chord progressions and garden-path sentences in a modified rapid serial visual presentation paradigm. The Early Right Anterior Negativity (ERAN) was observed in response to both attended and unattended musical syntax violations. In contrast, an N400 was only observed in response to attended linguistic syntax violations, and a P3/P600 only in response to attended musical syntax violations. Results suggest that early processing of musical syntax, as indexed by the ERAN, is relatively automatic; however, top-down allocation of attention changes the processing of syntax in both music and language at later stages of cognitive processing.

MVPA Analysis of Intertrial Phase Coherence of Neuromagnetic Responses to Words Reliably Classifies Multiple Levels of Language Processing in the Brain

Neural processing of language is still among the most poorly understood functions of the human brain, whereas a need to objectively assess the neurocognitive status of the language function in a participant-friendly and noninvasive fashion arises in various situations. Here, we propose a solution for this based on a short task-free recording of MEG responses to a set of spoken linguistic contrasts. We used spoken stimuli that diverged lexically (words/pseudowords), semantically (action-related/abstract), or morphosyntactically (grammatically correct/ungrammatical). Based on beamformer source reconstruction we investigated intertrial phase coherence (ITPC) in five canonical bands (α, β, and low, medium, and high γ) using multivariate pattern analysis (MVPA). Using this approach, we could successfully classify brain responses to meaningful words from meaningless pseudowords, correct from incorrect syntax, as well as semantic differences. The best classification results indicated distributed patterns of activity dominated by core temporofrontal language circuits and complemented by other areas. They varied between the different neurolinguistic properties across frequency bands, with lexical processes classified predominantly by broad γ, semantic distinctions by α and β, and syntax by low γ feature patterns. Crucially, all types of processing commenced in a near-parallel fashion from ∼100 ms after the auditory information allowed for disambiguating the spoken input. This shows that individual neurolinguistic processes take place simultaneously and involve overlapping yet distinct neuronal networks that operate at different frequency bands. This brings further hope that brain imaging can be used to assess neurolinguistic processes objectively and noninvasively in a range of populations.

Neural correlates of semantic and syntactic processing in German Sign Language

The study of deaf and hearing native users of signed languages can offer unique insights into how biological constraints and environmental input interact to shape the neural bases of language processing. Here, we use functional magnetic resonance imaging (fMRI) to address two questions: (1) Do semantic and syntactic processing in a signed language rely on anatomically and functionally distinct neural substrates as it has been shown for spoken languages? and (2) Does hearing status affect the neural correlates of these two types of linguistic processing? Deaf and hearing native signers performed a sentence judgement task on German Sign Language (Deutsche Gebärdensprache: DGS) sentences which were correct or contained either syntactic or semantic violations. We hypothesized that processing of semantic and syntactic violations in DGS relies on distinct neural substrates as it has been shown for spoken languages. Moreover, we hypothesized that effects of hearing status are observed within auditory regions, as deaf native signers have been shown to activate auditory areas to a greater extent than hearing native signers when processing a signed language. Semantic processing activated low-level visual areas and the left inferior frontal gyrus (IFG), suggesting both modality-dependent and independent processing mechanisms. Syntactic processing elicited increased activation in the right supramarginal gyrus (SMG). Moreover, psychophysiological interaction (PPI) analyses revealed a cluster in left middle occipital regions showing increased functional coupling with the right SMG during syntactic relative to semantic processing, possibly indicating spatial processing mechanisms that are specific to signed syntax. Effects of hearing status were observed in the right superior temporal cortex (STC): deaf but not hearing native signers showed greater activation for semantic violations than for syntactic violations in this region. Taken together, the present findings suggest that the neural correlates of language processing are partly determined by biological constraints, but that they may additionally be influenced by the unique processing demands of the language modality and different sensory experiences.

Art looks different - Semantic and syntactic processing of paintings and associated neurophysiological brain responses

The concept of semantics (meaning) and syntax (structure) seems to be an integral way of how humans perceive and order their environment. Processing natural scenes with semantic or syntactic inconsistencies evokes distinct Event-Related Potentials, ERPs, in the N300/400 and P600, respectively (Vo & Wolfe, 2013). Artworks, however, can by definition use violations of natural relationships as a means of style, especially in surrealist art. To test whether inconsistencies are processed differently in artworks, we presented participants with surrealist paintings containing semantic or syntactic inconsistencies, edited versions without inconsistencies, and as control real photographic versions of each painting. Photographs elicited more pronounced negative ERP amplitudes than paintings in all time windows, N300, N400 and P600. However, the lack of an interaction between image type and inconsistency type indicates that all presented images were processed as artworks, probably due to context effects. The ERPs were largely opposite to those reported previously with everyday life pictures, with syntactic inconsistencies driving the earlier components and eliciting higher amplitudes than semantic ones in the N400, and semantic inconsistencies eliciting a higher amplitude in the P600. We conclude that viewing artworks includes a specific processing mode, entailing syntactic and semantic expectations different from those in natural scenes.

Your Brain on Comics: A Cognitive Model of Visual Narrative Comprehension

The past decade has seen a rapid growth of cognitive and brain research focused on visual narratives like comics and picture stories. This paper will summarize and integrate this emerging literature into the Parallel Interfacing Narrative‐Semantics Model (PINS Model)—a theory of sequential image processing characterized by an interaction between two representational levels: semantics and narrative structure. Ongoing semantic processes build meaning into an evolving mental model of a visual discourse. Updating of spatial, referential, and event information then incurs costs when they are discontinuous with the growing context. In parallel, a narrative structure organizes semantic information into coherent sequences by assigning images to categorical roles, which are then embedded within a hierarchic constituent structure. Narrative constructional schemas allow for specific predictions of structural sequencing, independent of semantics. Together, these interacting levels of representation engage in an iterative process of retrieval of semantic and narrative information, prediction of upcoming information based on those assessments, and subsequent updating based on discontinuity. These core mechanisms are argued to be domain‐general—spanning across expressive systems—as suggested by similar electrophysiological brain responses (N400, P600, anterior negativities) generated in response to manipulation of sequential images, music, and language. Such similarities between visual narratives and other domains thus pose fundamental questions for the linguistic and cognitive sciences.

Visual narratives like comics involve a range of complex cognitive operations in order to be understood. The Parallel Interfacing Narrative‐Semantics (PINS) Model integrates an emerging literature showing that comprehension of wordless image sequences balances two representational levels of semantic and narrative structure. The neurocognitive mechanisms that guide these processes are argued to overlap with other domains, such as language and music.

The P3b and P600(s): Positive contributions to language comprehension

Since its discovery in the 1960s, the P300 has been contributing both directly and indirectly to language research. Perhaps most notably, it has been suggested that the P600, an ERP component that was first characterized in the context of syntactic processing, could be a variant of the P3b subcomponent of the P300. Here, we review studies on both sides of the debate. We also review the "semantic P600," a positivity with a similar time course and distribution to the P600 seen for syntactic manipulations but that is obtained in response to some types of semantic anomalies. Because most current theories of the P600 try to account for both the syntactic and the semantic variant, linking the syntactic P600 to the P3b might also imply a similar link for the semantic P600. However, we describe emerging research in our lab that casts doubt on the idea that the syntactic P600 and the semantic P600 are the same effect. We argue that grouping ERP responses primarily by domain (language vs. nonlanguage) is likely to be misleading and suggest alternative ways of determining whether ERP effects reflect similar or different processing mechanisms.

Toward a Neurobiologically Plausible Model of Language-Related, Negative Event-Related Potentials

Language-related event-related potential (ERP) components such as the N400 have traditionally been associated with linguistic or cognitive functional interpretations. By contrast, it has been considerably more difficult to relate these components to neurobiologically grounded accounts of language. Here, we propose a theoretical framework based on a predictive coding architecture, within which negative language-related ERP components such as the N400 can be accounted for in a neurobiologically plausible manner. Specifically, we posit that the amplitude of negative language-related ERP components reflects precision-weighted prediction error signals, i.e., prediction errors weighted by the relevance of the information source leading to the error. From this perspective, precision has a direct link to cue validity in a particular language and, thereby, to relevance of individual linguistic features for internal model updating. We view components such as the N400 and LAN as members of a family with similar functional characteristics and suggest that latency and topography differences between these components reflect the locus of prediction errors and model updating within a hierarchically organized cortical predictive coding architecture. This account has the potential to unify findings from the full range of the N400 literature, including word-level, sentence-, and discourse-level results as well as cross-linguistic differences.

Hierarchical structure guides rapid linguistic predictions during naturalistic listening

The grammar, or syntax, of human language is typically understood in terms of abstract hierarchical structures. However, theories of language processing that emphasize sequential information, not hierarchy, successfully model diverse phenomena. Recent work probing brain signals has shown mixed evidence for hierarchical information in some tasks. We ask whether sequential or hierarchical information guides the expectations that a human listener forms about a word's part-of-speech when simply listening to every-day language. We compare the predictions of three computational models against electroencephalography signals recorded from human participants who listen passively to an audiobook story. We find that predictions based on hierarchical structure correlate with the human brain response above-and-beyond predictions based only on sequential information. This establishes a link between hierarchical linguistic structure and neural signals that generalizes across the range of syntactic structures found in every-day language.

Subliminal syntactic priming

Subliminally presented words have been shown to cause priming at orthographic and semantic levels. Here, we investigate whether subliminal priming can also occur at the syntactic level, and use such priming as a tool to probe the architecture for processing the syntactic features of written words. We studied the impact of masked and unmasked written word primes on response times to a subsequent visible target that shared or did not share syntactic features such as grammatical category and grammatical number. Methodological precautions included the use of distinct lists of subliminal primes that were never consciously seen, and the verification that participants were at chance in a prime-classification task. Across five experiments, subliminal priming could be induced by the repetition of the same grammatical category (e.g. a noun followed by another noun), by the transition between two categories (e.g. a determiner followed by a noun), or by the repetition of a single grammatical feature, even if syntax is violated (e.g. "they lemons", where the expression is ungrammatical but the plural feature is repeated). The orthographic endings of prime words also provided unconscious cues to their grammatical category. Those results indicate the existence of a representation of abstract syntactic features, shared between several categories of words, and which is quickly and unconsciously extracted from a flashed visual word.

Backward-Looking Sentence Processing in Typically Disfluent versus Stuttered Speech: ERP Evidence

The aim was to determine how backward-looking sentence processing is affected by typically-disfluent versus stuttered speech. Two listener groups heard Garden Path (GP) and control sentences. GP sentences contained no disfluency, a silent pause, or a filled pause before the disambiguating verb. For one group, the sentence preambles additionally contained stuttering-like disfluencies. Comprehension accuracy, event-related potentials (ERPs) time-locked to disambiguating verbs, and perceptual speaker ratings, were compared between groups. The With Stuttering group perceived the speaker as less competent but had better comprehension accuracy for GP sentences. ERPs to disambiguating verbs in GP sentences included a P600 component, indexing backward-looking sentence processing, but only for the No Stuttering group. Other ERP components, elicited to GP sentences with silent/filled pauses, did not differ between groups. Results suggest that listeners abandon prior expectations when processing sentences containing stuttering-like disfluencies, possibly because they lack a speaker model defined by the presence of stuttering.

Is There a Processing Preference for Object Relative Clauses in Chinese? Evidence From ERPs

A consistent finding across head-initial languages, such as English, is that subject relative clauses (SRCs) are easier to comprehend than object relative clauses (ORCs). However, several studies in Mandarin Chinese, a head-final language, revealed the opposite pattern, which might be modulated by working memory (WM) as suggested by recent results from self-paced reading performance. In the present study, event-related potentials (ERPs) were recorded when participants with high and low WM spans (measured by forward digit span and operation span tests) read Chinese ORCs and SRCs. The results revealed an N400-P600 complex elicited by ORCs on the relativizer, whose magnitude was modulated by the WM span. On the other hand, a P600 effect was elicited by SRCs on the head noun, whose magnitude was not affected by the WM span. These findings paint a complex picture of relative clause processing in Chinese such that opposing factors involving structural ambiguities and integration of filler-gap dependencies influence processing dynamics in Chinese relative clauses.