The Electrophysiology of Basic Phrase Building

Decoding the Neural Dynamics of Headed Syntactic Structure Building

The brain builds hierarchical phrases during language comprehension; however, the details and dynamics of the phrase-building process remain underspecified. This study directly probes whether the neural code of verb phrases involves reactivating the syntactic property of a key subcomponent (the "head" verb). To this end, we train a part-of-speech sliding-window verb/adverb decoder on EEG signals recorded while 30 participants read sentences in a controlled experiment. The decoder reaches above-chance performance that is spatiotemporally consistent and generalizes to unseen data across sentence positions. Applying the decoder to held-out data yields predicted activation levels for the verbal "head" of a verb phrase at a distant nonhead word (adverb); the critical adverb appeared either at the end of a verb phrase or at a sequentially and lexically matched position with no verb phrase boundary. There is stronger verb activation beginning at ∼600 milliseconds at the critical adverb when it appears at a verb phrase boundary; this effect is not modulated by the strength of conceptual association nor does it reflect word predictability. Time-locked analyses additionally reveal a negativity waveform component and increased beta-delta inter-trial phase coherence, both previously linked to linguistic composition, in a similar time window. With a novel application of neural decoding, our findings delineate the dynamics by which the brain encodes phrasal representations by, in part, reactivating the representation of key subcomponents. We thus establish a link between cognitive accounts of phrasal representations and electrophysiological dynamics.

Use of context in updating affective representations of words in older adults

Older adults (OAs) often prioritize positive over negative information during word processing, termed as positivity bias. However, it is unclear how OAs update the affective representation of a word in contexts. The present study examined whether age-related positivity bias influences the update of the affective representation of a word in different emotional contexts. In Experiment 1 (web-based), younger and older participants read positive and negative target words in positive and negative contexts and rated the valence of the target words. Negative contexts biased the ratings more than positive ones, reflecting a negativity bias during offline valence evaluation in both age groups. In Experiment 2 (EEG), another group of participants read positive and negative target words in positive and negative contexts first, and then the same target words again, and made valence judgment on the target words. OAs showed a larger P2 (180-300 ms) difference before and after contexts for positive target words than younger adults (YAs). This suggests OAs' early attention to positive features of words in contexts. YAs showed a larger late positive complex (LPC) difference for target words before and after negative contexts than before and after positive contexts, while older adults showed comparable LPC effects across all the conditions. This suggests that YAs use negative contexts to update the affective representation of a word, whereas OAs do so in both positive and negative contexts. Our findings supported a reduced negativity bias in OAs in using (emotional) contexts to update the affective neural representation of a word.

Lexical cues and discourse integration: An ERP study of the N400 and P600 components

In a sentence reading ERP study in Swedish we investigated the roles of the N400 and P600 components. By manipulating ease of lexical retrieval and discourse integration of the critical words in four conditions (contextually primed/non-primed and degree of contextual fit), we explored these components from a sentence processing perspective. The results indicate that the N400 indexes lexical retrieval and access of stored conceptual knowledge, whereas the P600 component indexes pragmatic processes, such as integration of a word into the discourse context, or the information structural status of the word. The results support single-stream models of sentence processing where lexical retrieval and integration do not take place in parallel, as in multi-stream models.

Electrophysiological correlates of basic semantic composition in people with aphasia

The neuroanatomical correlates of basic semantic composition have been investigated in previous neuroimaging and lesion studies, but research on the electrophysiology of the involved processes is scarce. A large literature on sentence-level event-related potentials (ERPs) during semantic processing has identified at least two relevant components - the N400 and the P600. Other studies demonstrated that these components are reduced and/or delayed in people with aphasia (PWA). However, it remains to be shown if these findings generalize beyond the sentence level. Specifically, it is an open question if an alteration in ERP responses in PWA can also be observed during basic semantic composition, providing a potential future diagnostic tool. The present study aimed to elucidate the electrophysiological dynamics of basic semantic composition in a group of post-stroke PWA. We included 20 PWA and 20 age-matched controls (mean age 58 years) and measured ERP responses while they performed a plausibility judgment task on two-word phrases that were either meaningful ("anxious horse"), anomalous ("anxious wood") or had the noun replaced by a pseudoword ("anxious gufel"). The N400 effect for anomalous versus meaningful phrases was similar in both groups. In contrast, unlike the control group, PWA did not show an N400 effect between pseudoword and meaningful phrases. Moreover, both groups exhibited a parietal P600 effect towards pseudoword phrases, while PWA showed an additional P600 over frontal electrodes. Finally, PWA showed an inverse correlation between the magnitude of the N400 and P600 effects: PWA exhibiting no or even reversed N400 effects towards anomalous and pseudoword phrases showed a stronger P600 effect. These results may reflect a compensatory mechanism which allows PWA to arrive at the correct interpretation of the phrase. When compositional processing capacities are impaired in the early N400 time-window, PWA may make use of a more elaborate re-analysis process reflected in the P600.

Exploring the neurobiology of Merge at a basic level: insights from a novel artificial grammar paradigm

Introduction

Human language allows us to generate an infinite number of linguistic expressions. It's proposed that this competence is based on a binary syntactic operation, Merge, combining two elements to form a new constituent. An increasing number of recent studies have shifted from complex syntactic structures to two-word constructions to investigate the neural representation of this operation at the most basic level.

Methods

This fMRI study aimed to develop a highly flexible artificial grammar paradigm for testing the neurobiology of human syntax at a basic level. During scanning, participants had to apply abstract syntactic rules to assess whether a given two-word artificial phrase could be further merged with a third word. To control for lower-level template-matching and working memory strategies, an additional non-mergeable word-list task was set up.

Results

Behavioral data indicated that participants complied with the experiment. Whole brain and region of interest (ROI) analyses were performed under the contrast of "structure > word-list." Whole brain analysis confirmed significant involvement of the posterior inferior frontal gyrus [pIFG, corresponding to Brodmann area (BA) 44]. Furthermore, both the signal intensity in Broca's area and the behavioral performance showed significant correlations with natural language performance in the same participants. ROI analysis within the language atlas and anatomically defined Broca's area revealed that only the pIFG was reliably activated.

Discussion

Taken together, these results support the notion that Broca's area, particularly BA 44, works as a combinatorial engine where words are merged together according to syntactic information. Furthermore, this study suggests that the present artificial grammar may serve as promising material for investigating the neurobiological basis of syntax, fostering future cross-species studies.

Hierarchy, Not Lexical Regularity, Modulates Low-Frequency Neural Synchrony During Language Comprehension

Neural responses appear to synchronize with sentence structure. However, researchers have debated whether this response in the delta band (0.5-3 Hz) really reflects hierarchical information or simply lexical regularities. Computational simulations in which sentences are represented simply as sequences of high-dimensional numeric vectors that encode lexical information seem to give rise to power spectra similar to those observed for sentence synchronization, suggesting that sentence-level cortical tracking findings may reflect sequential lexical or part-of-speech information, and not necessarily hierarchical syntactic information. Using electroencephalography (EEG) data and the frequency-tagging paradigm, we develop a novel experimental condition to tease apart the predictions of the lexical and the hierarchical accounts of the attested low-frequency synchronization. Under a lexical model, synchronization should be observed even when words are reversed within their phrases (e.g., "sheep white grass eat" instead of "white sheep eat grass"), because the same lexical items are preserved at the same regular intervals. Critically, such stimuli are not syntactically well-formed; thus a hierarchical model does not predict synchronization of phrase- and sentence-level structure in the reversed phrase condition. Computational simulations confirm these diverging predictions. EEG data from N = 31 native speakers of Mandarin show robust delta synchronization to syntactically well-formed isochronous speech. Importantly, no such pattern is observed for reversed phrases, consistent with the hierarchical, but not the lexical, accounts.

Minimal Phrase Composition Revealed by Intracranial Recordings

The ability to comprehend phrases is an essential integrative property of the brain. Here, we evaluate the neural processes that enable the transition from single-word processing to a minimal compositional scheme. Previous research has reported conflicting timing effects of composition, and disagreement persists with respect to inferior frontal and posterior temporal contributions. To address these issues, 19 patients (10 male, 9 female) implanted with penetrating depth or surface subdural intracranial electrodes, heard auditory recordings of adjective-noun, pseudoword-noun, and adjective-pseudoword phrases and judged whether the phrase matched a picture. Stimulus-dependent alterations in broadband gamma activity, low-frequency power, and phase-locking values across the language-dominant left hemisphere were derived. This revealed a mosaic located on the lower bank of the posterior superior temporal sulcus (pSTS), in which closely neighboring cortical sites displayed exclusive sensitivity to either lexicality or phrase structure, but not both. Distinct timings were found for effects of phrase composition (210-300 ms) and pseudoword processing (∼300-700 ms), and these were localized to neighboring electrodes in pSTS. The pars triangularis and temporal pole encoded anticipation of composition in broadband low frequencies, and both regions exhibited greater functional connectivity with pSTS during phrase composition. Our results suggest that the pSTS is a highly specialized region composed of sparsely interwoven heterogeneous constituents that encodes both lower and higher level linguistic features. This hub in pSTS for minimal phrase processing may form the neural basis for the human-specific computational capacity for forming hierarchically organized linguistic structures.SIGNIFICANCE STATEMENT Linguists have claimed that the integration of multiple words into a phrase demands a computational procedure distinct from single-word processing. Here, we provide intracranial recordings from a large patient cohort, with high spatiotemporal resolution, to track the cortical dynamics of phrase composition. Epileptic patients volunteered to participate in a task in which they listened to phrases (red boat), word-pseudoword or pseudoword-word pairs (e.g., red fulg). At the onset of the second word in phrases, greater broadband high gamma activity was found in posterior superior temporal sulcus in electrodes that exclusively indexed phrasal meaning and not lexical meaning. These results provide direct, high-resolution signatures of minimal phrase composition in humans, a potentially species-specific computational capacity.

Cursed Concepts: New insights on combinatorial processing from ERP correlates of swearing in context

Expressives (damn) convey speaker attitude and when used in context (Tom lost the damn dog) can be flexibly applied locally to the noun (dog) or globally to the whole sentence (the situation). We used ERPs to explore brain responses to expressives in sentences. Participants read expressive, descriptive, and pseudoword adjectives followed by nouns in sentences (The damn/black/flerg dog peed on the couch). At the adjective late-positivity-component (LPC), expressives and descriptives showed no difference, suggesting reduced social threat and that readers employ a 'wait-and-see' strategy to interpret expressives. Nouns preceded by expressives elicited a larger frontal P200, as well as reduced N400 and LPC than nouns preceded by descriptives. We associated the frontal P200 with emotional salience, the frontal N400 with mental imagery, and the LPC with cognitive load for combinatorics. We suggest that expressive adjectives are not bound to conceptual integration and conclude that parsers wait-and-see what is being damned.

Composition within and between Languages in the Bilingual Mind: MEG Evidence from Korean/English Bilinguals

The ability of the human brain to build complex expressions from simpler parts is fascinating, but the ability of the bilingual brain to do so is perhaps even more remarkable. When highly proficient bilinguals converse, they can fluidly switch from one language to another even inside sentences. Thus, they build expressions using words from more than one language. How are bilinguals able to compose words across different languages in real time? While robust evidence has implicated the left anterior temporal lobe (LATL) for the composition of words within one language, we do not know how the LATL, or other regions implicated for composition, operates when the language switches. We also do not know whether prefrontal regions associated with language control are recruited for language switching during composition. We addressed these questions with magnetoencephalography measurements in bilinguals who are fluent in two typologically distant languages, English and Korean. We observed early composition effects in the LATL at ∼200 ms that were unaffected by either language or orthography switching, which was also varied (Hangul vs Roman alphabet). Thus, the combinatory mechanism at 200 ms housed in the anterior temporal cortex appears blind to the language in which its input concepts are expressed. However, in later time windows, language and orthography switching interacted both in regions implicated for composition [LATL, ventromedial prefrontal cortex, left inferior frontal gyrus (LIFG)] as well as in regions associated with language control (ACC, LIFG). This establishes a starting point for understanding how bilingual brains code switch: words are initially combined without consideration of which language they come from, but language switching affects later processing.

Electrophysiological evidence for the coexistence of expectancy fulfillment and semantic integration during the processing of binding and compound nouns

This study employed electrophysiological measures to investigate the processes of expectancy fulfillment and semantic integration during the processing of binding and compound nouns in Chinese. Sequential expectancy and cloze probability between the two constituents of a two-character noun were manipulated in two experiments, resulting in four types of target characters (i.e., the final characters of two-character nouns): (i) high-cloze, binding character (HB); (ii) high-cloze, compound character (HC); (iii) low-cloze, binding character (LB); (iv) low-cloze, compound character (LC). Participants were asked to judge as quickly and accurately as possible whether the two characters, presented sequentially, formed a real word. The two experiments varied in stimulus onset asynchrony (SOA) between the initial and the target characters: 1000 ms and 300 ms. ERPs acquired at the target characters revealed a significant semantic integration effect in both experiments whereby low-cloze target characters elicited more negative-going activities than high-cloze target characters (L > H) with both long and short SOAs. Importantly, there was a graded N400 effect (LC = LB > HC > HB) at the central region: a clearly visible positive deflection for HB relative to HC in synchrony with an equivalence in negativity for LB and LC. This graded pattern was observed with an SOA of 1000 ms but not 300 ms. These results are discussed in terms of the possible coexistence of an expectancy fulfillment mechanism indexed by the P300 that monitors incoming information for realization of expectancies based on stored mental representations and a semantic integration mechanism indexed by the N400 that incorporates incoming information into the preceding context based on semantic-pragmatic knowledge. Manifestation of this coexistence appears sensitive to SOA-modulated attention orientation.

Semantic and syntactic composition of minimal adjective-noun phrases in Dutch: An MEG study

The possibility to combine smaller units of meaning (e.g., words) to create new and more complex meanings (e.g., phrases and sentences) is a fundamental feature of human language. In the present project, we investigated how the brain supports the semantic and syntactic composition of two-word adjective-noun phrases in Dutch, using magnetoencephalography (MEG). The present investigation followed up on previous studies reporting a composition effect in the left anterior temporal lobe (LATL) when comparing neural activity at nouns combined with adjectives, as opposed to nouns in a non-compositional context. The first aim of the present study was to investigate whether this effect, as well as its modulation by noun specificity and adjective class, can also be observed in Dutch. A second aim was to investigate to what extent these effects may be driven by syntactic composition rather than primarily by semantic composition as was previously proposed. To this end, a novel condition was administered in which participants saw nouns combined with pseudowords lacking meaning but agreeing with the nouns in terms of grammatical gender, as real adjectives would. We failed to observe a composition effect or its modulation in both a confirmatory analysis (focused on the cortical region and time-window where it has previously been reported) and in exploratory analyses (where we tested multiple regions and an extended potential time-window of the effect). A syntactically driven composition effect was also not observed in our data. We do, however, successfully observe an independent, previously reported effect on single word processing in our data, confirming that our MEG data processing pipeline does meaningfully capture language processing activity by the brain. The failure to observe the composition effect in LATL is surprising given that it has been previously reported in multiple studies. Reviewing all previous studies investigating this effect, we propose that materials and a task involving imagery might be necessary for this effect to be observed. In addition, we identified substantial variability in the regions of interest analyzed in previous studies, which warrants additional checks of robustness of the effect. Further research should identify limits and conditions under which this effect can be observed. The failure to observe specifically a syntactic composition effect in such minimal phrases is less surprising given that it has not been previously reported in MEG data.

EEG signatures of elementary composition: Disentangling genuine composition and expectancy processes

We adapted Bemis & Pylkkänen's (2011) paradigm to study elementary composition in Spanish using electroencephalography, to determine if EEG is sensitive enough to detect a composition-related activity and analyze whether the expectancy of participants to compose contributes to this signal. We found relevant activity at the expected channels and times, and a putative composition-related activity before the second word onset. Using threshold-free cluster permutation analysis and linear models we show a task-progression effect for the composition task that is not present for the list task. In a second experiment we evaluate two-word composition incorporating all conditions in a single task. In this case, we failed to find any significant composition-related activity suggesting that the activity measured with EEG may be in part carried by expectancy processes arising from the block design of the experiment, which can be prevented by using a non-blocked design and data-driven techniques to analyze the data.

Neural basis of basic composition: what we have learned from the red-boat studies and their extensions

Language is our mind's most powerful generative system for the expression of meaning and thought. What are the neural mechanisms of our ability to compose complex meanings from simpler representations? This question is impossible to answer unless we decompose the notion of 'meaning composition' in some theoretically guided way and then begin to assess the extent to which brain activity tracks the posited subroutines. Here, I summarize results from a body of MEG research that has begun to address this question from the ground up, first focusing on simple combinations of two words. The work sets off with a hypothesis space offered by theoretical linguistics, positing syntactic and logico-semantic composition as the main combinatory routines, but then reveals that the most consistent and prominent reflection of composition, localized in the left anterior temporal cortex at 200-250 ms, cannot be described with this toolkit. Instead, this activity tracks a much more conceptually driven process, robustly sensitive to the density of the conceptual feature space of the composing items. I will describe our functional understanding of this activity and how it may operate within a broader 'combinatory network.' This article is part of the theme issue 'Towards mechanistic models of meaning composition'.

Distinct Neural Processes for Memorizing Form and Meaning Within Sentences

In order to memorize sentences we use both processes of language comprehension during encoding and processes of language production during maintenance. While the former processes are easily testable via controlled presentation of the input, the latter are more difficult to assess directly as language production is typically initiated and controlled internally. In the present event-related potential (ERP) study we track subvocal rehearsal of sentences, with the goal of studying the concomitant planning processes with the help of a silent cued-production task. Native German participants read different types of sentences word-by-word, then were prompted by a visual cue to silently repeat each individual word, in a rehearsal phase. In order to assess both local and global effects of sentence planning, we presented correct sentences, syntactically or semantically violated sentences, or random word order sequences. Semantic violations during reading elicited an N400 effect at the noun violating the selectional restrictions of the preceding verb. Syntactic violations, induced by a gender incongruency between determiner and noun, led to a P600 effect at the same position. Different ERP patterns occurred during the silent production phase. Here, semantically violated sentences elicited an early fronto-central negativity at the verb, while syntactically violated sentences elicited a late right-frontal positivity at the determiner. Random word order was accompanied by long-lasting slow waves during the production phase. The findings are consistent with models of hierarchical sentence planning and further indicate that the ongoing working memory processes are qualitatively distinct from comprehension mechanisms and neurophysiologically specific for syntactic and lexical-semantic level planning. In conclusion, active working memory maintenance of sentences is likely to comprise specific stages of sentence production that are indicated by ERP correlates of syntactic and semantic planning at the phrasal and clausal level respectively.