Distinguishing integration and prediction accounts of ERP N400 modulations in language processing through experimental design

The virtual hand paradigm: A new method for studying prediction and language-vision interactions

We introduce a new method for measuring prediction and language-vision interactions: tracking the trajectories of hand-reaching movements in Virtual Reality (VR) environments. Spatiotemporal trajectory tracking of hand-reaching movements in VR offers an ecologically valid yet controlled medium for conducting experiments in an environment that mirrors characteristics of real-world behaviors. Importantly, it enables tracking the continuous dynamics of processing on a single-trial level. In an exploratory experiment, L2 speakers heard predictive or non-predictive sentences (e.g., "The barber cuts the hair" vs. "The coach remembers the hair"). Participants' task was to move their hands as quickly and as accurately as possible towards the object most relevant to the sentence. We measured reaction times (RTs) and hand-reaching trajectories as indicators of predictive behavior. There was a main effect of predictability: Predictable items were touched faster than unpredictable ones. Importantly, uncertainty was captured using spatiotemporal survival analysis by prolonged fluctuations in upward and downward vertical hand movements before making a final move to target or distractor. Self-correction of prediction errors was revealed by participants switching the direction of hand-reaching movements mid-trial. We conclude that the virtual hand paradigm enables measuring the onset and dynamics of predictive behavior in real time in single and averaged trial data and captures (un)certainty about target objects and the self-correction of prediction error online in 'close to real-world' experimental settings. The new method has great potential to provide additional insights about time-course and intermediate states of processing, provisional interpretations and partial target commitments that go beyond other state-of-the-art methods.

Prosodic intonation modulates semantic incongruence: Evidence from an electrophysiological study

People always make semantic predictions based on preceding contexts which, however, can be beyond semantic information. This study examines the role of prosodic intonation as a non-semantic cue in semantic prediction. To compare effects of different intonation conditions on attenuating semantic incongruence between preceding contexts and target utterances, we recorded electroencephalogram when the participants listened to emotional utterances with congruent or incongruent endings and focused on two event-related potential components, N400 and P600, which relate to semantic and pragmatic processing, respectively. Interestingly, we observed that surprising intonation can mitigate the N400 in response to semantic incongruence, and this modulation was strongly correlated (r = 0.78) with the increase of P600 amplitude induced by the same intonation across individual participants. These findings consistently indicate the importance of prosodic intonation in promoting semantic prediction by lessening listeners' perceived semantic incongruence, broadening our understanding of how non-semantic cues affect human verbal communication.

Performing a task with a friend does not change semantic processes but preparation: a social N400 and CNV event-related potential study

The N400 event-related potential (ERP) indexes the semantic processing of words. Recently, social N400 effects were reported: N400 amplitudes were found to be larger in the presence of a confederate. We tested whether this increase would be even larger in participants with friends (Pwfs). This was not the case: whether the words were coherent, incoherent or equivocal, N400s were not larger in Pwfs than in alones. According to the N400 inhibition hypothesis, the social N400 effects previously reported with confederates could then be due to the automatic sidelining of information that occurs when building a common ground with a stranger. Interestingly, contingent negative variations (CNVs) developed as the words had to be classified at the occurrence of an imperative stimulus that followed. PwFs had larger CNVs than alones, suggesting heightened preparation to this imperative stimulus. Unexpectedly, the larger this effect, the less confident PwFs were in their classifications. Given their higher levels of state anxiety before and after the experiment, it thus seems that the presence of someone else completing the same task, even if it is a friend, induces performance pressure, enhances anxiety and preparation, and diminishes self-confidence.

Semantic and Phonological Prediction in Language Comprehension: Pretarget Attraction Toward Semantic and Phonological Competitors in a Mouse Tracking Task

Recent evidence increasingly suggests that comprehenders are capable of generating probabilistic predictions about forthcoming linguistic inputs during language comprehension. However, it remains debated whether language comprehenders predict low-level word forms and whether they always make predictions. In this study, we investigated semantic and phonological prediction in high- and low-constraining sentence contexts, utilizing the mouse-tracking paradigm to trace mouse movement trajectories. Mandarin Chinese speakers listened to high- and low-constraining sentences which resulted in high and low predictability for the critical target words. While listening, participants viewed a visual display featuring two objects: one corresponding to the critical target word (the target object) and the other being either semantically related, phonologically related, or unrelated to the target word. Participants were instructed to click on the target object. The analysis of mouse movement trajectories revealed two key findings: (1) In both high- and low-constraining contexts, there was a spatial attraction of the cursor toward semantic competitors, notably occurring before the target word was heard; (2) there are indications that phonological pretarget attraction effects were observed primarily in high-constraining contexts. These findings suggest that the constraints of sentences have the potential to modulate the representational contents of linguistic prediction during language comprehension. Methodologically, the mouse-tracking paradigm presents a promising tool for further exploration of linguistic prediction.

Prior Context and Individual Alpha Frequency Influence Predictive Processing during Language Comprehension

The extent to which the brain predicts upcoming information during language processing remains controversial. To shed light on this debate, the present study reanalyzed Nieuwland and colleagues' (2018) [Nieuwland, M. S., Politzer-Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., et al. Large-scale replication study reveals a limit on probabilistic prediction in language comprehension. eLife, 7, e33468, 2018] replication of DeLong and colleagues (2015) [DeLong, K. A., Urbach, T. P., & Kutas, M. Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nature Neuroscience, 8, 1117-1121, 2005]. Participants (n = 356) viewed sentences containing articles and nouns of varying predictability, while their EEG was recorded. We measured ERPs preceding the critical words (namely, the semantic prediction potential), in conjunction with postword N400 patterns and individual neural metrics. ERP activity was compared with two measures of word predictability: cloze probability and lexical surprisal. In contrast to prior literature, semantic prediction potential amplitudes did not increase as cloze probability increased, suggesting that the component may not reflect prediction during natural language processing. Initial N400 results at the article provided evidence against phonological prediction in language, in line with Nieuwland and colleagues' findings. Strikingly, however, when the surprisal of the prior words in the sentence was included in the analysis, increases in article surprisal were associated with increased N400 amplitudes, consistent with prediction accounts. This relationship between surprisal and N400 amplitude was not observed when the surprisal of the two prior words was low, suggesting that expectation violations at the article may be overlooked under highly predictable conditions. Individual alpha frequency also modulated the relationship between article surprisal and the N400, emphasizing the importance of individual neural factors for prediction. The present study extends upon existing neurocognitive models of language and prediction more generally, by illuminating the flexible and subject-specific nature of predictive processing.

Understanding words in context: A naturalistic EEG study of children's lexical processing

When listening to speech, adults rely on context to anticipate upcoming words. Evidence for this comes from studies demonstrating that the N400, an event-related potential (ERP) that indexes ease of lexical-semantic processing, is influenced by the predictability of a word in context. We know far less about the role of context in children's speech comprehension. The present study explored lexical processing in adults and 5-10-year-old children as they listened to a story. ERPs time-locked to the onset of every word were recorded. Each content word was coded for frequency, semantic association, and predictability. In both children and adults, N400s reflect word predictability, even when controlling for frequency and semantic association. These findings suggest that both adults and children use top-down constraints from context to anticipate upcoming words when listening to stories.

Hand Gestures Have Predictive Potential During Conversation: An Investigation of the Timing of Gestures in Relation to Speech

During face-to-face conversation, transitions between speaker turns are incredibly fast. These fast turn exchanges seem to involve next speakers predicting upcoming semantic information, such that next turn planning can begin before a current turn is complete. Given that face-to-face conversation also involves the use of communicative bodily signals, an important question is how bodily signals such as co-speech hand gestures play into these processes of prediction and fast responding. In this corpus study, we found that hand gestures that depict or refer to semantic information started before the corresponding information in speech, which held both for the onset of the gesture as a whole, as well as the onset of the stroke (the most meaningful part of the gesture). This early timing potentially allows listeners to use the gestural information to predict the corresponding semantic information to be conveyed in speech. Moreover, we provided further evidence that questions with gestures got faster responses than questions without gestures. However, we found no evidence for the idea that how much a gesture precedes its lexical affiliate (i.e., its predictive potential) relates to how fast responses were given. The findings presented here highlight the importance of the temporal relation between speech and gesture and help to illuminate the potential mechanisms underpinning multimodal language processing during face-to-face conversation.

Combining EEG and 3D-eye-tracking to study the prediction of upcoming speech in naturalistic virtual environments: A proof of principle

EEG and eye-tracking provide complementary information when investigating language comprehension. Evidence that speech processing may be facilitated by speech prediction comes from the observation that a listener's eye gaze moves towards a referent before it is mentioned if the remainder of the spoken sentence is predictable. However, changes to the trajectory of anticipatory fixations could result from a change in prediction or an attention shift. Conversely, N400 amplitudes and concurrent spectral power provide information about the ease of word processing the moment the word is perceived. In a proof-of-principle investigation, we combined EEG and eye-tracking to study linguistic prediction in naturalistic, virtual environments. We observed increased processing, reflected in theta band power, either during verb processing - when the verb was predictive of the noun - or during noun processing - when the verb was not predictive of the noun. Alpha power was higher in response to the predictive verb and unpredictable nouns. We replicated typical effects of noun congruence but not predictability on the N400 in response to the noun. Thus, the rich visual context that accompanied speech in virtual reality influenced language processing compared to previous reports, where the visual context may have facilitated processing of unpredictable nouns. Finally, anticipatory fixations were predictive of spectral power during noun processing and the length of time fixating the target could be predicted by spectral power at verb onset, conditional on the object having been fixated. Overall, we show that combining EEG and eye-tracking provides a promising new method to answer novel research questions about the prediction of upcoming linguistic input, for example, regarding the role of extralinguistic cues in prediction during language comprehension.

The dissociation of semantically congruent and incongruent cross-modal effects on the visual attentional blink

Introduction

Recent studies have found that the sound-induced alleviation of visual attentional blink, a well-known phenomenon exemplifying the beneficial influence of multisensory integration on time-based attention, was larger when that sound was semantically congruent relative to incongruent with the second visual target (T2). Although such an audiovisual congruency effect has been attributed mainly to the semantic conflict carried by the incongruent sound restraining that sound from facilitating T2 processing, it is still unclear whether the integrated semantic information carried by the congruent sound benefits T2 processing.

Methods

To dissociate the congruence-induced benefit and incongruence-induced reduction in the alleviation of visual attentional blink at the behavioral and neural levels, the present study combined behavioral measures and event-related potential (ERP) recordings in a visual attentional blink task wherein the T2-accompanying sound, when delivered, could be semantically neutral in addition to congruent or incongruent with respect to T2.

Results

The behavioral data clearly showed that compared to the neutral sound, the congruent sound improved T2 discrimination during the blink to a higher degree while the incongruent sound improved it to a lesser degree. The T2-locked ERP data revealed that the early occipital cross-modal N195 component (192-228 ms after T2 onset) was uniquely larger in the congruent-sound condition than in the neutral-sound and incongruent-sound conditions, whereas the late parietal cross-modal N440 component (400-500 ms) was prominent only in the incongruent-sound condition.

Discussion

These findings provide strong evidence that the modulating effect of audiovisual semantic congruency on the sound-induced alleviation of visual attentional blink contains not only a late incongruence-induced cost but also an early congruence-induced benefit, thereby demonstrating for the first time an unequivocal congruent-sound-induced benefit in alleviating the limitation of time-based visual attention.

Trial-by-trial fluctuations of pre-stimulus alpha power predict language ERPs

Anticipatory mechanisms are known to play a key role in language, but they have been mostly investigated with violation paradigms, which only consider what happens after predictions have been (dis)confirmed. Relatively few studies focused on the pre-stimulus interval and found that stronger expectations are associated with lower pre-stimulus alpha power. However, alpha power also fluctuates spontaneously, in the absence of experimental manipulations; and in the attention and perception domains, spontaneously low pre-stimulus power is associated with better behavioral performance and with event-related potential (ERPs) with shorter latencies and higher amplitudes. Importantly, little is known about the role of alpha fluctuations in other domains, as it is in language. To this aim, we investigated whether spontaneous fluctuations in pre-stimulus alpha power modulate language-related ERPs in a semantic congruence task. Electrophysiology data were analyzed using Generalized Additive Mixed Models to model nonlinear interactions between pre-stimulus alpha power and EEG amplitude, at the single-trial level. We found that the N400 and the late posterior positivity/P600 were larger in the case of lower pre-stimulus alpha power. Still, while the N400 was observable regardless of the level of pre-stimulus power, a late posterior positivity/P600 effect was only observable for low pre-stimulus alpha power. We discuss these findings in light of the different, albeit connected, functional interpretations of pre-stimulus alpha and the ERPs according to both a nonpredictive interpretation focused on attentional mechanisms and under a predictive processing framework.

The role of multimodal cues in second language comprehension

In face-to-face communication, multimodal cues such as prosody, gestures, and mouth movements can play a crucial role in language processing. While several studies have addressed how these cues contribute to native (L1) language processing, their impact on non-native (L2) comprehension is largely unknown. Comprehension of naturalistic language by L2 comprehenders may be supported by the presence of (at least some) multimodal cues, as these provide correlated and convergent information that may aid linguistic processing. However, it is also the case that multimodal cues may be less used by L2 comprehenders because linguistic processing is more demanding than for L1 comprehenders, leaving more limited resources for the processing of multimodal cues. In this study, we investigated how L2 comprehenders use multimodal cues in naturalistic stimuli (while participants watched videos of a speaker), as measured by electrophysiological responses (N400) to words, and whether there are differences between L1 and L2 comprehenders. We found that prosody, gestures, and informative mouth movements each reduced the N400 in L2, indexing easier comprehension. Nevertheless, L2 participants showed weaker effects for each cue compared to L1 comprehenders, with the exception of meaningful gestures and informative mouth movements. These results show that L2 comprehenders focus on specific multimodal cues - meaningful gestures that support meaningful interpretation and mouth movements that enhance the acoustic signal - while using multimodal cues to a lesser extent than L1 comprehenders overall.

Prediction during language comprehension: what is next?

Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.

Cross-modal enhancement of spatially unpredictable visual target discrimination during the attentional blink

The attentional blink can be substantially reduced by delivering a task-irrelevant sound synchronously with the second target (T2) embedded in a rapid serial visual presentation stream, which is further modulated by the semantic congruency between the sound and T2. The present study extended the cross-modal boost during attentional blink and the modulation of audiovisual semantic congruency in the spatial domain by showing that a spatially uninformative, semantically congruent (but not incongruent) sound could even improve the discrimination of spatially unpredictable T2 during attentional blink. T2-locked event-related potential (ERP) data yielded that the early cross-modal P195 difference component (184-234 ms) over the occipital scalp contralateral to the T2 location was larger preceding accurate than inaccurate discriminations of semantically congruent, but not incongruent, audiovisual T2s. Interestingly, the N2pc component (194-244 ms) associated with visual-spatial attentional allocation was enlarged for incongruent audiovisual T2s relative to congruent audiovisual and unisensory visual T2s only when they were accurately discriminated. These ERP findings suggest that the spatially extended cross-modal boost during attentional blink involves an early cross-modal interaction strengthening the perceptual processing of T2, without any sound-induced enhancement of visual-spatial attentional allocation toward T2. In contrast, the absence of an accuracy decrease in response to semantically incongruent audiovisual T2s may originate from the semantic mismatch capturing extra visual-spatial attentional resources toward T2.

Children with autism spectrum disorder who demonstrate normal language scores use a bottom-up semantic processing strategy: Evidence from N400 recordings

INTRODUCTION

The N400 is an electrophysiological component that reflects lexical access and integration of words with mental representations.

METHODS

Thirty-five young children with a range of language capabilities (n = 21 neurotypical controls, 10 males, mean age = 6.3 ± 0.9 years; n = 14 children with autism, 12 males, mean age = 6.4 ± 1.1 years) completed an auditory semantic categorization paradigm to evoke the N400. Electroencephalograph (EEG) data were acquired with a 64-channel electrode cap as children listened via ear inserts to binaurally presented single syllable words and decided whether the words were congruent (in) or incongruent (out) with a pre-specified category. EEG data were filtered, epoched, and averaged referenced, and global field power (GFP) was computed. The amplitude of the N400 peak in the GFP was submitted to a multiple linear regression analysis.

RESULTS

N400 amplitude was found to predict language scores only for the children with ASD who have language scores in the normal range (r2  = 0.72).

CONCLUSIONS

This finding that N400 amplitude only predicted language scores in children with ASD and normal language scores suggests that these children may rely more on basic semantic processing (as reflected by the N400) and less on anticipating and predicting upcoming words. This suggests preferential utilization of a bottom-up strategy to access higher order language.

Unveiling the need of interactions for social N400s and supporting the N400 inhibition hypothesis

When participants (Pps) are presented with stimuli in the presence of another person, they may consider that person's perspective. Indeed, five recent ERP studies show that the amplitudes of their N400s are increased. The two most recent ones reveal that these social-N400 increases occur even when instructions do not require a focus on the other's perspective. These increases also happen when Pps know that this other person has the same stimulus information as they have. However, in all these works, Pps could see the other person. Here, we tested whether the interaction occurring with this sight is important or whether these social N400 increases also occur when the other person is seated a bit behind Pps, who are aware of it. All had to decide whether the word ending short stories was coherent, incoherent, or equivocal. No social N400 increase was observed: N400s elicited by those words in Pps who were with a confederate (n = 50) were similar to those of Pps who were alone (n = 51). On the other hand, equivocal endings did not elicit larger N400s than coherent ones but triggered larger late posterior positivities (LPPs), like in previous studies. The discussion focuses on the circumstances in which perspective-taking occurs and on the functional significance of the N400 and the LPP.

Is the Mind Inherently Predicting? Exploring Forward and Backward Looking in Language Processing

Prediction is one characteristic of the human mind. But what does it mean to say the mind is a "prediction machine" and inherently forward looking as is frequently claimed? In natural languages, many contexts are not easily predictable in a forward fashion. In English, for example, many frequent verbs do not carry unique meaning on their own but instead, rely on another word or words that follow them to become meaningful. Upon reading take a the processor often cannot easily predict walk as the next word. But the system can "look back" and integrate walk more easily when it follows take a (e.g., as opposed to *make|get|have a walk). In the present paper, we provide further evidence for the importance of both forward and backward-looking in language processing. In two self-paced reading tasks and an eye-tracking reading task, we found evidence that adult English native speakers' sensitivity to word forward and backward conditional probability significantly predicted reading times over and above psycholinguistic predictors of reading latencies. We conclude that both forward and backward-looking (prediction and integration) appear to be important characteristics of language processing. Our results thus suggest that it makes just as much sense to call the mind an "integration machine" which is inherently backward 'looking.'

The electrophysiological correlates of word pre-activation during associative word learning

Human beings continuously make use of learned associations to generate predictions about future occurrences in the environment. Such memory-related predictive processes provide a scaffold for learning in that mental representations of foreseeable events can be adjusted or strengthened based on a specific outcome. Learning the meaning of novel words through picture-word associations constitutes a prime example of associative learning because pictures preceding words can trigger word prediction through the pre-activation of the related mnemonic representations. In the present electroencephalography (EEG) study, we used event-related potentials (ERPs) to compare neural indices of word pre-activation between a word learning condition with maximal prediction likelihood and a non-learning control condition with low prediction. Results revealed that prediction-related N400 amplitudes in response to pictures decreased over time at central electrodes as a function of word learning, whereas late positive component (LPC) amplitudes increased. Notably, N400 but not LPC changes were also predictive of word learning performance, suggesting that the N400 component constitutes a sensitive marker of word pre-activation during associative word learning.

A hierarchy of linguistic predictions during natural language comprehension

Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analyzing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable neural signatures of predictions about syntactic category (parts of speech), phonemes, and semantics. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction.

A hierarchy of linguistic predictions during natural language comprehension

Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analyzing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable neural signatures of predictions about syntactic category (parts of speech), phonemes, and semantics. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction.

Developmental Language Disorder as Syntactic Prediction Impairment

We provide evidence that children with Developmental Language Disorder (DLD) are impaired in predictive syntactic processing. In the current study, children listened passively to auditorily-presented sentences, where the critical condition included an unexpected "filled gap" in the direct object position of the relative clause verb. A filled gap is illustrated by the underlined phrase in "The zebra that the hippo kissed the camel on the nose…", rather than the expected "the zebra that the hippo kissed [e] on the nose", where [e] denotes the gap. Brain responses to the filled gap were compared to a control condition using adverb-relative clauses with identical substrings: "The weekend that the hippo kissed the camel on the nose [e]…". Here, the same noun phrase is not unexpected because the adverb gap occurs later in the structure. We hypothesized that a filled gap would elicit a prediction error brain signal in the form of an early anterior negativity, as we have previously observed in adults. We found an early (bilateral) anterior negativity to the filled gap in a control group of children with Typical Development (TD), but the children with DLD exhibited no brain response to the filled gap during the same early time window. This suggests that children with DLD fail to predict that a relativized object should correspond to an empty position after the relative clause verb, suggesting an impairment in predictive processing. We discuss how this lack of a prediction error signal can interact with language acquisition and result in DLD.

The interplay between audiovisual temporal synchrony and semantic congruency in the cross-modal boost of the visual target discrimination during the attentional blink

The visual attentional blink can be substantially reduced by delivering a task-irrelevant sound synchronously with the second visual target (T2), and this effect is further modulated by the semantic congruency between the sound and T2. However, whether the cross-modal benefit originates from audiovisual interactions or sound-induced alertness remains controversial, and whether the semantic congruency effect is contingent on audiovisual temporal synchrony needs further investigation. The current study investigated these questions by recording event-related potentials (ERPs) in a visual attentional blink task wherein a sound could either synchronize with T2, precede T2 by 200 ms, be delayed by 100 ms, or be absent, and could be either semantically congruent or incongruent with T2 when delivered. The behavioral data showed that both the cross-modal boost of T2 discrimination and the further semantic modulation were the largest when the sound synchronized with T2. In parallel, the ERP data yielded that both the early occipital cross-modal P195 component (192-228 ms after T2 onset) and late parietal cross-modal N440 component (424-448 ms) were prominent only when the sound synchronized with T2, with the former being elicited solely when the sound was further semantically congruent whereas the latter occurring only when that sound was incongruent. These findings demonstrate not only that the cross-modal boost of T2 discrimination during the attentional blink stems from early audiovisual interactions and the semantic congruency effect depends on audiovisual temporal synchrony, but also that the semantic modulation can unfold at the early stage of visual discrimination processing.

Listeners track talker-specific prosody to deal with talker-variability

One of the challenges in speech perception is that listeners must deal with considerable segmental and suprasegmental variability in the acoustic signal due to differences between talkers. Most previous studies have focused on how listeners deal with segmental variability. In this EEG experiment, we investigated whether listeners track talker-specific usage of suprasegmental cues to lexical stress to recognize spoken words correctly. In a three-day training phase, Dutch participants learned to map non-word minimal stress pairs onto different object referents (e.g., USklot meant "lamp"; usKLOT meant "train"). These non-words were produced by two male talkers. Critically, each talker used only one suprasegmental cue to signal stress (e.g., Talker A used only F0 and Talker B only intensity). We expected participants to learn which talker used which cue to signal stress. In the test phase, participants indicated whether spoken sentences including these non-words were correct ("The word for lamp is…"). We found that participants were slower to indicate that a stimulus was correct if the non-word was produced with the unexpected cue (e.g., Talker A using intensity). That is, if in training Talker A used F0 to signal stress, participants experienced a mismatch between predicted and perceived phonological word-forms if, at test, Talker A unexpectedly used intensity to cue stress. In contrast, the N200 amplitude, an event-related potential related to phonological prediction, was not modulated by the cue mismatch. Theoretical implications of these contrasting results are discussed. The behavioral findings illustrate talker-specific prediction of prosodic cues, picked up through perceptual learning during training.

The role of crosslinguistic differences in second language anticipatory processing: An event-related potentials study

The present study uses event-related potentials to investigate how crosslinguistic (dis)similarities modulate anticipatory processing in the second language (L2). Participants read predictive stories in English that made a genitive construction consisting of a third-person singular possessive pronoun and a kinship noun (e.g., his mother) likely in an upcoming continuation. The possessive pronoun's form depended on the antecedent's natural gender, which had been previously established in the stories. The continuation included either the expected genitive construction or an unexpected one with a possessive pronoun of the opposite gender. We manipulated crosslinguistic (dis)similarity by comparing advanced English learners with either Swedish or Spanish as their L1. While Swedish has equivalent possessive pronouns that mark the antecedent's natural gender (i.e., hans/hennes "his/her"), Spanish does not. In fact, Spanish possessive pronouns mark the syntactic features (number, gender) of the possessed noun (e.g., nosotros queremos a nuestra madre "we-_MASC love our-_FEM mother-_FEM). Twenty-four native speakers of English elicited an N400 effect for prenominal possessives that were unexpected based on the possessor noun's natural gender, consistent with the possibility that they activated the pronoun's form or its semantic features (natural gender). Thirty-two Swedish-speaking learners yielded a qualitatively and quantitatively native-like N400 for unexpected prenominal possessives. In contrast, twenty-five Spanish-speaking learners showed a P600 effect for unexpected possessives, consistent with the possibility that they experienced difficulty integrating a pronoun that mismatched the expected gender. Results suggest that differences with respect to the features encoded in the activated representation result in different predictive mechanisms among adult L2 learners.

Theta oscillations support the interface between language and memory

Recent evidence shows that hippocampal theta oscillations, usually linked to memory and navigation, are also observed during online language processing, suggesting a shared neurophysiological mechanism between language and memory. However, it remains to be established what specific roles hippocampal theta oscillations may play in language, and whether and how theta mediates the communication between the hippocampus and the perisylvian cortical areas, generally thought to support language processing. With whole-head magnetoencephalographic (MEG) recordings, the present study investigated these questions with two experiments. Using a violation paradigm, extensively used for studying neural underpinnings of different aspects of linguistic processing, we found increased theta power (4-8 ​Hz) in the hippocampal formation, when participants read a semantically incorrect vs. correct sentence ending. Such a pattern of results was replicated using different sentence stimuli in another cohort of participants. Importantly, no significant hippocampal theta power increase was found when participants read a semantically correct but syntactically incorrect sentence ending vs. a correct sentence ending. These findings may suggest that hippocampal theta oscillations are specifically linked to lexical-semantic related processing, and not general information processing in sentence reading. Furthermore, we found significantly transient theta phase coupling between the hippocampus and the left superior temporal gyrus, a hub area of the cortical network for language comprehension. This transient theta phase coupling may provide an important channel that links the memory and language systems for the generation of sentence meaning. Overall, these findings help specify the role of hippocampal theta in language, and provide a novel neurophysiological mechanism at the network level that may support the interface between memory and language.