1
|
Liu S, Liu X. Context influence on speech perception: evidence for acoustic-level mechanism across the voice onset time continuum. Neuroimage 2025; 310:121140. [PMID: 40089219 DOI: 10.1016/j.neuroimage.2025.121140] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2024] [Revised: 02/25/2025] [Accepted: 03/12/2025] [Indexed: 03/17/2025] Open
Abstract
Contextual information plays a significant role in shaping our perception of speech, yet it remains uncertain at which level of processing such information integrates with acoustic cues. A key area of debate is whether top-down information influences acoustic encoding within the lower levels of the speech-processing hierarchy. This study employed a machine learning algorithm to decode the voice onset time (VOT) of speech and investigated how the gender of the speaker of a precursor sentence impacted subsequent speech perception. Using EEG recordings, we examined neural responses to a VOT continuum following male and female voices. Our results reveal that a linear representation of the VOT continuum emerged at an early EEG time window and that gender-based contextual cues modulated speech perception at this stage. Notably, since context information was not involved in the decoding procedure itself, we conclude that this modulation reflected the true effects of context on the perception of VOT. Moreover, the contextual influence extended across the entire VOT continuum, not just at specific sounds, suggesting a broad and consistent modulation of speech perception by gender-based context. These findings support the idea of a general acoustic-level mechanism through which contextual information influences the early stage of speech processing, contributing to ongoing debates about the interaction between top-down and bottom-up processes in speech perception.
Collapse
Affiliation(s)
- Shun Liu
- Department of Education, Hunan First Normal University, Changsha, China; Department of Psychology, University of Jyvaskyla, Jyväskylä, Finland
| | - Xiqin Liu
- School of Foreign Languages, South China University of Technology, Guangzhou, China.
| |
Collapse
|
2
|
Ye W, Qu Q. Semantic and Phonological Prediction in Language Comprehension: Pretarget Attraction Toward Semantic and Phonological Competitors in a Mouse Tracking Task. Cogn Sci 2025; 49:e70054. [PMID: 40100145 DOI: 10.1111/cogs.70054] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2024] [Revised: 02/25/2025] [Accepted: 03/01/2025] [Indexed: 03/20/2025]
Abstract
Recent evidence increasingly suggests that comprehenders are capable of generating probabilistic predictions about forthcoming linguistic inputs during language comprehension. However, it remains debated whether language comprehenders predict low-level word forms and whether they always make predictions. In this study, we investigated semantic and phonological prediction in high- and low-constraining sentence contexts, utilizing the mouse-tracking paradigm to trace mouse movement trajectories. Mandarin Chinese speakers listened to high- and low-constraining sentences which resulted in high and low predictability for the critical target words. While listening, participants viewed a visual display featuring two objects: one corresponding to the critical target word (the target object) and the other being either semantically related, phonologically related, or unrelated to the target word. Participants were instructed to click on the target object. The analysis of mouse movement trajectories revealed two key findings: (1) In both high- and low-constraining contexts, there was a spatial attraction of the cursor toward semantic competitors, notably occurring before the target word was heard; (2) there are indications that phonological pretarget attraction effects were observed primarily in high-constraining contexts. These findings suggest that the constraints of sentences have the potential to modulate the representational contents of linguistic prediction during language comprehension. Methodologically, the mouse-tracking paradigm presents a promising tool for further exploration of linguistic prediction.
Collapse
Affiliation(s)
- Wenting Ye
- Key Laboratory of Cognition and Personality (SWU), Ministry of Education
- Faculty of Psychology, Southwest University (SWU)
- Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences
| | - Qingqing Qu
- Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences
- Department of Psychology, University of Chinese Academy of Sciences
| |
Collapse
|
3
|
Lewendon J, Britton J, Politzer-Ahles S. The Phonological Mapping Negativity (PMN) as a language-specific component: Exploring responses to linguistic vs musical mismatch. PLoS One 2024; 19:e0315537. [PMID: 39700196 DOI: 10.1371/journal.pone.0315537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Accepted: 11/26/2024] [Indexed: 12/21/2024] Open
Abstract
The Phonological Mismatch Negativity (PMN) is an ERP component said to index the processing of phonological information, and is known to increase in amplitude when phonological expectations are violated. For example, in a context that generates expectation of a certain phoneme, the PMN will become relatively more negative if the phoneme is switched for an alternative. The response is comparable to other temporally-proximate components, insofar as it indicates a neurological response to unexpected auditory input, but remains considered distinct by the field on the basis of its proposed specific sensitivity to phonology. Despite this, reports of the PMN overlap notably, both in temporal and topographic distribution, with the Mismatch Negativity (MMN) and the N400, and limited research to date has been conducted to establish whether these extant distinctions withstand testing. In the present study, we investigate the PMN's sensitivity to non-linguistic mismatches so as to test the response's specific language sensitivity. Participants heard primes-three-syllable words-played simultaneously to three-note tunes, with the instructions to attend exclusively to either the linguistic or musical content. They were then tasked with removing the first syllable (phoneme manipulation) or note (music manipulation) to form the target. Targets either matched or mismatched primes, thus achieving physically identical note or phoneme mismatches. Results show that a PMN was not elicited during the musical mismatch condition, a finding which supports suggestions that the PMN may be a language-specific response. However, our results also indicate that further research is necessary to determine the relationship between the PMN and N400. Though our paper probes a previously unstudied dimension of the PMN, questions still remain surrounding whether the PMN, although seemingly language-specific, is truly a phonology-specific component.
Collapse
Affiliation(s)
- Jen Lewendon
- Department of Psychology, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| | - James Britton
- Department of Chinese and Bilingual Studies, Hong Kong Polytechnic University, Hong Hum, Hong Kong
| | - Stephen Politzer-Ahles
- Department of Linguistics, The University of Kansas, Lawrence, Kansas, United States of America
| |
Collapse
|
4
|
Wong R, Reichle ED, Veldre A. Prediction in reading: A review of predictability effects, their theoretical implications, and beyond. Psychon Bull Rev 2024:10.3758/s13423-024-02588-z. [PMID: 39482486 DOI: 10.3758/s13423-024-02588-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/13/2024] [Indexed: 11/03/2024]
Abstract
Historically, prediction during reading has been considered an inefficient and cognitively expensive processing mechanism given the inherently generative nature of language, which allows upcoming text to unfold in an infinite number of possible ways. This article provides an accessible and comprehensive review of the psycholinguistic research that, over the past 40 or so years, has investigated whether readers are capable of generating predictions during reading, typically via experiments on the effects of predictability (i.e., how well a word can be predicted from its prior context). Five theoretically important issues are addressed: What is the best measure of predictability? What is the functional relationship between predictability and processing difficulty? What stage(s) of processing does predictability affect? Are predictability effects ubiquitous? What processes do predictability effects actually reflect? Insights from computational models of reading about how predictability manifests itself to facilitate the reading of text are also discussed. This review concludes by arguing that effects of predictability can, to a certain extent, be taken as demonstrating evidence that prediction is an important but flexible component of real-time language comprehension, in line with broader predictive accounts of cognitive functioning. However, converging evidence, especially from concurrent eye-tracking and brain-imaging methods, is necessary to refine theories of prediction.
Collapse
Affiliation(s)
- Roslyn Wong
- School of Psychological Sciences, Macquarie University, Sydney, Australia.
| | - Erik D Reichle
- School of Psychological Sciences, Macquarie University, Sydney, Australia
| | - Aaron Veldre
- School of Psychological Sciences, Macquarie University, Sydney, Australia
- Graduate School of Health, University of Technology Sydney, Sydney, Australia
| |
Collapse
|
5
|
Banaraki AK, Toghi A, Mohammadzadeh A. RDoC Framework Through the Lens of Predictive Processing: Focusing on Cognitive Systems Domain. COMPUTATIONAL PSYCHIATRY (CAMBRIDGE, MASS.) 2024; 8:178-201. [PMID: 39478691 PMCID: PMC11523845 DOI: 10.5334/cpsy.119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/25/2024] [Accepted: 10/11/2024] [Indexed: 11/02/2024]
Abstract
In response to shortcomings of the current classification system in translating discoveries from basic science to clinical applications, NIMH offers a new framework for studying mental health disorders called Research Domain Criteria (RDoC). This framework holds a multidimensional outlook on psychopathologies focusing on functional domains of behavior and their implementing neural circuits. In parallel, the Predictive Processing (PP) framework stands as a leading theory of human brain function, offering a unified explanation for various types of information processing in the brain. While both frameworks share an interest in studying psychopathologies based on pathophysiology, their integration still needs to be explored. Here, we argued in favor of the explanatory power of PP to be a groundwork for the RDoC matrix in validating its constructs and creating testable hypotheses about mechanistic interactions between molecular biomarkers and clinical traits. Together, predictive processing may serve as a foundation for achieving the goals of the RDoC framework.
Collapse
Affiliation(s)
| | - Armin Toghi
- Institute for Cognitive and Brain Sciences, Shahid Beheshti University, Tehran, Iran
| | - Azar Mohammadzadeh
- Research Center for Cognitive and Behavioral Studies, Tehran University of Medical Science, Tehran, Iran
| |
Collapse
|
6
|
Alekseeva M, Myachykov A, Bermudez-Margaretto B, Shtyrov Y. Morphosyntactic prediction in automatic neural processing of spoken language: EEG evidence. Brain Res 2024; 1836:148949. [PMID: 38641266 DOI: 10.1016/j.brainres.2024.148949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 03/29/2024] [Accepted: 04/16/2024] [Indexed: 04/21/2024]
Abstract
Automatic parsing of syntactic information by the human brain is a well-established phenomenon, but its mechanisms remain poorly understood. Its best-known neurophysiological reflection is the so-called early left-anterior negativity (ELAN) component of event-related potentials (ERPs), with two alternative hypotheses for its origin: (1) error detection, or (2) morphosyntactic prediction/priming. To test these alternatives, we conducted two experiments using a non-attend passive design with visual distraction and recorded ERPs to spoken pronoun-verb phrases with/without agreement violations and to the same critical verbs presented in isolation without preceding pronouns. The results revealed an ELAN at ∼130-220 ms for pronoun-verb gender agreement violations, confirming a high degree of automaticity in early morphosyntactic parsing. Critically, the strongest ELAN was elicited by verbs outside phrasal context, which suggests that the typical ELAN pattern is underpinned by a reduction of ERP amplitudes for felicitous combinations, reflecting syntactic priming/predictability between related words/morphemes (potentially mediated by associative links formed during previous linguistic experience) rather than specialised error-detection processes.
Collapse
Affiliation(s)
- Maria Alekseeva
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, Higher School of Economics, Moscow, Russian Federation.
| | | | - Beatriz Bermudez-Margaretto
- Instituto de Integración en la Comunidad (INICO), Facultad de Psicología, Universidad de Salamanca, Salamanca, Spain
| | - Yury Shtyrov
- Center of Functionally Integrative Neuroscience, Aarhus University, Aarhus, Denmark
| |
Collapse
|
7
|
Levari T, Snedeker J. Understanding words in context: A naturalistic EEG study of children's lexical processing. JOURNAL OF MEMORY AND LANGUAGE 2024; 137:104512. [PMID: 38855737 PMCID: PMC11160963 DOI: 10.1016/j.jml.2024.104512] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2024]
Abstract
When listening to speech, adults rely on context to anticipate upcoming words. Evidence for this comes from studies demonstrating that the N400, an event-related potential (ERP) that indexes ease of lexical-semantic processing, is influenced by the predictability of a word in context. We know far less about the role of context in children's speech comprehension. The present study explored lexical processing in adults and 5-10-year-old children as they listened to a story. ERPs time-locked to the onset of every word were recorded. Each content word was coded for frequency, semantic association, and predictability. In both children and adults, N400s reflect word predictability, even when controlling for frequency and semantic association. These findings suggest that both adults and children use top-down constraints from context to anticipate upcoming words when listening to stories.
Collapse
Affiliation(s)
- Tatyana Levari
- Department of Psychology, Harvard University, United States
| | - Jesse Snedeker
- Department of Psychology, Harvard University, United States
| |
Collapse
|
8
|
Wu M, Li M, Wu D. The neurocognitive processing mechanism of English subject-verb agreement by Chinese-speaking learners. Front Psychol 2024; 15:1402355. [PMID: 39027056 PMCID: PMC11256399 DOI: 10.3389/fpsyg.2024.1402355] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2024] [Accepted: 06/17/2024] [Indexed: 07/20/2024] Open
Abstract
Determiner phrases (DPs), an overarching term, can be classified into two determiner types: referential determiner phrases (RDPs, e.g., the boy) and quantificational determiner phrases (QDPs, e.g., each boy). Using the event-related potential (ERP) technique, this study explored the modulation of RDP vs. QDP in the online processing of English subject-verb agreement with omission errors by Chinese learners of English, addressing the question of whether singular quantification increases or decreases Chinese learners' sensitivity to agreement violations. The experiment manipulated the determiner type, specifically RDP vs. QDP, and grammaticality (grammatical vs. ungrammatical). The results indicated that similar to previous studies, a P600 effect was elicited in response to subject-verb agreement violations with omission errors, demonstrating that Chinese L2 learners are sensitive to such agreement violations. Additionally, the ERP patterns exhibited variations due to D-linking and number specification of RDP and QDP. Regarding D-linking, subject-verb agreement violations in the QDP conditions, necessitating integration of discourse-related knowledge, elicited laterally and frontally distributed P600 effects associated with integration complexity at the discourse level; however, non-D-linked referential determiners elicited the posteriorly-distributed P600 effects. Differences in number specification resulted in the distinctive P600 latencies and whether P600 was preceded by N400 or not. While both the RDP and QDP conditions exhibited the P600 effects, the onset latency of this effect in the number-unspecified RDP condition was 300 ms later compared to the number-specified QDP condition. Furthermore, an additional N400 component observed in the RDP condition suggests that L2 learners acquire morphologically complex subject-verb agreements by rote, treating them as unanalyzed chunks. This N400 component was absent in the QDP condition. From these results, the conclusion can be drawn that L2 learners are sensitive to the subject-verb agreement violations with omission errors, and L2 processing patterns of subject-verb agreement vary with different features of determiners, providing further evidence for the cue-based retrieval model during comprehension of grammatical sentences. Pedagogical implications are provided, and the future research direction is suggested.
Collapse
Affiliation(s)
| | | | - Di Wu
- School of Foreign Languages, Jiangsu University of Science and Technology, Zhenjiang, China
| |
Collapse
|
9
|
Weissman B, Cohn N, Tanner D. The electrophysiology of lexical prediction of emoji and text. Neuropsychologia 2024; 198:108881. [PMID: 38579906 DOI: 10.1016/j.neuropsychologia.2024.108881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2023] [Revised: 02/18/2024] [Accepted: 03/27/2024] [Indexed: 04/07/2024]
Abstract
As emoji often appear naturally alongside text in utterances, they provide a way to study how prediction unfolds in multimodal sentences in direct comparison to unimodal sentences. In this experiment, participants (N = 40) read sentences in which the sentence-final noun appeared in either word form or emoji form, a between-subjects manipulation. The experiment featured both high constraint sentences and low constraint sentences to examine how the lexical processing of emoji interacts with prediction processes in sentence comprehension. Two well-established ERP components linked to lexical processing and prediction - the N400 and the Late Frontal Positivity - are investigated for sentence-final words and emoji to assess whether, to what extent, and in what linguistic contexts emoji are processed like words. Results indicate that the expected effects, namely an N400 effect to an implausible lexical item compared to a plausible one and an LFP effect to an unexpected lexical item compared to an expected one, emerged for both words and emoji. This paper discusses the similarities and differences between the stimulus types and constraint conditions, contextualized within theories of linguistic prediction, ERP components, and a multimodal lexicon.
Collapse
Affiliation(s)
- Benjamin Weissman
- Department of Cognitive Science Rensselaer Polytechnic Institute 110 8th Street, Troy, NY, 12180, USA; Department of Linguistics University of Illinois at Urbana-Champaign 707 S Mathews Ave, Urbana, IL, 61801, USA.
| | - Neil Cohn
- Department of Communication and Cognition Tilburg University PO Box 90153, 5000, LE Tilburg, the Netherlands
| | - Darren Tanner
- Department of Linguistics University of Illinois at Urbana-Champaign 707 S Mathews Ave, Urbana, IL, 61801, USA; AI For Good Lab Microsoft 1 Microsoft Way, Redmond, WA, USA
| |
Collapse
|
10
|
Ding R, Ten Oever S, Martin AE. Delta-band Activity Underlies Referential Meaning Representation during Pronoun Resolution. J Cogn Neurosci 2024; 36:1472-1492. [PMID: 38652108 DOI: 10.1162/jocn_a_02163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/25/2024]
Abstract
Human language offers a variety of ways to create meaning, one of which is referring to entities, objects, or events in the world. One such meaning maker is understanding to whom or to what a pronoun in a discourse refers to. To understand a pronoun, the brain must access matching entities or concepts that have been encoded in memory from previous linguistic context. Models of language processing propose that internally stored linguistic concepts, accessed via exogenous cues such as phonological input of a word, are represented as (a)synchronous activities across a population of neurons active at specific frequency bands. Converging evidence suggests that delta band activity (1-3 Hz) is involved in temporal and representational integration during sentence processing. Moreover, recent advances in the neurobiology of memory suggest that recollection engages neural dynamics similar to those which occurred during memory encoding. Integrating from these two research lines, we here tested the hypothesis that neural dynamic patterns, especially in delta frequency range, underlying referential meaning representation, would be reinstated during pronoun resolution. By leveraging neural decoding techniques (i.e., representational similarity analysis) on a magnetoencephalogram data set acquired during a naturalistic story-listening task, we provide evidence that delta-band activity underlies referential meaning representation. Our findings suggest that, during spoken language comprehension, endogenous linguistic representations such as referential concepts may be proactively retrieved and represented via activation of their underlying dynamic neural patterns.
Collapse
Affiliation(s)
- Rong Ding
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
| | - Sanne Ten Oever
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
- Radboud University Donders Centre for Cognitive Neuroimaging, Nijmegen, The Netherlands
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, The Netherlands
| | - Andrea E Martin
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
- Radboud University Donders Centre for Cognitive Neuroimaging, Nijmegen, The Netherlands
| |
Collapse
|
11
|
Nour Eddine S, Brothers T, Wang L, Spratling M, Kuperberg GR. A predictive coding model of the N400. Cognition 2024; 246:105755. [PMID: 38428168 PMCID: PMC10984641 DOI: 10.1016/j.cognition.2024.105755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 02/14/2024] [Accepted: 02/19/2024] [Indexed: 03/03/2024]
Abstract
The N400 event-related component has been widely used to investigate the neural mechanisms underlying real-time language comprehension. However, despite decades of research, there is still no unifying theory that can explain both its temporal dynamics and functional properties. In this work, we show that predictive coding - a biologically plausible algorithm for approximating Bayesian inference - offers a promising framework for characterizing the N400. Using an implemented predictive coding computational model, we demonstrate how the N400 can be formalized as the lexico-semantic prediction error produced as the brain infers meaning from the linguistic form of incoming words. We show that the magnitude of lexico-semantic prediction error mirrors the functional sensitivity of the N400 to various lexical variables, priming, contextual effects, as well as their higher-order interactions. We further show that the dynamics of the predictive coding algorithm provides a natural explanation for the temporal dynamics of the N400, and a biologically plausible link to neural activity. Together, these findings directly situate the N400 within the broader context of predictive coding research. More generally, they raise the possibility that the brain may use the same computational mechanism for inference across linguistic and non-linguistic domains.
Collapse
Affiliation(s)
- Samer Nour Eddine
- Department of Psychology and Center for Cognitive Science, Tufts University, United States of America.
| | - Trevor Brothers
- Department of Psychology and Center for Cognitive Science, Tufts University, United States of America; Department of Psychology, North Carolina A&T, United States of America
| | - Lin Wang
- Department of Psychology and Center for Cognitive Science, Tufts University, United States of America; Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, United States of America
| | | | - Gina R Kuperberg
- Department of Psychology and Center for Cognitive Science, Tufts University, United States of America; Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, United States of America
| |
Collapse
|
12
|
Verhoef T, Marghetis T, Walker E, Coulson S. Brain responses to a lab-evolved artificial language with space-time metaphors. Cognition 2024; 246:105763. [PMID: 38442586 DOI: 10.1016/j.cognition.2024.105763] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 01/05/2024] [Accepted: 02/26/2024] [Indexed: 03/07/2024]
Abstract
What is the connection between the cultural evolution of a language and the rapid processing response to that language in the brains of individual learners? In an iterated communication study that was conducted previously, participants were asked to communicate temporal concepts such as "tomorrow," "day after," "year," and "past" using vertical movements recorded on a touch screen. Over time, participants developed simple artificial 'languages' that used space metaphorically to communicate in nuanced ways about time. Some conventions appeared rapidly and universally (e.g., using larger vertical movements to convey greater temporal durations). Other conventions required extensive social interaction and exhibited idiosyncratic variation (e.g., using vertical location to convey past or future). Here we investigate whether the brain's response during acquisition of such a language reflects the process by which the language's conventions originally evolved. We recorded participants' EEG as they learned one of these artificial space-time languages. Overall, the brain response to this artificial communication system was language-like, with, for instance, violations to the system's conventions eliciting an N400-like component. Over the course of learning, participants' brain responses developed in ways that paralleled the process by which the language had originally evolved, with early neural sensitivity to violations of a rapidly-evolving universal convention, and slowly developing neural sensitivity to an idiosyncratic convention that required slow social negotiation to emerge. This study opens up exciting avenues of future work to disentangle how neural biases influence learning and transmission in the emergence of structure in language.
Collapse
Affiliation(s)
- Tessa Verhoef
- Leiden Institute of Advanced Computer Science, Leiden University, Gorlaeus Building, Einsteinweg 55, 2333 CC Leiden, the Netherlands; Department of Cognitive Science, University of California, San Diego, Mail Code 0515; 9500, Gilman Drive, La Jolla, CA 92093-0515, USA.
| | - Tyler Marghetis
- Department of Cognitive and Information Sciences, University of California, Merced, 5200 North Lake Rd., Merced, CA 95343, USA
| | - Esther Walker
- Department of Cognitive Science, University of California, San Diego, Mail Code 0515; 9500, Gilman Drive, La Jolla, CA 92093-0515, USA
| | - Seana Coulson
- Department of Cognitive Science, University of California, San Diego, Mail Code 0515; 9500, Gilman Drive, La Jolla, CA 92093-0515, USA
| |
Collapse
|
13
|
Blache P. A neuro-cognitive model of comprehension based on prediction and unification. Front Hum Neurosci 2024; 18:1356541. [PMID: 38655372 PMCID: PMC11035797 DOI: 10.3389/fnhum.2024.1356541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Accepted: 03/20/2024] [Indexed: 04/26/2024] Open
Abstract
Most architectures and models of language processing have been built upon a restricted view of language, which is limited to sentence processing. These approaches fail to capture one primordial characteristic: efficiency. Many facilitation effects are known to be at play in natural situations such as conversation (shallow processing, no real access to the lexicon, etc.) without any impact on the comprehension. In this study, on the basis of a new model integrating into a unique architecture, we present these facilitation effects for accessing the meaning into the classical compositional architecture. This model relies on two mechanisms, prediction and unification, and provides a unique architecture for the description of language processing in its natural environment.
Collapse
Affiliation(s)
- Philippe Blache
- Laboratoire Parole et Langage (LPL-CNRS), Aix-en-Provence, France
- Institute of Language, Communication and the Brain (ILCB), Marseille, France
| |
Collapse
|
14
|
Wang L, Brothers T, Jensen O, Kuperberg GR. Dissociating the pre-activation of word meaning and form during sentence comprehension: Evidence from EEG representational similarity analysis. Psychon Bull Rev 2024; 31:862-873. [PMID: 37783897 PMCID: PMC10985416 DOI: 10.3758/s13423-023-02385-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/10/2023] [Indexed: 10/04/2023]
Abstract
During language comprehension, the processing of each incoming word is facilitated in proportion to its predictability. Here, we asked whether anticipated upcoming linguistic information is actually pre-activated before new bottom-up input becomes available, and if so, whether this pre-activation is limited to the level of semantic features, or whether extends to representations of individual word-forms (orthography/phonology). We carried out Representational Similarity Analysis on EEG data while participants read highly constraining sentences. Prior to the onset of the expected target words, sentence pairs predicting semantically related words (financial "bank" - "loan") and form-related words (financial "bank" - river "bank") produced more similar neural patterns than pairs predicting unrelated words ("bank" - "lesson"). This provides direct neural evidence for item-specific semantic and form predictive pre-activation. Moreover, the semantic pre-activation effect preceded the form pre-activation effect, suggesting that top-down pre-activation is propagated from higher to lower levels of the linguistic hierarchy over time.
Collapse
Affiliation(s)
- Lin Wang
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, 02129, USA.
- Department of Psychology, Tufts University, Medford, MA, USA.
| | - Trevor Brothers
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, 02129, USA
- Department of Psychology, Tufts University, Medford, MA, USA
| | - Ole Jensen
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Gina R Kuperberg
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, 02129, USA
- Department of Psychology, Tufts University, Medford, MA, USA
| |
Collapse
|
15
|
Farrell M, Recanatesi S, Shea-Brown E. From lazy to rich to exclusive task representations in neural networks and neural codes. Curr Opin Neurobiol 2023; 83:102780. [PMID: 37757585 DOI: 10.1016/j.conb.2023.102780] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Revised: 08/04/2023] [Accepted: 08/16/2023] [Indexed: 09/29/2023]
Abstract
Neural circuits-both in the brain and in "artificial" neural network models-learn to solve a remarkable variety of tasks, and there is a great current opportunity to use neural networks as models for brain function. Key to this endeavor is the ability to characterize the representations formed by both artificial and biological brains. Here, we investigate this potential through the lens of recently developing theory that characterizes neural networks as "lazy" or "rich" depending on the approach they use to solve tasks: lazy networks solve tasks by making small changes in connectivity, while rich networks solve tasks by significantly modifying weights throughout the network (including "hidden layers"). We further elucidate rich networks through the lens of compression and "neural collapse", ideas that have recently been of significant interest to neuroscience and machine learning. We then show how these ideas apply to a domain of increasing importance to both fields: extracting latent structures through self-supervised learning.
Collapse
Affiliation(s)
- Matthew Farrell
- John A. Paulson School of Engineering and Applied Sciences, Harvard University and Center for Brain Science, Harvard University, United States
| | - Stefano Recanatesi
- Applied Mathematics, Physiology and Biophysics, and Computational Neuroscience Center, University of Washington, United States
| | - Eric Shea-Brown
- Applied Mathematics, Physiology and Biophysics, and Computational Neuroscience Center, University of Washington, United States.
| |
Collapse
|
16
|
Ryskin R, Nieuwland MS. Prediction during language comprehension: what is next? Trends Cogn Sci 2023; 27:1032-1052. [PMID: 37704456 PMCID: PMC11614350 DOI: 10.1016/j.tics.2023.08.003] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 08/03/2023] [Accepted: 08/04/2023] [Indexed: 09/15/2023]
Abstract
Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.
Collapse
Affiliation(s)
- Rachel Ryskin
- Department of Cognitive and Information Sciences, University of California Merced, 5200 Lake Road, Merced, CA 95343, USA.
| | - Mante S Nieuwland
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands; Donders Institute for Brain, Cognition, and Behaviour, Nijmegen, The Netherlands
| |
Collapse
|
17
|
Wei W, Huang Z, Feng C, Qu Q. Predicting phonological information in language comprehension: evidence from ERP representational similarity analysis and Chinese idioms. Cereb Cortex 2023; 33:9367-9375. [PMID: 37317031 PMCID: PMC10786090 DOI: 10.1093/cercor/bhad209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Revised: 05/22/2023] [Accepted: 05/23/2023] [Indexed: 06/16/2023] Open
Abstract
Do comprehenders predict the meaning and even the phonological form of upcoming words during language comprehension? With a growing body of evidence suggesting that semantic representations may be predicted, the evidence for phonological prediction is less clear and largely derived from studies conducted in languages utilizing an alphabetic script. In this research, we aim to examine the prediction of phonological information in the processing of Chinese idioms through the use of ERP representational similarity analysis (RSA). The study utilizes four-character Chinese idioms, and phonological overlap was manipulated by varying the syllable at the idiom-final part between idiom pairs so that pairs of idioms share a syllable (i.e. within-pairs) or not (between-pairs). We quantified the similarity between patterns of neural activity of idioms for within- and between-pairs. RSA results revealed greater similarity in neural activity patterns for idioms within-pairs, compared with between-pairs, and critically this similarity effect was observed prior to the presentation of the phonological similarity, providing evidence for the pre-activation of upcoming phonological information, under circumstances that encourage predictive processing.
Collapse
Affiliation(s)
- Wei Wei
- Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China
- Sino-Danish College, University of Chinese Academy of Sciences, Beijing 100040, China
| | - Zirui Huang
- Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China
| | - Chen Feng
- Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing 100040, China
| | - Qingqing Qu
- Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing 100040, China
| |
Collapse
|
18
|
Tezcan F, Weissbart H, Martin AE. A tradeoff between acoustic and linguistic feature encoding in spoken language comprehension. eLife 2023; 12:e82386. [PMID: 37417736 PMCID: PMC10328533 DOI: 10.7554/elife.82386] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Accepted: 06/18/2023] [Indexed: 07/08/2023] Open
Abstract
When we comprehend language from speech, the phase of the neural response aligns with particular features of the speech input, resulting in a phenomenon referred to as neural tracking. In recent years, a large body of work has demonstrated the tracking of the acoustic envelope and abstract linguistic units at the phoneme and word levels, and beyond. However, the degree to which speech tracking is driven by acoustic edges of the signal, or by internally-generated linguistic units, or by the interplay of both, remains contentious. In this study, we used naturalistic story-listening to investigate (1) whether phoneme-level features are tracked over and above acoustic edges, (2) whether word entropy, which can reflect sentence- and discourse-level constraints, impacted the encoding of acoustic and phoneme-level features, and (3) whether the tracking of acoustic edges was enhanced or suppressed during comprehension of a first language (Dutch) compared to a statistically familiar but uncomprehended language (French). We first show that encoding models with phoneme-level linguistic features, in addition to acoustic features, uncovered an increased neural tracking response; this signal was further amplified in a comprehended language, putatively reflecting the transformation of acoustic features into internally generated phoneme-level representations. Phonemes were tracked more strongly in a comprehended language, suggesting that language comprehension functions as a neural filter over acoustic edges of the speech signal as it transforms sensory signals into abstract linguistic units. We then show that word entropy enhances neural tracking of both acoustic and phonemic features when sentence- and discourse-context are less constraining. When language was not comprehended, acoustic features, but not phonemic ones, were more strongly modulated, but in contrast, when a native language is comprehended, phoneme features are more strongly modulated. Taken together, our findings highlight the flexible modulation of acoustic, and phonemic features by sentence and discourse-level constraint in language comprehension, and document the neural transformation from speech perception to language comprehension, consistent with an account of language processing as a neural filter from sensory to abstract representations.
Collapse
Affiliation(s)
- Filiz Tezcan
- Language and Computation in Neural Systems Group, Max Planck Institute for PsycholinguisticsNijmegenNetherlands
| | - Hugo Weissbart
- Donders Centre for Cognitive Neuroimaging, Radboud UniversityNijmegenNetherlands
| | - Andrea E Martin
- Language and Computation in Neural Systems Group, Max Planck Institute for PsycholinguisticsNijmegenNetherlands
- Donders Centre for Cognitive Neuroimaging, Radboud UniversityNijmegenNetherlands
| |
Collapse
|
19
|
Cocquyt EM, Depuydt E, Santens P, van Mierlo P, Duyck W, Szmalec A, De Letter M. Effects of Healthy Aging and Gender on the Electrophysiological Correlates of Semantic Sentence Comprehension: The Development of Dutch Normative Data. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2023; 66:1694-1717. [PMID: 37093923 DOI: 10.1044/2023_jslhr-22-00545] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
PURPOSE The clinical use of event-related potentials in patients with language disorders is increasingly acknowledged. For this purpose, normative data should be available. Within this context, healthy aging and gender effects on the electrophysiological correlates of semantic sentence comprehension were investigated. METHOD One hundred and ten healthy subjects (55 men and 55 women), divided among three age groups (young, middle aged, and elderly), performed a semantic sentence congruity task in the visual modality during electroencephalographic recording. RESULTS The early visual complex was affected by increasing age as shown by smaller P2 amplitudes in the elderly compared to the young. Moreover, the N400 effect in the elderly was smaller than in the young and was delayed compared to latency measures in both middle-aged and young subjects. The topography of age-related amplitude changes of the N400 effect appeared to be gender specific. The late positive complex effect was increased at frontal electrode sites from middle age on, but this was not statistically significant. No gender effects were detected regarding the early P1, N1, and P2, or the late positive complex effect. CONCLUSION Especially aging effects were found during semantic sentence comprehension, and this from the level of perceptual processing on. Normative data are now available for clinical use.
Collapse
Affiliation(s)
| | - Emma Depuydt
- Medical Image and Signal Processing Group, Department of Electronics and Information Systems, Ghent University, Belgium
| | | | - Pieter van Mierlo
- Medical Image and Signal Processing Group, Department of Electronics and Information Systems, Ghent University, Belgium
| | - Wouter Duyck
- Department of Experimental Psychology, Faculty of Psychology and Educational Sciences, Ghent University, Belgium
| | - Arnaud Szmalec
- Department of Experimental Psychology, Faculty of Psychology and Educational Sciences, Ghent University, Belgium
- Psychological Sciences Research Institute, Université catholique de Louvain, Louvain-la-Neuve, Belgium
| | - Miet De Letter
- Department of Rehabilitation Sciences, Ghent University, Belgium
| |
Collapse
|
20
|
Seghier ML. Multiple functions of the angular gyrus at high temporal resolution. Brain Struct Funct 2023; 228:7-46. [PMID: 35674917 DOI: 10.1007/s00429-022-02512-y] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Accepted: 05/22/2022] [Indexed: 02/07/2023]
Abstract
Here, the functions of the angular gyrus (AG) are evaluated in the light of current evidence from transcranial magnetic/electric stimulation (TMS/TES) and EEG/MEG studies. 65 TMS/TES and 52 EEG/MEG studies were examined in this review. TMS/TES literature points to a causal role in semantic processing, word and number processing, attention and visual search, self-guided movement, memory, and self-processing. EEG/MEG studies reported AG effects at latencies varying between 32 and 800 ms in a wide range of domains, with a high probability to detect an effect at 300-350 ms post-stimulus onset. A three-phase unifying model revolving around the process of sensemaking is then suggested: (1) early AG involvement in defining the current context, within the first 200 ms, with a bias toward the right hemisphere; (2) attention re-orientation and retrieval of relevant information within 200-500 ms; and (3) cross-modal integration at late latencies with a bias toward the left hemisphere. This sensemaking process can favour accuracy (e.g. for word and number processing) or plausibility (e.g. for comprehension and social cognition). Such functions of the AG depend on the status of other connected regions. The much-debated semantic role is also discussed as follows: (1) there is a strong TMS/TES evidence for a causal semantic role, (2) current EEG/MEG evidence is however weak, but (3) the existing arguments against a semantic role for the AG are not strong. Some outstanding questions for future research are proposed. This review recognizes that cracking the role(s) of the AG in cognition is possible only when its exact contributions within the default mode network are teased apart.
Collapse
Affiliation(s)
- Mohamed L Seghier
- Department of Biomedical Engineering, Khalifa University of Science and Technology, Abu Dhabi, UAE. .,Healthcare Engineering Innovation Center (HEIC), Khalifa University of Science and Technology, Abu Dhabi, UAE.
| |
Collapse
|
21
|
Lee SY, Nam Y. Electrophysiological evidence for a subject-first strategy in visually situated auditory sentence processing in Korean. Acta Psychol (Amst) 2022; 231:103799. [PMID: 36473388 DOI: 10.1016/j.actpsy.2022.103799] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2021] [Revised: 11/06/2022] [Accepted: 11/20/2022] [Indexed: 12/11/2022] Open
Abstract
This study investigated a subject-first strategy in prediction mechanism in visually situated sentence processing in Korean, using event-related potentials (ERPs). According to the subject-first strategy, parsers tend to generate sentences conforming to canonical sentence word order (i.e., SOV in Korean), subject-first sentence, mapping conceptually more prominent referent such as agent of the event on the subject position of the sentence. Therefore, in the predictive mechanism of language comprehension, the subject is pre-activated and anticipated for the first NP of the sentence at the initial phase of bottom-up language processing. This study tested this subject-first strategy in Korean by examining brain responses to object-initial sentences (OV) compared with subject-initial sentences (SV) under the context of clear thematic role relations set by a visual image. The results of an ERP experiment with 30 native Korean speakers identified neural effects for object-initial sentences compared with subject-initial sentences at the NP and Verb, reflecting a conflict between the pre-activated representation in the parser's mind and the encountered bottom-up input. An N400 effect was elicited at the NP, as early as at the noun, not at the following object case marker. Late frontal positivity (LFP) was also found in the sentence-final verb, proving the processing difficulty of non-canonical object-initial sentences compared with canonical subject-initial sentences. These results indicate that Korean native speakers build linguistic representation conforming to a canonical sentence in SOV language in the predictive mechanism supporting subject-first strategy but revise the predicted event structure rapidly upon newly encountering input.
Collapse
Affiliation(s)
- Sun-Young Lee
- Division of English, Cyber Hankuk University of Foreign Studies, 107 Immun-ro, Dongdaemun-gu, Seoul 02450, South Korea.
| | - Yunju Nam
- Department of German Language and Literature, Hanyang University, 222, Wangsimni-ro, Seongdong-gu, Seoul 04763, South Korea.
| |
Collapse
|
22
|
Maran M, Numssen O, Hartwigsen G, Zaccarella E. Online neurostimulation of Broca's area does not interfere with syntactic predictions: A combined TMS-EEG approach to basic linguistic combination. Front Psychol 2022; 13:968836. [PMID: 36619118 PMCID: PMC9815778 DOI: 10.3389/fpsyg.2022.968836] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Accepted: 09/13/2022] [Indexed: 01/11/2023] Open
Abstract
Categorical predictions have been proposed as the key mechanism supporting the fast pace of syntactic composition in language. Accordingly, grammar-based expectations are formed-e.g., the determiner "a" triggers the prediction for a noun-and facilitate the analysis of incoming syntactic information, which is then checked against a single or few other word categories. Previous functional neuroimaging studies point towards Broca's area in the left inferior frontal gyrus (IFG) as one fundamental cortical region involved in categorical prediction during incremental language processing. Causal evidence for this hypothesis is however still missing. In this study, we combined Electroencephalography (EEG) and Transcranial Magnetic Stimulation (TMS) to test whether Broca's area is functionally relevant in predictive mechanisms for language. We transiently perturbed Broca's area during the first word in a two-word construction, while simultaneously measuring the Event-Related Potential (ERP) correlates of syntactic composition. We reasoned that if Broca's area is involved in predictive mechanisms for syntax, disruptive TMS during the first word would mitigate the difference in the ERP responses for predicted and unpredicted categories in basic two-word constructions. Contrary to this hypothesis, perturbation of Broca's area at the predictive stage did not affect the ERP correlates of basic composition. The correlation strength between the electrical field induced by TMS and the ERP responses further confirmed this pattern. We discuss the present results considering an alternative account of the role of Broca's area in syntactic composition, namely the bottom-up integration of words into constituents, and of compensatory mechanisms within the language predictive network.
Collapse
Affiliation(s)
- Matteo Maran
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany,International Max Planck Research School on Neuroscience of Communication: Function, Structure, and Plasticity, Leipzig, Germany,*Correspondence: Matteo Maran,
| | - Ole Numssen
- Lise Meitner Research Group Cognition and Plasticity, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Gesa Hartwigsen
- Lise Meitner Research Group Cognition and Plasticity, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Emiliano Zaccarella
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| |
Collapse
|
23
|
Mangardich H, Sabbagh MA. Event-related potential studies of cross-situational word learning in four-year-old children. J Exp Child Psychol 2022; 222:105468. [DOI: 10.1016/j.jecp.2022.105468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Revised: 03/18/2022] [Accepted: 05/02/2022] [Indexed: 10/18/2022]
|
24
|
Poulton VR, Nieuwland MS. Can You Hear What's Coming? Failure to Replicate ERP Evidence for Phonological Prediction. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2022; 3:556-574. [PMID: 37215344 PMCID: PMC10158594 DOI: 10.1162/nol_a_00078] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Accepted: 07/18/2022] [Indexed: 05/24/2023]
Abstract
Prediction-based theories of language comprehension assume that listeners predict both the meaning and phonological form of likely upcoming words. In alleged event-related potential (ERP) demonstrations of phonological prediction, prediction-mismatching words elicit a phonological mismatch negativity (PMN), a frontocentral negativity that precedes the centroparietal N400 component. However, classification and replicability of the PMN has proven controversial, with ongoing debate on whether the PMN is a distinct component or merely an early part of the N400. In this electroencephalography (EEG) study, we therefore attempted to replicate the PMN effect and its separability from the N400, using a participant sample size (N = 48) that was more than double that of previous studies. Participants listened to sentences containing either a predictable word or an unpredictable word with/without phonological overlap with the predictable word. Preregistered analyses revealed a widely distributed negative-going ERP in response to unpredictable words in both the early (150-250 ms) and the N400 (300-500 ms) time windows. Bayes factor analysis yielded moderate evidence against a different scalp distribution of the effects in the two time windows. Although our findings do not speak against phonological prediction during sentence comprehension, they do speak against the PMN effect specifically as a marker of phonological prediction mismatch. Instead of an PMN effect, our results demonstrate the early onset of the auditory N400 effect associated with unpredictable words. Our failure to replicate further highlights the risk associated with commonly employed data-contingent analyses (e.g., analyses involving time windows or electrodes that were selected based on visual inspection) and small sample sizes in the cognitive neuroscience of language.
Collapse
Affiliation(s)
- Victoria R. Poulton
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK
| | - Mante S. Nieuwland
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
- Donders Institute for Brain, Cognition and Behaviour, Nijmegen, Radboud University, The Netherlands
| |
Collapse
|
25
|
Heilbron M, Armeni K, Schoffelen JM, Hagoort P, de Lange FP. A hierarchy of linguistic predictions during natural language comprehension. Proc Natl Acad Sci U S A 2022; 119:e2201968119. [PMID: 35921434 PMCID: PMC9371745 DOI: 10.1073/pnas.2201968119] [Citation(s) in RCA: 106] [Impact Index Per Article: 35.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Accepted: 06/28/2022] [Indexed: 02/05/2023] Open
Abstract
Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analyzing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable neural signatures of predictions about syntactic category (parts of speech), phonemes, and semantics. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction.
Collapse
Affiliation(s)
- Micha Heilbron
- Donders Institute, Radboud University, 6525 EN Nijmegen, The Netherlands
- Max Planck Institute for Psycholinguistics, 6525 XD Nijmegen, The Netherlands
| | - Kristijan Armeni
- Donders Institute, Radboud University, 6525 EN Nijmegen, The Netherlands
| | | | - Peter Hagoort
- Donders Institute, Radboud University, 6525 EN Nijmegen, The Netherlands
- Max Planck Institute for Psycholinguistics, 6525 XD Nijmegen, The Netherlands
| | - Floris P. de Lange
- Donders Institute, Radboud University, 6525 EN Nijmegen, The Netherlands
| |
Collapse
|
26
|
Heilbron M, Armeni K, Schoffelen JM, Hagoort P, de Lange FP. A hierarchy of linguistic predictions during natural language comprehension. Proc Natl Acad Sci U S A 2022; 119:e2201968119. [PMID: 35921434 DOI: 10.1101/2020.12.03.410399] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/21/2023] Open
Abstract
Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analyzing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable neural signatures of predictions about syntactic category (parts of speech), phonemes, and semantics. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction.
Collapse
Affiliation(s)
- Micha Heilbron
- Donders Institute, Radboud University, 6525 EN Nijmegen, The Netherlands
- Max Planck Institute for Psycholinguistics, 6525 XD Nijmegen, The Netherlands
| | - Kristijan Armeni
- Donders Institute, Radboud University, 6525 EN Nijmegen, The Netherlands
| | | | - Peter Hagoort
- Donders Institute, Radboud University, 6525 EN Nijmegen, The Netherlands
- Max Planck Institute for Psycholinguistics, 6525 XD Nijmegen, The Netherlands
| | - Floris P de Lange
- Donders Institute, Radboud University, 6525 EN Nijmegen, The Netherlands
| |
Collapse
|
27
|
Knoeferle P, Mishra RK, Peña M. Editorial: Socially Situated? Effects of Social and Cultural Context on Language Processing and Learning. Front Psychol 2022; 13:855733. [PMID: 35418910 PMCID: PMC8995506 DOI: 10.3389/fpsyg.2022.855733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2022] [Accepted: 02/15/2022] [Indexed: 11/24/2022] Open
Affiliation(s)
- Pia Knoeferle
- Institute of German Studies and Linguistics, Humboldt-Universität zu Berlin, Berlin, Germany.,Berlin School of Mind and Brain, Berlin, Germany.,Einstein Center for Neursciences Berlin, Berlin, Germany
| | - Ramesh Kumar Mishra
- Center for Neural and Cognitive Sciences, University of Hyderabad, Hyderabad, India
| | - Marcela Peña
- Cognitive Neuroscience Laboratory, Pontificia Universidad Católica de Chile, Santiago, Chile.,Centro Nacional de Inteligencia Artificial, Santiago, Chile
| |
Collapse
|
28
|
Huang Z, Yang S, Xue L, Yang H, Lv Y, Zhao J. Level of Orthographic Knowledge Helps to Reveal Automatic Predictions in Visual Word Processing. Front Neurosci 2022; 15:809574. [PMID: 35221891 PMCID: PMC8864072 DOI: 10.3389/fnins.2021.809574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Accepted: 12/27/2021] [Indexed: 11/13/2022] Open
Abstract
The brain generates predictions about visual word forms to support efficient reading. The “interactive account” suggests that the predictions in visual word processing can be strategic or automatic (non-strategic). Strategic predictions are frequently demonstrated in studies that manipulated task demands, however, few studies have investigated automatic predictions. Orthographic knowledge varies greatly among individuals and it offers a unique opportunity in revealing automatic predictions. The present study grouped the participants by level of orthographic knowledge and recorded EEGs in a non-linguistic color matching task. The visual word-selective N170 response was much stronger to pseudo than to real characters in participants with low orthographic knowledge, but not in those with high orthographic knowledge. Previous work on predictive coding has demonstrated that N170 is a good index for prediction errors, i.e., the mismatches between predictions and visual inputs. The present findings provide unambiguous evidence that automatic predictions modulate the early stage of visual word processing.
Collapse
Affiliation(s)
- Zehao Huang
- Jing Hengyi School of Education, Hangzhou Normal University, Hangzhou, China
- Zhejiang Key Laboratory for Research in Assessment of Cognitive Impairments, Hangzhou, China
- Center for Cognition and Brain Disorders, Hangzhou Normal University, Hangzhou, China
| | - Shimeng Yang
- Jing Hengyi School of Education, Hangzhou Normal University, Hangzhou, China
- Zhejiang Key Laboratory for Research in Assessment of Cognitive Impairments, Hangzhou, China
- Center for Cognition and Brain Disorders, Hangzhou Normal University, Hangzhou, China
| | - Licheng Xue
- Institute for Brain Research and Rehabilitation, South China Normal University, Guangzhou, China
- Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education, Guangzhou, China
| | - Hang Yang
- Faculty of Science and Engineering, Bernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen, Groningen, Netherlands
| | - Yating Lv
- Zhejiang Key Laboratory for Research in Assessment of Cognitive Impairments, Hangzhou, China
- Center for Cognition and Brain Disorders, Hangzhou Normal University, Hangzhou, China
| | - Jing Zhao
- Jing Hengyi School of Education, Hangzhou Normal University, Hangzhou, China
- Zhejiang Key Laboratory for Research in Assessment of Cognitive Impairments, Hangzhou, China
- *Correspondence: Jing Zhao,
| |
Collapse
|
29
|
Federmeier KD. Connecting and considering: Electrophysiology provides insights into comprehension. Psychophysiology 2022; 59:e13940. [PMID: 34520568 PMCID: PMC9009268 DOI: 10.1111/psyp.13940] [Citation(s) in RCA: 48] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2021] [Revised: 08/27/2021] [Accepted: 08/30/2021] [Indexed: 11/29/2022]
Abstract
The ability to rapidly and systematically access knowledge stored in long-term memory in response to incoming sensory information-that is, to derive meaning from the world-lies at the core of human cognition. Research using methods that can precisely track brain activity over time has begun to reveal the multiple cognitive and neural mechanisms that make this possible. In this article, I delineate how a process of connecting affords an effortless, continuous infusion of meaning into human perception. In a relatively invariant time window, uncovered through studies using the N400 component of the event-related potential, incoming sensory information naturally induces a graded landscape of activation across long-term semantic memory, creating what might be called "proto-concepts". Connecting can be (but is not always) followed by a process of further considering those activations, wherein a set of more attentionally demanding "active comprehension" mechanisms mediate the selection, augmentation, and transformation of the initial semantic representations. The result is a limited set of more stable bindings that can be arranged in time or space, revised as needed, and brought to awareness. With this research, we are coming closer to understanding how the human brain is able to fluidly link sensation to experience, to appreciate language sequences and event structures, and, sometimes, to even predict what might be coming up next.
Collapse
Affiliation(s)
- Kara D Federmeier
- Department of Psychology, Program in Neuroscience, and the Beckman Institute for Advanced Science and Technology, University of Illinois, Champaign, Illinois, USA
| |
Collapse
|
30
|
Eisenhauer S, Gagl B, Fiebach CJ. Predictive pre-activation of orthographic and lexical-semantic representations facilitates visual word recognition. Psychophysiology 2021; 59:e13970. [PMID: 34813664 DOI: 10.1111/psyp.13970] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2020] [Revised: 09/24/2021] [Accepted: 10/26/2021] [Indexed: 11/30/2022]
Abstract
To a crucial extent, the efficiency of reading results from the fact that visual word recognition is faster in predictive contexts. Predictive coding models suggest that this facilitation results from pre-activation of predictable stimulus features across multiple representational levels before stimulus onset. Still, it is not sufficiently understood which aspects of the rich set of linguistic representations that are activated during reading-visual, orthographic, phonological, and/or lexical-semantic-contribute to context-dependent facilitation. To investigate in detail which linguistic representations are pre-activated in a predictive context and how they affect subsequent stimulus processing, we combined a well-controlled repetition priming paradigm, including words and pseudowords (i.e., pronounceable nonwords), with behavioral and magnetoencephalography measurements. For statistical analysis, we used linear mixed modeling, which we found had a higher statistical power compared to conventional multivariate pattern decoding analysis. Behavioral data from 49 participants indicate that word predictability (i.e., context present vs. absent) facilitated orthographic and lexical-semantic, but not visual or phonological processes. Magnetoencephalography data from 38 participants show sustained activation of orthographic and lexical-semantic representations in the interval before processing the predicted stimulus, suggesting selective pre-activation at multiple levels of linguistic representation as proposed by predictive coding. However, we found more robust lexical-semantic representations when processing predictable in contrast to unpredictable letter strings, and pre-activation effects mainly resembled brain responses elicited when processing the expected letter string. This finding suggests that pre-activation did not result in "explaining away" predictable stimulus features, but rather in a "sharpening" of brain responses involved in word processing.
Collapse
Affiliation(s)
- Susanne Eisenhauer
- Department of Psychology, Goethe University Frankfurt, Frankfurt am Main, Germany
| | - Benjamin Gagl
- Department of Psychology, Goethe University Frankfurt, Frankfurt am Main, Germany.,Center for Individual Development and Adaptive Education of Children at Risk (IDeA), Frankfurt am Main, Germany
| | - Christian J Fiebach
- Department of Psychology, Goethe University Frankfurt, Frankfurt am Main, Germany.,Center for Individual Development and Adaptive Education of Children at Risk (IDeA), Frankfurt am Main, Germany.,Brain Imaging Center, Goethe University Frankfurt, Frankfurt am Main, Germany
| |
Collapse
|
31
|
Ferreira F, Qiu Z. Predicting syntactic structure. Brain Res 2021; 1770:147632. [PMID: 34453937 DOI: 10.1016/j.brainres.2021.147632] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Revised: 08/20/2021] [Accepted: 08/21/2021] [Indexed: 11/19/2022]
Abstract
Prediction in language processing has been a topic of major interest in psycholinguistics for at least the last two decades, but most investigations focus on semantic rather than syntactic prediction. This review begins with a discussion of some influential models of parsing which assume that comprehenders have the ability to anticipate syntactic nodes, beginning with left-corner parsers and the garden-path model and ending with current information-theoretic approaches that emphasize online probabilistic prediction. We then turn to evidence for the prediction of specific syntactic forms, including coordinate clauses and noun phrases, verb arguments, and individual nouns, as well as studies that use morphosyntactic constraints to assess whether a specific semantic prediction has been made. The last section considers the implications of syntactic prediction for theories of language architecture and describes four avenues for future research.
Collapse
Affiliation(s)
| | - Zhuang Qiu
- University of California, Davis, United States.
| |
Collapse
|
32
|
Dave S, Brothers T, Hoversten LJ, Traxler MJ, Swaab TY. Cognitive control mediates age-related changes in flexible anticipatory processing during listening comprehension. Brain Res 2021; 1768:147573. [PMID: 34216583 PMCID: PMC8403152 DOI: 10.1016/j.brainres.2021.147573] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Revised: 05/16/2021] [Accepted: 06/25/2021] [Indexed: 11/21/2022]
Abstract
Effective listening comprehension not only requires processing local linguistic input, but also necessitates incorporating contextual cues available in the global communicative environment. Local sentence processing can be facilitated by pre-activation of likely upcoming input, or predictive processing. Recent evidence suggests that young adults can flexibly adapt local predictive processes based on cues provided by the global communicative environment, such as the reliability of specific speakers. Whether older comprehenders can also flexibly adapt to global contextual cues is currently unknown. Moreover, it is unclear whether the underlying mechanisms supporting local predictive processing differ from those supporting adaptation to global contextual cues. Critically, it is unclear whether these mechanisms change as a function of typical aging. We examined the flexibility of prediction in young and older adults by presenting sentences from speakers whose utterances were typically more or less predictable (i.e., reliable speakers who produced expected words 80% of the time, versus unreliable speakers who produced expected words 20% of the time). For young listeners, global speaker reliability cues modulated neural effects of local predictability on the N400. In contrast, older adults, on average, did not show global modulation of local processing. Importantly, however, cognitive control (i.e., Stroop interference effects) mediated age-related reductions in sensitivity to the reliability of the speaker. Both young and older adults with high cognitive control showed greater N400 effects of predictability during sentences produced by a reliable speaker, suggesting that cognitive control is required to regulate the strength of top-down predictions based on global contextual information. Critically, cognitive control predicted sensitivity to global speaker-specific information but not local predictability cues, suggesting that predictive processing in local sentence contexts may be supported by separable neural mechanisms from adaptation of prediction as a function of global context. These results have important implications for interpreting age-related change in predictive processing, and for drawing more generalized conclusions regarding domain-general versus language-specific accounts of prediction.
Collapse
Affiliation(s)
- Shruti Dave
- Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
| | | | - Liv J Hoversten
- Department of Psychology, University of California Santa Cruz, Santa Cruz, CA, USA
| | - Matthew J Traxler
- Department of Psychology and Center for Mind and Brain, University of California Davis, Davis, CA, USA
| | - Tamara Y Swaab
- Department of Psychology and Center for Mind and Brain, University of California Davis, Davis, CA, USA
| |
Collapse
|
33
|
Zhou L, Perfetti C. Consistency and regularity effects in character identification: A greater role for global than local mapping congruence. BRAIN AND LANGUAGE 2021; 221:104997. [PMID: 34399241 DOI: 10.1016/j.bandl.2021.104997] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/23/2021] [Revised: 07/14/2021] [Accepted: 07/15/2021] [Indexed: 06/13/2023]
Abstract
Consistency and regularity, concepts that arise, respectively, from the connectionist and classical cognitive modeling work in alphabetic reading, are two ways to characterize the orthography-to-phonology mappings of written languages. These concepts have been applied to Chinese reading research despite important differences across writing systems, with mixed results concerning their relative importance. The present study of covert naming in Chinese is distinctive in testing the ERP effects of regularity and consistency in a fully orthogonal design. We found that consistency, but not regularity, affected the N170, P200 and N400 as well as pronunciation transcription accuracies, demonstrating a more prominent role of consistency than regularity in character naming, consistent with conclusions from English word naming. To capture a generalization across writing systems, we propose mapping congruence as a writing-system-independent way of referring to orthography-to-phonology mappings and illustrate these congruence effects in an interactive framework of character identification.
Collapse
Affiliation(s)
- Lin Zhou
- Learning Research and Development Center, University of Pittsburgh, Pittsburgh, PA, USA; Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA.
| | - Charles Perfetti
- Learning Research and Development Center, University of Pittsburgh, Pittsburgh, PA, USA; Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA, USA
| |
Collapse
|
34
|
Gallo F, Bermudez-Margaretto B, Shtyrov Y, Abutalebi J, Kreiner H, Chitaya T, Petrova A, Myachykov A. First Language Attrition: What It Is, What It Isn't, and What It Can Be. Front Hum Neurosci 2021; 15:686388. [PMID: 34557079 PMCID: PMC8452950 DOI: 10.3389/fnhum.2021.686388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Accepted: 08/16/2021] [Indexed: 11/30/2022] Open
Abstract
This review aims at clarifying the concept of first language attrition by tracing its limits, identifying its phenomenological and contextual constraints, discussing controversies associated with its definition, and suggesting potential directions for future research. We start by reviewing different definitions of attrition as well as associated inconsistencies. We then discuss the underlying mechanisms of first language attrition and review available evidence supporting different background hypotheses. Finally, we attempt to provide the groundwork to build a unified theoretical framework allowing for generalizable results. To this end, we suggest the deployment of a rigorous neuroscientific approach, in search of neural markers of first language attrition in different linguistic domains, putting forward hypothetical experimental ways to identify attrition's neural traces and formulating predictions for each of the proposed experimental paradigms.
Collapse
Affiliation(s)
- Federico Gallo
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Russian Federation, Moscow, Russia
- Centre for Neurolinguistics and Psycholinguistics (CNPL), Vita-Salute San Raffaele University, Milan, Italy
| | - Beatriz Bermudez-Margaretto
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Russian Federation, Moscow, Russia
| | - Yury Shtyrov
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Russian Federation, Moscow, Russia
- Center of Functionally Integrative Neuroscience (CFIN), Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Jubin Abutalebi
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Russian Federation, Moscow, Russia
- Centre for Neurolinguistics and Psycholinguistics (CNPL), Vita-Salute San Raffaele University, Milan, Italy
| | - Hamutal Kreiner
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Russian Federation, Moscow, Russia
- Department of Behavioral Sciences, Linguistic Cognition Laboratory, Ruppin Academic Center, Emek Hefer, Israel
| | - Tamara Chitaya
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Russian Federation, Moscow, Russia
| | - Anna Petrova
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Russian Federation, Moscow, Russia
| | - Andriy Myachykov
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Russian Federation, Moscow, Russia
- Department of Psychology, Northumbria University, Newcastle upon Tyne, United Kingdom
| |
Collapse
|
35
|
Kessler R, Weber A, Friedrich CK. Activation of Literal Word Meanings in Idioms: Evidence from Eye-tracking and ERP Experiments. LANGUAGE AND SPEECH 2021; 64:594-624. [PMID: 32715872 PMCID: PMC8406370 DOI: 10.1177/0023830920943625] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
How the language processing system handles formulaic language such as idioms is a matter of debate. We investigated the activation of constituent meanings by means of predictive processing in an eye-tracking experiment and in two ERP experiments (auditory and visual). In the eye-tracking experiment, German-speaking participants listened to idioms in which the final word was excised (Hannes let the cat out of the . . .). Well before the offset of these idiom fragments, participants fixated on the correct idiom completion (bag) more often than on unrelated distractors (stomach). Moreover, there was an early fixation bias towards semantic associates (basket) of the correct completion, which ended shortly after the offset of the fragment. In the ERP experiments, sentences (spoken or written) either contained complete idioms, or the final word of the idiom was replaced with a semantic associate or with an unrelated word. Across both modalities, ERPs reflected facilitated processing of correct completions across several regions of interest (ROIs) and time windows. Facilitation of semantic associates was only reliably evident in early components for auditory idiom processing. The ERP findings for spoken idioms compliment the eye-tracking data by pointing to early decompositional processing of idioms. It seems that in spoken idiom processing, holistic representations do not solely determine lexical processing.
Collapse
|
36
|
Listeners track talker-specific prosody to deal with talker-variability. Brain Res 2021; 1769:147605. [PMID: 34363790 DOI: 10.1016/j.brainres.2021.147605] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Revised: 07/21/2021] [Accepted: 07/27/2021] [Indexed: 11/20/2022]
Abstract
One of the challenges in speech perception is that listeners must deal with considerable segmental and suprasegmental variability in the acoustic signal due to differences between talkers. Most previous studies have focused on how listeners deal with segmental variability. In this EEG experiment, we investigated whether listeners track talker-specific usage of suprasegmental cues to lexical stress to recognize spoken words correctly. In a three-day training phase, Dutch participants learned to map non-word minimal stress pairs onto different object referents (e.g., USklot meant "lamp"; usKLOT meant "train"). These non-words were produced by two male talkers. Critically, each talker used only one suprasegmental cue to signal stress (e.g., Talker A used only F0 and Talker B only intensity). We expected participants to learn which talker used which cue to signal stress. In the test phase, participants indicated whether spoken sentences including these non-words were correct ("The word for lamp is…"). We found that participants were slower to indicate that a stimulus was correct if the non-word was produced with the unexpected cue (e.g., Talker A using intensity). That is, if in training Talker A used F0 to signal stress, participants experienced a mismatch between predicted and perceived phonological word-forms if, at test, Talker A unexpectedly used intensity to cue stress. In contrast, the N200 amplitude, an event-related potential related to phonological prediction, was not modulated by the cue mismatch. Theoretical implications of these contrasting results are discussed. The behavioral findings illustrate talker-specific prediction of prosodic cues, picked up through perceptual learning during training.
Collapse
|
37
|
Pierotti E, Coffey-Corina S, Schaefer T, Corina DP. Semantic word integration in children with cochlear implants: Electrophysiological evidence. LANGUAGE, COGNITION AND NEUROSCIENCE 2021; 37:224-240. [PMID: 35187189 PMCID: PMC8849536 DOI: 10.1080/23273798.2021.1957954] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Accepted: 06/15/2021] [Indexed: 06/14/2023]
Abstract
Differential auditory experiences of children with hearing-loss who receive cochlear implants (CIs) may influence the integration of lexical and conceptual information. Here we measured event-related potentials during a word-picture priming task in CI-using children (n = 29, mean age = 81 months) and typically-hearing children (n = 19, mean age = 75 months) while they viewed audiovisual-word primes and picture targets that were semantically congruent or incongruent. In both groups, semantic relatedness modulated ERP amplitude 300-500ms after picture onset, signifying an N400 semantic effect. Critically, the CI-using children's responses to unrelated pairs were significantly more negative than hearing children's responses. Group differences were mirrored in an earlier 150-275ms time window associated with a P2 response. The present findings suggest attentional and/or strategic differences impact semantic processing and contribute to the N400 differences observed between groups.
Collapse
Affiliation(s)
- Elizabeth Pierotti
- Center for Mind and Brain, University of California, Davis, CA
- Department of Psychology, University of California, Davis, CA
| | | | - Tristan Schaefer
- Center for Mind and Brain, University of California, Davis, CA
- Department of Linguistics, University of California, Davis, CA
| | - David P. Corina
- Center for Mind and Brain, University of California, Davis, CA
- Department of Psychology, University of California, Davis, CA
- Department of Linguistics, University of California, Davis, CA
| |
Collapse
|
38
|
Ten Oever S, Martin AE. An oscillating computational model can track pseudo-rhythmic speech by using linguistic predictions. eLife 2021; 10:68066. [PMID: 34338196 PMCID: PMC8328513 DOI: 10.7554/elife.68066] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2021] [Accepted: 07/16/2021] [Indexed: 11/19/2022] Open
Abstract
Neuronal oscillations putatively track speech in order to optimize sensory processing. However, it is unclear how isochronous brain oscillations can track pseudo-rhythmic speech input. Here we propose that oscillations can track pseudo-rhythmic speech when considering that speech time is dependent on content-based predictions flowing from internal language models. We show that temporal dynamics of speech are dependent on the predictability of words in a sentence. A computational model including oscillations, feedback, and inhibition is able to track pseudo-rhythmic speech input. As the model processes, it generates temporal phase codes, which are a candidate mechanism for carrying information forward in time. The model is optimally sensitive to the natural temporal speech dynamics and can explain empirical data on temporal speech illusions. Our results suggest that speech tracking does not have to rely only on the acoustics but could also exploit ongoing interactions between oscillations and constraints flowing from internal language models.
Collapse
Affiliation(s)
- Sanne Ten Oever
- Language and Computation in Neural Systems group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands.,Donders Centre for Cognitive Neuroimaging, Radboud University, Nijmegen, Netherlands.,Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
| | - Andrea E Martin
- Language and Computation in Neural Systems group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands.,Donders Centre for Cognitive Neuroimaging, Radboud University, Nijmegen, Netherlands
| |
Collapse
|
39
|
Yacovone A, Moya E, Snedeker J. Unexpected words or unexpected languages? Two ERP effects of code-switching in naturalistic discourse. Cognition 2021; 215:104814. [PMID: 34303181 DOI: 10.1016/j.cognition.2021.104814] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2019] [Revised: 06/10/2021] [Accepted: 06/13/2021] [Indexed: 11/19/2022]
Abstract
Bilingual speakers often switch between languages in conversation without any advance notice. Psycholinguistic research has found that these language shifts (or code-switches) can be costly for comprehenders in certain situations. The present study explores the nature of these costs by comparing code-switches to other types of unexpected linguistic material. To do this, we used a novel EEG paradigm, the Storytime task, in which we record readings of natural texts, and then experimentally manipulate their properties by splicing in words. In this study, we manipulated the language of our target words (English, Spanish) and their fit with the preceding context (strong-fit, weak-fit). If code-switching incurs a unique cost beyond that incurred by an unexpected word, then we should see an additive pattern in our ERP indices. If an effect is driven by lexical expectation alone, then there should be a non-additive interaction such that all unexpected forms incur a similar cost. We found three effects: a general prediction effect (a non-additive N400), a post-lexical recognition of the switch in languages (an LPC for code-switched words), and a prolonged integration difficulty associated with weak-fitting words regardless of language (a sustained negativity). We interpret these findings as suggesting that the processing difficulties experienced by bilinguals can largely be understood within more general frameworks for understanding language comprehension. Our findings are consistent with the broader literature demonstrating that bilinguals do not have two wholly separate language systems but rather a single language system capable of using two coding systems.
Collapse
|
40
|
Nieuwland MS. How 'rational' is semantic prediction? A critique and re-analysis of. Cognition 2021; 215:104848. [PMID: 34274557 DOI: 10.1016/j.cognition.2021.104848] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2020] [Revised: 05/08/2021] [Accepted: 07/11/2021] [Indexed: 01/04/2023]
Abstract
In a recent article in Cognition, Delaney-Busch et al. (2019) claim evidence for 'rational', Bayesian adaptation of semantic predictions, using ERP data from Lau, Holcomb, and Kuperberg (2013). Participants read associatively related and unrelated prime-target word pairs in a first block with only 10% related trials and a second block with 50%. Related words elicited smaller N400s than unrelated words, and this difference was strongest in the second block, suggesting greater engagement in predictive processing. Using a rational adaptor model, Delaney-Busch et al. argue that the stronger N400 reduction for related words in the second block developed as a function of the number of related trials, and concluded therefore that participants predicted related words more strongly when their predictions were fulfilled more often. In this critique, I discuss two critical flaws in their analyses, namely the confounding of prediction effects with those of lexical frequency and the neglect of data from the first block. Re-analyses suggest a different picture: related words by themselves did not yield support for their conclusion, and the effect of relatedness gradually strengthened in othe two blocks in a similar way. Therefore, the N400 did not yield evidence that participants rationally adapted their semantic predictions. Within the framework proposed by Delaney-Busch et al., presumed semantic predictions may even be thought of as 'irrational'. While these results yielded no evidence for rational or probabilistic prediction, they do suggest that participants became increasingly better at predicting target words from prime words.
Collapse
Affiliation(s)
- Mante S Nieuwland
- Max-Planck-Institute for Psycholinguistics, The Netherlands; Donders Centre for Cognition, Brain and Behaviour, The Netherlands.
| |
Collapse
|
41
|
Silcox JW, Payne BR. The costs (and benefits) of effortful listening on context processing: A simultaneous electrophysiology, pupillometry, and behavioral study. Cortex 2021; 142:296-316. [PMID: 34332197 DOI: 10.1016/j.cortex.2021.06.007] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2020] [Revised: 04/02/2021] [Accepted: 06/10/2021] [Indexed: 11/24/2022]
Abstract
There is an apparent disparity between the fields of cognitive audiology and cognitive electrophysiology as to how linguistic context is used when listening to perceptually challenging speech. To gain a clearer picture of how listening effort impacts context use, we conducted a pre-registered study to simultaneously examine electrophysiological, pupillometric, and behavioral responses when listening to sentences varying in contextual constraint and acoustic challenge in the same sample. Participants (N = 44) listened to sentences that were highly constraining and completed with expected or unexpected sentence-final words ("The prisoners were planning their escape/party") or were low-constraint sentences with unexpected sentence-final words ("All day she thought about the party"). Sentences were presented either in quiet or with +3 dB SNR background noise. Pupillometry and EEG were simultaneously recorded and subsequent sentence recognition and word recall were measured. While the N400 expectancy effect was diminished by noise, suggesting impaired real-time context use, we simultaneously observed a beneficial effect of constraint on subsequent recognition memory for degraded speech. Importantly, analyses of trial-to-trial coupling between pupil dilation and N400 amplitude showed that when participants' showed increased listening effort (i.e., greater pupil dilation), there was a subsequent recovery of the N400 effect, but at the same time, higher effort was related to poorer subsequent sentence recognition and word recall. Collectively, these findings suggest divergent effects of acoustic challenge and listening effort on context use: while noise impairs the rapid use of context to facilitate lexical semantic processing in general, this negative effect is attenuated when listeners show increased effort in response to noise. However, this effort-induced reliance on context for online word processing comes at the cost of poorer subsequent memory.
Collapse
Affiliation(s)
| | - Brennan R Payne
- Department of Psychology, University of Utah, USA; Interdepartmental Neuroscience Program, University of Utah, USA
| |
Collapse
|
42
|
Li F, Xu X. Electrophysiological evidence for the coexistence of expectancy fulfillment and semantic integration during the processing of binding and compound nouns. Int J Psychophysiol 2021; 166:25-37. [PMID: 33984365 DOI: 10.1016/j.ijpsycho.2021.05.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2020] [Revised: 05/06/2021] [Accepted: 05/07/2021] [Indexed: 11/30/2022]
Abstract
This study employed electrophysiological measures to investigate the processes of expectancy fulfillment and semantic integration during the processing of binding and compound nouns in Chinese. Sequential expectancy and cloze probability between the two constituents of a two-character noun were manipulated in two experiments, resulting in four types of target characters (i.e., the final characters of two-character nouns): (i) high-cloze, binding character (HB); (ii) high-cloze, compound character (HC); (iii) low-cloze, binding character (LB); (iv) low-cloze, compound character (LC). Participants were asked to judge as quickly and accurately as possible whether the two characters, presented sequentially, formed a real word. The two experiments varied in stimulus onset asynchrony (SOA) between the initial and the target characters: 1000 ms and 300 ms. ERPs acquired at the target characters revealed a significant semantic integration effect in both experiments whereby low-cloze target characters elicited more negative-going activities than high-cloze target characters (L > H) with both long and short SOAs. Importantly, there was a graded N400 effect (LC = LB > HC > HB) at the central region: a clearly visible positive deflection for HB relative to HC in synchrony with an equivalence in negativity for LB and LC. This graded pattern was observed with an SOA of 1000 ms but not 300 ms. These results are discussed in terms of the possible coexistence of an expectancy fulfillment mechanism indexed by the P300 that monitors incoming information for realization of expectancies based on stored mental representations and a semantic integration mechanism indexed by the N400 that incorporates incoming information into the preceding context based on semantic-pragmatic knowledge. Manifestation of this coexistence appears sensitive to SOA-modulated attention orientation.
Collapse
Affiliation(s)
- Fei Li
- School of Foreign Languages, Shanghai Jiao Tong University, China.
| | - Xu Xu
- School of Foreign Languages, Shanghai Jiao Tong University, China.
| |
Collapse
|
43
|
Clinical applications of neurolinguistics in neurosurgery. Front Med 2021; 15:562-574. [PMID: 33983605 DOI: 10.1007/s11684-020-0771-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2019] [Accepted: 03/05/2020] [Indexed: 11/27/2022]
Abstract
The protection of language function is one of the major challenges of brain surgery. Over the past century, neurosurgeons have attempted to seek the optimal strategy for the preoperative and intraoperative identification of language-related brain regions. Neurosurgeons have investigated the neural mechanism of language, developed neurolinguistics theory, and provided unique evidence to further understand the neural basis of language functions by using intraoperative cortical and subcortical electrical stimulation. With the emergence of modern neuroscience techniques and dramatic advances in language models over the last 25 years, novel language mapping methods have been applied in the neurosurgical practice to help neurosurgeons protect the brain and reduce morbidity. The rapid advancements in brain-computer interface have provided the perfect platform for the combination of neurosurgery and neurolinguistics. In this review, the history of neurolinguistics models, advancements in modern technology, role of neurosurgery in language mapping, and modern language mapping methods (including noninvasive neuroimaging techniques and invasive cortical electroencephalogram) are presented.
Collapse
|
44
|
Making ERP research more transparent: Guidelines for preregistration. Int J Psychophysiol 2021; 164:52-63. [PMID: 33676957 DOI: 10.1016/j.ijpsycho.2021.02.016] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2020] [Revised: 02/17/2021] [Accepted: 02/18/2021] [Indexed: 12/16/2022]
Abstract
A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration: researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and the intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice.
Collapse
|
45
|
Vega-Mendoza M, Pickering MJ, Nieuwland MS. Concurrent use of animacy and event-knowledge during comprehension: Evidence from event-related potentials. Neuropsychologia 2021; 152:107724. [PMID: 33347913 DOI: 10.1016/j.neuropsychologia.2020.107724] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2019] [Revised: 10/02/2020] [Accepted: 12/08/2020] [Indexed: 10/22/2022]
Abstract
In two ERP experiments, we investigated whether readers prioritize animacy over real-world event-knowledge during sentence comprehension. We used the paradigm of Paczynski and Kuperberg (2012), who argued that animacy is prioritized based on the observations that the 'related anomaly effect' (reduced N400s for context-related anomalous words compared to unrelated words) does not occur for animacy violations, and that animacy violations but not relatedness violations elicit P600 effects. Participants read passive sentences with plausible agents (e.g., The prescription for the mental disorder was written by the psychiatrist) or implausible agents that varied in animacy and semantic relatedness (schizophrenic/guard/pill/fence). In Experiment 1 (with a plausibility judgment task), plausible sentences elicited smaller N400s relative to all types of implausible sentences. Crucially, animate words elicited smaller N400s than inanimate words, and related words elicited smaller N400s than unrelated words, but Bayesian analysis revealed substantial evidence against an interaction between animacy and relatedness. Moreover, at the P600 time-window, we observed more positive ERPs for animate than inanimate words and for related than unrelated words at anterior regions. In Experiment 2 (without judgment task), we observed an N400 effect with animacy violations, but no other effects. Taken together, the results of our experiments fail to support a prioritized role of animacy information over real-world event-knowledge, but they support an interactive, constraint-based view on incremental semantic processing.
Collapse
Affiliation(s)
- Mariana Vega-Mendoza
- Department of Psychology, School of Philosophy, Psychology and Language Sciences, University of Edinburgh, Edinburgh, Scotland, UK; Department of Business Administration, Technology and Social Sciences. Engineering Psychology, Luleå University of Technology, Luleå, Sweden.
| | - Martin J Pickering
- Department of Psychology, School of Philosophy, Psychology and Language Sciences, University of Edinburgh, Edinburgh, Scotland, UK
| | - Mante S Nieuwland
- Department of Psychology, School of Philosophy, Psychology and Language Sciences, University of Edinburgh, Edinburgh, Scotland, UK; Max Planck Institute for Psycholinguistics, Nijmegen, the Netherlands; Donders Institute for Cognition, Brain and Behaviour, Nijmegen, the Netherlands; Heinrich-Heine-University, Düsseldorf, Germany
| |
Collapse
|
46
|
Nieuwland MS, Arkhipova Y, Rodríguez-Gómez P. Anticipating words during spoken discourse comprehension: A large-scale, pre-registered replication study using brain potentials. Cortex 2020; 133:1-36. [PMID: 33096395 PMCID: PMC7526661 DOI: 10.1016/j.cortex.2020.09.007] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2020] [Revised: 09/08/2020] [Accepted: 09/09/2020] [Indexed: 11/18/2022]
Abstract
Numerous studies report brain potential evidence for the anticipation of specific words during language comprehension. In the most convincing demonstrations, highly predictable nouns exert an influence on processing even before they appear to a reader or listener, as indicated by the brain's neural response to a prenominal adjective or article when it mismatches the expectations about the upcoming noun. However, recent studies suggest that some well-known demonstrations of prediction may be hard to replicate. This could signal the use of data-contingent analysis, but might also mean that readers and listeners do not always use prediction-relevant information in the way that psycholinguistic theories typically suggest. To shed light on this issue, we performed a close replication of one of the best-cited ERP studies on word anticipation (Van Berkum, Brown, Zwitserlood, Kooijman & Hagoort, 2005; Experiment 1), in which participants listened to Dutch spoken mini-stories. In the original study, the marking of grammatical gender on pre-nominal adjectives ('groot/grote') elicited an early positivity when mismatching the gender of an unheard, highly predictable noun, compared to matching gender. The current pre-registered study involved that same manipulation, but used a novel set of materials twice the size of the original set, an increased sample size (N = 187), and Bayesian mixed-effects model analyses that better accounted for known sources of variance than the original. In our study, mismatching gender elicited more negative voltage than matching gender at posterior electrodes. However, this N400-like effect was small in size and lacked support from Bayes Factors. In contrast, we successfully replicated the original's noun effects. While our results yielded some support for prediction, they do not support the Van Berkum et al. effect and highlight the risks associated with commonly employed data-contingent analyses and small sample sizes. Our results also raise the question whether Dutch listeners reliably or consistently use adjectival inflection information to inform their noun predictions.
Collapse
Affiliation(s)
- Mante S Nieuwland
- Max Planck Institute for Psycholinguistics, Nijmegen, the Netherlands; Donders Institute for Brain, Cognition and Behaviour, Nijmegen, the Netherlands; Heinrich-Heine-University, Düsseldorf, Germany.
| | - Yana Arkhipova
- Max Planck Institute for Psycholinguistics, Nijmegen, the Netherlands; Cognitive Neuroscience Lab, Potsdam University, Potsdam, Germany
| | - Pablo Rodríguez-Gómez
- Instituto Pluridisciplinar, Universidad Complutense de Madrid, Madrid, Spain; Facultad de Psicología, Universidad Complutense de Madrid, Madrid, Spain
| |
Collapse
|
47
|
Tan Y, Hagoort P. Catecholaminergic Modulation of Semantic Processing in Sentence Comprehension. Cereb Cortex 2020; 30:6426-6443. [PMID: 32776103 PMCID: PMC7609945 DOI: 10.1093/cercor/bhaa204] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2019] [Revised: 05/26/2020] [Accepted: 07/06/2020] [Indexed: 12/23/2022] Open
Abstract
Catecholamine (CA) function has been widely implicated in cognitive functions that are tied to the prefrontal cortex and striatal areas. The present study investigated the effects of methylphenidate, which is a CA agonist, on the electroencephalogram (EEG) response related to semantic processing using a double-blind, placebo-controlled, randomized, crossover, within-subject design. Forty-eight healthy participants read semantically congruent or incongruent sentences after receiving 20-mg methylphenidate or a placebo while their brain activity was monitored with EEG. To probe whether the catecholaminergic modulation is task-dependent, in one condition participants had to focus on comprehending the sentences, while in the other condition, they only had to attend to the font size of the sentence. The results demonstrate that methylphenidate has a task-dependent effect on semantic processing. Compared to placebo, when semantic processing was task-irrelevant, methylphenidate enhanced the detection of semantic incongruence as indexed by a larger N400 amplitude in the incongruent sentences; when semantic processing was task-relevant, methylphenidate induced a larger N400 amplitude in the semantically congruent condition, which was followed by a larger late positive complex effect. These results suggest that CA-related neurotransmitters influence language processing, possibly through the projections between the prefrontal cortex and the striatum, which contain many CA receptors.
Collapse
Affiliation(s)
- Yingying Tan
- Max Planck Institute for Psycholinguistics, Nijmegen 6525 XD, The Netherlands
| | - Peter Hagoort
- Max Planck Institute for Psycholinguistics, Nijmegen 6525 XD, The Netherlands.,Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen 6500 HB, The Netherlands
| |
Collapse
|
48
|
Fleur DS, Flecken M, Rommers J, Nieuwland MS. Definitely saw it coming? The dual nature of the pre-nominal prediction effect. Cognition 2020; 204:104335. [DOI: 10.1016/j.cognition.2020.104335] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Revised: 05/15/2020] [Accepted: 05/22/2020] [Indexed: 01/29/2023]
|
49
|
Signoret C, Andersen LM, Dahlström Ö, Blomberg R, Lundqvist D, Rudner M, Rönnberg J. The Influence of Form- and Meaning-Based Predictions on Cortical Speech Processing Under Challenging Listening Conditions: A MEG Study. Front Neurosci 2020; 14:573254. [PMID: 33100961 PMCID: PMC7546411 DOI: 10.3389/fnins.2020.573254] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2020] [Accepted: 09/01/2020] [Indexed: 01/07/2023] Open
Abstract
Under adverse listening conditions, prior linguistic knowledge about the form (i.e., phonology) and meaning (i.e., semantics) help us to predict what an interlocutor is about to say. Previous research has shown that accurate predictions of incoming speech increase speech intelligibility, and that semantic predictions enhance the perceptual clarity of degraded speech even when exact phonological predictions are possible. In addition, working memory (WM) is thought to have specific influence over anticipatory mechanisms by actively maintaining and updating the relevance of predicted vs. unpredicted speech inputs. However, the relative impact on speech processing of deviations from expectations related to form and meaning is incompletely understood. Here, we use MEG to investigate the cortical temporal processing of deviations from the expected form and meaning of final words during sentence processing. Our overall aim was to observe how deviations from the expected form and meaning modulate cortical speech processing under adverse listening conditions and investigate the degree to which this is associated with WM capacity. Results indicated that different types of deviations are processed differently in the auditory N400 and Mismatch Negativity (MMN) components. In particular, MMN was sensitive to the type of deviation (form or meaning) whereas the N400 was sensitive to the magnitude of the deviation rather than its type. WM capacity was associated with the ability to process phonological incoming information and semantic integration.
Collapse
Affiliation(s)
- Carine Signoret
- Linnaeus Centre HEAD, Swedish Institute for Disability Research, Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden
| | - Lau M Andersen
- The National Research Facility for Magnetoencephalography, Department of Clinical Neuroscience, Karolinska Institutet, Solna, Sweden.,Center of Functionally Integrative Neuroscience, Institute of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Örjan Dahlström
- Linnaeus Centre HEAD, Swedish Institute for Disability Research, Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden
| | - Rina Blomberg
- Linnaeus Centre HEAD, Swedish Institute for Disability Research, Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden
| | - Daniel Lundqvist
- The National Research Facility for Magnetoencephalography, Department of Clinical Neuroscience, Karolinska Institutet, Solna, Sweden
| | - Mary Rudner
- Linnaeus Centre HEAD, Swedish Institute for Disability Research, Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden
| | - Jerker Rönnberg
- Linnaeus Centre HEAD, Swedish Institute for Disability Research, Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden
| |
Collapse
|
50
|
Wagner-Altendorf TA, Gottschlich C, Robert C, Cirkel A, Heldmann M, Münte TF. The Suppression of Taboo Word Spoonerisms Is Associated With Altered Medial Frontal Negativity: An ERP Study. Front Hum Neurosci 2020; 14:368. [PMID: 33088266 PMCID: PMC7498727 DOI: 10.3389/fnhum.2020.00368] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2020] [Accepted: 08/11/2020] [Indexed: 12/02/2022] Open
Abstract
The constant internal monitoring of speech is a crucial feature to ensure the fairly error-free process of speech production. It has been argued that internal speech monitoring takes place through detection of conflict between different response options or “speech plans.” Speech errors are thought to occur because two (or more) competing speech plans become activated, and the speaker is unable to inhibit the erroneous plan(s) prior to vocalization. A prime example for a speech plan that has to be suppressed is the involuntary utterance of a taboo word. The present study seeks to examine the suppression of involuntary taboo word utterances. We used the “Spoonerisms of Laboratory Induced Predisposition” (SLIP) paradigm to elicit two competing speech plans, one being correct and one embodying either a taboo word or a non-taboo word spoonerism. Behavioral data showed that inadequate speech plans generally were effectively suppressed, although more effectively in the taboo word spoonerism condition. Event-related potential (ERP) analysis revealed a broad medial frontal negativity (MFN) after the target word pair presentation, interpreted as reflecting conflict detection and resolution to suppress the inadequate speech plan. The MFN was found to be more pronounced in the taboo word spoonerism compared to the neutral word spoonerism condition, indicative of a higher level of conflict when subjects suppressed the involuntary utterance of taboo words.
Collapse
Affiliation(s)
| | | | - Carina Robert
- Department of Neurology, University of Lübeck, Lübeck, Germany.,Institute of Psychology II, University of Lübeck, Lübeck, Germany
| | - Anna Cirkel
- Department of Neurology, University of Lübeck, Lübeck, Germany
| | - Marcus Heldmann
- Department of Neurology, University of Lübeck, Lübeck, Germany.,Institute of Psychology II, University of Lübeck, Lübeck, Germany
| | - Thomas F Münte
- Department of Neurology, University of Lübeck, Lübeck, Germany.,Institute of Psychology II, University of Lübeck, Lübeck, Germany
| |
Collapse
|