1
|
Yu S, Gu C, Huang K, Li P. Predicting the next sentence (not word) in large language models: What model-brain alignment tells us about discourse comprehension. SCIENCE ADVANCES 2024; 10:eadn7744. [PMID: 38781343 PMCID: PMC11114233 DOI: 10.1126/sciadv.adn7744] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/01/2024] [Accepted: 04/18/2024] [Indexed: 05/25/2024]
Abstract
Current large language models (LLMs) rely on word prediction as their backbone pretraining task. Although word prediction is an important mechanism underlying language processing, human language comprehension occurs at multiple levels, involving the integration of words and sentences to achieve a full understanding of discourse. This study models language comprehension by using the next sentence prediction (NSP) task to investigate mechanisms of discourse-level comprehension. We show that NSP pretraining enhanced a model's alignment with brain data especially in the right hemisphere and in the multiple demand network, highlighting the contributions of nonclassical language regions to high-level language understanding. Our results also suggest that NSP can enable the model to better capture human comprehension performance and to better encode contextual information. Our study demonstrates that the inclusion of diverse learning objectives in a model leads to more human-like representations, and investigating the neurocognitive plausibility of pretraining tasks in LLMs can shed light on outstanding questions in language neuroscience.
Collapse
Affiliation(s)
- Shaoyun Yu
- Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong SAR, China
| | - Chanyuan Gu
- Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong SAR, China
| | - Kexin Huang
- Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong SAR, China
| | - Ping Li
- Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong SAR, China
- Centre for Immersive Learning and Metaverse in Education, The Hong Kong Polytechnic University, Hong Kong SAR, China
| |
Collapse
|
2
|
Inguscio BMS, Cartocci G, Sciaraffa N, Nicastri M, Giallini I, Aricò P, Greco A, Babiloni F, Mancini P. Two are better than one: Differences in cortical EEG patterns during auditory and visual verbal working memory processing between Unilateral and Bilateral Cochlear Implanted children. Hear Res 2024; 446:109007. [PMID: 38608331 DOI: 10.1016/j.heares.2024.109007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/26/2023] [Revised: 03/28/2024] [Accepted: 04/04/2024] [Indexed: 04/14/2024]
Abstract
Despite the proven effectiveness of cochlear implant (CI) in the hearing restoration of deaf or hard-of-hearing (DHH) children, to date, extreme variability in verbal working memory (VWM) abilities is observed in both unilateral and bilateral CI user children (CIs). Although clinical experience has long observed deficits in this fundamental executive function in CIs, the cause to date is still unknown. Here, we have set out to investigate differences in brain functioning regarding the impact of monaural and binaural listening in CIs compared with normal hearing (NH) peers during a three-level difficulty n-back task undertaken in two sensory modalities (auditory and visual). The objective of this pioneering study was to identify electroencephalographic (EEG) marker pattern differences in visual and auditory VWM performances in CIs compared to NH peers and possible differences between unilateral cochlear implant (UCI) and bilateral cochlear implant (BCI) users. The main results revealed differences in theta and gamma EEG bands. Compared with hearing controls and BCIs, UCIs showed hypoactivation of theta in the frontal area during the most complex condition of the auditory task and a correlation of the same activation with VWM performance. Hypoactivation in theta was also observed, again for UCIs, in the left hemisphere when compared to BCIs and in the gamma band in UCIs compared to both BCIs and NHs. For the latter two, a correlation was found between left hemispheric gamma oscillation and performance in the audio task. These findings, discussed in the light of recent research, suggest that unilateral CI is deficient in supporting auditory VWM in DHH. At the same time, bilateral CI would allow the DHH child to approach the VWM benchmark for NH children. The present study suggests the possible effectiveness of EEG in supporting, through a targeted approach, the diagnosis and rehabilitation of VWM in DHH children.
Collapse
Affiliation(s)
- Bianca Maria Serena Inguscio
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena 291, Rome 00161, Italy; BrainSigns Srl, Via Tirso, 14, Rome 00198, Italy.
| | - Giulia Cartocci
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena 291, Rome 00161, Italy; BrainSigns Srl, Via Tirso, 14, Rome 00198, Italy
| | | | - Maria Nicastri
- Department of Sense Organs, Sapienza University of Rome, Viale dell'Università 31, Rome 00161, Italy
| | - Ilaria Giallini
- Department of Sense Organs, Sapienza University of Rome, Viale dell'Università 31, Rome 00161, Italy
| | - Pietro Aricò
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena 291, Rome 00161, Italy; BrainSigns Srl, Via Tirso, 14, Rome 00198, Italy; Department of Computer, Control, and Management Engineering "Antonio Ruberti", Sapienza University of Rome, Via Ariosto 125, Rome 00185, Italy
| | - Antonio Greco
- Department of Sense Organs, Sapienza University of Rome, Viale dell'Università 31, Rome 00161, Italy
| | - Fabio Babiloni
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena 291, Rome 00161, Italy; BrainSigns Srl, Via Tirso, 14, Rome 00198, Italy; Department of Computer Science, Hangzhou Dianzi University, Xiasha Higher Education Zone, Hangzhou 310018, China
| | - Patrizia Mancini
- Department of Sense Organs, Sapienza University of Rome, Viale dell'Università 31, Rome 00161, Italy
| |
Collapse
|
3
|
Schneider JM, Scott TL, Legault J, Qi Z. Limited but specific engagement of the mature language network during linguistic statistical learning. Cereb Cortex 2024; 34:bhae123. [PMID: 38566510 PMCID: PMC10987970 DOI: 10.1093/cercor/bhae123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 03/04/2024] [Accepted: 03/05/2024] [Indexed: 04/04/2024] Open
Abstract
Statistical learning (SL) is the ability to detect and learn regularities from input and is foundational to language acquisition. Despite the dominant role of SL as a theoretical construct for language development, there is a lack of direct evidence supporting the shared neural substrates underlying language processing and SL. It is also not clear whether the similarities, if any, are related to linguistic processing, or statistical regularities in general. The current study tests whether the brain regions involved in natural language processing are similarly recruited during auditory, linguistic SL. Twenty-two adults performed an auditory linguistic SL task, an auditory nonlinguistic SL task, and a passive story listening task as their neural activation was monitored. Within the language network, the left posterior temporal gyrus showed sensitivity to embedded speech regularities during auditory, linguistic SL, but not auditory, nonlinguistic SL. Using a multivoxel pattern similarity analysis, we uncovered similarities between the neural representation of auditory, linguistic SL, and language processing within the left posterior temporal gyrus. No other brain regions showed similarities between linguistic SL and language comprehension, suggesting that a shared neurocomputational process for auditory SL and natural language processing within the left posterior temporal gyrus is specific to linguistic stimuli.
Collapse
Affiliation(s)
- Julie M Schneider
- Department of Communication Sciences and Disorders, Louisiana State University, 77 Hatcher Hall, Field House Dr., Baton Rouge, LA 70803, United States
- Department of Linguistics & Cognitive Science, University of Delaware, 125 E Main St, Newark, DE 19716, United States
| | - Terri L Scott
- School of Medicine, University of California San Francisco, 533 Parnassus Ave, San Francisco, CA 94143, United States
| | - Jennifer Legault
- Department of Psychology, Elizabethtown College, One Alpha Dr, Elizabethtown, PA 17022, United States
| | - Zhenghan Qi
- Department of Linguistics & Cognitive Science, University of Delaware, 125 E Main St, Newark, DE 19716, United States
- Bouvé College of Health Sciences, Northeastern University, 360 Huntington Ave, Boston, MA 02115, United States
| |
Collapse
|
4
|
Wei X, Gunter TC, Adamson H, Schwendemann M, Friederici AD, Goucha T, Anwander A. White matter plasticity during second language learning within and across hemispheres. Proc Natl Acad Sci U S A 2024; 121:e2306286121. [PMID: 38175869 PMCID: PMC10786292 DOI: 10.1073/pnas.2306286121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 11/06/2023] [Indexed: 01/06/2024] Open
Abstract
Adult second language (L2) learning is a challenging enterprise inducing neuroplastic changes in the human brain. However, it remains unclear how the structural language connectome and its subnetworks change during adult L2 learning. The current study investigated longitudinal changes in white matter (WM) language networks in each hemisphere, as well as their interconnection, in a large group of Arabic-speaking adults who learned German intensively for 6 mo. We found a significant increase in WM-connectivity within bilateral temporal-parietal semantic and phonological subnetworks and right temporal-frontal pathways mainly in the second half of the learning period. At the same time, WM-connectivity between the two hemispheres decreased significantly. Crucially, these changes in WM-connectivity are correlated with L2 performance. The observed changes in subnetworks of the two hemispheres suggest a network reconfiguration due to lexical learning. The reduced interhemispheric connectivity may indicate a key role of the corpus callosum in L2 learning by reducing the inhibition of the language-dominant left hemisphere. Our study highlights the dynamic changes within and across hemispheres in adult language-related networks driven by L2 learning.
Collapse
Affiliation(s)
- Xuehu Wei
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Thomas C. Gunter
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Helyne Adamson
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Matthias Schwendemann
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Angela D. Friederici
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Tomás Goucha
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Alfred Anwander
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| |
Collapse
|
5
|
Schneider JM, McIlvain G, Johnson CL. Mechanical Properties of the Developing Brain are Associated with Language Input and Vocabulary Outcome. Dev Neuropsychol 2022; 47:258-272. [PMID: 35938379 PMCID: PMC9397825 DOI: 10.1080/87565641.2022.2108425] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2022] [Revised: 07/05/2022] [Accepted: 07/26/2022] [Indexed: 11/03/2022]
Abstract
The quality of language that children hear in their environment is associated with the development of language-related brain regions, in turn promoting vocabulary knowledge. Although informative, it remains unknown how these environmental influences alter the structure of neural tissue and subsequent vocabulary outcomes. The current study uses magnetic resonance elastography (MRE) to examine how children's language environments underlie brain tissue mechanical properties, characterized as brain tissue stiffness and damping ratio, and promote vocabulary knowledge. Twenty-five children, ages 5-7, had their audio and video recorded while engaging in a play session with their parents. Children also completed the Picture Vocabulary Task (from NIH Toolbox) and participated in an MRI, where MRE and anatomical images were acquired. Higher quality input was associated with greater stiffness in the bilateral inferior frontal gyrus and right superior temporal gyrus, whereas greater vocabulary knowledge was associated with lower damping ratio in the right inferior frontal gyrus. These findings suggest changes in neural tissue composition are sensitive to malleable aspects of the environment, whereas tissue organization is more strongly associated with vocabulary outcome. Notably, these associations were independent of maternal education, suggesting more proximal measures of a child's environment may be the source of differences in neural tissue structure underlying variability in vocabulary outcomes.
Collapse
Affiliation(s)
- Julie M. Schneider
- Department of Communication Sciences and Disorders, Louisiana State University, Baton Rouge, LA
| | - Grace McIlvain
- Department of Biomedical Engineering, University of Delaware; Newark, DE
| | - Curtis L. Johnson
- Department of Biomedical Engineering, University of Delaware; Newark, DE
| |
Collapse
|
6
|
Yeh C, Chen MH, Chen PH, Lee CL. Lateralization as a symphony: Joint influence of interhemispheric inhibition and transmission on brain asymmetry and syntactic processing. BRAIN AND LANGUAGE 2022; 228:105095. [PMID: 35248863 DOI: 10.1016/j.bandl.2022.105095] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Revised: 01/16/2022] [Accepted: 02/24/2022] [Indexed: 06/14/2023]
Abstract
This study investigated the roles of cross-hemispheric communications in promoting left-lateralized syntactic processing in the brain. Fifty-six young right-handers without familial sinistrality background underwent a divided visual field ERP grammaticality judgment experiment to assess syntactic processing in each hemisphere. Two behavioral tasks -the bilateral flanker task and bilateral word matching task, were used to assess cross-hemispheric inhibition and transmission. Grand average ERP data showed a significant P600 grammaticality effect in the left hemisphere (LH) only; however, individual variations in the P600 responses were observed in both hemispheres. Results of correlational analyses showed that larger LH P600 effects were associated with slower inter-hemispheric transmissions; smaller right hemisphere (RH) P600 effects were associated with more effective RH inhibition. These results yielded support for both the callosal distance hypothesis and the inhibition hypothesis for language lateralization and demonstrated that different aspects of cross-hemispheric communications jointly influence the degree of syntactic lateralization.
Collapse
Affiliation(s)
- Chih Yeh
- Max Planck School of Cognition, Germany; Graduate Institute of Linguistics, National Taiwan University, Taiwan
| | - Min-Hsin Chen
- Graduate Institute of Linguistics, National Taiwan University, Taiwan
| | - Po-Heng Chen
- Graduate Institute of Linguistics, National Taiwan University, Taiwan
| | - Chia-Lin Lee
- Graduate Institute of Linguistics, National Taiwan University, Taiwan; Department of Psychology, National Taiwan University, Taiwan; Graduate Institute of Brain and Mind Sciences, National Taiwan University, Taiwan; Neurobiology and Cognitive Neuroscience Center, National Taiwan University, Taiwan.
| |
Collapse
|
7
|
Fong MCM, Ma MKH, Chui JYT, Law TST, Hui NY, Au A, Wang WS. Foreign Language Learning in Older Adults: Anatomical and Cognitive Markers of Vocabulary Learning Success. Front Hum Neurosci 2022; 16:787413. [PMID: 35340542 PMCID: PMC8942782 DOI: 10.3389/fnhum.2022.787413] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 02/08/2022] [Indexed: 12/03/2022] Open
Abstract
In recent years, foreign language learning (FLL) has been proposed as a possible cognitive intervention for older adults. However, the brain network and cognitive functions underlying FLL has remained largely unconfirmed in older adults. In particular, older and younger adults have markedly different cognitive profile—while older adults tend to exhibit decline in most cognitive domains, their semantic memory usually remains intact. As such, older adults may engage the semantic functions to a larger extent than the other cognitive functions traditionally considered the most important (e.g., working memory capacity and phonological awareness). Using anatomical measurements and a cognitive test battery, the present study examined this hypothesis in twenty cognitively normal older adults (58–69 years old), who participated in a two-month Italian learning programme. Results showed that the immediate learning success and long-term retention of Italian vocabularies were most consistently predicted by the anatomical measures of the left pars orbitalis and left caudal middle frontal cortex, which are implicated in semantic and episodic memory functions. Convergent evidence was also found based on the pattern of cognitive associations. Our results are consistent with a prominent role of semantic and episodic memory functions in vocabulary learning in older learners.
Collapse
Affiliation(s)
- Manson Cheuk-Man Fong
- Research Centre for Language, Cognition, and Neuroscience, Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
- Research Institute for Smart Ageing, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
- *Correspondence: Manson Cheuk-Man Fong
| | - Matthew King-Hang Ma
- Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, Hong Kong SAR, China
| | - Jeremy Yin To Chui
- Research Centre for Language, Cognition, and Neuroscience, Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - Tammy Sheung Ting Law
- Research Centre for Language, Cognition, and Neuroscience, Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - Nga-Yan Hui
- Research Centre for Language, Cognition, and Neuroscience, Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - Alma Au
- Department of Applied Social Science, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - William Shiyuan Wang
- Research Centre for Language, Cognition, and Neuroscience, Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
- Research Institute for Smart Ageing, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
- Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, Hong Kong SAR, China
- William Shiyuan Wang
| |
Collapse
|
8
|
Borodkin K, Livny A, Kushnir T, Tsarfaty G, Maliniak O, Faust M. Linking L2 proficiency and patterns of functional connectivity during L1 word retrieval. BRAIN AND LANGUAGE 2021; 216:104931. [PMID: 33677174 DOI: 10.1016/j.bandl.2021.104931] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Revised: 12/29/2020] [Accepted: 02/08/2021] [Indexed: 06/12/2023]
Abstract
Second language (L2) learners differ greatly in language proficiency, which is partially explained by variability in native language (L1) skills. The present fMRI study explored the neural underpinnings of the L1-L2 link. Twenty L2 learners completed a tip-of-the-tongue (TOT) task that required retrieving words in L1. Low-proficiency L2 learners showed greater functional connectivity for correct and TOT responses between the left inferior frontal gyrus and right-sided homologues of the temporoparietal regions that support phonological processing (e.g., supramarginal gyrus), possibly reflecting difficulty with phonological retrieval. High-proficiency L2 learners showed greater connectivity for erroneous responses (TOT in particular) between the left inferior frontal gyrus and regions of left medial temporal lobe (e.g., hippocampus), associated with implicit learning processes. The difference between low- and high-proficiency L2 learners in functional connectivity, which is evident even during L1 processing, may affect L2 learning processes and outcomes.
Collapse
Affiliation(s)
- Katy Borodkin
- Department of Communication Disorders, Sackler Faculty of Medicine, Tel Aviv University, Israel.
| | - Abigail Livny
- Department of Diagnostic Imaging, Sheba Medical Center, Tel-Hashomer, Israel; The Joseph Sagol Neuroscience Center, Sheba Medical Center, Tel Hashomer, Israel; Sackler Faculty of Medicine, Tel Aviv University, Israel
| | - Tammar Kushnir
- Department of Diagnostic Imaging, Sheba Medical Center, Tel-Hashomer, Israel; Sackler Faculty of Medicine, Tel Aviv University, Israel
| | - Galia Tsarfaty
- Department of Diagnostic Imaging, Sheba Medical Center, Tel-Hashomer, Israel; Sackler Faculty of Medicine, Tel Aviv University, Israel
| | - Omer Maliniak
- Department of Psychology, Bar-Ilan University, Ramat Gan, Israel
| | - Miriam Faust
- Department of Psychology, Bar-Ilan University, Ramat Gan, Israel; Gonda Brain Research Center, Bar-Ilan University, Ramat Gan, Israel
| |
Collapse
|
9
|
Jouravlev O, Mineroff Z, Blank IA, Fedorenko E. The Small and Efficient Language Network of Polyglots and Hyper-polyglots. Cereb Cortex 2021; 31:62-76. [PMID: 32820332 DOI: 10.1093/cercor/bhaa205] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2019] [Revised: 07/06/2020] [Accepted: 07/06/2020] [Indexed: 11/13/2022] Open
Abstract
Acquiring a foreign language is challenging for many adults. Yet certain individuals choose to acquire sometimes dozens of languages and often just for fun. Is there something special about the minds and brains of such polyglots? Using robust individual-level markers of language activity, measured with fMRI, we compared native language processing in polyglots versus matched controls. Polyglots (n = 17, including nine "hyper-polyglots" with proficiency in 10-55 languages) used fewer neural resources to process language: Their activations were smaller in both magnitude and extent. This difference was spatially and functionally selective: The groups were similar in their activation of two other brain networks-the multiple demand network and the default mode network. We hypothesize that the activation reduction in the language network is experientially driven, such that the acquisition and use of multiple languages makes language processing generally more efficient. However, genetic and longitudinal studies will be critical to distinguish this hypothesis from the one whereby polyglots' brains already differ at birth or early in development. This initial characterization of polyglots' language network opens the door to future investigations of the cognitive and neural architecture of individuals who gain mastery of multiple languages, including changes in this architecture with linguistic experiences.
Collapse
Affiliation(s)
- Olessia Jouravlev
- Brain & Cognitive Sciences Department, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.,Department of Cognitive Science, Carleton University, Ottawa, ON K1S5B6, Canada
| | - Zachary Mineroff
- Brain & Cognitive Sciences Department, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Idan A Blank
- Brain & Cognitive Sciences Department, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.,Department of Psychology, University of California Los Angeles, Los Angeles, CA 90095, USA
| | - Evelina Fedorenko
- Brain & Cognitive Sciences Department, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.,McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| |
Collapse
|