1
|
Liu B, Wang X, Wang X, Li Y, Han Y, Lu J, Zhang H, Wang X, Bi Y. Object knowledge representation in the human visual cortex requires a connection with the language system. PLoS Biol 2025; 23:e3003161. [PMID: 40392802 PMCID: PMC12091770 DOI: 10.1371/journal.pbio.3003161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2024] [Accepted: 04/12/2025] [Indexed: 05/22/2025] Open
Abstract
How world knowledge is stored in the human brain is a central question in cognitive neuroscience. Object knowledge effects have been commonly observed in higher-order sensory association cortices, with the role of language being highly debated. Using object color as a test case, we investigated whether communication with the language system plays a necessary role in knowledge neural representation in the visual cortex and corresponding behaviors, combining diffusion imaging (measuring white-matter structural integrity), functional MRI (fMRI; measuring functional neural representation of knowledge), and neuropsychological assessments (measuring behavioral integrity) in a group of patients who suffered from stroke (N = 33; 18 with left-hemisphere lesions, 11 with right-hemisphere lesions, and 4 with bilateral lesions). The structural integrity loss of the white-matter connection between the anterior temporal language region and the ventral visual cortex had a significant effect on the neural representation strength of object color knowledge in the ventral visual cortex and on object color knowledge behavior across modalities. These contributions could not be explained by the potential effects of the early visual perception pathway or potential confounding brain or cognitive variables. Our experiments reveal the contribution of the vision-language connection in the ventral occipitotemporal cortex (VOTC) object knowledge neural representation and object knowledge behaviors, highlighting the significance of the language-sensory system interface.
Collapse
Affiliation(s)
- Bo Liu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Department of Radiology, First Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
- College of Medical Imaging, Shanxi Medical University, Taiyuan, Shanxi, China
- Shanxi Key Laboratory of Intelligent Imaging, First Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Xiaosha Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Xiaoying Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Yan Li
- Department of Radiology, First Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
- College of Medical Imaging, Shanxi Medical University, Taiyuan, Shanxi, China
- Shanxi Key Laboratory of Intelligent Imaging, First Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Yang Han
- College of Medical Imaging, Shanxi Medical University, Taiyuan, Shanxi, China
- Xi’an Key Laboratory of Metabolic Disease Imaging, Xi’an No.3 Hospital, Affiliated Hospital of Northwest University, Xi’an, China,
| | - Jiahui Lu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Hui Zhang
- Department of Radiology, First Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
- College of Medical Imaging, Shanxi Medical University, Taiyuan, Shanxi, China
- Shanxi Key Laboratory of Intelligent Imaging, First Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Xiaochun Wang
- Department of Radiology, First Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
- College of Medical Imaging, Shanxi Medical University, Taiyuan, Shanxi, China
- Shanxi Key Laboratory of Intelligent Imaging, First Hospital of Shanxi Medical University, Taiyuan, Shanxi, China
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
- IDG/McGovern Institute for Brain Research, Peking University, Beijing, China
- Institute for Artificial Intelligence, Peking University, Beijing, China
- Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing, China
| |
Collapse
|
2
|
Vogt L, Strömert P, Matentzoglu N, Karam N, Konrad M, Prinz M, Baum R. Suggestions for extending the FAIR Principles based on a linguistic perspective on semantic interoperability. Sci Data 2025; 12:688. [PMID: 40274834 PMCID: PMC12022272 DOI: 10.1038/s41597-025-05011-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2024] [Accepted: 04/15/2025] [Indexed: 04/26/2025] Open
Abstract
FAIR (meta)data presuppose their successful communication between machines and humans while preserving meaning and reference. The FAIR Guiding Principles lack specificity regarding semantic interoperability. We adopt a linguistic perspective on semantic interoperability and investigate the structures and conventions ensuring reliable communication of textual information, drawing parallels with data structures by understanding both as models. We propose a conceptual model of semantic interoperability, comprising intensional and extensional terminological interoperability, as well as logical and schema propositional interoperability. Since there cannot be a universally accepted best vocabulary and best (meta)data schema, establishing semantic interoperability necessitates the provision of comprehensive sets of intensional and extensional entity mappings and schema crosswalks. In accordance with our conceptual model, we suggest additions to the FAIR Guiding Principles that encompass the requirements for semantic interoperability. Additionally, we argue that attaining FAIRness of (meta)data requires not only their organization into FAIR Digital Objects, but also the establishment of a FAIR ecosystem of FAIR Services, that include a terminology, a schema, and an operations service.
Collapse
Affiliation(s)
- Lars Vogt
- TIB Leibniz Information Centre for Science and Technology, Welfengarten 1B, 30167, Hanover, Germany.
| | - Philip Strömert
- TIB Leibniz Information Centre for Science and Technology, Welfengarten 1B, 30167, Hanover, Germany
| | | | - Naouel Karam
- Institute for Applied Informatics (InfAI), University of Leipzig, Leipzig, Germany
| | - Marcel Konrad
- TIB Leibniz Information Centre for Science and Technology, Welfengarten 1B, 30167, Hanover, Germany
| | - Manuel Prinz
- TIB Leibniz Information Centre for Science and Technology, Welfengarten 1B, 30167, Hanover, Germany
| | - Roman Baum
- ZB MED - Information Centre for Life Sciences, Gleueler Straβe 60, 50931, Cologne, Germany
| |
Collapse
|
3
|
Zhu Z, Yang H, Wen H, Hung J, Hu Y, Bi Y, Yu X. Innate network mechanisms of temporal pole for semantic cognition in neonatal and adult twin studies. Nat Commun 2025; 16:3835. [PMID: 40268914 PMCID: PMC12019161 DOI: 10.1038/s41467-025-58896-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2024] [Accepted: 04/04/2025] [Indexed: 04/25/2025] Open
Abstract
What are the innate neural mechanisms scaffolding the protracted development of sophisticated human cognition observable later in life? We investigate this question by focusing on the putative hub of the human semantic memory system-the temporal pole. Combining infant- and twin-based imaging analyses, we examine the ontogenetic mechanisms and network characteristics of the functional subdivisions within the temporal pole that are specialized for semantic processing of different types in adults. Our findings reveal topologically similar temporal pole parcellations in the adult and neonatal brains. Notably, the specific functional connectivity of the dorsal and ventrolateral subdivisions with semantic-related networks are evident in neonates, significantly heritable, and associated with semantic functions in adult twins. These results demonstrate the neonatal emergence of genetically programmed functional connectivity characteristics in the temporal pole parcellations that underlie its crucial role in semantic processing, highlighting the innate network mechanisms that support semantic cognition in humans.
Collapse
Affiliation(s)
- Ziliang Zhu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| | - Huichao Yang
- College of Education, Hebei Normal University, Shijiazhuang, China
| | - Haojie Wen
- School of Systems Science, Beijing Normal University, Beijing, China
| | - Jinyi Hung
- Department of Audiology and Speech-Language Pathology, Mackay Medical College, New Taipei City, Taiwan
| | - Yueqin Hu
- Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China.
- School of Psychological and Cognitive Science and Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing, China.
- IDG/McGovern Institute for Brain Research, Peking University, Beijing, China.
- Institute for Artificial Intelligence, Peking University, Beijing, China.
| | - Xi Yu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China.
| |
Collapse
|
4
|
Hauptman M, Elli G, Pant R, Bedny M. Neural specialization for 'visual' concepts emerges in the absence of vision. Cognition 2025; 257:106058. [PMID: 39827755 DOI: 10.1016/j.cognition.2024.106058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2024] [Revised: 12/20/2024] [Accepted: 12/28/2024] [Indexed: 01/22/2025]
Abstract
The 'different-body/different-concepts hypothesis' central to some embodiment theories proposes that the sensory capacities of our bodies shape the cognitive and neural basis of our concepts. We tested this hypothesis by comparing behavioral semantic similarity judgments and neural signatures (fMRI) of 'visual' categories ('living things,' or animals, e.g., tiger, and light events, e.g., sparkle) across congenitally blind (n = 21) and sighted (n = 22) adults. Words referring to 'visual' entities/nouns and events/verbs (animals and light events) were compared to less vision-dependent categories from the same grammatical class (animal vs. place nouns, light vs. sound, mouth, and hand verbs). Within-category semantic similarity judgments about animals (e.g., sparrow vs. finch) were partially different across groups, consistent with the idea that sighted people rely on visually learned information to make such judgments about animals. However, robust neural specialization for living things in temporoparietal semantic networks, including in the precuneus, was observed in blind and sighted people alike. For light events, which are directly accessible only through vision, behavioral judgments were indistinguishable across groups. Neural responses to light events were also similar across groups: in both blind and sighted people, the left middle temporal gyrus (LMTG+) responded more to event concepts, including light events, compared to entity concepts. Multivariate patterns of neural activity in LMTG+ distinguished among different event types, including light events vs. other event types. In sum, we find that neural signatures of concepts previously attributed to visual experience do not require vision. Across a wide range of semantic types, conceptual representations develop independent of sensory experience.
Collapse
Affiliation(s)
- Miriam Hauptman
- Department of Psychological & Brain Sciences, Johns Hopkins University, Baltimore, MD, USA.
| | - Giulia Elli
- Department of Psychological & Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
| | - Rashi Pant
- Department of Psychological & Brain Sciences, Johns Hopkins University, Baltimore, MD, USA; Department of Biological Psychology & Neuropsychology, Universität Hamburg, Germany.
| | - Marina Bedny
- Department of Psychological & Brain Sciences, Johns Hopkins University, Baltimore, MD, USA.
| |
Collapse
|
5
|
Jonauskaite D, Gierlinger N, Geiger K, Busse C, Frick A, Mohr C, Leder H. Non-visual colour: A qualitative study of how the totally blind and an achromatope navigate colour in the sighted world. Acta Psychol (Amst) 2025; 253:104682. [PMID: 39809045 DOI: 10.1016/j.actpsy.2024.104682] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2024] [Revised: 12/11/2024] [Accepted: 12/17/2024] [Indexed: 01/16/2025] Open
Abstract
Colour plays an important role in the sighted world, not only by guiding and warning, but also by helping to make decisions, form opinions, and influence emotional landscape. While not everyone has direct access to this information, even people without colour vision (i.e., blind, achromatope) understand the meanings of colour terms and can assign sensory and affective properties to colours. To learn which aspects of colour are transmitted non-visually, and thus, are pertinent to those without colour vision, we conducted qualitative interviews with 11 participants (2 congenitally blind, 2 early blind, 4 late blind, 2 late blind with synaesthesia, and 1 achromatope). Our thematic analysis revealed that all participants had detailed knowledge of colours and displayed opinions and attitudes. Colour was important to them as it allowed to take part in the sighted world, navigate the surroundings, and communicate with the sighted peers. While participants with non-congenital colour vision absence could remember and even visualise colours, colour was more abstract to participants with congenital colour vision absence. This was possibly a reason why colour was not very important to their personal lives. Nonetheless, all our participants associated colours with diverse objects, concepts, and emotions, and also had colour preferences, indicating that semantic (conceptual, symbolic, affective) meanings of colour can be transmitted without direct visual experience. Future quantitative and qualitative studies are needed for a systematic understanding of such connotations in the visually impaired population, and their implications to those who can and cannot see colour.
Collapse
Affiliation(s)
- Domicele Jonauskaite
- Institute of Psychology, University of Lausanne, Lausanne, Switzerland; Faculty of Psychology, University of Vienna, Vienna, Austria.
| | | | - Klara Geiger
- Faculty of Psychology, University of Vienna, Vienna, Austria
| | - Claudia Busse
- Faculty of Psychology, University of Vienna, Vienna, Austria
| | - Aline Frick
- Faculty of Psychology, University of Vienna, Vienna, Austria; Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, the Netherlands
| | - Christine Mohr
- Institute of Psychology, University of Lausanne, Lausanne, Switzerland
| | - Helmut Leder
- Faculty of Psychology, University of Vienna, Vienna, Austria; Vienna Cognitive Science Hub, Vienna, Austria
| |
Collapse
|
6
|
Zhang Y, Fan L, Hao Y, Dagher A, Jiang T, Bellec P. Connectome-constrained neural decoding reveals a representational hierarchy from perception to cognition to action. Sci Bull (Beijing) 2025; 70:478-482. [PMID: 39256082 DOI: 10.1016/j.scib.2024.08.029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 07/05/2024] [Accepted: 08/23/2024] [Indexed: 09/12/2024]
Affiliation(s)
- Yu Zhang
- Zhejiang Lab, Hangzhou 311100, China.
| | - Lingzhong Fan
- Brainnetome Center, Chinese Academy of Sciences, Beijing 100190, China; Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
| | | | - Alain Dagher
- McConnell Brain Imaging Center, Montreal Neurological Institute, McGill University, Montreal, QC, H3A 2B4, Canada
| | - Tianzi Jiang
- Brainnetome Center, Chinese Academy of Sciences, Beijing 100190, China; Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
| | - Pierre Bellec
- Centre de recherche de l'Institut universitaire de gériatrie de Montréal, Montreal, QC, H3W 1W6, Canada; Department of Psychology, University of Montreal, Montreal, QC, H3C 3J7, Canada.
| |
Collapse
|
7
|
Reilly J, Shain C, Borghesani V, Kuhnke P, Vigliocco G, Peelle JE, Mahon BZ, Buxbaum LJ, Majid A, Brysbaert M, Borghi AM, De Deyne S, Dove G, Papeo L, Pexman PM, Poeppel D, Lupyan G, Boggio P, Hickok G, Gwilliams L, Fernandino L, Mirman D, Chrysikou EG, Sandberg CW, Crutch SJ, Pylkkänen L, Yee E, Jackson RL, Rodd JM, Bedny M, Connell L, Kiefer M, Kemmerer D, de Zubicaray G, Jefferies E, Lynott D, Siew CSQ, Desai RH, McRae K, Diaz MT, Bolognesi M, Fedorenko E, Kiran S, Montefinese M, Binder JR, Yap MJ, Hartwigsen G, Cantlon J, Bi Y, Hoffman P, Garcea FE, Vinson D. What we mean when we say semantic: Toward a multidisciplinary semantic glossary. Psychon Bull Rev 2025; 32:243-280. [PMID: 39231896 PMCID: PMC11836185 DOI: 10.3758/s13423-024-02556-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/19/2024] [Indexed: 09/06/2024]
Abstract
Tulving characterized semantic memory as a vast repository of meaning that underlies language and many other cognitive processes. This perspective on lexical and conceptual knowledge galvanized a new era of research undertaken by numerous fields, each with their own idiosyncratic methods and terminology. For example, "concept" has different meanings in philosophy, linguistics, and psychology. As such, many fundamental constructs used to delineate semantic theories remain underspecified and/or opaque. Weak construct specificity is among the leading causes of the replication crisis now facing psychology and related fields. Term ambiguity hinders cross-disciplinary communication, falsifiability, and incremental theory-building. Numerous cognitive subdisciplines (e.g., vision, affective neuroscience) have recently addressed these limitations via the development of consensus-based guidelines and definitions. The project to follow represents our effort to produce a multidisciplinary semantic glossary consisting of succinct definitions, background, principled dissenting views, ratings of agreement, and subjective confidence for 17 target constructs (e.g., abstractness, abstraction, concreteness, concept, embodied cognition, event semantics, lexical-semantic, modality, representation, semantic control, semantic feature, simulation, semantic distance, semantic dimension). We discuss potential benefits and pitfalls (e.g., implicit bias, prescriptiveness) of these efforts to specify a common nomenclature that other researchers might index in specifying their own theoretical perspectives (e.g., They said X, but I mean Y).
Collapse
Affiliation(s)
| | - Cory Shain
- Massachusetts Institute of Technology, Cambridge, MA, USA
| | | | - Philipp Kuhnke
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- Leipzig University, Leipzig, Germany
| | | | | | | | - Laurel J Buxbaum
- Thomas Jefferson University, Moss Rehabilitation Research Institute, Elkins Park, PA, USA
| | | | | | | | | | - Guy Dove
- University of Louisville, Louisville, KY, USA
| | - Liuba Papeo
- Centre National de La Recherche Scientifique (CNRS), University Claude-Bernard Lyon, Lyon, France
| | | | | | | | - Paulo Boggio
- Universidade Presbiteriana Mackenzie, São Paulo, Brazil
| | | | | | | | | | | | | | | | | | - Eiling Yee
- University of Connecticut, Storrs, CT, USA
| | | | | | | | | | | | | | | | | | | | | | | | - Ken McRae
- Western University, London, ON, Canada
| | | | | | | | | | | | | | - Melvin J Yap
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- National University of Singapore, Singapore, Singapore
| | - Gesa Hartwigsen
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- Leipzig University, Leipzig, Germany
| | | | - Yanchao Bi
- University of Edinburgh, Edinburgh, UK
- Beijing Normal University, Beijing, China
| | | | | | | |
Collapse
|
8
|
Fu Z, Chen H, Liu Z, Sun M, Liu Z, Bi Y. Pathogen stress heightens sensorimotor dimensions in the human collective semantic space. COMMUNICATIONS PSYCHOLOGY 2025; 3:2. [PMID: 39757308 DOI: 10.1038/s44271-024-00183-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/23/2024] [Accepted: 12/19/2024] [Indexed: 01/07/2025]
Abstract
Infectious diseases have been major causes of death throughout human history and are assumed to broadly affect human psychology. However, whether and how conceptual processing, an internal world model central to various cognitive processes, adapts to such salient stress variables remains largely unknown. To address this, we conducted three studies examining the relationship between pathogen severity and semantic space, probed through the main neurocognitive semantic dimensions revealed by large-scale text analyses: one cross-cultural study (across 43 countries) and two historical studies (over the past 100 years). Across all three studies, we observed that increasing pathogen severity was associated with an enhancement of the sensory-motor dimension in the collective semantic space. These patterns remained robust after controlling for the effects of sociocultural variables, including economic wealth and societal norms of tightness. These results highlight the universal dynamic mechanisms of collective semantics, such that pathogen stress potentially drives sensorially oriented semantic processing.
Collapse
Affiliation(s)
- Ze Fu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China
| | - Huimin Chen
- School of Journalism and Communication, Tsinghua University, Beijing, 100084, China
| | - Zhan Liu
- Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China
| | - Maosong Sun
- Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China
| | - Zhiyuan Liu
- Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China.
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China.
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, 100871, China.
- IDG/McGovern Institute for Brain Research, Peking University, Beijing, 100871, China.
- Institute for Artificial Intelligence, Peking University, Beijing, 100871, China.
- Chinese Institute for Brain Research, Beijing, 102206, China.
| |
Collapse
|
9
|
Federico G, Lesourd M, Fournel A, Bluet A, Bryche C, Metaireau M, Baldi D, Brandimonte MA, Soricelli A, Rossetti Y, Osiurak F. Two distinct neural pathways for mechanical versus digital technology. Neuroimage 2025; 305:120971. [PMID: 39667539 DOI: 10.1016/j.neuroimage.2024.120971] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2024] [Revised: 12/05/2024] [Accepted: 12/09/2024] [Indexed: 12/14/2024] Open
Abstract
Technology pervades every aspect of our lives, making it crucial to investigate how the human mind deals with it. Here we examine the cognitive and neural foundations of technological cognition. In the first fMRI experiment, participants viewed videos depicting the use of mechanical tools (e.g., a screwdriver) and digital tools (e.g., a smartphone) compared to simple object movements. Results revealed a key dissociation: mechanical tools extensively activated the dorsal and ventro-dorsal visual streams, which are motor- and action-oriented brain systems. Conversely, digital tools largely engaged the ventral visual stream, associated with conceptual and social cognition. A second behavioral experiment showed a pronounced tendency to anthropomorphize digital tools. A third experiment involving a priming task confirmed that digital tools activate the social brain. The discovery of two different neurocognitive systems for mechanical versus digital technology offers new insights into human-technology interaction and its implications for the evolution of the human mind.
Collapse
Affiliation(s)
- Giovanni Federico
- Laboratory of Experimental Psychology and Cognitive Neuroscience, Suor Orsola Benincasa University, Naples, Italy.
| | - Mathieu Lesourd
- Laboratoire de Recherches Intégratives en Neurosciences et Psychologie Cognitive (UMR INSERM 1322), Université de Bourgogne Franche-Comté, Besançon, France; MSHE Ledoux, CNRS, Université de Bourgogne Franche-Comté, Besançon, France
| | - Arnaud Fournel
- Laboratoire d'Étude des Mécanismes Cognitifs (EA 3082), Université de Lyon, France
| | - Alexandre Bluet
- Laboratoire d'Étude des Mécanismes Cognitifs (EA 3082), Université de Lyon, France; Karolinska Institutet, Stockholm, Sweden
| | - Chloé Bryche
- Laboratoire d'Étude des Mécanismes Cognitifs (EA 3082), Université de Lyon, France
| | - Maximilien Metaireau
- Laboratoire de Recherches Intégratives en Neurosciences et Psychologie Cognitive (UMR INSERM 1322), Université de Bourgogne Franche-Comté, Besançon, France; Laboratoire d'Étude des Mécanismes Cognitifs (EA 3082), Université de Lyon, France
| | | | - Maria Antonella Brandimonte
- Laboratory of Experimental Psychology and Cognitive Neuroscience, Suor Orsola Benincasa University, Naples, Italy
| | | | - Yves Rossetti
- Mouvement, Handicap, et Neuro-Immersion, Hospices Civils de Lyon et Centre de Recherche en Neurosciences de Lyon, Hôpital Henry Gabrielle, St Genis Laval, France; Centre de Recherche en Neurosciences de Lyon, Trajectoires Team, CNRS U5292, Inserm U1028, Université de Lyon, France
| | - François Osiurak
- Laboratoire d'Étude des Mécanismes Cognitifs (EA 3082), Université de Lyon, France; Institut Universitaire de France, Paris, France
| |
Collapse
|
10
|
Zhao M, Xin Y, Deng H, Zuo Z, Wang X, Bi Y, Liu N. Object color knowledge representation occurs in the macaque brain despite the absence of a developed language system. PLoS Biol 2024; 22:e3002863. [PMID: 39466847 PMCID: PMC11542842 DOI: 10.1371/journal.pbio.3002863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2024] [Revised: 11/07/2024] [Accepted: 09/21/2024] [Indexed: 10/30/2024] Open
Abstract
Animals guide their behaviors through internal representations of the world in the brain. We aimed to understand how the macaque brain stores such general world knowledge, focusing on object color knowledge. Three functional magnetic resonance imaging (fMRI) experiments were conducted in macaque monkeys: viewing chromatic and achromatic gratings, viewing grayscale images of their familiar fruits and vegetables (e.g., grayscale strawberry), and viewing true- and false-colored objects (e.g., red strawberry and green strawberry). We observed robust object knowledge representations in the color patches, especially the one located around TEO: the activity patterns could classify grayscale pictures of objects based on their memory color and response patterns in these regions could translate between chromatic grating viewing and grayscale object viewing (e.g., red grating-grayscale images of strawberry), such that classifiers trained by viewing chromatic gratings could successfully classify grayscale object images according to their memory colors. Our results showed direct positive evidence of object color memory in macaque monkeys. These results indicate the perceptually grounded knowledge representation as a conservative memory mechanism and open a new avenue to study this particular (semantic) memory representation with macaque models.
Collapse
Affiliation(s)
- Minghui Zhao
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
- College of Life Sciences, University of Chinese Academy of Sciences, Beijing, China
| | - Yumeng Xin
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
- College of Life Sciences, University of Chinese Academy of Sciences, Beijing, China
| | - Haoyun Deng
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
- College of Life Sciences, University of Chinese Academy of Sciences, Beijing, China
| | - Zhentao Zuo
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
- College of Life Sciences, University of Chinese Academy of Sciences, Beijing, China
- Sino-Danish College, University of Chinese Academy of Sciences, Beijing, China
| | - Xiaoying Wang
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
- IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
- IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China
- Chinese Institute for Brain Research, Beijing, China
| | - Ning Liu
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China
- College of Life Sciences, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
11
|
Wen H, Wang D, Bi Y. Processing Language Partly Shares Neural Genetic Basis with Processing Tools and Body Parts. eNeuro 2024; 11:ENEURO.0138-24.2024. [PMID: 38886065 PMCID: PMC11298957 DOI: 10.1523/eneuro.0138-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Revised: 05/24/2024] [Accepted: 05/28/2024] [Indexed: 06/20/2024] Open
Abstract
Language is an evolutionarily salient faculty for humans that relies on a distributed brain network spanning across frontal, temporal, parietal, and subcortical regions. To understand whether the complex language network shares common or distinct genetic mechanisms, we examined the relationships between the genetic effects underlying the brain responses to language and a set of object domains that have been suggested to coevolve with language: tools, faces (indicating social), and body parts (indicating social and gesturing). Analyzing the twin datasets released by the Human Connectome Project that had functional magnetic resonance imaging data from human twin subjects (monozygotic and dizygotic) undergoing language and working memory tasks contrasting multiple object domains (198 females and 144 males for the language task; 192 females and 142 males for the working memory task), we identified a set of cortical regions in the frontal and temporal cortices and subcortical regions whose activity to language was significantly genetically influenced. The heterogeneity of the genetic effects among these language clusters was corroborated by significant differences of the human gene expression profiles (Allen Human Brain Atlas dataset). Among them, the bilateral basal ganglia (mainly dorsal caudate) exhibited a common genetic basis for language, tool, and body part processing, and the right superior temporal gyrus exhibited a common genetic basis for language and tool processing across multiple types of analyses. These results uncovered the heterogeneous genetic patterns of language neural processes, shedding light on the evolution of language and its shared origins with tools and bodily functions.
Collapse
Affiliation(s)
- Haojie Wen
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China
- IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
- Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
- School of Systems Science, Beijing Normal University, Beijing 100875, China
| | - Dahui Wang
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China
- Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
- School of Systems Science, Beijing Normal University, Beijing 100875, China
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China
- IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
- Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
- Chinese Institute for Brain Research, Beijing 102206, China
| |
Collapse
|
12
|
Tao Y, Ma L, Yu J, Zhang H. Memory-Based Cross-Modal Semantic Alignment Network for Radiology Report Generation. IEEE J Biomed Health Inform 2024; 28:4145-4156. [PMID: 38656853 DOI: 10.1109/jbhi.2024.3393018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/26/2024]
Abstract
Generating radiology reports automatically reduces the workload of radiologists and helps the diagnoses of specific diseases. Many existing methods take this task as modality transfer process. However, since the key information related to disease accounts for a small proportion in both image and report, it is hard for the model to learn the latent relation between the radiology image and its report, thus failing to generate fluent and accurate radiology reports. To tackle this problem, we propose a memory-based cross-modal semantic alignment model (MCSAM) following an encoder-decoder paradigm. MCSAM includes a well initialized long-term clinical memory bank to learn disease-related representations as well as prior knowledge for different modalities to retrieve and use the retrieved memory to perform feature consolidation. To ensure the semantic consistency of the retrieved cross modal prior knowledge, a cross-modal semantic alignment module (SAM) is proposed. SAM is also able to generate semantic visual feature embeddings which can be added to the decoder and benefits report generation. More importantly, to memorize the state and additional information while generating reports with the decoder, we use learnable memory tokens which can be seen as prompts. Extensive experiments demonstrate the promising performance of our proposed method which generates state-of-the-art performance on the MIMIC-CXR dataset.
Collapse
|
13
|
Tian S, Chen L, Wang X, Li G, Fu Z, Ji Y, Lu J, Wang X, Shan S, Bi Y. Vision matters for shape representation: Evidence from sculpturing and drawing in the blind. Cortex 2024; 174:241-255. [PMID: 38582629 DOI: 10.1016/j.cortex.2024.02.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 01/23/2024] [Accepted: 02/27/2024] [Indexed: 04/08/2024]
Abstract
Shape is a property that could be perceived by vision and touch, and is classically considered to be supramodal. While there is mounting evidence for the shared cognitive and neural representation space between visual and tactile shape, previous research tended to rely on dissimilarity structures between objects and had not examined the detailed properties of shape representation in the absence of vision. To address this gap, we conducted three explicit object shape knowledge production experiments with congenitally blind and sighted participants, who were asked to produce verbal features, 3D clay models, and 2D drawings of familiar objects with varying levels of tactile exposure, including tools, large nonmanipulable objects, and animals. We found that the absence of visual experience (i.e., in the blind group) led to stronger differences in animals than in tools and large objects, suggesting that direct tactile experience of objects is essential for shape representation when vision is unavailable. For tools with rich tactile/manipulation experiences, the blind produced overall good shapes comparable to the sighted, yet also showed intriguing differences. The blind group had more variations and a systematic bias in the geometric property of tools (making them stubbier than the sighted), indicating that visual experience contributes to aligning internal representations and calibrating overall object configurations, at least for tools. Taken together, the object shape representation reflects the intricate orchestration of vision, touch and language.
Collapse
Affiliation(s)
- Shuang Tian
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG, McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Lingjuan Chen
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG, McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Xiaoying Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG, McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Guochao Li
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG, McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Ze Fu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG, McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Yufeng Ji
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China
| | - Jiahui Lu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG, McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Xiaosha Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG, McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Shiguang Shan
- Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG, McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China.
| |
Collapse
|
14
|
Versace E, Freeland L, Emmerson MG. First-sight recognition of touched objects shows that chicks can solve Molyneux's problem. Biol Lett 2024; 20:20240025. [PMID: 38565149 PMCID: PMC10987231 DOI: 10.1098/rsbl.2024.0025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Accepted: 03/06/2024] [Indexed: 04/04/2024] Open
Abstract
If a congenitally blind person learns to distinguish between a cube and a sphere by touch, would they immediately recognize these objects by sight once their vision is restored? This question, posed by Molyneux in 1688, has puzzled philosophers and scientists since then. To overcome ethical and practical difficulties in the investigation of cross-modal recognition, we studied inexperienced poultry chicks, which can be reared in darkness until the moment of a visual test with no detrimental consequences. After hatching chicks in darkness, we exposed them to either tactile smooth or tactile bumpy stimuli for 24 h. Immediately after the tactile exposure, chicks were tested in a visual recognition task, during their first experience with light. At first sight, chicks that had been exposed in the tactile modality to smooth stimuli approached the visual smooth stimulus significantly more than those exposed to the tactile bumpy stimuli. These results show that visually inexperienced chicks can solve Molyneux's problem, indicating cross-modal recognition does not require previous multimodal experience. At least in this precocial species, supra-modal brain areas appear functional already at birth. This discovery paves the way for the investigation of predisposed cross-modal cognition that does not depend on visual experience.
Collapse
Affiliation(s)
- Elisabetta Versace
- Department of Biological and Experimental Psychology, School of Biological and Behavioural Sciences, Queen Mary University of London, 327 Mile End Road, London E1 4NS, UK
| | - Laura Freeland
- Department of Biological and Experimental Psychology, School of Biological and Behavioural Sciences, Queen Mary University of London, 327 Mile End Road, London E1 4NS, UK
| | - Michael G. Emmerson
- Department of Biological and Experimental Psychology, School of Biological and Behavioural Sciences, Queen Mary University of London, 327 Mile End Road, London E1 4NS, UK
| |
Collapse
|
15
|
Lettieri G, Handjaras G, Cappello EM, Setti F, Bottari D, Bruno V, Diano M, Leo A, Tinti C, Garbarini F, Pietrini P, Ricciardi E, Cecchetti L. Dissecting abstract, modality-specific and experience-dependent coding of affect in the human brain. SCIENCE ADVANCES 2024; 10:eadk6840. [PMID: 38457501 PMCID: PMC10923499 DOI: 10.1126/sciadv.adk6840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Accepted: 02/06/2024] [Indexed: 03/10/2024]
Abstract
Emotion and perception are tightly intertwined, as affective experiences often arise from the appraisal of sensory information. Nonetheless, whether the brain encodes emotional instances using a sensory-specific code or in a more abstract manner is unclear. Here, we answer this question by measuring the association between emotion ratings collected during a unisensory or multisensory presentation of a full-length movie and brain activity recorded in typically developed, congenitally blind and congenitally deaf participants. Emotional instances are encoded in a vast network encompassing sensory, prefrontal, and temporal cortices. Within this network, the ventromedial prefrontal cortex stores a categorical representation of emotion independent of modality and previous sensory experience, and the posterior superior temporal cortex maps the valence dimension using an abstract code. Sensory experience more than modality affects how the brain organizes emotional information outside supramodal regions, suggesting the existence of a scaffold for the representation of emotional states where sensory inputs during development shape its functioning.
Collapse
Affiliation(s)
- Giada Lettieri
- Crossmodal Perception and Plasticity Laboratory, Institute of Research in Psychology & Institute of Neuroscience, Université Catholique de Louvain, Louvain-la-Neuve, Belgium
- Social and Affective Neuroscience Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
| | - Giacomo Handjaras
- Social and Affective Neuroscience Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
| | - Elisa M. Cappello
- Social and Affective Neuroscience Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
| | - Francesca Setti
- Sensorimotor Experiences and Mental Representations Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
| | - Davide Bottari
- Sensorimotor Experiences and Mental Representations Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
- Sensory Experience Dependent Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
| | | | - Matteo Diano
- Department of Psychology, University of Turin, Turin, Italy
| | - Andrea Leo
- Department of of Translational Research and Advanced Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy
| | - Carla Tinti
- Department of Psychology, University of Turin, Turin, Italy
| | | | - Pietro Pietrini
- Forensic Neuroscience and Psychiatry Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
| | - Emiliano Ricciardi
- Sensorimotor Experiences and Mental Representations Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
- Sensory Experience Dependent Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
| | - Luca Cecchetti
- Social and Affective Neuroscience Group, MoMiLab, IMT School for Advanced Studies Lucca, Lucca, Italy
| |
Collapse
|
16
|
Lin N, Zhang X, Wang X, Wang S. The organization of the semantic network as reflected by the neural correlates of six semantic dimensions. BRAIN AND LANGUAGE 2024; 250:105388. [PMID: 38295716 DOI: 10.1016/j.bandl.2024.105388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 01/23/2024] [Accepted: 01/26/2024] [Indexed: 03/03/2024]
Abstract
Multiple sensory-motor and non-sensory-motor dimensions have been proposed for semantic representation, but it remains unclear how the semantic system is organized along them in the human brain. Using naturalistic fMRI data and large-scale semantic ratings, we investigated the overlaps and dissociations between the neural correlates of six semantic dimensions: vision, motor, socialness, emotion, space, and time. Our findings revealed a more complex semantic atlas than what is predicted by current neurobiological models of semantic representation. Brain regions that are selectively sensitive to specific semantic dimensions were found both within and outside the brain networks assumed to represent multimodal general and/or abstract semantics. Overlaps between the neural correlates of different semantic dimensions were mainly found inside the default mode network, concentrated in the left anterior superior temporal gyrus and angular gyrus, which have been proposed as two connector hubs that bridge the multimodal experiential semantic system and the language-supported semantic system.
Collapse
Affiliation(s)
- Nan Lin
- CAS Key Laboratory of Behavioural Sciences, Institute of Psychology, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China.
| | - Xiaohan Zhang
- State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS, Beijing, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Xiuyi Wang
- CAS Key Laboratory of Behavioural Sciences, Institute of Psychology, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
| | - Shaonan Wang
- State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS, Beijing, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
17
|
Hao F, Shi L, Sun Q, Che L, Jiang Y, Huang Z, Cheng X, Fan Z, Ding X. Space-time mapping on the sagittal axis in congenital blindness. PSYCHOLOGICAL RESEARCH 2024; 88:338-347. [PMID: 37620731 DOI: 10.1007/s00426-023-01871-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Accepted: 08/18/2023] [Indexed: 08/26/2023]
Abstract
Previous evolutionary perspectives proposed that the space-time mapping on the sagittal axis originates from visuo-locomotion coupling when walking/running forward. Accordingly, the congenitally blind could not have developed a sagittal mental timeline if the latter depends on such a visuo-locomotion coupling. However, this conclusion was reached in only a single empirical study (Rinaldi et al. in J Exp Psychol General 147:444-450, 2018), and its theoretical underpinnings are not entirely convincing as locally static and continuous auditory input undergoes a relatively similar change as function of self-locomotion, but this type of sensory-locomotion coupling is spared even in congenital blindness. Therefore, the present study systematically explored whether the congenitally blind show space-time mappings on the sagittal axis using different paradigms in three experiments. In Experiment 1, using a typical implicit RT task, the congenitally blind showed the same preferred space-time mapping in the sagittal dimension as normally sighted participants did. In Experiment 2, this space-time mapping occurred even automatically when temporal relations were task-irrelevant in a naming task. In Experiment 3, in an explicit space-time mapping task, the congenitally blind were more likely to locate the past behind and the future in front of their bodies. Moreover, most blind participants used spatial metaphors for their space-time mapping on the sagittal axis. These results supported the conclusion that the congenitally blind have a sagittal mental timeline, and that their sensory-locomotion coupling experience was either more similar to that of sighted participants or not critical for the space-time mapping. The present study, thus, also helps to clarify the origin of the sagittal mental timeline.
Collapse
Affiliation(s)
- Fengxiao Hao
- School of Psychology, Central China Normal University (CCNU), Wuhan, 430079, China
| | - Lingzheng Shi
- School of Psychology, Central China Normal University (CCNU), Wuhan, 430079, China
| | - Qiang Sun
- School of Psychology, Central China Normal University (CCNU), Wuhan, 430079, China
- Mental Health Education Center, Fuyang Institute of Technology, Fuyang, China
| | - Lu Che
- Xi'an Blind and Deafmute School, Xi'an, China
| | - Yuewen Jiang
- School of Psychology, Central China Normal University (CCNU), Wuhan, 430079, China
| | - Zhenyi Huang
- School of Psychology, Central China Normal University (CCNU), Wuhan, 430079, China
| | - Xiaorong Cheng
- School of Psychology, Central China Normal University (CCNU), Wuhan, 430079, China
| | - Zhao Fan
- School of Psychology, Central China Normal University (CCNU), Wuhan, 430079, China
| | - Xianfeng Ding
- School of Psychology, Central China Normal University (CCNU), Wuhan, 430079, China.
| |
Collapse
|
18
|
Fairhall SL. Sentence-level embeddings reveal dissociable word- and sentence-level cortical representation across coarse- and fine-grained levels of meaning. BRAIN AND LANGUAGE 2024; 250:105389. [PMID: 38306958 DOI: 10.1016/j.bandl.2024.105389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Revised: 01/09/2024] [Accepted: 01/26/2024] [Indexed: 02/04/2024]
Abstract
In this large-sample (N = 64) fMRI study, sentence embeddings (text-embedding-ada-002, OpenAI) and representational similarity analysis were used to contrast sentence-level and word-level semantic representation. Overall, sentence-level information resulted in a 20-25 % increase in the model's ability to captures neural representation when compared to word-level only information (word-order scrambled embeddings). This increase was relatively undifferentiated across the cortex. However, when coarse-grained (across thematic category) and fine-grained (within thematic category) combinatorial meaning were separately assessed, word- and sentence-level representations were seen to strongly dissociate across the cortex and to do so differently as a function of grain. Coarse-grained sentence-level representations were evident in occipitotemporal, ventral temporal and medial prefrontal cortex, while fine-grained differences were seen in lateral prefrontal and parietal cortex, middle temporal gyrus, the precuneus, and medial prefrontal cortex. This result indicates dissociable cortical substrates underly single concept versus combinatorial meaning and that different cortical regions specialise for fine- and coarse-grained meaning.
Collapse
Affiliation(s)
- Scott L Fairhall
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Italy.
| |
Collapse
|
19
|
Chen L. Synesthetic Correspondence: An Overview. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2024; 1437:101-119. [PMID: 38270856 DOI: 10.1007/978-981-99-7611-9_7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/26/2024]
Abstract
Intramodal and cross-modal perceptual grouping based on the spatial proximity and temporal closeness between multiple sensory stimuli, as an operational principle has built a coherent and meaningful representation of the multisensory event/object. To implement and investigate the cross-modal perceptual grouping, researchers have employed excellent paradigms of spatial/temporal ventriloquism and cross-modal dynamic capture and have revealed the conditional constraints as well as the functional facilitations among various correspondence of sensory properties, with featured behavioral evidence, computational framework as well as brain oscillation patterns. Typically, synesthetic correspondence as a special type of cross-modal correspondence can shape the efficiency and effect-size of cross-modal interaction. For example, factors such as pitch/loudness in the auditory dimension with size/brightness in the visual dimension could modulate the strength of the cross-modal temporal capture. The empirical behavioral findings, as well as psychophysical and neurophysiological evidence to address the cross-modal perceptual grouping and synesthetic correspondence, were summarized in this review. Finally, the potential applications (such as artificial synesthesia device) and how synesthetic correspondence interface with semantics (sensory linguistics), as well as the promising research questions in this field have been discussed.
Collapse
Affiliation(s)
- Lihan Chen
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.
- Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China.
- Key Laboratory of Machine Perception (Ministry of Education), Peking University, Beijing, China.
- National Key Laboratory of General Artificial Intelligence, Peking University, Beijing, China.
- National Engineering Laboratory for Big Data Analysis and Applications, Peking University, Beijing, China.
| |
Collapse
|
20
|
Kauf C, Ivanova AA, Rambelli G, Chersoni E, She JS, Chowdhury Z, Fedorenko E, Lenci A. Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely. Cogn Sci 2023; 47:e13386. [PMID: 38009752 DOI: 10.1111/cogs.13386] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Revised: 10/27/2023] [Accepted: 11/04/2023] [Indexed: 11/29/2023]
Abstract
Word co-occurrence patterns in language corpora contain a surprising amount of conceptual knowledge. Large language models (LLMs), trained to predict words in context, leverage these patterns to achieve impressive performance on diverse semantic tasks requiring world knowledge. An important but understudied question about LLMs' semantic abilities is whether they acquire generalized knowledge of common events. Here, we test whether five pretrained LLMs (from 2018's BERT to 2023's MPT) assign a higher likelihood to plausible descriptions of agent-patient interactions than to minimally different implausible versions of the same event. Using three curated sets of minimal sentence pairs (total n = 1215), we found that pretrained LLMs possess substantial event knowledge, outperforming other distributional language models. In particular, they almost always assign a higher likelihood to possible versus impossible events (The teacher bought the laptop vs. The laptop bought the teacher). However, LLMs show less consistent preferences for likely versus unlikely events (The nanny tutored the boy vs. The boy tutored the nanny). In follow-up analyses, we show that (i) LLM scores are driven by both plausibility and surface-level sentence features, (ii) LLM scores generalize well across syntactic variants (active vs. passive constructions) but less well across semantic variants (synonymous sentences), (iii) some LLM errors mirror human judgment ambiguity, and (iv) sentence plausibility serves as an organizing dimension in internal LLM representations. Overall, our results show that important aspects of event knowledge naturally emerge from distributional linguistic patterns, but also highlight a gap between representations of possible/impossible and likely/unlikely events.
Collapse
Affiliation(s)
- Carina Kauf
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
- McGovern Institute for Brain Research, Massachusetts Institute of Technology
| | - Anna A Ivanova
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
- McGovern Institute for Brain Research, Massachusetts Institute of Technology
- Computer Science and Artificial Intelligence Lab, Massachusetts Institute of Technology
| | - Giulia Rambelli
- Department of Modern Languages, Literatures and Cultures, University of Bologna
| | - Emmanuele Chersoni
- Department of Chinese and Bilingual Studies, Hong Kong Polytechnic University
| | - Jingyuan Selena She
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
- McGovern Institute for Brain Research, Massachusetts Institute of Technology
| | | | - Evelina Fedorenko
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
- McGovern Institute for Brain Research, Massachusetts Institute of Technology
| | - Alessandro Lenci
- Department of Philology, Literature, and Linguistics, University of Pisa
| |
Collapse
|
21
|
Dove G. Concepts require flexible grounding. BRAIN AND LANGUAGE 2023; 245:105322. [PMID: 37713771 DOI: 10.1016/j.bandl.2023.105322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 07/30/2023] [Accepted: 09/06/2023] [Indexed: 09/17/2023]
Abstract
Research on semantic memory has a problem. On the one hand, a robust body of evidence implicates sensorimotor regions in conceptual processing. On the other hand, a different body of evidence implicates a modality independent semantic system. The standard solution to this tension is to posit a hub-and-spoke system with modality independent hubs and modality specific spokes. In this paper, I argue in support of an alternative view of grounding which remains committed to neural reenactment but emphasizes the multimodal and multilevel nature of the semantic system. This view is built upon the recognition that abstraction is a design feature of concepts. Semantic memory employs hierarchically structured representations to capture different degrees of abstraction. Grounding does not work the way that many embodied approaches have assumed.
Collapse
Affiliation(s)
- Guy Dove
- Department of Philosophy, University of Louisville, United States.
| |
Collapse
|
22
|
Vignali L, Xu Y, Turini J, Collignon O, Crepaldi D, Bottini R. Spatiotemporal dynamics of abstract and concrete semantic representations. BRAIN AND LANGUAGE 2023; 243:105298. [PMID: 37399687 DOI: 10.1016/j.bandl.2023.105298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/11/2022] [Revised: 05/19/2023] [Accepted: 06/21/2023] [Indexed: 07/05/2023]
Abstract
Dual Coding Theories (DCT) suggest that meaning is represented in the brain by a double code: a language-derived code in the Anterior Temporal Lobe (ATL) and a sensory-derived code in perceptual and motor regions. Concrete concepts should activate both codes, while abstract ones rely solely on the linguistic code. To test these hypotheses, the present magnetoencephalography (MEG) experiment had participants judge whether visually presented words relate to the senses while we recorded brain responses to abstract and concrete semantic components obtained from 65 independently rated semantic features. Results evidenced early involvement of anterior-temporal and inferior-frontal brain areas in both abstract and concrete semantic information encoding. At later stages, occipital and occipito-temporal regions showed greater responses to concrete compared to abstract features. The present findings suggest that the concreteness of words is processed first with a transmodal/linguistic code, housed in frontotemporal brain systems, and only after with an imagistic/sensorimotor code in perceptual regions.
Collapse
Affiliation(s)
- Lorenzo Vignali
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy; International School for Advanced Studies (SISSA), Trieste, Italy
| | - Yangwen Xu
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy; International School for Advanced Studies (SISSA), Trieste, Italy
| | | | - Olivier Collignon
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy; Psychological Sciences Research Institute (IPSY) and Institute of NeuroScience (IoNS), University of Louvain, Louvain-la-Neuve, Belgium; School of Health Sciences, HES-SO Valais-Wallis, The Sense Innovation and Research Center, Lausanne and Sion, Switzerland
| | - Davide Crepaldi
- International School for Advanced Studies (SISSA), Trieste, Italy
| | - Roberto Bottini
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy.
| |
Collapse
|
23
|
Carota F, Nili H, Kriegeskorte N, Pulvermüller F. Experientially-grounded and distributional semantic vectors uncover dissociable representations of conceptual categories. LANGUAGE, COGNITION AND NEUROSCIENCE 2023; 39:1020-1044. [PMID: 39777206 PMCID: PMC11706208 DOI: 10.1080/23273798.2023.2232481] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/11/2022] [Accepted: 06/26/2023] [Indexed: 01/11/2025]
Abstract
Neuronal populations code similar concepts by similar activity patterns across the human brain's semantic networks. However, it is unclear to what extent such meaning-to-symbol mapping reflects distributional statistics, or experiential information grounded in sensorimotor and emotional knowledge. We asked whether integrating distributional and experiential data better distinguished conceptual categories than each method taken separately. We examined the similarity structure of fMRI patterns elicited by visually presented action- and object-related words using representational similarity analysis (RSA). We found that the distributional and experiential/integrative models respectively mapped the high-dimensional semantic space in left inferior frontal, anterior temporal, and in left precentral, posterior inferior/middle temporal cortex. Furthermore, results from model comparisons uncovered category-specific similarity patterns, as both distributional and experiential models matched the similarity patterns for action concepts in left fronto-temporal cortex, whilst the experiential/integrative (but not distributional) models matched the similarity patterns for object concepts in left fusiform and angular gyrus.
Collapse
Affiliation(s)
- Francesca Carota
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
- Donders Institute for Cognitive Neuroscience, Radboud University, Nijmegen, The Netherlands
- Cognition and Brain Sciences Unit, Medical Research Council, Cambridge, UK
| | - Hamed Nili
- Cognition and Brain Sciences Unit, Medical Research Council, Cambridge, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Nikolaus Kriegeskorte
- Cognition and Brain Sciences Unit, Medical Research Council, Cambridge, UK
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK
- Department of Psychology, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
- Department of Neuroscience, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
- Department of Electrical Engineering, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Friedemann Pulvermüller
- Cognition and Brain Sciences Unit, Medical Research Council, Cambridge, UK
- Brain Language Laboratory, Department of Philosophy and Humanities, WE4, Freie Universität Berlin, Berlin, Germany
- School of Mind and Brain, Humboldt Universität zu Berlin, Berlin, Germany
- Cluster of Excellence “Matters of Activity”, Humboldt Universität zu Berlin, Berlin, Germany
- Einstein Center for Neuroscience Berlin, Berlin, Germany
| |
Collapse
|
24
|
Xu Y, Vignali L, Sigismondi F, Crepaldi D, Bottini R, Collignon O. Similar object shape representation encoded in the inferolateral occipitotemporal cortex of sighted and early blind people. PLoS Biol 2023; 21:e3001930. [PMID: 37490508 PMCID: PMC10368275 DOI: 10.1371/journal.pbio.3001930] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 06/23/2023] [Indexed: 07/27/2023] Open
Abstract
We can sense an object's shape by vision or touch. Previous studies suggested that the inferolateral occipitotemporal cortex (ILOTC) implements supramodal shape representations as it responds more to seeing or touching objects than shapeless textures. However, such activation in the anterior portion of the ventral visual pathway could be due to the conceptual representation of an object or visual imagery triggered by touching an object. We addressed these possibilities by directly comparing shape and conceptual representations of objects in early blind (who lack visual experience/imagery) and sighted participants. We found that bilateral ILOTC in both groups showed stronger activation during a shape verification task than during a conceptual verification task made on the names of the same manmade objects. Moreover, the distributed activity in the ILOTC encoded shape similarity but not conceptual association among objects. Besides the ILOTC, we also found shape representation in both groups' bilateral ventral premotor cortices and intraparietal sulcus (IPS), a frontoparietal circuit relating to object grasping and haptic processing. In contrast, the conceptual verification task activated both groups' left perisylvian brain network relating to language processing and, interestingly, the cuneus in early blind participants only. The ILOTC had stronger functional connectivity to the frontoparietal circuit than to the left perisylvian network, forming a modular structure specialized in shape representation. Our results conclusively support that the ILOTC selectively implements shape representation independently of visual experience, and this unique functionality likely comes from its privileged connection to the frontoparietal haptic circuit.
Collapse
Affiliation(s)
- Yangwen Xu
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy
| | - Lorenzo Vignali
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy
- International School for Advanced Studies (SISSA), Trieste, Italy
| | | | - Davide Crepaldi
- International School for Advanced Studies (SISSA), Trieste, Italy
| | - Roberto Bottini
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy
| | - Olivier Collignon
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy
- Psychological Sciences Research Institute (IPSY) and Institute of NeuroScience (IoNS), University of Louvain, Louvain-la-Neuve, Belgium
- School of Health Sciences, HES-SO Valais-Wallis, The Sense Innovation and Research Center, Lausanne and Sion, Switzerland
| |
Collapse
|
25
|
EMERY KARAJ, ISHERWOOD ZOEYJ, WEBSTER MICHAELA. Gaining the system: limits to compensating color deficiencies through post-receptoral gain changes. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2023; 40:A16-A25. [PMID: 37132998 PMCID: PMC10157001 DOI: 10.1364/josaa.480035] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 12/14/2022] [Indexed: 05/04/2023]
Abstract
Color percepts of anomalous trichromats are often more similar to normal trichromats than predicted from their receptor spectral sensitivities, suggesting that post-receptoral mechanisms can compensate for chromatic losses. The basis for these adjustments and the extent to which they could discount the deficiency are poorly understood. We modeled the patterns of compensation that might result from increasing the gains in post-receptoral neurons to offset their weakened inputs. Individual neurons and the population responses jointly encode luminance and chromatic signals. As a result, they cannot independently adjust for a change in the chromatic inputs, predicting only partial recovery of the chromatic responses and increased responses to achromatic contrast. These analyses constrain the potential sites and mechanisms of compensation for a color loss and characterize the utility and limits of neural gain changes for calibrating color vision.
Collapse
Affiliation(s)
- KARA J. EMERY
- Department of Psychology and Graduate Program in Integrative Neuroscience, University of Nevada, Reno, Reno NV 89557
- Center for Data Science, New York University, New York NY 10011
| | - ZOEY J. ISHERWOOD
- Department of Psychology and Graduate Program in Integrative Neuroscience, University of Nevada, Reno, Reno NV 89557
| | - MICHAEL A. WEBSTER
- Department of Psychology and Graduate Program in Integrative Neuroscience, University of Nevada, Reno, Reno NV 89557
| |
Collapse
|
26
|
Dove GO. Rethinking the role of language in embodied cognition. Philos Trans R Soc Lond B Biol Sci 2023; 378:20210375. [PMID: 36571130 PMCID: PMC9791473 DOI: 10.1098/rstb.2021.0375] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 06/26/2022] [Indexed: 12/27/2022] Open
Abstract
There has been a lot of recent interest in the way that language might enhance embodied cognition. This interest is driven in large part by a growing body of evidence implicating the language system in various aspects of semantic memory-including, but not limited to, its apparent contribution to abstract concepts. In this essay, I develop and defend a novel account of the cognitive role played by language in our concepts. This account relies on the embodied nature of the language system itself, diverges in significant ways from traditional accounts, and is part of a flexible, multimodal and multilevel view of our conceptual system. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.
Collapse
Affiliation(s)
- Guy O. Dove
- Department of Philosophy, University of Louisville, 313 Humanities Building, Louisville, KY 40292, USA
| |
Collapse
|
27
|
Fu Z, Wang X, Wang X, Yang H, Wang J, Wei T, Liao X, Liu Z, Chen H, Bi Y. Different computational relations in language are captured by distinct brain systems. Cereb Cortex 2023; 33:997-1013. [PMID: 35332914 DOI: 10.1093/cercor/bhac117] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2021] [Revised: 02/25/2022] [Accepted: 02/26/2022] [Indexed: 11/14/2022] Open
Abstract
A critical way for humans to acquire information is through language, yet whether and how language experience drives specific neural semantic representations is still poorly understood. We considered statistical properties captured by 3 different computational principles of language (simple co-occurrence, network-(graph)-topological relations, and neural-network-vector-embedding relations) and tested the extent to which they can explain the neural patterns of semantic representations, measured by 2 functional magnetic resonance imaging experiments that shared common semantic processes. Distinct graph-topological word relations, and not simple co-occurrence or neural-network-vector-embedding relations, had unique explanatory power for the neural patterns in the anterior temporal lobe (capturing graph-common-neighbors), inferior frontal gyrus, and posterior middle/inferior temporal gyrus (capturing graph-shortest-path). These results were relatively specific to language: they were not explained by sensory-motor similarities and the same computational relations of visual objects (based on visual image database) showed effects in the visual cortex in the picture naming experiment. That is, different topological properties within language and the same topological computations (common-neighbors) for language and visual inputs are captured by different brain regions. These findings reveal the specific neural semantic representations along graph-topological properties of language, highlighting the information type-specific and statistical property-specific manner of semantic representations in the human brain.
Collapse
Affiliation(s)
- Ze Fu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Xiaosha Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Xiaoying Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Huichao Yang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China.,School of Systems Science, Beijing Normal University, Beijing 100875, China
| | - Jiahuan Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Tao Wei
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Xuhong Liao
- School of Systems Science, Beijing Normal University, Beijing 100875, China
| | - Zhiyuan Liu
- Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
| | - Huimin Chen
- School of Journalism and Communication, Tsinghua University, Beijing 100084, China
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China.,Chinese Institute for Brain Research, Beijing 102206, China
| |
Collapse
|
28
|
Xiong Z, Tian Y, Wang X, Wei K, Bi Y. Gravity matters for the neural representations of action semantics. Cereb Cortex 2023:6995384. [PMID: 36682884 DOI: 10.1093/cercor/bhad006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 12/31/2022] [Accepted: 01/02/2023] [Indexed: 01/24/2023] Open
Abstract
The dynamic relationship between the neural representation of action word semantics and specific sensorimotor experience remains controversial. Here, we temporarily altered human subjects' sensorimotor experience in a 15-day head-down tilt bed rest setting, a ground-based analog of microgravity that disproportionally affects sensorimotor experiences of the lower limbs, and examined whether such effector-dependent activity deprivation specifically affected the neural processes of comprehending verbs of lower-limb actions (e.g. to kick) relative to upper-limb ones (e.g. to pinch). Using functional magnetic resonance imaging, we compared the multivoxel neural patterns for such action words prior to and after bed rest. We found an effector-specific (lower vs. upper limb) experience modulation in subcortical sensorimotor-related and anterior temporal regions. The neural action semantic representations in other effector-specific verb semantic regions (e.g. left lateral posterior temporal cortex) and motor execution regions were robust against such experience alterations. These effector-specific, sensorimotor-experience-sensitive and experience-independent patterns of verb neural representation highlight the multidimensional and dynamic nature of semantic neural representation, and the broad influence of microgravity (hence gravity) environment on cognition.
Collapse
Affiliation(s)
- Ziyi Xiong
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Yu Tian
- National Key Laboratory of Human Factors Engineering, China Astronaut Research and Training Center, Beijing 100094, China.,School of Psychological and Cognitive Sciences, Peking University, Beijing 100871, China
| | - Xiaosha Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China
| | - Kunlin Wei
- School of Psychological and Cognitive Sciences, Peking University, Beijing 100871, China.,Beijing Key Laboratory of Behavior and Mental Health, Beijing 100871, China.,Peking-Tsinghua Center for Life Sciences, Peking University, Beijing 100871, China
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China.,Chinese Institute for Brain Research, Beijing 102206, China
| |
Collapse
|
29
|
Liu D, Hao L, Han L, Zhou Y, Qin S, Niki K, Shen W, Shi B, Luo J. The optimal balance of controlled and spontaneous processing in insight problem solving: fMRI evidence from Chinese idiom guessing. Psychophysiology 2023:e14240. [PMID: 36651323 DOI: 10.1111/psyp.14240] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Revised: 10/28/2022] [Accepted: 10/28/2022] [Indexed: 01/19/2023]
Abstract
Cognitive control is a key factor in insight generation. However, the neurocognitive mechanisms underlying the generation of insight for different cognitive control remain poorly understood. This study developed a parametric fMRI design, wherein hints for solving Chinese idiom riddles were gradually provided in a stepwise manner (from the first hint, H1, to the final hint, H4). By classifying the step-specific items solved in different hint-uncovering steps/conditions, we could identify insightful responses for different levels of spontaneous or controlled processing. At the behavioral level, the number of insightful problem solving trials reached the maximum at a intermediate level of the cognitively controlled processing and the spontaneously idea generating in H3, while the bilateral insular cortex and thalamus showed the robust engagement, implying the function of these regions in making the optimal balance between external hint processing and internal generated ideas. In addition, we identified brain areas, including the dorsolateral prefrontal cortex (dlPFC), angular gyrus (AG), dorsal anterior cingulate cortex (dACC), and precuneus (PreC), whose activities were parametrically increased with the levels of controlled (from H1 to H4) insightful processing which were increasingly produced by the sequentially revealed hints. Further representational similarity analysis (RSA) found that spontaneous processing in insight featured greater within-condition representational variabilities in widely distributed regions in the executive, salience, and default networks. Altogether, the present study provided new evidence for the relationship between the process of cognitive control and that of spontaneous idea generation in insight problem solving and demystified the function of the insula and thalamus as an interactive interface for the optimal balance of these two processes.
Collapse
Affiliation(s)
- Di Liu
- Beijing Key Laboratory of Learning and Cognition & School of Psychology, Capital Normal University, Beijing, China
| | - Lei Hao
- College of Teacher Education, Southwest University, Chongqing, China.,State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Faculty of Psychology at Beijing Normal University, Beijing, China
| | - Lei Han
- School of Psychology, Shandong Normal University, Jinan, China
| | - Ying Zhou
- Beijing Key Laboratory of Learning and Cognition & School of Psychology, Capital Normal University, Beijing, China
| | - Shaozheng Qin
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Faculty of Psychology at Beijing Normal University, Beijing, China
| | - Kazuhisa Niki
- Human Informatics Research Institute, Advanced Industrial Science and Technology, Tsukuba, Japan.,Keio University Graduate School of Human Relations, Keio University, Tokyo, Japan
| | - Wangbing Shen
- School of Public Administration and Institute of Applied Psychology, Hohai University, Nanjing, China
| | - Baoguo Shi
- Beijing Key Laboratory of Learning and Cognition & School of Psychology, Capital Normal University, Beijing, China.,College of Teacher Education, Southwest University, Chongqing, China
| | - Jing Luo
- Beijing Key Laboratory of Learning and Cognition & School of Psychology, Capital Normal University, Beijing, China.,Department of Psychology, Shaoxing University, Shaoxing, China
| |
Collapse
|
30
|
Bosten JM, Coen-Cagli R, Franklin A, Solomon SG, Webster MA. Calibrating Vision: Concepts and Questions. Vision Res 2022; 201:108131. [PMID: 37139435 PMCID: PMC10151026 DOI: 10.1016/j.visres.2022.108131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.
Collapse
Affiliation(s)
| | - Ruben Coen-Cagli
- Department of Systems Computational Biology, and Dominick P. Purpura Department of Neuroscience, and Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Bronx NY
| | | | - Samuel G Solomon
- Institute of Behavioural Neuroscience, Department of Experimental Psychology, University College London, UK
| | | |
Collapse
|
31
|
Xue G. From remembering to reconstruction: The transformative neural representation of episodic memory. Prog Neurobiol 2022; 219:102351. [PMID: 36089107 DOI: 10.1016/j.pneurobio.2022.102351] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2022] [Revised: 08/23/2022] [Accepted: 09/05/2022] [Indexed: 11/29/2022]
Abstract
Although memory has long been recognized as a generative process, neural research of memory in recent decades has been predominantly influenced by Tulving's "mental time traveling" perspective and focused on the reactivation and consolidation of encoded memory representations. With the development of multiple powerful analytical approaches to characterize the contents and formats of neural representations, recent studies are able to provide detailed examinations of the representations at various processing stages and have provided exciting new insights into the transformative nature of episodic memory. These studies have revealed the rapid, substantial, and continuous transformation of memory representation during the encoding, maintenance, consolidation, and retrieval of both single and multiple events, as well as event sequences. These transformations are characterized by the abstraction, integration, differentiation, and reorganization of memory representations, enabling the long-term retention and generalization of memory. These studies mark a significant shift in perspective from remembering to reconstruction, which might better reveal the nature of memory and its roles in supporting more effective learning, adaptive decision-making, and creative problem solving.
Collapse
Affiliation(s)
- Gui Xue
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China; Chinese Institute for Brain Research, Beijing 102206, PR China.
| |
Collapse
|
32
|
Wang X, Krieger-Redwood K, Zhang M, Cui Z, Wang X, Karapanagiotidis T, Du Y, Leech R, Bernhardt BC, Margulies DS, Smallwood J, Jefferies E. Physical distance to sensory-motor landmarks predicts language function. Cereb Cortex 2022; 33:4305-4318. [PMID: 36066439 PMCID: PMC10110440 DOI: 10.1093/cercor/bhac344] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Revised: 08/01/2022] [Accepted: 08/02/2022] [Indexed: 11/14/2022] Open
Abstract
Auditory language comprehension recruits cortical regions that are both close to sensory-motor landmarks (supporting auditory and motor features) and far from these landmarks (supporting word meaning). We investigated whether the responsiveness of these regions in task-based functional MRI is related to individual differences in their physical distance to primary sensorimotor landmarks. Parcels in the auditory network, that were equally responsive across story and math tasks, showed stronger activation in individuals who had less distance between these parcels and transverse temporal sulcus, in line with the predictions of the "tethering hypothesis," which suggests that greater proximity to input regions might increase the fidelity of sensory processing. Conversely, language and default mode parcels, which were more active for the story task, showed positive correlations between individual differences in activation and sensory-motor distance from primary sensory-motor landmarks, consistent with the view that physical separation from sensory-motor inputs supports aspects of cognition that draw on semantic memory. These results demonstrate that distance from sensorimotor regions provides an organizing principle of functional differentiation within the cortex. The relationship between activation and geodesic distance to sensory-motor landmarks is in opposite directions for cortical regions that are proximal to the heteromodal (DMN and language network) and unimodal ends of the principal gradient of intrinsic connectivity.
Collapse
Affiliation(s)
- Xiuyi Wang
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China.,Department of Psychology, University of York, Heslington, York YO10 5DD, UK
| | | | - Meichao Zhang
- Department of Psychology, University of York, Heslington, York YO10 5DD, UK
| | - Zaixu Cui
- Chinese Institute for Brain Research, Beijing 102206, China
| | - Xiaokang Wang
- Department of Biomedical Engineering, University of California, Davis, CA 95616, USA
| | | | - Yi Du
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China.,Chinese Institute for Brain Research, Beijing 102206, China.,CAS Center for Excellence in Brain Science and Intelligence Technology, Shanghai 200031, China.,Department of Psychology, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Robert Leech
- Centre for Neuroimaging Science, Kings College London, London, UK
| | - Boris C Bernhardt
- McConnell Brain Imaging Centre, McGill University, Montreal, Quebec, Canada
| | - Daniel S Margulies
- Integrative Neuroscience and Cognition Center (UMR 8002), Centre National de la Recherche Scientifique (CNRS) and Université de Paris, Paris, France.,Wellcome Centre for Integrative Neuroimaging, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | | | | |
Collapse
|
33
|
Zhou Y, Li J, Chi J, Tang W, Zheng Y. Set-CNN: A text convolutional neural network based on semantic extension for short text classification. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.109948] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
34
|
Shao X, Chen C, Loftus EF, Xue G, Zhu B. Dynamic changes in neural representations underlie the repetition effect on false memory. Neuroimage 2022; 259:119442. [PMID: 35788042 DOI: 10.1016/j.neuroimage.2022.119442] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Revised: 06/19/2022] [Accepted: 06/30/2022] [Indexed: 11/18/2022] Open
Abstract
Restudying word lists (e.g., dream, awake, and bed) strengthens true memory of the studied words and reduces false memory for unstudied but semantically related lures (e.g., sleep). Yet, the neural mechanisms involved in this repetition effect on false memory remain unclear. Possible mechanisms involve item-specific and semantic neural representations at encoding, and the memory strength between encoding and retrieval. This study first replicated the behavioral results (Exp. 1) and then investigated various neural mechanisms by using slow event-related functional magnetic resonance imaging (fMRI) and representational similarity analysis (Exp. 2). Behavioral results confirmed that restudy improved true memory and reduced false memory. The fMRI results showed that restudy induced item-specific neural representations at encoding in the left occipital pole, but reduced neural overlap between semantic representations at encoding in the left temporal pole. Individual differences in these two encoding neural mechanisms were correlated with the behavioral measure of false memory, with greater restudy-induced representational changes at encoding (item-specific neural representations and reduced neural overlap between semantic representations) being associated with lower false memory. Moreover, restudy enhanced the memory strength between encoding and retrieval in the visuoparietal cortex but reduced it in the frontal cortex. These findings suggest that dynamic changes in neural representations underlie the repetition effect on false memory, supporting a dual-coding neural framework.
Collapse
Affiliation(s)
- Xuhao Shao
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China; Institute of Developmental Psychology, Beijing Normal University, Beijing 100875, China; Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China; IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Chuansheng Chen
- Department of Psychological Science, University of California, Irvine, CA 92697, United States
| | - Elizabeth F Loftus
- Department of Psychological Science, University of California, Irvine, CA 92697, United States
| | - Gui Xue
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China
| | - Bi Zhu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China; Institute of Developmental Psychology, Beijing Normal University, Beijing 100875, China; Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing 100875, China; IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.
| |
Collapse
|
35
|
What the study of spinal cord injured patients can tell us about the significance of the body in cognition. Psychon Bull Rev 2022; 29:2052-2069. [PMID: 35697914 DOI: 10.3758/s13423-022-02129-6] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/18/2022] [Indexed: 11/08/2022]
Abstract
Although in the last three decades philosophers, psychologists and neuroscientists have produced numerous studies on human cognition, the debate concerning its nature is still heated and current views on the subject are somewhat antithetical. On the one hand, there are those who adhere to a view implying 'disembodiment' which suggests that cognition is based entirely on symbolic processes. On the other hand, a family of theories referred to as the Embodied Cognition Theories (ECT) postulate that creating and maintaining cognition is linked with varying degrees of inherence to somatosensory and motor representations. Spinal cord injury induces a massive body-brain disconnection with the loss of sensory and motor bodily functions below the lesion level but without directly affecting the brain. Thus, SCI may represent an optimal model for testing the role of the body in cognition. In this review, we describe post-lesional cognitive modifications in relation to body, space and action representations and various instances of ECT. We discuss the interaction between body-grounded and symbolic processes in adulthood with relevant modifications after body-brain disconnection.
Collapse
|
36
|
Wang Y, Zeng Y. Multisensory Concept Learning Framework Based on Spiking Neural Networks. Front Syst Neurosci 2022; 16:845177. [PMID: 35645741 PMCID: PMC9133338 DOI: 10.3389/fnsys.2022.845177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2021] [Accepted: 04/20/2022] [Indexed: 11/13/2022] Open
Abstract
Concept learning highly depends on multisensory integration. In this study, we propose a multisensory concept learning framework based on brain-inspired spiking neural networks to create integrated vectors relying on the concept's perceptual strength of auditory, gustatory, haptic, olfactory, and visual. With different assumptions, two paradigms: Independent Merge (IM) and Associate Merge (AM) are designed in the framework. For testing, we employed eight distinct neural models and three multisensory representation datasets. The experiments show that integrated vectors are closer to human beings than the non-integrated ones. Furthermore, we systematically analyze the similarities and differences between IM and AM paradigms and validate the generality of our framework.
Collapse
Affiliation(s)
- Yuwei Wang
- Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Yi Zeng
- Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
- National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- *Correspondence: Yi Zeng
| |
Collapse
|
37
|
Cherukunnath D, Singh AP. Exploring Cognitive Processes of Knowledge Acquisition to Upgrade Academic Practices. Front Psychol 2022; 13:682628. [PMID: 35602694 PMCID: PMC9120965 DOI: 10.3389/fpsyg.2022.682628] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Accepted: 03/09/2022] [Indexed: 11/13/2022] Open
Abstract
The development of cognitive functions follows certain pathways through brain maturation. Concepts taught at school can be reinforced by understanding the related cognitive functions that enhance learning. The cultural and social diversities faced by the education system worldwide can be solved by understanding the unifying cognitive processes of learning. This knowledge can be effectively used to devise better curriculum and training for students. Cognition, conation, and emotional regulation are the main components that determine an individual’s efficiency to deal with various situations. How the brain receives input, perceives, and organizes these information lays the foundation for learning. The objectives of the study were (i) to explore age-group specific inputs for knowledge acquisition, (ii) to relate knowledge organization to the cognitive processes, and (iii) to identify factors that strengthen the knowledge ensemble through subject-domain allied training. The review focused on studies related to elementary school age (below 7 years), middle school age (7–12 years), and high school age (12 years and above). Published journal articles related to the objectives were randomly reviewed to establish a possible relationship. The findings of this review can help to advance student learning practices and instructional strategies. The findings are listed below. (i) Acquisition of knowledge during early childhood is based on sensory-motor integration on which attentional, perceptual, memory, language, and socialization systems develop. As brain development progresses toward adolescence, meta-awareness and social-emotional cognition influence the student learning process. (ii) Knowledge representations can be strengthened by domain-specific training inputs. (iii) Associational integration of the developmental, cognitive, and conative processes are indicators of curriculum strength. (iv) The strengthening of cognitive processes by rerouting through complementary neural circuitry, such as music, arts, real-life-based experiments, and physical exercises, is an effective way to improve child-friendly instructions.
Collapse
Affiliation(s)
| | - Anita Puri Singh
- Department of Psychology, Government M L B Girls PG College, Bhopal, Bhopal, India
| |
Collapse
|
38
|
Wang Y, Zeng Y. Statistical Analysis of Multisensory and Text-Derived Representations on Concept Learning. Front Comput Neurosci 2022; 16:861265. [PMID: 35615056 PMCID: PMC9125787 DOI: 10.3389/fncom.2022.861265] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Accepted: 03/31/2022] [Indexed: 11/20/2022] Open
Abstract
When learning concepts, cognitive psychology research has revealed that there are two types of concept representations in the human brain: language-derived codes and sensory-derived codes. For the objective of human-like artificial intelligence, we expect to provide multisensory and text-derived representations for concepts in AI systems. Psychologists and computer scientists have published lots of datasets for the two kinds of representations, but as far as we know, no systematic work exits to analyze them together. We do a statistical study on them in this work. We want to know if multisensory vectors and text-derived vectors reflect conceptual understanding and if they are complementary in terms of cognition. Four experiments are presented in this work, all focused on multisensory representations labeled by psychologists and text-derived representations generated by computer scientists for concept learning, and the results demonstrate that (1) for the same concept, both forms of representations can properly reflect the concept, but (2) the representational similarity analysis findings reveal that the two types of representations are significantly different, (3) as the concreteness of the concept grows larger, the multisensory representation of the concept becomes closer to human beings than the text-derived representation, and (4) we verified that combining the two improves the concept representation.
Collapse
Affiliation(s)
- Yuwei Wang
- Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Yi Zeng
- Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
- National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- *Correspondence: Yi Zeng
| |
Collapse
|
39
|
Isella V, Rosazza C, Ferri F, Gazzotti M, Impagnatiello V, Mapelli C, Morzenti S, Crivellaro C, Appollonio IM, Ferrarese C. Learning From Mistakes: Cognitive and Metabolic Correlates of Errors on Picture Naming in the Alzheimer’s Disease Spectrum. J Alzheimers Dis 2022; 87:1033-1053. [DOI: 10.3233/jad-220053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
Background: Analysis of subtypes of picture naming errors produced by patients with Alzheimer’s disease (AD) have seldom been investigated yet may clarify the cognitive and neural underpinnings of naming in the AD spectrum. Objective: To elucidate the neurocognitive bases of picture naming in AD through a qualitative analysis of errors. Methods: Over 1000 naming errors produced by 70 patients with amnestic, visuospatial, linguistic, or frontal AD were correlated with general cognitive tests and with distribution of hypometabolism on FDG-PET. Results: Principal component analysis identified 1) a Visual processing factor clustering visuospatial tests and unrecognized stimuli, pure visual errors and visual-semantic errors, associated with right parieto-occipital hypometabolism; 2) a Concept-Lemma factor grouping language tests and anomias, circumlocutions, superordinates, and coordinates, correlated with left basal temporal hypometabolism; 3) a Lemma-Phonology factor including the digit span and phonological errors, linked with left temporo-parietal hypometabolism. Regression of brain metabolism on individual errors showed that errors due to impairment of basic and higher-order processing of object visual attributes or of their interaction with semantics, were related with bilateral occipital and left occipito-temporal dysfunction. Omissions and superordinates were linked to degradation of broad and basic concepts in the left basal temporal cortex. Semantic-lexical errors derived from faulty semantically- and phonologically-driven lexical retrieval in the left superior and middle temporal gyri. Generation of nonwords was underpinned by of phonological impairment within the left inferior parietal cortex. Conclusion: Analysis of individual naming errors allowed to outline a comprehensive anatomo-functional model of picture naming in classical and atypical AD.
Collapse
Affiliation(s)
- Valeria Isella
- Department of Neurology, S. Gerardo Hospital, Monza, University of Milano - Bicocca, Italy
- NeuroMI, University of Milano - Bicocca, Italy
| | - Cristina Rosazza
- Dipartimento di Studi Umanistici (DISTUM), Università degli Studi di Urbino Carlo Bo, Urbino, Italy
- Neuroradiology Unit, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan, Italy
| | - Francesca Ferri
- Department of Neurology, S. Gerardo Hospital, Monza, University of Milano - Bicocca, Italy
- NeuroMI, University of Milano - Bicocca, Italy
| | - Maria Gazzotti
- Department of Neurology, S. Gerardo Hospital, Monza, University of Milano - Bicocca, Italy
| | | | - Cristina Mapelli
- Department of Neurology, S. Gerardo Hospital, Monza, University of Milano - Bicocca, Italy
- NeuroMI, University of Milano - Bicocca, Italy
| | - Sabrina Morzenti
- Medical Physics, S. Gerardo Hospital, Monza, Italy
- NeuroMI, University of Milano - Bicocca, Italy
| | - Cinzia Crivellaro
- Nuclear Medicine, S. Gerardo Hospital, Monza, Italy
- NeuroMI, University of Milano - Bicocca, Italy
| | - Ildebrando M. Appollonio
- Department of Neurology, S. Gerardo Hospital, Monza, University of Milano - Bicocca, Italy
- NeuroMI, University of Milano - Bicocca, Italy
| | - Carlo Ferrarese
- Department of Neurology, S. Gerardo Hospital, Monza, University of Milano - Bicocca, Italy
- NeuroMI, University of Milano - Bicocca, Italy
| |
Collapse
|
40
|
Bottini R, Nava E, De Cuntis I, Benetti S, Collignon O. Synesthesia in a congenitally blind individual. Neuropsychologia 2022; 170:108226. [DOI: 10.1016/j.neuropsychologia.2022.108226] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2021] [Revised: 02/25/2022] [Accepted: 03/23/2022] [Indexed: 11/25/2022]
|
41
|
Chen M, Zhang L, Jiang Q. Gender Difference in Cognitive Function Among Stable Schizophrenia: A Network Perspective. Neuropsychiatr Dis Treat 2022; 18:2991-3000. [PMID: 36578902 PMCID: PMC9792107 DOI: 10.2147/ndt.s393586] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/15/2022] [Accepted: 12/06/2022] [Indexed: 12/24/2022] Open
Abstract
OBJECTIVE To investigate the gender differences and influencing factors of cognitive function in stable schizophrenic patients, and to explore the cognitive characteristics of male and female patients. METHODS A total of 298 patients with chronic schizophrenia were divided into two groups according to gender. The differences of demographic and clinical characteristics between the two groups were firstly analyzed. Then the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) was used to measure their cognitive function, and the correlation between cognitive function and demographic characteristics and clinical characteristics was analyzed. Finally, the gender-based cognitive characteristics were explored through network analysis. RESULTS There was no significant difference in the RBANS total score and sub-item score between the male schizophrenia and female schizophrenia patients. Correlation analysis showed that RBANS total score was inversely proportional with age, duration and Positive and Negative Syndrome Scale (PANSS) score in male schizophrenia, while being directly proportional with age at onset and inversely proportional with PANSS score in female schizophrenia. Network analysis showed that language was the core of cognitive function for male schizophrenia, and the delayed memory was the core of cognitive function for female schizophrenia. CONCLUSION There was no significant gender difference in cognitive function score among patients with stable schizophrenia. The core cognitive functions of male and female schizophrenia are language and delayed memory, respectively.
Collapse
Affiliation(s)
- Mengyi Chen
- Department of Geriatric, Shanghai Pudong New Area Mental Health Center, Tongji University School of Medicine, Shanghai, People's Republic of China
| | - Lei Zhang
- Department of Geriatric, Shanghai Pudong New Area Mental Health Center, Tongji University School of Medicine, Shanghai, People's Republic of China
| | - Qi Jiang
- Department of Geriatric, Shanghai Pudong New Area Mental Health Center, Tongji University School of Medicine, Shanghai, People's Republic of China
| |
Collapse
|
42
|
Wang X, Li G, Zhao G, Li Y, Wang B, Lin CP, Liu X, Bi Y. Social and emotion dimensional organizations in the abstract semantic space: the neuropsychological evidence. Sci Rep 2021; 11:23572. [PMID: 34876617 PMCID: PMC8651696 DOI: 10.1038/s41598-021-02824-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Accepted: 11/12/2021] [Indexed: 11/25/2022] Open
Abstract
An essential aspect of human cognition is supported by a rich reservoir of abstract concepts without tangible external referents (e.g., “honor”, “relationship”, “direction”). While decades of research showed that the neural organization of conceptual knowledge referring to concrete words respects domains of evolutionary salience and sensorimotor attributes, the organization principles of abstract word meanings are poorly understood. Here, we provide neuropsychological evidence for a domain (sociality) and attribute (emotion) structure in abstract word processing. Testing 34 brain-damaged patients on a word-semantic judgment task, we observed double dissociations between social and nonsocial words and a single dissociation of sparing of emotional (relative to non-emotional) words. The lesion profiles of patients with specific dissociations suggest potential neural correlates positively or negatively associated with each dimension. These results unravel a general domain-attribute architecture of word meanings and highlight the roles of the social domain and the emotional attribute in the non-object semantic space.
Collapse
Affiliation(s)
- Xiaosha Wang
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China
| | - Guochao Li
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China
| | - Gang Zhao
- Department of Neurosurgery, First Hospital of Jilin University, Changchun, China
| | - Yunqian Li
- Department of Neurosurgery, First Hospital of Jilin University, Changchun, China
| | - Bijun Wang
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China
| | - Ching-Po Lin
- Department of Neurosurgery, First Hospital of Jilin University, Changchun, China.,Institute of Neuroscience, National Yang-Ming University, Taipei, Taiwan.,Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
| | - Xinrui Liu
- Department of Neurosurgery, First Hospital of Jilin University, Changchun, China.
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China. .,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, China. .,Chinese Institute for Brain Research, Beijing, China.
| |
Collapse
|
43
|
Liu J, Zhang H, Yu T, Ren L, Ni D, Yang Q, Lu B, Zhang L, Axmacher N, Xue G. Transformative neural representations support long-term episodic memory. SCIENCE ADVANCES 2021; 7:eabg9715. [PMID: 34623910 PMCID: PMC8500506 DOI: 10.1126/sciadv.abg9715] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/07/2021] [Accepted: 08/17/2021] [Indexed: 06/13/2023]
Abstract
Memory is often conceived as a dynamic process that involves substantial transformations of mental representations. However, the neural mechanisms underlying these transformations and their role in memory formation and retrieval have only started to be elucidated. Combining intracranial EEG recordings with deep neural network models, we provide a detailed picture of the representational transformations from encoding to short-term memory maintenance and long-term memory retrieval that underlie successful episodic memory. We observed substantial representational transformations during encoding. Critically, more pronounced semantic representational formats predicted better subsequent long-term memory, and this effect was mediated by more consistent item-specific representations across encoding events. The representations were further transformed right after stimulus offset, and the representations during long-term memory retrieval were more similar to those during short-term maintenance than during encoding. Our results suggest that memory representations pass through multiple stages of transformations to achieve successful long-term memory formation and recall.
Collapse
Affiliation(s)
- Jing Liu
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Hui Zhang
- Department of Neuropsychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Bochum 44801, Germany
| | - Tao Yu
- Beijing Institute of Functional Neurosurgery, Xuanwu Hospital, Capital Medical University, Beijing 100053, China
| | - Liankun Ren
- Comprehensive Epilepsy Center of Beijing, Department of Neurology, Xuanwu Hospital, Capital Medical University, Beijing 100053, China
| | - Duanyu Ni
- Beijing Institute of Functional Neurosurgery, Xuanwu Hospital, Capital Medical University, Beijing 100053, China
| | - Qinhao Yang
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Baoqing Lu
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Liang Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Nikolai Axmacher
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
- Department of Neuropsychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Bochum 44801, Germany
| | - Gui Xue
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| |
Collapse
|
44
|
Wang X, Bi Y. Idiosyncratic Tower of Babel: Individual Differences in Word-Meaning Representation Increase as Word Abstractness Increases. Psychol Sci 2021; 32:1617-1635. [PMID: 34546824 DOI: 10.1177/09567976211003877] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Humans primarily rely on language to communicate, on the basis of a shared understanding of the basic building blocks of communication: words. Do we mean the same things when we use the same words? Although cognitive neural research on semantics has revealed the common principles of word-meaning representation, the factors underlying the potential individual variations in word meanings are unknown. Here, we empirically characterized the intersubject consistency of 90 words across 20 adult subjects (10 female) using both behavioral measures (rating-based semantic-relationship patterns) and neuroimaging measures (word-evoked brain activity patterns). Across both the behavioral and neuroimaging experiments, we showed that the magnitude of individual disagreements on word meanings could be modeled on the basis of how much language or sensory experience is associated with a word and that this variation increases with word abstractness. Uncovering the cognitive and neural origins of word-meaning disagreements across individuals has implications for potential mechanisms to modulate such disagreements.
Collapse
Affiliation(s)
- Xiaosha Wang
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University.,IDG/McGovern Institute for Brain Research, Beijing Normal University.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University.,IDG/McGovern Institute for Brain Research, Beijing Normal University.,Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University.,Chinese Institute for Brain Research, Beijing, China
| |
Collapse
|
45
|
Dual coding of knowledge in the human brain. Trends Cogn Sci 2021; 25:883-895. [PMID: 34509366 DOI: 10.1016/j.tics.2021.07.006] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2021] [Revised: 07/10/2021] [Accepted: 07/14/2021] [Indexed: 11/23/2022]
Abstract
How does the human brain code knowledge about the world? While disciplines such as artificial intelligence represent world knowledge based on human language, neurocognitive models of knowledge have been dominated by sensory embodiment, in which knowledge is derived from sensory/motor experience and supported by high-level sensory/motor and association cortices. The neural correlates of an alternative disembodied symbolic system had previously been difficult to establish. A recent line of studies exploring knowledge about visual properties, such as color, in visually deprived individuals converge to provide positive, compelling evidence for non-sensory, language-derived, knowledge representation in dorsal anterior temporal lobe and extended language network, in addition to the sensory-derived representations, leading to a sketch of a dual-coding knowledge neural framework.
Collapse
|
46
|
Abstract
Empiricist philosophers such as Locke famously argued that people born blind might learn arbitrary color facts (e.g., marigolds are yellow) but would lack color understanding. Contrary to this intuition, we find that blind and sighted adults share causal understanding of color, despite not always agreeing about arbitrary color facts. Relative to sighted people, blind individuals are less likely to generate "yellow" for banana and "red" for stop sign but make similar generative inferences about real and novel objects' colors, and provide similar causal explanations. For example, people infer that two natural kinds (e.g., bananas) and two artifacts with functional colors (e.g., stop signs) are more likely to have the same color than two artifacts with nonfunctional colors (e.g., cars). People develop intuitive and inferentially rich "theories" of color regardless of visual experience. Linguistic communication is more effective at aligning intuitive theories than knowledge of arbitrary facts.
Collapse
|
47
|
Yu X, Ferradal SL, Sliva DD, Dunstan J, Carruthers C, Sanfilippo J, Zuk J, Zöllei L, Boyd E, Gagoski B, Ou Y, Grant PE, Gaab N. Functional Connectivity in Infancy and Toddlerhood Predicts Long-Term Language and Preliteracy Outcomes. Cereb Cortex 2021; 32:bhab230. [PMID: 34347052 PMCID: PMC10847903 DOI: 10.1093/cercor/bhab230] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023] Open
Abstract
Functional connectivity (FC) techniques can delineate brain organization as early as infancy, enabling the characterization of early brain characteristics associated with subsequent behavioral outcomes. Previous studies have identified specific functional networks in infant brains that underlie cognitive abilities and pathophysiology subsequently observed in toddlers and preschoolers. However, it is unknown whether and how functional networks emerging within the first 18 months of life contribute to the development of higher order, complex functions of language/literacy at school-age. This 5-year longitudinal imaging project starting in infancy, utilized resting-state functional magnetic resonance imaging and demonstrated prospective associations between FC in infants/toddlers and subsequent language and foundational literacy skills at 6.5 years old. These longitudinal associations were shown independently of key environmental influences and further present in a subsample of infant imaging data (≤12 months), suggesting early emerged functional networks specifically linked to high-order language and preliteracy skills. Moreover, emergent language skills in infancy and toddlerhood contributed to the prospective associations, implicating a role of early linguistic experiences in shaping the FC correlates of long-term oral language skills. The current results highlight the importance of functional organization established in infancy and toddlerhood as a neural scaffold underlying the learning process of complex cognitive functions.
Collapse
Affiliation(s)
- Xi Yu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China
- Laboratories of Cognitive Neuroscience, Division of Developmental Medicine, Department of Medicine, Boston Children’s Hospital, Boston, MA 02115, USA
| | - Silvina L Ferradal
- Department of Intelligent Systems Engineering, Indiana University, Bloomington, IN 47408, USA
| | - Danielle D Sliva
- Laboratories of Cognitive Neuroscience, Division of Developmental Medicine, Department of Medicine, Boston Children’s Hospital, Boston, MA 02115, USA
- Department of Neuroscience, Brown University, Providence, RI 02912, USA
| | - Jade Dunstan
- Laboratories of Cognitive Neuroscience, Division of Developmental Medicine, Department of Medicine, Boston Children’s Hospital, Boston, MA 02115, USA
| | - Clarisa Carruthers
- Laboratories of Cognitive Neuroscience, Division of Developmental Medicine, Department of Medicine, Boston Children’s Hospital, Boston, MA 02115, USA
| | - Joseph Sanfilippo
- Laboratories of Cognitive Neuroscience, Division of Developmental Medicine, Department of Medicine, Boston Children’s Hospital, Boston, MA 02115, USA
| | - Jennifer Zuk
- Laboratories of Cognitive Neuroscience, Division of Developmental Medicine, Department of Medicine, Boston Children’s Hospital, Boston, MA 02115, USA
- Harvard Medical School, Boston, MA 02115, USA
- Department of Speech, Language & Hearing Sciences, Boston University, Boston, MA 02215, USA
| | - Lilla Zöllei
- A.A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA 02129, USA
| | - Emma Boyd
- A.A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA 02129, USA
| | - Borjan Gagoski
- Harvard Medical School, Boston, MA 02115, USA
- Fetal Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, Boston, MA 02115, USA
- Department of Radiology, Boston Children's Hospital, Boston, MA 02215, USA
| | - Yangming Ou
- Harvard Medical School, Boston, MA 02115, USA
- Fetal Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, Boston, MA 02115, USA
- Department of Radiology, Boston Children's Hospital, Boston, MA 02215, USA
- Computational Health Informatics Program, Boston Children's Hospital, Boston, MA 02115, USA
| | - P Ellen Grant
- Harvard Medical School, Boston, MA 02115, USA
- Fetal Neonatal Neuroimaging and Developmental Science Center, Boston Children’s Hospital, Boston, MA 02115, USA
- Department of Radiology, Boston Children's Hospital, Boston, MA 02215, USA
- Department of Medicine, Boston Children's Hospital, Boston, MA 02115, USA
| | - Nadine Gaab
- Laboratories of Cognitive Neuroscience, Division of Developmental Medicine, Department of Medicine, Boston Children’s Hospital, Boston, MA 02115, USA
- Harvard Medical School, Boston, MA 02115, USA
- Harvard Graduate School of Education Boston, Boston, MA 02115, USA
| |
Collapse
|
48
|
Deng Y, Wang Y, Qiu C, Hu Z, Sun W, Gong Y, Zhao X, He W, Cao L. A Chinese Conceptual Semantic Feature Dataset (CCFD). Behav Res Methods 2021; 53:1697-1709. [PMID: 33532892 DOI: 10.3758/s13428-020-01525-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/12/2020] [Indexed: 11/08/2022]
Abstract
Memory and language are important high-level cognitive functions of humans, and the study of conceptual representation of the human brain is a key approach to reveal the principles of cognition. However, this research is often constrained by the availability of stimulus materials. The research on concept representation often needs to be based on a standardized and large-scale database of conceptual semantic features. Although Western scholars have established a variety of English conceptual semantic feature datasets, there is still a lack of a comprehensive Chinese version. In the present study, a Chinese Conceptual semantic Feature Dataset (CCFD) was established with 1,410 concepts including their semantic features and the similarity between concepts. The concepts were grouped into 28 subordinate categories and seven superior categories artificially. The results showed that concepts within the same category were closer to each other, while concepts between categories were farther apart. The CCFD proposed in this study can provide stimulation materials and data support for related research fields. All the data and supplementary materials can be found at https://osf.io/ug5dt/ .
Collapse
Affiliation(s)
- Yaling Deng
- State Key Laboratory of Media Convergence and Communication, Communication University of China, No.1 of Dingfuzhuang East Street, Chaoyang District, Beijing, China.
- Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, 100024, China.
| | - Ye Wang
- State Key Laboratory of Media Convergence and Communication, Communication University of China, No.1 of Dingfuzhuang East Street, Chaoyang District, Beijing, China
- Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, 100024, China
| | - Chenyang Qiu
- Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, 100024, China
| | - Zhenchao Hu
- TV School, Communication University of China, Beijing, 100024, China
| | - Wenyang Sun
- Animation and Digital Arts school, Communication University of China, Beijing, 100024, China
| | - Yanzhu Gong
- Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, 100024, China
| | - Xue Zhao
- College of Humanities, Communication University of China, Beijing, 100024, China
| | - Wei He
- College of Humanities, Communication University of China, Beijing, 100024, China
| | - Lihong Cao
- State Key Laboratory of Media Convergence and Communication, Communication University of China, No.1 of Dingfuzhuang East Street, Chaoyang District, Beijing, China.
- Neuroscience and Intelligent Media Institute, Communication University of China, Beijing, 100024, China.
- State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi, 214125, China.
| |
Collapse
|
49
|
Liu Y, Jiang W, Bi Y, Wei K. Sensorimotor knowledge from task-irrelevant feedback contributes to motor learning. J Neurophysiol 2021; 126:723-735. [PMID: 34259029 DOI: 10.1152/jn.00174.2021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Exposure to task-irrelevant feedback leads to perceptual learning, but its effect on motor learning has been understudied. Here, we asked human participants to reach a visual target with a hand-controlled cursor while observing another cursor moving independently in a different direction. Although the task-irrelevant feedback did not change the main task's performance, it elicited robust savings in subsequent adaptation to classical visuomotor rotation perturbation. We demonstrated that the saving effect resulted from a faster formation of strategic learning through a series of experiments, not from gains in the implicit learning process. Furthermore, the saving effect was robust against drastic changes in stimulus features (i.e., rotation size or direction) or task types (i.e., for motor adaptation and skill learning). However, the effect was absent when the task-irrelevant feedback did not carry the visuomotor relationship embedded in visuomotor rotation. Thus, though previous research on perceptual learning has related task-irrelevant feedback to changes in early sensory processes, our findings support its role in acquiring abstract sensorimotor knowledge during motor learning. Motor learning studies have traditionally focused on task-relevant feedback, but our study extends the scope of feedback processes and sheds new light on the dichotomy of explicit and implicit learning in motor adaptation and motor structure learning.NEW & NOTEWORTHY When the motor system faces perturbations, such as fatigue or new environmental changes, it adapts to these changes by voluntarily selecting new action plans or implicitly fine-tuning the control. We show that the action selection part can be enhanced without practice or explicit instruction. We further demonstrate that this enhancement is probably linked to the acquisition of abstract knowledge about the to-be-adapted novel visual feedback. Our findings draw an interesting parallel between motor and perceptual learning by showing that top-down information affects both types of procedural learning.
Collapse
Affiliation(s)
- Yajie Liu
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China
| | - Wanying Jiang
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,Peking-Tsinghua Center for Life Sciences, Beijing, China
| | - Yuqing Bi
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China
| | - Kunlin Wei
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,Beijing Key Laboratory of Behavior and Mental Health, Beijing, China.,Peking-Tsinghua Center for Life Sciences, Beijing, China
| |
Collapse
|
50
|
Jonauskaite D, Camenzind L, Parraga CA, Diouf CN, Mercapide Ducommun M, Müller L, Norberg M, Mohr C. Colour-emotion associations in individuals with red-green colour blindness. PeerJ 2021; 9:e11180. [PMID: 33868822 PMCID: PMC8035895 DOI: 10.7717/peerj.11180] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2020] [Accepted: 03/08/2021] [Indexed: 12/25/2022] Open
Abstract
Colours and emotions are associated in languages and traditions. Some of us may convey sadness by saying feeling blue or by wearing black clothes at funerals. The first example is a conceptual experience of colour and the second example is an immediate perceptual experience of colour. To investigate whether one or the other type of experience more strongly drives colour-emotion associations, we tested 64 congenitally red-green colour-blind men and 66 non-colour-blind men. All participants associated 12 colours, presented as terms or patches, with 20 emotion concepts, and rated intensities of the associated emotions. We found that colour-blind and non-colour-blind men associated similar emotions with colours, irrespective of whether colours were conveyed via terms (r = .82) or patches (r = .80). The colour-emotion associations and the emotion intensities were not modulated by participants’ severity of colour blindness. Hinting at some additional, although minor, role of actual colour perception, the consistencies in associations for colour terms and patches were higher in non-colour-blind than colour-blind men. Together, these results suggest that colour-emotion associations in adults do not require immediate perceptual colour experiences, as conceptual experiences are sufficient.
Collapse
Affiliation(s)
| | - Lucia Camenzind
- Institute of Psychology, University of Lausanne, Lausanne, Vaud, Switzerland
| | - C Alejandro Parraga
- Comp. Vision Centre/Comp. Sci. Department, Universitat Autònoma de Barcelona, Barcelona, Spain
| | - Cécile N Diouf
- Institute of Psychology, University of Lausanne, Lausanne, Vaud, Switzerland
| | | | - Lauriane Müller
- Institute of Psychology, University of Lausanne, Lausanne, Vaud, Switzerland
| | - Mélanie Norberg
- Institute of Psychology, University of Lausanne, Lausanne, Vaud, Switzerland
| | - Christine Mohr
- Institute of Psychology, University of Lausanne, Lausanne, Vaud, Switzerland
| |
Collapse
|