1
|
Vong WK, Wang W, Orhan AE, Lake BM. Grounded language acquisition through the eyes and ears of a single child. Science 2024; 383:504-511. [PMID: 38300999 DOI: 10.1126/science.adi1374] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Accepted: 12/31/2023] [Indexed: 02/03/2024]
Abstract
Starting around 6 to 9 months of age, children begin acquiring their first words, linking spoken words to their visual counterparts. How much of this knowledge is learnable from sensory input with relatively generic learning mechanisms, and how much requires stronger inductive biases? Using longitudinal head-mounted camera recordings from one child aged 6 to 25 months, we trained a relatively generic neural network on 61 hours of correlated visual-linguistic data streams, learning feature-based representations and cross-modal associations. Our model acquires many word-referent mappings present in the child's everyday experience, enables zero-shot generalization to new visual referents, and aligns its visual and linguistic conceptual systems. These results show how critical aspects of grounded word meaning are learnable through joint representation and associative learning from one child's input.
Collapse
Affiliation(s)
- Wai Keen Vong
- Center for Data Science, New York University, New York, NY, USA
| | - Wentao Wang
- Center for Data Science, New York University, New York, NY, USA
| | - A Emin Orhan
- Center for Data Science, New York University, New York, NY, USA
| | - Brenden M Lake
- Center for Data Science, New York University, New York, NY, USA
- Department of Psychology, New York University, New York, NY, USA
| |
Collapse
|
2
|
Nour MM, McNamee DC, Liu Y, Dolan RJ. Trajectories through semantic spaces in schizophrenia and the relationship to ripple bursts. Proc Natl Acad Sci U S A 2023; 120:e2305290120. [PMID: 37816054 PMCID: PMC10589662 DOI: 10.1073/pnas.2305290120] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 07/31/2023] [Indexed: 10/12/2023] Open
Abstract
Human cognition is underpinned by structured internal representations that encode relationships between entities in the world (cognitive maps). Clinical features of schizophrenia-from thought disorder to delusions-are proposed to reflect disorganization in such conceptual representations. Schizophrenia is also linked to abnormalities in neural processes that support cognitive map representations, including hippocampal replay and high-frequency ripple oscillations. Here, we report a computational assay of semantically guided conceptual sampling and exploit this to test a hypothesis that people with schizophrenia (PScz) exhibit abnormalities in semantically guided cognition that relate to hippocampal replay and ripples. Fifty-two participants [26 PScz (13 unmedicated) and 26 age-, gender-, and intelligence quotient (IQ)-matched nonclinical controls] completed a category- and letter-verbal fluency task, followed by a magnetoencephalography (MEG) scan involving a separate sequence-learning task. We used a pretrained word embedding model of semantic similarity, coupled to a computational model of word selection, to quantify the degree to which each participant's verbal behavior was guided by semantic similarity. Using MEG, we indexed neural replay and ripple power in a post-task rest session. Across all participants, word selection was strongly influenced by semantic similarity. The strength of this influence showed sensitivity to task demands (category > letter fluency) and predicted performance. In line with our hypothesis, the influence of semantic similarity on behavior was reduced in schizophrenia relative to controls, predicted negative psychotic symptoms, and correlated with an MEG signature of hippocampal ripple power (but not replay). The findings bridge a gap between phenomenological and neurocomputational accounts of schizophrenia.
Collapse
Affiliation(s)
- Matthew M. Nour
- Department of Psychiatry, University of Oxford, OxfordOX3 7JX, United Kingdom
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, LondonWC1B 5EH, United Kingdom
| | - Daniel C. McNamee
- Champalimaud Research, Centre for the Unknown, 1400-038Lisbon, Portugal
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Chinese Institute for Brain Research, Beijing102206, China
| | - Raymond J. Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, LondonWC1B 5EH, United Kingdom
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3AR, United Kingdom
| |
Collapse
|
3
|
Stavropoulos A, Lakshminarasimhan KJ, Angelaki DE. Belief embodiment through eye movements facilitates memory-guided navigation. bioRxiv 2023:2023.08.21.554107. [PMID: 37662309 PMCID: PMC10473632 DOI: 10.1101/2023.08.21.554107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/05/2023]
Abstract
Neural network models optimized for task performance often excel at predicting neural activity but do not explain other properties such as the distributed representation across functionally distinct areas. Distributed representations may arise from animals' strategies for resource utilization, however, fixation-based paradigms deprive animals of a vital resource: eye movements. During a naturalistic task in which humans use a joystick to steer and catch flashing fireflies in a virtual environment lacking position cues, subjects physically track the latent task variable with their gaze. We show this strategy to be true also during an inertial version of the task in the absence of optic flow and demonstrate that these task-relevant eye movements reflect an embodiment of the subjects' dynamically evolving internal beliefs about the goal. A neural network model with tuned recurrent connectivity between oculomotor and evidence-integrating frontoparietal circuits accounted for this behavioral strategy. Critically, this model better explained neural data from monkeys' posterior parietal cortex compared to task-optimized models unconstrained by such an oculomotor-based cognitive strategy. These results highlight the importance of unconstrained movement in working memory computations and establish a functional significance of oculomotor signals for evidence-integration and navigation computations via embodied cognition.
Collapse
Affiliation(s)
| | | | - Dora E. Angelaki
- Center for Neural Science, New York University, New York, NY, USA
- Tandon School of Engineering, New York University, New York, NY, USA
| |
Collapse
|
4
|
Brown KS, Yee E, Joergensen G, Troyer M, Saltzman E, Rueckl J, Magnuson JS, McRae K. Investigating the Extent to which Distributional Semantic Models Capture a Broad Range of Semantic Relations. Cogn Sci 2023; 47:e13291. [PMID: 37183557 DOI: 10.1111/cogs.13291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Revised: 03/20/2023] [Accepted: 04/07/2023] [Indexed: 05/16/2023]
Abstract
Distributional semantic models (DSMs) are a primary method for distilling semantic information from corpora. However, a key question remains: What types of semantic relations among words do DSMs detect? Prior work typically has addressed this question using limited human data that are restricted to semantic similarity and/or general semantic relatedness. We tested eight DSMs that are popular in current cognitive and psycholinguistic research (positive pointwise mutual information; global vectors; and three variations each of Skip-gram and continuous bag of words (CBOW) using word, context, and mean embeddings) on a theoretically motivated, rich set of semantic relations involving words from multiple syntactic classes and spanning the abstract-concrete continuum (19 sets of ratings). We found that, overall, the DSMs are best at capturing overall semantic similarity and also can capture verb-noun thematic role relations and noun-noun event-based relations that play important roles in sentence comprehension. Interestingly, Skip-gram and CBOW performed the best in terms of capturing similarity, whereas GloVe dominated the thematic role and event-based relations. We discuss the theoretical and practical implications of our results, make recommendations for users of these models, and demonstrate significant differences in model performance on event-based relations.
Collapse
Affiliation(s)
- Kevin S Brown
- Department of Pharmaceutical Sciences, Oregon State University
- School of Chemical, Biological, and Environmental Engineering, Oregon State University
| | - Eiling Yee
- Department of Psychological Sciences, University of Connecticut
| | | | | | | | - Jay Rueckl
- Department of Psychological Sciences, University of Connecticut
| | - James S Magnuson
- Department of Psychological Sciences, University of Connecticut
- BCBL, Basque Center on Cognition, Brain, & Language
- Ikerbasque, Basque Foundation for Science
| | - Ken McRae
- Department of Psychology, University of Western Ontario
| |
Collapse
|
5
|
Digutsch J, Kosinski M. Overlap in meaning is a stronger predictor of semantic activation in GPT-3 than in humans. Sci Rep 2023; 13:5035. [PMID: 36977744 PMCID: PMC10050205 DOI: 10.1038/s41598-023-32248-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Accepted: 03/24/2023] [Indexed: 03/30/2023] Open
Abstract
Modern large language models generate texts that are virtually indistinguishable from those written by humans and achieve near-human performance in comprehension and reasoning tests. Yet, their complexity makes it difficult to explain and predict their functioning. We examined a state-of-the-art language model (GPT-3) using lexical decision tasks widely used to study the structure of semantic memory in humans. The results of four analyses showed that GPT-3's patterns of semantic activation are broadly similar to those observed in humans, showing significantly higher semantic activation in related (e.g., "lime-lemon") word pairs than in other-related (e.g., "sour-lemon") or unrelated (e.g., "tourist-lemon") word pairs. However, there are also significant differences between GPT-3 and humans. GPT-3's semantic activation is better predicted by similarity in words' meaning (i.e., semantic similarity) rather than their co-occurrence in the language (i.e., associative similarity). This suggests that GPT-3's semantic network is organized around word meaning rather than their co-occurrence in text.
Collapse
Affiliation(s)
- Jan Digutsch
- Leibniz Research Centre for Working Environment and Human Factors at the Technical University of Dortmund, Dortmund, Germany.
- Institute of Behavioral Science and Technology, University of St. Gallen, St. Gallen, Switzerland.
| | | |
Collapse
|
6
|
Avcu E, Hwang M, Brown KS, Gow DW. A tale of two lexica: Investigating computational pressures on word representation with neural networks. Front Artif Intell 2023; 6:1062230. [PMID: 37051161 PMCID: PMC10083378 DOI: 10.3389/frai.2023.1062230] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Accepted: 03/10/2023] [Indexed: 03/28/2023] Open
Abstract
Introduction The notion of a single localized store of word representations has become increasingly less plausible as evidence has accumulated for the widely distributed neural representation of wordform grounded in motor, perceptual, and conceptual processes. Here, we attempt to combine machine learning methods and neurobiological frameworks to propose a computational model of brain systems potentially responsible for wordform representation. We tested the hypothesis that the functional specialization of word representation in the brain is driven partly by computational optimization. This hypothesis directly addresses the unique problem of mapping sound and articulation vs. mapping sound and meaning. Results We found that artificial neural networks trained on the mapping between sound and articulation performed poorly in recognizing the mapping between sound and meaning and vice versa. Moreover, a network trained on both tasks simultaneously could not discover the features required for efficient mapping between sound and higher-level cognitive states compared to the other two models. Furthermore, these networks developed internal representations reflecting specialized task-optimized functions without explicit training. Discussion Together, these findings demonstrate that different task-directed representations lead to more focused responses and better performance of a machine or algorithm and, hypothetically, the brain. Thus, we imply that the functional specialization of word representation mirrors a computational optimization strategy given the nature of the tasks that the human brain faces.
Collapse
Affiliation(s)
- Enes Avcu
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
| | | | - Kevin Scott Brown
- Department of Pharmaceutical Sciences and School of Chemical, Biological, and Environmental Engineering, Oregon State University, Corvallis, OR, United States
| | - David W. Gow
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
- Athinoula A. Martinos Center for Biomedical Imaging Massachusetts General Hospital, Charlestown, MA, United States
- Department of Psychology, Salem State University, Salem, MA, United States
- Harvard-MIT Division of Health Sciences and Technology, Boston, MA, United States
| |
Collapse
|
7
|
Caucheteux C, Gramfort A, King JR. Evidence of a predictive coding hierarchy in the human brain listening to speech. Nat Hum Behav 2023; 7:430-441. [PMID: 36864133 PMCID: PMC10038805 DOI: 10.1038/s41562-022-01516-2] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 12/15/2022] [Indexed: 03/04/2023]
Abstract
Considerable progress has recently been made in natural language processing: deep learning algorithms are increasingly able to generate, summarize, translate and classify texts. Yet, these language models still fail to match the language abilities of humans. Predictive coding theory offers a tentative explanation to this discrepancy: while language models are optimized to predict nearby words, the human brain would continuously predict a hierarchy of representations that spans multiple timescales. To test this hypothesis, we analysed the functional magnetic resonance imaging brain signals of 304 participants listening to short stories. First, we confirmed that the activations of modern language models linearly map onto the brain responses to speech. Second, we showed that enhancing these algorithms with predictions that span multiple timescales improves this brain mapping. Finally, we showed that these predictions are organized hierarchically: frontoparietal cortices predict higher-level, longer-range and more contextual representations than temporal cortices. Overall, these results strengthen the role of hierarchical predictive coding in language processing and illustrate how the synergy between neuroscience and artificial intelligence can unravel the computational bases of human cognition.
Collapse
Affiliation(s)
- Charlotte Caucheteux
- Meta AI, Paris, France.
- Université Paris-Saclay, Inria, Commissariat à l'Énergie Atomique et aux Énergies Alternatives, Paris, France.
| | - Alexandre Gramfort
- Meta AI, Paris, France
- Université Paris-Saclay, Inria, Commissariat à l'Énergie Atomique et aux Énergies Alternatives, Paris, France
| | - Jean-Rémi King
- Meta AI, Paris, France.
- Laboratoire des systèmes perceptifs, Département d'études cognitives, École normale supérieure, PSL University, CNRS, Paris, France.
| |
Collapse
|
8
|
Kurth-Nelson Z, Behrens T, Wayne G, Miller K, Luettgau L, Dolan R, Liu Y, Schwartenbeck P. Replay and compositional computation. Neuron 2023; 111:454-469. [PMID: 36640765 DOI: 10.1016/j.neuron.2022.12.028] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/11/2022] [Accepted: 12/18/2022] [Indexed: 01/15/2023]
Abstract
Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.
Collapse
Affiliation(s)
- Zeb Kurth-Nelson
- DeepMind, London, UK; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK.
| | - Timothy Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - Kevin Miller
- DeepMind, London, UK; Institute of Ophthalmology, University College London, London, UK
| | - Lennart Luettgau
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Ray Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK; Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Philipp Schwartenbeck
- Max Planck Institute for Biological Cybernetics, Tubingen, Germany; University of Tubingen, Tubingen, Germany
| |
Collapse
|
9
|
Rastelli C, Greco A, De Pisapia N, Finocchiaro C. Balancing novelty and appropriateness leads to creative associations in children. PNAS Nexus 2022; 1:pgac273. [PMID: 36712330 PMCID: PMC9802071 DOI: 10.1093/pnasnexus/pgac273] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Accepted: 12/01/2022] [Indexed: 12/05/2022]
Abstract
Creative problem solving is a fundamental skill of human cognition and is conceived as a search process whereby a novel and appropriate solution is generated. However, it is unclear whether children are able to balance novelty and appropriateness to generate creative solutions and what are the underlying computational mechanisms. Here, we asked children, ranging from 10 to 11 years old, to perform a word association task according to three instructions, which triggered a more appropriate (ordinary), novel (random), or balanced (creative) response. Results revealed that children exhibited greater cognitive flexibility in the creative condition compared to the control conditions, as revealed by the structure and resiliency of the semantic networks. Moreover, responses' word embeddings extracted from pretrained deep neural networks showed that semantic distance and category switching index increased in the creative condition with respect to the ordinary condition and decreased compared to the random condition. Critically, we showed how children efficiently solved the exploration/exploitation trade-off to generate creative associations by fitting a computational reinforcement learning (RL) model that simulates semantic search strategies. Our findings provide compelling evidence that children balance novelty and appropriateness to generate creative associations by optimally regulating the level of exploration in the semantic search. This corroborates previous findings on the adult population and highlights the crucial contribution of both components to the overall creative process. In conclusion, these results shed light on the connections between theoretical concepts such as bottom-up/top-down modes of thinking in creativity research and the exploration/exploitation trade-off in human RL research.
Collapse
Affiliation(s)
| | - Antonino Greco
- MEG Center, University of Tübingen, 72076 Tübingen, Germany,Department of Neural Dynamics and Magnetoencephalography, Hertie Institute for Clinical Brain Research, University of Tübingen, 72076 Tübingen, Germany,Werner Reichardt Center for Integrative Neuroscience, University of Tübingen, 72076 Tübingen, Germany
| | - Nicola De Pisapia
- Department of Psychology and Cognitive Science, University of Trento, 38068 Rovereto, Italy
| | - Chiara Finocchiaro
- Department of Psychology and Cognitive Science, University of Trento, 38068 Rovereto, Italy
| |
Collapse
|
10
|
Abstract
How are abstract concepts grounded in perceptual experiences for shaping human conceptual knowledge? Recent studies on abstract concepts emphasizing the role of language have argued that abstract concepts are grounded indirectly in perceptual experiences and language (or words) functions as a bridge between abstract concepts and perceptual experiences. However, this “indirect grounding” view remains largely speculative and has hardly been supported directly by empirical evidence. In this paper, therefore, we test the indirect grounding view by means of multimodal distributional semantics, in which the meaning of a word (i.e., a concept) is represented as the combination of textual and visual vectors. The newly devised multimodal distributional semantic model incorporates the indirect grounding view by computing the visual vector of an abstract word through the visual vectors of concrete words semantically related to that abstract word. An evaluation experiment is conducted in which conceptual representation is predicted from multimodal vectors using a multilayer feed-forward neural network. The analysis of prediction performance demonstrates that the indirect grounding model achieves significantly better performance in predicting human conceptual representation of abstract words than other models that mimic competing views on abstract concepts, especially than the direct grounding model in which the visual vectors of abstract words are computed directly from the images of abstract concepts. This result lends some plausibility to the indirect grounding view as a cognitive mechanism of grounding abstract concepts.
Collapse
|
11
|
Abstract
Deep language algorithms, like GPT-2, have demonstrated remarkable abilities to process text, and now constitute the backbone of automatic translation, summarization and dialogue. However, whether these models encode information that relates to human comprehension still remains controversial. Here, we show that the representations of GPT-2 not only map onto the brain responses to spoken stories, but they also predict the extent to which subjects understand the corresponding narratives. To this end, we analyze 101 subjects recorded with functional Magnetic Resonance Imaging while listening to 70 min of short stories. We then fit a linear mapping model to predict brain activity from GPT-2's activations. Finally, we show that this mapping reliably correlates ([Formula: see text]) with subjects' comprehension scores as assessed for each story. This effect peaks in the angular, medial temporal and supra-marginal gyri, and is best accounted for by the long-distance dependencies generated in the deep layers of GPT-2. Overall, this study shows how deep language models help clarify the brain computations underlying language comprehension.
Collapse
Affiliation(s)
- Charlotte Caucheteux
- Meta AI Research, Paris, France.
- Université Paris-Saclay, Inria, CEA, Palaiseau, France.
| | | | - Jean-Rémi King
- Meta AI Research, Paris, France
- École normale supérieure, PSL University, CNRS, Paris, France
| |
Collapse
|
12
|
Forthmann B, Beaty RE, Johnson DR. Semantic Spaces Are Not Created Equal – How Should We Weigh Them in the Sequel? European Journal of Psychological Assessment 2022. [DOI: 10.1027/1015-5759/a000723] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
Abstract. Semantic distance scoring provides an attractive alternative to other scoring approaches for responses in creative thinking tasks. In addition, evidence in support of semantic distance scoring has increased over the last few years. In one recent approach, it has been proposed to combine multiple semantic spaces to better balance the idiosyncratic influences of each space. Thereby, final semantic distance scores for each response are represented by a composite or factor score. However, semantic spaces are not necessarily equally weighted in mean scores, and the usage of factor scores requires high levels of factor determinacy (i.e., the correlation between estimates and true factor scores). Hence, in this work, we examined the weighting underlying mean scores, mean scores of standardized variables, factor loadings, weights that maximize reliability, and equally effective weights on common verbal creative thinking tasks. Both empirical and simulated factor determinacy, as well as Gilmer-Feldt’s composite reliability, were mostly good to excellent (i.e., > .80) across two task types (Alternate Uses and Creative Word Association), eight samples of data, and all weighting approaches. Person-level validity findings were further highly comparable across weighting approaches. Observed nuances and challenges of different weightings and the question of using composites vs. factor scores are thoroughly provided.
Collapse
Affiliation(s)
- Boris Forthmann
- Institute of Psychology in Education, University of Münster, Germany
| | - Roger E. Beaty
- Department of Psychology, Pennsylvania State University, PA, USA
| | - Dan R. Johnson
- Department of Cognitive and Behavioral Science, Washington and Lee University, Lexington, VA, USA
| |
Collapse
|
13
|
Wojcik EH, Zettersten M, Benitez VL. The map trap: Why and how word learning research should move beyond mapping. Wiley Interdiscip Rev Cogn Sci 2022; 13:e1596. [PMID: 35507459 DOI: 10.1002/wcs.1596] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Revised: 03/22/2022] [Accepted: 03/25/2022] [Indexed: 11/08/2022]
Abstract
A pervasive goal in the study of how children learn word meanings is to explain how young children solve the mapping problem. The mapping problem asks how language learners connect a label to its referent. Mapping is one part of word learning, however, it does not reflect other critical components of word meaning construction, such as the encoding of lexico-semantic relations and socio-pragmatic context. In this paper, we argue that word learning researchers' overemphasis of mapping has constrained our experimental paradigms and hypotheses, leading to misconceived theories and policy interventions. We first explain how the mapping focus limits our ability to study the richness and complexity of what infants and children learn about, and do with, word meanings. Then, we describe how our focus on mapping has constrained theory development. Specifically, we show how it has led to (a) the misguided emphasis on referent selection and ostensive labeling, and (b) the undervaluing of diverse pathways to word knowledge, both within and across cultures. We also review the consequences of the mapping focus outside of the lab, including myopic language learning interventions. Last, we outline an alternative, more inclusive approach to experimental study and theory construction in word learning research. This article is categorized under: Psychology > Language Psychology > Theory and Methods Psychology > Learning.
Collapse
Affiliation(s)
- Erica H Wojcik
- Department of Psychology, Skidmore College, Saratoga Springs, New York, USA
| | - Martin Zettersten
- Department of Psychology, Princeton University, Princeton, New Jersey, USA
| | | |
Collapse
|
14
|
Bailey AH, Williams A, Cimpian A. Based on billions of words on the internet, people = men. Sci Adv 2022; 8:eabm2463. [PMID: 35363515 PMCID: PMC10938580 DOI: 10.1126/sciadv.abm2463] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/03/2021] [Accepted: 02/10/2022] [Indexed: 06/14/2023]
Abstract
Recent advances have made it possible to precisely measure the extent to which any two words are used in similar contexts. In turn, this measure of similarity in linguistic context also captures the extent to which the concepts being denoted are similar. When extracted from massive corpora of text written by millions of individuals, this measure of linguistic similarity can provide insight into the collective concepts of a linguistic community, concepts that both reflect and reinforce widespread ways of thinking. Using this approach, we investigated the collective concept person/people, which forms the basis for nearly all societal decision- and policy-making. In three studies and three preregistered replications with similarity metrics extracted from a corpus of over 630 billion English words, we found that the collective concept person/people is not gender-neutral but rather prioritizes men over women-a fundamental bias in our species' collective view of itself.
Collapse
Affiliation(s)
- April H. Bailey
- Department of Psychology, New York University, 6 Washington Place, New York, NY 10003, USA
| | - Adina Williams
- Facebook Artificial Intelligence Research, Meta Platforms Inc., 770 Broadway, Floor 7, New York, NY 10003, USA
| | - Andrei Cimpian
- Department of Psychology, New York University, 6 Washington Place, New York, NY 10003, USA
| |
Collapse
|
15
|
Günther F, Marelli M. Patterns in CAOSS: Distributed representations predict variation in relational interpretations for familiar and novel compound words. Cogn Psychol 2022; 134:101471. [PMID: 35339747 DOI: 10.1016/j.cogpsych.2022.101471] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Revised: 01/25/2022] [Accepted: 02/28/2022] [Indexed: 12/01/2022]
Abstract
While distributional semantic models that represent word meanings as high-dimensional vectors induced from large text corpora have been shown to successfully predict human behavior across a wide range of tasks, they have also received criticism from different directions. These include concerns over their interpretability (how can numbers specifying abstract, latent dimensions represent meaning?) and their ability to capture variation in meaning (how can a single vector representation capture multiple different interpretations for the same expression?). Here, we demonstrate that semantic vectors can indeed rise up to these challenges, by training a mapping system (a simple linear regression) that predicts inter-individual variation in relational interpretations for compounds such as wood brush (for example brush FOR wood, or brush MADE OF wood) from (compositional) semantic vectors representing the meanings of these compounds. These predictions consistently beat different random baselines, both for familiar compounds (moon light, Experiment 1) as well as novel compounds (wood brush, Experiment 2), demonstrating that distributional semantic vectors encode variations in qualitative interpretations that can be decoded using techniques as simple as linear regression.
Collapse
Affiliation(s)
| | - Marco Marelli
- University of Milano-Bicocca, Milan, Italy; NeuroMI, Milan Center for Neuroscience, Milan, Italy
| |
Collapse
|
16
|
Igarashi T, Okuda S, Sasahara K. Development of the Japanese Version of the Linguistic Inquiry and Word Count Dictionary 2015. Front Psychol 2022; 13:841534. [PMID: 35330723 PMCID: PMC8940168 DOI: 10.3389/fpsyg.2022.841534] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Accepted: 02/04/2022] [Indexed: 12/03/2022] Open
Abstract
The Linguistic Inquiry and Word Count Dictionary 2015 (LIWC2015) is a standard text analysis dictionary that quantifies the linguistic and psychometric properties of English words. A Japanese version of the LIWC2015 dictionary (J-LIWC2015) has been expected in the fields of natural language processing and cross-cultural research. This study aims to create the J-LIWC2015 through systematic investigations of the original dictionary and Japanese corpora. The entire LIWC2015 dictionary was initially subjected to human and machine translation into Japanese. After verifying the frequency of use of the words in large corpora, frequent words and phrases that are unique to Japanese were added to the dictionary, followed by recategorization by psychologists. The updated dictionary indicated good internal consistency, semantic equivalence with the original LIWC2015 dictionary, and good construct validity in each category. The evidence suggests that the J-LIWC2015 dictionary is a powerful research tool in computational social science to scrutinize the psychological processes behind Japanese texts and promote standardized cross-cultural investigations in combination with LIWC dictionaries in different languages.
Collapse
Affiliation(s)
- Tasuku Igarashi
- Graduate School of Education and Human Development, Nagoya University, Nagoya, Japan
| | - Shimpei Okuda
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Kazutoshi Sasahara
- School of Environment and Society, Tokyo Institute of Technology, Tokyo, Japan
| |
Collapse
|
17
|
Abstract
Deep learning algorithms trained to predict masked words from large amount of text have recently been shown to generate activations similar to those of the human brain. However, what drives this similarity remains currently unknown. Here, we systematically compare a variety of deep language models to identify the computational principles that lead them to generate brain-like representations of sentences. Specifically, we analyze the brain responses to 400 isolated sentences in a large cohort of 102 subjects, each recorded for two hours with functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). We then test where and when each of these algorithms maps onto the brain responses. Finally, we estimate how the architecture, training, and performance of these models independently account for the generation of brain-like representations. Our analyses reveal two main findings. First, the similarity between the algorithms and the brain primarily depends on their ability to predict words from context. Second, this similarity reveals the rise and maintenance of perceptual, lexical, and compositional representations within each cortical region. Overall, this study shows that modern language algorithms partially converge towards brain-like solutions, and thus delineates a promising path to unravel the foundations of natural language processing. Charlotte Caucheteux and Jean-Rémi King examine the ability of transformer neural networks trained on word prediction tasks to fit representations in the human brain measured with fMRI and MEG. Their results provide further insight into the workings of transformer language models and their relevance to brain responses.
Collapse
Affiliation(s)
- Charlotte Caucheteux
- Facebook AI Research, Paris, France. .,Université Paris-Saclay, Inria, CEA, Palaiseau, France.
| | - Jean-Rémi King
- Facebook AI Research, Paris, France. .,École normale supérieure, PSL University, CNRS, Paris, France.
| |
Collapse
|
18
|
McCrae JP, Fransen T, Ahmadi S, Buitelaar P, Goswami K. Toward an Integrative Approach for Making Sense Distinctions. Front Artif Intell 2022; 5:745626. [PMID: 35198970 PMCID: PMC8859323 DOI: 10.3389/frai.2022.745626] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2021] [Accepted: 01/11/2022] [Indexed: 11/29/2022] Open
Abstract
Word senses are the fundamental unit of description in lexicography, yet it is rarely the case that different dictionaries reach any agreement on the number and definition of senses in a language. With the recent rise in natural language processing and other computational approaches there is an increasing demand for quantitatively validated sense catalogues of words, yet no consensus methodology exists. In this paper, we look at four main approaches to making sense distinctions: formal, cognitive, distributional, and intercultural and examine the strengths and weaknesses of each approach. We then consider how these may be combined into a single sound methodology. We illustrate this by examining two English words, “wing” and “fish,” using existing resources for each of these four approaches and illustrate the weaknesses of each. We then look at the impact of such an integrated method and provide some future perspectives on the research that is necessary to reach a principled method for making sense distinctions.
Collapse
|
19
|
Nacaroğlu O, Bektaş O, Tüysüz M. Examining the Emotional Semantic Orientation of Gifted Students Towards the Flipped Learning Model. Tech Know Learn 2021. [PMCID: PMC8628495 DOI: 10.1007/s10758-021-09581-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The aim of this study was to examine the emotional semantic orientation of gifted students towards the flipped learning model (FLM). An explanatory sequential design, one of the mixed research methods, was utilized in this research. Participants were 53 gifted students, who continued their education in a Science and Art Center in the Eastern Anatolia Region, in the first semester of the 2019–2020 academic year. Participants stated the FLM to be fun, different, instructive, useful, advantageous, and flexible in terms of in-class practices. They also found the FLM to be fun owing to its features such as facilitating learning, being flexible, and providing opportunities for practice. Moreover, the participants regarded the FLM as fun, useful, advantageous, flexible, and effective in terms of out-of-class practices. Another result was that no significant difference was found between the emotional semantic orientations of the female and male gifted students in terms of in-class practices. However, in terms of out-of-class practices, there was a significant difference between the scores obtained from the answers given for the effective-ineffective adjective pair in favor of male participants, while there a significant difference between the scores obtained from the answers given for the fun-boring adjective pair in favor of female participants. Investigation of integrating hybrid learning approaches such as the FLM and evaluating students’ cognitive and affective developments in other disciplines and subjects should be conducted to obtain more data on this approach.
Collapse
|