Linguistic Distributional Knowledge and Sensorimotor Grounding both Contribute to Semantic Category Production

Mapping semantic space: Exploring the higher-order structure of word meaning

Multiple representation theories posit that concepts are represented via a combination of properties derived from sensorimotor, affective, and linguistic experiences. Recently, it has been proposed that information derived from social experience, or socialness, represents another key aspect of conceptual representation. How these various dimensions interact to form a coherent conceptual space has yet to be fully explored. To address this, we capitalized on openly available word property norms for 6339 words and conducted a large-scale investigation into the relationships between 18 dimensions. An exploratory factor analysis reduced the dimensions to six higher-order factors: sub-lexical, distributional, visuotactile, body action, affective and social interaction. All these factors explained unique variance in performance on lexical and semantic tasks, demonstrating that they make important contributions to the representation of word meaning. An important and novel finding was that the socialness dimension clustered with the auditory modality and with mouth and head actions. We suggest this reflects experiential learning from verbal interpersonal interactions. Moreover, formally modelling the network structure of semantic space revealed pairwise partial correlations between most dimensions and highlighted the centrality of the interoception dimension. Altogether, these findings provide new insights into the architecture of conceptual space, including the importance of inner and social experience, and highlight promising avenues for future research.

Raising the Roof: Situating Verbs in Symbolic and Embodied Language Processing

Recent investigations on how people derive meaning from language have focused on task-dependent shifts between two cognitive systems. The symbolic (amodal) system represents meaning as the statistical relationships between words. The embodied (modal) system represents meaning through neurocognitive simulation of perceptual or sensorimotor systems associated with a word's referent. A primary finding of literature in this field is that the embodied system is only dominant when a task necessitates it, but in certain paradigms, this has only been demonstrated using nouns and adjectives. The purpose of this paper is to study whether similar effects hold with verbs. Experiment 1 evaluated a novel task in which participants rated a selection of verbs on their implied vertical movement. Ratings correlated well with distributional semantic models, establishing convergent validity, though some variance was unexplained by language statistics alone. Experiment 2 replicated previous noun-based location-cue congruency experimental paradigms with verbs and showed that the ratings obtained in Experiment 1 predicted reaction times more strongly than language statistics. Experiment 3 modified the location-cue paradigm by adding movement to create an animated, temporally decoupled, movement-verb judgment task designed to examine the relative influence of symbolic and embodied processing for verbs. Results were generally consistent with linguistic shortcut hypotheses of symbolic-embodied integrated language processing; location-cue congruence elicited processing facilitation in some conditions, and perceptual information accounted for reaction times and accuracy better than language statistics alone. These studies demonstrate novel ways in which embodied and linguistic information can be examined while using verbs as stimuli.

The effects of sensorimotor and linguistic information on the basic-level advantage

The basic-level advantage is one of the best-known effects in human categorisation. Traditional accounts argue that basic-level categories present a maximally informative or entry level into a taxonomic organisation of concepts in semantic memory. However, these explanations are not fully compatible with most recent views on the structure of the conceptual system such as linguistic-simulation accounts, which emphasise the dual role of sensorimotor (i.e., perception-action experience of the world) and linguistic distributional information (i.e., statistical distribution of words in language) in conceptual processing. In four preregistered word→picture categorisation studies, we examined whether novel measures of sensorimotor and linguistic distance contribute to the basic level-advantage in categorical decision-making. Results showed that overlap in sensorimotor experience between category concept and member concept (e.g., animal→dog) predicted RT and accuracy at least as well as a traditional division into discrete subordinate, basic, and superordinate taxonomic levels. Furthermore, linguistic distributional information contributed to capturing effects of graded category structure where typicality ratings did not. Finally, when image label production frequency was taken into account (i.e., how often people actually produced specific labels for images), linguistic distributional information predicted RT and accuracy above and beyond sensorimotor information. These findings add to our understanding of how sensorimotor-linguistic theories of the conceptual system can explain categorisation behaviour.

The Development of Abstract Word Meanings

Extensive research has shown that children's early words are learned through sensorimotor experience. Thus, early-acquired words tend to have more concrete meanings. Abstract word meanings tend to be learned later but less is known about their acquisition. We collected meaning-specific concreteness ratings and examined their relationship with age-of-acquisition data from large-scale vocabulary testing with children in grade 2 to college age. Earlier-acquired meanings were rated as more concrete while later-acquired meanings as more abstract, particularly for words typically considered to be concrete. The results suggest that sensorimotor experiences are important to early-acquired word meanings, and other experiences (e.g., linguistic) are important to later-acquired meanings, consistent with a multi-representational view of lexical semantics.

Sensorimotor distance: A grounded measure of semantic similarity for 800 million concept pairs

Experimental design and computational modelling across the cognitive sciences often rely on measures of semantic similarity between concepts. Traditional measures of semantic similarity are typically derived from distance in taxonomic databases (e.g. WordNet), databases of participant-produced semantic features, or corpus-derived linguistic distributional similarity (e.g. CBOW), all of which are theoretically problematic in their lack of grounding in sensorimotor experience. We present a new measure of sensorimotor distance between concepts, based on multidimensional comparisons of their experiential strength across 11 perceptual and action-effector dimensions in the Lancaster Sensorimotor Norms. We demonstrate that, in modelling human similarity judgements, sensorimotor distance has comparable explanatory power to other measures of semantic similarity, explains variance in human judgements which is missed by other measures, and does so with the advantages of remaining both grounded and computationally efficient. Moreover, sensorimotor distance is equally effective for both concrete and abstract concepts. We further introduce a web-based tool ( https://lancaster.ac.uk/psychology/smdistance ) for easily calculating and visualising sensorimotor distance between words, featuring coverage of nearly 800 million word pairs. Supplementary materials are available at https://osf.io/d42q6/ .

Category production norms for 117 concrete and abstract categories

We present a database of category production (aka semantic fluency) norms collected in the UK for 117 categories (67 concrete and 50 abstract). Participants verbally named as many category members as possible within 60 seconds, resulting in a large variety of over 2000 generated member concepts. The norms feature common measures of category production (production frequency, mean ordinal rank, first-rank frequency), as well as response times for all first-named category members, and typicality ratings collected from a separate participant sample. We provide two versions of the dataset: a referential version that groups together responses that relate to the same referent (e.g., hippo, hippopotamus) and a full version that retains all original responses to enable future lexical analysis. Correlational analyses with previous norms from the USA and UK demonstrate both consistencies and differences in English-language norms over time and between geographical regions. Further exploration of the norms reveals a number of structural and psycholinguistic differences between abstract and concrete categories. The data and analyses will be of use in the fields of cognitive psychology, neuropsychology, psycholinguistics, and cognitive modelling, and to any researchers interested in semantic category structure. All data, including original participant recordings, are available at https://osf.io/jgcu6/ .

Multi-dimensional sensorimotor grounding of concrete and abstract categories

Semantic categories, and the concepts belonging to them, have commonly been defined by their relative concreteness, that is, their reliance on perception. However, sensorimotor grounding must be regarded as going beyond the basic five senses and incorporate a multi-dimensional variety of perceptual and action experience. We present a series of exploratory analyses examining the sensorimotor grounding of participant-produced member concepts for 117 categories, spanning concrete (e.g. animal and furniture) and highly abstract (e.g. unit of time and science) categories. We found that both concrete and abstract categories are strongly grounded in multi-dimensional sensorimotor experience. Both domains were dominated by vision and, to a lesser extent, head movements, but concrete categories were more grounded in touch and hand-arm action, while abstract categories were more grounded in hearing and interoception. Importantly, this pattern of grounding was not uniform, and subdomains of concrete (e.g. ingestibles, animates, natural categories and artefacts) and abstract (e.g. internal, social and non-social) categories were grounded in different profiles of sensorimotor experience. Overall, these findings suggest that the distinction between abstract and concrete categories is not as clearcut as ontological assumptions might suggest, and that the strength and diversity of sensorimotor grounding in abstract categories must not be underestimated. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Semantic flow and its relation to controlled semantic retrieval deficits in the narrative production of people with aphasia

Aphasia has had a profound influence on our understanding of how language is instantiated within the human brain. Historically, aphasia has yielded an in vivo model for elucidating the effects of impaired lexical-semantic access on language comprehension and production. Aphasiology has focused intensively on single word dissociations. Yet, less is known about the integrity of combinatorial semantic processes required to construct well-formed narratives. Here we addressed the question of how controlled lexical-semantic retrieval deficits (a hallmark of aphasia) might compound over the course of longer narratives. We specifically examined word-by-word flow of taxonomic vs. thematic semantic distance in the storytelling narratives of individuals with chronic post-stroke aphasia (n = 259) relative to age-matched controls (n = 203). We first parsed raw transcribed narratives into content words and computed inter-word semantic distances for every running pair of words in each narrative (N = 232,490 word transitions). The narratives of people with aphasia showed significant reductions in taxonomic and thematic semantic distance relative to controls. Both distance metrics were strongly predictive of offline measures of semantic impairment and aphasia severity. Since individuals with aphasia often exhibit perseverative language output (i.e., repetitions), we performed additional analyses with repetitions excluded. When repetitions were excluded, group differences in semantic distances persisted and thematic distance was still predictive of semantic impairment, although some findings changed. These results demonstrate the cumulative impact of deficits in controlled word retrieval over the course of narrative production. We discuss the nature of semantic flow between words as a novel metric of characterizing discourse and elucidating the nature of lexical-semantic access impairment in aphasia at multiword levels.