Embodied Semantics in a Second Language: Critical Review and Clinical Implications

Non-native language comprehenders encode implied shapes of objects in memory

Barsalou (1999) proposes that conceptual knowledge is represented by mental simulations containing perceptual information derived from actual experiences. Although a substantial number of studies have provided evidence consistent with this view in native language comprehension, it remains unclear whether the non-native language comprehension processes also include mental simulations. The current study successfully replicates the shape match effect in sentence-picture verification (Zwaan et al., 2002) for non-native English language comprehenders, indicating native-like visual simulations. In addition, participants displayed better delayed recognition memory when the shape of the depicted objects matched the shape that was implied by the sentence than when it did not, suggesting that visual simulations were generated spontaneously in naturalistic non-native language comprehension. Additional correlational analyses revealed no relationship between English proficiency and the size of the match effect.

Embodied Semantics: Early Simultaneous Motor Grounding in First and Second Languages

BACKGROUND/OBJECTIVES

Although the embodiment of action-related language is well-established in the mother tongue (L1), less is known about the embodiment of a second language (L2) acquired later in life through formal instruction. We used the high temporal resolution of ERPs and topographic ERP analyses to compare embodiment in L1 and L2 and to investigate whether L1 and L2 are embodied with different strengths at different stages of linguistic processing.

METHODS

Subjects were presented with action-related and non-action-related verbs in a silent reading task. Subjects were late French-German and German-French bilinguals, respectively, and we could therefore collapse across languages to avoid common confounding between language (French and German) and order of acquisition (L1, L2).

RESULTS

We could show distinct effects of embodiment and language. Embodiment affected only the sensory and lexical stages of processing with increased strength and power of the N1 component for motor vs. non-motor verbs, and language affected the lexical and semantic stages of processing with stronger P2/N400 components for L2 than for L1. Non-motor verbs elicited a stronger P1 component in L2.

CONCLUSIONS

Our results suggest that processing words in L2 requires more effortful processing. Importantly, L1 and L2 are not embodied differently, and embodiment affects early and similar stages of processing in L1 and L2, possibly integrating other process of action-language interaction.

Automatic mental simulation in native and non-native speakers

Pictures of objects are verified faster when they match the implied orientation, shape, and color in a sentence-picture verification task, suggesting that people mentally simulate these features during language comprehension. Previous studies had an unintended correlation between match status and the required response, which may have influenced participants' responses by eliciting strategic use of this correlation. We removed this correlation by including color-matching filler trials and investigated if the color-match effect was still obtained. In both a native sample (Experiment 1) and a non-native sample (Experiment 2), we found strong evidence for a color-match advantage on median reaction time and error rates. Our results are consistent with the view that color is automatically simulated during language comprehension as predicted by the grounded cognition framework.

The role of embodied cognition in action language comprehension in L1 and L2

In this study we carried out a behavioral experiment comparing action language comprehension in L1 (Italian) and L2 (English). Participants were Italian native speakers who had acquired the second language late (after the age of 10). They performed semantic judgments on L1 and L2 literal, idiomatic and metaphorical action sentences after viewing a video of a hand performing an action that was related or unrelated to the verb used in the sentence. Results showed that responses to literal and metaphorical L1 sentences were faster when the action depicted was related to the verb used rather than when the action depicted was unrelated to the verb used. No differences were found for the idiomatic condition. In L2 we found that all responses to the three conditions were facilitated when the action depicted was related to the verb used. Moreover, we found that the difference between the unrelated and the related modalities was greater in L2 than in L1 for the literal and the idiomatic condition but not for the metaphorical condition. These findings are consistent with the embodied cognition hypothesis of language comprehension.

Horizontal mapping of time-related words in first and second language

The existence of a consistent horizontal spatial-conceptual mapping for words denoting time is a well-established phenomenon. For example, words related to the past or future (e.g., yesterday/tomorrow) facilitate respective leftward/rightward attentional shifts and responses, suggesting the visual-spatial grounding of temporal semantics, at least in the native language (L1). To examine whether similar horizontal bias also accompanies access to time-related words in a second language (L2), we tested 53 Russian-English (Experiment 1) and 48 German-English (Experiment 2) bilinguals, who classified randomly presented L1 and L2 time-related words as past- or future-related using left or right response keys. The predicted spatial congruency effect was registered in all tested languages and, furthermore, was positively associated with higher L2 proficiency in Experiment 2. Our findings (1) support the notion of horizontal spatial-conceptual mapping in diverse L1s, (2) demonstrate the existence of a similar spatial bias when processing temporal words in L2, and (3) show that the strength of time-space association in L2 may depend on individual L2 proficiency.

The Role of the Motor System in L1 and L2 Action Verb Processing for Chinese Learners of English: Evidence from Mu Rhythm Desynchronization

The nature of semantic representation has long been a key question in linguistic research. The Embodied Cognition theory challenges the traditional view of language representation, stating that semantic information stems from the sensory-motor cortex, which is activated automatically during semantic processing. However, most of the evidence comes from monolingual studies; it remains unclear whether second-language (L2) comprehension involves different semantic representations or mirrors the pattern seen in first-language (L1) processing. Therefore, the present study investigated the role of the sensory-motor system in language processing via making Electroencephalography (EEG) recordings during the processing of L1 and L2 action verbs. The results showed that L1 (Chinese) action verbs generated higher mu-event-related desynchronization (ERD) than L1 abstract verbs in the early processing stage (250 ms after verb presentation), and the same phenomenon was also observed for L2 (English). The results also indicated that language modulated the processing of action verbs, with L1 action verbs eliciting stronger ERD than L2 action verbs. These results demonstrate that the sensory-motor cortex plays a crucial role in comprehending both L1 and L2 action verbs.

Embodiment of action-related language in the native and a late foreign language - An fMRI-study

Theories of embodied cognition postulate that language processing activates similar sensory-motor structures as when interacting with the environment. Only little is known about the neural substrate of embodiment in a foreign language (L2) as compared to the mother tongue (L1). In this fMRI study, we investigated embodiment of motor and non-motor action verbs in L1 and L2 including 31 late bilinguals. Half had German as L1 and French as L2, and the other half vice-versa. We collapsed across languages to avoid the confound between language and order of language acquisition. Region of interest analyses showed stronger activation in motor regions during L2 than during L1 processing, independently of the motor-relatedness of the verbs. Moreover, a stronger involvement of motor regions for motor-related as compared to non-motor-related verbs, similarly for L1 and L2, was found. Overall, the similarity between L1 and L2 embodiment seems to depend on individual and contextual factors.

Spatiotemporal Dynamics of Activation in Motor and Language Areas Suggest a Compensatory Role of the Motor Cortex in Second Language Processing

The involvement of the motor cortex in language understanding has been intensively discussed in the framework of embodied cognition. Although some studies have provided evidence for the involvement of the motor cortex in different receptive language tasks, the role that it plays in language perception and understanding is still unclear. In the present study, we explored the degree of involvement of language and motor areas in a visually presented sentence comprehension task, modulated by language proficiency (L1: native language, L2: second language) and linguistic abstractness (literal, metaphorical, and abstract). Magnetoencephalography data were recorded from 26 late Chinese learners of English. A cluster-based permutation F test was performed on the amplitude of the source waveform for each motor and language region of interest (ROI). Results showed a significant effect of language proficiency in both language and motor ROIs, manifested as overall greater involvement of language ROIs (short insular gyri and planum polare of the superior temporal gyrus) in the L1 than the L2 during 300-500 ms, and overall greater involvement of motor ROI (central sulcus) in the L2 than the L1 during 600-800 ms. We interpreted the over-recruitment of the motor area in the L2 as a higher demand for cognitive resources to compensate for the inadequate engagement of the language network. In general, our results indicate a compensatory role of the motor cortex in L2 understanding.

Involvement of the sensorimotor system in less advanced L2 processing: Evidence from a semantic category decision task

There is increasing evidence indicating that the sensorimotor system is involved in advanced L2 processing, which raises the question of what role sensorimotor information plays in the course of less advanced L2 comprehension. In the current study, two experiments were conducted using a lexical decision task (LDT) and semantic category task (SCT). The results showed that, in the LDT, a task more likely to result in participants making judgments based on the physical properties of words (e.g., familiarity, orthography), “up” words (e.g., sun, plane) did not result in faster upward than downward responses, and “down” words (e.g., tunnel, cave) also did not result in faster downward than upward responses. In the SCT, compatibility effects were found; specifically, searching for the up target after “up” words was faster than after “down” words and searching for the bottom target after “down” words was faster than after “upward-pointing” words. Hence, we concluded that L2 sensorimotor association, at least for L2 with low proficiency, not automatic in nature and is dependent upon deeper semantic task demands.

An Exploratory Study for Assessment of Multimodal Semantic Memory in Colombian Children

Semantic memory (SM) is a type of long-term memory associated with the storage of general information about the world. Here we assessed the characteristics of the SM battery, developed by Catricalà et al. (2013), in a sample of Colombian children. This battery was originally conceived to evaluate adults, and features six subtests that assess SM in different modalities, using a common set of 48 stimuli in both living and nonliving categories. The design of the current study is of a cross-sectional and exploratory type. The sample was composed of 111 children, 57 boys (51%) and 54 girls (49%), who were 6 (n = 68) and 7 (n = 43) years old and had no intellectual disability. Robust linear regression models and correlation networks were used. We found an effect of age on general intelligence after correcting for gender, and no differences on the six subtest scores after corrections for gender and age were performed. Furthermore, age was found to be positively associated with the naming of colored photographs (β = .75, p = .039), naming in response to an oral description (β = 1.81, p = .039), picture sorting at four levels (β = 7.22, p = .029), and sentence verification (β = 26.66, p = .01). In addition, there were differences between the results obtained in adults in the original study and in the children of our study. This exploratory study supports the feasibility of the Spanish translation of the Catricalà et al. (2013) battery to assess SM in children with a nonclinical condition. Future studies are needed to evaluate the psychometric properties of this SM battery, and to corroborate and expand our findings in a larger sample of control children, and in children with some degree of intellectual disability or suffering of some neurodegenerative or psychiatric conditions.

How words ripple through bilingual hands: Motor-language coupling during L1 and L2 writing

The speed of our hand movements can be affected by concurrent processing of manual action verbs (MaVs). Whereas this phenomenon is well established for native languages (L1s), it remains underexplored in late foreign languages (L2s), especially during highly automatized tasks. Here we timed keystroke activity while Spanish-English bilinguals copied MaVs, non-manual action verbs, and non-action verbs in their L1 and L2. Motor planning and execution dynamics were indexed by first-letter lag (the time-lapse between word presentation and first keystroke) and whole-word lag (the time-lapse between first and last keystroke), respectively. Despite yielding no effects on motor planning, MaVs facilitated typing execution in L1 but delayed it in L2, irrespective of the subjects' typing skills, age of L2 learning, and L2 competence. Therefore, motor-language coupling effects seem to be present in both languages though they can arise differently in each. These results extend language grounding models, illuminating the role of embodied mechanisms throughout life.

An Interdisciplinary Approach to Understanding the Psychological Impact of Different Grammaticalizations of the Future

Considering how fundamental and ubiquitous temporal information is in discourse (e.g., Zwaan & Radvansky, 1998), it seems rather surprising that the impact of the grammaticalization of the future on the way we perceive the future has only been scarcely studied. We argue that this may be due to its rather abstract nature and how it has been previously operationalized. In this review, we lay the foundation for studying the impact of the grammaticalization of the future on mental representations of the future by taking an interdisciplinary perspective, connecting cognitive sciences, linguistics, psycholinguistics, economics, and health psychology. More specifically, we argue that experimental psycholinguistics, combined with more applied domains, constitute a promising research avenue.

Motor-system dynamics during naturalistic reading of action narratives in first and second language

Do embodied semantic systems play different roles depending on when and how well a given language was learned? Emergent evidence suggests that this is the case for isolated, decontextualized stimuli, but no study has addressed the issue considering naturalistic narratives. Seeking to bridge this gap, we assessed motor-system dynamics in 26 Spanish-English bilinguals as they engaged in free, unconstrained reading of naturalistic action texts (ATs, highlighting the characters’ movements) and neutral texts (NTs, featuring low motility) in their first and second language (L1, L2). To explore functional connectivity spread over each reading session, we recorded ongoing high-density electroencephalographic signals and subjected them to functional connectivity analysis via a spatial clustering approach. Results showed that, in L1, AT (relative to NT) reading involved increased connectivity between left and right central electrodes consistently implicated in action-related processes, as well as distinct source-level modulations in motor regions. In L2, despite null group-level effects, enhanced motor-related connectivity during AT reading correlated positively with L2 proficiency and negatively with age of L2 learning. Taken together, these findings suggest that action simulations during unconstrained narrative reading involve neural couplings between motor-sensitive mechanisms, in proportion to how consolidated a language is. More generally, such evidence addresses recent calls to test the ecological validity of motor-resonance effects while offering new insights on their relation with experiential variables.

Perceptual Representations in L1, L2 and L3 Comprehension: Delayed Sentence-Picture Verification

We used delayed sentence-picture verification tasks to investigate multilingual perceptual representations. In experiment 1, participants listened to sentences with implied shapes. After a 10-min interval, they judged whether pictures had been mentioned in the preceding sentences or not. Results in experiment 1 showed significant match effect in L1, but not in high proficient L2 or low proficient L3. In experiment 2, Participants listened to one language block, then immediately judged one picture block, totally three language-picture blocks. Results in experiment 2 were parallel to results in experiment 1. Our study supports the view of distributed conception: L2 and L3 are associated with less perceptual symbols than L1, indicating great impact of acquisition styles on perceptual representations. Our results show little impact of language proficiency levels on perceptual representations in delayed tasks.