The Effect of Training-Induced Visual Imageability on Electrophysiological Correlates of Novel Word Processing

The electrophysiology of lexical prediction of emoji and text

As emoji often appear naturally alongside text in utterances, they provide a way to study how prediction unfolds in multimodal sentences in direct comparison to unimodal sentences. In this experiment, participants (N = 40) read sentences in which the sentence-final noun appeared in either word form or emoji form, a between-subjects manipulation. The experiment featured both high constraint sentences and low constraint sentences to examine how the lexical processing of emoji interacts with prediction processes in sentence comprehension. Two well-established ERP components linked to lexical processing and prediction - the N400 and the Late Frontal Positivity - are investigated for sentence-final words and emoji to assess whether, to what extent, and in what linguistic contexts emoji are processed like words. Results indicate that the expected effects, namely an N400 effect to an implausible lexical item compared to a plausible one and an LFP effect to an unexpected lexical item compared to an expected one, emerged for both words and emoji. This paper discusses the similarities and differences between the stimulus types and constraint conditions, contextualized within theories of linguistic prediction, ERP components, and a multimodal lexicon.

The influence of conceptual concreteness on the reading acquisition and integration of novel words into semantic memory via thematic relations

Plenty of studies have been conducted to reveal neurocognitive underpinnings of conceptual representation. Compared with that of concrete concepts, the neurocognitive correlates of abstract concepts remain elusive. The current study aimed to investigate the influence of conceptual concreteness on the reading acquisition and integration of novel words into semantic memory. We constructed two-sentence contexts in which two-character pseudowords were embedded as novel words. Participants read the contexts to infer the meaning of novel words which were either concrete or abstract, and then performed a lexical decision task and a cued-recall memory task. In lexical decision task, primed by the learned novel words, their corresponding concepts, thematically related or unrelated words as well as unlearned pseudowords were judged whether they were words or not. In memory task, participants were presented with the novel words and asked to write down their meaning. The contextual reading and memory test can demonstrate the modulation of conceptual concreteness on novel word learning and the lexical decision task can reveal whether concrete and abstract novel words are integrated into semantic memory similarly or not. During contextual reading, abstract novel words presented for the first time elicited a larger N400 than concrete ones. In memory task, the meaning of concrete novel words was recollected better than abstract novel words. These results indicate that abstract novel words are more difficult to acquire during contextual reading, and to retain afterwards. For lexical decision task behavioral and ERPs were graded, with the longest reaction time, the lowest accuracy and the largest N400s for the unrelated words, then the thematically related words and finally the corresponding concepts of the novel words, regardless of conceptual concreteness. The results suggest that both concrete and abstract novel words can be integrated into semantic memory via thematic relations. These findings are discussed in terms of differential representational framework which posits that concrete words connect with each other via semantic similarities, and abstract ones via thematic relations.

When a Sunny Day Gives You Butterflies: An Electrophysiological Investigation of Concreteness and Context Effects in Semantic Word Processing

Theories on controlled semantic cognition assume that word concreteness and linguistic context interact during semantic word processing. Methodological approaches and findings on how this interaction manifests at the electrophysiological and behavioral levels are heterogeneous. We measured ERPs and RTs applying a validated cueing paradigm with 19 healthy participants, who performed similarity judgments on concrete or abstract words (e.g., "butterfly" or "tolerance") after reading contextual and irrelevant sentential cues. Data-driven analyses showed that concreteness increased and context decreased negative-going deflections in broadly distributed bilateral clusters covering the N400 and N700/late positive component time range, whereas both reduced RTs. Crucially, within a frontotemporal cluster in the N400 time range, contextual (vs. irrelevant) information reduced negative-going amplitudes in response to concrete but not abstract words, whereas a contextual cue reduced RTs only in response to abstract but not concrete words. The N400 amplitudes did not explain additional variance in the RT data, which showed a stronger contextual facilitation for abstract than concrete words. Our results support separate but interacting effects of concreteness and context on automatic and controlled stages of contextual semantic processing and suggest that effects on the electrophysiological versus behavioral level obtained with this paradigm are dissociated.

Corroborating behavioral evidence for the interplay of representational richness and semantic control in semantic word processing

This study aimed to replicate and validate concreteness and context effects on semantic word processing. In Experiment 1, we replicated the behavioral findings of Hoffman et al. (Cortex 63,250–266, https://doi.org/10.1016/j.cortex.2014.09.001, 2015) by applying their cueing paradigm with their original stimuli translated into German. We found concreteness and contextual cues to facilitate word processing in a semantic judgment task with 55 healthy adults. The two factors interacted in their effect on reaction times: abstract word processing profited more strongly from a contextual cue, while the concrete words’ processing advantage was reduced but still present. For accuracy, the descriptive pattern of results suggested an interaction, which was, however, not significant. In Experiment 2, we reformulated the contextual cues to avoid repetition of the to-be-processed word. In 83 healthy adults, the same pattern of results emerged, further validating the findings. Our corroborating evidence supports theories integrating representational richness and semantic control mechanisms as complementary mechanisms in semantic word processing.

Time course of brain activity during the processing of motor- and vision-related abstract concepts: flexibility and task dependency

Grounded cognition theories assume that conceptual processing depends on modality-specific brain systems in a context-dependent fashion. Although the relation of abstract concepts to modality-specific systems is less obvious than for concrete concepts, recent behavioral and neuroimaging studies indicated a foundation of abstract concepts in vision and action. However, due to their poor temporal resolution, neuroimaging studies cannot determine whether sensorimotor activity reflects rapid access to conceptual information or later conceptual processes. The present study therefore assessed the time course of abstract concept processing using event-related potentials (ERPs) and compared ERP responses to abstract concepts with a strong relation to vision or action. We tested whether possible ERP effects to abstract word categories would emerge in early or in later time windows and whether these effects would depend on the depth of the conceptual task. In Experiment 1, a shallow lexical decision task, early feature-specific effects starting at 178 ms were revealed, but later effects beyond 300 ms were also observed. In Experiment 2, a deep conceptual decision task, feature-specific effects with an onset of 22 ms were obtained, but effects again extended beyond 300 ms. In congruency with earlier neuroimaging work, the present feature-specific ERP effects suggest a grounding of abstract concepts in modal brain systems. The presence of early and late feature-specific effects indicates that sensorimotor activity observed in neuroimaging experiments may reflect both rapid conceptual and later post-conceptual processing. Results furthermore suggest that a deep conceptual task accelerates access to conceptual sensorimotor features, thereby demonstrating conceptual flexibility.