The time course of cross-modal representations of conceptual categories

Graspability in disguise: The cognitive and neural differences in processing words representing small and big objects

Size is a fundamental visual-spatial characteristic of the physical world. Previous studies have revealed distinct brain responses to small and big objects represented by pictures, implying that object size is a key dimension in organizing concrete concepts. However, it remains unknown whether the brain responses reflecting size-based categorization extend to symbolic input like words. Furthermore, several behavioral studies have indicated faster lexical decisions for words representing big objects (big words) than those representing small objects (small words). However, how this behavioral finding relates to potential neural differences in processing small and big words, as well as the underlying cognitive processes, remains unclear. Therefore, the present study investigates the cognitive and neural differences in processing small and big words. We compared the behavioral and neural responses (EEG) to small and big words using a lexical decision task (LDT) and a semantic decision task (SDT). Our results showed that in the LDT, reaction times to big words were significantly shorter than those to small words in the by-participant but not by-item analysis, suggesting a potential rather than robust processing advantage for big words. By contrast, no behavioral differences were observed in the SDT. Our EEG decoding results revealed distinct brain responses to small and big words at 190-250 msec in both tasks, with additional distinct neural responses at 390-520 msec only in the SDT. Importantly, the regression representational similarity analysis (RSA) suggested that these distinct brain responses could be explained by object graspability represented by small and big words, rather than object size. These findings illustrate the cognitive and neural differences in processing small and big words, identify graspability as the key influencing dimension, and demonstrate flexible, two-stage processing of semantic concepts. Moreover, we propose a novel hypothesis to explain the potential processing advantage for big words over small words.

Tracking neural correlates of contextualized meanings with representational similarity analysis

Although it is uncontroversial that word meanings shift depending on their context, our understanding of contextualized lexical meaning remains poor. How is a contextualized semantic space organized? In this MEG study (27 human participants, 16 women, 10 men, 1 non-binary), we manipulated the semantic and syntactic contexts of word forms to query the organization of this space. All wordforms were noun/verb ambiguous and varied in the semantic distance between their noun and verb uses: unambiguous stems, polysemes with distinct but related meanings, and homonyms with completely unrelated meanings. The senses of each stem were disambiguated by a unique discourse sentence and the items were placed in syntactic contexts of varying sizes. Univariate results characterized syntactic context as a bilateral and distributed effect. A multivariate Representational Similarity Analysis correlated one-hot models of the categorical factors as well as contextualized embedding-based models with MEG activity. Of all models representing ambiguity, only a model differentiating between syntactic categories across contexts correlated with the brain. An All-Embeddings model, where each contextualized word had a distinct representation, explained distributed neural activity across the left hemisphere. Finally, a Syntactic Context model and Within-Context-Stem model were significant in left occipito-parietal regions. While the noun vs. verb contrast affected neural signals robustly, we saw no evidence of the homonym-polyseme-unambiguous contrast, over and above the evidence for fully itemized representations. These findings suggest that in contexts devoid of ambiguity, the neural representation of a word is mainly shaped by its syntactic category and its contextually informed, unique semantic representation.Significance statement A word's context can define its meaning. Context is an integral part of understanding language, yet the organization of the semantic space formed by words in context remains unclear. We used magnetoencephalography (MEG) to investigate the dynamic interaction between contextualized semantic representations, syntactic categories, ambiguity and local syntactic contexts. We find a left-lateralized network encoding a semantic space where each contextualized instance of a word has a distinct neural representation, while syntactic category had a broad bilateral representation. Our study provides a link between naturalistic multivariate studies of item/word-level semantic processing and more traditional controlled factorial investigations of lexical meaning. These findings enrich our understanding of the neural underpinnings of words in context and highlights the role of syntactic context.

Optimizing magnetometers arrays and analysis pipelines for multivariate pattern analysis

BACKGROUND

Multivariate pattern analysis (MVPA) has proven an excellent tool in cognitive neuroscience. It also holds a strong promise when applied to optically-pumped magnetometer-based magnetoencephalography.

NEW METHOD

To optimize OPM-MEG systems for MVPA experiments this study examines data from a conventional MEG magnetometer array, focusing on appropriate noise reduction techniques for magnetometers. We determined the least required number of sensors needed for robust MVPA for image categorization experiments.

RESULTS

We found that the use of signal space separation (SSS) without a proper regularization significantly lowered the classification accuracy considering a sub-array of 102 magnetometers or a sub-array of 204 gradiometers. We also found that classification accuracy did not improve when going beyond 30 sensors irrespective of whether SSS has been applied.

COMPARISON WITH EXISTING METHODS

The power spectra of data filtered with SSS has a substantially higher noise floor that data cleaned with SSP or HFC. Consequently, MVPA decoding results obtained from the SSS-filtered data are significantly lower compared to all other methods employed.

CONCLUSIONS

When designing MEG system based on SQUID magnetometers optimized for multivariate analysis for image categorization experiments, about 30 magnetometers are sufficient. We advise against applying SSS filters without a proper regularization to data from MEG and OPM systems prior to performing MVPA as this method, albeit reducing low-frequency external noise contributions, also introduces an increase in broadband noise. We recommend employing noise reduction techniques that either decrease or maintain the noise floor of the data like signal-space projection, homogeneous field correction and gradient noise reduction.

MEG Evidence That Modality-Independent Conceptual Representations Contain Semantic and Visual Features

The semantic knowledge stored in our brains can be accessed from different stimulus modalities. For example, a picture of a cat and the word "cat" both engage similar conceptual representations. While existing research has found evidence for modality-independent representations, their content remains unknown. Modality-independent representations could be semantic, or they might also contain perceptual features. We developed a novel approach combining word/picture cross-condition decoding with neural network classifiers that learned latent modality-independent representations from MEG data (25 human participants, 15 females, 10 males). We then compared these representations to models representing semantic, sensory, and orthographic features. Results show that modality-independent representations correlate both with semantic and visual representations. There was no evidence that these results were due to picture-specific visual features or orthographic features automatically activated by the stimuli presented in the experiment. These findings support the notion that modality-independent concepts contain both perceptual and semantic representations.

Spatiotemporal Properties of Common Semantic Categories for Words and Pictures

The timing of semantic processing during object recognition in the brain is a topic of ongoing discussion. One way of addressing this question is by applying multivariate pattern analysis to human electrophysiological responses to object images of different semantic categories. However, although multivariate pattern analysis can reveal whether neuronal activity patterns are distinct for different stimulus categories, concerns remain on whether low-level visual features also contribute to the classification results. To circumvent this issue, we applied a cross-decoding approach to magnetoencephalography data from stimuli from two different modalities: images and their corresponding written words. We employed items from three categories and presented them in a randomized order. We show that if the classifier is trained on words, pictures are classified between 150 and 430 msec after stimulus onset, and when training on pictures, words are classified between 225 and 430 msec. The topographical map, identified using a searchlight approach for cross-modal activation in both directions, showed left lateralization, confirming the involvement of linguistic representations. These results point to semantic activation of pictorial stimuli occurring at ∼150 msec, whereas for words, the semantic activation occurs at ∼230 msec.

Overt Word Reading and Visual Object Naming in Adults with Dyslexia: Electroencephalography Study in Transparent Orthography

The study aimed to investigate overt reading and naming processes in adult people with dyslexia (PDs) in shallow (transparent) language orthography. The results of adult PDs are compared with adult healthy controls HCs. Comparisons are made in three phases: pre-lexical (150-260 ms), lexical (280-700 ms), and post-lexical stage of processing (750-1000 ms) time window. Twelve PDs and HCs performed overt reading and naming tasks under EEG recording. The word reading and naming task consisted of sparse neighborhoods with closed phonemic onset (words/objects sharing the same onset). For the analysis of the mean ERP amplitude for pre-lexical, lexical, and post-lexical time window, a mixed design ANOVA was performed with the right (F4, FC2, FC6, C4, T8, CP2, CP6, P4) and left (F3, FC5, FC1, T7, C3, CP5, CP1, P7, P3) electrode sites, within-subject factors and group (PD vs. HC) as between-subject factor. Behavioral response latency results revealed significantly prolonged reading latency between HCs and PDs, while no difference was detected in naming response latency. ERP differences were found between PDs and HCs in the right hemisphere's pre-lexical time window (160-200 ms) for word reading aloud. For visual object naming aloud, ERP differences were found between PDs and HCs in the right hemisphere's post-lexical time window (900-1000 ms). The present study demonstrated different distributions of the electric field at the scalp in specific time windows between two groups in the right hemisphere in both word reading and visual object naming aloud, suggesting alternative processing strategies in adult PDs. These results indirectly support the view that adult PDs in shallow language orthography probably rely on the grapho-phonological route during overt word reading and have difficulties with phoneme and word retrieval during overt visual object naming in adulthood.

Early language dissociation in bilingual minds: magnetoencephalography evidence through a machine learning approach

Does neural activity reveal how balanced bilinguals choose languages? Despite using diverse neuroimaging techniques, prior studies haven't provided a definitive solution to this problem. Nonetheless, studies involving direct brain stimulation in bilinguals have identified distinct brain regions associated with language production in different languages. In this magnetoencephalography study with 45 proficient Spanish-Basque bilinguals, we investigated language selection during covert picture naming and word reading tasks. Participants were prompted to name line drawings or read words if the color of the stimulus changed to green, in 10% of trials. The task was performed either in Spanish or Basque. Despite similar sensor-level evoked activity for both languages in both tasks, decoding analyses revealed language-specific classification ~100 ms post-stimulus onset. During picture naming, right occipital-temporal sensors predominantly contributed to language decoding, while left occipital-temporal sensors were crucial for decoding during word reading. Cross-task decoding analysis unveiled robust generalization effects from picture naming to word reading. Our methodology involved a fine-grained examination of neural responses using magnetoencephalography, offering insights into the dynamics of language processing in bilinguals. This study refines our understanding of the neural underpinnings of language selection and bridges the gap between non-invasive and invasive experimental evidence in bilingual language production.