Compositionality and the angular gyrus: A multi-voxel similarity analysis of the semantic composition of nouns and verbs

Computational Language Modeling and the Promise of In Silico Experimentation

Language neuroscience currently relies on two major experimental paradigms: controlled experiments using carefully hand-designed stimuli, and natural stimulus experiments. These approaches have complementary advantages which allow them to address distinct aspects of the neurobiology of language, but each approach also comes with drawbacks. Here we discuss a third paradigm-in silico experimentation using deep learning-based encoding models-that has been enabled by recent advances in cognitive computational neuroscience. This paradigm promises to combine the interpretability of controlled experiments with the generalizability and broad scope of natural stimulus experiments. We show four examples of simulating language neuroscience experiments in silico and then discuss both the advantages and caveats of this approach.

Neural Correlates of Semantic Interference and Phonological Facilitation in Picture Naming: A Systematic Review and Coordinate-Based Meta-analysis

Semantic interference (SI) and phonological facilitation (PF) effects occur when multiple representations are co-activated simultaneously in complex naming paradigms, manipulating the context in which word production is set. Although the behavioral consequences of these psycholinguistic effects are well-known, the involved brain structures are still controversial. This paper aims to provide a systematic review and a coordinate-based meta-analysis of the available functional neuroimaging studies investigating SI and PF in picture naming paradigms. The included studies were fMRI experiments on healthy subjects, employing paradigms in which co-activations of representations were obtained by manipulating the naming context using semantically or phonologically related items. We examined the principal methodological aspects of the included studies, emphasizing the existing commonalities and discrepancies across single investigations. We then performed an exploratory coordinate-based meta-analysis of the reported activation peaks of neural response related to SI and PF. Our results consolidated previous findings regarding the involvement of the left inferior frontal gyrus and the left middle temporal gyrus in SI and brought out the role of bilateral inferior parietal regions in PF.

Representation of event and object concepts in ventral anterior temporal lobe and angular gyrus

Semantic knowledge includes understanding of objects and their features and also understanding of the characteristics of events. The hub-and-spoke theory holds that these conceptual representations rely on multiple information sources that are integrated in a central hub in the ventral anterior temporal lobes. The dual-hub theory expands this framework with the claim that the ventral anterior temporal lobe hub is specialized for object representation, while a second hub in angular gyrus is specialized for event representation. To test these ideas, we used representational similarity analysis, univariate and psychophysiological interaction analyses of fMRI data collected while participants processed object and event concepts (e.g. "an apple," "a wedding") presented as images and written words. Representational similarity analysis showed that angular gyrus encoded event concept similarity more than object similarity, although the left angular gyrus also encoded object similarity. Bilateral ventral anterior temporal lobes encoded both object and event concept structure, and left ventral anterior temporal lobe exhibited stronger coding for events. Psychophysiological interaction analysis revealed greater connectivity between left ventral anterior temporal lobe and right pMTG, and between right angular gyrus and bilateral ITG and middle occipital gyrus, for event concepts compared to object concepts. These findings support the specialization of angular gyrus for event semantics, though with some involvement in object coding, but do not support ventral anterior temporal lobe specialization for object concepts.

Spatiotemporal characteristics of the neural representation of event concepts

Events are a fundamentally important part of our understanding of the world. How lexical concepts denoting events are represented in the brain remains controversial. We conducted two experiments using event and object nouns matched on a range of psycholinguistic variables, including concreteness, to examine spatial and temporal characteristics of event concepts. Both experiments used magnitude and valence tasks on event and object nouns. The fMRI experiment revealed a distributed set of regions for events, including the angular gyrus, anterior temporal lobe, and posterior cingulate across tasks. In the EEG experiment, events and objects differed in amplitude within the 300-500 ms window. Together these results shed light into the spatiotemporal characteristics of event concept representation and show that event concepts are represented in the putative hubs of the semantic system. While these hubs are typically associated with object semantics, they also represent events, and have a likely role in temporal integration.

Constructing complex social categories under uncertainty

Conceptual combination is the act of building complex concepts from simpler ones. Although research has examined how inferences about compound objects (e.g., fuzzy chair) are produced from their constituent concepts, little is known about the combinatorial processes that produce inferences about compound social categories (e.g., Irish musician). Using a computational approach, we investigated the relationship between ratings of 25 nationality-occupation combinations and ratings of their constituent concepts along the attribute dimensions of warmth and competence. We found that people incorporate uncertainty into their perceptions of compound social categories. Further, people are more likely to use a linear combination strategy when they are more certain about the attributes of the constituents and less familiar with the combination. Conversely, when social combinations are more familiar, their judged attributes deviate further from the predictions of a combinatorial model and are shared across participants, suggesting that stereotype-based knowledge plays a central role in the representation of complex social groups. Twenty-five non-human animal combinations (e.g., circus snake) serve as a comparison and were rated on size and ferocity. We found evidence that familiarity has different effects on the strategies used to combine person concepts and animal concepts, pointing to the possible existence of both common and distinct mechanisms for constructing social and non-social categories.

Altered Default Mode Network Functional Connectivity in Parkinson’s Disease: A Resting-State Functional Magnetic Resonance Imaging Study

Purpose

Whether the intrinsic functional connectivity pattern of the default mode network (DMN) is involved in the progression of cognitive decline in Parkinson’s disease (PD) remains unclear. This study aimed to investigate the intrinsic functional connectivity (FC) pattern of the DMN anchored on the posterior cingulate cortex (PCC) in patients with PD by resting-state functional magnetic resonance imaging (fMRI).

Methods

Fifty patients with PD and 50 healthy controls (HCs) were included for resting-state fMRI scanning. A seed-based FC method was used to reveal FC patterns in the DMN with region of interest (ROI) in the PCC. Relationships between FC patterns and disease severity (UPDRS-III) were detected.

Results

Compared with the HCs, the patients with PD showed increased FC between the PCC and the right precuneus, left cuneus, and right angular gyrus. In the PD group, the increased FC values in the right precuneus were significantly and positively correlated with motor severity as assessed with UPDRS-III scores (rho = 0.337, p = 0.02).

Conclusion

Our result highlights that the patients with PD showed increased FC between the PCC and the right precuneus, left cuneus, and right angular gyrus in the DMN. The altered connectivity pattern in the DMN may play a crucial role in the neurophysiological mechanism of cognitive decline in patients with PD. These findings might provide new insights into neural mechanisms of cognitive decline in PD.

Common Neural Functions during Children's Learning from Naturalistic and Controlled Mathematics Paradigms

Two major goals of human neuroscience are to understand how the brain functions in the real world and to measure neural processes under conditions that are ecologically valid. A critical step toward these goals is understanding how brain activity during naturalistic tasks that mimic the real world relates to brain activity in more traditional laboratory tasks. In this study, we used intersubject correlations to locate reliable stimulus-driven cerebral processes among children and adults in a naturalistic video lesson and a laboratory forced-choice task that shared the same arithmetic concept. We show that relative to a control condition with grammatical content, naturalistic and laboratory arithmetic tasks evoked overlapping activation within brain regions previously associated with math semantics. The regions of specific functional overlap between the naturalistic mathematics lesson and laboratory mathematics task included bilateral intraparietal cortex, which confirms that this region processes mathematical content independently of differences in task mode. These findings suggest that regions of the intraparietal cortex process mathematical content when children are learning about mathematics in a naturalistic setting.

Deep neural networks reveal topic-level representations of sentences in medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus

When reading a sentence, individual words can be combined to create more complex meaning. In this study, we sought to uncover brain regions that reflect the representation of the meaning of sentences at the topic level, as opposed to the meaning of their individual constituent words when considered irrespective of their context. Using fMRI, we recorded the neural activity of participants while reading sentences. We constructed a topic-level sentence representations using the final layer of a convolutional neural network (CNN) trained to classify Wikipedia sentences into broad semantic categories. This model was contrasted with word-level sentence representations constructed using the average of the word embeddings constituting the sentence. Using representational similarity analysis, we found that the medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus more strongly represent sentence topic-level, compared to word-level, meaning, uncovering the important role of these semantic system regions in the representation of topic-level meaning. Results were comparable when sentence meaning was modelled with a multilayer perceptron that was not sensitive to word order within a sentence, suggesting that the learning objective, in the terms of the topic being modelled, is the critical factor in capturing these neural representational spaces.

The Structure of Systematicity in the Brain

A hallmark of human intelligence is the ability to adapt to new situations, by applying learned rules to new content (systematicity) and thereby enabling an open-ended number of inferences and actions (generativity). Here, we propose that the human brain accomplishes these feats through pathways in the parietal cortex that encode the abstract structure of space, events, and tasks, and pathways in the temporal cortex that encode information about specific people, places, and things (content). Recent neural network models show how the separation of structure and content might emerge through a combination of architectural biases and learning, and these networks show dramatic improvements in the ability to capture systematic, generative behavior. We close by considering how the hippocampal formation may form integrative memories that enable rapid learning of new structure and content representations.

Context free and context-dependent conceptual representation in the brain

How concepts are coded in the brain is a core issue in cognitive neuroscience. Studies have focused on how individual concepts are processed, but the way in which conceptual representation changes to suit the context is unclear. We parametrically manipulated the association strength between words, presented in pairs one word at a time using a slow event-related fMRI design. We combined representational similarity analysis and computational linguistics to probe the neurocomputational content of these trials. Individual word meaning was maintained in supramarginal gyrus (associated with verbal short-term memory) when items were judged to be unrelated, but not when a linking context was retrieved. Context-dependent meaning was instead represented in left lateral prefrontal gyrus (associated with controlled retrieval), angular gyrus, and ventral temporal lobe (regions associated with integrative aspects of memory). Analyses of informational connectivity, examining the similarity of activation patterns across trials between sites, showed that control network regions had more similar multivariate responses across trials when association strength was weak, reflecting a common controlled retrieval state when the task required more unusual associations. These findings indicate that semantic control and representational sites amplify contextually relevant meanings in trials judged to be related.

Conceptual Combination in the LATL With and Without Syntactic Composition

The relationship among syntactic, semantic, and conceptual processes in language comprehension is a central question to the neurobiology of language. Several studies have suggested that conceptual combination in particular can be localized to the left anterior temporal lobe (LATL), while syntactic processes are more often associated with the posterior temporal lobe or inferior frontal gyrus. However, LATL activity can also correlate with syntactic computations, particularly in narrative comprehension. Here we investigated the degree to which LATL conceptual combination is dependent on syntax, specifically asking whether rapid (∼200 ms) magnetoencephalography effects of conceptual combination in the LATL can occur in the absence of licit syntactic phrase closure and in the absence of a semantically plausible output for the composition. We find that such effects do occur: LATL effects of conceptual combination were observed even when there was no syntactic phrase closure or plausible meaning. But syntactic closure did have an additive effect such that LATL signals were the highest for expressions that composed both conceptually and syntactically. Our findings conform to an account in which LATL conceptual composition is influenced by local syntactic composition but is also able to operate without it.

Imbalanced Gamma-band Functional Brain Networks of Autism Spectrum Disorders

Resting gamma-band brain networks are known as an inhibitory component in functional brain networks. Although autism spectrum disorder (ASD) is considered as with imbalanced brain networks, the inhibitory component remains not fully explored. The study reported 10 children with ASD and 10 typically-developing (TD) controls. The power spectral density analysis of the gamma-band signal in the cerebral cortex was performed at the source level. The normalized phase transfer entropy values (nPTEs) were calculated to construct brain connectivity. Gamma-band activity of the ASD group was lower than the TD children. The significantly inhibited brain regions were mainly distributed in the bilateral frontal and temporal lobes. Connectivity analysis showed alterations in the connections from key nodes of the social brain network. The behavior assessments in the ASD group revealed a significantly positive correlation between the total score of Childhood Autism Rating Scale and the regional nPTEs of the right transverse temporal gyrus. Our results provide strong evidence that the gamma-band brain networks of ASD children have a lower level of brain activities and different distribution of information flows. Clinical meanings of such imbalances of both activity and connectivity were also worthy of further explorations.

The role of left angular gyrus in the representation of linguistic composition relations

Language comprehension is compositional: individual words are combined structurally to form larger meaning representations. The neural basis for compositionality is at the center of a growing body of recent research. Previous work has largely used univariate analysis to investigate the question, a technique that could potentially lead to the loss of fined‐grained information due to the procedure of averaging over neural responses. In a functional magnetic resonance imaging experiment, the present study examined different types of composition relations in Chinese phrases, using a 1‐back composition relation probe (CRP) task and a 1‐back word probe (WP) task. We first analyzed the data using the multivariate representation similarity analysis, which better captures the fine‐grained representational differences in the stimuli. The results showed that the left angular gyrus (AG) represents different types of composition relations in the CRP task, but no brain areas were identified in the WP task. We also conducted a traditional univariate analysis and found greater activations in the bilateral inferior frontal gyrus in the CRP task relative to the WP task. We discuss the methodological and theoretical implications of our findings in the context of the larger language neural network identified in previous studies. Our findings highlight the role of left AG in representing and distinguishing fine‐grained linguistic composition relations.

Algebra dissociates from arithmetic in the brain semantic network

Background

Mathematical expressions mainly include arithmetic (such as 8 − (1 + 3)) and algebra (such as a − (b + c)). Previous studies have shown that both algebraic processing and arithmetic involved the bilateral parietal brain regions. Although previous studies have revealed that algebra was dissociated from arithmetic, the neural bases of the dissociation between algebraic processing and arithmetic is still unclear. The present study uses functional magnetic resonance imaging (fMRI) to identify the specific brain networks for algebraic and arithmetic processing.

Methods

Using fMRI, this study scanned 30 undergraduates and directly compared the brain activation during algebra and arithmetic. Brain activations, single-trial (item-wise) interindividual correlation and mean-trial interindividual correlation related to algebra processing were compared with those related to arithmetic. The functional connectivity was analyzed by a seed-based region of interest (ROI)-to-ROI analysis.

Results

Brain activation analyses showed that algebra elicited greater activation in the angular gyrus and arithmetic elicited greater activation in the bilateral supplementary motor area, left insula, and left inferior parietal lobule. Interindividual single-trial brain-behavior correlation revealed significant brain-behavior correlations in the semantic network, including the middle temporal gyri, inferior frontal gyri, dorsomedial prefrontal cortices, and left angular gyrus, for algebra. For arithmetic, the significant brain-behavior correlations were located in the phonological network, including the precentral gyrus and supplementary motor area, and in the visuospatial network, including the bilateral superior parietal lobules. For algebra, significant positive functional connectivity was observed between the visuospatial network and semantic network, whereas for arithmetic, significant positive functional connectivity was observed only between the visuospatial network and phonological network.

Conclusion

These findings suggest that algebra relies on the semantic network and conversely, arithmetic relies on the phonological and visuospatial networks.

Dissociating nouns and verbs in temporal and perisylvian networks: Evidence from neurodegenerative diseases

Naming of nouns and verbs can be selectively impaired in neurological disorders, but the specificity of the neural and cognitive correlates of such dissociation remains unclear. Functional imaging and stroke research sought to identify cortical regions selectively recruited for nouns versus verbs, yet findings are inconsistent. The present study investigated this issue in neurodegenerative diseases known to selectively affect different brain networks, thus providing new critical evidence of network specificity. We examined naming performances on nouns and verbs in 146 patients with different neurodegenerative syndromes (Primary Progressive Aphasia - PPA, Alzheimer's disease - AD, and behavioral variant Frontotemporal Dementia - FTD) and 30 healthy adults. We then correlated naming scores with MRI-derived cortical thickness values as well as with performances in semantic and syntactic tasks, across all subjects. Results indicated that patients with the semantic variant PPA named significantly fewer nouns than verbs. Instead, nonfluent/agrammatic PPA patients named fewer verbs than nouns. Across all subjects, performance on nouns (adjusted for verbs) specifically correlated with cortical atrophy in left anterior temporal regions, and performance on verbs (adjusted for nouns) with atrophy in left inferior and middle frontal, inferior parietal and posterior temporal regions. Furthermore, lower lexical-semantic abilities correlated with deficits in naming both nouns and verbs, while lower syntactic abilities only correlated with naming verbs. Our results show that different neural and cognitive mechanisms underlie naming of specific grammatical categories in neurodegenerative diseases. Importantly, our findings showed that verb processing depends on a widespread perisylvian networks, suggesting that some regions might be involved in processing different types of action knowledge. These findings have important implications for early differential diagnosis of neurodegenerative disorders.

From letters to composed concepts: A magnetoencephalography study of reading

Language comprehension requires the recognition of individual words and the combination of their meanings to yield complex concepts or interpretations. This combinatory process often requires the insertion of unstated semantic material between words, based on thematic or feature knowledge. For example, the phrase horse barn is not interpreted as a blend of a horse and a barn, but specifically a barn where horses are kept. Previous neuroscientific evidence suggests that left posterior and anterior temporal cortex underpin thematic and feature‐based concept knowledge, respectively, but much remains unclear about how these areas contribute to combinatory language processing. Using magnetoencephalography, we contrasted source‐localized responses to modifier‐noun phrases involving thematic relations versus feature modifications, while also examining how lower‐level orthographic processing fed composition. Participants completed three procedures examining responses to letter‐strings, adjective‐noun phrases, and noun–noun combinations that varied the semantic relations between words. We found that sections of the left anterior temporal lobe, posterior temporal lobe, and cortex surrounding the angular gyrus were all engaged in the minimal composition of adjective‐noun phrases, a more distributed network than in most prior studies of minimal composition. Of these regions, only the left posterior temporal lobe was additionally sensitive to implicit thematic relations between composing words, suggesting that it houses a specialized relational processing component in a wider composition network. We additionally identified a left occipitotemporal progression from orthographic to lexical processing, feeding ventral anterior areas engaged in the combination of word meanings. Finally, by examining source signal leakage, we characterized the degree to which these responses could be distinguished from one another using source estimation.

Asymmetric Underlying Mechanisms of Relation-Based and Property-Based Noun-Noun Conceptual Combination

Conceptual combination is a fundamental human cognitive ability by which people can experience infinite thinking by artfully combining finite knowledge. For example, one can instantly combine “cactus” and “fish” together as “prickly fish” even if one has never previously heard of a “cactus fish.” Although two major combinatorial types—property and relational combinations—have been identified, the underlying processes of each remain elusive. This study investigates the asymmetric processing mechanisms underlying property and relational combinations by examining differential semantic activation during noun–noun conceptual combination. Across two experiments utilizing each combinatorial process as semantic priming and implementing a lexical decision task immediately after combination, we measure and compare the semantic activation patterns of intrinsic and extrinsic semantic features in these two combinatorial types. We found converging evidence that property and relational combinations involve asymmetric semantic information and entail distinct processing mechanisms. In property combination, the intrinsic feature in the modifier concept showed greater activation than the semantic feature of the same dimension in the head concept. In contrast, in relational combination, the extrinsic semantic feature in the head concept and the whole modifier concept showed similar levels of activation. Moreover, our findings also showed that these patterns of semantic activation occurred only when the combinatorial process was complete, indicating that accessing the same lexical-semantic information is not sufficient to observe asymmetric patterns. These findings demonstrate that property combination involves replacing a specific semantic feature of the head noun with that of the modifier noun, whereas relational combination involves completing the semantic feature of the head noun with the whole modifier concept. We discuss the implications of these findings, research limitations, and future research directions.

How the conceptual specificity of individual words affects incremental sentence composition: MEG evidence

While much research has addressed the neural basis of lexical access and the composition of lexical items into larger meanings, little is known about how the semantic properties of individual words affect composition. Research on modifier-noun combinations has, however, shown that composition related activity in the left anterior temporal lobe (LATL) is sensitive to the conceptual specificity of the composing words. Here we tested whether this pattern extends to verb-argument combinations in minimal subject-verb-object sentences. If the LATL specificity effects extend to verb-argument integration, this would suggest a general mechanism that composes not only entity concepts, but also propositions describing events. Results showed an overall similar modulation by conceptual specificity in the verb domain, suggesting a central, category-insensitive, role for the LATL as a conceptual combiner. Additionally, we saw specificity effects in the left mid-superior temporal cortex, but the angular gyrus, often hypothesized as combinatory, showed no effects of composition.

The Cortical Organization of Syntax

Syntax, the structure of sentences, enables humans to express an infinite range of meanings through finite means. The neurobiology of syntax has been intensely studied but with little consensus. Two main candidate regions have been identified: the posterior inferior frontal gyrus (pIFG) and the posterior middle temporal gyrus (pMTG). Integrating research in linguistics, psycholinguistics, and neuroscience, we propose a neuroanatomical framework for syntax that attributes distinct syntactic computations to these regions in a unified model. The key theoretical advances are adopting a modern lexicalized view of syntax in which the lexicon and syntactic rules are intertwined, and recognizing a computational asymmetry in the role of syntax during comprehension and production. Our model postulates a hierarchical lexical-syntactic function to the pMTG, which interconnects previously identified speech perception and conceptual-semantic systems in the temporal and inferior parietal lobes, crucial for both sentence production and comprehension. These relational hierarchies are transformed via the pIFG into morpho-syntactic sequences, primarily tied to production. We show how this architecture provides a better account of the full range of data and is consistent with recent proposals regarding the organization of phonological processes in the brain.

Compositionality in a Parallel Architecture for Language Processing

Compositionality has been a central concept in linguistics and philosophy for decades, and it is increasingly prominent in many other areas of cognitive science. Its status, however, remains contentious. Here, I reassess the nature and scope of the principle of compositionality (Partee, 1995) from the perspective of psycholinguistics and cognitive neuroscience. First, I review classic arguments for compositionality and conclude that they fail to establish compositionality as a property of human language. Next, I state a new competence argument, acknowledging the fact that any competent user of a language L can assign to most expressions in L at least one meaning which is a function only of the meanings of the expression's parts and of its syntactic structure. I then discuss selected results from cognitive neuroscience, indicating that the human brain possesses the processing capacities presupposed by the competence argument. Finally, I outline a language processing architecture consistent with the neuroscience results, where semantic representations may be generated by a syntax-driven stream and by an "asyntactic" processing stream, jointly or independently. Compositionality is viewed as a constraint on computation in the former stream only.

Differential contributions of left-hemispheric language regions to basic semantic composition

Semantic composition, the ability to combine single words to form complex meanings, is a core feature of human language. Despite growing interest in the basis of semantic composition, the neural correlates and the interaction of regions within this network remain a matter of debate. We designed a well-controlled two-word fMRI paradigm in which phrases only differed along the semantic dimension while keeping syntactic information alike. Healthy participants listened to meaningful ("fresh apple"), anomalous ("awake apple") and pseudoword phrases ("awake gufel") while performing an implicit and an explicit semantic task. We identified neural signatures for distinct processes during basic semantic composition. When lexical information is kept constant across conditions and the evaluation of phrasal plausibility is examined (meaningful vs. anomalous phrases), a small set of mostly left-hemispheric semantic regions, including the anterior part of the left angular gyrus, is found active. Conversely, when the load of lexical information-independently of phrasal plausibility-is varied (meaningful or anomalous vs. pseudoword phrases), conceptual combination involves a wide-spread left-hemispheric network comprising executive semantic control regions and general conceptual representation regions. Within this network, the functional coupling between the left anterior inferior frontal gyrus, the bilateral pre-supplementary motor area and the posterior angular gyrus specifically increases for meaningful phrases relative to pseudoword phrases. Stronger effects in the explicit task further suggest task-dependent neural recruitment. Overall, we provide a separation between distinct nodes of the semantic network, whose functional contributions depend on the type of compositional process under analysis.

Common and distinct neural substrates of sentence production and comprehension

Functional neuroimaging and lesion-symptom mapping investigations implicate a left frontal-temporal-parietal network for sentence processing. The majority of studies have focused on sentence comprehension, with fewer in the domain of sentence production, which have not fully elucidated overlapping and/or unique brain structures associated with the two domains, particularly for sentences with noncanonical word order. Using voxel-based lesion symptom mapping (VLSM) we examined the relationship between lesions within the left hemisphere language network and both sentence comprehension and production of simple and complex syntactic structures in 76 participants with chronic stroke-induced aphasia. Results revealed shared regions across domains in the anterior and posterior superior temporal gyri (aSTG, pSTG), and the temporal pole (adjusted for verb production/comprehension). Additionally, comprehension was associated with lesions in the anterior and posterior middle temporal gyri (aMTG, pMTG), the MTG temporooccipital regions, SMG/AG, central and parietal operculum, and the insula. Subsequent VLSM analyses (production versus comprehension) revealed critical regions associated with each domain: anterior temporal lesions were associated with production; posterior temporo-parietal lesions were associated with comprehension, implicating important roles for regions within the ventral and dorsal stream processing routes, respectively. Processing of syntactically complex, noncanonical (adjusted for canonical), sentences was associated with damage to the pSTG across domains, with additional damage to the pMTG and IPL associated with impaired sentence comprehension, suggesting that the pSTG is crucial for computing noncanonical sentences across domains and that the pMTG, and IPL are necessary for re-analysis of thematic roles as required for resolution of long-distance dependencies. These findings converge with previous studies and extend our knowledge of the neural mechanisms of sentence comprehension to production, highlighting critical regions associated with both domains, and further address the mechanism engaged for syntactic computation, controlled for the contribution of verb processing.

Feature Uncertainty Predicts Behavioral and Neural Responses to Combined Concepts

The cognitive and neural structure of conceptual knowledge affects how concepts combine in language and thought. Examining the principles by which individual concepts (e.g., diamond, baseball) combine into more complex phrases (e.g., "baseball diamond") can illuminate not only how the brain combines concepts but also the key ingredients of conceptual structure. Here we specifically tested the role of feature uncertainty in the modulation of conceptual brightness evoked by adjective-noun combinations (e.g., "dark diamond") in male and female human subjects. We collected explicit ratings of conceptual brightness for 45 noun concepts and their "dark" and "light" combinations, resulting in a measure reflecting the degree of conceptual brightness modulation in each noun concept. Feature uncertainty was captured in an entropy measure, as well as in a predictive Bayesian model of feature modulation. We found that feature uncertainty (i.e., entropy) and the Bayesian model were both strong predictors of these behavioral effects. Using fMRI, we observed the neural responses evoked by the concepts and combinations in a priori ROIs. Feature uncertainty predicted univariate responses in left inferior frontal gyrus, and multivariate responses in left anterior temporal lobe were predicted by degree of conceptual brightness modulation. These findings suggest that feature uncertainty is a key ingredient of conceptual structure, and inform cognitive neuroscience theories of conceptual combination by highlighting the role of left inferior frontal gyrus and left anterior temporal lobe in the process of flexible feature modulation during comprehension of complex language.SIGNIFICANCE STATEMENT The meaning of a word depends on the words surrounding it. The challenge of understanding how flexible meaning emerges in language can be simplified by studying adjective-noun phrases. We tested whether the uncertainty of a feature (i.e., brightness) in a given noun concept (e.g., diamond) influences how the adjective and noun concepts combine. We analyzed feature uncertainty using two probabilistic measures, and found that feature uncertainty predicted people's explicit interpretations of adjective-noun phrases (e.g., "dark diamond"). Using fMRI, we found that combined concepts evoked responses in left inferior frontal gyrus and left anterior temporal lobe that related to our measures of feature modulation and uncertainty. These findings reveal the cognitive and neural processes supporting conceptual combination and complex language use.

Two Ways to Build a Thought: Distinct Forms of Compositional Semantic Representation across Brain Regions

To understand a simple sentence such as "the woman chased the dog", the human mind must dynamically organize the relevant concepts to represent who did what to whom. This structured recombination of concepts (woman, dog, chased) enables the representation of novel events, and is thus a central feature of intelligence. Here, we use functional magnetic resonance (fMRI) and encoding models to delineate the contributions of three brain regions to the representation of relational combinations. We identify a region of anterior-medial prefrontal cortex (amPFC) that shares representations of noun-verb conjunctions across sentences: for example, a combination of "woman" and "chased" to encode woman-as-chaser, distinct from woman-as-chasee. This PFC region differs from the left-mid superior temporal cortex (lmSTC) and hippocampus, two regions previously implicated in representing relations. lmSTC represents broad role combinations that are shared across verbs (e.g., woman-as-agent), rather than narrow roles, limited to specific actions (woman-as-chaser). By contrast, a hippocampal sub-region represents events sharing narrow conjunctions as dissimilar. The success of the hippocampal conjunctive encoding model is anti-correlated with generalization performance in amPFC on a trial-by-trial basis, consistent with a pattern separation mechanism. Thus, these three regions appear to play distinct, but complementary, roles in encoding compositional event structure.

Concepts and Compositionality: In Search of the Brain's Language of Thought

Imagine Genghis Khan, Aretha Franklin, and the Cleveland Cavaliers performing an opera on Maui. This silly sentence makes a serious point: As humans, we can flexibly generate and comprehend an unbounded number of complex ideas. Little is known, however, about how our brains accomplish this. Here we assemble clues from disparate areas of cognitive neuroscience, integrating recent research on language, memory, episodic simulation, and computational models of high-level cognition. Our review is framed by Fodor's classic language of thought hypothesis, according to which our minds employ an amodal, language-like system for combining and recombining simple concepts to form more complex thoughts. Here, we highlight emerging work on combinatorial processes in the brain and consider this work's relation to the language of thought. We review evidence for distinct, but complementary, contributions of map-like representations in subregions of the default mode network and sentence-like representations of conceptual relations in regions of the temporal and prefrontal cortex.

A Double Dissociation in Sensitivity to Verb and Noun Semantics Across Cortical Networks

What is the neural organization of the mental lexicon? Previous research suggests that partially distinct cortical networks are active during verb and noun processing, but what information do these networks represent? We used multivoxel pattern analysis (MVPA) to investigate whether these networks are sensitive to lexicosemantic distinctions among verbs and among nouns and, if so, whether they are more sensitive to distinctions among words in their preferred grammatical class. Participants heard 4 types of verbs (light emission, sound emission, hand-related actions, mouth-related actions) and 4 types of nouns (birds, mammals, manmade places, natural places). As previously shown, the left posterior middle temporal gyrus (LMTG+), and inferior frontal gyrus (LIFG) responded more to verbs, whereas the inferior parietal lobule (LIP), precuneus (LPC), and inferior temporal (LIT) cortex responded more to nouns. MVPA revealed a double-dissociation in lexicosemantic sensitivity: classification was more accurate among verbs than nouns in the LMTG+, and among nouns than verbs in the LIP, LPC, and LIT. However, classification was similar for verbs and nouns in the LIFG, and above chance for the nonpreferred category in all regions. These results suggest that the lexicosemantic information about verbs and nouns is represented in partially nonoverlapping networks.

How Grammar Introduces Asymmetry Into Cognitive Structures: Compositional Semantics, Metaphors, and Schematological Hybrids

This paper presents a preliminary and tentative formulation of a novel empirical generalization governing the relationship between grammar and cognition across a variety of independent domains. Its point of departure is an abstract distinction between two kinds of cognitive structures: symmetric and asymmetric. While in principle any feature whatsoever has the potential for introducing asymmetry, this paper focuses on one specific feature, namely thematic-role assignment. Our main empirical finding concerns the role of language, or, more specifically, grammar, in effecting and maintaining the distinction between symmetric and asymmetric cognitive structures. Specifically, whereas symmetric structures devoid of thematic-role assignment more commonly occur in a non-grammatical and usually also non-verbal medium, asymmetric structures involving thematic-role assignment are more likely to be associated with a grammatical medium. Our work draws together three independent strands of empirical research associated with three diverse phenomenological domains: compositional semantics, metaphors and schematological hybrids. These three domains instantiate conceptual combinations, bringing together two or more subordinate entities into a single superordinate entity. For compositional semantics this consists of a juxtaposition of constituent signs to form a single more complex sign; for metaphors this entails the bringing together of two different concepts in order to produce a comparison; while for schematological hybrids this involves the combination of different entities to form a single new hybrid entity. Our empirical results reveal a remarkable parallelism between the above three domains. Within each domain, symmetric structures tend to be associated with a non-verbal or otherwise non-grammatical medium, while asymmetric structures are more frequently associated with a grammatical medium. Thus, within each domain, grammar introduces asymmetry. More specifically, we find that in all three domains, the asymmetry in question is one that involves the assignment of thematic roles. To capture this effect, we posit two distinct levels, or tiers, of cognition: non-grammatical cognition, more commonly associated with symmetric structures, and grammatical cognition more conducive to asymmetric structures. Within each of the three phenomenological domains, we find the distinction between non-grammatical and grammatical cognition to be manifest in three independent realms, phylogeny, ontogeny, and the architecture of human cognition. Thus, grammar constitutes the driving force behind the transition from symmetric to asymmetric cognitive structures.

Neural dynamics of semantic composition

Human speech comprehension is remarkable for its immediacy and rapidity. The listener interprets an incrementally delivered auditory input, millisecond by millisecond as it is heard, in terms of complex multilevel representations of relevant linguistic and nonlinguistic knowledge. Central to this process are the neural computations involved in semantic combination, whereby the meanings of words are combined into more complex representations, as in the combination of a verb and its following direct object (DO) noun (e.g., "eat the apple"). These combinatorial processes form the backbone for incremental interpretation, enabling listeners to integrate the meaning of each word as it is heard into their dynamic interpretation of the current utterance. Focusing on the verb-DO noun relationship in simple spoken sentences, we applied multivariate pattern analysis and computational semantic modeling to source-localized electro/magnetoencephalographic data to map out the specific representational constraints that are constructed as each word is heard, and to determine how these constraints guide the interpretation of subsequent words in the utterance. Comparing context-independent semantic models of the DO noun with contextually constrained noun models reflecting the semantic properties of the preceding verb, we found that only the contextually constrained model showed a significant fit to the brain data. Pattern-based measures of directed connectivity across the left hemisphere language network revealed a continuous information flow among temporal, inferior frontal, and inferior parietal regions, underpinning the verb's modification of the DO noun's activated semantics. These results provide a plausible neural substrate for seamless real-time incremental interpretation on the observed millisecond time scales.

Semantic memory for objects, actions, and events: A novel test of event-related conceptual semantic knowledge

People possess significant knowledge about how real-world events typically unfold. Such event-related semantic memory connects action and object knowledge, is essential for multiple stages of language processing, and may be impaired in neurological conditions like aphasia. However, current assessments are not well designed for measuring this knowledge. This study presents and tests a novel measure of event-related semantic memory. Task-performance data were collected from unimpaired adults across the lifespan and a sample of stroke survivors with aphasia. Individuals with aphasia also completed measures of language processing and action-/object-related semantic memory, to establish the novel measure's convergent validity. Results demonstrate that performance on the event-knowledge measure correlated with action and object semantic-memory measures and was also associated with a broader range of language-processing performance than other semantic-memory measures. These findings suggest that the novel measure can be used to detect the presence and impact of event-knowledge impairments in neurological conditions.

Verb-argument integration in primary progressive aphasia: Real-time argument access and selection

BACKGROUND

Impaired sentence comprehension is observed in the three major subtypes of PPA, with distinct performance patterns relating to impairments in comprehending complex sentences in the agrammatic (PPA-G) and logopenic (PPA-L) variants and word comprehension in the semantic subtype (PPA-S). However, little is known about basic combinatory processes during sentence comprehension in PPA, such the integration of verbs with their subject and object(s) (verb-argument integration).

METHODS

The present study used visual-world eye-tracking to examine real-time verb-argument integration in individuals with PPA (12 with PPA-G, 10 with PPA-L, and 6 with PPA-S) and neurotypical older adults (15). Two baseline experiments probed eye movement control, using a non-linguistic task, and noun comprehension, respectively. Two verb-argument integration experiments examined the effects of verb meaning on (a) lexical access of the verb's direct object (argument access) and (b) selection of a semantically-appropriate direct object (argument selection), respectively. Eye movement analyses were conducted only for trials with correct behavioral responses, allowing us to distinguish accuracy and online processing.

RESULTS

The eye movement control experiment revealed no significant impairments in PPA, whereas the noun comprehension experiment revealed reduced accuracy and eye-movement latencies in PPA-S, and to a lesser extent PPA-G. In the argument access experiment, verb meaning facilitated argument access normally in PPA-G and PPA-L; in PPA-S, verb-meaning effects emerged on an atypical time course. In the argument selection experiment, significant impairments in accuracy were observed only in PPA-G, accompanied by markedly atypical eye movement patterns.

CONCLUSION

This study revealed two distinct patterns of impaired verb-argument integration in PPA. In PPA-S, impaired verb-argument integration was observed in the argument access experiment, indicating impairments in basic semantic combinatory processes which likely relate to damage in ventral language pathways. In contrast, listeners with PPA-G showed marked impairments of argument selection, likely relating to damage to left inferior frontal regions.

Neurocognitive correlates of category ambiguous verb processing: The single versus dual lexical entry hypotheses

Word-class ambiguous words engender greater processing time and fMRI (BOLD signal) activation than unambiguous ones. Theoretical accounts of this phenomenon suggest that words with multiple meanings (1) are associated with multiple lexical entries and thus require greater selection demands, or (2) undergo computationally expensive grammatical processes that convert words from one word-class to another. Using an fMRI grammaticality judgment task, we tested these accounts by examining word-class ambiguous polysemic (e.g., brush) and homonymic (e.g., bear) verbs, and unambiguous verbs (e.g., bake). Results showed that ambiguous verbs evoked longer response times and greater neural activation in the left inferior frontal and parietal gyri. However, homonymic verbs also showed increased left inferior frontal and temporal neural activations compared to polysemic verbs. This indicates that rather than having multiple lexical representations like homonyms, polysemic verbs may share a core representation with their noun counterparts.

The multifaceted abstract brain

concepts play a central role in human behaviour and constitute a critical component of the human conceptual system. Here, we investigate the neural basis of four types of abstract concepts, examining their similarities and differences through neuroimaging meta-analyses. We examine numerical and emotional concepts, and two higher-order abstract processes, morality judgements and theory of mind. Three main findings emerge. First, representation of abstract concepts is more widespread than is often assumed. Second, representations of different types of abstract concepts differ in important respects. Each of the domains examined here was associated with some unique areas. Third, some areas were commonly activated across domains and included inferior parietal, posterior cingulate and medial prefrontal cortex. We interpret these regions in terms of their role in episodic recall, event representation and social-emotional processing. We suggest that different types of abstract concepts can be represented and grounded through differing contributions from event-based, interoceptive, introspective and sensory-motor representations. The results underscore the richness and diversity of abstract concepts, argue against single-mechanism accounts for representation of all types of abstract concepts and suggest mechanisms for their direct and indirect grounding.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.

Neural Connectivity in Syntactic Movement Processing

Linguistic theory suggests non-canonical sentences subvert the dominant agent-verb-theme order in English via displacement of sentence constituents to argument (NP-movement) or non-argument positions (wh-movement). Both processes have been associated with the left inferior frontal gyrus and posterior superior temporal gyrus, but differences in neural activity and connectivity between movement types have not been investigated. In the current study, functional magnetic resonance imaging data were acquired from 21 adult participants during an auditory sentence-picture verification task using passive and active sentences contrasted to isolate NP-movement, and object- and subject-cleft sentences contrasted to isolate wh-movement. Then, functional magnetic resonance imaging data from regions common to both movement types were entered into a dynamic causal modeling analysis to examine effective connectivity for wh-movement and NP-movement. Results showed greater left inferior frontal gyrus activation for Wh > NP-movement, but no activation for NP > Wh-movement. Both types of movement elicited activity in the opercular part of the left inferior frontal gyrus, left posterior superior temporal gyrus, and left medial superior frontal gyrus. The dynamic causal modeling analyses indicated that neither movement type significantly modulated the connection from the left inferior frontal gyrus to the left posterior superior temporal gyrus, nor vice-versa, suggesting no connectivity differences between wh- and NP-movement. These findings support the idea that increased complexity of wh-structures, compared to sentences with NP-movement, requires greater engagement of cognitive resources via increased neural activity in the left inferior frontal gyrus, but both movement types engage similar neural networks.

The temporal dynamics of structure and content in sentence comprehension: Evidence from fMRI-constrained MEG

Humans have a striking capacity to combine words into sentences that express new meanings. Previous research has identified key brain regions involved in this capacity, but little is known about the time course of activity in these regions, as hemodynamic methods such as fMRI provide little insight into temporal dynamics of neural activation. We performed an MEG experiment to elucidate the temporal dynamics of structure and content processing within four brain regions implicated by fMRI data from the same experiment: the temporo-parietal junction (TPJ), the posterior temporal lobe (PTL), the anterior temporal lobe (ATL), and the anterior inferior frontal gyrus (IFG). The TPJ showed increased activity for both structure and content near the end of the sentence, consistent with a role in incremental interpretation of event semantics. The PTL, a region not often associated with core aspects of syntax, showed a strong early effect of structure, consistent with predictive parsing models, and both structural and semantic context effects on function words. These results provide converging evidence that the PTL plays an important role in lexicalized syntactic processing. The ATL and IFG, regions traditionally associated with syntax, showed minimal effects of sentence structure. The ATL, PTL and IFG all showed effects of semantic content: increased activation for real words relative to nonwords. Our fMRI-guided MEG investigation therefore helps identify syntactic and semantic aspects of sentence comprehension in the brain in both spatial and temporal dimensions.

Neural networks for sentence comprehension and production: An ALE-based meta-analysis of neuroimaging studies

Comprehending and producing sentences is a complex endeavor requiring the coordinated activity of multiple brain regions. We examined three issues related to the brain networks underlying sentence comprehension and production in healthy individuals: First, which regions are recruited for sentence comprehension and sentence production? Second, are there differences for auditory sentence comprehension vs. visual sentence comprehension? Third, which regions are specifically recruited for the comprehension of syntactically complex sentences? Results from activation likelihood estimation (ALE) analyses (from 45 studies) implicated a sentence comprehension network occupying bilateral frontal and temporal lobe regions. Regions implicated in production (from 15 studies) overlapped with the set of regions associated with sentence comprehension in the left hemisphere, but did not include inferior frontal cortex, and did not extend to the right hemisphere. Modality differences between auditory and visual sentence comprehension were found principally in the temporal lobes. Results from the analysis of complex syntax (from 37 studies) showed engagement of left inferior frontal and posterior temporal regions, as well as the right insula. The involvement of the right hemisphere in the comprehension of these structures has potentially important implications for language treatment and recovery in individuals with agrammatic aphasia following left hemisphere brain damage.

Composition of event concepts: Evidence for distinct roles for the left and right anterior temporal lobes

Characterizing the precise computations carried out by the various nodes of the semantic network remains a central challenge. One of the better understood nodes within this system is the left anterior temporal lobe (LATL), which shows an early (∼250 ms) amplitude increase if the semantic composition between the current word and its context is in some ways "simple." As this type of effect has only been demonstrated for noun-modifier composition, we asked if a similar pattern is elicited for verb phrase composition. Agentive, resultative, and eventive adverbs were employed to vary whether the meaning of the adverb directly applies to the verb or not, with only eventives exemplifying direct and straightforward composition. Results showed that eventives, but not agentives or resultatives, elicited a significant increase in the LATL at 250 ms. The RATL showed a sharply contrasting pattern, with agentives showing the largest activity, suggesting a distinct role in semantic composition.

Relational vs. attributive interpretation of nominal compounds differentially engages angular gyrus and anterior temporal lobe

The angular gyrus (AG) and anterior temporal lobe (ATL) have been found to respond to a number of tasks involving combinatorial processing. In this study, we investigate the conceptual combination of nominal compounds, and ask whether ATL/AG activity is modulated by the type of combinatorial operation applied to a nominal compound. We compare relational and attributive interpretations of nominal compounds and find that ATL and AG both discriminate these two types, but in distinct ways. While right AG demonstrated greater positive task-responsive activity for relational compounds, there was a greater negative deflection in the BOLD response in left AG for relational compounds. In left ATL, we found an earlier peak in subjects' BOLD response curves for attributive interpretations. In other words, we observed dissociations in both AG and ATL between relational and attributive nominal compounds, with regard to magnitude in the former and to timing in the latter. These findings expand on prior studies that posit roles for both AG and ATL in conceptual processing generally, and in conceptual combination specifically, by indicating possible functional specializations of these two regions within a larger conceptual knowledge network.

Taxonomic and thematic semantic systems

Object concepts are critical for nearly all aspects of human cognition, from perception tasks like object recognition, to understanding and producing language, to making meaningful actions. Concepts can have 2 very different kinds of relations: similarity relations based on shared features (e.g., dog-bear), which are called "taxonomic" relations, and contiguity relations based on co-occurrence in events or scenarios (e.g., dog-leash), which are called "thematic" relations. Here, we report a systematic review of experimental psychology and cognitive neuroscience evidence of this distinction in the structure of semantic memory. We propose 2 principles that may drive the development of distinct taxonomic and thematic semantic systems: differences between which features determine taxonomic versus thematic relations, and differences in the processing required to extract taxonomic versus thematic relations. This review brings together distinct threads of behavioral, computational, and neuroscience research on semantic memory in support of a functional and neural dissociation, and defines a framework for future studies of semantic memory. (PsycINFO Database Record