Supramodal Sentence Processing in the Human Brain: fMRI Evidence for the Influence of Syntactic Complexity in More Than 200 Participants

The neurobiology of sentence production: A narrative review and meta-analysis

Although there is a sizeable body of literature on sentence comprehension and processing both in healthy and disordered language users, the literature on sentence production remains much more sparse. Linguistic and computational descriptions of expressive syntactic deficits in aphasia are especially rare. In addition, the neuroimaging and (psycho) linguistic literatures operate largely separately. In this paper, I will first lay out the theoretical lay of the land with regard to psycholinguistic models of sentence production. I will then provide a brief narrative overview and large-scale meta-analysis of the neuroimaging literature as it pertains to syntactic computation, followed by an attempt to integrate the psycholinguistic models with the findings from functional and clinical neuroimaging. Finally, I provide a brief overview of the literature surrounding expressive syntactic deficits and propose a path forward to close some of the existing gaps.

Evaluating the factor structure of the Dutch individual differences in language skills (IDLaS-NL) test battery

Individual differences in using language are prevalent in our daily lives. Language skills are often assessed in vocational (predominantly written language) and diagnostic contexts. Not much is known, however, about individual differences in spoken language skills. The lack of research is in part due to the lack of suitable test instruments. We introduce the Individual Differences in Language Skills (IDLaS-NL) test battery, a set of 31 behavioural tests that can be used to capture variability in language and relevant general cognitive skills in adult speakers of Dutch. The battery was designed to measure word and sentence production and comprehension skills, linguistic knowledge, nonverbal processing speed, working memory, and nonverbal reasoning. The present article outlines the structure of the battery, describes the materials and procedure of each test, and evaluates the battery's factor structure based on the results of a sample of 748 Dutch adults, aged between 18 and 30 years, most of them students. The analyses demonstrate that the battery has good construct validity and can be reliably administered both in the lab and via the internet. We therefore recommend the battery as a valuable new tool to assess individual differences in language knowledge and skills; this future work may include linking language skills to other aspects of human cognition and life outcomes.

The role of variation in phonological and semantic working memory capacities in sentence comprehension: neural evidence from healthy and brain-damaged individuals

Research on the role of working memory (WM) in language processing has typically focused on WM for phonological information. However, considerable behavioral evidence supports the existence of a separate semantic WM system that plays a greater role in language processing. We review the neural evidence that supports the distinction between phonological and semantic WM capacities and discuss how individual differences in these capacities relate to sentence processing. In terms of neural substrates, findings from multivariate functional MRI for healthy participants and voxel-based lesion-symptom mapping for brain-damaged participants imply that the left supramarginal gyrus supports phonological WM, whereas the left inferior frontal gyrus (LIFG) and angular gyrus support semantic WM. In sentence comprehension, individual variation in semantic but not phonological WM related to performance in resolving semantic information and the LIFG region implicated in semantic WM showed fMRI activation during the resolution of semantic interference. Moreover, variation for brain-damaged participants in the integrity of a fiber tract supporting semantic WM had a greater relation to the processing of complex sentences than did the integrity of fiber tracts supporting phonological WM. Overall, the neural findings provide converging evidence regarding the distinction of these two capacities and the greater contribution of individual differences in semantic than phonological WM capacity to sentence processing.

Functional and structural brain asymmetries in language processing

The lateralization of language to the left hemisphere of the human brain constitutes one of the classic examples of asymmetry in biology. At the same time, it is also commonly understood that damage to the left hemisphere does not lead to a complete loss of all linguistic abilities. These seemingly contradictory findings indicate that neither our cognitive capacity for language nor its neural substrates are monolithic. This chapter reviews the functional and structural lateralization of the neural substrates of different aspects of language as revealed in the past decades by neuroimaging research. Most aspects of language processing indeed tend to be functionally lateralized to the left hemisphere in the adult human brain. Nevertheless, both hemispheres exhibit a certain equipotentiality with regard to some aspects of language processing, especially with regard to processing meaning and sound. In contrast, the so-called "core language network" in the left hemisphere constitutes a functional and structural asymmetry: This network (i) is crucial for a core aspect of language processing, namely syntax, which refers to the generation of hierarchically structured representations of utterances linking meaning and sound, (ii) matures in accordance with a genetically determined biologic matrix, and (iii) its emergence may have constituted a prerequisite for the evolution of the human language capacity.

Functional and structural brain asymmetries in sign language processing

The capacity for language constitutes a cornerstone of human cognition and distinguishes our species from other animals. Research in the cognitive sciences has demonstrated that this capacity is not bound to speech but can also be externalized in the form of sign language. Sign languages are the naturally occurring languages of the deaf and rely on movements and configurations of hands, arms, face, and torso in space. This chapter reviews the functional and structural organization of the neural substrates of sign language, as identified by neuroimaging research over the past decades. Most aspects of sign language processing in adult deaf signers markedly mirror the well-known, functional left-lateralization of spoken and written language. However, both hemispheres exhibit a certain equipotentiality for processing linguistic information and the right hemisphere seems to specifically support processing of some constructions unique to the signed modality. Crucially, the so-called "core language network" in the left hemisphere constitutes a functional and structural asymmetry in typically developed deaf and hearing populations alike: This network is (i) pivotal for processing complex syntax independent of the modality of language use, (ii) matures in accordance with a genetically determined biologic matrix, and (iii) may have constituted an evolutionary prerequisite for the emergence of the human capacity for language.

Connectivity of Fronto-Temporal Regions in Syntactic Structure Building During Speaking and Listening

The neural infrastructure for sentence production and comprehension has been found to be mostly shared. The same regions are engaged during speaking and listening, with some differences in how strongly they activate depending on modality. In this study, we investigated how modality affects the connectivity between regions previously found to be involved in syntactic processing across modalities. We determined how constituent size and modality affected the connectivity of the pars triangularis of the left inferior frontal gyrus (LIFG) and of the left posterior temporal lobe (LPTL) with the pars opercularis of the LIFG, the left anterior temporal lobe (LATL), and the rest of the brain. We found that constituent size reliably increased the connectivity across these frontal and temporal ROIs. Connectivity between the two LIFG regions and the LPTL was enhanced as a function of constituent size in both modalities, and it was upregulated in production possibly because of linearization and motor planning in the frontal cortex. The connectivity of both ROIs with the LATL was lower and only enhanced for larger constituent sizes, suggesting a contributing role of the LATL in sentence processing in both modalities. These results thus show that the connectivity among fronto-temporal regions is upregulated for syntactic structure building in both sentence production and comprehension, providing further evidence for accounts of shared neural resources for sentence-level processing across modalities.

Verbal short term memory contribution to sentence comprehension decreases with increasing syntactic complexity in people with aphasia

Sentence comprehension requires the integration of linguistic units presented in a temporal sequence based on a non-linear underlying syntactic structure. While it is uncontroversial that storage is mandatory for this process, there are opposing views regarding the relevance of general short-term-/working-memory capacities (STM/WM) versus language specific resources. Here we report results from 43 participants with an acquired brain lesion in the extended left hemispheric language network and resulting language deficits, who performed a sentence-to-picture matching task and an experimental task assessing phonological short-term memory. The sentence task systematically varied syntactic complexity (embedding depth and argument order) while lengths, number of propositions and plausibility were kept constant. Clinical data including digit-/ block-spans and lesion size and site were additionally used in the analyses. Correlational analyses confirm that performance on STM/WM-tasks (experimental task and digit-span) are the only two relevant predictors for correct sentence-picture-matching, while reaction times only depended on age and lesion size. Notably increasing syntactic complexity reduced the correlational strength speaking for the additional recruitment of language specific resources independent of more general verbal STM/WM capacities, when resolving complex syntactic structure. The complementary lesion-behaviour analysis yielded different lesion volumes correlating with either the sentence-task or the STM-task. Factoring out STM measures lesions in the anterior temporal lobe correlated with a larger decrease in accuracy with increasing syntactic complexity. We conclude that overall sentence comprehension depends on STM/WM capacity, while increases in syntactic complexity tax another independent cognitive resource.

Diverging neural dynamics for syntactic structure building in naturalistic speaking and listening

The neural correlates of sentence production are typically studied using task paradigms that differ considerably from the experience of speaking outside of an experimental setting. In this fMRI study, we aimed to gain a better understanding of syntactic processing in spontaneous production versus naturalistic comprehension in three regions of interest (BA44, BA45, and left posterior middle temporal gyrus). A group of participants (n = 16) was asked to speak about the events of an episode of a TV series in the scanner. Another group of participants (n = 36) listened to the spoken recall of a participant from the first group. To model syntactic processing, we extracted word-by-word metrics of phrase-structure building with a top-down and a bottom-up parser that make different hypotheses about the timing of structure building. While the top-down parser anticipates syntactic structure, sometimes before it is obvious to the listener, the bottom-up parser builds syntactic structure in an integratory way after all of the evidence has been presented. In comprehension, neural activity was found to be better modeled by the bottom-up parser, while in production, it was better modeled by the top-down parser. We additionally modeled structure building in production with two strategies that were developed here to make different predictions about the incrementality of structure building during speaking. We found evidence for highly incremental and anticipatory structure building in production, which was confirmed by a converging analysis of the pausing patterns in speech. Overall, this study shows the feasibility of studying the neural dynamics of spontaneous language production.

IDLaS-NL - A platform for running customized studies on individual differences in Dutch language skills via the Internet

We introduce the Individual Differences in Language Skills (IDLaS-NL) web platform, which enables users to run studies on individual differences in Dutch language skills via the Internet. IDLaS-NL consists of 35 behavioral tests, previously validated in participants aged between 18 and 30 years. The platform provides an intuitive graphical interface for users to select the tests they wish to include in their research, to divide these tests into different sessions and to determine their order. Moreover, for standardized administration the platform provides an application (an emulated browser) wherein the tests are run. Results can be retrieved by mouse click in the graphical interface and are provided as CSV file output via e-mail. Similarly, the graphical interface enables researchers to modify and delete their study configurations. IDLaS-NL is intended for researchers, clinicians, educators and in general anyone conducting fundamental research into language and general cognitive skills; it is not intended for diagnostic purposes. All platform services are free of charge. Here, we provide a description of its workings as well as instructions for using the platform. The IDLaS-NL platform can be accessed at www.mpi.nl/idlas-nl .

Conversational production and comprehension: fMRI-evidence reminiscent of but deviant from the classical Broca-Wernicke model

A key question in research on the neurobiology of language is to which extent the language production and comprehension systems share neural infrastructure, but this question has not been addressed in the context of conversation. We utilized a public fMRI dataset where 24 participants engaged in unscripted conversations with a confederate outside the scanner, via an audio-video link. We provide evidence indicating that the two systems share neural infrastructure in the left-lateralized perisylvian language network, but diverge regarding the level of activation in regions within the network. Activity in the left inferior frontal gyrus was stronger in production compared to comprehension, while comprehension showed stronger recruitment of the left anterior middle temporal gyrus and superior temporal sulcus, compared to production. Although our results are reminiscent of the classical Broca-Wernicke model, the anterior (rather than posterior) temporal activation is a notable difference from that model. This is one of the findings that may be a consequence of the conversational setting, another being that conversational production activated what we interpret as higher-level socio-pragmatic processes. In conclusion, we present evidence for partial overlap and functional asymmetry of the neural infrastructure of production and comprehension, in the above-mentioned frontal vs temporal regions during conversation.

Automated Measures of Syntactic Complexity in Natural Speech Production: Older and Younger Adults as a Case Study

PURPOSE

Multiple methods have been suggested for quantifying syntactic complexity in speech. We compared eight automated syntactic complexity metrics to determine which best captured verified syntactic differences between old and young adults.

METHOD

We used natural speech samples produced in a picture description task by younger (n = 76, ages 18-22 years) and older (n = 36, ages 53-89 years) healthy participants, manually transcribed and segmented into sentences. We manually verified that older participants produced fewer complex structures. We developed a metric of syntactic complexity using automatically extracted syntactic structures as features in a multidimensional metric. We compared our metric to seven other metrics: Yngve score, Frazier score, Frazier-Roark score, developmental level, syntactic frequency, mean dependency distance, and sentence length. We examined the success of each metric in identifying the age group using logistic regression models. We repeated the analysis with automatic transcription and segmentation using an automatic speech recognition (ASR) system.

RESULTS

Our multidimensional metric was successful in predicting age group (area under the curve [AUC] = 0.87), and it performed better than the other metrics. High AUCs were also achieved by the Yngve score (0.84) and sentence length (0.84). However, in a fully automated pipeline with ASR, the performance of these two metrics dropped (to 0.73 and 0.46, respectively), while the performance of the multidimensional metric remained relatively high (0.81).

CONCLUSIONS

Syntactic complexity in spontaneous speech can be quantified by directly assessing syntactic structures and considering them in a multivariable manner. It can be derived automatically, saving considerable time and effort compared to manually analyzing large-scale corpora, while maintaining high face validity and robustness.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.24964179.

Neural correlates of pronoun processing: An activation likelihood estimation meta-analysis

Pronouns are unique linguistic devices that allow for the expression of referential relationships. Despite their communicative utility, the neural correlates of the operations involved in reference assignment and/or resolution, are not well-understood. The present study synthesized the neuroimaging literature on pronoun processing to test extant theories of pronoun comprehension. Following the PRISMA guidelines and thebest-practice recommendations for neuroimaging meta-analyses, a systematic literature search and record assessment were performed. As a result, 16 fMRI studies were included in the meta-analysis, and were coded in Scribe 3.6 for inclusion in the BrainMap database. The activation coordinates for the contrasts of interest were transformed into Talairach space and submitted to an Activation Likelihood Estimation (ALE) meta-analysis in GingerALE 3.0.1. The results indicated that pronoun processing had functional convergence in the left posterior middle and superior temporal gyri, potentially reflecting the retrieval, prediction and integration roles of these areas for pronoun processing.

Exploring the neurobiology of Merge at a basic level: insights from a novel artificial grammar paradigm

Introduction

Human language allows us to generate an infinite number of linguistic expressions. It's proposed that this competence is based on a binary syntactic operation, Merge, combining two elements to form a new constituent. An increasing number of recent studies have shifted from complex syntactic structures to two-word constructions to investigate the neural representation of this operation at the most basic level.

Methods

This fMRI study aimed to develop a highly flexible artificial grammar paradigm for testing the neurobiology of human syntax at a basic level. During scanning, participants had to apply abstract syntactic rules to assess whether a given two-word artificial phrase could be further merged with a third word. To control for lower-level template-matching and working memory strategies, an additional non-mergeable word-list task was set up.

Results

Behavioral data indicated that participants complied with the experiment. Whole brain and region of interest (ROI) analyses were performed under the contrast of "structure > word-list." Whole brain analysis confirmed significant involvement of the posterior inferior frontal gyrus [pIFG, corresponding to Brodmann area (BA) 44]. Furthermore, both the signal intensity in Broca's area and the behavioral performance showed significant correlations with natural language performance in the same participants. ROI analysis within the language atlas and anatomically defined Broca's area revealed that only the pIFG was reliably activated.

Discussion

Taken together, these results support the notion that Broca's area, particularly BA 44, works as a combinatorial engine where words are merged together according to syntactic information. Furthermore, this study suggests that the present artificial grammar may serve as promising material for investigating the neurobiological basis of syntax, fostering future cross-species studies.