Comprehension of computer code relies primarily on domain-general executive brain regions

Intuitive physical reasoning is not mediated by linguistic nor exclusively domain-general abstract representations

The ability to reason about the physical world is a critical tool in the human cognitive toolbox, but the nature of the representations that mediate physical reasoning remains debated. Here, we use fMRI to illuminate this question by investigating the relationship between the physical-reasoning system and two well-characterized systems: a) the domain-general Multiple Demand (MD) system, which supports abstract reasoning, including mathematical and logical reasoning, and b) the language system, which supports linguistic computations and has been hypothesized to mediate some forms of thought. We replicate prior findings of a network of frontal and parietal areas that are robustly engaged by physical reasoning and identify an additional physical-reasoning area in the left frontal cortex, which also houses components of the MD and language systems. Critically, direct comparisons with tasks that target the MD and the language systems reveal that the physical-reasoning system overlaps with the MD system, but is dissociable from it in fine-grained activation patterns, which replicates prior work. Moreover, the physical-reasoning system does not overlap with the language system. These results suggest that physical reasoning does not rely on linguistic representations, nor exclusively on the domain-general abstract reasoning that the MD system supports.

Constructed languages are processed by the same brain mechanisms as natural languages

What constitutes a language? Natural languages share features with other domains: from math, to music, to gesture. However, the brain mechanisms that process linguistic input are highly specialized, showing little response to diverse nonlinguistic tasks. Here, we examine constructed languages (conlangs) to ask whether they draw on the same neural mechanisms as natural languages or whether they instead pattern with domains like math and programming languages. Using individual-subject fMRI analyses, we show that understanding conlangs recruits the same brain areas as natural language comprehension. This result holds for Esperanto (n = 19 speakers) and four fictional conlangs [Klingon (n = 10), Na'vi (n = 9), High Valyrian (n = 3), and Dothraki (n = 3)]. These findings suggest that conlangs and natural languages share critical features that allow them to draw on the same representations and computations, implemented in the left-lateralized network of brain areas. The features of conlangs that differentiate them from natural languages-including recent creation by a single individual, often for an esoteric purpose, the small number of speakers, and the fact that these languages are typically learned in adulthood-appear to not be consequential for the reliance on the same cognitive and neural mechanisms. We argue that the critical shared feature of conlangs and natural languages is that they are symbolic systems capable of expressing an open-ended range of meanings about our outer and inner worlds.

Universality of representation in biological and artificial neural networks

Many artificial neural networks (ANNs) trained with ecologically plausible objectives on naturalistic data align with behavior and neural representations in biological systems. Here, we show that this alignment is a consequence of convergence onto the same representations by high-performing ANNs and by brains. We developed a method to identify stimuli that systematically vary the degree of inter-model representation agreement. Across language and vision, we then showed that stimuli from high- and low-agreement sets predictably modulated model-to-brain alignment. We also examined which stimulus features distinguish high- from low-agreement sentences and images. Our results establish representation universality as a core component in the model-to-brain alignment and provide a new approach for using ANNs to uncover the structure of biological representations and computations.

The language network ages well: Preserved selectivity, lateralization, and within-network functional synchronization in older brains

Healthy aging is associated with structural and functional brain changes. However, cognitive abilities differ from one another in how they change with age: whereas executive functions, like working memory, show age-related decline, aspects of linguistic processing remain relatively preserved (Hartshorne et al., 2015). This heterogeneity of the cognitive-behavioral landscape in aging predicts differences among brain networks in whether and how they should change with age. To evaluate this prediction, we used individual-subject fMRI analyses ('precision fMRI') to examine the language-selective network (Fedorenko et al., 2024) and the Multiple Demand (MD) network, which supports executive functions (Duncan et al., 2020), in older adults (n=77) relative to young controls (n=470). In line with past claims, relative to young adults, the MD network of older adults shows weaker and less spatially extensive activations during an executive function task and reduced within-network functional synchronization. However, in stark contrast to the MD network, we find remarkable preservation of the language network in older adults. Their language network responds to language as strongly and selectively as in younger adults, and is similarly lateralized and internally synchronized. In other words, the language network of older adults looks indistinguishable from that of younger adults. Our findings align with behavioral preservation of language skills in aging and suggest that some networks remain young-like, at least on standard measures of function and connectivity.

Language in Brains, Minds, and Machines

It has long been argued that only humans could produce and understand language. But now, for the first time, artificial language models (LMs) achieve this feat. Here we survey the new purchase LMs are providing on the question of how language is implemented in the brain. We discuss why, a priori, LMs might be expected to share similarities with the human language system. We then summarize evidence that LMs represent linguistic information similarly enough to humans to enable relatively accurate brain encoding and decoding during language processing. Finally, we examine which LM properties-their architecture, task performance, or training-are critical for capturing human neural responses to language and review studies using LMs as in silico model organisms for testing hypotheses about language. These ongoing investigations bring us closer to understanding the representations and processes that underlie our ability to comprehend sentences and express thoughts in language.

Early-emerging combinatorial thought: Human infants flexibly combine kind and quantity concepts

Combinatorial thought, or the ability to combine a finite set of concepts into a myriad of complex ideas and knowledge structures, is the key to the productivity of the human mind and underlies communication, science, technology, and art. Despite the importance of combinatorial thought for human cognition and culture, its developmental origins remain unknown. To address this, we tested whether 12-mo-old infants (N = 60), who cannot yet speak and only understand a handful of words, can combine quantity and kind concepts activated by verbal input. We proceeded in two steps: first, we taught infants two novel labels denoting quantity (e.g., "mize" for 1 item; "padu" for 2 items, Experiment 1). Then, we assessed whether they could combine quantity and kind concepts upon hearing complex expressions comprising their labels (e.g., "padu duck", Experiments 2-3). At test, infants viewed four different sets of objects (e.g., 1 duck, 2 ducks, 1 ball, 2 balls) while being presented with the target phrase (e.g., "padu duck") naming one of them (e.g., 2 ducks). They successfully retrieved and combined on-line the labeled concepts, as evidenced by increased looking to the named sets but not to distractor sets. Our results suggest that combinatorial processes for building complex representations are available by the end of the first year of life. The infant mind seems geared to integrate concepts in novel productive ways. This ability may be a precondition for deciphering the ambient language(s) and building abstract models of experience that enable fast and flexible learning.

Linguistic inputs must be syntactically parsable to fully engage the language network

Human language comprehension is remarkably robust to ill-formed inputs (e.g., word transpositions). This robustness has led some to argue that syntactic parsing is largely an illusion, and that incremental comprehension is more heuristic, shallow, and semantics-based than is often assumed. However, the available data are also consistent with the possibility that humans always perform rule-like symbolic parsing and simply deploy error correction mechanisms to reconstruct ill-formed inputs when needed. We put these hypotheses to a new stringent test by examining brain responses to a) stimuli that should pose a challenge for syntactic reconstruction but allow for complex meanings to be built within local contexts through associative/shallow processing (sentences presented in a backward word order), and b) grammatically well-formed but semantically implausible sentences that should impede semantics-based heuristic processing. Using a novel behavioral syntactic reconstruction paradigm, we demonstrate that backward-presented sentences indeed impede the recovery of grammatical structure during incremental comprehension. Critically, these backward-presented stimuli elicit a relatively low response in the language areas, as measured with fMRI. In contrast, semantically implausible but grammatically well-formed sentences elicit a response in the language areas similar in magnitude to naturalistic (plausible) sentences. In other words, the ability to build syntactic structures during incremental language processing is both necessary and sufficient to fully engage the language network. Taken together, these results provide strongest to date support for a generalized reliance of human language comprehension on syntactic parsing.

Precision fMRI reveals that the language network exhibits adult-like left-hemispheric lateralization by 4 years of age

Left hemisphere damage in adulthood often leads to linguistic deficits, but many cases of early damage leave linguistic processing preserved, and a functional language system can develop in the right hemisphere. To explain this early apparent equipotentiality of the two hemispheres for language, some have proposed that the language system is bilateral during early development and only becomes left-lateralized with age. We examined language lateralization using functional magnetic resonance imaging with two large pediatric cohorts (total n=273 children ages 4-16; n=107 adults). Strong, adult-level left-hemispheric lateralization (in activation volume and response magnitude) was evident by age 4. Thus, although the right hemisphere can take over language function in some cases of early brain damage, and although some features of the language system do show protracted development (magnitude of language response and strength of inter-regional correlations in the language network), the left-hemisphere bias for language is robustly present by 4 years of age. These results call for alternative accounts of early equipotentiality of the two hemispheres for language.

The Language Network Reliably "Tracks" Naturalistic Meaningful Nonverbal Stimuli

The language network, comprised of brain regions in the left frontal and temporal cortex, responds robustly and reliably during language comprehension but shows little or no response during many nonlinguistic cognitive tasks (e.g., Fedorenko & Blank, 2020). However, one domain whose relationship with language remains debated is semantics-our conceptual knowledge of the world. Given that the language network responds strongly to meaningful linguistic stimuli, could some of this response be driven by the presence of rich conceptual representations encoded in linguistic inputs? In this study, we used a naturalistic cognition paradigm to test whether the cognitive and neural resources that are responsible for language processing are also recruited for processing semantically rich nonverbal stimuli. To do so, we measured BOLD responses to a set of ∼5-minute-long video and audio clips that consisted of meaningful event sequences but did not contain any linguistic content. We then used the intersubject correlation (ISC) approach (Hasson et al., 2004) to examine the extent to which the language network "tracks" these stimuli, that is, exhibits stimulus-related variation. Across all the regions of the language network, meaningful nonverbal stimuli elicited reliable ISCs. These ISCs were higher than the ISCs elicited by semantically impoverished nonverbal stimuli (e.g., a music clip), but substantially lower than the ISCs elicited by linguistic stimuli. Our results complement earlier findings from controlled experiments (e.g., Ivanova et al., 2021) in providing further evidence that the language network shows some sensitivity to semantic content in nonverbal stimuli.

Dissociating language and thought in large language models

Large language models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their linguistic and cognitive capabilities remain split. Here, we evaluate LLMs using a distinction between formal linguistic competence (knowledge of linguistic rules and patterns) and functional linguistic competence (understanding and using language in the world). We ground this distinction in human neuroscience, which has shown that formal and functional competence rely on different neural mechanisms. Although LLMs are surprisingly good at formal competence, their performance on functional competence tasks remains spotty and often requires specialized fine-tuning and/or coupling with external modules. We posit that models that use language in human-like ways would need to master both of these competence types, which, in turn, could require the emergence of separate mechanisms specialized for formal versus functional linguistic competence.

Language is primarily a tool for communication rather than thought

Language is a defining characteristic of our species, but the function, or functions, that it serves has been debated for centuries. Here we bring recent evidence from neuroscience and allied disciplines to argue that in modern humans, language is a tool for communication, contrary to a prominent view that we use language for thinking. We begin by introducing the brain network that supports linguistic ability in humans. We then review evidence for a double dissociation between language and thought, and discuss several properties of language that suggest that it is optimized for communication. We conclude that although the emergence of language has unquestionably transformed human culture, language does not appear to be a prerequisite for complex thought, including symbolic thought. Instead, language is a powerful tool for the transmission of cultural knowledge; it plausibly co-evolved with our thinking and reasoning capacities, and only reflects, rather than gives rise to, the signature sophistication of human cognition.

The language network as a natural kind within the broader landscape of the human brain

Language behaviour is complex, but neuroscientific evidence disentangles it into distinct components supported by dedicated brain areas or networks. In this Review, we describe the 'core' language network, which includes left-hemisphere frontal and temporal areas, and show that it is strongly interconnected, independent of input and output modalities, causally important for language and language-selective. We discuss evidence that this language network plausibly stores language knowledge and supports core linguistic computations related to accessing words and constructions from memory and combining them to interpret (decode) or generate (encode) linguistic messages. We emphasize that the language network works closely with, but is distinct from, both lower-level - perceptual and motor - mechanisms and higher-level systems of knowledge and reasoning. The perceptual and motor mechanisms process linguistic signals, but, in contrast to the language network, are sensitive only to these signals' surface properties, not their meanings; the systems of knowledge and reasoning (such as the system that supports social reasoning) are sometimes engaged during language use but are not language-selective. This Review lays a foundation both for in-depth investigations of these different components of the language processing pipeline and for probing inter-component interactions.

Contribution of the language network to the comprehension of Python programming code

Does the perisylvian language network contribute to comprehension of programming languages, like Python? Univariate neuroimaging studies find high responses to code in fronto-parietal executive areas but not in fronto-temporal language areas, suggesting the language network does little. We used multivariate-pattern-analysis to test whether the language network encodes Python functions. Python programmers read functions while undergoing fMRI. A linear SVM decoded for-loops from if-conditionals based on activity in lateral temporal (LT) language cortex. In searchlight analysis, decoding accuracy was higher in LT language cortex than anywhere else. Follow up analysis showed that decoding was not driven by presence of different words across functions, "for" vs "if," but by compositional program properties. Finally, univariate responses to code peaked earlier in LT language-cortex than in the fronto-parietal network. We propose that the language system forms initial "surface meaning" representations of programs, which input to the reasoning network for processing of algorithms.

Computer programmers show distinct, expertise-dependent brain responses to violations in form and meaning when reading code

As computer programming becomes more central to the workforce, the need for better models of how it is effectively learned has become more apparent. The current study addressed this gap by recording electrophysiological brain responses as 62 Python programmers with varying skill levels read lines of code with manipulations of form (syntax) and meaning (semantics). At the group level, results showed that manipulations of form resulted in P600 effects, with syntactically invalid code generating more positive deflections in the 500-800 ms range than syntactically valid code. Meaning manipulations resulted in N400 effects, with semantically implausible code generating more negative deflections in the 300-500 ms range than semantically plausible code. Greater Python expertise within the group was associated with greater sensitivity to violations in form. These results support the notion that skilled programming, like skilled natural language learning, is associated with the incorporation of rule-based knowledge into online comprehension processes. Conversely, programmers at all skill levels showed neural sensitivity to meaning manipulations, suggesting that reliance on pre-existing semantic relationships facilitates code comprehension across skill levels.

High-level language brain regions process sublexical regularities

A network of left frontal and temporal brain regions supports language processing. This "core" language network stores our knowledge of words and constructions as well as constraints on how those combine to form sentences. However, our linguistic knowledge additionally includes information about phonemes and how they combine to form phonemic clusters, syllables, and words. Are phoneme combinatorics also represented in these language regions? Across five functional magnetic resonance imaging experiments, we investigated the sensitivity of high-level language processing brain regions to sublexical linguistic regularities by examining responses to diverse nonwords-sequences of phonemes that do not constitute real words (e.g. punes, silory, flope). We establish robust responses in the language network to visually (experiment 1a, n = 605) and auditorily (experiments 1b, n = 12, and 1c, n = 13) presented nonwords. In experiment 2 (n = 16), we find stronger responses to nonwords that are more well-formed, i.e. obey the phoneme-combinatorial constraints of English. Finally, in experiment 3 (n = 14), we provide suggestive evidence that the responses in experiments 1 and 2 are not due to the activation of real words that share some phonology with the nonwords. The results suggest that sublexical regularities are stored and processed within the same fronto-temporal network that supports lexical and syntactic processes.

Functional characterization of the language network of polyglots and hyperpolyglots with precision fMRI

How do polyglots-individuals who speak five or more languages-process their languages, and what can this population tell us about the language system? Using fMRI, we identified the language network in each of 34 polyglots (including 16 hyperpolyglots with knowledge of 10+ languages) and examined its response to the native language, non-native languages of varying proficiency, and unfamiliar languages. All language conditions engaged all areas of the language network relative to a control condition. Languages that participants rated as higher proficiency elicited stronger responses, except for the native language, which elicited a similar or lower response than a non-native language of similar proficiency. Furthermore, unfamiliar languages that were typologically related to the participants' high-to-moderate-proficiency languages elicited a stronger response than unfamiliar unrelated languages. The results suggest that the language network's response magnitude scales with the degree of engagement of linguistic computations (e.g. related to lexical access and syntactic-structure building). We also replicated a prior finding of weaker responses to native language in polyglots than non-polyglot bilinguals. These results contribute to our understanding of how multiple languages coexist within a single brain and provide new evidence that the language network responds more strongly to stimuli that more fully engage linguistic computations.

Left-Hemisphere Cortical Language Regions Respond Equally to Observed Dialogue and Monologue

Much of the language we encounter in our everyday lives comes in the form of conversation, yet the majority of research on the neural basis of language comprehension has used input from only one speaker at a time. Twenty adults were scanned while passively observing audiovisual conversations using functional magnetic resonance imaging. In a block-design task, participants watched 20 s videos of puppets speaking either to another puppet (the dialogue condition) or directly to the viewer (the monologue condition), while the audio was either comprehensible (played forward) or incomprehensible (played backward). Individually functionally localized left-hemisphere language regions responded more to comprehensible than incomprehensible speech but did not respond differently to dialogue than monologue. In a second task, participants watched videos (1-3 min each) of two puppets conversing with each other, in which one puppet was comprehensible while the other's speech was reversed. All participants saw the same visual input but were randomly assigned which character's speech was comprehensible. In left-hemisphere cortical language regions, the time course of activity was correlated only among participants who heard the same character speaking comprehensibly, despite identical visual input across all participants. For comparison, some individually localized theory of mind regions and right-hemisphere homologues of language regions responded more to dialogue than monologue in the first task, and in the second task, activity in some regions was correlated across all participants regardless of which character was speaking comprehensibly. Together, these results suggest that canonical left-hemisphere cortical language regions are not sensitive to differences between observed dialogue and monologue.

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are highly predictive of human brain responses to language. Here, using fMRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of brain response associated with each sentence. Then, we use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also noninvasively control neural activity in higher-level cortical areas, like the language network.

Changes in functional brain activity patterns associated with computer programming learning in novices

BACKGROUND

Computer programming, the process of designing, writing, and testing executable computer code, is an essential skill in numerous fields. A description of the neural structures engaged and modified during programming skill acquisition could help improve training programs and provide clues to the neural substrates underlying the acquisition of related skills.

METHODS

Fourteen female university students without prior computer programing experience were examined by functional magnetic resonance imaging (fMRI) during the early and late stages of a 5-month 'Computer Processing' course. Brain regions involved in task performance and learning were identified by comparing responses to programming and control tasks during the early and late stages.

RESULTS

The accuracy of performing a programming task was significantly improved during the late stage. Various regions of the frontal, temporal, parietal, and occipital cortex as well as several subcortical structures (caudate nuclei and cerebellum) were activated during programming tasks. Brain activity in the right inferior frontal gyrus was greater during the late stage and significantly correlated with improved task performance. Although the left inferior frontal gyrus was also highly active during the programming task, there were no learning-induced changes in activity or a significant correlation between activity and improved task performances.

CONCLUSION

Computer programming learning among novices induces functional neuroplasticity within the right inferior frontal gyrus but not the left inferior gyrus (Broca's area).

The human language system, including its inferior frontal component in "Broca's area," does not support music perception

Language and music are two human-unique capacities whose relationship remains debated. Some have argued for overlap in processing mechanisms, especially for structure processing. Such claims often concern the inferior frontal component of the language system located within "Broca's area." However, others have failed to find overlap. Using a robust individual-subject fMRI approach, we examined the responses of language brain regions to music stimuli, and probed the musical abilities of individuals with severe aphasia. Across 4 experiments, we obtained a clear answer: music perception does not engage the language system, and judgments about music structure are possible even in the presence of severe damage to the language network. In particular, the language regions' responses to music are generally low, often below the fixation baseline, and never exceed responses elicited by nonmusic auditory conditions, like animal sounds. Furthermore, the language regions are not sensitive to music structure: they show low responses to both intact and structure-scrambled music, and to melodies with vs. without structural violations. Finally, in line with past patient investigations, individuals with aphasia, who cannot judge sentence grammaticality, perform well on melody well-formedness judgments. Thus, the mechanisms that process structure in language do not appear to process music, including music syntax.

Non-literal language processing is jointly supported by the language and theory of mind networks: Evidence from a novel meta-analytic fMRI approach

Going beyond the literal meaning of language is key to communicative success. However, the mechanisms that support non-literal inferences remain debated. Using a novel meta-analytic approach, we evaluate the contribution of linguistic, social-cognitive, and executive mechanisms to non-literal interpretation. We identified 74 fMRI experiments (n = 1,430 participants) from 2001 to 2021 that contrasted non-literal language comprehension with a literal control condition, spanning ten phenomena (e.g., metaphor, irony, indirect speech). Applying the activation likelihood estimation approach to the 825 activation peaks yielded six left-lateralized clusters. We then evaluated the locations of both the individual-study peaks and the clusters against probabilistic functional atlases (cf. anatomical locations, as is typically done) for three candidate brain networks-the language-selective network (Fedorenko, Behr, & Kanwisher, 2011), which supports language processing, the Theory of Mind (ToM) network (Saxe & Kanwisher, 2003), which supports social inferences, and the domain-general Multiple-Demand (MD) network (Duncan, 2010), which supports executive control. These atlases were created by overlaying individual activation maps of participants who performed robust and extensively validated 'localizer' tasks that selectively target each network in question (n = 806 for language; n = 198 for ToM; n = 691 for MD). We found that both the individual-study peaks and the ALE clusters fell primarily within the language network and the ToM network. These results suggest that non-literal processing is supported by both i) mechanisms that process literal linguistic meaning, and ii) mechanisms that support general social inference. They thus undermine a strong divide between literal and non-literal aspects of language and challenge the claim that non-literal processing requires additional executive resources.

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production

A fronto-temporal brain network has long been implicated in language comprehension. However, this network's role in language production remains debated. In particular, it remains unclear whether all or only some language regions contribute to production, and which aspects of production these regions support. Across 3 functional magnetic resonance imaging experiments that rely on robust individual-subject analyses, we characterize the language network's response to high-level production demands. We report 3 novel results. First, sentence production, spoken or typed, elicits a strong response throughout the language network. Second, the language network responds to both phrase-structure building and lexical access demands, although the response to phrase-structure building is stronger and more spatially extensive, present in every language region. Finally, contra some proposals, we find no evidence of brain regions-within or outside the language network-that selectively support phrase-structure building in production relative to comprehension. Instead, all language regions respond more strongly during production than comprehension, suggesting that production incurs a greater cost for the language network. Together, these results align with the idea that language comprehension and production draw on the same knowledge representations, which are stored in a distributed manner within the language-selective network and are used to both interpret and generate linguistic utterances.

Dissociating Language and Thought in Human Reasoning

What is the relationship between language and complex thought? In the context of deductive reasoning there are two main views. Under the first, which we label here the language-centric view, language is central to the syntax-like combinatorial operations of complex reasoning. Under the second, which we label here the language-independent view, these operations are dissociable from the mechanisms of natural language. We applied continuous theta burst stimulation (cTBS), a form of noninvasive neuromodulation, to healthy adult participants to transiently inhibit a subregion of Broca's area (left BA44) associated in prior work with parsing the syntactic relations of natural language. We similarly inhibited a subregion of dorsomedial frontal cortex (left medial BA8) which has been associated with core features of logical reasoning. There was a significant interaction between task and stimulation site. Post hoc tests revealed that performance on a linguistic reasoning task, but not deductive reasoning task, was significantly impaired after inhibition of left BA44, and performance on a deductive reasoning task, but not linguistic reasoning task, was decreased after inhibition of left medial BA8 (however not significantly). Subsequent linear contrasts supported this pattern. These novel results suggest that deductive reasoning may be dissociable from linguistic processes in the adult human brain, consistent with the language-independent view.

Causal Contributions of the Domain-General (Multiple Demand) and the Language-Selective Brain Networks to Perceptual and Semantic Challenges in Speech Comprehension

Listening to spoken language engages domain-general multiple demand (MD; frontoparietal) regions of the human brain, in addition to domain-selective (frontotemporal) language regions, particularly when comprehension is challenging. However, there is limited evidence that the MD network makes a functional contribution to core aspects of understanding language. In a behavioural study of volunteers (n = 19) with chronic brain lesions, but without aphasia, we assessed the causal role of these networks in perceiving, comprehending, and adapting to spoken sentences made more challenging by acoustic-degradation or lexico-semantic ambiguity. We measured perception of and adaptation to acoustically degraded (noise-vocoded) sentences with a word report task before and after training. Participants with greater damage to MD but not language regions required more vocoder channels to achieve 50% word report, indicating impaired perception. Perception improved following training, reflecting adaptation to acoustic degradation, but adaptation was unrelated to lesion location or extent. Comprehension of spoken sentences with semantically ambiguous words was measured with a sentence coherence judgement task. Accuracy was high and unaffected by lesion location or extent. Adaptation to semantic ambiguity was measured in a subsequent word association task, which showed that availability of lower-frequency meanings of ambiguous words increased following their comprehension (word-meaning priming). Word-meaning priming was reduced for participants with greater damage to language but not MD regions. Language and MD networks make dissociable contributions to challenging speech comprehension: Using recent experience to update word meaning preferences depends on language-selective regions, whereas the domain-general MD network plays a causal role in reporting words from degraded speech.

Programming ability prediction: Applying an attention-based convolutional neural network to functional near-infrared spectroscopy analyses of working memory

Although theoretical studies have suggested that working-memory capacity is crucial for academic achievement, few empirical studies have directly investigated the relationship between working-memory capacity and programming ability, and no direct neural evidence has been reported to support this relationship. The present study aimed to fill this gap in the literature. Using a between-subject design, 17 programming novices and 18 advanced students performed an n-back working-memory task. During the experiment, their prefrontal hemodynamic responses were measured using a 48-channel functional near-infrared spectroscopy (fNIRS) device. The results indicated that the advanced students had a higher working-memory capacity than the novice students, validating the relationship between programming ability and working memory. The analysis results also showed that the hemodynamic responses in the prefrontal cortex can be used to discriminate between novices and advanced students. Additionally, we utilized an attention-based convolutional neural network to analyze the spatial domains of the fNIRS signals and demonstrated that the left prefrontal cortex was more important than other brain regions for programming ability prediction. This result was consistent with the results of statistical analysis, which in turn improved the interpretability of neural networks.

Structural neuroplasticity in computer programming beginners

We examined the structural neuroplastic changes associated with the learning of computer programming in university students with no previous programming experience. They participated in a 15-week course (26 lessons) on the “Processing” computer programming language. We have conducted a longitudinal analysis of gray matter volume (GMV) in the magnetic resonance images obtained before and after learning computer programming. Significant neuroplastic changes appeared in the following 8 sites: the left frontal pole; the right frontal pole; the right medial frontal gyrus; the left cuneus; the left lateral cerebellum (posterior lobule and tuber); the medial cerebellum (uvula and tonsil); the right pallidum; and the left pallidum. The amount of change in the GMV of the right frontal pole correlated positively with the final product score. Furthermore, the amount of change in the GMV of the right medial frontal gyrus and the bilateral pallidum correlated positively with the test scores. Thus, the right frontal pole was presumably associated with the function of persistent attempts to accomplish tasks (goal achievement-related function). The right medial frontal gyrus and the bilateral pallidum were presumably related to deduction and reward functions, respectively. Therefore, multiple brain regions appear to be involved in programming learning through different functions.

Probabilistic atlas for the language network based on precision fMRI data from >800 individuals

Two analytic traditions characterize fMRI language research. One relies on averaging activations across individuals. This approach has limitations: because of inter-individual variability in the locations of language areas, any given voxel/vertex in a common brain space is part of the language network in some individuals but in others, may belong to a distinct network. An alternative approach relies on identifying language areas in each individual using a functional 'localizer'. Because of its greater sensitivity, functional resolution, and interpretability, functional localization is gaining popularity, but it is not always feasible, and cannot be applied retroactively to past studies. To bridge these disjoint approaches, we created a probabilistic functional atlas using fMRI data for an extensively validated language localizer in 806 individuals. This atlas enables estimating the probability that any given location in a common space belongs to the language network, and thus can help interpret group-level activation peaks and lesion locations, or select voxels/electrodes for analysis. More meaningful comparisons of findings across studies should increase robustness and replicability in language research.

Differential Tracking of Linguistic vs. Mental State Content in Naturalistic Stimuli by Language and Theory of Mind (ToM) Brain Networks

Language and social cognition, especially the ability to reason about mental states, known as theory of mind (ToM), are deeply related in development and everyday use. However, whether these cognitive faculties rely on distinct, overlapping, or the same mechanisms remains debated. Some evidence suggests that, by adulthood, language and ToM draw on largely distinct-though plausibly interacting-cortical networks. However, the broad topography of these networks is similar, and some have emphasized the importance of social content / communicative intent in the linguistic signal for eliciting responses in the language areas. Here, we combine the power of individual-subject functional localization with the naturalistic-cognition inter-subject correlation approach to illuminate the language-ToM relationship. Using functional magnetic resonance imaging (fMRI), we recorded neural activity as participants (n = 43) listened to stories and dialogues with mental state content (+linguistic, +ToM), viewed silent animations and live action films with mental state content but no language (-linguistic, +ToM), or listened to an expository text (+linguistic, -ToM). The ToM network robustly tracked stimuli rich in mental state information regardless of whether mental states were conveyed linguistically or non-linguistically, while tracking a +linguistic / -ToM stimulus only weakly. In contrast, the language network tracked linguistic stimuli more strongly than (a) non-linguistic stimuli, and than (b) the ToM network, and showed reliable tracking even for the linguistic condition devoid of mental state content. These findings suggest that in spite of their indisputably close links, language and ToM dissociate robustly in their neural substrates-and thus plausibly cognitive mechanisms-including during the processing of rich naturalistic materials.

Music, Math, and Working Memory: Magnetoencephalography Mapping of Brain Activation in Musicians

Musical transposing is highly demanding of working memory, as it involves mentally converting notes from one musical key (i.e., pitch scale) to another key for singing or instrumental performance. Because musical transposing involves mental adjustment of notes up or down by a specific amount, it may share cognitive elements with arithmetical operations of addition and subtraction. We compared brain activity during high and low working memory load conditions of musical transposing versus math calculations in classically trained musicians. Magnetoencephalography (MEG) was sensitive to differences of task and working memory load. Frontal-occipital connections were highly active during transposing, but not during math calculations. Right motor and premotor regions were highly active in the more difficult condition of the transposing task. Multiple frontal lobe regions were highly active across tasks, including the left medial frontal area during both transposing and calculation tasks but the right medial frontal area only during calculations. In the more difficult calculation condition, right temporal regions were highly active. In coherence analyses and neural synchrony analyses, several similarities were seen across calculation tasks; however, latency analyses were sensitive to differences in task complexity across the calculation tasks due to the high temporal resolution of MEG. MEG can be used to examine musical cognition and the neural consequences of music training. Further systematic study of brain activity during high versus low memory load conditions of music and other cognitive tasks is needed to illuminate the neural bases of enhanced working memory ability in musicians as compared to non-musicians.

Software Bug Detection Causes a Shift From Bottom-Up to Top-Down Effective Connectivity Involving the Insula Within the Error-Monitoring Network

The neural correlates of software programming skills have been the target of an increasing number of studies in the past few years. Those studies focused on error-monitoring during software code inspection. Others have studied task-related cognitive load as measured by distinct neurophysiological measures. Most studies addressed only syntax errors (shallow level of code monitoring). However, a recent functional MRI (fMRI) study suggested a pivotal role of the insula during error-monitoring when challenging deep-level analysis of code inspection was required. This raised the hypothesis that the insula is causally involved in deep error-monitoring. To confirm this hypothesis, we carried out a new fMRI study where participants performed a deep source-code comprehension task that included error-monitoring to detect bugs in the code. The generality of our paradigm was enhanced by comparison with a variety of tasks related to text reading and bugless source-code understanding. Healthy adult programmers (N = 21) participated in this 3T fMRI experiment. The activation maps evoked by error-related events confirmed significant activations in the insula [p(Bonferroni) < 0.05]. Importantly, a posterior-to-anterior causality shift was observed concerning the role of the insula: in the absence of error, causal directions were mainly bottom-up, whereas, in their presence, the strong causal top-down effects from frontal regions, in particular, the anterior cingulate cortex was observed.

The Domain-General Multiple Demand Network Is More Active in Early Balanced Bilinguals Than Monolinguals During Executive Processing

The bilingual experience may place special cognitive demands on speakers and has been argued to lead to improvements in domain-general executive abilities, like cognitive control and working memory. Such improvements have been argued for based on both behavioral and brain imaging evidence. However, the empirical landscape is complex and ridden with controversy. Here we attempt to shed light on this question through an fMRI investigation of relatively large, relatively homogeneous, and carefully matched samples of early balanced bilinguals (n = 55) and monolinguals (n = 54), using robust, previously validated individual-level markers of neural activity in the domain-general multiple demand (MD) network, which supports executive functions. We find that the bilinguals, compared to the monolinguals, show significantly stronger neural responses to an executive (spatial working memory) task, and a larger difference between a harder and an easier condition of the task, across the MD network. These stronger neural responses are accompanied by better behavioral performance on the working memory task. We further show that the bilingual-vs.-monolingual difference in neural responses is not ubiquitous across the brain as no group difference in magnitude is observed in primary visual areas, which also respond to the task. Although the neural group difference in the MD network appears robust, it remains difficult to causally link it to bilingual experience specifically.

Reading and Calculation Neural Systems and Their Weighted Adaptive Use for Programming Skills

Software programming is a modern activity that poses strong challenges to the human brain. The neural mechanisms that support this novel cognitive faculty are still unknown. On the other hand, reading and calculation abilities represent slightly less recent human activities, in which neural correlates are relatively well understood. We hypothesize that calculus and reading brain networks provide joint underpinnings with distinctly weighted contributions which concern programming tasks, in particular concerning error identification. Based on a meta-analysis of the core regions involved in both reading and math and recent experimental evidence on the neural basis of programming tasks, we provide a theoretical account that integrates the role of these networks in program understanding. In this connectivity-based framework, error-monitoring processing regions in the frontal cortex influence the insula, which is a pivotal hub within the salience network, leading into efficient causal modulation of parietal networks involved in reading and mathematical operations. The core role of the anterior insula and anterior midcingulate cortex is illuminated by their relation to performance in error processing and novelty. The larger similarity that we observed between the networks underlying calculus and programming skills does not exclude a more limited but clear overlap with the reading network, albeit with differences in hemispheric lateralization when compared with prose reading. Future work should further elucidate whether other features of computer program understanding also use distinct weights of phylogenetically “older systems” for this recent human activity, based on the adjusting influence of fronto-insular networks. By unraveling the neural correlates of program understanding and bug detection, this work provides a framework to understand error monitoring in this novel complex faculty.

The Language Network Is Recruited but Not Required for Nonverbal Event Semantics

The ability to combine individual concepts of objects, properties, and actions into complex representations of the world is often associated with language. Yet combinatorial event-level representations can also be constructed from nonverbal input, such as visual scenes. Here, we test whether the language network in the human brain is involved in and necessary for semantic processing of events presented nonverbally. In Experiment 1, we scanned participants with fMRI while they performed a semantic plausibility judgment task versus a difficult perceptual control task on sentences and line drawings that describe/depict simple agent-patient interactions. We found that the language network responded robustly during the semantic task performed on both sentences and pictures (although its response to sentences was stronger). Thus, language regions in healthy adults are engaged during a semantic task performed on pictorial depictions of events. But is this engagement necessary? In Experiment 2, we tested two individuals with global aphasia, who have sustained massive damage to perisylvian language areas and display severe language difficulties, against a group of age-matched control participants. Individuals with aphasia were severely impaired on the task of matching sentences to pictures. However, they performed close to controls in assessing the plausibility of pictorial depictions of agent-patient interactions. Overall, our results indicate that the left frontotemporal language network is recruited but not necessary for semantic processing of nonverbally presented events.

Expert Programmers Have Fine-Tuned Cortical Representations of Source Code

Expertise enables humans to achieve outstanding performance on domain-specific tasks, and programming is no exception. Many studies have shown that expert programmers exhibit remarkable differences from novices in behavioral performance, knowledge structure, and selective attention. However, the underlying differences in the brain of programmers are still unclear. We here address this issue by associating the cortical representation of source code with individual programming expertise using a data-driven decoding approach. This approach enabled us to identify seven brain regions, widely distributed in the frontal, parietal, and temporal cortices, that have a tight relationship with programming expertise. In these brain regions, functional categories of source code could be decoded from brain activity and the decoding accuracies were significantly correlated with individual behavioral performances on a source-code categorization task. Our results suggest that programming expertise is built on fine-tuned cortical representations specialized for the domain of programming.

Comprehension of computer code relies primarily on domain-general executive brain regions

Computer programming is a novel cognitive tool that has transformed modern society. What cognitive and neural mechanisms support this skill? Here, we used functional magnetic resonance imaging to investigate two candidate brain systems: the multiple demand (MD) system, typically recruited during math, logic, problem solving, and executive tasks, and the language system, typically recruited during linguistic processing. We examined MD and language system responses to code written in Python, a text-based programming language (Experiment 1) and in ScratchJr, a graphical programming language (Experiment 2); for both, we contrasted responses to code problems with responses to content-matched sentence problems. We found that the MD system exhibited strong bilateral responses to code in both experiments, whereas the language system responded strongly to sentence problems, but weakly or not at all to code problems. Thus, the MD system supports the use of novel cognitive tools even when the input is structurally similar to natural language.