Fronto-parietal network supports context-dependent speech comprehension

Developmental dyslexia susceptibility genes DNAAF4, DCDC2, and NRSN1 are associated with brain function in fluently reading adolescents and young adults

Reading skills and developmental dyslexia, characterized by difficulties in developing reading skills, have been associated with brain anomalies within the language network. Genetic factors contribute to developmental dyslexia risk, but the mechanisms by which these genes influence reading skills remain unclear. In this preregistered study (https://osf.io/7sehx), we explored if developmental dyslexia susceptibility genes DNAAF4, DCDC2, NRSN1, and KIAA0319 are associated with brain function in fluently reading adolescents and young adults. Functional MRI and task performance data were collected during tasks involving written and spoken sentence processing, and DNA sequence variants of developmental dyslexia susceptibility genes previously associated with brain structure anomalies were genotyped. The results revealed that variation in DNAAF4, DCDC2, and NRSN1 is associated with brain activity in key language regions: the left inferior frontal gyrus, middle temporal gyrus, and intraparietal sulcus. Furthermore, NRSN1 was associated with task performance, but KIAA0319 did not yield any significant associations. Our findings suggest that individuals with a genetic predisposition to developmental dyslexia may partly employ compensatory neural and behavioral mechanisms to maintain typical task performance. Our study highlights the relevance of these developmental dyslexia susceptibility genes in language-related brain function, even in individuals without developmental dyslexia, providing valuable insights into the genetic factors influencing language processing.

Functional network connectivity patterns predicting the efficacy of repetitive transcranial magnetic stimulation in the spectrum of Alzheimer's disease

BACKGROUND

Neuro-navigated repetitive transcranial magnetic stimulation (rTMS) is potentially effective in enhancing cognitive performance in the spectrum of Alzheimer's disease (AD). We explored the effect of rTMS-induced network reorganization and its predictive value for individual treatment response.

METHODS

Sixty-two amnestic mild cognitive impairment (aMCI) and AD patients were recruited. These subjects were assigned to multimodal magnetic resonance imaging scanning before and after a 4-week stimulation. Then, we investigated the neural mechanism underlying rTMS treatment based on static functional network connectivity (sFNC) and dynamic functional network connectivity (dFNC) analyses. Finally, the support vector regression was used to predict the individual rTMS treatment response through these functional features at baseline.

RESULTS

We found that rTMS at the left angular gyrus significantly induced cognitive improvement in multiple cognitive domains. Participants after rTMS treatment exhibited significantly the increased sFNC between the right frontoparietal network (rFPN) and left frontoparietal network (lFPN) and decreased sFNC between posterior visual network and medial visual network. We revealed remarkable dFNC characteristics of brain connectivity, which was increased mainly in higher-order cognitive networks and decreased in primary networks or between primary networks and higher-order cognitive networks. dFNC characteristics in state 1 and state 4 could further predict individual higher memory improvement after rTMS treatment (state 1, R = 0.58; state 4, R = 0.54).

CONCLUSION

Our findings highlight that neuro-navigated rTMS could suppress primary network connections to compensate for higher-order cognitive networks. Crucially, dynamic regulation of brain networks at baseline may serve as an individualized predictor of rTMS treatment response.

RELEVANCE STATEMENT

Dynamic reorganization of brain networks could predict the efficacy of repetitive transcranial magnetic stimulation in the spectrum of Alzheimer's disease.

KEY POINTS

• rTMS at the left angular gyrus could induce cognitive improvement. • rTMS could suppress primary network connections to compensate for higher-order networks. • Dynamic reorganization of brain networks could predict individual treatment response to rTMS.

Weaving a story: Narrative formation over prolonged time scales engages social cognition and frontoparietal networks

Forming narratives is of key importance to human experience, enabling one to render large amounts of information into relatively compacted stories for future retrieval, giving meaning to otherwise fragmented occurrences. The neural mechanisms that underlie coherent narrative construction of causally connected information over prolonged temporal periods are yet unclear. Participants in this fMRI study observed consecutive scenes from a full-length movie either in their original order, enabling causal inferences over time, or in reverse order, impeding a key component of coherent narratives-causal inference. In between scenes, we presented short periods of blank screens for examining post-encoding processing effects. Using multivariate pattern analysis (MVPA) followed by seed-base correlation analysis, we hypothesized that networks involved in online monitoring of incoming information on the one hand, and offline processing of previous occurrences on the other would differ between the groups. We found that despite the exposure to the same scenes, the chronological-order condition exhibited enhanced functional connectivity in frontoparietal regions associated with information integration and working memory. The reverse-order condition yielded offline, post-scene coactivation of neural networks involved in social cognition and particularly theory of mind and action comprehension. These findings shed light on offline processes of narrative construction efforts, highlighting the role of social cognition networks in seeking for narrative coherence.

Different kinds of simulation during literary reading: Insights from a combined fMRI and eye-tracking study

Mental simulation is an important aspect of narrative reading. In a previous study, we found that gaze durations are differentially impacted by different kinds of mental simulation. Motor simulation, perceptual simulation, and mentalizing as elicited by literary short stories influenced eye movements in distinguishable ways (Mak & Willems, 2019). In the current study, we investigated the existence of a common neural locus for these different kinds of simulation. We additionally investigated whether individual differences during reading, as indexed by the eye movements, are reflected in domain-specific activations in the brain. We found a variety of brain areas activated by simulation-eliciting content, both modality-specific brain areas and a general simulation area. Individual variation in percent signal change in activated areas was related to measures of story appreciation as well as personal characteristics (i.e., transportability, perspective taking). Taken together, these findings suggest that mental simulation is supported by both domain-specific processes grounded in previous experiences, and by the neural mechanisms that underlie higher-order language processing (e.g., situation model building, event indexing, integration).

Individual theta-band cortical entrainment to speech in quiet predicts word-in-noise comprehension

Speech elicits brain activity time-locked to its amplitude envelope. The resulting speech-brain synchrony (SBS) is thought to be crucial to speech parsing and comprehension. It has been shown that higher speech-brain coherence is associated with increased speech intelligibility. However, studies depending on the experimental manipulation of speech stimuli do not allow conclusion about the causality of the observed tracking. Here, we investigate whether individual differences in the intrinsic propensity to track the speech envelope when listening to speech-in-quiet is predictive of individual differences in speech-recognition-in-noise, in an independent task. We evaluated the cerebral tracking of speech in source-localized magnetoencephalography, at timescales corresponding to the phrases, words, syllables and phonemes. We found that individual differences in syllabic tracking in right superior temporal gyrus and in left middle temporal gyrus (MTG) were positively associated with recognition accuracy in an independent words-in-noise task. Furthermore, directed connectivity analysis showed that this relationship is partially mediated by top-down connectivity from premotor cortex-associated with speech processing and active sensing in the auditory domain-to left MTG. Thus, the extent of SBS-even during clear speech-reflects an active mechanism of the speech processing system that may confer resilience to noise.

Objective speech intelligibility prediction using a deep learning model with continuous speech-evoked cortical auditory responses

Auditory prostheses provide an opportunity for rehabilitation of hearing-impaired patients. Speech intelligibility can be used to estimate the extent to which the auditory prosthesis improves the user’s speech comprehension. Although behavior-based speech intelligibility is the gold standard, precise evaluation is limited due to its subjectiveness. Here, we used a convolutional neural network to predict speech intelligibility from electroencephalography (EEG). Sixty-four–channel EEGs were recorded from 87 adult participants with normal hearing. Sentences spectrally degraded by a 2-, 3-, 4-, 5-, and 8-channel vocoder were used to set relatively low speech intelligibility conditions. A Korean sentence recognition test was used. The speech intelligibility scores were divided into 41 discrete levels ranging from 0 to 100%, with a step of 2.5%. Three scores, namely 30.0, 37.5, and 40.0%, were not collected. The speech features, i.e., the speech temporal envelope (ENV) and phoneme (PH) onset, were used to extract continuous-speech EEGs for speech intelligibility prediction. The deep learning model was trained by a dataset of event-related potentials (ERP), correlation coefficients between the ERPs and ENVs, between the ERPs and PH onset, or between ERPs and the product of the multiplication of PH and ENV (PHENV). The speech intelligibility prediction accuracies were 97.33% (ERP), 99.42% (ENV), 99.55% (PH), and 99.91% (PHENV). The models were interpreted using the occlusion sensitivity approach. While the ENV models’ informative electrodes were located in the occipital area, the informative electrodes of the phoneme models, i.e., PH and PHENV, were based on the occlusion sensitivity map located in the language processing area. Of the models tested, the PHENV model obtained the best speech intelligibility prediction accuracy. This model may promote clinical prediction of speech intelligibility with a comfort speech intelligibility test.

Infant Formula Supplemented With Milk Fat Globule Membrane, Long-Chain Polyunsaturated Fatty Acids, and Synbiotics Is Associated With Neurocognitive Function and Brain Structure of Healthy Children Aged 6 Years: The COGNIS Study

Background

Adequate nutrient intake during the first few months of life plays a critical role on brain structure and function development.

Objectives

To analyze the long-term effects of an experimental infant formula (EF) on neurocognitive function and brain structure in healthy children aged 6 years compared to those fed with a standard infant formula or breastfed.

Methods

The current study involved 108 healthy children aged 6 years and participating in the COGNIS Study. At 0-2 months, infants were randomized to receive up to 18 months of life a standard infant formula (SF) or EF enriched with milk fat globule membrane (MFGM), long-chain polyunsaturated fatty acids (LC-PUFAs) and synbiotics. Furthermore, a reference group of breastfed (BF) infants were also recruited. Children were assessed using neurocognitive tests and structural Magnetic Resonance Imaging (MRI) at 6 years old.

Results

Experimental infant formula (EF) children showed greater volumes in the left orbital cortex, higher vocabulary scores and IQ, and better performance in an attention task than BF children. EF children also presented greater volumes in parietal regions than SF kids. Additionally, greater cortical thickness in the insular, parietal, and temporal areas were found in children from the EF group than those fed with SF or BF groups. Further correlation analyses suggest that higher volumes and cortical thickness of different parietal and frontal regions are associated with better cognitive development in terms of language (verbal comprehension) and executive function (working memory). Finally, arachidonic acid (ARA), adrenic acid (AdA), docosahexaenoic acid (DHA) levels in cheek cell glycerophospholipids, ARA/DHA ratio, and protein, fatty acid, and mineral intake during the first 18 months of life seem to be associated with changes in the brain structures at 6 years old.

Conclusions

Supplemented infant formula with MFGM components, LC-PUFAs, and synbiotics seems to be associated to long-term effects on neurocognitive development and brain structure in children at 6 years old.

Clinical Trial Registration

https://www.clinicaltrials.gov/, identifier: NCT02094547.

Behavioral Experience-Sampling Methods in Neuroimaging Studies With Movie and Narrative Stimuli

Movies and narratives are increasingly utilized as stimuli in functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and electroencephalography (EEG) studies. Emotional reactions of subjects, what they pay attention to, what they memorize, and their cognitive interpretations are all examples of inner experiences that can differ between subjects during watching of movies and listening to narratives inside the scanner. Here, we review literature indicating that behavioral measures of inner experiences play an integral role in this new research paradigm via guiding neuroimaging analysis. We review behavioral methods that have been developed to sample inner experiences during watching of movies and listening to narratives. We also review approaches that allow for joint analyses of the behaviorally sampled inner experiences and neuroimaging data. We suggest that building neurophenomenological frameworks holds potential for solving the interrelationships between inner experiences and their neural underpinnings. Finally, we tentatively suggest that recent developments in machine learning approaches may pave way for inferring different classes of inner experiences directly from the neuroimaging data, thus potentially complementing the behavioral self-reports.

Linguistic processing of task-irrelevant speech at a cocktail party

Paying attention to one speaker in a noisy place can be extremely difficult, because to-be-attended and task-irrelevant speech compete for processing resources. We tested whether this competition is restricted to acoustic-phonetic interference or if it extends to competition for linguistic processing as well. Neural activity was recorded using Magnetoencephalography as human participants were instructed to attend to natural speech presented to one ear, and task-irrelevant stimuli were presented to the other. Task-irrelevant stimuli consisted either of random sequences of syllables, or syllables structured to form coherent sentences, using hierarchical frequency-tagging. We find that the phrasal structure of structured task-irrelevant stimuli was represented in the neural response in left inferior frontal and posterior parietal regions, indicating that selective attention does not fully eliminate linguistic processing of task-irrelevant speech. Additionally, neural tracking of to-be-attended speech in left inferior frontal regions was enhanced when competing with structured task-irrelevant stimuli, suggesting inherent competition between them for linguistic processing.

More Than Words: Extra-Sylvian Neuroanatomic Networks Support Indirect Speech Act Comprehension and Discourse in Behavioral Variant Frontotemporal Dementia

Indirect speech acts—responding “I forgot to wear my watch today” to someone who asked for the time—are ubiquitous in daily conversation, but are understudied in current neurobiological models of language. To comprehend an indirect speech act like this one, listeners must not only decode the lexical-semantic content of the utterance, but also make a pragmatic, bridging inference. This inference allows listeners to derive the speaker’s true, intended meaning—in the above dialog, for example, that the speaker cannot provide the time. In the present work, we address this major gap by asking non-aphasic patients with behavioral variant frontotemporal dementia (bvFTD, n = 21) and brain-damaged controls with amnestic mild cognitive impairment (MCI, n = 17) to judge simple question-answer dialogs of the form: “Do you want some cake for dessert?” “I’m on a very strict diet right now,” and relate the results to structural and diffusion MRI. Accuracy and reaction time results demonstrate that subjects with bvFTD, but not MCI, are selectively impaired in indirect relative to direct speech act comprehension, due in part to their social and executive limitations, and performance is related to caregivers’ judgment of communication efficacy. MRI imaging associates the observed impairment in bvFTD to cortical thinning not only in traditional language-associated regions, but also in fronto-parietal regions implicated in social and executive cerebral networks. Finally, diffusion tensor imaging analyses implicate white matter tracts in both dorsal and ventral projection streams, including superior longitudinal fasciculus, frontal aslant, and uncinate fasciculus. These results have strong implications for updated neurobiological models of language, and emphasize a core, language-mediated social disorder in patients with bvFTD.

Movies and narratives as naturalistic stimuli in neuroimaging

Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.

Breaking down the cocktail party: Attentional modulation of cerebral audiovisual speech processing

Recent studies utilizing electrophysiological speech envelope reconstruction have sparked renewed interest in the cocktail party effect by showing that auditory neurons entrain to selectively attended speech. Yet, the neural networks of attention to speech in naturalistic audiovisual settings with multiple sound sources remain poorly understood. We collected functional brain imaging data while participants viewed audiovisual video clips of lifelike dialogues with concurrent distracting speech in the background. Dialogues were presented in a full-factorial design, comprising task (listen to the dialogues vs. ignore them), audiovisual quality and semantic predictability. We used univariate analyses in combination with multivariate pattern analysis (MVPA) to study modulations of brain activity related to attentive processing of audiovisual speech. We found attentive speech processing to cause distinct spatiotemporal modulation profiles in distributed cortical areas including sensory and frontal-control networks. Semantic coherence modulated attention-related activation patterns in the earliest stages of auditory cortical processing, suggesting that the auditory cortex is involved in high-level speech processing. Our results corroborate views that emphasize the dynamic nature of attention, with task-specificity and context as cornerstones of the underlying neuro-cognitive mechanisms.

No title, no theme: The joined neural space between speakers and listeners during production and comprehension of multi-sentence discourse

Speakers and listeners usually interact in larger discourses than single words or even single sentences. The goal of the present study was to identify the neural bases reflecting how the mental representation of the situation denoted in a multi-sentence discourse (situation model) is constructed and shared between speakers and listeners. An fMRI study using a variant of the ambiguous text paradigm was designed. Speakers (n = 15) produced ambiguous texts in the scanner and listeners (n = 27) subsequently listened to these texts in different states of ambiguity: preceded by a highly informative, intermediately informative or no title at all. Conventional BOLD activation analyses in listeners, as well as inter-subject correlation analyses between the speakers' and the listeners' hemodynamic time courses were performed. Critically, only the processing of disambiguated, coherent discourse with an intelligible situation model representation involved (shared) activation in bilateral lateral parietal and medial prefrontal regions. This shared spatiotemporal pattern of brain activation between the speaker and the listener suggests that the process of memory retrieval in medial prefrontal regions and the binding of retrieved information in the lateral parietal cortex constitutes a core mechanism underlying the communication of complex conceptual representations.

Predictive Impact of Contextual Objects during Action Observation: Evidence from Functional Magnetic Resonance Imaging

The processing of congruent stimuli, such as an object or action in its typical location, is usually associated with reduced neural activity, probably due to facilitated recognition. However, in some situations, congruency increases neural activity—for example, when objects next to observed actions are likely versus unlikely to be involved in forthcoming action steps. Here, we investigated using fMRI whether the processing of contextual cues during action perception is driven by their (in)congruency and, thus, informative value to make sense of an observed scene. Specifically, we tested whether both highly congruent contextual objects (COs), which strongly indicate a future action step, and highly incongruent COs, which require updating predictions about possible forthcoming action steps, provide more anticipatory information about the action course than moderately congruent COs. In line with our hypothesis that especially the inferior frontal gyrus (IFG) subserves the integration of the additional information into the predictive model of the action, we found highly congruent and incongruent COs to increase bilateral activity in action observation nodes, that is, the IFG, the occipitotemporal cortex, and the intraparietal sulcus. Intriguingly, BA 47 was significantly stronger engaged for incongruent COs reflecting the updating of prediction in response to conflicting information. Our findings imply that the IFG reflects the informative impact of COs on observed actions by using contextual information to supply and update the currently operating predictive model. In the case of an incongruent CO, this model has to be reconsidered and extended toward a new overarching action goal.

Mapping language with resting-state functional magnetic resonance imaging: A study on the functional profile of the language network

Resting-state functional magnetic resonance imaging (rsfMRI) is a promising technique for language mapping that does not require task-execution. This can be an advantage when language mapping is limited by poor task performance, as is common in clinical settings. Previous studies have shown that language maps extracted with rsfMRI spatially match their task-based homologs, but no study has yet demonstrated the direct participation of the rsfMRI language network in language processes. This demonstration is critically important because spatial similarity can be influenced by the overlap of domain-general regions that are recruited during task-execution. Furthermore, it is unclear which processes are captured by the language network: does it map rather low-level or high-level (e.g., syntactic and lexico-semantic) language processes? We first identified the rsfMRI language network and then investigated task-based responses within its regions when processing stimuli of increasing linguistic content: symbols, pseudowords, words, pseudosentences and sentences. The language network responded only to language stimuli (not to symbols), and higher linguistic content elicited larger brain responses. The left fronto-parietal, the default mode, and the dorsal attention networks were examined and yet none showed language involvement. These findings demonstrate for the first time that the language network extracted through rsfMRI is able to map language in the brain, including regions subtending higher-level syntactic and semantic processes.

Emotions amplify speaker-listener neural alignment

Individuals often align their emotional states during conversation. Here, we reveal how such emotional alignment is reflected in synchronization of brain activity across speakers and listeners. Two "speaker" subjects told emotional and neutral autobiographical stories while their hemodynamic brain activity was measured with functional magnetic resonance imaging (fMRI). The stories were recorded and played back to 16 "listener" subjects during fMRI. After scanning, both speakers and listeners rated the moment-to-moment valence and arousal of the stories. Time-varying similarity of the blood-oxygenation-level-dependent (BOLD) time series was quantified by intersubject phase synchronization (ISPS) between speaker-listener pairs. Telling and listening to the stories elicited similar emotions across speaker-listener pairs. Arousal was associated with increased speaker-listener neural synchronization in brain regions supporting attentional, auditory, somatosensory, and motor processing. Valence was associated with increased speaker-listener neural synchronization in brain regions involved in emotional processing, including amygdala, hippocampus, and temporal pole. Speaker-listener synchronization of subjective feelings of arousal was associated with increased neural synchronization in somatosensory and subcortical brain regions; synchronization of valence was associated with neural synchronization in parietal cortices and midline structures. We propose that emotion-dependent speaker-listener neural synchronization is associated with emotional contagion, thereby implying that listeners reproduce some aspects of the speaker's emotional state at the neural level.

Pragmatic abilities in multiple sclerosis: The contribution of the temporo-parietal junction

Recent studies showed that multiple sclerosis (MS) patients might experience communicative deficits, specifically in pragmatics (i.e., the ability to integrate the context-dependent aspects of language). A crucial region for pragmatics is the temporo-parietal junction, in particular the so-called Geschwind's area (GA), which is involved in high-level language processes, including the comprehension of narratives, metaphor, and irony. We evaluated the relationship between pragmatic abilities, measured through the Assessment of Pragmatic Abilities and Cognitive Substrates (APACS) test, and the functional connectivity (FC) of the bilateral GAs, assessed through a seed-based analysis of Resting-State fMRI in patients with MS. A positive correlation was observed between APACS scores and the FC for both the right and the left GA and the paracingulate cortex. Our findings suggest that the brain FC for social communication involves connections extending over both hemispheres, including right and left GAs and right and left paracingulate cortex, possibly impaired in patients with MS. This study offers preliminary evidence for future researches enrolling also a control sample to explore the involvement of GA in pragmatics in neurological disorders as well as in healthy conditions.

Grounding the neurobiology of language in first principles: The necessity of non-language-centric explanations for language comprehension

Recent decades have ushered in tremendous progress in understanding the neural basis of language. Most of our current knowledge on language and the brain, however, is derived from lab-based experiments that are far removed from everyday language use, and that are inspired by questions originating in linguistic and psycholinguistic contexts. In this paper we argue that in order to make progress, the field needs to shift its focus to understanding the neurobiology of naturalistic language comprehension. We present here a new conceptual framework for understanding the neurobiological organization of language comprehension. This framework is non-language-centered in the computational/neurobiological constructs it identifies, and focuses strongly on context. Our core arguments address three general issues: (i) the difficulty in extending language-centric explanations to discourse; (ii) the necessity of taking context as a serious topic of study, modeling it formally and acknowledging the limitations on external validity when studying language comprehension outside context; and (iii) the tenuous status of the language network as an explanatory construct. We argue that adopting this framework means that neurobiological studies of language will be less focused on identifying correlations between brain activity patterns and mechanisms postulated by psycholinguistic theories. Instead, they will be less self-referential and increasingly more inclined towards integration of language with other cognitive systems, ultimately doing more justice to the neurobiological organization of language and how it supports language as it is used in everyday life.

Sharing the social world via intersubject neural synchronisation

Sociability and capability of shared mental states are hallmarks of the human species, and pursuing shared goals oftentimes requires coordinating both behaviour and mental states. Here we review recent work using indices of intersubject neural synchronisation for measuring similarity of mental states across individuals. We discuss the methodological advances and limitations in the analyses based on intersubject synchrony, and discuss how these kinds of model-free analysis techniques enable the investigation of the brain basis of complex social processes. We argue that similarity of brain activity across individuals can be used, under certain conditions, to index the similarity of their subjective states of consciousness, and thus be used for investigating brain basis of mutual understanding and cooperation.

Predicting "When" in Discourse Engages the Human Dorsal Auditory Stream: An fMRI Study Using Naturalistic Stories

The hierarchical organization of human cortical circuits integrates information across different timescales via temporal receptive windows, which increase in length from lower to higher levels of the cortical hierarchy (Hasson et al., 2015). A recent neurobiological model of higher-order language processing (Bornkessel-Schlesewsky et al., 2015) posits that temporal receptive windows in the dorsal auditory stream provide the basis for a hierarchically organized predictive coding architecture (Friston and Kiebel, 2009). In this stream, a nested set of internal models generates time-based (“when”) predictions for upcoming input at different linguistic levels (sounds, words, sentences, discourse). Here, we used naturalistic stories to test the hypothesis that multi-sentence, discourse-level predictions are processed in the dorsal auditory stream, yielding attenuated BOLD responses for highly predicted versus less strongly predicted language input. The results were as hypothesized: discourse-related cues, such as passive voice, which effect a higher predictability of remention for a character at a later point within a story, led to attenuated BOLD responses for auditory input of high versus low predictability within the dorsal auditory stream, specifically in the inferior parietal lobule, middle frontal gyrus, and dorsal parts of the inferior frontal gyrus, among other areas. Additionally, we found effects of content-related (“what”) predictions in ventral regions. These findings provide novel evidence that hierarchical predictive coding extends to discourse-level processing in natural language. Importantly, they ground language processing on a hierarchically organized predictive network, as a common underlying neurobiological basis shared with other brain functions.

SIGNIFICANCE STATEMENT Language is the most powerful communicative medium available to humans. Nevertheless, we lack an understanding of the neurobiological basis of language processing in natural contexts: it is not clear how the human brain processes linguistic input within the rich contextual environments of our everyday language experience. This fMRI study provides the first demonstration that, in natural stories, predictions concerning the probability of remention of a protagonist at a later point are processed in the dorsal auditory stream. Results are congruent with a hierarchical predictive coding architecture assuming temporal receptive windows of increasing length from auditory to higher-order cortices. Accordingly, language processing in rich contextual settings can be explained via domain-general, neurobiological mechanisms of information processing in the human brain.

Neural mechanisms for integrating consecutive and interleaved natural events

To understand temporally extended events, the human brain needs to accumulate information continuously across time. Interruptions that require switching of attention to other event sequences disrupt this process. To reveal neural mechanisms supporting integration of event information, we measured brain activity with functional magnetic resonance imaging (fMRI) from 18 participants while they viewed 6.5-minute excerpts from three movies (i) consecutively and (ii) as interleaved segments of approximately 50-s in duration. We measured inter-subject reliability of brain activity by calculating inter-subject correlations (ISC) of fMRI signals and analyzed activation timecourses with a general linear model (GLM). Interleaving decreased the ISC in posterior temporal lobes, medial prefrontal cortex, superior precuneus, medial occipital cortex, and cerebellum. In the GLM analyses, posterior temporal lobes were activated more consistently by instances of speech when the movies were viewed consecutively than as interleaved segments. By contrast, low-level auditory and visual stimulus features and editing boundaries caused similar activity patterns in both conditions. In the medial occipital cortex, decreases in ISC were seen in short bursts throughout the movie clips. By contrast, the other areas showed longer-lasting differences in ISC during isolated scenes depicting socially-relevant and suspenseful content, such as deception or inter-subject conflict. The areas in the posterior temporal lobes also showed sustained activity during continuous actions and were deactivated when actions ended at scene boundaries. Our results suggest that the posterior temporal and dorsomedial prefrontal cortices, as well as the cerebellum and dorsal precuneus, support integration of events into coherent event sequences. Hum Brain Mapp 38:3360-3376, 2017. © 2017 Wiley Periodicals, Inc.

Context matters: Anterior and posterior cortical midline responses to sad movie scenes

Narrative movies can create powerful emotional responses. While recent research has advanced the understanding of neural networks involved in immersive movie viewing, their modulation within a movie's dynamic context remains inconclusive. In this study, 24 healthy participants passively watched sad scene climaxes taken from 24 romantic comedies, while brain activity was measured using functional magnetic resonance (fMRI). To study effects of context, the sad scene climaxes were presented with either coherent scene context, replaced non-coherent context or without context. In a second viewing, the same clips were rated continuously for sadness. The ratings varied over time with peaks of experienced sadness within the assumed climax intervals. Activations in anterior and posterior cortical midline regions increased if presented with both coherent and replaced context, while activation in the temporal gyri decreased. This difference was more pronounced for the coherent context condition. Psycho-Physiological interactions (PPI) analyses showed a context-dependent coupling of midline regions with occipital visual and sub-cortical reward regions. Our results demonstrate the pivotal role of midline structures and their interaction with perceptual and reward areas in processing contextually embedded socio-emotional information in movies.

Metonymy as Referential Dependency: Psycholinguistic and Neurolinguistic Arguments for a Unified Linguistic Treatment

We examine metonymy at psycho- and neurolinguistic levels, seeking to adjudicate between two possible processing implementations (one- vs. two-mechanism). We compare highly conventionalized systematic metonymy (producer-for-product: "All freshmen read O'Connell") to lesser-conventionalized circumstantial metonymy ("[a waitress says to another:] 'Table 2 asked for more coffee."'). Whereas these two metonymy types differ in terms of contextual demands, they each reveal a similar dependency between the named and intended conceptual entities (e.g., Jackendoff, 1997; Nunberg, 1979, 1995). We reason that if each metonymy yields a distinct processing time course and substantially non-overlapping preferential localization pattern, it would not only support a two-mechanism view (one lexical, one pragmatic) but would suggest that conventionalization acts as a linguistic categorizer. By contrast, a similar behavior in time course and localization would support a one-mechanism view and the inference that conventionalization acts instead as a modulator of contextual felicitousness, and that differences in interpretation introduced by conventionalization are of degree, not of kind. Results from three paradigms: self-paced reading (SPR), event-related potentials (ERP), and functional magnetic resonance imaging (fMRI), reveal the following: no main effect by condition (metonymy vs. matched literal control) for either metonymy type immediately after the metonymy trigger, and a main effect for only the Circumstantial metonymy one word post-trigger (SPR); a N400 effect across metonymy types and a late positivity for Circumstantial metonymy (ERP); and a highly overlapping activation connecting the left ventrolateral prefrontal cortex and the left dorsolateral prefrontal cortex (fMRI). Altogether, the pattern observed does not reach the threshold required to justify a two-mechanism system. Instead, the pattern is more naturally (and conservatively) understood as resulting from the implementation of a generalized referential dependency mechanism, modulated by degree of context dependence/conventionalization, thus supporting architectures of language whereby "lexical" and "pragmatic" meaning relations are encoded along a cline of contextual underspecification.

Comprehension through explanation as the interaction of the brain's coherence and cognitive control networks

Discourse comprehension processes attempt to produce an elaborate and well-connected representation in the reader’s mind. A common network of regions including the angular gyrus, posterior cingulate, and dorsal frontal cortex appears to be involved in constructing coherent representations in a variety of tasks including social cognition tasks, narrative comprehension, and expository text comprehension. Reading strategies that require the construction of explicit inferences are used in the present research to examine how this coherence network interacts with other brain regions. A psychophysiological interaction analysis was used to examine regions showing changed functional connectivity with this coherence network when participants were engaged in either a non-inferencing reading strategy, paraphrasing, or a strategy requiring coherence-building inferences, self-explanation. Results of the analysis show that the coherence network increases in functional connectivity with a cognitive control network that may be specialized for the manipulation of semantic representations and the construction of new relations among these representations.