Semantics-weighted lexical surprisal modeling of naturalistic functional MRI time-series during spoken narrative listening

The organization of the semantic network as reflected by the neural correlates of six semantic dimensions

Multiple sensory-motor and non-sensory-motor dimensions have been proposed for semantic representation, but it remains unclear how the semantic system is organized along them in the human brain. Using naturalistic fMRI data and large-scale semantic ratings, we investigated the overlaps and dissociations between the neural correlates of six semantic dimensions: vision, motor, socialness, emotion, space, and time. Our findings revealed a more complex semantic atlas than what is predicted by current neurobiological models of semantic representation. Brain regions that are selectively sensitive to specific semantic dimensions were found both within and outside the brain networks assumed to represent multimodal general and/or abstract semantics. Overlaps between the neural correlates of different semantic dimensions were mainly found inside the default mode network, concentrated in the left anterior superior temporal gyrus and angular gyrus, which have been proposed as two connector hubs that bridge the multimodal experiential semantic system and the language-supported semantic system.

Will you read how I will read? Naturalistic fMRI predictors of emergent reading

Despite reading being an essential and almost universal skill in the developed world, reading proficiency varies substantially from person to person. To study why, the fMRI field is beginning to turn from single-word or nonword reading tasks to naturalistic stimuli like connected text and listening to stories. To study reading development in children just beginning to read, listening to stories is an appropriate paradigm because speech perception and phonological processing are important for, and are predictors of, reading proficiency. Our study examined the relationship between behavioral reading-related skills and the neural response to listening to stories in the fMRI environment. Functional MRI were gathered in a 3T TIM-Trio scanner. During the fMRI scan, children aged approximately 7 years listened to professionally narrated common short stories and answered comprehension questions following the narration. Analyses of the data used inter-subject correlation (ISC), and representational similarity analysis (RSA). Our primary finding is that ISC reveals areas of increased synchrony in both high- and low-performing emergent readers previously implicated in reading ability/disability. Of particular interest are that several previously identified brain regions (medial temporal gyrus (MTG), inferior frontal gyrus (IFG), inferior temporal gyrus (ITG)) were found to "synchronize" across higher reading ability participants, while lower reading ability participants had idiosyncratic activation patterns in these regions. Additionally, two regions (superior frontal gyrus (SFG) and another portion of ITG) were recruited by all participants, but their specific timecourse of activation depended on reading performance. These analyses support the idea that different brain regions involved in reading follow different developmental trajectories that correlate with reading proficiency on a spectrum rather than the usual dichotomy of poor readers versus strong readers.

Listening Effort in Tinnitus: A Pilot Study Employing a Light EEG Headset and Skin Conductance Assessment during the Listening to a Continuous Speech Stimulus under Different SNR Conditions

Background noise elicits listening effort. What else is tinnitus if not an endogenous background noise? From such reasoning, we hypothesized the occurrence of increased listening effort in tinnitus patients during listening tasks. Such a hypothesis was tested by investigating some indices of listening effort through electroencephalographic and skin conductance, particularly parietal and frontal alpha and electrodermal activity (EDA). Furthermore, tinnitus distress questionnaires (THI and TQ12-I) were employed. Parietal alpha values were positively correlated to TQ12-I scores, and both were negatively correlated to EDA; Pre-stimulus frontal alpha correlated with the THI score in our pilot study; finally, results showed a general trend of increased frontal alpha activity in the tinnitus group in comparison to the control group. Parietal alpha during the listening to stimuli, positively correlated to the TQ12-I, appears to reflect a higher listening effort in tinnitus patients and the perception of tinnitus symptoms. The negative correlation between both listening effort (parietal alpha) and tinnitus symptoms perception (TQ12-I scores) with EDA levels could be explained by a less responsive sympathetic nervous system to prepare the body to expend increased energy during the "fight or flight" response, due to pauperization of energy from tinnitus perception.

Explaining neural activity in human listeners with deep learning via natural language processing of narrative text

Deep learning (DL) approaches may also inform the analysis of human brain activity. Here, a state-of-art DL tool for natural language processing, the Generative Pre-trained Transformer version 2 (GPT-2), is shown to generate meaningful neural encodings in functional MRI during narrative listening. Linguistic features of word unpredictability (surprisal) and contextual importance (saliency) were derived from the GPT-2 applied to the text of a 12-min narrative. Segments of variable duration (from 15 to 90 s) defined the context for the next word, resulting in different sets of neural predictors for functional MRI signals recorded in 27 healthy listeners of the narrative. GPT-2 surprisal, estimating word prediction errors from the artificial network, significantly explained the neural data in superior and middle temporal gyri (bilaterally), in anterior and posterior cingulate cortices, and in the left prefrontal cortex. GPT-2 saliency, weighing the importance of context words, significantly explained the neural data for longer segments in left superior and middle temporal gyri. These results add novel support to the use of DL tools in the search for neural encodings in functional MRI. A DL language model like the GPT-2 may feature useful data about neural processes subserving language comprehension in humans, including next-word context-related prediction.

Auditory cortex hypoperfusion: a metabolic hallmark in Beta Thalassemia

Background

Sensorineural hearing loss in beta-thalassemia is common and it is generally associated with iron chelation therapy. However, data are scarce, especially on adult populations, and a possible involvement of the central auditory areas has not been investigated yet. We performed a multicenter cross-sectional audiological and single-center 3Tesla brain perfusion MRI study enrolling 77 transfusion-dependent/non transfusion-dependent adult patients and 56 healthy controls. Pure tone audiometry, demographics, clinical/laboratory and cognitive functioning data were recorded.

Results

Half of patients (52%) presented with high-frequency hearing deficit, with overt hypoacusia (Pure Tone Average (PTA) > 25 dB) in 35%, irrespective of iron chelation or clinical phenotype. Bilateral voxel clusters of significant relative hypoperfusion were found in the auditory cortex of beta-thalassemia patients, regardless of clinical phenotype. In controls and transfusion-dependent (but not in non-transfusion-dependent) patients, the relative auditory cortex perfusion values increased linearly with age (p < 0.04). Relative auditory cortex perfusion values showed a significant U-shaped correlation with PTA values among hearing loss patients, and a linear correlation with the full scale intelligence quotient (right side p = 0.01, left side p = 0.02) with its domain related to communication skills (right side p = 0.04, left side p = 0.07) in controls but not in beta-thalassemia patients. Audiometric test results did not correlate to cognitive test scores in any subgroup.

Conclusions

In conclusion, primary auditory cortex perfusion changes are a metabolic hallmark of adult beta-thalassemia, thus suggesting complex remodeling of the hearing function, that occurs regardless of chelation therapy and before clinically manifest hearing loss. The cognitive impact of perfusion changes is intriguing but requires further investigations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13023-021-01969-0.

Cortical Tracking of Speech: Toward Collaboration between the Fields of Signal and Sentence Processing

In recent years, a growing number of studies have used cortical tracking methods to investigate auditory language processing. Although most studies that employ cortical tracking stem from the field of auditory signal processing, this approach should also be of interest to psycholinguistics-particularly the subfield of sentence processing-given its potential to provide insight into dynamic language comprehension processes. However, there has been limited collaboration between these fields, which we suggest is partly because of differences in theoretical background and methodological constraints, some mutually exclusive. In this paper, we first review the theories and methodological constraints that have historically been prioritized in each field and provide concrete examples of how some of these constraints may be reconciled. We then elaborate on how further collaboration between the two fields could be mutually beneficial. Specifically, we argue that the use of cortical tracking methods may help resolve long-standing debates in the field of sentence processing that commonly used behavioral and neural measures (e.g., ERPs) have failed to adjudicate. Similarly, signal processing researchers who use cortical tracking may be able to reduce noise in the neural data and broaden the impact of their results by controlling for linguistic features of their stimuli and by using simple comprehension tasks. Overall, we argue that a balance between the methodological constraints of the two fields will lead to an overall improved understanding of language processing as well as greater clarity on what mechanisms cortical tracking of speech reflects. Increased collaboration will help resolve debates in both fields and will lead to new and exciting avenues for research.