The left fusiform gyrus hosts trisensory representations of manipulable objects

Classifying schizophrenia using functional MRI and investigating underlying functional phenomena

BACKGROUND

Existing studies have revealed functional abnormalities in certain brain regions of patients with schizophrenia (SZ), but the relationships between these abnormalities and their impact on disease progression remain unclear.

METHODS

Fifty-six patients with SZ and 56 healthy controls were included. Based on resting-state functional magnetic resonance imaging, we analyzed fractional amplitude of low-frequency fluctuations (fALFF), regional homogeneity (ReHo), and degree centrality (DC). Statistically significant metrics were selected as features, and machine learning models were used to distinguish between patients and controls. Analyze the importance of features in the optimal model. The Louvain community detection algorithm and structural equation modeling were used to investigate community relationships and potential causal effects.

RESULTS

The average prediction accuracy of various ML classifiers reached 0.9241 by fALFF, ReHo, and DC values. The SVM model have the highest performance with an accuracy of 0.9464. Abnormal ReHo in the right middle frontal gyrus contributed most to this optimal classifier and participated in the direct impact on SZ. All the features we analyzed ultimately constituted two functional clusters (FClus), which exhibit internal causal influences. FClus1 had a positive influence on SZ, with the cascade starting from abnormal fALFF in the right inferior temporal gyrus. FClus2 had a negative influence on SZ, with the cascade starting from abnormal fALFF in the left fusiform gyrus.Abnormal fALFF in the right caudate nucleus, degree centrality in the right angular gyrus, and ReHo in the right lentiform nucleus do not have a causal impact on the disease.

CONCLUSIONS

We identified interactions among features within FClus that potentially influence the onset and progression of schizophrenia, including epicenter phenomenon of FClus, FClus for inhibiting schizophrenia, and abnormal function of brain regions without direct impact. Additionally, we believe that the contribution of features to the disease classification model may indicate the size of their direct impact on the disease, not necessarily their importance in the disease process.

Functional gradient of the fusiform gyrus and its underlying molecular basis

Evidence has evinced the functional complexity, anatomical heterogeneity, connectivity diversity, and clinical relevance of the fusiform gyrus. We aimed to investigate the hierarchical organization of the fusiform gyrus and its underlying molecular basis. Resting-state functional MRI data of 793 healthy subjects were collected from a discovery dataset and two independent cross-scanner, cross-race validation datasets. Functional gradients of the fusiform gyrus were calculated based on the voxel-wise fusiform gyrus-to-cerebrum functional connectivity to reflect its functional organization. Transcription-neuroimaging spatial correlation analysis was performed to determine genes with expression levels tracking the fusiform gyrus functional gradient. The dominant functional gradient that explained the greatest connectivity variance showed an anterior-posterior axis across the fusiform gyrus. More important, there was a strong association between the fusiform gyrus-dominant gradient and gene expression profiles, with two gene sets contributing significantly to this association yet differing in their specific expression and functional annotation. In addition, the fusiform gyrus-dominant gradient was linked closely to intrinsic geometry, slightly to cortical morphology, and gradually to behavioral domains from high-level cognitive processes to low-level sensory functions. Our findings add to the extant knowledge regarding the topographic organization of the fusiform gyrus by informing a novel conceptualization of how functional heterogeneity and multiplicity co-occur within the fusiform gyrus.

A meta-analysis of letter-sound integration: Assimilation and accommodation in the superior temporal gyrus

Despite being a relatively new cultural phenomenon, the ability to perform letter-sound integration is readily acquired even though it has not had time to evolve in the brain. Leading theories of how the brain accommodates literacy acquisition include the neural recycling hypothesis and the assimilation-accommodation hypothesis. The neural recycling hypothesis proposes that a new cultural skill is developed by "invading" preexisting neural structures to support a similar cognitive function, while the assimilation-accommodation hypothesis holds that a new cognitive skill relies on direct invocation of preexisting systems (assimilation) and adds brain areas based on task requirements (accommodation). Both theories agree that letter-sound integration may be achieved by reusing pre-existing functionally similar neural bases, but differ in their proposals of how this occurs. We examined the evidence for each hypothesis by systematically comparing the similarities and differences between letter-sound integration and two other types of preexisting and functionally similar audiovisual (AV) processes, namely object-sound and speech-sound integration, by performing an activation likelihood estimation (ALE) meta-analysis. All three types of AV integration recruited the left posterior superior temporal gyrus (STG), while speech-sound integration additionally activated the bilateral middle STG and letter-sound integration directly invoked the AV areas involved in speech-sound integration. These findings suggest that letter-sound integration may reuse the STG for speech-sound and object-sound integration through an assimilation-accommodation mechanism.

Prediction of image interpretation cognitive ability under different mental workloads: a task-state fMRI study

Visual imaging experts play an important role in multiple fields, and studies have shown that the combination of functional magnetic resonance imaging and machine learning techniques can predict cognitive abilities, which provides a possible method for selecting individuals with excellent image interpretation skills. We recorded behavioral data and neural activity of 64 participants during image interpretation tasks under different workloads. Based on the comprehensive image interpretation ability, participants were divided into two groups. general linear model analysis showed that during image interpretation tasks, the high-ability group exhibited higher activation in middle frontal gyrus (MFG), fusiform gyrus, inferior occipital gyrus, superior parietal gyrus, inferior parietal gyrus, and insula compared to the low-ability group. The radial basis function Support Vector Machine (SVM) algorithm shows the most excellent performance in predicting participants' image interpretation abilities (Pearson correlation coefficient = 0.54, R2 = 0.31, MSE = 0.039, RMSE = 0.002). Variable importance analysis indicated that the activation features of the fusiform gyrus and MFG played an important role in predicting this ability. Our study revealed the neural basis related to image interpretation ability when exposed to different mental workloads. Additionally, our results demonstrated the efficacy of machine learning algorithms in extracting neural activation features to predict such ability.

Exploration of Hippocampal Functional Connectivity Alterations in Patients with High Myopia via Seed-Based Functional Connectivity Analysis

Aim

The objective of this study was to examine changes in functional connectivity (FC) in the hippocampus among patients with high myopia (HM) compared to healthy controls (HCs) through the utilization of seed-based functional connectivity (FC) analysis.

Methods

Resting-state functional magnetic resonance imaging (fMRI) was conducted on a sample of 82 patients diagnosed with high myopia (HM) and 59 HCs. The two groups were matched based on age, weight and other relevant variables. Using seed-based FC analysis to detect alterations in hippocampal FC patterns in HM patients and HCs. Furthermore, a correlation analysis was performed to examine the associations between the mean functional connectivity (FC) signals in various brain regions of patients with HM and their corresponding clinical manifestations.

Results

The FC values in the left temporal pole-temporal gyrus (L-TPOsup), right hippocampus (R-HIP), left medial temporal gyrus (L-MTG) and left hippocampus in HM patients were significantly lower than those of healthy subjects. In the left temporal pole-superior temporal gyrus (L-TPOsup), right orbital part of middle frontal gyrus (RO-MFG), left fusiform gyrus (L-FG), left cerebellum superior (L-Cbe6), left middle temporal gyrus (L-MTG), right thalamus (R-THA), and right hippocampus, FC values were also significantly lower.

Conclusion

Brain dysfunction was observed in various regions of the HM patients, suggesting the existence of neurobiological alterations that could lead to impairments in visual cognition, movement, emotional processing, and visual memory.

Evidence for an amodal domain-general object recognition ability

A general object recognition ability predicts performance across a variety of high-level visual tests, categories, and performance in haptic recognition. Does this ability extend to auditory recognition? Vision and haptics tap into similar representations of shape and texture. In contrast, features of auditory perception like pitch, timbre, or loudness do not readily translate into shape percepts related to edges, surfaces, or spatial arrangement of parts. We find that an auditory object recognition ability correlates highly with a visual object recognition ability after controlling for general intelligence, perceptual speed, low-level visual ability, and memory ability. Auditory object recognition was a stronger predictor of visual object recognition than all control measures across two experiments, even though those control variables were also tested visually. These results point towards a single high-level ability used in both vision and audition. Much work highlights how the integration of visual and auditory information is important in specific domains (e.g., speech, music), with evidence for some overlap of visual and auditory neural representations. Our results are the first to reveal a domain-general ability, o, that predicts object recognition performance in both visual and auditory tests. Because o is domain-general, it reveals mechanisms that apply across a wide range of situations, independent of experience and knowledge. As o is distinct from general intelligence, it is well positioned to potentially add predictive validity when explaining individual differences in a variety of tasks, above and beyond measures of common cognitive abilities like general intelligence and working memory.

Elevated Insulin Levels Engage the Salience Network during Multisensory Perception

INTRODUCTION

Brain insulin reactivity has been reported in connection with systematic energy metabolism, enhancement in cognition, olfactory sensitivity, and neuroendocrine circuits. High receptor densities exist in regions important for sensory processing. The main aim of the study was to examine whether intranasal insulin would modulate the activity of areas in charge of olfactory-visual integration.

METHODS

As approach, a placebo-controlled double-blind within crossover design was chosen. The experiments were conducted in a research unit of a university hospital. On separate mornings, twenty-six healthy normal-weight males aged between 19 and 31 years received either 40 IU intranasal insulin or placebo vehicle. Subsequently, they underwent 65 min of functional magnetic resonance imaging whilst performing an odor identification task. Functional brain activations of olfactory, visual, and multisensory integration as well as insulin versus placebo were assessed. Regarding the odor identification task, reaction time, accuracy, pleasantness, and intensity measurements were taken to examine the role of integration and treatment. Blood samples were drawn to control for peripheral hormone concentrations.

RESULTS

Intranasal insulin administration during olfactory-visual stimulation revealed strong bilateral engagement of frontoinsular cortices, anterior cingulate, prefrontal cortex, mediodorsal thalamus, striatal, and hippocampal regions (p ≤ 0.001 familywise error [FWE] corrected). In addition, the integration contrast showed increased activity in left intraparietal sulcus, left inferior frontal gyrus, left superior frontal gyrus, and left middle frontal gyrus (p ≤ 0.013 FWE corrected).

CONCLUSIONS

Intranasal insulin application in lean men led to enhanced activation in multisensory olfactory-visual integration sites and salience hubs which indicates stimuli valuation modulation. This effect can serve as a basis for understanding the connection of intracerebral insulin and olfactory-visual processing.

Verbal Working Memory-Balance program training alters the left fusiform gyrus resting-state functional connectivity: A randomized clinical trial study on children with dyslexia

Sufficient activation of the left fusiform gyrus is important in reading ability acquisition due to its role in reading and naming, working memory (WM), and balance tasks. Recently, a newly-designed training program, Verbal Working Memory-Balance (VWM-B), has been evaluated on children with dyslexia, and its positive effects were shown on reading ability, WM capacity, and postural control. In the present study, we aimed to estimate the functional connectivity alterations of the left fusiform gyrus following training by the VWM-B. Before and after 15 sessions of training, the fMRI and other tools data were collected on a sample of children with dyslexia, who were allocated into two control and experiment groups. Data analyses showed the increased functional connectivity of the left fusiform gyrus between the left anterior temporal fusiform cortex, left and right Crus II regions of the cerebellum, and the left middle frontal gyrus. Moreover, VWM-B training significantly improved the reading and naming ability, WM capacity, and postural control of participants in the experiment group in comparison to the control. The current study findings emphasize the critical role of the left fusiform gyrus in reading ability. Moreover, it provides evidence to support the existence of cerebellar deficits in dyslexia.

Learning without contingencies: A loss of synergy between memory and reward circuits in schizophrenia

Motivational deficits in schizophrenia may interact with foundational cognitive processes including learning and memory to induce impaired cognitive proficiency. If such a loss of synergy exists, it is likely to be underpinned by a loss of synchrony between the brains learning and reward sub-networks. Moreover, this loss should be observed even during tasks devoid of explicit reward contingencies given that such tasks are better models of real world performance than those with artificial contingencies. Here we applied undirected functional connectivity (uFC) analyses to fMRI data acquired while participants engaged in an associative learning task without contingencies or feedback. uFC was estimated and inter-group differences (between schizophrenia patients and controls, n = 54 total, n = 28 patients) were assessed within and between reward (VTA and NAcc) and learning/memory (Basal Ganglia, DPFC, Hippocampus, Parahippocampus, Occipital Lobe) sub-networks. The task paradigm itself alternated between Encoding, Consolidation, and Retrieval conditions, and uFC differences were quantified for each of the conditions. Significantly reduced uFC dominated the connectivity profiles of patients across all conditions. More pertinent to our motivations, these reductions were observed within and across classes of sub-networks (reward-related and learning/memory related). We suggest that disrupted functional connectivity between reward and learning sub-networks may drive many of the performance deficits that characterize schizophrenia. Thus, cognitive deficits in schizophrenia may in fact be underpinned by a loss of synergy between reward-sensitivity and cognitive processes.

Comparing encoding mechanisms in realistic virtual reality and conventional 2D laboratory settings: Event-related potentials in a repetition suppression paradigm

Although the human brain is adapted to function within three-dimensional environments, conventional laboratory research commonly investigates cognitive mechanisms in a reductionist approach using two-dimensional stimuli. However, findings regarding mnemonic processes indicate that realistic experiences in Virtual Reality (VR) are stored in richer and more intertwined engrams than those obtained from the conventional laboratory. Our study aimed to further investigate the generalizability of laboratory findings and to differentiate whether the processes underlying memory formation differ between VR and the conventional laboratory already in early encoding stages. Therefore, we investigated the Repetition Suppression (RS) effect as a correlate of the earliest instance of mnemonic processes under conventional laboratory conditions and in a realistic virtual environment. Analyses of event-related potentials (ERPs) indicate that the ERP deflections at several electrode clusters were lower in VR compared to the PC condition. These results indicate an optimized distribution of cognitive resources in realistic contexts. The typical RS effect was replicated under both conditions at most electrode clusters for a late time window. Additionally, a specific RS effect was found in VR at anterior electrodes for a later time window, indicating more extensive encoding processes in VR compared to the laboratory. Specifically, electrotomographic results (VARETA) indicate multimodal integration involving a broad cortical network and higher cognitive processes during the encoding of realistic objects. Our data suggest that object perception under realistic conditions, in contrast to the conventional laboratory, requires multisensory integration involving an interconnected functional system, facilitating the formation of intertwined memory traces in realistic environments.

Alteration of Degree Centrality in Adolescents With Early Blindness

Congenital nystagmus in infants and young children can lead to early blindness (EB). Previous neuroimaging studies have demonstrated that EB is accompanied by alterations in brain structure and function. However, the effects of visual impairment and critical developmental periods on brain functional connectivity at rest have been unclear. Here, we used the voxel-wise degree centrality (DC) method to explore the underlying functional network brain activity in adolescents with EB. Twenty-one patients with EBs and 21 sighted controls (SCs) underwent magnetic resonance imaging. Differences between the two groups were assessed using the DC method. Moreover, the support vector machine (SVM) method was used to differentiate patients with EB patients from the SCs according to DC values. Compared with the SCs, the patients with EB had increased DC values in the bilateral cerebellum_6, cerebellum vermis_4_5, bilateral supplementary motor areas (SMA), and left fusiform gyrus; the patients with EB had decreased DC values in the bilateral rectal gyrus and left medial orbital frontal gyrus. The SVM classification of the DC values achieved an overall accuracy of 70.45% and an area under the curve of 0.86 in distinguishing between the patients with EB and the SCs. Our study may reveal the neuromechanism of neuroplasticity in EB; the findings provide an imaging basis for future development of restorative visual therapies and sensory substitution devices, and future assessments of visual rehabilitation efficacy.

OUP accepted manuscript

Myasthenia gravis is an autoimmune disease affecting neuromuscular transmission and causing skeletal muscle weakness. Additionally, systemic inflammation, cognitive deficits and autonomic dysfunction have been described. However, little is known about myasthenia gravis-related reorganization of the brain. In this study, we thus investigated the structural and functional brain changes in myasthenia gravis patients. Eleven myasthenia gravis patients (age: 70.64 ± 9.27; 11 males) were compared to age-, sex- and education-matched healthy controls (age: 70.18 ± 8.98; 11 males). Most of the patients (n = 10, 0.91%) received cholinesterase inhibitors. Structural brain changes were determined by applying voxel-based morphometry using high-resolution T₁-weighted sequences. Functional brain changes were assessed with a neuropsychological test battery (including attention, memory and executive functions), a spatial orientation task and brain-derived neurotrophic factor blood levels. Myasthenia gravis patients showed significant grey matter volume reductions in the cingulate gyrus, in the inferior parietal lobe and in the fusiform gyrus. Furthermore, myasthenia gravis patients showed significantly lower performance in executive functions, working memory (Spatial Span, P = 0.034, d = 1.466), verbal episodic memory (P = 0.003, d = 1.468) and somatosensory-related spatial orientation (Triangle Completion Test, P = 0.003, d = 1.200). Additionally, serum brain-derived neurotrophic factor levels were significantly higher in myasthenia gravis patients (P = 0.001, d = 2.040). Our results indicate that myasthenia gravis is associated with structural and functional brain alterations. Especially the grey matter volume changes in the cingulate gyrus and the inferior parietal lobe could be associated with cognitive deficits in memory and executive functions. Furthermore, deficits in somatosensory-related spatial orientation could be associated with the lower volumes in the inferior parietal lobe. Future research is needed to replicate these findings independently in a larger sample and to investigate the underlying mechanisms in more detail.

Association of Poorer Hearing With Longitudinal Change in Cerebral White Matter Microstructure

Importance

There is a dearth of studies that examine the association between poorer hearing and change in cerebral white matter (WM) microstructure.

Objective

To examine the association of poorer hearing with baseline and change in WM microstructure among older adults.

Design, Setting, and Participants

This was a prospective cohort study that evaluated speech-in-noise, pure-tone audiometry, and WM microstructure, as measured by mean diffusivity (MD) and fractional anisotropy (FA), both of which were evaluated by diffusion tensor imaging (DTI) in 17 WM regions. Data were collected between October 2012 and December 2018 and analyzed between March 2019 and August 2019 with a mean follow-up time of 1.7 years. The study evaluated responses to the Baltimore Longitudinal Study of Aging among 356 cognitively normal adults who had at least 1 hearing assessment and DTI session. Excluded were those with baseline cognitive impairment, stroke, head injuries, Parkinson disease, and/or bipolar disorder.

Exposures

Peripheral auditory function was measured by pure-tone average in the better-hearing ear. Central auditory function was measured by signal-to-noise ratio score from a speech-in-noise task and adjusted by pure-tone average.

Main Outcomes and Measures

Linear mixed-effects models with random intercepts and slopes were used to examine the association of poorer peripheral and central auditory function with baseline and longitudinal DTI metrics in 17 WM regions, adjusting for baseline characteristics (age, sex, race, hypertension, elevated total cholesterol, and obesity).

Results

Of 356 cognitively normal adults included in the study, the mean (SD) age was 73.5 (8.8) years, and 204 (57.3%) were women. There were no baseline associations between hearing and DTI measures. Longitudinally, poorer peripheral hearing was associated with increases in MD in the inferior fronto-occipital fasciculus (β = 0.025; 95% CI, 0.008-0.042) and the body (β = 0.050; 95% CI, 0.015-0.085) of the corpus callosum, but there were no associations of peripheral hearing with FA changes in these tracts. Poorer central auditory function was associated with longitudinal MD increases (β = 0.031; 95% CI, 0.010-0.052) and FA declines (β = -1.624; 95% CI, -2.511 to -0.738) in the uncinate fasciculus.

Conclusions and Relevance

Findings of this cohort study suggest that poorer hearing is related to change in integrity of specific WM regions involved with auditory processing.

Trimodal processing of complex stimuli in inferior parietal cortex is modality-independent

In humans, multisensory mechanisms facilitate object processing through integration of sensory signals that match in their temporal and spatial occurrence as well as their meaning. The generalizability of such integration processes across different sensory modalities is, however, to date not well understood. As such, it remains unknown whether there are cerebral areas that process object-related signals independently of the specific senses from which they arise, and whether these areas show different response profiles depending on the number of sensory channels that carry information. To address these questions, we presented participants with dynamic stimuli that simultaneously emitted object-related sensory information via one, two, or three channels (sight, sound, smell) in the MR scanner. By comparing neural activation patterns between various integration processes differing in type and number of stimulated senses, we showed that the left inferior frontal gyrus and areas within the left inferior parietal cortex were engaged independently of the number and type of sensory input streams. Activation in these areas was enhanced during bimodal stimulation, compared to the sum of unimodal activations, and increased even further during trimodal stimulation. Taken together, our findings demonstrate that activation of the inferior parietal cortex during processing and integration of meaningful multisensory stimuli is both modality-independent and modulated by the number of available sensory modalities. This suggests that the processing demand placed on the parietal cortex increases with the number of sensory input streams carrying meaningful information, likely due to the increasing complexity of such stimuli.

Typical and atypical language brain organization based on intrinsic connectivity and multitask functional asymmetries

Based on the joint investigation in 287 healthy volunteers (150 left-Handers (LH)) of language task-induced asymmetries and intrinsic connectivity strength of the sentence-processing supramodal network, we show that individuals with atypical rightward language lateralization (N = 30, 25 LH) do not rely on an organization that simply mirrors that of typical leftward lateralized individuals. Actually, the resting-state organization in the atypicals showed that their sentence processing was underpinned by left and right networks both wired for language processing and highly interacting by strong interhemispheric intrinsic connectivity and larger corpus callosum volume. Such a loose hemispheric specialization for language permits the hosting of language in either the left and/or right hemisphere as assessed by a very high incidence of dissociations across various language task-induced asymmetries in this group.

Properties of cross-modal occipital responses in early blindness: An ALE meta-analysis

•

ALE meta-analysis reveals distributed brain networks for object and spatial functions in individuals with early blindness.

•

ALE contrast analysis reveals specific activations in the left cuneus and lingual gyrus for language function, suggesting a reverse hierarchical organization of the visual cortex for early blind individuals.

•

The findings contribute to visual rehabilitation in blind individuals by revealing the function-dependent and sensory-independent networks during nonvisual processing.

Cross-modal occipital responses appear to be essential for nonvisual processing in individuals with early blindness. However, it is not clear whether the recruitment of occipital regions depends on functional domain or sensory modality. The current study utilized a coordinate-based meta-analysis to identify the distinct brain regions involved in the functional domains of object, spatial/motion, and language processing and the common brain regions involved in both auditory and tactile modalities in individuals with early blindness. Following the PRISMA guidelines, a total of 55 studies were included in the meta-analysis. The specific analyses revealed the brain regions that are consistently recruited for each function, such as the dorsal fronto-parietal network for spatial function and ventral occipito-temporal network for object function. This is consistent with the literature, suggesting that the two visual streams are preserved in early blind individuals. The contrast analyses found specific activations in the left cuneus and lingual gyrus for language function. This finding is novel and suggests a reverse hierarchical organization of the visual cortex for early blind individuals. The conjunction analyses found common activations in the right middle temporal gyrus, right precuneus and a left parieto-occipital region. Clinically, this work contributes to visual rehabilitation in early blind individuals by revealing the function-dependent and sensory-independent networks during nonvisual processing.

Hand-Feel Touch Cues and Their Influences on Consumer Perception and Behavior with Respect to Food Products: A Review

There has been a great deal of research investigating intrinsic/extrinsic cues and their influences on consumer perception and purchasing decisions at points of sale, product usage, and consumption. Consumers create expectations toward a food product through sensory information extracted from its surface (intrinsic cues) or packaging (extrinsic cues) at retail stores. Packaging is one of the important extrinsic cues that can modulate consumer perception, liking, and decision making of a product. For example, handling a product packaging during consumption, even just touching the packaging while opening or holding it during consumption, may result in a consumer expectation of the package content. Although hand-feel touch cues are an integral part of the food consumption experience, as can be observed in such an instance, little has been known about their influences on consumer perception, acceptability, and purchase behavior of food products. This review therefore provided a better understanding about hand-feel touch cues and their influences in the context of food and beverage experience with a focus on (1) an overview of touch as a sensory modality, (2) factors influencing hand-feel perception, (3) influences of hand-feel touch cues on the perception of other sensory modalities, and (4) the effects of hand-feel touch cues on emotional responses and purchase behavior.

Functional Preference for Object Sounds and Voices in the Brain of Early Blind and Sighted Individuals

Sounds activate occipital regions in early blind individuals. However, how different sound categories map onto specific regions of the occipital cortex remains a matter of debate. We used fMRI to characterize brain responses of early blind and sighted individuals to familiar object sounds, human voices, and their respective low-level control sounds. In addition, sighted participants were tested while viewing pictures of faces, objects, and phase-scrambled control pictures. In both early blind and sighted, a double dissociation was evidenced in bilateral auditory cortices between responses to voices and object sounds: Voices elicited categorical responses in bilateral superior temporal sulci, whereas object sounds elicited categorical responses along the lateral fissure bilaterally, including the primary auditory cortex and planum temporale. Outside the auditory regions, object sounds also elicited categorical responses in the left lateral and in the ventral occipitotemporal regions in both groups. These regions also showed response preference for images of objects in the sighted group, thus suggesting a functional specialization that is independent of sensory input and visual experience. Between-group comparisons revealed that, only in the blind group, categorical responses to object sounds extended more posteriorly into the occipital cortex. Functional connectivity analyses evidenced a selective increase in the functional coupling between these reorganized regions and regions of the ventral occipitotemporal cortex in the blind group. In contrast, vocal sounds did not elicit preferential responses in the occipital cortex in either group. Nevertheless, enhanced voice-selective connectivity between the left temporal voice area and the right fusiform gyrus were found in the blind group. Altogether, these findings suggest that, in the absence of developmental vision, separate auditory categories are not equipotent in driving selective auditory recruitment of occipitotemporal regions and highlight the presence of domain-selective constraints on the expression of cross-modal plasticity.

Remembering verbally-presented items as pictures: Brain activity underlying visual mental images in schizophrenia patients with visual hallucinations

BACKGROUND

Previous research suggests that visual hallucinations in schizophrenia consist of mental images mistaken for percepts due to failure of the reality-monitoring processes. However, the neural substrates that underpin such dysfunction are currently unknown. We conducted a brain imaging study to investigate the role of visual mental imagery in visual hallucinations.

METHOD

Twenty-three patients with schizophrenia and 26 healthy participants were administered a reality-monitoring task whilst undergoing an fMRI protocol. At the encoding phase, a mixture of pictures of common items and labels designating common items were presented. On the memory test, participants were requested to remember whether a picture of the item had been presented or merely its label.

RESULTS

Visual hallucination scores were associated with a liberal response bias reflecting propensity to erroneously remember pictures of the items that had in fact been presented as words. At encoding, patients with visual hallucinations differentially activated the right fusiform gyrus when processing the words they later remembered as pictures, which suggests the formation of visual mental images. On the memory test, the whole patient group activated the anterior cingulate and medial superior frontal gyrus when falsely remembering pictures. However, no differential activation was observed in patients with visual hallucinations, whereas in the healthy sample, the production of visual mental images at encoding led to greater activation of a fronto-parietal decisional network on the memory test.

CONCLUSIONS

Visual hallucinations are associated with enhanced visual imagery and possibly with a failure of the reality-monitoring processes that enable discrimination between imagined and perceived events.

Functional and Structural Neuroplasticity Induced by Short-Term Tactile Training Based on Braille Reading

Neuroplastic changes induced by sensory learning have been recognized within the cortices of specific modalities as well as within higher ordered multimodal areas. The interplay between these areas is not fully understood, particularly in the case of somatosensory learning. Here we examined functional and structural changes induced by short-term tactile training based of Braille reading, a task that requires both significant tactile expertise and mapping of tactile input onto multimodal representations. Subjects with normal vision were trained for 3 weeks to read Braille exclusively by touch and scanned before and after training, while performing a same-different discrimination task on Braille characters and meaningless characters. Functional and diffusion-weighted magnetic resonance imaging sequences were used to assess resulting changes. The strongest training-induced effect was found in the primary somatosensory cortex (SI), where we observed bilateral augmentation in activity accompanied by an increase in fractional anisotropy (FA) within the contralateral SI. Increases of white matter fractional anisotropy were also observed in the secondary somatosensory area (SII) and the thalamus. Outside of somatosensory system, changes in both structure and function were found in i.e., the fusiform gyrus, the medial frontal gyri and the inferior parietal lobule. Our results provide evidence for functional remodeling of the somatosensory pathway and higher ordered multimodal brain areas occurring as a result of short-lasting tactile learning, and add to them a novel picture of extensive white matter plasticity.

The neural network for tool-related cognition: An activation likelihood estimation meta-analysis of 70 neuroimaging contrasts

The ability to recognize and use a variety of tools is an intriguing human cognitive function. Multiple neuroimaging studies have investigated neural activations with various types of tool-related tasks. In the present paper, we reviewed tool-related neural activations reported in 70 contrasts from 56 neuroimaging studies and performed a series of activation likelihood estimation (ALE) meta-analyses to identify tool-related cortical circuits dedicated either to general tool knowledge or to task-specific processes. The results indicate the following: (a) Common, task-general processing regions for tools are located in the left inferior parietal lobule (IPL) and ventral premotor cortex; and (b) task-specific regions are located in superior parietal lobule (SPL) and dorsal premotor area for imagining/executing actions with tools and in bilateral occipito-temporal cortex for recognizing/naming tools. The roles of these regions in task-general and task-specific activities are discussed with reference to evidence from neuropsychology, experimental psychology and other neuroimaging studies.

Massive cortical reorganization in sighted Braille readers

The brain is capable of large-scale reorganization in blindness or after massive injury. Such reorganization crosses the division into separate sensory cortices (visual, somatosensory...). As its result, the visual cortex of the blind becomes active during tactile Braille reading. Although the possibility of such reorganization in the normal, adult brain has been raised, definitive evidence has been lacking. Here, we demonstrate such extensive reorganization in normal, sighted adults who learned Braille while their brain activity was investigated with fMRI and transcranial magnetic stimulation (TMS). Subjects showed enhanced activity for tactile reading in the visual cortex, including the visual word form area (VWFA) that was modulated by their Braille reading speed and strengthened resting-state connectivity between visual and somatosensory cortices. Moreover, TMS disruption of VWFA activity decreased their tactile reading accuracy. Our results indicate that large-scale reorganization is a viable mechanism recruited when learning complex skills.

DOI:http://dx.doi.org/10.7554/eLife.10762.001

According to most textbooks, our brain is divided into separate areas that are dedicated to specific senses. We have a visual cortex for vision, a tactile cortex for touch, and so on. However, researchers suspect that this division might not be as fixed as the textbooks say. For example, blind people can switch their 'leftover' visual cortex to non-visual purposes, such as reading Braille – a tactile alphabet.

Can this switch in functional organization also happen in healthy people with normal vision? To investigate this, Siuda-Krzywicka, Bola et al. taught a group of healthy, sighted people to read Braille by touch, and monitored the changes in brain activity that this caused using a technique called functional magnetic resonance imaging. According to textbooks, tactile reading should engage the tactile cortex. Yet, the experiment revealed that the brain activity critical for reading Braille by touch did not occur in the volunteers’ tactile cortex, but in their visual cortex.

Further experiments used a technique called transcranial magnetic stimulation to suppress the activity of the visual cortex of the volunteers. This impaired their ability to read Braille by touch. This is a clear-cut proof that sighted adults can re-program their visual cortex for non-visual, tactile purposes.

These results show that intensive training in a complex task can overcome the sensory division-of-labor of our brain. This indicates that our brain is much more flexible than previously thought, and that such flexibility might occur when we learn everyday, complex skills such as driving a car or playing a musical instrument.

The next question that follows from this work is: what enables the brain’s activity to change after learning to read Braille? To understand this, Siuda-Krzywicka, Bola et al. are currently exploring how the physical structure of the brain changes as a result of a person acquiring the ability to read Braille by touch.

DOI:http://dx.doi.org/10.7554/eLife.10762.002

Preserved Haptic Shape Processing after Bilateral LOC Lesions

The visual and haptic perceptual systems are understood to share a common neural representation of object shape. A region thought to be critical for recognizing visual and haptic shape information is the lateral occipital complex (LOC). We investigated whether LOC is essential for haptic shape recognition in humans by studying behavioral responses and brain activation for haptically explored objects in a patient (M.C.) with bilateral lesions of the occipitotemporal cortex, including LOC. Despite severe deficits in recognizing objects using vision, M.C. was able to accurately recognize objects via touch. M.C.'s psychophysical response profile to haptically explored shapes was also indistinguishable from controls. Using fMRI, M.C. showed no object-selective visual or haptic responses in LOC, but her pattern of haptic activation in other brain regions was remarkably similar to healthy controls. Although LOC is routinely active during visual and haptic shape recognition tasks, it is not essential for haptic recognition of object shape.

SIGNIFICANCE STATEMENT

The lateral occipital complex (LOC) is a brain region regarded to be critical for recognizing object shape, both in vision and in touch. However, causal evidence linking LOC with haptic shape processing is lacking. We studied recognition performance, psychophysical sensitivity, and brain response to touched objects, in a patient (M.C.) with extensive lesions involving LOC bilaterally. Despite being severely impaired in visual shape recognition, M.C. was able to identify objects via touch and she showed normal sensitivity to a haptic shape illusion. M.C.'s brain response to touched objects in areas of undamaged cortex was also very similar to that observed in neurologically healthy controls. These results demonstrate that LOC is not necessary for recognizing objects via touch.

Crossmodal enhancement in the LOC for visuohaptic object recognition over development

Research has provided strong evidence of multisensory convergence of visual and haptic information within the visual cortex. These studies implement crossmodal matching paradigms to examine how systems use information from different sensory modalities for object recognition. Developmentally, behavioral evidence of visuohaptic crossmodal processing has suggested that communication within sensory systems develops earlier than across systems; nonetheless, it is unknown how the neural mechanisms driving these behavioral effects develop. To address this gap in knowledge, BOLD functional Magnetic Resonance Imaging (fMRI) was measured during delayed match-to-sample tasks that examined intramodal (visual-to-visual, haptic-to-haptic) and crossmodal (visual-to-haptic, haptic-to-visual) novel object recognition in children aged 7-8.5 years and adults. Tasks were further divided into sample encoding and test matching phases to dissociate the relative contributions of each. Results of crossmodal and intramodal object recognition revealed the network of known visuohaptic multisensory substrates, including the lateral occipital complex (LOC) and the intraparietal sulcus (IPS). Critically, both adults and children showed crossmodal enhancement within the LOC, suggesting a sensitivity to changes in sensory modality during recognition. These groups showed similar regions of activation, although children generally exhibited more widespread activity during sample encoding and weaker BOLD signal change during test matching than adults. Results further provided evidence of a bilateral region in the occipitotemporal cortex that was haptic-preferring in both age groups. This region abutted the bimodal LOtv, and was consistent with a medial to lateral organization that transitioned from a visual to haptic bias within the LOC. These findings converge with existing evidence of visuohaptic processing in the LOC in adults, and extend our knowledge of crossmodal processing in adults and children.

Convergent and invariant object representations for sight, sound, and touch

We continuously perceive objects in the world through multiple sensory channels. In this study, we investigated the convergence of information from different sensory streams within the cerebral cortex. We presented volunteers with three common objects via three different modalities-sight, sound, and touch-and used multivariate pattern analysis of functional magnetic resonance imaging data to map the cortical regions containing information about the identity of the objects. We could reliably predict which of the three stimuli a subject had seen, heard, or touched from the pattern of neural activity in the corresponding early sensory cortices. Intramodal classification was also successful in large portions of the cerebral cortex beyond the primary areas, with multiple regions showing convergence of information from two or all three modalities. Using crossmodal classification, we also searched for brain regions that would represent objects in a similar fashion across different modalities of presentation. We trained a classifier to distinguish objects presented in one modality and then tested it on the same objects presented in a different modality. We detected audiovisual invariance in the right temporo-occipital junction, audiotactile invariance in the left postcentral gyrus and parietal operculum, and visuotactile invariance in the right postcentral and supramarginal gyri. Our maps of multisensory convergence and crossmodal generalization reveal the underlying organization of the association cortices, and may be related to the neural basis for mental concepts.

Short-term plasticity of visuo-haptic object recognition

Functional magnetic resonance imaging (fMRI) studies have provided ample evidence for the involvement of the lateral occipital cortex (LO), fusiform gyrus (FG), and intraparietal sulcus (IPS) in visuo-haptic object integration. Here we applied 30 min of sham (non-effective) or real offline 1 Hz repetitive transcranial magnetic stimulation (rTMS) to perturb neural processing in left LO immediately before subjects performed a visuo-haptic delayed-match-to-sample task during fMRI. In this task, subjects had to match sample (S1) and target (S2) objects presented sequentially within or across vision and/or haptics in both directions (visual-haptic or haptic-visual) and decide whether or not S1 and S2 were the same objects. Real rTMS transiently decreased activity at the site of stimulation and remote regions such as the right LO and bilateral FG during haptic S1 processing. Without affecting behavior, the same stimulation gave rise to relative increases in activation during S2 processing in the right LO, left FG, bilateral IPS, and other regions previously associated with object recognition. Critically, the modality of S2 determined which regions were recruited after rTMS. Relative to sham rTMS, real rTMS induced increased activations during crossmodal congruent matching in the left FG for haptic S2 and the temporal pole for visual S2. In addition, we found stronger activations for incongruent than congruent matching in the right anterior parahippocampus and middle frontal gyrus for crossmodal matching of haptic S2 and in the left FG and bilateral IPS for unimodal matching of visual S2, only after real but not sham rTMS. The results imply that a focal perturbation of the left LO triggers modality-specific interactions between the stimulated left LO and other key regions of object processing possibly to maintain unimpaired object recognition. This suggests that visual and haptic processing engage partially distinct brain networks during visuo-haptic object matching.

Spontaneous neuronal activity predicts intersubject variations in executive control of attention

Executive control of attention regulates our thoughts, emotion and behavior. Individual differences in executive control are associated with task-related differences in brain activity. But it is unknown whether attentional differences depend on endogenous (resting state) brain activity and to what extent regional fluctuations and functional connectivity contribute to individual variations in executive control processing. Here, we explored the potential contribution of intrinsic brain activity to executive control by using resting-state functional magnetic resonance imaging (fMRI). Using the amplitude of low-frequency fluctuations (ALFF) as an index of spontaneous brain activity, we found that ALFF in the right precuneus (PCUN) and the medial part of left superior frontal gyrus (msFC) was significantly correlated with the efficiency of executive control processing. Crucially, the strengths of functional connectivity between the right PCUN/left msFC and distributed brain regions, including the left fusiform gyrus, right inferior frontal gyrus, left superior frontal gyrus and right precentral gyrus, were correlated with individual differences in executive performance. Together, the ALFF and functional connectivity accounted for 67% of the variability in behavioral performance. Moreover, the strength of functional connectivity between specific regions could predict more individual variability in executive control performance than regionally specific fluctuations. In conclusion, our findings suggest that spontaneous brain activity may reflect or underpin executive control of attention. It will provide new insights into the origins of inter-individual variability in human executive control processing.

Spectral parameters modulation and source localization of blink-related alpha and low-beta oscillations differentiate minimally conscious state from vegetative state/unresponsive wakefulness syndrome

Recently, the cortical source of blink-related delta oscillations (delta BROs) in resting healthy subjects has been localized in the posterior cingulate cortex/precuneus (PCC/PCu), one of the main core-hubs of the default-mode network. This has been interpreted as the electrophysiological signature of the automatic monitoring of the surrounding environment while subjects are immersed in self-reflecting mental activities. Although delta BROs were directly correlated to the degree of consciousness impairment in patients with disorders of consciousness, they failed to differentiate vegetative state/unresponsive wakefulness syndrome (VS/UWS) from minimally conscious state (MCS). In the present study, we have extended the analysis of BROs to frequency bands other than delta in the attempt to find a biological marker that could support the differential diagnosis between VS/UWS and MCS. Four patients with VS/UWS, 5 patients with MCS, and 12 healthy matched controls (CTRL) underwent standard 19-channels EEG recordings during resting conditions. Three-second-lasting EEG epochs centred on each blink instance were submitted to time-frequency analyses in order to extract the normalized Blink-Related Synchronization/Desynchronization (nBRS/BRD) of three bands of interest (low-alpha, high-alpha and low-beta) in the time-window of 50–550 ms after the blink-peak and to estimate the corresponding cortical sources of electrical activity. VS/UWS nBRS/BRD levels of all three bands were lower than those related to both CTRL and MCS, thus enabling the differential diagnosis between MCS and VS/UWS. Furthermore, MCS showed an intermediate signal intensity on PCC/PCu between CTRL and VS/UWS and a higher signal intensity on the left temporo-parieto-occipital junction and inferior occipito-temporal regions when compared to VS/UWS. This peculiar pattern of activation leads us to hypothesize that resting MCS patients have a bottom-up driven activation of the task positive network and thus are tendentially prone to respond to environmental stimuli, even though in an almost unintentional way.

When vision is not an option: children's integration of auditory and haptic information is suboptimal

When visual information is available, human adults, but not children, have been shown to reduce sensory uncertainty by taking a weighted average of sensory cues. In the absence of reliable visual information (e.g. extremely dark environment, visual disorders), the use of other information is vital. Here we ask how humans combine haptic and auditory information from childhood. In the first experiment, adults and children aged 5 to 11 years judged the relative sizes of two objects in auditory, haptic, and non-conflicting bimodal conditions. In Experiment 2, different groups of adults and children were tested in non-conflicting and conflicting bimodal conditions. In Experiment 1, adults reduced sensory uncertainty by integrating the cues optimally, while children did not. In Experiment 2, adults and children used similar weighting strategies to solve audio–haptic conflict. These results suggest that, in the absence of visual information, optimal integration of cues for discrimination of object size develops late in childhood.

Multisensory integration mechanisms during aging

The rapid demographical shift occurring in our society implies that understanding of healthy aging and age-related diseases is one of our major future challenges. Sensory impairments have an enormous impact on our lives and are closely linked to cognitive functioning. Due to the inherent complexity of sensory perceptions, we are commonly presented with a complex multisensory stimulation and the brain integrates the information from the individual sensory channels into a unique and holistic percept. The cerebral processes involved are essential for our perception of sensory stimuli and becomes especially important during the perception of emotional content. Despite ongoing deterioration of the individual sensory systems during aging, there is evidence for an increase in, or maintenance of, multisensory integration processing in aging individuals. Within this comprehensive literature review on multisensory integration we aim to highlight basic mechanisms and potential compensatory strategies the human brain utilizes to help maintain multisensory integration capabilities during healthy aging to facilitate a broader understanding of age-related pathological conditions. Further our goal was to identify where further research is needed.

White matter structural connectivity underlying semantic processing: evidence from brain damaged patients

Widely distributed brain regions in temporal, parietal and frontal cortex have been found to be involved in semantic processing, but the anatomical connections supporting the semantic system are not well understood. In a group of 76 right-handed brain-damaged patients, we tested the relationship between the integrity of major white matter tracts and the presence of semantic deficits. The integrity of white matter tracts was measured by percentage of lesion voxels obtained in structural imaging and mean fractional anisotropy values obtained in diffusion tensor imaging. Semantic deficits were assessed by jointly considering the performance on three semantic tasks that vary in the modalities of input (visual and auditory stimuli) and output (oral naming and associative judgement). We found that the lesion volume and fractional anisotropy value of the left inferior fronto-occipital fasciculus, left anterior thalamic radiation, and left uncinate fasciculus significantly correlated with severity of impairment in all three semantic tasks. These associations remained significant even when we controlled for a wide range of potential confounding variables, including overall cognitive state, whole lesion volume, or type of brain damage. The effects of these three white matter tracts could not be explained by potential involvement of relevant grey matter, and were (relatively) specific to object semantic processing, as no correlation with performance on non-object semantic control tasks (oral repetition and number processing tasks) was observed. These results underscore the causal role of left inferior fronto-occipital fasciculus, left anterior thalamic radiation, and left uncinate fasciculus in semantic processing, providing direct evidence for (part of) the anatomical skeleton of the semantic network.

The development of visual areas depends differently on visual experience

Visual experience plays an important role in the development of the visual cortex; however, recent functional imaging studies have shown that the functional organization is preserved in several higher-tier visual areas in congenitally blind subjects, indicating that maturation of visual areas depend unequally on visual experience. In this study, we aim to validate this hypothesis using a multimodality MRI approach. We found increased cortical thickness in the congenitally blind was present in the early visual areas and absent in the higher-tier ones, suggesting that the structural development of the visual cortex depends hierarchically on visual experience. In congenitally blind subjects, the decreased resting-state functional connectivity with the primary somatosensory cortex was more prominent in the early visual areas than in the higher-tier ones and were more pronounced in the ventral stream than in the dorsal one, suggesting that the development of functional organization of the visual cortex also depends differently on visual experience. Moreover, congenitally blind subjects showed normal or increased functional connectivity between ipsilateral higher-tier and early visual areas, suggesting an indirect corticocortical pathway through which somatosenroy information can reach the early visual areas. These findings support our hypothesis that the development of visual areas depends differently on visual experience.

Neural substrates for semantic memory of familiar songs: is there an interface between lyrics and melodies?

Findings on song perception and song production have increasingly suggested that common but partially distinct neural networks exist for processing lyrics and melody. However, the neural substrates of song recognition remain to be investigated. The purpose of this study was to examine the neural substrates involved in the accessing “song lexicon” as corresponding to a representational system that might provide links between the musical and phonological lexicons using positron emission tomography (PET). We exposed participants to auditory stimuli consisting of familiar and unfamiliar songs presented in three ways: sung lyrics (song), sung lyrics on a single pitch (lyrics), and the sung syllable ‘la’ on original pitches (melody). The auditory stimuli were designed to have equivalent familiarity to participants, and they were recorded at exactly the same tempo. Eleven right-handed nonmusicians participated in four conditions: three familiarity decision tasks using song, lyrics, and melody and a sound type decision task (control) that was designed to engage perceptual and prelexical processing but not lexical processing. The contrasts (familiarity decision tasks versus control) showed no common areas of activation between lyrics and melody. This result indicates that essentially separate neural networks exist in semantic memory for the verbal and melodic processing of familiar songs. Verbal lexical processing recruited the left fusiform gyrus and the left inferior occipital gyrus, whereas melodic lexical processing engaged the right middle temporal sulcus and the bilateral temporo-occipital cortices. Moreover, we found that song specifically activated the left posterior inferior temporal cortex, which may serve as an interface between verbal and musical representations in order to facilitate song recognition.

Long-term memory search across the visual brain

Signal transmission from the human retina to visual cortex and connectivity of visual brain areas are relatively well understood. How specific visual perceptions transform into corresponding long-term memories remains unknown. Here, I will review recent Blood Oxygenation Level-Dependent functional Magnetic Resonance Imaging (BOLD fMRI) in humans together with molecular biology studies (animal models) aiming to understand how the retinal image gets transformed into so-called visual (retinotropic) maps. The broken object paradigm has been chosen in order to illustrate the complexity of multisensory perception of simple objects subject to visual —rather than semantic— type of memory encoding. The author explores how amygdala projections to the visual cortex affect the memory formation and proposes the choice of experimental techniques needed to explain our massive visual memory capacity. Maintenance of the visual long-term memories is suggested to require recycling of GluR2-containing α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptors (AMPAR) and β₂-adrenoreceptors at the postsynaptic membrane, which critically depends on the catalytic activity of the N-ethylmaleimide-sensitive factor (NSF) and protein kinase PKMζ.

Crossmodal recruitment of the ventral visual stream in congenital blindness

We used functional MRI (fMRI) to test the hypothesis that blind subjects recruit the ventral visual stream during nonhaptic tactile-form recognition. Congenitally blind and blindfolded sighted control subjects were scanned after they had been trained during four consecutive days to perform a tactile-form recognition task with the tongue display unit (TDU). Both groups learned the task at the same rate. In line with our hypothesis, the fMRI data showed that during nonhaptic shape recognition, blind subjects activated large portions of the ventral visual stream, including the cuneus, precuneus, inferotemporal (IT), cortex, lateral occipital tactile vision area (LOtv), and fusiform gyrus. Control subjects activated area LOtv and precuneus but not cuneus, IT and fusiform gyrus. These results indicate that congenitally blind subjects recruit key regions in the ventral visual pathway during nonhaptic tactile shape discrimination. The activation of LOtv by nonhaptic tactile shape processing in blind and sighted subjects adds further support to the notion that this area subserves an abstract or supramodal representation of shape. Together with our previous findings, our data suggest that the segregation of the efferent projections of the primary visual cortex into a dorsal and ventral visual stream is preserved in individuals blind from birth.

Multisensory interactions between auditory and haptic object recognition

Object manipulation produces characteristic sounds and causes specific haptic sensations that facilitate the recognition of the manipulated object. To identify the neural correlates of audio-haptic binding of object features, healthy volunteers underwent functional magnetic resonance imaging while they matched a target object to a sample object within and across audition and touch. By introducing a delay between the presentation of sample and target stimuli, it was possible to dissociate haptic-to-auditory and auditory-to-haptic matching. We hypothesized that only semantically coherent auditory and haptic object features activate cortical regions that host unified conceptual object representations. The left fusiform gyrus (FG) and posterior superior temporal sulcus (pSTS) showed increased activation during crossmodal matching of semantically congruent but not incongruent object stimuli. In the FG, this effect was found for haptic-to-auditory and auditory-to-haptic matching, whereas the pSTS only displayed a crossmodal matching effect for congruent auditory targets. Auditory and somatosensory association cortices showed increased activity during crossmodal object matching which was, however, independent of semantic congruency. Together, the results show multisensory interactions at different hierarchical stages of auditory and haptic object processing. Object-specific crossmodal interactions culminate in the left FG, which may provide a higher order convergence zone for conceptual object knowledge.

Speech comprehension aided by multiple modalities: behavioural and neural interactions

Speech comprehension is a complex human skill, the performance of which requires the perceiver to combine information from several sources - e.g. voice, face, gesture, linguistic context - to achieve an intelligible and interpretable percept. We describe a functional imaging investigation of how auditory, visual and linguistic information interact to facilitate comprehension. Our specific aims were to investigate the neural responses to these different information sources, alone and in interaction, and further to use behavioural speech comprehension scores to address sites of intelligibility-related activation in multifactorial speech comprehension. In fMRI, participants passively watched videos of spoken sentences, in which we varied Auditory Clarity (with noise-vocoding), Visual Clarity (with Gaussian blurring) and Linguistic Predictability. Main effects of enhanced signal with increased auditory and visual clarity were observed in overlapping regions of posterior STS. Two-way interactions of the factors (auditory × visual, auditory × predictability) in the neural data were observed outside temporal cortex, where positive signal change in response to clearer facial information and greater semantic predictability was greatest at intermediate levels of auditory clarity. Overall changes in stimulus intelligibility by condition (as determined using an independent behavioural experiment) were reflected in the neural data by increased activation predominantly in bilateral dorsolateral temporal cortex, as well as inferior frontal cortex and left fusiform gyrus. Specific investigation of intelligibility changes at intermediate auditory clarity revealed a set of regions, including posterior STS and fusiform gyrus, showing enhanced responses to both visual and linguistic information. Finally, an individual differences analysis showed that greater comprehension performance in the scanning participants (measured in a post-scan behavioural test) were associated with increased activation in left inferior frontal gyrus and left posterior STS. The current multimodal speech comprehension paradigm demonstrates recruitment of a wide comprehension network in the brain, in which posterior STS and fusiform gyrus form sites for convergence of auditory, visual and linguistic information, while left-dominant sites in temporal and frontal cortex support successful comprehension.

Smoking experience modulates the cortical integration of vision and haptics

Human neuroplasticity of multisensory integration has been studied mainly in the context of natural or artificial training situations in healthy subjects. However, regular smokers also offer the opportunity to assess the impact of intensive daily multisensory interactions with smoking-related objects on the neural correlates of crossmodal object processing. The present functional magnetic resonance imaging study revealed that smokers show a comparable visuo-haptic integration pattern for both smoking paraphernalia and control objects in the left lateral occipital complex, a region playing a crucial role in crossmodal object recognition. Moreover, the degree of nicotine dependence correlated positively with the magnitude of visuo-haptic integration in the left lateral occipital complex (LOC) for smoking-associated but not for control objects. In contrast, in the left LOC non-smokers displayed a visuo-haptic integration pattern for control objects, but not for smoking paraphernalia. This suggests that prolonged smoking-related multisensory experiences in smokers facilitate the merging of visual and haptic inputs in the lateral occipital complex for the respective stimuli. Studying clinical populations who engage in compulsive activities may represent an ecologically valid approach to investigating the neuroplasticity of multisensory integration.

The interactive account of ventral occipitotemporal contributions to reading

The ventral occipitotemporal cortex (vOT) is involved in the perception of visually presented objects and written words. The Interactive Account of vOT function is based on the premise that perception involves the synthesis of bottom-up sensory input with top-down predictions that are generated automatically from prior experience. We propose that vOT integrates visuospatial features abstracted from sensory inputs with higher level associations such as speech sounds, actions and meanings. In this context, specialization for orthography emerges from regional interactions without assuming that vOT is selectively tuned to orthographic features. We discuss how the Interactive Account explains left vOT responses during normal reading and developmental dyslexia; and how it accounts for the behavioural consequences of left vOT damage.

The interactive account of ventral occipitotemporal contributions to reading

The ventral occipitotemporal cortex (vOT) is involved in the perception of visually presented objects and written words. The Interactive Account of vOT function is based on the premise that perception involves the synthesis of bottom-up sensory input with top-down predictions that are generated automatically from prior experience. We propose that vOT integrates visuospatial features abstracted from sensory inputs with higher level associations such as speech sounds, actions and meanings. In this context, specialization for orthography emerges from regional interactions without assuming that vOT is selectively tuned to orthographic features. We discuss how the Interactive Account explains left vOT responses during normal reading and developmental dyslexia; and how it accounts for the behavioural consequences of left vOT damage.