The intraparietal sulcus governs multisensory integration of audiovisual information based on task difficulty

Visuotactile integration in individuals with fibromyalgia

Our brain constantly integrates afferent information, such as visual and tactile information, to perceive the world around us. According to the maximum-likelihood estimation (MLE) model, imprecise information will be weighted less than precise, making the multisensory percept as precise as possible. Individuals with fibromyalgia (FM), a chronic pain syndrome, show alterations in the integration of tactile information. This could lead to a decrease in their weight in a multisensory percept or a general disruption of multisensory integration, making it less beneficial. To assess multisensory integration, 15 participants with FM and 18 pain-free controls performed a temporal-order judgment task in which they received pairs of sequential visual, tactile (unisensory conditions), or visuotactile (multisensory condition) stimulations on the index and the thumb of the non-dominant hand and had to determine which finger was stimulated first. The task enabled us to measure the precision and accuracy of the percept in each condition. Results indicate an increase in precision in the visuotactile condition compared to the unimodal conditions in controls only, although we found no intergroup differences. The observed visuotactile precision was correlated to the precision predicted by the MLE model in both groups, suggesting an optimal integration. Finally, the weights of the sensory information were not different between the groups; however, in the group with FM, higher pain intensity was associated with smaller tactile weight. This study shows no alterations of the visuotactile integration in individuals with FM, though pain may influence tactile weight in these participants.

The impact of degraded vision on emotional perception of audiovisual stimuli: An event-related potential study

Emotion recognition will be challenged for individuals when visual signals are degraded in real-life scenarios. Recently, researchers have conducted many studies on the distinct neural activity between clear and degraded audiovisual stimuli. These findings addressed the "how" question, but the precise stage of the distinct activity that occurred remains unknown. Therefore, it is crucial to use event-related potential (ERP) to explore the "when" question, just the time course of the neural activity of degraded audiovisual stimuli. In the present research, we established two conditions: clear auditory + degraded visual (AcVd) and clear auditory + clear visual (AcVc) multisensory conditions. We enlisted 31 participants to evaluate the emotional valence of audiovisual stimuli. The resulting data were analyzed using ERP in time domains and Microstate analysis. Current results suggest that degraded vision impairs the early-stage processing of audiovisual stimuli, with the superior parietal lobule (SPL) regulating audiovisual processing in a top-down fashion. Additionally, our findings indicate that negative and positive stimuli elicit greater EPN compared to neutral stimuli, pointing towards a subjective motivation-related attentional regulation. To sum up, in the early stage of emotional audiovisual processing, the degraded visual signal affected the perception of the physical attributes of audiovisual stimuli and had a further influence on emotion extraction processing, leading to the different regulation of top-down attention resources in the later stage.

Multi-modal Representation of the Size of Space in the Human Brain

To estimate the size of an indoor space, we must analyze the visual boundaries that limit the spatial extent and acoustic cues from reflected interior surfaces. We used fMRI to examine how the brain processes the geometric size of indoor scenes when various types of sensory cues are presented individually or together. Specifically, we asked whether the size of space is represented in a modality-specific way or in an integrative way that combines multimodal cues. In a block-design study, images or sounds that depict small- and large-sized indoor spaces were presented. Visual stimuli were real-world pictures of empty spaces that were small or large. Auditory stimuli were sounds convolved with different reverberations. By using a multivoxel pattern classifier, we asked whether the two sizes of space can be classified in visual, auditory, and visual-auditory combined conditions. We identified both sensory-specific and multimodal representations of the size of space. To further investigate the nature of the multimodal region, we specifically examined whether it contained multimodal information in a coexistent or integrated form. We found that angular gyrus and the right medial frontal gyrus had modality-integrated representation, displaying sensitivity to the match in the spatial size information conveyed through image and sound. Background functional connectivity analysis further demonstrated that the connection between sensory-specific regions and modality-integrated regions increases in the multimodal condition compared with single modality conditions. Our results suggest that spatial size perception relies on both sensory-specific and multimodal representations, as well as their interplay during multimodal perception.

Multisensory integration in the mammalian brain: diversity and flexibility in health and disease

Multisensory integration (MSI) occurs in a variety of brain areas, spanning cortical and subcortical regions. In traditional studies on sensory processing, the sensory cortices have been considered for processing sensory information in a modality-specific manner. The sensory cortices, however, send the information to other cortical and subcortical areas, including the higher association cortices and the other sensory cortices, where the multiple modality inputs converge and integrate to generate a meaningful percept. This integration process is neither simple nor fixed because these brain areas interact with each other via complicated circuits, which can be modulated by numerous internal and external conditions. As a result, dynamic MSI makes multisensory decisions flexible and adaptive in behaving animals. Impairments in MSI occur in many psychiatric disorders, which may result in an altered perception of the multisensory stimuli and an abnormal reaction to them. This review discusses the diversity and flexibility of MSI in mammals, including humans, primates and rodents, as well as the brain areas involved. It further explains how such flexibility influences perceptual experiences in behaving animals in both health and disease. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Audiovisual n-Back Training Alters the Neural Processes of Working Memory and Audiovisual Integration: Evidence of Changes in ERPs

(1) Background: This study investigates whether audiovisual n-back training leads to training effects on working memory and transfer effects on perceptual processing. (2) Methods: Before and after training, the participants were tested using the audiovisual n-back task (1-, 2-, or 3-back), to detect training effects, and the audiovisual discrimination task, to detect transfer effects. (3) Results: For the training effect, the behavioral results show that training leads to greater accuracy and faster response times. Stronger training gains in accuracy and response time using 3- and 2-back tasks, compared to 1-back, were observed in the training group. Event-related potentials (ERPs) data revealed an enhancement of P300 in the frontal and central regions across all working memory levels after training. Training also led to the enhancement of N200 in the central region in the 3-back condition. For the transfer effect, greater audiovisual integration in the frontal and central regions during the post-test rather than pre-test was observed at an early stage (80-120 ms) in the training group. (4) Conclusion: Our findings provide evidence that audiovisual n-back training enhances neural processes underlying a working memory and demonstrate a positive influence of higher cognitive functions on lower cognitive functions.

Identifying cortical areas that underlie the transformation from 2D retinal to 3D head-centric motion signals

Accurate motion perception requires that the visual system integrate the 2D retinal motion signals received by the two eyes into a single representation of 3D motion. However, most experimental paradigms present the same stimulus to the two eyes, signaling motion limited to a 2D fronto-parallel plane. Such paradigms are unable to dissociate the representation of 3D head-centric motion signals (i.e., 3D object motion relative to the observer) from the associated 2D retinal motion signals. Here, we used stereoscopic displays to present separate motion signals to the two eyes and examined their representation in visual cortex using fMRI. Specifically, we presented random-dot motion stimuli that specified various 3D head-centric motion directions. We also presented control stimuli, which matched the motion energy of the retinal signals, but were inconsistent with any 3D motion direction. We decoded motion direction from BOLD activity using a probabilistic decoding algorithm. We found that 3D motion direction signals can be reliably decoded in three major clusters in the human visual system. Critically, in early visual cortex (V1-V3), we found no significant difference in decoding performance between stimuli specifying 3D motion directions and the control stimuli, suggesting that these areas represent the 2D retinal motion signals, rather than 3D head-centric motion itself. In voxels in and surrounding hMT and IPS0 however, decoding performance was consistently superior for stimuli that specified 3D motion directions compared to control stimuli. Our results reveal the parts of the visual processing hierarchy that are critical for the transformation of retinal into 3D head-centric motion signals and suggest a role for IPS0 in their representation, in addition to its sensitivity to 3D object structure and static depth.

Neural correlates of multisensory integration in the human brain: an ALE meta-analysis

Previous fMRI research identified superior temporal sulcus as central integration area for audiovisual stimuli. However, less is known about a general multisensory integration network across senses. Therefore, we conducted activation likelihood estimation meta-analysis with multiple sensory modalities to identify a common brain network. We included 49 studies covering all Aristotelian senses i.e., auditory, visual, tactile, gustatory, and olfactory stimuli. Analysis revealed significant activation in bilateral superior temporal gyrus, middle temporal gyrus, thalamus, right insula, and left inferior frontal gyrus. We assume these regions to be part of a general multisensory integration network comprising different functional roles. Here, thalamus operate as first subcortical relay projecting sensory information to higher cortical integration centers in superior temporal gyrus/sulcus while conflict-processing brain regions as insula and inferior frontal gyrus facilitate integration of incongruent information. We additionally performed meta-analytic connectivity modelling and found each brain region showed co-activations within the identified multisensory integration network. Therefore, by including multiple sensory modalities in our meta-analysis the results may provide evidence for a common brain network that supports different functional roles for multisensory integration.

Audiovisual integration of the dynamic hand-held tool at different stimulus intensities in aging

Introduction: In comparison to the audiovisual integration of younger adults, the same process appears more complex and unstable in older adults. Previous research has found that stimulus intensity is one of the most important factors influencing audiovisual integration. Methods: The present study compared differences in audiovisual integration between older and younger adults using dynamic hand-held tool stimuli, such as holding a hammer hitting the floor. Meanwhile, the effects of stimulus intensity on audiovisual integration were compared. The intensity of the visual and auditory stimuli was regulated by modulating the contrast level and sound pressure level. Results: Behavioral results showed that both older and younger adults responded faster and with higher hit rates to audiovisual stimuli than to visual and auditory stimuli. Further results of event-related potentials (ERPs) revealed that during the early stage of 60-100 ms, in the low-intensity condition, audiovisual integration of the anterior brain region was greater in older adults than in younger adults; however, in the high-intensity condition, audiovisual integration of the right hemisphere region was greater in younger adults than in older adults. Moreover, audiovisual integration was greater in the low-intensity condition than in the high-intensity condition in older adults during the 60-100 ms, 120-160 ms, and 220-260 ms periods, showing inverse effectiveness. However, there was no difference in the audiovisual integration of younger adults across different intensity conditions. Discussion: The results suggested that there was an age-related dissociation between high- and low-intensity conditions with audiovisual integration of the dynamic hand-held tool stimulus. Older adults showed greater audiovisual integration in the lower intensity condition, which may be due to the activation of compensatory mechanisms.

Frontal medial cortex and angular gyrus functional connectivity is related to sex and age differences in odor sensitivity

BACKGROUND AND PURPOSE

Odor preference is one of the key factors for the rehabilitation of the swallowing function. On the other hand, sensitivity to odor differs between sexes and decreases with age. These factors rely on brain neuronal circuits. However, it remains not fully clarified which neuronal circuit determines the sex and age differences in odor sensitivity. In this study, we carried out both the odor sensitivity test and functional MRI (fMRI) to find the key neuronal circuits determining sex and age differences in odor sensitivity.

METHODS

Healthy volunteers (28 males, aged 27-62 years, and 30 females, aged 21-59 years) participated in this study. Some of them (seven males and seven females) underwent fMRI. We prepared five odorous test substances and presented each substance at 1 minute intervals. After 5 minutes of questioning about food intake, the subjects were asked to recall each of the test substances presented from the list. In the fMRI study, all the subjects underwent 15 minutes of the prestimulation, stimulation with peppermint odor, and poststimulation sessions.

RESULTS

The odor test score was significantly higher in females than in males and showed an age-dependent decrease. We found four functional connectivities whose degrees were significantly different between males and females. One of them, the functional connectivity between the frontal medial cortex (MedFC) and the left angular gyrus (AG. l), showed an age-dependent change.

CONCLUSIONS

The functional MedFC-AG.l connectivity is one of the important neuronal circuits that affect the sex- and age-dependent odor sensitivity.

Odor-induced sound localization bias under unilateral intranasal trigeminal stimulation

As a stereo odor cue, internostril odor influx could help us in many spatial tasks, including localization and navigation. Studies have also revealed that this benefit could be modulated by the asymmetric concentrations of both influxes (left nose vs right nose). The interaction between olfaction and vision, such as in object recognition and visual direction judgment, has been documented; however, little has been revealed about the impact of odor cues on sound localization. Here we adopted the ventriloquist paradigm in auditory-odor interactions and investigated sound localization with the concurrent unilateral odor influx. Specifically, we teased apart both the "nature" of the odors (pure olfactory stimulus vs. mixed olfactory/trigeminal stimulus) and the location of influx (left nose vs. right nose) and examined sound localization with the method of constant stimuli. Forty-one participants, who passed the Chinese Smell Identification Test, perceived sounds with different azimuths (0°, 5°, 10°, and 20° unilaterally deflected from the sagittal plane by head-related transfer function) and performed sound localization (leftward or rightward) tasks under concurrent, different unilateral odor influxes (10% v/v phenylethyl alcohol, PEA, as pure olfactory stimulus, 1% m/v menthol as mixed olfactory/trigeminal stimulus, and propylene glycol as the control). Meanwhile, they reported confidence levels of the judgments. Results suggested that unilateral PEA influx did not affect human sound localization judgments. However, unilateral menthol influx systematically biased the perceived sound localization, shifting toward the odor source. Our study provides evidence that unilateral odor influx could bias perceived sound localization only when the odor activates the trigeminal nerves.

Predicting Post-Stroke Somatosensory Function from Resting-State Functional Connectivity: A Feasibility Study

Accumulating evidence shows that brain functional deficits may be impacted by damage to remote brain regions. Recent advances in neuroimaging suggest that stroke impairment can be better predicted based on disruption to brain networks rather than from lesion locations or volumes only. Our aim was to explore the feasibility of predicting post-stroke somatosensory function from brain functional connectivity through the application of machine learning techniques. Somatosensory impairment was measured using the Tactile Discrimination Test. Functional connectivity was employed to model the global brain function. Behavioral measures and MRI were collected at the same timepoint. Two machine learning models (linear regression and support vector regression) were chosen to predict somatosensory impairment from disrupted networks. Along with two feature pools (i.e., low-order and high-order functional connectivity, or low-order functional connectivity only) engineered, four predictive models were built and evaluated in the present study. Forty-three chronic stroke survivors participated this study. Results showed that the regression model employing both low-order and high-order functional connectivity can predict outcomes based on correlation coefficient of r = 0.54 (p = 0.0002). A machine learning predictive approach, involving high- and low-order modelling, is feasible for the prediction of residual somatosensory function in stroke patients using functional brain networks.

Acquired olfactory loss alters functional connectivity and morphology

Removing function from a developed and functional sensory system is known to alter both cerebral morphology and functional connections. To date, a majority of studies assessing sensory-dependent plasticity have focused on effects from either early onset or long-term sensory loss and little is known how the recent sensory loss affects the human brain. With the aim of determining how recent sensory loss affects cerebral morphology and functional connectivity, we assessed differences between individuals with acquired olfactory loss (duration 7-36 months) and matched healthy controls in their grey matter volume, using multivariate pattern analyses, and functional connectivity, using dynamic connectivity analyses, within and from the olfactory cortex. Our results demonstrate that acquired olfactory loss is associated with altered grey matter volume in, among others, posterior piriform cortex, a core olfactory processing area, as well as the inferior frontal gyrus and angular gyrus. In addition, compared to controls, individuals with acquired anosmia displayed significantly stronger dynamic functional connectivity from the posterior piriform cortex to, among others, the angular gyrus, a known multisensory integration area. When assessing differences in dynamic functional connectivity from the angular gyrus, individuals with acquired anosmia had stronger connectivity from the angular gyrus to areas primary responsible for basic visual processing. These results demonstrate that recently acquired sensory loss is associated with both changed cerebral morphology within core olfactory areas and increase dynamic functional connectivity from olfactory cortex to cerebral areas processing multisensory integration.

Multisensory Information Facilitates the Categorization of Untrained Stimuli

Although it has been demonstrated that multisensory information can facilitate object recognition and object memory, it remains unclear whether such facilitation effect exists in category learning. To address this issue, comparable car images and sounds were first selected by a discrimination task in Experiment 1. Then, those selected images and sounds were utilized in a prototype category learning task in Experiments 2 and 3, in which participants were trained with auditory, visual, and audiovisual stimuli, and were tested with trained or untrained stimuli within the same categories presented alone or accompanied with a congruent or incongruent stimulus in the other modality. In Experiment 2, when low-distortion stimuli (more similar to the prototypes) were trained, there was higher accuracy for audiovisual trials than visual trials, but no significant difference between audiovisual and auditory trials. During testing, accuracy was significantly higher for congruent trials than unisensory or incongruent trials, and the congruency effect was larger for untrained high-distortion stimuli than trained low-distortion stimuli. In Experiment 3, when high-distortion stimuli (less similar to the prototypes) were trained, there was higher accuracy for audiovisual trials than visual or auditory trials, and the congruency effect was larger for trained high-distortion stimuli than untrained low-distortion stimuli during testing. These findings demonstrated that higher degree of stimuli distortion resulted in more robust multisensory effect, and the categorization of not only trained but also untrained stimuli in one modality could be influenced by an accompanying stimulus in the other modality.

Neural Basis of the Sound-Symbolic Crossmodal Correspondence Between Auditory Pseudowords and Visual Shapes

Sound symbolism refers to the association between the sounds of words and their meanings, often studied using the crossmodal correspondence between auditory pseudowords, e.g., 'takete' or 'maluma', and pointed or rounded visual shapes, respectively. In a functional magnetic resonance imaging study, participants were presented with pseudoword-shape pairs that were sound-symbolically congruent or incongruent. We found no significant congruency effects in the blood oxygenation level-dependent (BOLD) signal when participants were attending to visual shapes. During attention to auditory pseudowords, however, we observed greater BOLD activity for incongruent compared to congruent audiovisual pairs bilaterally in the intraparietal sulcus and supramarginal gyrus, and in the left middle frontal gyrus. We compared this activity to independent functional contrasts designed to test competing explanations of sound symbolism, but found no evidence for mediation via language, and only limited evidence for accounts based on multisensory integration and a general magnitude system. Instead, we suggest that the observed incongruency effects are likely to reflect phonological processing and/or multisensory attention. These findings advance our understanding of sound-to-meaning mapping in the brain.

Seeing Beyond Your Nose? The Effects of Lifelong Olfactory Sensory Deprivation on Cerebral Audio-visual Integration

Lifelong auditory and visual sensory deprivation have been demonstrated to alter both perceptual acuity and the neural processing of remaining senses. Recently, it was demonstrated that individuals with anosmia, i.e. complete olfactory sensory deprivation, displayed enhanced multisensory integration performance. Whether this ability is due to a reorganization of olfactory processing regions to focus on cross-modal multisensory information or whether it is due to enhanced processing within multisensory integration regions is not known. To dissociate these two outcomes, we investigated the neural processing of dynamic audio-visual stimuli in individuals with congenital anosmia and matched controls (both groups, n = 33) using functional magnetic resonance imaging. Specifically, we assessed whether the previously demonstrated multisensory enhancement is related to cross-modal processing of multisensory stimuli in olfactory associated regions, the piriform and olfactory orbitofrontal cortices, or enhanced multisensory processing in established multisensory integration regions, the superior temporal and intraparietal sulci. No significant group differences were found in the a priori hypothesized regions using region of interest analyses. However, exploratory whole-brain analysis suggested higher activation related to multisensory integration within the posterior superior temporal sulcus, in close proximity to the multisensory region of interest, in individuals with congenital anosmia. No group differences were demonstrated in olfactory associated regions. Although results were outside our hypothesized regions, combined, they tentatively suggest that enhanced processing of audio-visual stimuli in individuals with congenital anosmia may be mediated by multisensory, and not primary sensory, cerebral regions.

Trimodal processing of complex stimuli in inferior parietal cortex is modality-independent

In humans, multisensory mechanisms facilitate object processing through integration of sensory signals that match in their temporal and spatial occurrence as well as their meaning. The generalizability of such integration processes across different sensory modalities is, however, to date not well understood. As such, it remains unknown whether there are cerebral areas that process object-related signals independently of the specific senses from which they arise, and whether these areas show different response profiles depending on the number of sensory channels that carry information. To address these questions, we presented participants with dynamic stimuli that simultaneously emitted object-related sensory information via one, two, or three channels (sight, sound, smell) in the MR scanner. By comparing neural activation patterns between various integration processes differing in type and number of stimulated senses, we showed that the left inferior frontal gyrus and areas within the left inferior parietal cortex were engaged independently of the number and type of sensory input streams. Activation in these areas was enhanced during bimodal stimulation, compared to the sum of unimodal activations, and increased even further during trimodal stimulation. Taken together, our findings demonstrate that activation of the inferior parietal cortex during processing and integration of meaningful multisensory stimuli is both modality-independent and modulated by the number of available sensory modalities. This suggests that the processing demand placed on the parietal cortex increases with the number of sensory input streams carrying meaningful information, likely due to the increasing complexity of such stimuli.

Meta-Analyses Support a Taxonomic Model for Representations of Different Categories of Audio-Visual Interaction Events in the Human Brain

Our ability to perceive meaningful action events involving objects, people, and other animate agents is characterized in part by an interplay of visual and auditory sensory processing and their cross-modal interactions. However, this multisensory ability can be altered or dysfunctional in some hearing and sighted individuals, and in some clinical populations. The present meta-analysis sought to test current hypotheses regarding neurobiological architectures that may mediate audio-visual multisensory processing. Reported coordinates from 82 neuroimaging studies (137 experiments) that revealed some form of audio-visual interaction in discrete brain regions were compiled, converted to a common coordinate space, and then organized along specific categorical dimensions to generate activation likelihood estimate (ALE) brain maps and various contrasts of those derived maps. The results revealed brain regions (cortical "hubs") preferentially involved in multisensory processing along different stimulus category dimensions, including 1) living versus nonliving audio-visual events, 2) audio-visual events involving vocalizations versus actions by living sources, 3) emotionally valent events, and 4) dynamic-visual versus static-visual audio-visual stimuli. These meta-analysis results are discussed in the context of neurocomputational theories of semantic knowledge representations and perception, and the brain volumes of interest are available for download to facilitate data interpretation for future neuroimaging studies.

Monkeys and humans implement causal inference to simultaneously localize auditory and visual stimuli

The environment is sampled by multiple senses, which are woven together to produce a unified perceptual state. However, optimally unifying such signals requires assigning particular signals to the same or different underlying objects or events. Many prior studies (especially in animals) have assumed fusion of cross-modal information, whereas recent work in humans has begun to probe the appropriateness of this assumption. Here we present results from a novel behavioral task in which both monkeys (Macaca mulatta) and humans localized visual and auditory stimuli and reported their perceived sources through saccadic eye movements. When the locations of visual and auditory stimuli were widely separated, subjects made two saccades, while when the two stimuli were presented at the same location they made only a single saccade. Intermediate levels of separation produced mixed response patterns: a single saccade to an intermediate position on some trials or separate saccades to both locations on others. The distribution of responses was well described by a hierarchical causal inference model that accurately predicted both the explicit "same vs. different" source judgments as well as biases in localization of the source(s) under each of these conditions. The results from this task are broadly consistent with prior work in humans across a wide variety of analogous tasks, extending the study of multisensory causal inference to nonhuman primates and to a natural behavioral task with both a categorical assay of the number of perceived sources and a continuous report of the perceived position of the stimuli.NEW & NOTEWORTHY We developed a novel behavioral paradigm for the study of multisensory causal inference in both humans and monkeys and found that both species make causal judgments in the same Bayes-optimal fashion. To our knowledge, this is the first demonstration of behavioral causal inference in animals, and this cross-species comparison lays the groundwork for future experiments using neuronal recording techniques that are impractical or impossible in human subjects.

Externalization Errors of Olfactory Source Monitoring in Healthy Controls-An fMRI Study

Using a combined approach of functional magnetic resonance imaging (fMRI) and noninvasive brain stimulation (transcranial direct current stimulation [tDCS]), the present study investigated source memory and its link to mental imagery in the olfactory domain, as well as in the auditory domain. Source memory refers to the knowledge of the origin of mental experiences, differentiating events that have occurred and memories of imagined events. Because of a confusion between internally generated and externally perceived information, patients that are prone to hallucinations show decreased source memory accuracy; also, vivid mental imagery can lead to similar results in healthy controls. We tested source memory following cathodal tDCS stimulation using a mental imagery task, which required participants to perceive or imagine a set of the same olfactory and auditory stimuli during fMRI. The supplementary motor area (SMA) is involved in mental imagery across different modalities and potentially linked to source memory. Therefore, we attempted to modulate participants' SMA activation before entering the scanner using tDCS to influence source memory accuracy in healthy participants. Our results showed the same source memory accuracy between the olfactory and auditory modalities with no effects of stimulation. Finally, we found SMA's subregions differentially involved in olfactory and auditory imagery, with activation of dorsal SMA correlated with auditory source memory.

Audio-visual and olfactory-visual integration in healthy participants and subjects with autism spectrum disorder

The human capacity to integrate sensory signals has been investigated with respect to different sensory modalities. A common denominator of the neural network underlying the integration of sensory clues has yet to be identified. Additionally, brain imaging data from patients with autism spectrum disorder (ASD) do not cover disparities in neuronal sensory processing. In this fMRI study, we compared the underlying neural networks of both olfactory-visual and auditory-visual integration in patients with ASD and a group of matched healthy participants. The aim was to disentangle sensory-specific networks so as to derive a potential (amodal) common source of multisensory integration (MSI) and to investigate differences in brain networks with sensory processing in individuals with ASD. In both groups, similar neural networks were found to be involved in the olfactory-visual and auditory-visual integration processes, including the primary visual cortex, the inferior parietal sulcus (IPS), and the medial and inferior frontal cortices. Amygdala activation was observed specifically during olfactory-visual integration, with superior temporal activation having been seen during auditory-visual integration. A dynamic causal modeling analysis revealed a nonlinear top-down IPS modulation of the connection between the respective primary sensory regions in both experimental conditions and in both groups. Thus, we demonstrate that MSI has shared neural sources across olfactory-visual and audio-visual stimulation in patients and controls. The enhanced recruitment of the IPS to modulate changes between areas is relevant to sensory perception. Our results also indicate that, with respect to MSI processing, adults with ASD do not significantly differ from their healthy counterparts.

Visual input shapes the auditory frequency responses in the inferior colliculus of mouse

The integration of visual and auditory information is important for humans or animals to build an accurate and coherent perception of the external world. Although some evidence has shown some principles of the audiovisual integration, little insight has been gained into its functional purpose. In this study, we investigated the functional influence of dynamic visual input on auditory frequency processing by recording single unit activity in the central nucleus of the inferior colliculus (ICc). Results showed that the auditory responses of ICc neurons to sound frequencies could be enhanced or suppressed by visual stimuli even though the same visual stimuli induced no neural responses when presented alone. For each ICc neuron, the most effective visual stimuli were located in the same azimuth as for auditory stimuli and preceded for ∼20 ms. Additionally, visual stimuli could steepen or flatten the frequency tuning curves (FTCs) of ICc neurons by various visual effects at each responsive frequency. The modulation degree of auditory FTCs was dependent on the minimal thresholds (MTs) of ICc neurons, i.e., with MTs increasing, the modulation degree decreased. Due to the non-homogeneous distribution of MTs which was lowest at 10 kHz, visual modulation of auditory FTCs exhibited a frequency-specific manner, the closer it reached the characteristic frequency (CF) of 10 kHz, the greater modulation. Thus, visual modulation of auditory frequency responses in ICc is dependent not only on the visual stimulus but also on the auditory characteristics of ICc neurons. These results suggest a moment-to-moment visual modulation of auditory frequency responses that in real time increase auditory frequency sensitivity to audiovisual stimuli. Furthermore, in the long term such modulation could serve to instruct auditory adaptive plasticity to maintain necessary and accurate auditory detection and perceptual behavior.

The intraparietal sulcus governs multisensory integration of audiovisual information based on task difficulty

Object recognition benefits maximally from multimodal sensory input when stimulus presentation is noisy, or degraded. Whether this advantage can be attributed specifically to the extent of overlap in object-related information, or rather, to object-unspecific enhancement due to the mere presence of additional sensory stimulation, remains unclear. Further, the cortical processing differences driving increased multisensory integration (MSI) for degraded compared with clear information remain poorly understood. Here, two consecutive studies first compared behavioral benefits of audio-visual overlap of object-related information, relative to conditions where one channel carried information and the other carried noise. A hierarchical drift diffusion model indicated performance enhancement when auditory and visual object-related information was simultaneously present for degraded stimuli. A subsequent fMRI study revealed visual dominance on a behavioral and neural level for clear stimuli, while degraded stimulus processing was mainly characterized by activation of a frontoparietal multisensory network, including IPS. Connectivity analyses indicated that integration of degraded object-related information relied on IPS input, whereas clear stimuli were integrated through direct information exchange between visual and auditory sensory cortices. These results indicate that the inverse effectiveness observed for identification of degraded relative to clear objects in behavior and brain activation might be facilitated by selective recruitment of an executive cortical network which uses IPS as a relay mediating crossmodal sensory information exchange.