Ten years at the Multisensory Forum: musings on the evolution of a field

Neural correlates of audiovisual narrative speech perception in children and adults on the autism spectrum: A functional magnetic resonance imaging study

Autistic individuals show substantially reduced benefit from observing visual articulations during audiovisual speech perception, a multisensory integration deficit that is particularly relevant to social communication. This has mostly been studied using simple syllabic or word-level stimuli and it remains unclear how altered lower-level multisensory integration translates to the processing of more complex natural multisensory stimulus environments in autism. Here, functional neuroimaging was used to examine neural correlates of audiovisual gain (AV-gain) in 41 autistic individuals to those of 41 age-matched non-autistic controls when presented with a complex audiovisual narrative. Participants were presented with continuous narration of a story in auditory-alone, visual-alone, and both synchronous and asynchronous audiovisual speech conditions. We hypothesized that previously identified differences in audiovisual speech processing in autism would be characterized by activation differences in brain regions well known to be associated with audiovisual enhancement in neurotypicals. However, our results did not provide evidence for altered processing of auditory alone, visual alone, audiovisual conditions or AV- gain in regions associated with the respective task when comparing activation patterns between groups. Instead, we found that autistic individuals responded with higher activations in mostly frontal regions where the activation to the experimental conditions was below baseline (de-activations) in the control group. These frontal effects were observed in both unisensory and audiovisual conditions, suggesting that these altered activations were not specific to multisensory processing but reflective of more general mechanisms such as an altered disengagement of Default Mode Network processes during the observation of the language stimulus across conditions.

Neural correlates of multisensory enhancement in audiovisual narrative speech perception: a fMRI investigation

This fMRI study investigated the effect of seeing articulatory movements of a speaker while listening to a naturalistic narrative stimulus. It had the goal to identify regions of the language network showing multisensory enhancement under synchronous audiovisual conditions. We expected this enhancement to emerge in regions known to underlie the integration of auditory and visual information such as the posterior superior temporal gyrus as well as parts of the broader language network, including the semantic system. To this end we presented 53 participants with a continuous narration of a story in auditory alone, visual alone, and both synchronous and asynchronous audiovisual speech conditions while recording brain activity using BOLD fMRI. We found multisensory enhancement in an extensive network of regions underlying multisensory integration and parts of the semantic network as well as extralinguistic regions not usually associated with multisensory integration, namely the primary visual cortex and the bilateral amygdala. Analysis also revealed involvement of thalamic brain regions along the visual and auditory pathways more commonly associated with early sensory processing. We conclude that under natural listening conditions, multisensory enhancement not only involves sites of multisensory integration but many regions of the wider semantic network and includes regions associated with extralinguistic sensory, perceptual and cognitive processing.

Early Neuro-Psychomotor Therapy Intervention for Theory of Mind and Emotion Recognition in Neurodevelopmental Disorders: A Pilot Study

The aim of the present study is to explore the effect of early neuro-psychomotor therapy to improve theory of mind skills and emotion recognition in children with neurodevelopmental disorders. A pilot study was set up, consisting of in-group training activities based on the neuro-psychomotor approach. Children were evaluated using Neuropsychological Assessment for Child (Nepsy-II), Test of Emotion Comprehension (TEC), and Social Communication Questionnaire (SCQ). For data analysis, one-sample Wilcoxon signed rank test was used with a significance of p < 0.05. Two children with a developmental language disorder and four children with autism spectrum disorders participated in a 3-month training program. Our findings revealed significant improvement in emotion recognition, as measured with Nepsy-II (p = 0.04), while no statistical improvement was found for theory of mind. Despite the limited sample, early neuro-psychomotor therapy improves emotion recognition skills in children with neurodevelopmental disorders. However, considering the explorative nature of the study, findings should be interpreted with caution.

Resolution of impaired multisensory processing in autism and the cost of switching sensory modality

Children with autism spectrum disorders (ASD) exhibit alterations in multisensory processing, which may contribute to the prevalence of social and communicative deficits in this population. Resolution of multisensory deficits has been observed in teenagers with ASD for complex, social speech stimuli; however, whether this resolution extends to more basic multisensory processing deficits remains unclear. Here, in a cohort of 364 participants we show using simple, non-social audiovisual stimuli that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Computational modelling indicated that multisensory processing transitions from a default state of competition to one of facilitation, and that this transition is delayed in ASD. Further analysis revealed group differences in how sensory channels are weighted, and how this is impacted by preceding cross-sensory inputs. Our findings indicate that there is a complex and dynamic interplay among the sensory systems that differs considerably in individuals with ASD.

Crosse et al. study a cohort of 364 participants with autism spectrum disorders (ASD) and matched controls, and show that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Using computational modelling they go on to demonstrate that there is a delayed transition of multisensory processing from a default state of competition to one of facilitation in ASD, as well as differences in sensory weighting and the ability to switch between sensory modalities, which sheds light on the interplay among sensory systems that differ in ASD individuals.

Reduced multisensory facilitation exists at different periods of development in autism

Atypical sensory processing is now recognised as a key component of an autism diagnosis. The integration of multiple sensory inputs (multisensory integration (MSI)) is thought to be idiosyncratic in autistic individuals and may have cascading effects on the development of higher-level skills such as social communication. Multisensory facilitation was assessed using a target detection paradigm in 45 autistic and 111 neurotypical individuals, matched on age and IQ. Target stimuli were: auditory (A; 3500 Hz tone), visual (V; white disk 'flash') or audiovisual (AV; simultaneous tone and flash), and were presented on a dark background in a randomized order with varying stimulus onset delays. Reaction time (RT) was recorded via button press. In order to assess possible developmental effects, participants were divided into younger (age 14 or younger) and older (age 15 and older) groups. Redundancy gain (RG) was significantly greater in neurotypical, compared to autistic individuals. No significant effect of age or interaction was found. Race model analysis was used to compute a bound value that represented the facilitation effect provided by MSI. Our results revealed that MSI facilitation occurred (violation of the race model) in neurotypical individuals, with more efficient MSI in older participants. In both the younger and older autistic groups, we found reduced MSI facilitation (no or limited violation of the race model). Autistic participants showed reduced multisensory facilitation compared to neurotypical participants in a simple target detection task, void of social context. This remained consistent across age. Our results support evidence that autistic individuals may not integrate low-level, non-social information in a typical fashion, adding to the growing discussion around the influential effect that basic perceptual atypicalities may have on the development of higher-level, core aspects of autism.

Operating in a Multisensory Context: Assessing the Interplay Between Multisensory Reaction Time Facilitation and Inter-sensory Task-switching Effects

Individuals respond faster to presentations of bisensory stimuli (e.g. audio-visual targets) than to presentations of either unisensory constituent in isolation (i.e. to the auditory-alone or visual-alone components of an audio-visual stimulus). This well-established multisensory speeding effect, termed the redundant signals effect (RSE), is not predicted by simple linear summation of the unisensory response time probability distributions. Rather, the speeding is typically faster than this prediction, leading researchers to ascribe the RSE to a so-called co-activation account. According to this account, multisensory neural processing occurs whereby the unisensory inputs are integrated to produce more effective sensory-motor activation. However, the typical paradigm used to test for RSE involves random sequencing of unisensory and bisensory inputs in a mixed design, raising the possibility of an alternate attention-switching account. This intermixed design requires participants to switch between sensory modalities on many task trials (e.g. from responding to a visual stimulus to an auditory stimulus). Here we show that much, if not all, of the RSE under this paradigm can be attributed to slowing of reaction times to unisensory stimuli resulting from modality switching, and is not in fact due to speeding of responses to AV stimuli. As such, the present data do not support a co-activation account, but rather suggest that switching and mixing costs akin to those observed during classic task-switching paradigms account for the observed RSE.

Multisensory Audiovisual Processing in Children With a Sensory Processing Disorder (I): Behavioral and Electrophysiological Indices Under Speeded Response Conditions

Background

Maladaptive reactivity to sensory inputs is commonly observed in neurodevelopmental disorders (e.g., autism, ADHD). Little is known, however, about the underlying neural mechanisms. For some children, atypical sensory reactivity is the primary complaint, despite absence of another identifiable neurodevelopmental diagnosis. Studying Sensory Processing Disorder (SPD) may well provide a window into the neuropathology of these symptoms. It has been proposed that a deficit in sensory integration underlies the SPD phenotype, but objective quantification of sensory integration is lacking. Here we used neural and behavioral measures of multisensory integration (MSI), which would be affected by impaired sensory integration and for which there are well accepted objective measures, to test whether failure to integrate across the senses is associated with atypical sensory reactivity in SPD. An autism group served to determine if observed differences were unique to SPD.

Methods

We tested whether children aged 6–16 years with SPD (N = 14) integrate multisensory inputs differently from age-matched typically developing controls (TD: N = 54), or from children with an autism spectrum disorder (ASD: N = 44). Participants performed a simple reaction-time task to the occurrence of auditory, visual, and audiovisual stimuli presented in random order, while high-density recordings of electrical brain activity were made.

Results

Children with SPD showed large reductions in the extent to which they benefited from multisensory inputs compared to TDs. The ASD group showed similarly reduced response speeding to multisensory relative to unisensory inputs. Neural evidence for MSI was seen across all three groups, with the multisensory response differing from the sum of the unisensory responses. Post hoc tests suggested the possibility of enhanced MSI in SPD in timeframes consistent with cortical sensory registration (∼60 ms), followed by reduced MSI during a timeframe consistent with object formation (∼130 ms). The ASD group also showed reduced MSI in the later timeframe.

Conclusion

Children with SPD showed reduction in their ability to benefit from redundant audio-visual inputs, similar to children with ASD. Neurophysiological recordings, on the other hand, showed that major indices of MSI were largely intact, although post hoc testing pointed to periods of potential differential processing. While these exploratory electrophysiological observations point to potential sensory-perceptual differences in multisensory processing in SPD, it remains equally plausible at this stage that later attentional processing differences may yet prove responsible for the multisensory behavioral deficits uncovered here.

Sensory-to-Cognitive Systems Integration Is Associated With Clinical Severity in Autism Spectrum Disorder

OBJECTIVE

Impaired multisensory integration in autism spectrum disorder (ASD) may arise from functional dysconnectivity among brain systems. Our study examines the functional connectivity integration between primary modal sensory regions and heteromodal processing cortex in ASD, and whether abnormalities in network integration relate to clinical severity.

METHOD

We studied a sample of 55 high-functioning ASD and 64 healthy control (HC) male children and adolescents (total n = 119, age range 7-18 years). Stepwise functional connectivity analysis (SFC) was applied to resting state functional magnetic resonance images (rsfMRI) to characterize the connectivity paths that link primary sensory cortices to higher-order brain cognitive functional circuits and to relate alterations in functional connectivity integration with three clinical scales: Social Communication Questionnaire, Social Responsiveness Scale, and Vineland Adaptive Behavior Scales.

RESULTS

HC displayed typical functional connectivity transitions from primary sensory systems to association areas, but the ASD group showed altered patterns of multimodal sensory integration to heteromodal systems. Specifically, compared to the HC group, the ASD group showed the following: (1) hyperconnectivity in the visual cortex at initial link step distances; (2) hyperconnectivity between sensory unimodal regions and regions of the default mode network; and (3) hypoconnectivity between sensory unimodal regions and areas of the fronto-parietal and attentional networks. These patterns of hyper- and hypoconnectivity were associated with increased clinical severity in ASD.

CONCLUSION

Networkwise reorganization in high-functioning ASD individuals affects strategic regions of unimodal-to-heteromodal cortical integration predicting clinical severity. In addition, SFC analysis appears to be a promising approach for studying the neural pathophysiology of multisensory integration deficits in ASD.

Reduced multisensory facilitation in adolescents and adults on the Autism Spectrum

Individuals with autism are reported to integrate information from visual and auditory channels in an idiosyncratic way. Multisensory integration (MSI) of simple, non-social stimuli (i.e., flashes and beeps) was evaluated in adolescents and adults with (n = 20) and without autism (n = 19) using a reaction time (RT) paradigm using audio, visual, and audiovisual stimuli. For each participant, the race model analysis compares the RTs on the audiovisual condition to a bound value computed from the unimodal RTs that reflects the effect of redundancy. If the actual audiovisual RTs are significantly faster than this bound, the race model is violated, indicating evidence of MSI. Our results show that the race model violation occurred only for the typically-developing (TD) group. While the TD group shows evidence of MSI, the autism group does not. These results suggest that multisensory integration of simple information, void of social content or complexity, is altered in autism. Individuals with autism may not benefit from the advantage conferred by multisensory stimulation to the same extent as TD individuals. Altered MSI for simple, non-social information may have cascading effects on more complex perceptual processes related to language and behaviour in autism.

Audiovisual Multisensory Integration and Evoked Potentials in Young Adults With and Without Attention-Deficit/Hyperactivity Disorder

The purpose of this study was to assess how young adults with attention-deficit/hyperactivity disorder (ADHD) process audiovisual (AV) multisensory stimuli using behavioral and neurological measures. Adults with a clinical diagnosis of ADHD (n = 10) and neurotypical controls (n = 11) completed a simple response time task, consisting of auditory, visual, and AV multisensory conditions. Continuous 64-electrode electroencephalography (EEG) was collected to assess neurological responses to each condition. The AV multisensory condition resulted in the shortest response times for both populations. Analysis using the race model (Miller, 1982) demonstrated that those with ADHD had violation of the race model earlier in the response, which may be a marker for impulsivity. EEG analysis revealed that both groups had early multisensory integration (MSI) occur following multisensory stimulus onset. There were also significant group differences in event-related potentials (ERPs) in frontal, parietal, and occipital brain regions, which are regions reported to be altered in those with ADHD. This study presents results examining multisensory processing in the population of adults with ADHD, and can be used as a foundation for future ADHD research using developmental research designs as well as the development of novel technological supports.

Audiovisual multisensory integration in individuals with autism spectrum disorder: A systematic review and meta-analysis

An ever-growing literature has aimed to determine how individuals with autism spectrum disorder (ASD) differ from their typically developing (TD) peers on measures of multisensory integration (MSI) and to ascertain the degree to which differences in MSI are associated with the broad range of symptoms associated with ASD. Findings, however, have been highly variable across the studies carried out to date. The present work systematically reviews and quantitatively synthesizes the large literature on audiovisual MSI in individuals with ASD to evaluate the cumulative evidence for (a) group differences between individuals with ASD and TD peers, (b) correlations between MSI and autism symptoms in individuals with ASD and (c) study level factors that may moderate findings (i.e., explain differential effects) observed across studies. To identify eligible studies, a comprehensive search strategy was employed using the ProQuest search engine, PubMed database, forwards and backwards citation searches, direct author contact, and hand-searching of select conference proceedings. A significant between-group difference in MSI was evident in the literature, with individuals with ASD demonstrating worse audiovisual integration on average across studies compared to TD controls. This effect was moderated by mean participant age, such that between-group differences were more pronounced in younger samples. The mean correlation between MSI and autism and related symptomatology was also significant, indicating that increased audiovisual integration in individuals with ASD is associated with better language/communication abilities and/or reduced autism symptom severity in the extant literature. This effect was moderated by whether the stimuli were linguistic versus non-linguistic in nature, such that correlation magnitudes tended to be significantly greater when linguistic stimuli were utilized in the measure of MSI. Limitations and future directions for primary and meta-analytic research are discussed.

Multisensory integration of redundant trisensory stimulation

Integration of sensory information across modalities can confer behavioral advantages by decreasing perceptual ambiguity, increasing reaction time, and increasing detection accuracy relative to unisensory stimuli. We asked how combinations of auditory, visual, and somatosensory events alter response time. Participants detected stimulation on one side of space (right or left) while ignoring stimulation on the other side of space. There were seven types of suprathreshold stimuli: auditory (tones from speakers), visual (sinusoidal contrast gratings), somatosensory (fingertip vibrations), audio-visual, somato-visual, audio-somatosensory, and audio-somato-visual. Response enhancement and race model analysis confirmed that bisensory and trisensory trials enhanced response time relative to unisensory trials. Exploratory analysis of individual differences in intersensory facilitation revealed that participants fit into one of two groups: those who benefitted from trisensory information and those who did not.

Basic and complex emotion recognition in children with autism: cross-cultural findings

BACKGROUND

Children with autism spectrum conditions (ASC) have emotion recognition deficits when tested in different expression modalities (face, voice, body). However, these findings usually focus on basic emotions, using one or two expression modalities. In addition, cultural similarities and differences in emotion recognition patterns in children with ASC have not been explored before. The current study examined the similarities and differences in the recognition of basic and complex emotions by children with ASC and typically developing (TD) controls across three cultures: Israel, Britain, and Sweden.

METHODS

Fifty-five children with high-functioning ASC, aged 5-9, were compared to 58 TD children. On each site, groups were matched on age, sex, and IQ. Children were tested using four tasks, examining recognition of basic and complex emotions from voice recordings, videos of facial and bodily expressions, and emotional video scenarios including all modalities in context.

RESULTS

Compared to their TD peers, children with ASC showed emotion recognition deficits in both basic and complex emotions on all three modalities and their integration in context. Complex emotions were harder to recognize, compared to basic emotions for the entire sample. Cross-cultural agreement was found for all major findings, with minor deviations on the face and body tasks.

CONCLUSIONS

Our findings highlight the multimodal nature of ER deficits in ASC, which exist for basic as well as complex emotions and are relatively stable cross-culturally. Cross-cultural research has the potential to reveal both autism-specific universal deficits and the role that specific cultures play in the way empathy operates in different countries.

Visual, Auditory, and Cross Modal Sensory Processing in Adults with Autism: An EEG Power and BOLD fMRI Investigation

Electroencephalography (EEG) and blood oxygen level dependent functional magnetic resonance imagining (BOLD fMRI) assessed the neurocorrelates of sensory processing of visual and auditory stimuli in 11 adults with autism (ASD) and 10 neurotypical (NT) controls between the ages of 20-28. We hypothesized that ASD performance on combined audiovisual trials would be less accurate with observable decreased EEG power across frontal, temporal, and occipital channels and decreased BOLD fMRI activity in these same regions; reflecting deficits in key sensory processing areas. Analysis focused on EEG power, BOLD fMRI, and accuracy. Lower EEG beta power and lower left auditory cortex fMRI activity were seen in ASD compared to NT when they were presented with auditory stimuli as demonstrated by contrasting the activity from the second presentation of an auditory stimulus in an all auditory block vs. the second presentation of a visual stimulus in an all visual block (AA2-VV2).We conclude that in ASD, combined audiovisual processing is more similar than unimodal processing to NTs.

Multisensory integration compensates loss of sensitivity of visual temporal order in the elderly

Here, we examined sensitivity of visual, auditory, and audiovisual temporal order in five age-groups (20 to 70 years old). We also measured multisensory integration (MSI) using a phenomenon known as "temporal ventriloquism," in which click sounds improve sensitivity of visual temporal order. Results showed that sensitivity of visual, auditory, and audiovisual temporal order declined from 50 years on. However, there was no corresponding decline in MSI as the click sounds actually compensated the loss of sensitivity of visual temporal order in the elderly. Sensitivity of audiovisual temporal order did not correlate with MSI, suggesting that well-preserved explicit judgments about cross-modal temporal order are not required for MSI to occur.

Severe multisensory speech integration deficits in high-functioning school-aged children with Autism Spectrum Disorder (ASD) and their resolution during early adolescence

Under noisy listening conditions, visualizing a speaker's articulations substantially improves speech intelligibility. This multisensory speech integration ability is crucial to effective communication, and the appropriate development of this capacity greatly impacts a child's ability to successfully navigate educational and social settings. Research shows that multisensory integration abilities continue developing late into childhood. The primary aim here was to track the development of these abilities in children with autism, since multisensory deficits are increasingly recognized as a component of the autism spectrum disorder (ASD) phenotype. The abilities of high-functioning ASD children (n = 84) to integrate seen and heard speech were assessed cross-sectionally, while environmental noise levels were systematically manipulated, comparing them with age-matched neurotypical children (n = 142). Severe integration deficits were uncovered in ASD, which were increasingly pronounced as background noise increased. These deficits were evident in school-aged ASD children (5-12 year olds), but were fully ameliorated in ASD children entering adolescence (13-15 year olds). The severity of multisensory deficits uncovered has important implications for educators and clinicians working in ASD. We consider the observation that the multisensory speech system recovers substantially in adolescence as an indication that it is likely amenable to intervention during earlier childhood, with potentially profound implications for the development of social communication abilities in ASD children.

The development of multisensory integration in high-functioning autism: high-density electrical mapping and psychophysical measures reveal impairments in the processing of audiovisual inputs

Successful integration of auditory and visual inputs is crucial for both basic perceptual functions and for higher-order processes related to social cognition. Autism spectrum disorders (ASD) are characterized by impairments in social cognition and are associated with abnormalities in sensory and perceptual processes. Several groups have reported that individuals with ASD are impaired in their ability to integrate socially relevant audiovisual (AV) information, and it has been suggested that this contributes to the higher-order social and cognitive deficits observed in ASD. However, successful integration of auditory and visual inputs also influences detection and perception of nonsocial stimuli, and integration deficits may impair earlier stages of information processing, with cascading downstream effects. To assess the integrity of basic AV integration, we recorded high-density electrophysiology from a cohort of high-functioning children with ASD (7-16 years) while they performed a simple AV reaction time task. Children with ASD showed considerably less behavioral facilitation to multisensory inputs, deficits that were paralleled by less effective neural integration. Evidence for processing differences relative to typically developing children was seen as early as 100 ms poststimulation, and topographic analysis suggested that children with ASD relied on different cortical networks during this early multisensory processing stage.

An emerging paradigm: a strength-based approach to exploring mental imagery

Mental imagery, or the ability to simulate in the mind information that is not currently perceived by the senses, has attracted considerable research interest in psychology since the early 1970's. Within the past two decades, research in this field-as in cognitive psychology more generally-has been dominated by neuroscientific methods that typically involve comparisons between imagery performance of participants from clinical populations with those who exhibit apparently normal cognitive functioning. Although this approach has been valuable in identifying key neural substrates of visual imagery, it has been less successful in understanding the possible mechanisms underlying another simulation process, namely, motor imagery or the mental rehearsal of actions without engaging in the actual movements involved. In order to address this oversight, a "strength-based" approach has been postulated which is concerned with understanding those on the high ability end of the imagery performance spectrum. Guided by the expert performance approach and principles of ecological validity, converging methods have the potential to enable imagery researchers to investigate the neural "signature" of elite performers, for example. Therefore, the purpose of this paper is to explain the origin, nature, and implications of the strength-based approach to mental imagery. Following a brief explanation of the background to this latter approach, we highlight some important theoretical advances yielded by recent research on mental practice, mental travel, and meta-imagery processes in expert athletes and dancers. Next, we consider the methodological implications of using a strength-based approach to investigate imagery processes. The implications for the field of motor cognition are outlined and specific research questions, in dynamic imagery, imagery perspective, measurement, multi-sensory imagery, and metacognition that may benefit from this approach in the future are sketched briefly.

Diminished sensitivity of audiovisual temporal order in autism spectrum disorder

We examined sensitivity of audiovisual temporal order in adolescents with autism spectrum disorder (ASD) using an audiovisual temporal order judgment (TOJ) task. In order to assess domain-specific impairments, the stimuli varied in social complexity from simple flash/beeps to videos of a handclap or a speaking face. Compared to typically-developing controls, individuals with ASD were generally less sensitive in judgments of audiovisual temporal order (larger just noticeable differences, JNDs), but there was no specific impairment with social stimuli. This suggests that people with ASD suffer from a more general impairment in audiovisual temporal processing.

Neural dynamics of audiovisual synchrony and asynchrony perception in 6-month-old infants

Young infants are sensitive to multisensory temporal synchrony relations, but the neural dynamics of temporal interactions between vision and audition in infancy are not well understood. We investigated audiovisual synchrony and asynchrony perception in 6-month-old infants using event-related brain potentials (ERP). In a prior behavioral experiment (n = 45), infants were habituated to an audiovisual synchronous stimulus and tested for recovery of interest by presenting an asynchronous test stimulus in which the visual stream was delayed with respect to the auditory stream by 400 ms. Infants who behaviorally discriminated the change in temporal alignment were included in further analyses. In the EEG experiment (final sample: n = 15), synchronous and asynchronous stimuli (visual delay of 400 ms) were presented in random order. Results show latency shifts in the auditory ERP components N1 and P2 as well as the infant ERP component Nc. Latencies in the asynchronous condition were significantly longer than in the synchronous condition. After video onset but preceding the auditory onset, amplitude modulations propagating from posterior to anterior sites and related to the Pb component of infants' ERP were observed. Results suggest temporal interactions between the two modalities. Specifically, they point to the significance of anticipatory visual motion for auditory processing, and indicate young infants' predictive capacities for audiovisual temporal synchrony relations.

Behavioral impact of unisensory and multisensory audio-tactile events: pros and cons for interlimb coordination in juggling

Recent behavioral neuroscience research revealed that elementary reactive behavior can be improved in the case of cross-modal sensory interactions thanks to underlying multisensory integration mechanisms. Can this benefit be generalized to an ongoing coordination of movements under severe physical constraints? We choose a juggling task to examine this question. A central issue well-known in juggling lies in establishing and maintaining a specific temporal coordination among balls, hands, eyes and posture. Here, we tested whether providing additional timing information about the balls and hands motions by using external sound and tactile periodic stimulations, the later presented at the wrists, improved the behavior of jugglers. One specific combination of auditory and tactile metronome led to a decrease of the spatiotemporal variability of the juggler's performance: a simple sound associated to left and right tactile cues presented antiphase to each other, which corresponded to the temporal pattern of hands movement in the juggling task. A contrario, no improvements were obtained in the case of other auditory and tactile combinations. We even found a degraded performance when tactile events were presented alone. The nervous system thus appears able to integrate in efficient way environmental information brought by different sensory modalities, but only if the information specified matches specific features of the coordination pattern. We discuss the possible implications of these results for the understanding of the neuronal integration process implied in audio-tactile interaction in the context of complex voluntary movement, and considering the well-known gating effect of movement on vibrotactile perception.

The N1 auditory evoked potential component as an endophenotype for schizophrenia: high-density electrical mapping in clinically unaffected first-degree relatives, first-episode, and chronic schizophrenia patients

The N1 component of the auditory evoked potential (AEP) is a robust and easily recorded metric of auditory sensory-perceptual processing. In patients with schizophrenia, a diminution in the amplitude of this component is a near-ubiquitous finding. A pair of recent studies has also shown this N1 deficit in first-degree relatives of schizophrenia probands, suggesting that the deficit may be linked to the underlying genetic risk of the disease rather than to the disease state itself. However, in both these studies, a significant proportion of the relatives had other psychiatric conditions. As such, although the N1 deficit represents an intriguing candidate endophenotype for schizophrenia, it remains to be shown whether it is present in a group of clinically unaffected first-degree relatives. In addition to testing first-degree relatives, we also sought to replicate the N1 deficit in a group of first-episode patients and in a group of chronic schizophrenia probands. Subject groups consisted of 35 patients with schizophrenia, 30 unaffected first-degree relatives, 13 first-episode patients, and 22 healthy controls. Subjects sat in a dimly lit room and listened to a series of simple 1,000-Hz tones, indicating with a button press whenever they heard a deviant tone (1,500 Hz; 17% probability), while the AEP was recorded from 72 scalp electrodes. Both chronic and first-episode patients showed clear N1 amplitude decrements relative to healthy control subjects. Crucially, unaffected first-degree relatives also showed a clear N1 deficit. This study provides further support for the proposal that the auditory N1 deficit in schizophrenia is linked to the underlying genetic risk of developing this disorder. In light of recent studies, these results point to the N1 deficit as an endophenotypic marker for schizophrenia. The potential future utility of this metric as one element of a multivariate endophenotype is discussed.

Sensory processing in autism: a review of neurophysiologic findings

Atypical sensory-based behaviors are a ubiquitous feature of autism spectrum disorders (ASDs). In this article, we review the neural underpinnings of sensory processing in autism by reviewing the literature on neurophysiological responses to auditory, tactile, and visual stimuli in autistic individuals. We review studies of unimodal sensory processing and multisensory integration that use a variety of neuroimaging techniques, including electroencephalography (EEG), magnetoencephalography (MEG), and functional MRI. We then explore the impact of covert and overt attention on sensory processing. With additional characterization, neurophysiologic profiles of sensory processing in ASD may serve as valuable biomarkers for diagnosis and monitoring of therapeutic interventions for autism and reveal potential strategies and target brain regions for therapeutic interventions.

Auditory enhancement of visual perception at threshold depends on visual abilities

Whether or not multisensory interactions can improve detection thresholds, and thus widen the range of perceptible events is a long-standing debate. Here we revisit this question, by testing the influence of auditory stimuli on visual detection threshold, in subjects exhibiting a wide range of visual-only performance. Above the perceptual threshold, crossmodal interactions have indeed been reported to depend on the subject's performance when the modalities are presented in isolation. We thus tested normal-seeing subjects and short-sighted subjects wearing their usual glasses. We used a paradigm limiting potential shortcomings of previous studies: we chose a criterion-free threshold measurement procedure and precluded exogenous cueing effects by systematically presenting a visual cue whenever a visual target (a faint Gabor patch) might occur. Using this carefully controlled procedure, we found that concurrent sounds only improved visual detection thresholds in the sub-group of subjects exhibiting the poorest performance in the visual-only conditions. In these subjects, for oblique orientations of the visual stimuli (but not for vertical or horizontal targets), the auditory improvement was still present when visual detection was already helped with flanking visual stimuli generating a collinear facilitation effect. These findings highlight that crossmodal interactions are most efficient to improve perceptual performance when an isolated modality is deficient.

Multisensory processing in children with autism: high-density electrical mapping of auditory-somatosensory integration

Successful integration of signals from the various sensory systems is crucial for normal sensory-perceptual functioning, allowing for the perception of coherent objects rather than a disconnected cluster of fragmented features. Several prominent theories of autism suggest that automatic integration is impaired in this population, but there have been few empirical tests of this thesis. A standard electrophysiological metric of multisensory integration (MSI) was used to test the integrity of auditory-somatosensory integration in children with autism (N=17, aged 6-16 years), compared to age- and IQ-matched typically developing (TD) children. High-density electrophysiology was recorded while participants were presented with either auditory or somatosensory stimuli alone (unisensory conditions), or as a combined auditory-somatosensory stimulus (multisensory condition), in randomized order. Participants watched a silent movie during testing, ignoring concurrent stimulation. Significant differences between neural responses to the multisensory auditory-somatosensory stimulus and the unisensory stimuli (the sum of the responses to the auditory and somatosensory stimuli when presented alone) served as the dependent measure. The data revealed group differences in the integration of auditory and somatosensory information that appeared at around 175 ms, and were characterized by the presence of MSI for the TD but not the autism spectrum disorder (ASD) children. Overall, MSI was less extensive in the ASD group. These findings are discussed within the framework of current knowledge of MSI in typical development as well as in relation to theories of ASD.

Tactile shape discrimination recruits human lateral occipital complex during early perceptual processing

Neuroimaging studies investigating somatosensory-based object recognition in humans have revealed activity in the lateral occipital complex, a cluster of regions primarily associated with visual object recognition. To date, determining whether this activity occurs during or subsequent to recognition per se, has been difficult to assess due to the low temporal resolution of the hemodynamic response. To more finely measure the timing of somatosensory object recognition processes we employed high density EEG using a modified version of a paradigm previously applied to neuroimaging experiments. Simple geometric shapes were presented to the right index finger of 10 participants while the ongoing EEG was measured time locked to the stimulus. In the condition of primary interest participants discriminated the shape of the stimulus. In the alternate condition they judged stimulus duration. Using traditional event-related potential analysis techniques we found significantly greater amplitudes in the evoked potentials of the shape discrimination condition between 140 and 160 ms, a timeframe in which LOC mediated perceptual processes are believed to occur during visual object recognition. Scalp voltage topography and source analysis procedures indicated the lateral occipital complex as the likely source behind this effect. This finding supports a multisensory role for the lateral occipital complex during object recognition.

The multifaceted interplay between attention and multisensory integration

Multisensory integration has often been characterized as an automatic process. Recent findings indicate that multisensory integration can occur across various stages of stimulus processing that are linked to, and can be modulated by, attention. Stimulus-driven, bottom-up mechanisms induced by crossmodal interactions can automatically capture attention towards multisensory events, particularly when competition to focus elsewhere is relatively low. Conversely, top-down attention can facilitate the integration of multisensory inputs and lead to a spread of attention across sensory modalities. These findings point to a more intimate and multifaceted interplay between attention and multisensory integration than was previously thought. We review developments in the current understanding of the interactions between attention and multisensory processing, and propose a framework that unifies previous, apparently discordant, findings.