Resolution of impaired multisensory processing in autism and the cost of switching sensory modality

Cerebellar contribution to multisensory integration: A computational modeling exploration

The remarkable ability of the human brain to create a coherent perception of reality relies heavily on multisensory integration-the complex process of combining inputs from different senses. While this mechanism is fundamental to our understanding of the world, its underlying neural architecture remains partially unknown. This study investigates the role of the cerebellum in multisensory integration through a novel computational approach inspired by clinical observations of a patient with cerebellar agenesis. With reference to the clinical data comparing an acerebellar patient with age-matched control subjects, we exploited biologically realistic spiking neural networks to model both conditions. Our computational framework enables testing multiple network configurations and parameters, effectively replicating and extending the clinical experiments in silico. To enhance accessibility and promote broader adoption among researchers, we complemented this framework with a user-friendly web-based interface, eliminating the need for programming expertise. The computational results closely mirror the clinical findings, providing support for the critical contribution of the cerebellum in multisensory integration. Beyond being a consistent proof of concept for the previous clinical observations, this study introduces a versatile platform for testing brain models through our newly developed framework and interface. Thus, this work not only advances our understanding of the cerebellar role in sensory processing but also establishes a robust methodology for future computational investigations of neural mechanisms.

Neural mechanisms of intersensory switching: Evidence for delayed sensory processing and increased cognitive demands

Intersensory switching (IS), the ability to shift attention between different sensory systems, is essential for cognitive flexibility, yet leads to slower responses compared to repeating the same sensory modality. The underlying neural mechanisms of IS remain largely unknown. In this study, high-density EEG was used to investigate these mechanisms in healthy adults (n = 53; mean age 26±7.39; 30 female) performing a speeded reaction time (RT) task involving visual and auditory stimuli. Trials were categorized as Repeat (same preceding modality) or Switch (different preceding modality). Switch trials showed slower RTs and delayed sensory responses (N1 and P2 components). Furthermore, across both Repeat and Switch trials, RT correlated with the latency of these neural responses. Additionally, lower alpha-band inter-trial phase coherence (ITPC) in primary sensory regions was noted for Switch compared to Repeat trials, suggesting reduced efficiency of sensory processing. Greater induced theta activity over fronto-central scalp regions in Switch trials suggested increased cognitive control demands. These findings support a model where the prior stimulus primes the sensory cortex for faster processing of Repeat trials, while Switch trials lead to heightened cognitive resources for adjustment, likely reflecting attentional reallocation mediated by the anterior cingulate cortex (ACC). The consistent effects across auditory and visual modalities indicate that IS relies on a core, modality-independent mechanism grounded in fundamental principles of sensory and attentional reorganization.

Atypical audio-visual neural synchrony and speech processing in early autism

BACKGROUND

Children with Autism Spectrum disorder (ASD) often exhibit communication difficulties that may stem from basic auditory temporal integration impairment but also be aggravated by an audio-visual integration deficit, resulting in a lack of interest in face-to-face communication. This study addresses whether speech processing anomalies in young autistic children (mean age 3.09-year-old) are associated with alterations of audio-visual temporal integration.

METHODS

We used high-density electroencephalography (HD-EEG) and eye tracking to record brain activity and gaze patterns in 31 children with ASD (6 females) and 33 typically developing (TD) children (11 females), while they watched cartoon videos. Neural responses to temporal audio-visual stimuli were analyzed using Temporal Response Functions model and phase analyses for audiovisual temporal coordination.

RESULTS

The reconstructability of speech signals from auditory responses was reduced in children with ASD compared to TD, but despite more restricted gaze patterns in ASD it was similar for visual responses in both groups. Speech reception was most strongly affected when visual speech information was also present, an interference that was not seen in TD children. These differences were associated with a broader phase angle distribution (exceeding pi/2) in the EEG theta range in children with ASD, signaling reduced reliability of audio-visual temporal alignment.

CONCLUSION

These findings show that speech processing anomalies in ASD do not stand alone and that they are associated already at a very early development stage with audio-visual imbalance with poor auditory response encoding and disrupted audio-visual temporal coordination.

Unveiling neurodevelopmental changes in multisensory integration while controlling attention

It is well-accepted that multisensory integration (MSI) undergoes protracted maturation from childhood to adulthood. However, existing evidence may have been confounded by potential age-related differences in attention. To unveil neurodevelopmental changes in MSI while matching top-down attention between children and adults, we recorded event-related potentials of healthy children aged 7 to 9 years and young adults in the visual-to-auditory attentional spreading paradigm wherein attention and MSI could be measured concurrently. The absence of children versus adults differences in the visual selection negativity component and behavioral measures of auditory interference first demonstrates that the child group could maintain top-down visual attention and ignore task-irrelevant auditory information to a similar extent as adults. Then, the stimulus-driven attentional spreading quantified by the auditory negative difference (Nd) component was found to be overall absent in the child group, revealing the children's largely immature audiovisual binding process. These findings furnish strong evidence for the protracted maturation of MSI per se from childhood to adulthood, hence providing a new benchmark for characterizing the developmental course of MSI. In addition, we also found that the representation-driven attentional spreading measured by another Nd was present but less robust in children, suggesting their substantially but not fully developed audiovisual representation coactivation process.

Somatosensory temporal sensitivity in adults on the autism spectrum: A high-density electrophysiological mapping study using the mismatch negativity (MMN) sensory memory paradigm

Atypical reactivity to somatosensory inputs is common in autism spectrum disorder and carries considerable impact on downstream social communication and quality of life. While behavioral and survey work have established differences in the perception of somatosensory information, little has been done to elucidate the underlying neurophysiological processes that drive these characteristics. Here, we implemented a duration-based somatosensory mismatch negativity (MMN) paradigm to examine the role of temporal sensitivity and sensory memory in the processing of vibrotactile information in autistic (n = 30) and neurotypical (n = 30) adults. To capture the variability in responses between groups across a range of duration discrepancies, we compared the electrophysiological responses to frequent standard vibrations (100 ms) and four infrequent deviant vibrations (115, 130, 145, and 160 ms). The same stimuli were used in a follow-up behavioral task to determine active detection of the infrequent vibrations. We found no differences between the two groups with regard to discrimination between standard and deviant vibrations, demonstrating comparable neurologic and behavioral temporal somatosensory perception. However, exploratory analyses yielded subtle differences in amplitude at the N1 and P220 time points. Together, these results indicate that the temporal mechanisms of somatosensory discrimination are conserved in adults on the autism spectrum, though more general somatosensory processing may be affected. We discuss these findings in the broader context of the MMN literature in autism, as well as the potential role of cortical maturity in somatosensory mechanisms.

Age, not autism, influences multisensory integration of speech stimuli among adults in a McGurk/MacDonald paradigm

Differences between autistic and non-autistic individuals in perception of the temporal relationships between sights and sounds are theorized to underlie difficulties in integrating relevant sensory information. These, in turn, are thought to contribute to problems with speech perception and higher level social behaviour. However, the literature establishing this connection often involves limited sample sizes and focuses almost entirely on children. To determine whether these differences persist into adulthood, we compared 496 autistic and 373 non-autistic adults (aged 17 to 75 years). Participants completed an online version of the McGurk/MacDonald paradigm, a multisensory illusion indicative of the ability to integrate audiovisual speech stimuli. Audiovisual asynchrony was manipulated, and participants responded both to the syllable they perceived (revealing their susceptibility to the illusion) and to whether or not the audio and video were synchronized (allowing insight into temporal processing). In contrast with prior research with smaller, younger samples, we detected no evidence of impaired temporal or multisensory processing in autistic adults. Instead, we found that in both groups, multisensory integration correlated strongly with age. This contradicts prior presumptions that differences in multisensory perception persist and even increase in magnitude over the lifespan of autistic individuals. It also suggests that the compensatory role multisensory integration may play as the individual senses decline with age is intact. These findings challenge existing theories and provide an optimistic perspective on autistic development. They also underline the importance of expanding autism research to better reflect the age range of the autistic population.

A multimodal interface for speech perception: the role of the left superior temporal sulcus in social cognition and autism

Multimodal integration is crucial for human interaction, in particular for social communication, which relies on integrating information from various sensory modalities. Recently a third visual pathway specialized in social perception was proposed, which includes the right superior temporal sulcus (STS) playing a key role in processing socially relevant cues and high-level social perception. Importantly, it has also recently been proposed that the left STS contributes to audiovisual integration of speech processing. In this article, we propose that brain areas along the right STS that support multimodal integration for social perception and cognition can be considered homologs to those in the left, language-dominant hemisphere, sustaining multimodal integration of speech and semantic concepts fundamental for social communication. Emphasizing the significance of the left STS in multimodal integration and associated processes such as multimodal attention to socially relevant stimuli, we underscore its potential relevance in comprehending neurodevelopmental conditions characterized by challenges in social communication such as autism spectrum disorder (ASD). Further research into this left lateral processing stream holds the promise of enhancing our understanding of social communication in both typical development and ASD, which may lead to more effective interventions that could improve the quality of life for individuals with atypical neurodevelopment.

NSF DARE-transforming modeling in neurorehabilitation: a patient-in-the-loop framework

In 2023, the National Science Foundation (NSF) and the National Institute of Health (NIH) brought together engineers, scientists, and clinicians by sponsoring a conference on computational modelling in neurorehabiilitation. To facilitate multidisciplinary collaborations and improve patient care, in this perspective piece we identify where and how computational modelling can support neurorehabilitation. To address the where, we developed a patient-in-the-loop framework that uses multiple and/or continual measurements to update diagnostic and treatment model parameters, treatment type, and treatment prescription, with the goal of maximizing clinically-relevant functional outcomes. This patient-in-the-loop framework has several key features: (i) it includes diagnostic and treatment models, (ii) it is clinically-grounded with the International Classification of Functioning, Disability and Health (ICF) and patient involvement, (iii) it uses multiple or continual data measurements over time, and (iv) it is applicable to a range of neurological and neurodevelopmental conditions. To address the how, we identify state-of-the-art and highlight promising avenues of future research across the realms of sensorimotor adaptation, neuroplasticity, musculoskeletal, and sensory & pain computational modelling. We also discuss both the importance of and how to perform model validation, as well as challenges to overcome when implementing computational models within a clinical setting. The patient-in-the-loop approach offers a unifying framework to guide multidisciplinary collaboration between computational and clinical stakeholders in the field of neurorehabilitation.

Intact Somatosensory Temporal Sensitivity in Adults on the Autism Spectrum: A High-Density Electrophysiological Mapping Study Using the Mismatch Negativity (MMN) Sensory Memory Paradigm

Atypical reactivity to somatosensory inputs is common in autism spectrum disorder and carries considerable impact on downstream social communication and quality of life. While behavioral and survey work have established differences in the perception of somatosensory information, little has been done to elucidate the underlying neurophysiological processes that drive these characteristics. Here, we implemented a duration-based somatosensory mismatch negativity paradigm to examine the role of temporal sensitivity and sensory memory in the processing of vibrotactile information in autistic (n=30) and neurotypical (n=30) adults. To capture the variability in responses between groups across a range of duration discrepancies, we compared the electrophysiological responses to frequent standard vibrations (100 ms) and four infrequent deviant vibrations (115, 130, 145, and 160 ms). The same stimuli were used in a follow-up behavioral task to determine active detection of the infrequent vibrations. We found no differences between the two groups with regard to discrimination between standard and deviant vibrations, demonstrating comparable neurologic and behavioral temporal somatosensory perception. However, exploratory analyses yielded subtle differences in amplitude at the N1 and P220 time points. Together, these results indicate that the temporal mechanisms of somatosensory discrimination are conserved in adults on the autism spectrum, though more general somatosensory processing may be affected. We discuss these findings in the broader context of the MMN literature in autism, as well as the potential role of cortical maturity in somatosensory mechanisms.

Neural correlates of audiovisual narrative speech perception in children and adults on the autism spectrum: A functional magnetic resonance imaging study

Autistic individuals show substantially reduced benefit from observing visual articulations during audiovisual speech perception, a multisensory integration deficit that is particularly relevant to social communication. This has mostly been studied using simple syllabic or word-level stimuli and it remains unclear how altered lower-level multisensory integration translates to the processing of more complex natural multisensory stimulus environments in autism. Here, functional neuroimaging was used to examine neural correlates of audiovisual gain (AV-gain) in 41 autistic individuals to those of 41 age-matched non-autistic controls when presented with a complex audiovisual narrative. Participants were presented with continuous narration of a story in auditory-alone, visual-alone, and both synchronous and asynchronous audiovisual speech conditions. We hypothesized that previously identified differences in audiovisual speech processing in autism would be characterized by activation differences in brain regions well known to be associated with audiovisual enhancement in neurotypicals. However, our results did not provide evidence for altered processing of auditory alone, visual alone, audiovisual conditions or AV- gain in regions associated with the respective task when comparing activation patterns between groups. Instead, we found that autistic individuals responded with higher activations in mostly frontal regions where the activation to the experimental conditions was below baseline (de-activations) in the control group. These frontal effects were observed in both unisensory and audiovisual conditions, suggesting that these altered activations were not specific to multisensory processing but reflective of more general mechanisms such as an altered disengagement of Default Mode Network processes during the observation of the language stimulus across conditions.

PERMUTOOLS: A MATLAB PACKAGE FOR MULTIVARIATE PERMUTATION TESTING

Statistical hypothesis testing and effect size measurement are routine parts of quantitative research. Advancements in computer processing power have greatly improved the capability of statistical inference through the availability of resampling methods. However, many of the statistical practices used today are based on traditional, parametric methods that rely on assumptions about the underlying population. These assumptions may not always be valid, leading to inaccurate results and misleading interpretations. Permutation testing, on the other hand, generates the sampling distribution empirically by permuting the observed data, providing distribution-free hypothesis testing. Furthermore, this approach lends itself to a powerful method for multiple comparison correction - known as max correction - which is less prone to type II errors than conventional correction methods. Parametric methods have also traditionally been utilized for estimating the confidence interval of various test statistics and effect size measures. However, these too can be estimated empirically using permutation or bootstrapping techniques. Whilst resampling methods are generally considered preferable, many popular programming languages and statistical software packages lack efficient implementations. Here, we introduce PERMUTOOLS, a MATLAB package for multivariate permutation testing and effect size measurement.

Semantic congruency modulates the speed-up of multisensory responses

Responses to multisensory signals are often faster compared to their unisensory components. This speed-up is typically attributed to target redundancy in that a correct response can be triggered by one or the other signal. In addition, semantic congruency of signals can also modulate multisensory responses; however, the contribution of semantic content is difficult to isolate as its manipulation commonly changes signal redundancy as well. To disentangle the effects of redundancy and semantic congruency, we manipulated semantic content but kept redundancy constant. We presented semantically congruent/incongruent animal pictures and sounds and asked participants to respond with the same response to two target animals (cats and dogs). We find that the speed-up of multisensory responses is larger for congruent (e.g., barking dogs) than incongruent combinations (e.g., barking cats). We then used a computational modelling approach to analyse audio-visual processing interferences that may underlie the effect. Our data is best described by a model that explains the semantic congruency modulation with a parameter that was previously linked to trial sequence effects, which in our experiment occur from the repetition/switching of both sensory modality and animal category. Yet, a systematic analysis of such trial sequence effects shows that the reported congruency effect is an independent phenomenon. Consequently, we discuss potential contributors to the semantic modulation of multisensory responses.

The multisensory mind: a systematic review of multisensory integration processing in Anorexia and Bulimia Nervosa

Individuals with Anorexia Nervosa and Bulimia Nervosa present alterations in the way they experience their bodies. Body experience results from a multisensory integration process in which information from different sensory domains and spatial reference frames is combined into a coherent percept. Given the critical role of the body in the onset and maintenance of both Anorexia Nervosa and Bulimia Nervosa, we conducted a systematic review to examine multisensory integration abilities of individuals affected by these two conditions and investigate whether they exhibit impairments in crossmodal integration. We searched for studies evaluating crossmodal integration in individuals with a current diagnosis of Anorexia Nervosa and Bulimia Nervosa as compared to healthy individuals from both behavioral and neurobiological perspectives. A search of PubMed, PsycINFO, and Web of Sciences databases was performed to extract relevant articles. Of the 2348 studies retrieved, 911 were unique articles. After the screening, 13 articles were included. Studies revealed multisensory integration abnormalities in patients affected by Anorexia Nervosa; only one included individuals with Bulimia Nervosa and observed less severe impairments compared to healthy controls. Overall, results seemed to support the presence of multisensory deficits in Anorexia Nervosa, especially when integrating interoceptive and exteroceptive information. We proposed the Predictive Coding framework for understanding our findings and suggested future lines of investigation.

Atypical development of causal inference in autism inferred through a neurocomputational model

In everyday life, the brain processes a multitude of stimuli from the surrounding environment, requiring the integration of information from different sensory modalities to form a coherent perception. This process, known as multisensory integration, enhances the brain's response to redundant congruent sensory cues. However, it is equally important for the brain to segregate sensory inputs from distinct events, to interact with and correctly perceive the multisensory environment. This problem the brain must face, known as the causal inference problem, is strictly related to multisensory integration. It is widely recognized that the ability to integrate information from different senses emerges during the developmental period, as a function of our experience with multisensory stimuli. Consequently, multisensory integrative abilities are altered in individuals who have atypical experiences with cross-modal cues, such as those on the autistic spectrum. However, no research has been conducted on the developmental trajectories of causal inference and its relationship with experience thus far. Here, we used a neuro-computational model to simulate and investigate the development of causal inference in both typically developing children and those in the autistic spectrum. Our results indicate that higher exposure to cross-modal cues accelerates the acquisition of causal inference abilities, and a minimum level of experience with multisensory stimuli is required to develop fully mature behavior. We then simulated the altered developmental trajectory of causal inference in individuals with autism by assuming reduced multisensory experience during training. The results suggest that causal inference reaches complete maturity much later in these individuals compared to neurotypical individuals. Furthermore, we discuss the underlying neural mechanisms and network architecture involved in these processes, highlighting that the development of causal inference follows the evolution of the mechanisms subserving multisensory integration. Overall, this study provides a computational framework, unifying causal inference and multisensory integration, which allows us to suggest neural mechanisms and provide testable predictions about the development of such abilities in typically developed and autistic children.

Cued motor processing in autism and typical development: A high-density electrical mapping study of response-locked neural activity in children and adolescents

Motor atypicalities are common in autism spectrum disorder (ASD) and are often evident prior to classical ASD symptoms. Despite evidence of differences in neural processing during imitation in autistic individuals, research on the integrity and spatiotemporal dynamics of basic motor processing is surprisingly sparse. To address this need, we analysed electroencephalography (EEG) data recorded from a large sample of autistic (n = 84) and neurotypical (n = 84) children and adolescents while they performed an audiovisual speeded reaction time (RT) task. Analyses focused on RTs and response-locked motor-related electrical brain responses over frontoparietal scalp regions: the late Bereitschaftspotential, the motor potential and the reafferent potential. Evaluation of behavioural task performance indicated greater RT variability and lower hit rates in autistic participants compared to typically developing age-matched neurotypical participants. Overall, the data revealed clear motor-related neural responses in ASD, but with subtle differences relative to typically developing participants evident over fronto-central and bilateral parietal scalp sites prior to response onset. Group differences were further parsed as a function of age (6-9, 9-12 and 12-15 years), sensory cue preceding the response (auditory, visual and bi-sensory audiovisual) and RT quartile. Group differences in motor-related processing were most prominent in the youngest group of children (age 6-9), with attenuated cortical responses observed for young autistic participants. Future investigations assessing the integrity of such motor processes in younger children, where larger differences may be present, are warranted.

Changing the Tendency to Integrate the Senses

Integration of sensory signals that emanate from the same source, such as the visual of lip articulations and the sound of the voice of a speaking individual, can improve perception of the source signal (e.g., speech). Because momentary sensory inputs are typically corrupted with internal and external noise, there is almost always a discrepancy between the inputs, facing the perceptual system with the problem of determining whether the two signals were caused by the same source or different sources. Thus, whether or not multisensory stimuli are integrated and the degree to which they are bound is influenced by factors such as the prior expectation of a common source. We refer to this factor as the tendency to bind stimuli, or for short, binding tendency. In theory, the tendency to bind sensory stimuli can be learned by experience through the acquisition of the probabilities of the co-occurrence of the stimuli. It can also be influenced by cognitive knowledge of the environment. The binding tendency varies across individuals and can also vary within an individual over time. Here, we review the studies that have investigated the plasticity of binding tendency. We discuss the protocols that have been reported to produce changes in binding tendency, the candidate learning mechanisms involved in this process, the possible neural correlates of binding tendency, and outstanding questions pertaining to binding tendency and its plasticity. We conclude by proposing directions for future research and argue that understanding mechanisms and recipes for increasing binding tendency can have important clinical and translational applications for populations or individuals with a deficiency in multisensory integration.

Age-related decrease in motor contribution to multisensory reaction times in primary school children

Traditional measurement of multisensory facilitation in tasks such as speeded motor reaction tasks (MRT) consistently show age-related improvement during early childhood. However, the extent to which motor function increases with age and hence contribute to multisensory motor reaction times in young children has seldom been examined. Thus, we aimed to investigate the contribution of motor development to measures of multisensory (auditory, visual, and audiovisual) and visuomotor processing tasks in three young school age groups of children (n = 69) aged (5-6, n = 21; 7-8, n = 25.; 9-10 n = 18 years). We also aimed to determine whether age-related sensory threshold times for purely visual inspection time (IT) tasks improved significantly with age. Bayesian results showed decisive evidence for age-group differences in multisensory MRT and visuo-motor processing tasks, though the evidence showed that threshold time for visual identification IT performance was only slower in the youngest age group children (5-6) compared to older groups. Bayesian correlations between performance on the multisensory MRT and visuo-motor processing tasks indicated moderate to decisive evidence in favor of the alternative hypothesis (BF10 = 4.71 to 91.346), though not with the threshold IT (BF10 < 1.35). This suggests that visual sensory system development in children older than 6 years makes a less significant contribution to the measure of multisensory facilitation, compared to motor development. In addition to this main finding, multisensory facilitation of MRT within race-model predictions was only found in the oldest group of children (9-10), supporting previous suggestions that multisensory integration is likely to continue into late childhood/early adolescence at least.