Multisensory perception and attention in school-age children

Dynamic causal modeling analysis reveals the modulation of motor cortex and integration in superior temporal gyrus during multisensory speech perception

The processing of speech information from various sensory modalities is crucial for human communication. Both left posterior superior temporal gyrus (pSTG) and motor cortex importantly involve in the multisensory speech perception. However, the dynamic integration of primary sensory regions to pSTG and the motor cortex remain unclear. Here, we implemented a behavioral experiment of classical McGurk effect paradigm and acquired the task functional magnetic resonance imaging (fMRI) data during synchronized audiovisual syllabic perception from 63 normal adults. We conducted dynamic causal modeling (DCM) analysis to explore the cross-modal interactions among the left pSTG, left precentral gyrus (PrG), left middle superior temporal gyrus (mSTG), and left fusiform gyrus (FuG). Bayesian model selection favored a winning model that included modulations of connections to PrG (mSTG → PrG, FuG → PrG), from PrG (PrG → mSTG, PrG → FuG), and to pSTG (mSTG → pSTG, FuG → pSTG). Moreover, the coupling strength of the above connections correlated with behavioral McGurk susceptibility. In addition, significant differences were found in the coupling strength of these connections between strong and weak McGurk perceivers. Strong perceivers modulated less inhibitory visual influence, allowed less excitatory auditory information flowing into PrG, but integrated more audiovisual information in pSTG. Taken together, our findings show that the PrG and pSTG interact dynamically with primary cortices during audiovisual speech, and support the motor cortex plays a specifically functional role in modulating the gain and salience between auditory and visual modalities.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-023-09945-z.

Age-related changes to the attentional modulation of temporal binding

During multisensory integration, the time range within which visual and auditory information can be perceived as synchronous and bound together is known as the temporal binding window (TBW). With increasing age, the TBW becomes wider, such that older adults erroneously, and often dangerously, integrate sensory inputs that are asynchronous. Recent research suggests that attentional cues can narrow the width of the TBW in younger adults, sharpening temporal perception and increasing the accuracy of integration. However, due to their age-related declines in attentional control, it is not yet known whether older adults can deploy attentional resources to narrow the TBW in the same way as younger adults. This study investigated the age-related changes to the attentional modulation of the TBW. Thirty younger and 30 older adults completed a cued-spatial-attention version of the stream-bounce illusion, assessing the extent to which the visual and auditory stimuli were integrated when presented at three different stimulus-onset asynchronies, and when attending to a validly cued or invalidly cued location. A 2 × 2 × 3 mixed ANOVA revealed that when participants attended to the validly cued location (i.e., when attention was present), susceptibility to the stream-bounce illusion decreased. However, crucially, this attentional manipulation significantly affected audiovisual integration in younger adults, but not in older adults. These findings suggest that older adults have multisensory integration-related attentional deficits. Directions for future research and practical applications surrounding treatments to improve the safety of older adults' perception and navigation through the environment are discussed.

Somatosensory Event-Related Potential as an Electrophysiological Correlate of Endogenous Spatial Tactile Attention: Prospects for Electrotactile Brain-Computer Interface for Sensory Training

Tactile attention tasks are used in the diagnosis and treatment of neurological and sensory processing disorders, while somatosensory event-related potentials (ERP) measured by electroencephalography (EEG) are used as neural correlates of attention processes. Brain-computer interface (BCI) technology provides an opportunity for the training of mental task execution via providing online feedback based on ERP measures. Our recent work introduced a novel electrotactile BCI for sensory training, based on somatosensory ERP; however, no previous studies have addressed specific somatosensory ERP morphological features as measures of sustained endogenous spatial tactile attention in the context of BCI control. Here we show the morphology of somatosensory ERP responses induced by a novel task introduced within our electrotactile BCI platform i.e., the sustained endogenous spatial electrotactile attention task. By applying pulsed electrical stimuli to the two proximal stimulation hotspots at the user's forearm, stimulating sequentially the mixed branches of radial and median nerves with equal probability of stimuli occurrence, we successfully recorded somatosensory ERPs for both stimulation locations, in the attended and unattended conditions. Waveforms of somatosensory ERP responses for both mixed nerve branches showed similar morphology in line with previous reports on somatosensory ERP components obtained by stimulation of exclusively sensory nerves. Moreover, we found statistically significant increases in ERP amplitude on several components, at both stimulation hotspots, while sustained endogenous spatial electrotactile attention task is performed. Our results revealed the existence of general ERP windows of interest and signal features that can be used to detect sustained endogenous tactile attention and classify between spatial attention locations in 11 healthy subjects. The current results show that features of N140, P3a and P3b somatosensory ERP components are the most prominent global markers of sustained spatial electrotactile attention, over all subjects, within our novel electrotactile BCI task/paradigm, and this work proposes the features of those components as markers of sustained endogenous spatial tactile attention in online BCI control. Immediate implications of this work are the possible improvement of online BCI control within our novel electrotactile BCI system, while these finding can be used for other tactile BCI applications in the diagnosis and treatment of neurological disorders by employing mixed nerve somatosensory ERPs and sustained endogenous electrotactile attention task as control paradigms.

Age-related decrease in motor contribution to multisensory reaction times in primary school children

Traditional measurement of multisensory facilitation in tasks such as speeded motor reaction tasks (MRT) consistently show age-related improvement during early childhood. However, the extent to which motor function increases with age and hence contribute to multisensory motor reaction times in young children has seldom been examined. Thus, we aimed to investigate the contribution of motor development to measures of multisensory (auditory, visual, and audiovisual) and visuomotor processing tasks in three young school age groups of children (n = 69) aged (5-6, n = 21; 7-8, n = 25.; 9-10 n = 18 years). We also aimed to determine whether age-related sensory threshold times for purely visual inspection time (IT) tasks improved significantly with age. Bayesian results showed decisive evidence for age-group differences in multisensory MRT and visuo-motor processing tasks, though the evidence showed that threshold time for visual identification IT performance was only slower in the youngest age group children (5-6) compared to older groups. Bayesian correlations between performance on the multisensory MRT and visuo-motor processing tasks indicated moderate to decisive evidence in favor of the alternative hypothesis (BF10 = 4.71 to 91.346), though not with the threshold IT (BF10 < 1.35). This suggests that visual sensory system development in children older than 6 years makes a less significant contribution to the measure of multisensory facilitation, compared to motor development. In addition to this main finding, multisensory facilitation of MRT within race-model predictions was only found in the oldest group of children (9-10), supporting previous suggestions that multisensory integration is likely to continue into late childhood/early adolescence at least.

Lighting in Kindergartens: Towards Innovative Design Concepts for Lighting Design in Kindergartens Based on Children’s Perception of Space

Light is the foundation of the visual perceptual process that initiates the evaluation of the surrounding area. Linked to various aspects and rhythms of the body, light connects us to both the natural environment and the interior space. The process of perceiving and assessing space for children and adults with different viewing heights and viewing perspectives as well as the role of light to facilitate this are the key elements of this study. The paper describes general lighting design strategies for kindergartens, specifically developed to create an environment that takes into account the children’s scale and cognitive processes. The objective was to discuss environments that support the child’s spatial perception along with shape and object recognition by means of lighting design, for example by creating distinct, well-placed shadows. The proposed strategies are informed by a literature review on the concepts and interrelations of light, human physiology and the perception of scale and space. In addition, our process also included visits to kindergartens to observe and analyse existing lighting as well as the use of simulation programs to test lighting scenarios and their patterns/distribution of light and shadow. The outcome described in this paper is a proposal and strategy to take into account children’s vantage points when designing lighting in kindergartens that is still open to practical interpretations in real-world sites.

The feeling of "kiki": Comparing developmental changes in sound-shape correspondence for audio-visual and audio-tactile stimuli

Sound-shape crossmodal correspondence, the naturally occurring associations between abstract visual shapes and nonsense sounds, is one aspect of multisensory processing that strengthens across early childhood. Little is known regarding whether school-aged children exhibit other variants of sound-shape correspondences such as audio-tactile (AT) associations between tactile shapes and nonsense sounds. Based on previous research in blind individuals suggesting the role of visual experience in establishing sound-shape correspondence, we hypothesized that children would show weaker AT association than adults and that children's AT association would be enhanced with visual experience of the shapes. In Experiment 1, we showed that, when asked to match shapes explored haptically via touch to nonsense words, 6- to 8-year-olds exhibited inconsistent AT associations, whereas older children and adults exhibited the expected AT associations, despite robust audio-visual (AV) associations found across all age groups in a related study. In Experiment 2, we confirmed the role of visual experience in enhancing AT association; here, 6- to 8-year-olds could exhibit the expected AT association if first exposed to the AV condition, whereas adults showed the expected AT association irrespective of whether the AV condition was tested first or second. Our finding suggests that AT sound-shape correspondence is weak early in development relative to AV sound-shape correspondence, paralleling previous findings on the development of other types of multisensory associations. The potential role of visual experience in the development of sound-shape correspondences in other senses is discussed.

An Experimenter's Influence on Motor Enhancements: The Effects of Letter Congruency and Sensory Switch-Costs on Multisensory Integration

Multisensory integration can alter information processing, and previous research has shown that such processes are modulated by sensory switch costs and prior experience (e.g., semantic or letter congruence). Here we report an incidental finding demonstrating, for the first time, the interplay between these processes and experimental factors, specifically the presence (vs. absence) of the experimenter in the testing room. Experiment 1 demonstrates that multisensory motor facilitation in response to audiovisual stimuli (circle and tone with no prior learnt associations) is higher in those trials in which the sensory modality switches than when it repeats. Those participants who completed the study while alone exhibited increased RT variability. Experiment 2 replicated these findings using the letters “b” and “d” presented as unisensory stimuli or congruent and incongruent multisensory stimuli (i.e., grapheme-phoneme pairs). Multisensory enhancements were inflated following a sensory switch; that is, congruent and incongruent multisensory stimuli resulted in significant gains following a sensory switch in the monitored condition. However, when the participants were left alone, multisensory enhancements were only observed for repeating incongruent multisensory stimuli. These incidental findings therefore suggest that the effects of letter congruence and sensory switching on multisensory integration are partly modulated by the presence of an experimenter.

Neonatal Multisensory Processing in Preterm and Term Infants Predicts Sensory Reactivity and Internalizing Tendencies in Early Childhood

Multisensory processes include the capacity to combine information from the different senses, often improving stimulus representations and behavior. The extent to which multisensory processes are an innate capacity or instead require experience with environmental stimuli remains debated. We addressed this knowledge gap by studying multisensory processes in prematurely born and full-term infants. We recorded 128-channel event-related potentials (ERPs) from a cohort of 55 full-term and 61 preterm neonates (at an equivalent gestational age) in response to auditory, somatosensory, and combined auditory-somatosensory multisensory stimuli. Data were analyzed within an electrical neuroimaging framework, involving unsupervised topographic clustering of the ERP data. Multisensory processing in full-term infants was characterized by a simple linear summation of responses to auditory and somatosensory stimuli alone, which furthermore shared common ERP topographic features. We refer to the ERP topography observed in full-term infants as "typical infantile processing" (TIP). In stark contrast, preterm infants exhibited non-linear responses and topographies less-often characterized by TIP; there were distinct patterns of ERP topographies to multisensory and summed unisensory conditions. We further observed that the better TIP characterized an infant's ERPs, independently of prematurity, the more typical was the score on the Infant/Toddler Sensory Profile (ITSP) at 12 months of age and the less likely was the child to the show internalizing tendencies at 24 months of age. Collectively, these results highlight striking differences in the brain's responses to multisensory stimuli in children born prematurely; differences that relate to later sensory and internalizing functions.

Problems in Audiovisual Filtering for Children with Special Educational Needs

There is pervasive evidence that problems in sensory processing occur across a range of developmental disorders, but their aetiology and clinical significance remain unclear. The present study investigated the relation between sensory processing and literacy skills in children with and without a background of special educational needs (SEN). Twenty-six children aged between 7 and 12 years old, from both regular classes and SEN programmes, participated. Following baseline tests of literacy, fine motor skills and naming speed, two sets of instruments were administered: the carer-assessed Child Sensory Profile-2 and a novel Audiovisual Animal Stroop (AVAS) test. The SEN group showed significantly higher ratings on three Child Sensory Profile-2 quadrants, together with body position ratings. The SEN participants also showed a specific deficit when required to ignore an accompanying incongruent auditory stimulus on the AVAS. Interestingly, AVAS performance correlated significantly with literacy scores and with the sensory profile scores. It is proposed that the children with SEN showed a specific deficit in "filtering out" irrelevant auditory input. The results highlight the importance of including analysis of sensory processes within theoretical and applied approaches to developmental differences and suggest promising new approaches to the understanding, assessment, and support of children with SEN.