Multisensory Integration: How Visual Experience Shapes Spatial Perception

Child development and the role of visual experience in the use of spatial and non-spatial features in haptic object perception

Previous work has suggested a different developmental timeline and role of visual experience for the use of spatial and non-spatial features in haptic object recognition. To investigate this conjecture, we used a haptic ambiguous odd-one-out task in which one object needed to be selected as being different from two other objects. The odd-one-out could be selected based on four characteristics: size, shape (spatial), texture, and weight (non-spatial). We tested sighted children from 4 to 12 years of age; congenitally blind, late blind, and adult participants with low vision; and normally sighted adults. Given the protracted developmental time course for spatial perception, we expected a shift from a preference for non-spatial features toward spatial features during typical development. Due to the dominant influence of vision for spatial perception, we expected congenitally blind adults to show a similar preference for non-spatial features as the youngest children. The results confirmed our first hypothesis; the 4-year-olds demonstrated a lower dominance for spatial features for object classification compared with older children and sighted adults. In contrast, our second hypothesis was not confirmed; congenitally blind adults' preferred categorization criteria were indistinguishable from those of sighted controls. These findings suggest an early development, but late maturation, of spatial processing in haptic object recognition independent of visual experience.

Developmental changes in the visual, haptic, and bimodal perception of geometric angles

Geometrical knowledge is typically taught to children through a combination of vision and repetitive drawing (i.e. haptics), yet our understanding of how different spatial senses contribute to geometric perception during childhood is poor. Studies of line orientation suggest a dominant role of vision affecting the calibration of haptics during development; however, the associated multisensory interactions underpinning angle perception are unknown. Here we examined visual, haptic, and bimodal perception of angles across three age groups of children: 6 to 8 years, 8 to 10 years, and 10 to 12 years, with age categories also representing their class (grade) in primary school. All participants first learned an angular shape, presented dynamically, in one of three sensory tracing conditions: visual only, haptic only, or bimodal exploration. At test, which was visual only, participants selected a target angle from four possible alternatives with distractor angle sizes varying relative to the target angle size. We found a clear improvement in accuracy of angle perception with development for all learning modalities. Angle perception in the youngest group was equally poor (but above chance) for all modalities; however, for the two older child groups, visual learning was better than haptics. Haptic perception did not improve to the level of vision with age (even in a comparison adult group), and we found no specific benefit for bimodal learning over visual learning in any age group, including adults. Our results support a developmental increment in both spatial accuracy and precision in all modalities, which was greater in vision than in haptics, and are consistent with previous accounts of cross-sensory calibration in the perception of geometric forms.

Dissecting abstract, modality-specific and experience-dependent coding of affect in the human brain

Emotion and perception are tightly intertwined, as affective experiences often arise from the appraisal of sensory information. Nonetheless, whether the brain encodes emotional instances using a sensory-specific code or in a more abstract manner is unclear. Here, we answer this question by measuring the association between emotion ratings collected during a unisensory or multisensory presentation of a full-length movie and brain activity recorded in typically developed, congenitally blind and congenitally deaf participants. Emotional instances are encoded in a vast network encompassing sensory, prefrontal, and temporal cortices. Within this network, the ventromedial prefrontal cortex stores a categorical representation of emotion independent of modality and previous sensory experience, and the posterior superior temporal cortex maps the valence dimension using an abstract code. Sensory experience more than modality affects how the brain organizes emotional information outside supramodal regions, suggesting the existence of a scaffold for the representation of emotional states where sensory inputs during development shape its functioning.

Visuospatial cognition in acute unilateral peripheral vestibulopathy

Background

This study aims to investigate the presence of spatial cognitive impairments in patients with acute unilateral peripheral vestibulopathy (vestibular neuritis, AUPV) during both the acute phase and the recovery phase.

Methods

A total of 72 AUPV patients (37 with right-sided AUPV and 35 with left-sided AUPV; aged 34-80 years, median 60.5; 39 males, 54.2%) and 35 healthy controls (HCs; aged 43-75 years, median 59; 20 males, 57.1%) participated in the study. Patients underwent comprehensive neurotological assessments, including video-oculography, video head impulse and caloric tests, ocular and cervical vestibular-evoked myogenic potentials, and pure-tone audiometry. Additionally, the Visual Object and Space Perception (VOSP) battery was used to evaluate visuospatial perception, while the Block design test and Corsi block-tapping test assessed visuospatial memory within the first 2 days (acute phase) and 4 weeks after symptom onset (recovery phase).

Results

Although AUPV patients were able to successfully perform visuospatial perception tasks within normal parameters, they demonstrated statistically worse performance on the visuospatial memory tests compared to HCs during the acute phase. When comparing right versus left AUPV groups, significant decreased scores in visuospatial perception and memory were observed in the right AUPV group relative to the left AUPV group. In the recovery phase, patients showed substantial improvements even in these previously diminished visuospatial cognitive performances.

Conclusion

AUPV patients showed different spatial cognition responses, like spatial memory, depending on the affected ear, improving with vestibular compensation over time. We advocate both objective and subjective visuospatial assessments and the development of tests to detect potential cognitive deficits after unilateral vestibular impairments.

Late Blindness and Deafness are Associated with Decreased Tactile Sensitivity, But Early Blindness is Not

Perceptual experience is shaped by a complex interaction between our sensory systems in which each sense conveys information on specific properties of our surroundings. This multisensory processing of complementary information improves the accuracy of our perceptual judgments and leads to more precise and faster reactions. Sensory impairment or loss in one modality leads to information deficiency that can impact other senses in various ways. For early auditory or visual loss, impairment and/or compensatory increase of the sensitivity of other senses are equally well described. Investigating individuals with deafness (N = 73), early (N = 51), late blindness (N = 49) and corresponding controls, we compared tactile sensitivity using the standard monofilament test on two locations, the finger and handback. Results indicate lower tactile sensitivity in people with deafness and late blindness but not in people with early blindness compared to respective controls, irrespective of stimulation location, gender, and age. Results indicate that neither sensory compensation nor simple use-dependency or a hindered development of the tactile sensory system is sufficient to explain changes in somatosensation after the sensory loss but that a complex interaction of effects is present.

Clinical assessment of the TechArm system on visually impaired and blind children during uni- and multi-sensory perception tasks

We developed the TechArm system as a novel technological tool intended for visual rehabilitation settings. The system is designed to provide a quantitative assessment of the stage of development of perceptual and functional skills that are normally vision-dependent, and to be integrated in customized training protocols. Indeed, the system can provide uni- and multisensory stimulation, allowing visually impaired people to train their capability of correctly interpreting non-visual cues from the environment. Importantly, the TechArm is suitable to be used by very young children, when the rehabilitative potential is maximal. In the present work, we validated the TechArm system on a pediatric population of low-vision, blind, and sighted children. In particular, four TechArm units were used to deliver uni- (audio or tactile) or multi-sensory stimulation (audio-tactile) on the participant's arm, and subject was asked to evaluate the number of active units. Results showed no significant difference among groups (normal or impaired vision). Overall, we observed the best performance in tactile condition, while auditory accuracy was around chance level. Also, we found that the audio-tactile condition is better than the audio condition alone, suggesting that multisensory stimulation is beneficial when perceptual accuracy and precision are low. Interestingly, we observed that for low-vision children the accuracy in audio condition improved proportionally to the severity of the visual impairment. Our findings confirmed the TechArm system's effectiveness in assessing perceptual competencies in sighted and visually impaired children, and its potential to be used to develop personalized rehabilitation programs for people with visual and sensory impairments.

Early visual deprivation disrupts the mental representation of numbers in visually impaired children

Several shreds of evidence indicate that visual deprivation does not alter numerical competence neither in adults nor in children. However, studies reporting non-impaired numerical abilities in the visually impaired population present some limitations: (a) they mainly assessed the ability to process numbers (e.g. mathematical competence) rather than represent numbers (e.g. mental number line); (b) they principally focused on positive rather than negative number estimates; (c) they investigated numerical abilities in adult individuals except one focusing on children (Crollen et al. in Cognition 210:104586, 2021). Overall, this could limit a comprehensive explanation of the role exerted by vision on numerical processing when vision is compromised. Here we investigated how congenital visual deprivation affects the ability to represent positive and negative numbers in horizontal and sagittal planes in visually impaired children (thirteen children with low vision, eight children with complete blindness, age range 6-15 years old). We adapted the number-to-position paradigm adopted by Crollen et al. (Cognition 210:104586, 2021), asking children to indicate the spatial position of positive and negative numbers on a graduated rule positioned horizontally or sagittally in the frontal plane. Results suggest that long-term visual deprivation alters the ability to identify the spatial position of numbers independently of the spatial plane and the number polarity. Moreover, results indicate that relying on poor visual acuity is detrimental for low vision children when asked to localize both positive and negative numbers in space, suggesting that visual experience might have a differential role in numerical processing depending on number polarity. Such findings add knowledge related to the impact of visual experience on numerical processing. Since both positive and negative numbers are fundamental aspects of learning mathematical principles, the outcomes of the present study inform about the need to implement early rehabilitation strategies to prevent the risk of numerical difficulties in visually impaired children.

Variables influencing conditioning-evoked hallucinations: overview and future applications

Hallucinations occur in the absence of sensory stimulation and result in vivid perceptual experiences of nonexistent events that manifest across a range of sensory modalities. Approaches from the field of experimental and cognitive psychology have leveraged the idea that associative learning experiences can evoke conditioning-induced hallucinations in both animals and humans. In this review, we describe classical and contemporary findings and highlight the variables eliciting these experiences. We also provide an overview of the neurobiological mechanisms, along with the associative and computational factors that may explain hallucinations that are generated by representation-mediated conditioning phenomena. Through the integration of animal and human research, significant advances into the psychobiology of hallucinations are possible, which may ultimately translate to more effective clinical applications.

Sensory processing in young children with visual impairments: Use and extension of the Sensory Profile

BACKGROUND

Children with visual impairments (VI) are at risk for sensory processing difficulties. A widely used measure for sensory processing is the Sensory Profile (SP). However, the SP requires adaptation to accommodate for how children with VI experience sensory information.

AIMS

(1) To examine sensory processing patterns in young children with VI, (2) to develop VI-specific items to use in conjunction with the SP and to determine internal consistency and construct validity of these newly developed items, and (3) to examine the association between sensory processing and and emotional and behavioral problems.

METHODS

Twenty-six VI-specific items were added to the SP. The SP and these items were completed by caregivers of 90 children with VI between 3 and 8 years old. The Child Behavior Checklist (CBCL) was used to assess emotional and behavioral problems.

RESULTS

Three- to five-year-old children with VI have significantly more difficulties in three quadrants of the SP as compared to the norm group. Six- to eight-year-old children with VI have more difficulties in all quadrants. A reliable and valid VI-specific set of 15 items was established following psychometric evaluation. Age-related differences were found in the associations between the SP and CBCL.

CONCLUSION

Although further validation is recommended, this evaluation of the VI-specific item set suggests it has the potential to be a useful measure for children with VI.

Object Localization Assistive System Based on CV and Vibrotactile Encoding

Intelligent assistive systems can navigate blind people, but most of them could only give non-intuitive cues or inefficient guidance. Based on computer vision and vibrotactile encoding, this paper presents an interactive system that provides blind people with intuitive spatial cognition. Different from the traditional auditory feedback strategy based on speech cues, this paper firstly introduces a vibration-encoded feedback method that leverages the haptic neural pathway and enables the users to interact with objects other than manipulating an assistance device. Based on this strategy, a wearable visual module based on an RGB-D camera is adopted for 3D spatial object localization, which contributes to accurate perception and quick object localization in the real environment. The experimental results on target blind individuals indicate that vibrotactile feedback reduces the task completion time by over 25% compared with the mainstream voice prompt feedback scheme. The proposed object localization system provides a more intuitive spatial navigation and comfortable wearability for blindness assistance.

StereoPilot: A Wearable Target Location System for Blind and Visually Impaired Using Spatial Audio Rendering

Vision loss severely impacts object recognition and spatial cognition for limited vision individuals. It is a challenge to compensate for this using other sensory modalities, such as touch or hearing. This paper introduces StereoPilot, a wearable target location system to facilitate the spatial cognition of BVI. Through wearing a head-mounted RGB-D camera, the 3D spatial information of the environment is measured and processed into navigation cues. Leveraging spatial audio rendering (SAR) technology, it allows the navigation cues to be transmitted in a type of 3D sound from which the sound orientation can be distinguished by the sound localization instincts in humans. Three haptic and auditory display strategies were compared with SAR through experiments with three BVI and four sighted subjects. Compared with mainstream speech instructional feedback, the experimental results of the Fitts' law test showed that SAR increases the information transfer rate (ITR) by a factor of three for spatial navigation, while the positioning error is reduced by 40%. Furthermore, SAR has a lower learning effect than other sonification approaches such as vOICe. In desktop manipulation experiments, StereoPilot was able to obtain precise localization of desktop objects while reducing the completion time of target grasping tasks in half as compared to the voice instruction method. In summary, StereoPilot provides an innovative wearable target location solution that swiftly and intuitively transmits environmental information to BVI individuals in the real world.

Multisensory Rather than Unisensory Representations Contribute to Statistical Context Learning in Tactile Search

Using a combination of behavioral and EEG measures in a tactile odd-one-out search task with collocated visual items, we investigated the mechanisms underlying facilitation of search by repeated (vs. nonrepeated) spatial distractor-target configurations ("contextual cueing") when either the tactile (same-modality) or the visual array (different-modality) context was predictive of the location of the tactile singleton target. Importantly, in both conditions, the stimulation was multisensory, consisting of tactile plus visual items, although the target was singled out in the tactile modality and so the visual items were task-irrelevant. We found that when the predictive context was tactile, facilitation of search RTs by repeated configurations was accompanied by, and correlated with, enhanced lateralized ERP markers of pre-attentive (N1, N2) and, respectively focal-attentional processing (contralateral delay activity) not only over central ("somatosensory"), but also posterior ("visual") electrode sites, although the ERP effects were less marked over visual cortex. A similar pattern-of facilitated RTs and enhanced lateralized (N2 and contralateral delay activity) ERP components-was found when the predictive context was visual, although the ERP effects were less marked over somatosensory cortex. These findings indicate that both somatosensory and visual cortical regions contribute to the more efficient processing of the tactile target in repeated stimulus arrays, although their involvement is differentially weighted depending on the sensory modality that contains the predictive information.

Deprivation of Auditory Experience Influences Numerosity Discrimination, but Not Numerosity Estimation

Number sense is the ability to estimate the number of items, and it is common to many species. Despite the numerous studies dedicated to unveiling how numerosity is processed in the human brain, to date, it is not clear whether the representation of numerosity is supported by a single general mechanism or by multiple mechanisms. Since it is known that deafness entails a selective impairment in the processing of temporal information, we assessed the approximate numerical abilities of deaf individuals to disentangle these two hypotheses. We used a numerosity discrimination task (2AFC) and an estimation task, in both cases using sequential (temporal) or simultaneous (spatial) stimuli. The results showed a selective impairment of the deaf participants compared with the controls (hearing) in the temporal numerosity discrimination task, while no difference was found to discriminate spatial numerosity. Interestingly, the deaf and hearing participants did not differ in spatial or temporal numerosity estimation. Overall, our results suggest that the deficit in temporal processing induced by deafness also impacts perception in other domains such as numerosity, where sensory information is conveyed in a temporal format, which further suggests the existence of separate mechanisms subserving the processing of temporal and spatial numerosity.

Geometry intuitions without vision? A study in blind children and adults

Geometry intuitions seem to be rooted in a non-verbal system that humans possess since early age. However, the mechanisms underlying the comprehension of basic geometric concepts remain elusive. Some authors have suggested that the starting point of geometry development could be found in the visual perception of specific features in our environment, thus conferring to vision a foundational role in the acquisition of geometric skills. To examine this assumption, a test probing intuitive understanding of basic geometric concepts was presented to congenitally blind children and adults. Participants had to detect the intruder among four different shapes, from which three instantiated a specific geometrical concept and one (the intruder) violated it. Although they performed above the chance level, the blind presented poorer performance than the sighted participants who did the task in the visual modality (i.e., with the eyes open), but performed equally well than the sighted who did the task in the tactile modality (i.e., with a blindfold). We therefore provide evidence that geometric abilities are impacted by the lack of vision.

Standardized and Experimental Tools to Assess Spatial Cognition in Visually Impaired Children: A Mini-Review

The acquisition of spatial cognition is essential for both everyday functioning (e.g., navigation) and more specific goals (e.g., mathematics), therefore being able to assess and monitor spatial cognition from the first years of life would be essential to predict developmental outcomes and timely intervene whenever spatial development is compromised. Several shreds of evidence have indicated that spatial development can be compromised in the case of development with atypical sensory experience such as blindness. Despite the massive importance of spatial abilities for the development of psychomotor competencies across childhood, only a few standardized and experimental methods have been developed to assess them in visually impaired children. In this review, we will give a short overview of current formal (standardized) and informal (experimental) methods to assess spatial cognition in visually impaired children, demonstrating that very few validated tools have been proposed to date. The main contribution of this current work is to highlight the need of ad hoc studies to create and validate clinical measures to assess spatial cognition in visually impaired individuals and address potential future developments in this area of research.

Spatial and Ecological Farmer Knowledge and Decision-Making about Ecosystem Services and Biodiversity

Amid climate change, biodiversity loss and food insecurity, there is the growing need to draw synergies between micro-scale environmental processes and practices, and macro-level ecosystem dynamics to facilitate conservation decision-making. Adopting this synergistic approach can improve crop yields and profitability more sustainably, enhance livelihoods and mitigate climate change. Using spatially explicit data generated through a public participatory geographic information system methodology (n = 37), complemented by spatial analysis, interviews (n = 68) and focus group discussions (n = 4), we explored the synergies between participatory farmer-to-farmer agroecology knowledge sharing, farm-level decisions and their links with macro-level prioritization of conservation strategies. We mapped farm conditions and ecosystem services (ES) of two village areas with varying knowledge systems about farming. Results of the farm-level analysis revealed variations in spatial perception among farmers, differences in understanding the dynamics of crop growth and varying priorities for extension services based on agroecological knowledge. The ES use pattern analysis revealed hotspots in the mapped ES indicators with similarities in both village areas. Despite the similarities in ES use, priorities for biodiversity conservation align with farmers’ understanding of farm processes and practices. Farmers with training in agroecology prioritized strategies that are ecologically friendly while farmers with no agroecology training prioritized the use of strict regulations. Importantly, the results show that agroecology can potentially contribute to biodiversity conservation and food security, with climate change mitigation co-benefits. The findings generally contribute to debates on land sparing and land sharing conservation strategies and advance social learning theory as it pertains to acquiring agroecological knowledge for improved yield and a sustainable environment.

A Multidimensional, Multisensory and Comprehensive Rehabilitation Intervention to Improve Spatial Functioning in the Visually Impaired Child: A Community Case Study

Congenital visual impairment may have a negative impact on spatial abilities and result in severe delays in perceptual, social, motor, and cognitive skills across life span. Despite several evidences have highlighted the need for an early introduction of re-habilitation interventions, such interventions are rarely adapted to children’s visual capabilities and very few studies have been conducted to assess their long-term efficacy. In this work, we present a case study of a visually impaired child enrolled in a newly developed re-habilitation intervention aimed at improving the overall development through the diversification of re-habilitation activities based on visual potential and developmental profile, with a focus on spatial functioning. We argue that intervention for visually impaired children should be (a) adapted to their visual capabilities, in order to increase re-habilitation outcomes, (b) multi-interdisciplinary and multidimensional, to improve adaptive abilities across development, (c) multisensory, to promote the integration of different perceptual information coming from the environment.

The Effects of an Ecological Diversifying Experience on Creativity: An Experimental Study

Sometimes, life houses rare and unexpected events, such as moving abroad or meeting a special person unexpectedly. Recently, these situations have been indicated as "diversifying experiences" (DEs), defined as unusual and unexpected events that drag people outside their daily routine and accustomed schemas. The core mechanism of DEs would entail the disruption of our mental schema, which can facilitate unexpected connections among even distant ideas, thus enhancing people's cognitive flexibility, that is, a key component of creative thinking. Despite both qualitative and lab-based studies have investigated the features of these experiences, an ecological assessment of their properties also in relation with creativity is still an open issue. The aim of this research is to study the DE-creativity link in a more ecological way, on the basis of a real-life disruptive experience of light deprivation. Specifically, we compared an ecological DE artistic established entertainment format (i.e., "dialogue in the dark," which is seeing people perform several daily life activities but in the absence of light) with an equivalent experience in which the same activities were acted in the sunlight. The absence of light played the role of violating mechanism, framed within the ecological experiential format of the "dialogue in the dark." We compared visitors' emotional profile [Positive and Negative Affect Schedule (PANAS), ad hoc Adjective Checklist], perceived impact of the experience [Centrality of Event Scale (CES)], and creative performance [Torrance Tests of Creative Thinking (TTCT)] in both groups of sighted people (in absence of light vs. in presence of light); and we also controlled for people's openness to experience and need for cognitive closure, as dispositions. Results showed that (vs. control group) "dialogue in the dark" (i) led to worse creative performances, (ii) produced more intense positive affect, and (iii) resulted as a more impacting experience. Intense short-term impact of DE could have been detrimental for participants' creativity. People may need more time to elaborate the DE and accommodate existing schema to generate more creative ideas. This is the first study proposing and succeeding in demonstrating the feasibility to investigate even real complex DEs in a controlled way, thus outlining how their link with creativity can take place in real life.

Visual vs vibrotactile feedback for posture assessment during upper-limb robot-aided rehabilitation

Repetitive and intensive exercises during robot-aided rehabilitation may expose patients to inappropriate and unsafe postures. The introduction of a sensory feedback can help the subject to perform the rehabilitation task with an ergonomic posture. In this work, the introduction of visual and vibrotactile feedback in a robotic platform for upper limb rehabilitation has been proposed to ensure ergonomic posture during rehabilitation. The two feedback modalities have been used to provide information about incorrect neck and trunk posture. Ten healthy subjects have been involved in this study. Each of them performed 3D reaching movements with the aid of the robotic platform in three different conditions, i.e. without feedback, with visual feedback and with vibrotactile feedback, and a comparative analysis has been carried out to evaluate feedback effectiveness, acceptance and performance. Experimental results show that in case of no feedback the subjects reach and maintain configurations that can lead to incorrect neck and trunk configurations and therefore, if repeated, to musculoskeletal disorders. Conversely, with visual or vibrotactile feedback, the subjects tend to correct inappropriate posture with both trunk and head during task performing.

Distance perception during self-movement

The perception of distance in open fields was widely studied with static observers. However, it is a fact that we and the world around us are in continuous relative movement, and that our perceptual experience is shaped by the complex interactions between our senses and the perception of our self-motion. This poses interesting questions about how our nervous system integrates this multisensory information to resolve specific tasks of our daily life, for example, distance estimation. This study provides new evidence about how visual and motor self-motion information affects our perception of distance and a hypothesis about how these two sources of information can be integrated to calibrate the estimation of distance. This model accounts for the biases found when visual and proprioceptive information is inconsistent.

Reaching measures and feedback effects in auditory peripersonal space

We analyse the effects of exploration feedback on reaching measures of perceived auditory peripersonal space (APS) boundary and the auditory distance perception (ADP) of sound sources located within it. We conducted an experiment in which the participants had to estimate if a sound source was (or not) reachable and to estimate its distance (40 to 150 cm in 5-cm steps) by reaching to a small loudspeaker. The stimulus consisted of a train of three bursts of Gaussian broadband noise. Participants were randomly assigned to two groups: Experimental (EG) and Control (CG). There were three phases in the following order: Pretest-Test-Posttest. For all phases, the listeners performed the same task except for the EG-Test phase where the participants reach in order to touch the sound source. We applied models to characterise the participants' responses and provide evidence that feedback significantly reduces the response bias of both the perceived boundary of the APS and the ADP of sound sources located within reach. In the CG, the repetition of the task did not affect APS and ADP accuracy, but it improved the performance consistency: the reachable uncertainty zone in APS was reduced and there was a tendency to decrease variability in ADP.

Audio Feedback Associated With Body Movement Enhances Audio and Somatosensory Spatial Representation

In the last years, the positive impact of sensorimotor rehabilitation training on spatial abilities has been taken into account, e.g., providing evidence that combined multimodal compared to unimodal feedback improves responsiveness to spatial stimuli. To date, it still remains unclear to which extent spatial learning is influenced by training conditions. Here we investigated the effects of active and passive audio-motor training on spatial perception in the auditory and proprioceptive domains on 36 healthy young adults. First, to investigate the role of voluntary movements on spatial perception, we compared the effects of active vs. passive multimodal training on auditory and proprioceptive spatial localization. Second, to investigate the effectiveness of unimodal training conditions on spatial perception, we compared the impact of only proprioceptive or only auditory sensory feedback on spatial localization. Finally, to understand whether the positive effects of multimodal and unimodal trainings generalize to the untrained part, both dominant and non-dominant arms were tested. Results indicate that passive multimodal training (guided movement) is more beneficial than active multimodal training (active exploration) and only in passive condition the improvement is generalized also on the untrained hand. Moreover, we found that combined audio-motor training provides the strongest benefit because it significantly affects both auditory and somatosensory localization, while the effect of a single feedback modality is limited to a single domain, indicating a cross-modal influence of the two domains. Therefore, the use of multimodal feedback is more efficient in improving spatial perception. These results indicate that combined sensorimotor signals are effective in recalibrating auditory and proprioceptive spatial perception and that the beneficial effect is mainly due to the combination of auditory and proprioceptive spatial cues.

The Impact of Early Visual Deprivation on Spatial Hearing: A Comparison between Totally and Partially Visually Deprived Children

The specific role of early visual deprivation on spatial hearing is still unclear, mainly due to the difficulty of comparing similar spatial skills at different ages and to the difficulty in recruiting young blind children from birth. In this study, the effects of early visual deprivation on the development of auditory spatial localization have been assessed in a group of seven 3–5 years old children with congenital blindness (n = 2; light perception or no perception of light) or low vision (n = 5; visual acuity range 1.1–1.7 LogMAR), with the main aim to understand if visual experience is fundamental to the development of specific spatial skills. Our study led to three main findings: firstly, totally blind children performed overall more poorly compared sighted and low vision children in all the spatial tasks performed; secondly, low vision children performed equally or better than sighted children in the same auditory spatial tasks; thirdly, higher residual levels of visual acuity are positively correlated with better spatial performance in the dynamic condition of the auditory localization task indicating that the more residual vision the better spatial performance. These results suggest that early visual experience has an important role in the development of spatial cognition, even when the visual input during the critical period of visual calibration is partially degraded like in the case of low vision children. Overall these results shed light on the importance of early assessment of spatial impairments in visually impaired children and early intervention to prevent the risk of isolation and social exclusion.

Uni- and Bimodal Threat Cueing with Vibrotactile and 3D Audio Technologies in a Combat Vehicle

An experiment investigated uni- and bimodal cueing of horizontal threat directions to the driver of the Combat Vehicle 90 (CV 90). 3D audio in headphones or tactile vibration onto the torso, or both in combination were utilized for threat cueing. Ten male CV 90 drivers from the Swedish Army Combat School were required to turn the vehicle toward the threat as quickly and accurate as possible. Each driver handled 45 threats in total. Each of five threat directions was presented three times for each of the three threat cueing conditions. The results show that the drivers had good overall performance, albeit the 3D audio needs improvement with regard to front - back confusion if not used with tactile cueing. That is, the 3D audio generated greater localization errors and reaction times with threats straight behind the vehicle compared with the tactile and the 3D audio/tactile combination, respectively.

Touching and Hearing Unseen Objects: Multisensory Effects on Scene Recognition

In three experiments, we investigated the influence of object-specific sounds on haptic scene recognition without vision. Blindfolded participants had to recognize, through touch, spatial scenes comprising six objects that were placed on a round platform. Critically, in half of the trials, object-specific sounds were played when objects were touched (bimodal condition), while sounds were turned off in the other half of the trials (unimodal condition). After first exploring the scene, two objects were swapped and the task was to report, which of the objects swapped positions. In Experiment 1, geometrical objects and simple sounds were used, while in Experiment 2, the objects comprised toy animals that were matched with semantically compatible animal sounds. In Experiment 3, we replicated Experiment 1, but now a tactile-auditory object identification task preceded the experiment in which the participants learned to identify the objects based on tactile and auditory input. For each experiment, the results revealed a significant performance increase only after the switch from bimodal to unimodal. Thus, it appears that the release of bimodal identification, from audio-tactile to tactile-only produces a benefit that is not achieved when having the reversed order in which sound was added after having experience with haptic-only. We conclude that task-related factors other than mere bimodal identification cause the facilitation when switching from bimodal to unimodal conditions.

Auditory and proprioceptive spatial impairments in blind children and adults

It is not clear what role visual information plays in the development of space perception. It has previously been shown that in absence of vision, both the ability to judge orientation in the haptic modality and bisect intervals in the auditory modality are severely compromised (Gori, Sandini, Martinoli & Burr, 2010; Gori, Sandini, Martinoli & Burr, 2014). Here we report for the first time also a strong deficit in proprioceptive reproduction and audio distance evaluation in early blind children and adults. Interestingly, the deficit is not present in a small group of adults with acquired visual disability. Our results support the idea that in absence of vision the audio and proprioceptive spatial representations may be delayed or drastically weakened due to the lack of visual calibration over the auditory and haptic modalities during the critical period of development.

Prestimulus oscillatory alpha power and connectivity patterns predispose perceptual integration of an audio and a tactile stimulus

To efficiently perceive and respond to the external environment, our brain has to perceptually integrate or segregate stimuli of different modalities. The temporal relationship between the different sensory modalities is therefore essential for the formation of different multisensory percepts. In this magnetoencephalography study, we created a paradigm where an audio and a tactile stimulus were presented by an ambiguous temporal relationship so that perception of physically identical audiotactile stimuli could vary between integrated (emanating from the same source) and segregated. This bistable paradigm allowed us to compare identical bimodal stimuli that elicited different percepts, providing a possibility to directly infer multisensory interaction effects. Local differences in alpha power over bilateral inferior parietal lobules (IPLs) and superior parietal lobules (SPLs) preceded integrated versus segregated percepts of the two stimuli (audio and tactile). Furthermore, differences in long-range cortical functional connectivity seeded in rIPL (region of maximum difference) revealed differential patterns that predisposed integrated or segregated percepts encompassing secondary areas of all different modalities and prefrontal cortex. We showed that the prestimulus brain states predispose the perception of the audiotactile stimulus both in a global and a local manner. Our findings are in line with a recent consistent body of findings on the importance of prestimulus brain states for perception of an upcoming stimulus. This new perspective on how stimuli originating from different modalities are integrated suggests a non-modality specific network predisposing multisensory perception.

Sensory Imagery in Individuals who Are Blind and Sighted: Examining Unimodal and Multimodal Forms

Introduction

Previous research has suggested that visual images are more easily generated, more vivid, and more memorable than other sensory modalities. This research examined whether or not imagery is experienced in similar ways by people with and without sight. Specifically, the imageability of visual, auditory, and tactile cue words was compared. The degree to which images were multimodal or unimodal was also examined.

Methods

Twelve participants who were totally blind from early infancy and 12 sighted participants generated images in response to 53 sensory and nonsensory words, rating imageability and the sensory modality, and describing images. From these 53 items, 4 subgroups of words that stimulated images that were predominantly visual, tactile, auditory, and low-imagery were created.

Results

T-tests comparing imageability ratings from blind and sighted participants found no differences for auditory and tactile words (both p > .1). Nevertheless, although participants without sight found auditory and tactile images equally imageable, sighted participants found images in response to tactile cue words harder to generate than visual cue words (mean difference: −0.51, p = .025). Participants with sight were also more likely to develop multisensory images than were participants without sight (both U ≥ 15.0, N1 = 12, N2 = 12, p ≤.008).

Discussion

For both the blind and sighted groups, auditory and tactile images were rich and varied, and similar language was used. Sighted participants were more likely to generate multimodal images, and this was particularly the case for tactile words. Nevertheless, cue words that resulted in multisensory images were not necessarily rated as more imageable. The discussion considers whether or not multimodal imagery represents a method of compensating for impoverished unimodal imagery.

Implications for practitioners

Imagery is important not only as a mnemonic in memory rehabilitation, but also in everyday uses for modes such as autobiographical memory. This research emphasizes the importance of not only auditory and tactile sensory imagery, but also spatial imagery for people without sight.

Resting state functional connectivity in early blind humans

Task-based neuroimaging studies in early blind humans (EB) have demonstrated heightened visual cortex responses to non-visual paradigms. Several prior functional connectivity studies in EB have shown altered connections consistent with these task-based results. But these studies generally did not consider behavioral adaptations to lifelong blindness typically observed in EB. Enhanced cognitive abilities shown in EB include greater serial recall and attention to memory. Here, we address the question of the extent to which brain intrinsic activity in EB reflects such adaptations. We performed a resting-state functional magnetic resonance imaging study contrasting 14 EB with 14 age/gender matched normally sighted controls (NS). A principal finding was markedly greater functional connectivity in EB between visual cortex and regions typically associated with memory and cognitive control of attention. In contrast, correlations between visual cortex and non-deprived sensory cortices were significantly lower in EB. Thus, the available data, including that obtained in prior task-based and resting state fMRI studies, as well as the present results, indicate that visual cortex in EB becomes more heavily incorporated into functional systems instantiating episodic recall and attention to non-visual events. Moreover, EB appear to show a reduction in interactions between visual and non-deprived sensory cortices, possibly reflecting suppression of inter-sensory distracting activity.

Improvement in spatial imagery following sight onset late in childhood

The factors contributing to the development of spatial imagery skills are not well understood. Here, we consider whether visual experience shapes these skills. Although differences in spatial imagery between sighted and blind individuals have been reported, it is unclear whether these differences are truly due to visual deprivation or instead are due to extraneous factors, such as reduced opportunities for the blind to interact with their environment. A direct way of assessing vision's contribution to the development of spatial imagery is to determine whether spatial imagery skills change soon after the onset of sight in congenitally blind individuals. We tested 10 children who gained sight after several years of congenital blindness and found significant improvements in their spatial imagery skills following sight-restoring surgeries. These results provide evidence of vision's contribution to spatial imagery and also have implications for the nature of internal spatial representations.

How does experience modulate auditory spatial processing in individuals with blindness?

Comparing early- and late-onset blindness in individuals offers a unique model for studying the influence of visual experience on neural processing. This study investigated how prior visual experience would modulate auditory spatial processing among blind individuals. BOLD responses of early- and late-onset blind participants were captured while performing a sound localization task. The task required participants to listen to novel “Bat-ears” sounds, analyze the spatial information embedded in the sounds, and specify out of 15 locations where the sound would have been emitted. In addition to sound localization, participants were assessed on visuospatial working memory and general intellectual abilities. The results revealed common increases in BOLD responses in the middle occipital gyrus, superior frontal gyrus, precuneus, and precentral gyrus during sound localization for both groups. Between-group dissociations, however, were found in the right middle occipital gyrus and left superior frontal gyrus. The BOLD responses in the left superior frontal gyrus were significantly correlated with accuracy on sound localization and visuospatial working memory abilities among the late-onset blind participants. In contrast, the accuracy on sound localization only correlated with BOLD responses in the right middle occipital gyrus among the early-onset counterpart. The findings support the notion that early-onset blind individuals rely more on the occipital areas as a result of cross-modal plasticity for auditory spatial processing, while late-onset blind individuals rely more on the prefrontal areas which subserve visuospatial working memory.

Visual sensory disability: rehabilitative treatment in an aquatic environment

The outcome of this study is based on the concept of the enormous potentiality that is expressed, after the elimination of myofascial and articular compensation, from the body system. 10 low vision and blind subjects aged between 35 and 50 years. The subjects performed 10 sessions (2/week) for 5 weeks. Each session consisted of a training in the gym (30 minutes) and hydrokinetic therapy (45 minutes) All the subjects at the beginning (T0) and at the end (T1) of the rehabilitation program were tested by static baropodometry and the stabilometric (Milletrix, Diagnostic support, Rome, Italy). Our results shows an increase of the total area of support surface (p0.05). The stabilometry results shows an improvement of the orthostatic balance, sway area and the ellipse area decrease respectively 15% (236,9vs201 p≤0.05) and 41% b(p≤0.05). Thus, the energy expenditure of the patient in maintaining the orthostatic position without visual afferences is lower. Rehabilitative treatment in an aquatic environment resulted effective in improving posture and balance in all patients with increased precision in the execution of the step, which implies less effort during walk, less energy consumption and better quality of life.

Spatial representations in blind people: the role of strategies and mobility skills

The role of vision in the construction of spatial representations has been the object of numerous studies and heated debate. The core question of whether visual experience is necessary to form spatial representations has found different, often contradictory answers. The present paper examines mental images generated from verbal descriptions of spatial environments. Previous evidence had shown that blind individuals have difficulty remembering information about spatial environments. By testing a group of congenitally blind people, we replicated this result and found that it is also present when the overall mental model of the environment is assessed. This was not always the case, however, but appeared to correlate with some blind participants' lower use of a mental imagery strategy and preference for a verbal rehearsal strategy, which was adopted particularly by blind people with more limited mobility skills. The more independent blind people who used a mental imagery strategy performed as well as sighted participants, suggesting that the difficulty blind people may have in processing spatial descriptions is not due to the absence of vision per se, but could be the consequence of both, their using less efficient verbal strategies and having poor mobility skills.

The role of visual experience for the neural basis of spatial cognition

Blindness often results in the adaptive neural reorganization of the remaining modalities, producing sharper auditory and haptic behavioral performance. Yet, non-visual modalities might not be able to fully compensate for the lack of visual experience as in the case of congenital blindness. For example, developmental visual experience seems to be necessary for the maturation of multisensory neurons for spatial tasks. Additionally, the ability of vision to convey information in parallel might be taken into account as the main attribute that cannot be fully compensated by the spared modalities. Therefore, the lack of visual experience might impair all spatial tasks that require the integration of inputs from different modalities, such as having to represent a set of objects on the basis of the spatial relationships among the objects, rather than the spatial relationship that each object has with oneself. Here we integrate behavioral and neural evidence to conclude that visual experience is necessary for the neural development of normal spatial cognition.

Defensive peripersonal space: the blink reflex evoked by hand stimulation is increased when the hand is near the face

Electrical stimulation of the median nerve at the wrist may elicit a blink reflex [hand blink reflex (HBR)] mediated by a neural circuit at brain stem level. As, in a Sherringtonian sense, the blink reflex is a defensive response, in a series of experiments we tested, in healthy volunteers, whether and how the HBR is modulated by the proximity of the stimulated hand to the face. Electromyographic activity was recorded from the orbicularis oculi, bilaterally. We observed that the HBR is enhanced when the stimulated hand is inside the peripersonal space of the face, compared with when it is outside, irrespective of whether the proximity of the hand to the face is manipulated by changing the position of the arm ( experiment 1) or by rotating the head while keeping the arm position constant ( experiment 3). Experiment 2 showed that such HBR enhancement has similar magnitude when the participants have their eyes closed. Experiments 4 and 5 showed, respectively, that the blink reflex elicited by the electrical stimulation of the supraorbital nerve, as well as the N20 wave of the somatosensory evoked potentials elicited by the median nerve stimulation, are entirely unaffected by hand position. Taken together, our results provide compelling evidence that the brain stem circuits mediating the HBR in humans undergo tonic and selective top-down modulation from higher order cortical areas responsible for encoding the location of somatosensory stimuli in external space coordinates. These findings support the existence of a “defensive” peripersonal space, representing a safety margin advantageous for survival.

Sequential vs simultaneous encoding of spatial information: a comparison between the blind and the sighted

The aim of this research is to assess whether the crucial factor in determining the characteristics of blind people's spatial mental images is concerned with the visual impairment per se or the processing style that the dominant perceptual modalities used to acquire spatial information impose, i.e. simultaneous (vision) vs sequential (kinaesthesis). Participants were asked to learn six positions in a large parking area via movement alone (congenitally blind, adventitiously blind, blindfolded sighted) or with vision plus movement (simultaneous sighted, sequential sighted), and then to mentally scan between positions in the path. The crucial manipulation concerned the sequential sighted group. Their visual exploration was made sequential by putting visual obstacles within the pathway in such a way that they could not see simultaneously the positions along the pathway. The results revealed a significant time/distance linear relation in all tested groups. However, the linear component was lower in sequential sighted and blind participants, especially congenital. Sequential sighted and congenitally blind participants showed an almost overlapping performance. Differences between groups became evident when mentally scanning farther distances (more than 5m). This threshold effect could be revealing of processing limitations due to the need of integrating and updating spatial information. Overall, the results suggest that the characteristics of the processing style rather than the visual impairment per se affect blind people's spatial mental images.

Audiotactile interactions in temporal perception

In the present review, we focus on how commonalities in the ontogenetic development of the auditory and tactile sensory systems may inform the interplay between these signals in the temporal domain. In particular, we describe the results of behavioral studies that have investigated temporal resolution (in temporal order, synchrony/asynchrony, and simultaneity judgment tasks), as well as temporal numerosity perception, and similarities in the perception of frequency across touch and hearing. The evidence reviewed here highlights features of audiotactile temporal perception that are distinctive from those seen for other pairings of sensory modalities. For instance, audiotactile interactions are characterized in certain tasks (e.g., temporal numerosity judgments) by a more balanced reciprocal influence than are other modality pairings. Moreover, relative spatial position plays a different role in the temporal order and temporal recalibration processes for audiotactile stimulus pairings than for other modality pairings. The effect exerted by both the spatial arrangement of stimuli and attention on temporal order judgments is described. Moreover, a number of audiotactile interactions occurring during sensory-motor synchronization are highlighted. We also look at the audiotactile perception of rhythm and how it may be affected by musical training. The differences emerging from this body of research highlight the need for more extensive investigation into audiotactile temporal interactions. We conclude with a brief overview of some of the key issues deserving of further research in this area.

Crossmodal plasticity in sensory loss

In this review, we describe crossmodal plasticity following sensory loss in three parts, with each section focusing on one sensory system. We summarize a wide range of studies showing that sensory loss may lead, depending of the affected sensory system, to functional changes in other, primarily not affected senses, which range from heightened to lowered abilities. In the first part, the effects of blindness on mainly audition and touch are described. The latest findings on brain reorganization in blindness are reported, with a particular emphasis on imaging studies illustrating how nonvisual inputs recruit the visually deafferented occipital cortex. The second part covers crossmodal processing in deafness, with a special focus on the effects of deafness on visual processing. In the last portion of this review, we present the effects that the loss of a chemical sense have on the sensitivity of the other chemical senses, that is, smell, taste, and trigeminal chemosensation. We outline how the convergence of the chemical senses to the same central processing areas may lead to the observed reduction in sensitivity of the primarily not affected senses. Altogether, the studies reviewed herein illustrate the fascinating plasticity of the brain when coping with sensory deprivation.

Audiotactile interactions in front and rear space

The last few years have seen a growing interest in the assessment of audiotactile interactions in information processing in peripersonal space. In particular, these studies have focused on investigating peri-hand space [corrected] and, more recently, on the functional differences that have been demonstrated between the space close to front and back of the head (i.e., the peri-head space). In this review, the issue of how audiotactile interactions vary as a function of the region of space in which stimuli are presented (i.e., front vs. rear, peripersonal vs. extra-personal) will be described. We review evidence from both monkey and human studies. This evidence, providing insight into the differential attributes qualifying the frontal and the rear regions of space, sheds light on an until now neglected research topic and may help to contribute to the formulation of new rehabilitative approaches to disorders of spatial representation. A tentative explanation of the evolutionary reasons underlying these particular patterns of results, as well as suggestions for possible future developments, are also provided.

Early visual deprivation alters multisensory processing in peripersonal space

Multisensory peripersonal space develops in a maturational process that is thought to be influenced by early sensory experience. We investigated the role of vision in the effective development of audiotactile interactions in peripersonal space. Early blind (EB), late blind (LB) and sighted control (SC) participants were asked to lateralize auditory, tactile and audiotactile stimuli. The experiment was conducted with the hands uncrossed or crossed over the body midline in order to alter the relationship between personal and peripersonal spatial representations. First, we observed that the crossed posture results in a greater detrimental effect for tactile performance in sighted subjects but a greater deficit in auditory performance in early blind ones. This result is interpreted as evidence for a visually driven developmental process that automatically remaps tactile and proprioceptive spatial representation into an external framework. Second, we demonstrate that improved reaction times observed in the bimodal conditions in SC and LB exceeds that predicted by probability summation in both conditions of postures, indicating neural integration of different sensory information. In EB, nonlinear summation was obtained in the uncrossed but not in the crossed posture. We argue that the default use of an anatomically anchored reference system in EB prevents effective audiotactile interactions in the crossed posture due to the poorly aligned spatial coordinates of these two modalities in such conditions. Altogether, these results provide compelling evidence for the critical role of early vision in the development of multisensory perception and action control in peripersonal space.

Viewing the body modulates neural mechanisms underlying sustained spatial attention in touch

Cross-modal links between vision and touch have been extensively shown with a variety of paradigms. The present event-related potential (ERP) study aimed to clarify whether neural mechanisms underlying sustained tactile-spatial attention may be modulated by visual input, and the sight of the stimulated body part (i.e. hands) in particular. Participants covertly attended to one of their hands throughout a block to detect infrequent tactile target stimuli at that hand while ignoring tactile targets at the unattended hand, and all tactile non-targets. In different blocks, participants performed this task under three viewing conditions: full vision; hands covered from view; and blindfolded. When the participants' hands were visible attention was found to modulate somatosensory ERPs at early latencies (i.e. in the time range of the somatosensory P100 and the N140 components), as well as at later time intervals, from 200 ms after stimulus onset. By contrast, when participants were blindfolded and, crucially, even when only their hands were not visible, attentional modulations were found to arise only at later intervals (i.e. from 200 ms post-stimulus), while earlier somatosensory components were not affected by spatial attention. The behavioural results tallied with these electrophysiological findings, showing faster response times to tactile targets under the full vision condition compared with conditions when participants' hands were covered, and when participants were blindfolded. The results from this study provide the first evidence of the profound impact of vision on mechanisms underlying sustained tactile-spatial attention, which is enhanced by the sight of the body parts (i.e. hands).

Movement and the Rubber Hand Illusion

When a participant views a rubber hand being stroked by a paintbrush while his/her real hand is unseen and similarly stroked by another paintbrush, a misperception known as the rubber hand illusion occurs whereby tactile sensations are falsely referred to the non-body part. The purpose of the current study was to further examine the rubber hand illusion with conditions of movement. An apparatus was devised that would synchronise visual with felt movement in an active condition and a passive condition. An asynchronous condition was included as a control in which visual and felt movement were purposely disconnected. The three movement conditions (active, passive, and asynchronous) were statistically compared in order to assess our prediction that synchronous conditions of movement (especially active) would generate more reports of the illusion. The performance of the movement conditions was evaluated against a visual-tactile condition, which is a known contributor to the rubber hand illusion. Not only significantly more robust reports of the illusion were obtained when visual movement and felt movement were synchronised but there was also a trend toward stronger reports in the active condition rather than the passive condition. Interestingly, the pattern of results differed according to the particular question on the self-report.

Improved selective and divided spatial attention in early blind subjects

Spatial attention paradigms using auditory or tactile stimulation were used to explore neural and behavioral reorganization in early blind subjects. Although it is commonly assumed that blind subjects outperform sighted subjects in such tasks, the empirical data to confirm this remain controversial. Moreover, previous studies have often confounded factors of sensory acuity with those of attention. In the present work, we compared the performance of individually matched early blind and sighted subjects during auditory and tactile tasks. These consisted of sensory acuity tests, simple reaction time task as well as selective and divided spatial attention tasks. Based on sensory measurements, we made sure that the reliability and salience of auditory and tactile information were identical between the two populations to estimate attentional performance independently of sensory influence. Results showed no difference between groups in either sensory sensitivity or simple reaction time task for both modalities. However, blind subjects displayed shorter reaction times than sighted subjects in both tactile and auditory selective spatial attention tasks and also in bimodal divided spatial attention tasks. The present study thus demonstrates an enhanced attentional performance in early blind subjects which is independent of sensory influence. These supra-normal abilities could be related to quantitative and qualitative changes in the way early visually deprived subjects process non-visual spatial information.