Ambient visual information confers a context-specific, long-term benefit on memory for haptic scenes

Sensory substitution increases robotic surgical performance and sets the ground for a mediating role of the sense of embodiment: a systematic review

Sensory Substitution (SS) allows the elaboration of information via non preferential sensory modalities. This phenomenon occurs in robotic-assisted surgery (RAS), in which haptic feedback is lacking. It has been suggested that SS could sustain surgeons' proficiency by means of visual clues for inferring tactile information, that also promotes the feeling of haptic phantom sensations. A critical role in reaching a good performance in procedural tasks is also sustained by the Sense of Embodiment (SE), that is, the capacity to integrate objects into subjective bodily self-representation. As SE is enhanced by haptic sensations, we hypothesize a role of SS in promoting SE in RAS. Accordingly, the goal of this systematic review is to summarize the evidence pertaining the study of SS in RAS in order to highlight the impact on the performance, and to identify a mediating role of the SE in increasing dexterity in RAS. Eight studies selected from the MEDLINE and Scopus® databases met inclusion criteria for a qualitative synthesis. Results indicated that haptic to other modalities SS enhanced force consistency and accuracy, and decreased surgeon fatigue. Expert surgeons, as compared to novices, showed a better natural SS processing, testified by a proficient performance with and without SS aids. No studies investigated the mediating role of SE. These findings indicate that SS is subjected to learning and memory processes that help surgeons to rapidly derive haptic-correlates from visual clues, which are highly required for a good performance. Also, the higher ability of doing SS and the associated perception of haptic sensations might increase multisensory integration, which might sustain performance.

The development of allocentric spatial frame in the auditory system

The ability to encode space is a crucial aspect of interacting with the external world. Therefore, this ability appears to be fundamental for the correct development of the capacity to integrate different spatial reference frames. The spatial reference frame seems to be present in all the sensory modalities. However, it has been demonstrated that different sensory modalities follow various developmental courses. Nevertheless, to date these courses have been investigated only in people with sensory impairments, where there is a possible bias due to compensatory strategies and it is complicated to assess the exact age when these skills emerge. For these reasons, we investigated the development of the allocentric frame in the auditory domain in a group of typically developing children aged 6-10 years. To do so, we used an auditory Simon task, a paradigm that involves implicit spatial processing, and we asked children to perform the task in both the uncrossed and crossed hands postures. We demonstrated that the crossed hands posture affected the performance only in younger children (6-7 years), whereas at 10 years of age children performed as adults and were not affected by such posture. Moreover, we found that this task's performance correlated with age and developmental differences in spatial abilities. Our results support the hypothesis that auditory spatial cognition's developmental course is similar to the visual modality development as reported in the literature.

Spotlights and Soundscapes

For persons with visual impairment, forming cognitive maps of unfamiliar interior spaces can be challenging. Various technical developments have converged to make it feasible, without specialized equipment, to represent a variety of useful landmark objects via spatial audio, rather than solely dispensing route information. Although such systems could be key to facilitating cognitive map formation, high-density auditory environments must be crafted carefully to avoid overloading the listener. This article recounts a set of research exercises with potential users, in which the optimization of such systems was explored. In Experiment 1, a virtual reality environment was used to rapidly prototype and adjust the auditory environment in response to participant comments. In Experiment 2, three variants of the system were evaluated in terms of their effectiveness in a real-world building. This methodology revealed a variety of optimization approaches and recommendations for designing dense mixed-reality auditory environments aimed at supporting cognitive map formation by visually impaired persons.

Adults' spatial scaling: evidence from the haptic domain

The current study investigated adults' spatial-scaling abilities using a haptic localization task. As a first aim, we examined the strategies used to solve this haptic task. Secondly, we explored whether irrelevant visual information influenced adults' spatial-scaling performance. Thirty-two adults were asked to locate targets as presented in maps on a larger or same-sized referent space. Maps varied in size in accordance with different scaling factors (1:4, 1:2, 1:1), whereas the referent space was constant in size throughout the experimental session. The availability of irrelevant, non-informative vision was manipulated by blindfolding half of the participants prior to the experiment (condition without non-informative vision), whereas the other half were able to see their surroundings with the stimuli being hidden behind a curtain (condition with non-informative vision). Analyses with absolute errors (after correcting for reversal errors) as the dependent variable revealed a significant interaction of the scaling factor and non-informative vision condition. Adults in the blindfolded condition showed constant errors and response times irrespective of scaling factor. Such a response pattern indicates the usage of relative strategies. Adults in the curtain condition showed a linear increase in errors with higher scaling factors, whereas their response times remained constant. This pattern of results supports the usage of absolute strategies or mental transformation strategies. Overall, our results indicate different scaling strategies depending on the availability of non-informative vision, highlighting the strong influence of (even irrelevant) vision on adults' haptic processing.

Brain networks require a network-conscious psychopathological approach

In experimental psychology and neuroscience, technological advances and multisensory research have contributed to gradually dismiss a version of reductionism. Empirical results no longer support a brain model in which distinct "modules" perform discrete functions, but rather, a brain of partially overlapping networks. A similarly changed brain model is extending to psychopathology and clinical psychology, and partly accounts for the problems of reductionism.

Scanning movements during haptic search: similarity with fixations during visual search

Finding relevant objects through vision, or visual search, is a crucial function that has received considerable attention in the literature. After decades of research, data suggest that visual fixations are more crucial to understanding how visual search works than are the attributes of stimuli. This idea receives further support from the field of haptic search.

The Influence of Tactile Cognitive Maps on Auditory Space Perception in Sighted Persons

We have recently shown that vision is important to improve spatial auditory cognition. In this study, we investigate whether touch is as effective as vision to create a cognitive map of a soundscape. In particular, we tested whether the creation of a mental representation of a room, obtained through tactile exploration of a 3D model, can influence the perception of a complex auditory task in sighted people. We tested two groups of blindfolded sighted people – one experimental and one control group – in an auditory space bisection task. In the first group, the bisection task was performed three times: specifically, the participants explored with their hands the 3D tactile model of the room and were led along the perimeter of the room between the first and the second execution of the space bisection. Then, they were allowed to remove the blindfold for a few minutes and look at the room between the second and third execution of the space bisection. Instead, the control group repeated for two consecutive times the space bisection task without performing any environmental exploration in between. Considering the first execution as a baseline, we found an improvement in the precision after the tactile exploration of the 3D model. Interestingly, no additional gain was obtained when room observation followed the tactile exploration, suggesting that no additional gain was obtained by vision cues after spatial tactile cues were internalized. No improvement was found between the first and the second execution of the space bisection without environmental exploration in the control group, suggesting that the improvement was not due to task learning. Our results show that tactile information modulates the precision of an ongoing space auditory task as well as visual information. This suggests that cognitive maps elicited by touch may participate in cross-modal calibration and supra-modal representations of space that increase implicit knowledge about sound propagation.

Touching and Hearing Unseen Objects: Multisensory Effects on Scene Recognition

In three experiments, we investigated the influence of object-specific sounds on haptic scene recognition without vision. Blindfolded participants had to recognize, through touch, spatial scenes comprising six objects that were placed on a round platform. Critically, in half of the trials, object-specific sounds were played when objects were touched (bimodal condition), while sounds were turned off in the other half of the trials (unimodal condition). After first exploring the scene, two objects were swapped and the task was to report, which of the objects swapped positions. In Experiment 1, geometrical objects and simple sounds were used, while in Experiment 2, the objects comprised toy animals that were matched with semantically compatible animal sounds. In Experiment 3, we replicated Experiment 1, but now a tactile-auditory object identification task preceded the experiment in which the participants learned to identify the objects based on tactile and auditory input. For each experiment, the results revealed a significant performance increase only after the switch from bimodal to unimodal. Thus, it appears that the release of bimodal identification, from audio-tactile to tactile-only produces a benefit that is not achieved when having the reversed order in which sound was added after having experience with haptic-only. We conclude that task-related factors other than mere bimodal identification cause the facilitation when switching from bimodal to unimodal conditions.

Multisensory integration substantiates distributed and overlapping neural networks

The hypothesis that highly overlapping networks underlie brain functions (neural reuse) is decisively supported by three decades of multisensory research. Multisensory areas process information from more than one sensory modality and therefore represent the best examples of neural reuse. Recent evidence of multisensory processing in primary visual cortices further indicates that neural reuse is a basic feature of the brain.

Sensory Substitution: The Spatial Updating of Auditory Scenes "Mimics" the Spatial Updating of Visual Scenes

Visual-to-auditory sensory substitution is used to convey visual information through audition, and it was initially created to compensate for blindness; it consists of software converting the visual images captured by a video-camera into the equivalent auditory images, or “soundscapes”. Here, it was used by blindfolded sighted participants to learn the spatial position of simple shapes depicted in images arranged on the floor. Very few studies have used sensory substitution to investigate spatial representation, while it has been widely used to investigate object recognition. Additionally, with sensory substitution we could study the performance of participants actively exploring the environment through audition, rather than passively localizing sound sources. Blindfolded participants egocentrically learnt the position of six images by using sensory substitution and then a judgment of relative direction task (JRD) was used to determine how this scene was represented. This task consists of imagining being in a given location, oriented in a given direction, and pointing towards the required image. Before performing the JRD task, participants explored a map that provided allocentric information about the scene. Although spatial exploration was egocentric, surprisingly we found that performance in the JRD task was better for allocentric perspectives. This suggests that the egocentric representation of the scene was updated. This result is in line with previous studies using visual and somatosensory scenes, thus supporting the notion that different sensory modalities produce equivalent spatial representation(s). Moreover, our results have practical implications to improve training methods with sensory substitution devices (SSD).

Rest boosts the long-term retention of spatial associative and temporal order information

People retain more new verbal episodic information for at least 7 days if they rest for a few minutes after learning than if they attend to new information. It is hypothesized that rest allows for superior consolidation of new memories. In rodents, rest periods promote hippocampal replay of a recently travelled route, and this replay is thought to be critical for memory consolidation and subsequent spatial navigation. If rest boosts human memory by promoting hippocampal replay/consolidation, then the beneficial effect of rest should extend to complex (hippocampal) memory tasks, for example, tasks probing associations and sequences. We investigated this question via a virtual reality route memory task. Healthy young participants learned two routes to a 100% criterion. One route was followed by a 10-min rest and the other by a 10-min spot the difference game. For each learned route, participants performed four delayed spatial memory tests probing: (i) associative (landmark-direction) memory, (ii) cognitive map formation, (iii) temporal (landmark) order memory, and (iv) route memory. Tests were repeated after 7 days to determine any long-term effects. No effect of rest was detected in the route memory or cognitive map tests, most likely due to ceiling and floor effects, respectively. Rest did, however, boost retention in the associative memory and temporal order memory tests, and this boost remained for at least 7 days. We therefore demonstrate that the benefit of rest extends to (spatial) associative and temporal order memory in humans. We hypothesise that rest allows superior consolidation/hippocampal replay of novel information pertaining to a recently learned route, thus boosting new memories over the long term.

From body shadows to bodily attention: automatic orienting of tactile attention driven by cast shadows

Body shadows orient attention to the body-part casting the shadow. We have investigated the automaticity of this phenomenon, by addressing its time-course and its resistance to contextual manipulations. When targets were tactile stimuli at the hands (Exp.1) or visual stimuli near the body-shadow (Exp.2), cueing effects emerged regardless of the delay between shadow and target onset (100, 600, 1200, 2400ms). This suggests a fast and sustained attention orienting to body-shadows, that involves both the space occupied by shadows (extra-personal space) and the space the shadow refers to (own body). When target type became unpredictable (tactile or visual), shadow-cueing effects remained robust only for tactile targets, as visual stimuli showed no overall reliable effects, regardless of whether they occurred near the shadow (Exp.3) or near the body (Exp.4). We conclude that mandatory attention shifts triggered by body-shadows are limited to tactile targets and, instead, are less automatic for visual stimuli.

The blind brain: How (lack of) vision shapes the morphological and functional architecture of the human brain

Since the early days, how we represent the world around us has been a matter of philosophical speculation. Over the last few decades, modern neuroscience, and specifically the development of methodologies for the structural and the functional exploration of the brain have made it possible to investigate old questions with an innovative approach. In this brief review, we discuss the main findings from a series of brain anatomical and functional studies conducted in sighted and congenitally blind individuals by our’s and others' laboratories. Historically, research on the ‘blind brain’ has focused mainly on the cross-modal plastic changes that follow sensory deprivation. More recently, a novel line of research has been developed to determine to what extent visual experience is truly required to achieve a representation of the surrounding environment. Overall, the results of these studies indicate that most of the brain fine morphological and functional architecture is programmed to develop and function independently from any visual experience. Distinct cortical areas are able to process information in a supramodal fashion, that is, independently from the sensory modality that carries that information to the brain. These observations strongly support the hypothesis of a modality-independent, i.e. more abstract, cortical organization, and may contribute to explain how congenitally blind individuals may interact efficiently with an external world that they have never seen.