Merging familiar and new senses to perceive and act in space

Development and experience-dependence of multisensory spatial processing

Multisensory spatial processes are fundamental for efficient interaction with the world. They include not only the integration of spatial cues across sensory modalities, but also the adjustment or recalibration of spatial representations to changing cue reliabilities, crossmodal correspondences, and causal structures. Yet how multisensory spatial functions emerge during ontogeny is poorly understood. New results suggest that temporal synchrony and enhanced multisensory associative learning capabilities first guide causal inference and initiate early coarse multisensory integration capabilities. These multisensory percepts are crucial for the alignment of spatial maps across sensory systems, and are used to derive more stable biases for adult crossmodal recalibration. The refinement of multisensory spatial integration with increasing age is further promoted by the inclusion of higher-order knowledge.

Shape detection beyond the visual field using a visual-to-auditory sensory augmentation device

Current advancements in both technology and science allow us to manipulate our sensory modalities in new and unexpected ways. In the present study, we explore the potential of expanding what we perceive through our natural senses by utilizing a visual-to-auditory sensory substitution device (SSD), the EyeMusic, an algorithm that converts images to sound. The EyeMusic was initially developed to allow blind individuals to create a spatial representation of information arriving from a video feed at a slow sampling rate. In this study, we aimed to use the EyeMusic for the blind areas of sighted individuals. We use it in this initial proof-of-concept study to test the ability of sighted subjects to combine visual information with surrounding auditory sonification representing visual information. Participants in this study were tasked with recognizing and adequately placing the stimuli, using sound to represent the areas outside the standard human visual field. As such, the participants were asked to report shapes' identities as well as their spatial orientation (front/right/back/left), requiring combined visual (90° frontal) and auditory input (the remaining 270°) for the successful performance of the task (content in both vision and audition was presented in a sweeping clockwise motion around the participant). We found that participants were successful at a highly above chance level after a brief 1-h-long session of online training and one on-site training session of an average of 20 min. They could even draw a 2D representation of this image in some cases. Participants could also generalize, recognizing new shapes they were not explicitly trained on. Our findings provide an initial proof of concept indicating that sensory augmentation devices and techniques can potentially be used in combination with natural sensory information in order to expand the natural fields of sensory perception.

Studying the Development of Navigation Using Virtual Environments

Research on spatial navigation is essential to understanding how mobile species adapt to their environments. Such research increasingly uses virtual environments (VEs) because, although VE has drawbacks, it allows for standardization of procedures, precision in measuring behaviors, ease in introducing variation, and cross-investigator comparability. Developmental researchers have used a wide range of VE testing methods, including desktop computers, gaming consoles, virtual reality, and phone applications. We survey the paradigms to guide researchers' choices, organizing them by their characteristics using a framework proposed by Girard (2022) in which navigation is reactive or deliberative, and may be tied to sensory input or not. This organization highlights what representations each paradigm indicates. VE tools have enriched our picture of the development of navigation, but much research remains to be done, e.g., determining retest reliability, comparing performance on different paradigms, validating performance against real-world behavior and open sharing. Reliable and valid assessments available on open-science repositories are essential for work on the development of navigation, its neural bases, and its implications for other cognitive domains.