Multisensory interactions in the depth plane in front and rear space: A review

Comparison of peripersonal space in front and rear spaces

The space immediately around the body, referred to as the peripersonal space (PPS), plays a crucial role in interactions with external objects and in avoiding unsafe situations. This study aimed to investigate whether the size of the PPS changes depending on direction, with a particular focus on the disparity between the front and rear spaces. A vibrotactile stimulus was presented to measure PPS while a task-irrelevant auditory stimulus (probe) approached the participant. In addition, to evaluate the effect of the probe, a baseline condition was used in which only tactile stimuli were presented. The results showed that the auditory facilitation effect of the tactile stimulus was greater in the rear condition than in the front condition. Conversely, the performance on tasks related to auditory distance perception and sound speed estimation did not differ between the two directions, indicating that the difference in the auditory facilitation effect between directions cannot be explained by these factors. These findings indicate that the strength of audio-tactile integration is greater in the rear space compared to the front space, suggesting that the representation of the PPS differed between the front and rear spaces.

"Rear bias" in spatial auditory perception: Attentional and affective vigilance to sounds occurring outside the visual field

Presented studies explored the rear bias phenomenon, that is, the attentional and affective bias to sounds occurring behind the listener. Physiological and psychological reactions (i.e., fEMG, EDA/SCR, Simple Reaction Task-SRT, and self-assessments of affect-related states) were measured in response to tones of different frequencies (Study 1) and emotional vocalizations (Study 2) presented in rear and front spatial locations. Results showed that emotional vocalizations, when located in the back, facilitate reactions related to attention orientation (i.e., auricularis muscle response and simple reaction times) and evoke higher arousal-both physiological (as measured by SCR) and psychological (self-assessment scale). Importantly, observed asymmetries were larger for negative and threat-related signals (e.g., anger) than positive/nonthreatening ones (e.g., achievement). By contrast, there were only small differences for the relatively higher frequency tones. The observed relationships are discussed in terms of one of the postulated auditory system's functions, which is monitoring of the environment in order to quickly detect potential threats that occur outside of the visual field (e.g., behind one's back).

Visual stimuli in the peripersonal space facilitate the spatial prediction of tactile events-A comparison between approach and nearness effects

Previous studies reported that an object in one's peripersonal space (PPS) attracts attention and facilitates subsequent processing of stimuli. Recent studies showed that visual stimuli approaching the body facilitated the spatial prediction of subsequent tactile events, even if these stimuli were task-irrelevant. However, it is unclear whether the approach is important for facilitating this prediction or if the simple existence of stimuli within the PPS is what matters. The present study aimed to scrutinize the predictive function of visuo-tactile interaction in the PPS by examining the effects of visual stimuli approaching the hand and of visual stimuli near the hand. For this purpose, we examined electroencephalograms (EEGs) during a simple reaction time task for tactile stimuli when visual stimuli were presented approaching the hand or were presented near the hand, and we analyzed event-related spectral perturbation (ERSP) as an index of prediction and event-related brain potentials (ERPs) as an index of attention and prediction error. The tactile stimulus was presented to the left (or right) wrist with a high probability (80%) and to the opposite wrist with a low probability (20%). In the approach condition, three visual stimuli were presented approaching the hand to which the high-probability tactile stimulus was presented; in the near condition, three visual stimuli were presented repeatedly near the hand with the high-probability tactile stimulus. Beta-band activity at the C3 and C4 electrodes, around the primary somatosensory area, was suppressed before the onset of the tactile stimulus, and this suppression was larger in the approach condition than in the near condition. The P3 amplitude for high-probability stimuli in the approach condition was larger than that in the near condition. These results revealed that the approach of visual stimuli facilitates spatial prediction and processing of subsequent tactile stimuli compared to situations in which visual stimuli just exist within the PPS. This study indicated that approaching visual stimuli facilitates the prediction of subsequent tactile events, even if they are task-irrelevant.

Peripersonal space in the front, rear, left and right directions for audio-tactile multisensory integration

Peripersonal space (PPS) is important for humans to perform body–environment interactions. However, many previous studies only focused on the specific direction of the PPS, such as the front space, despite suggesting that there were PPSs in all directions. We aimed to measure and compare the peri-trunk PPS in four directions (front, rear, left, and right). To measure the PPS, we used a tactile and an audio stimulus because auditory information is available at any time in all directions. We used the approaching and receding task-irrelevant sounds in the experiment. Observers were asked to respond as quickly as possible when a tactile stimulus was applied to a vibrator on their chest. We found that peri-trunk PPS representations exist with an approaching sound, irrespective of the direction.

Crossmodal spatial distraction across the lifespan

The ability to resist distracting stimuli whilst voluntarily focusing on a task is fundamental to our everyday cognitive functioning. Here, we investigated how this ability develops, and thereafter declines, across the lifespan using a single task/experiment. Young children (5-7 years), older children (10-11 years), young adults (20-27 years), and older adults (62-86 years) were presented with complex visual scenes. Endogenous (voluntary) attention was engaged by having the participants search for a visual target presented on either the left or right side of the display. The onset of the visual scenes was preceded - at stimulus onset asynchronies (SOAs) of 50, 200, or 500 ms - by a task-irrelevant sound (an exogenous crossmodal spatial distractor) delivered either on the same or opposite side as the visual target, or simultaneously on both sides (cued, uncued, or neutral trials, respectively). Age-related differences were revealed, especially in the extreme age-groups, which showed a greater impact of crossmodal spatial distractors. Young children were highly susceptible to exogenous spatial distraction at the shortest SOA (50 ms), whereas older adults were distracted at all SOAs, showing significant exogenous capture effects during the visual search task. By contrast, older children and young adults' search performance was not significantly affected by crossmodal spatial distraction. Overall, these findings present a detailed picture of the developmental trajectory of endogenous resistance to crossmodal spatial distraction from childhood to old age and demonstrate a different efficiency in coping with distraction across the four age-groups studied.

The additive nature of the human multisensory evoked pupil response

Pupillometry has received increased interest for its usefulness in measuring various sensory processes as an alternative to behavioural assessments. This is also apparent for multisensory investigations. Studies of the multisensory pupil response, however, have produced conflicting results. Some studies observed super-additive multisensory pupil responses, indicative of multisensory integration (MSI). Others observed additive multisensory pupil responses even though reaction time (RT) measures were indicative of MSI. Therefore, in the present study, we investigated the nature of the multisensory pupil response by combining methodological approaches of previous studies while using supra-threshold stimuli only. In two experiments we presented auditory and visual stimuli to observers that evoked a(n) (onset) response (be it constriction or dilation) in a simple detection task and a change detection task. In both experiments, the RT data indicated MSI as shown by race model inequality violation. Still, the multisensory pupil response in both experiments could best be explained by linear summation of the unisensory pupil responses. We conclude that the multisensory pupil response for supra-threshold stimuli is additive in nature and cannot be used as a measure of MSI, as only a departure from additivity can unequivocally demonstrate an interaction between the senses.

When self-prioritization crosses the senses: Crossmodal self-prioritization demonstrated between vision and touch

The investigation of self-prioritization via a simple matching paradigm represents a new way of enhancing our knowledge about the processing of self-relevant content and also increases our understanding of the self-concept itself. By associating formerly neutral material with the self, and assessing the resulting prioritization of these newly formed self-associations, conclusions can be drawn concerning the effects of self-relevance without the burden of highly overlearned materials such as one's own name. This approach was used to gain further insights into the structure and complexity of self-associations: a tactile pattern was associated with the self and thereafter, the prioritization of the exact same visual pattern was assessed - enabling the investigation of crossmodal self-associations. The results demonstrate a prioritization of self-associated material that rapidly extends beyond the borders of a sensory modality in which it was first established.

Comparison of auditory spatial bisection and minimum audible angle in front, lateral, and back space

Although vision is important for calibrating auditory spatial perception, it only provides information about frontal sound sources. Previous studies of blind and sighted people support the idea that azimuthal spatial bisection in frontal space requires visual calibration, while detection of a change in azimuth (minimum audible angle, MAA) does not. The influence of vision on the ability to map frontal, lateral and back space has not been investigated. Performance in spatial bisection and MAA tasks was assessed for normally sighted blindfolded subjects using bursts of white noise presented frontally, laterally, or from the back relative to the subjects. Thresholds for both tasks were similar in frontal space, lower for the MAA task than for the bisection task in back space, and higher for the MAA task in lateral space. Two interpretations of the results are discussed, one in terms of visual calibration and the use of internal representations of source location and the other based on comparison of the magnitude or direction of change of the available binaural cues. That bisection thresholds were increased in back space relative to front space, where visual calibration information is unavailable, suggests that an internal representation of source location was used for the bisection task.

Peri-Personal Space Tracing by Hand-Blink Reflex Modulation in Patients with Chronic Disorders of Consciousness

The assessment of awareness in patients with chronic Disorders of Consciousness (DoC), including Unresponsive Wakefulness Syndrome (UWS) and Minimally Conscious State (MCS), is challenging. The level of awareness impairment may depend on the degree of deterioration of the large-scale cortical-thalamo-cortical networks induced by brain injury. Electrophysiological approaches may shed light on awareness presence in patients with DoC by estimating cortical functions related to the cortical-thalamo-cortical networks including, for example, the cortico-subcortical processes generating motor responses to the perturbation of the peri-personal space (PPS). We measured the amplitude, latency, and duration of the hand-blink reflex (HBR) responses by recording electromyography (EMG) signals from both the orbicularis oculi muscles while electrically stimulating the median nerve at the wrist. Such a BR is thought to be mediated by a neural circuit at the brainstem level. Despite its defensive-response nature, HBR can be modulated by the distance between the stimulated hand and the face. This suggests a functional top-down control of HBR as reflected by HBR features changes (latency, amplitude, and magnitude). We therefore estimated HBR responses in a sample of patients with DoC (8 MCS and 12 UWS, compared to 15 healthy controls -HC) while performing a motor task targeting the PPS. This consisted of passive movements in which the hand of the subject was positioned at different distances from the participant's face. We aimed at demonstrating a residual top-down modulation of HBR properties, which could be useful to differentiate patients with DoC and, potentially, demonstrate awareness preservation. We found a decrease in latency, and an increase in duration and magnitude of HBR responses, which were all inversely related to the hand-to-face distance in HC and patients with MCS, but not in individuals with UWS. Our data suggest that only patients with MCS have preserved, residual, top-down modulation of the processes related to the PPS from higher-order cortical areas to sensory-motor integration network. Although the sample size was relatively small, being thus our data preliminary, HBR assessment seems a rapid, easy, and first-level tool to differentiate patients with MCS from those with UWS. We may also hypothesize that such a HBR modulation suggests awareness preservation.

Increased Neural Strength and Reliability to Audiovisual Stimuli at the Boundary of Peripersonal Space

The actionable space surrounding the body, referred to as peripersonal space (PPS), has been the subject of significant interest of late within the broader framework of embodied cognition. Neurophysiological and neuroimaging studies have shown the representation of PPS to be built from visuotactile and audiotactile neurons within a frontoparietal network and whose activity is modulated by the presence of stimuli in proximity to the body. In contrast to single-unit and fMRI studies, an area of inquiry that has received little attention is the EEG characterization associated with PPS processing. Furthermore, although PPS is encoded by multisensory neurons, to date there has been no EEG study systematically examining neural responses to unisensory and multisensory stimuli, as these are presented outside, near, and within the boundary of PPS. Similarly, it remains poorly understood whether multisensory integration is generally more likely at certain spatial locations (e.g., near the body) or whether the cross-modal tactile facilitation that occurs within PPS is simply due to a reduction in the distance between sensory stimuli when close to the body and in line with the spatial principle of multisensory integration. In the current study, to examine the neural dynamics of multisensory processing within and beyond the PPS boundary, we present auditory, visual, and audiovisual stimuli at various distances relative to participants' reaching limit—an approximation of PPS—while recording continuous high-density EEG. We question whether multisensory (vs. unisensory) processing varies as a function of stimulus–observer distance. Results demonstrate a significant increase of global field power (i.e., overall strength of response across the entire electrode montage) for stimuli presented at the PPS boundary—an increase that is largest under multisensory (i.e., audiovisual) conditions. Source localization of the major contributors to this global field power difference suggests neural generators in the intraparietal sulcus and insular cortex, hubs for visuotactile and audiotactile PPS processing. Furthermore, when neural dynamics are examined in more detail, changes in the reliability of evoked potentials in centroparietal electrodes are predictive on a subject-by-subject basis of the later changes in estimated current strength at the intraparietal sulcus linked to stimulus proximity to the PPS boundary. Together, these results provide a previously unrealized view into the neural dynamics and temporal code associated with the encoding of nontactile multisensory around the PPS boundary.

The Role of Visual Experience in Auditory Space Perception around the Legs

It is widely accepted that vision plays a key role in the development of spatial skills of the other senses. Recent works have shown that blindness is often associated with auditory spatial deficits. The majority of previous studies have focused on understanding the representation of the upper frontal body space where vision and actions have a central role in mapping the space, however less research has investigated the back space and the space around the legs. Here we investigate space perception around the legs and the role of previous visual experience, by studying sighted and blind participants in an audio localization task (front-back discrimination). Participants judged if a sound was delivered in their frontal or back space. The results showed that blindfolded sighted participants were more accurate than blind participants in the frontal space. However, both groups were similarly accurate when auditory information was delivered in the back space. Blind individuals performed the task with similar accuracy for sounds delivered in the frontal and back space, while sighted people performed better in the frontal space. These results suggest that visual experience influences auditory spatial representations around the legs. Moreover, these results suggest that hearing and vision play different roles in different spaces.

Identifying peripersonal space boundaries in newborns

Peripersonal space immediately surrounds the body and can be represented in the brain as a multisensory and sensorimotor interface mediating physical and social interactions between body and environment. Very little consideration has been given to the ontogeny of peripersonal spatial representations in early postnatal life, despite the crucial roles of peripersonal space and its adaptive relevance as the space where infants' earliest interactions take place. Here, we investigated whether peripersonal space could be considered a delimited portion of space with defined boundaries soon after birth. Our findings showed for the first time that newborns' saccadic reaction times to a tactile stimulus simultaneous to sounds with different intensities changed based on the sound intensity. In particular, they were significantly faster when the sound was lounder than a critical intensity, in a pattern that closely resembled that showed by adults. Therefore, provided that sound intensity on its own can cue newborns' sound distance perception, we speculate that this critical distance could be considered the boundary of newborns' rudimentary peripersonal space. Altogether, our findings suggest that soon after birth peripersonal space may be already considered as a bounded portion of space, perhaps instrumental to drive newborns' attention towards events and people within it.

Toward understanding the neurophysiological basis of peripersonal space: An EEG study on healthy individuals

The subcortical mechanisms subtending the sensorimotor processes related to the peripersonal space (PPS) have been well characterized, whereas less evidence is available concerning the cortical mechanisms. We investigated the theta, alpha and beta event-related spectral perturbations (ERSP) while holding the forearm in different positions into the PPS of the face. Fifty healthy individuals were subjected to EEG recording while being provided with median nerve electric stimulation at the wrist of the right hand held at different hand-to-face distances. Theta and beta rhythms were significantly perturbed depending on the hand-to-face distance, whereas alpha oscillations reflected a more general, non-specific oscillatory response to the motor task. The perturbation of theta and beta frequency bands may reflect the processes of top-down modulation overseeing the conscious spatiotemporal encoding of sensory-motor information within the PPS. In other words, such perturbation reflects the continuous update of the conscious internal representations of the PPS to build up a purposeful and reflexive motor response.

Impairments in Multisensory Integration after Stroke

The integration of information from multiple senses leads to a plethora of behavioral benefits, most predominantly to faster and better detection, localization, and identification of events in the environment. Although previous studies of multisensory integration (MSI) in humans have provided insights into the neural underpinnings of MSI, studies of MSI at a behavioral level in individuals with brain damage are scarce. Here, a well-known psychophysical paradigm (the redundant target paradigm) was employed to quantify MSI in a group of stroke patients. The relation between MSI and lesion location was analyzed using lesion subtraction analysis. Twenty-one patients with ischemic infarctions and 14 healthy control participants responded to auditory, visual, and audiovisual targets in the left and right visual hemifield. Responses to audiovisual targets were faster than to unisensory targets. This could be due to MSI or statistical facilitation. Comparing the audiovisual RTs to the winner of a race between unisensory signals allowed us to determine whether participants could integrate auditory and visual information. The results indicated that (1) 33% of the patients showed an impairment in MSI; (2) patients with MSI impairment had left hemisphere and brainstem/cerebellar lesions; and (3) the left caudate, left pallidum, left putamen, left thalamus, left insula, left postcentral and precentral gyrus, left central opercular cortex, left amygdala, and left OFC were more often damaged in patients with MSI impairments. These results are the first to demonstrate the impact of brain damage on MSI in stroke patients using a well-established psychophysical paradigm.

Peri-hand space expands beyond reach in the context of walk-and-reach movements

The brain incorporates sensory information across modalities to be able to interact with our environment. The peripersonal space (PPS), defined by a high level of crossmodal interaction, is centered on the relevant body part, e.g. the hand, but can spatially expand to encompass tools or reach targets during goal-directed behavior. Previous studies considered expansion of the PPS towards goals within immediate or tool-mediated reach, but not the translocation of the body as during walking. Here, we used the crossmodal congruency effect (CCE) to quantify the extension of the PPS and test if PPS can also expand further to include far located walk-and-reach targets accessible only by translocation of the body. We tested for orientation specificity of the hand-centered reference frame, asking if the CCE inverts with inversion of the hand orientation during reach. We show a high CCE with onset of the movement not only towards reach targets but also walk-and-reach targets. When participants must change hand orientation, the CCE decreases, if not vanishes, and does not simply invert. We conclude that the PPS can expand to the action space beyond immediate or tool-mediated reaching distance but is not purely hand-centered with respect to orientation.

An Action Field Theory of Peripersonal Space

Predominant conceptual frameworks often describe peripersonal space (PPS) as a single, distance-based, in-or-out zone within which stimuli elicit enhanced neural and behavioural responses. Here we argue that this intuitive framework is contradicted by neurophysiological and behavioural data. First, PPS-related measures are not binary, but graded with proximity. Second, they are strongly influenced by factors other than proximity, such as walking, tool use, stimulus valence, and social cues. Third, many different PPS-related responses exist, and each can be used to describe a different space. Here, we reconceptualise PPS as a set of graded fields describing behavioural relevance of actions aiming to create or avoid contact between objects and the body. This reconceptualisation incorporates PPS into mainstream theories of action selection and behaviour.

What is peripersonal space? An examination of unresolved empirical issues and emerging findings

Findings from diverse fields of study, including neuroscience, psychology, zoology, and sociology, demonstrate that human and non-human primates maintain a representation of the space immediately surrounding the body, known as peripersonal space (PPS). However, progress in this field has been hampered by the lack of an agreed upon definition of PPS. Since the beginning of its formal study, scientists have argued that PPS plays a crucial role in both defensive and non-defensive actions. Yet consensus is lacking about the cognitive and neural instantiation of these functions. In particular, researchers have begun to ask whether a single, unified system of spatial-attentional resources supports both the defensive and non-defensive functions of PPS or, rather, whether there are multiple, independent systems. Moreover, there are open questions about the specificity of PPS. For example: Does PPS dissociate from other well-known phenomena such as personal space and the body schema? Finally, emerging research has brought attention to important questions about individual differences in the flexibility of PPS and the distribution of PPS in front compared to behind the body. In this advanced review, we shed light on questions about the nature of PPS, offering answers when the research permits or providing recommendations for achieving answers in future research. In so doing, we lay the groundwork for a comprehensive definition of PPS. This article is categorized under: Cognitive Biology > Evolutionary Roots of Cognition Psychology > Attention Psychology > Perception and Psychophysics Neuroscience > Plasticity.

Frontier of Self and Impact Prediction

The construction of a coherent representation of our body and the mapping of the space immediately surrounding it are of the highest ecological importance. This space has at least three specificities: it is a space where actions are planned in order to interact with our environment; it is a space that contributes to the experience of self and self-boundaries, through tactile processing and multisensory interactions; last, it is a space that contributes to the experience of body integrity against external events. In the last decades, numerous studies have been interested in peripersonal space (PPS), defined as the space directly surrounding us and which we can interact with (for reviews, see Cléry et al., 2015b; de Vignemont and Iannetti, 2015; di Pellegrino and Làdavas, 2015). These studies have contributed to the understanding of how this space is constructed, encoded and modulated. The majority of these studies focused on subparts of PPS (the hand, the face or the trunk) and very few of them investigated the interaction between PPS subparts. In the present review, we summarize the latest advances in this research and we discuss the new perspectives that are set forth for futures investigations on this topic. We describe the most recent methods used to estimate PPS boundaries by the means of dynamic stimuli. We then highlight how impact prediction and approaching stimuli modulate this space by social, emotional and action-related components involving principally a parieto-frontal network. In a next step, we review evidence that there is not a unique representation of PPS but at least three sub-sections (hand, face and trunk PPS). Last, we discuss how these subspaces interact, and we question whether and how bodily self-consciousness (BSC) is functionally and behaviorally linked to PPS.

Front-Presented Looming Sound Selectively Alters the Perceived Size of a Visual Looming Object

In spite of accumulating evidence for the spatial rule governing cross-modal interaction according to the spatial consistency of stimuli, it is still unclear whether 3D spatial consistency (i.e., front/rear of the body) of stimuli also regulates audiovisual interaction. We investigated how sounds with increasing/decreasing intensity (looming/receding sound) presented from the front and rear space of the body impact the size perception of a dynamic visual object. Participants performed a size-matching task (Experiments 1 and 2) and a size adjustment task (Experiment 3) of visual stimuli with increasing/decreasing diameter, while being exposed to a front- or rear-presented sound with increasing/decreasing intensity. Throughout these experiments, we demonstrated that only the front-presented looming sound caused overestimation of the spatially consistent looming visual stimulus in size, but not of the spatially inconsistent and the receding visual stimulus. The receding sound had no significant effect on vision. Our results revealed that looming sound alters dynamic visual size perception depending on the consistency in the approaching quality and the front-rear spatial location of audiovisual stimuli, suggesting that the human brain differently processes audiovisual inputs based on their 3D spatial consistency. This selective interaction between looming signals should contribute to faster detection of approaching threats. Our findings extend the spatial rule governing audiovisual interaction into 3D space.

Audiovisual integration in depth: multisensory binding and gain as a function of distance

The integration of information across sensory modalities is dependent on the spatiotemporal characteristics of the stimuli that are paired. Despite large variation in the distance over which events occur in our environment, relatively little is known regarding how stimulus-observer distance affects multisensory integration. Prior work has suggested that exteroceptive stimuli are integrated over larger temporal intervals in near relative to far space, and that larger multisensory facilitations are evident in far relative to near space. Here, we sought to examine the interrelationship between these previously established distance-related features of multisensory processing. Participants performed an audiovisual simultaneity judgment and redundant target task in near and far space, while audiovisual stimuli were presented at a range of temporal delays (i.e., stimulus onset asynchronies). In line with the previous findings, temporal acuity was poorer in near relative to far space. Furthermore, reaction time to asynchronously presented audiovisual targets suggested a temporal window for fast detection-a range of stimuli asynchronies that was also larger in near as compared to far space. However, the range of reaction times over which multisensory response enhancement was observed was limited to a restricted range of relatively small (i.e., 150 ms) asynchronies, and did not differ significantly between near and far space. Furthermore, for synchronous presentations, these distance-related (i.e., near vs. far) modulations in temporal acuity and multisensory gain correlated negatively at an individual subject level. Thus, the findings support the conclusion that multisensory temporal binding and gain are asymmetrically modulated as a function of distance from the observer, and specifies that this relationship is specific for temporally synchronous audiovisual stimulus presentations.

Trajectory Discrimination and Peripersonal Space Perception in Newborns

The ability to discriminate the trajectories of moving objects is highly adaptive and fundamental for physical and social interactions. Therefore, we could reasonably expect sensitivity to different trajectories already at birth, as a precursor of later communicative and defensive abilities. To investigate this possibility, we measured newborns' looking behavior to evaluate their ability to discriminate between visual stimuli depicting motion along different trajectories happening within the space surrounding their body. Differently from previous studies, we did not take into account defensive reactions, which may not be elicited by impending collision as newborns might not categorize approaching stimuli as possible dangers. In two experiments, we showed that newborns display a spontaneous visual preference for trajectories directed toward their body. We found this visual preference when visual stimuli depicted motion in opposite directions (approaching vs. receding) as well as when they both moved toward the peripersonal space and differed only in their specific target (i.e., the body vs. the space around it). These findings suggest that at birth human infants seem to be already equipped with visual mechanisms predisposing them to perceive their presence in the environment and to adaptively focus their attention on the peripersonal space and their bodily self.

Audio Motor Training at the Foot Level Improves Space Representation

Spatial representation is developed thanks to the integration of visual signals with the other senses. It has been shown that the lack of vision compromises the development of some spatial representations. In this study we tested the effect of a new rehabilitation device called ABBI (Audio Bracelet for Blind Interaction) to improve space representation. ABBI produces an audio feedback linked to body movement. Previous studies from our group showed that this device improves the spatial representation of space in early blind adults around the upper part of the body. Here we evaluate whether the audio motor feedback produced by ABBI can also improve audio spatial representation of sighted individuals in the space around the legs. Forty five blindfolded sighted subjects participated in the study, subdivided into three experimental groups. An audio space localization (front-back discrimination) task was performed twice by all groups of subjects before and after different kind of training conditions. A group (experimental) performed an audio-motor training with the ABBI device placed on their foot. Another group (control) performed a free motor activity without audio feedback associated with body movement. The other group (control) passively listened to the ABBI sound moved at foot level by the experimenter without producing any body movement. Results showed that only the experimental group, which performed the training with the audio-motor feedback, showed an improvement in accuracy for sound discrimination. No improvement was observed for the two control groups. These findings suggest that the audio-motor training with ABBI improves audio space perception also in the space around the legs in sighted individuals. This result provides important inputs for the rehabilitation of the space representations in the lower part of the body.

Responding to sounds from unseen locations: crossmodal attentional orienting in response to sounds presented from the rear

To date, most of the research on spatial attention has focused on probing people's responses to stimuli presented in frontal space. That is, few researchers have attempted to assess what happens in the space that is currently unseen (essentially rear space). In a sense, then, 'out of sight' is, very much, 'out of mind'. In this review, we highlight what is presently known about the perception and processing of sensory stimuli (focusing on sounds) whose source is not currently visible. We briefly summarize known differences in the localizability of sounds presented from different locations in 3D space, and discuss the consequences for the crossmodal attentional and multisensory perceptual interactions taking place in various regions of space. The latest research now clearly shows that the kinds of crossmodal interactions that take place in rear space are very often different in kind from those that have been documented in frontal space. Developing a better understanding of how people respond to unseen sound sources in naturalistic environments by integrating findings emerging from multiple fields of research will likely lead to the design of better warning signals in the future. This review highlights the need for neuroscientists interested in spatial attention to spend more time researching what happens (in terms of the covert and overt crossmodal orienting of attention) in rear space.

Modeling the Perception of Audiovisual Distance: Bayesian Causal Inference and Other Models

Studies of audiovisual perception of distance are rare. Here, visual and auditory cue interactions in distance are tested against several multisensory models, including a modified causal inference model. In this causal inference model predictions of estimate distributions are included. In our study, the audiovisual perception of distance was overall better explained by Bayesian causal inference than by other traditional models, such as sensory dominance and mandatory integration, and no interaction. Causal inference resolved with probability matching yielded the best fit to the data. Finally, we propose that sensory weights can also be estimated from causal inference. The analysis of the sensory weights allows us to obtain windows within which there is an interaction between the audiovisual stimuli. We find that the visual stimulus always contributes by more than 80% to the perception of visual distance. The visual stimulus also contributes by more than 50% to the perception of auditory distance, but only within a mobile window of interaction, which ranges from 1 to 4 m.

Echoic Sensory Substitution Information in a Single Obstacle Circumvention Task

Accurate motor control is required when walking around obstacles in order to avoid collisions. When vision is unavailable, sensory substitution can be used to improve locomotion through the environment. Tactile sensory substitution devices (SSDs) are electronic travel aids, some of which indicate the distance of an obstacle using the rate of vibration of a transducer on the skin. We investigated how accurately such an SSD guided navigation in an obstacle circumvention task. Using an SSD, 12 blindfolded participants navigated around a single flat 0.6 x 2 m obstacle. A 3-dimensional Vicon motion capture system was used to quantify various kinematic indices of human movement. Navigation performance under full vision was used as a baseline for comparison. The obstacle position was varied from trial to trial relative to the participant, being placed at two distances 25 cm to the left, right or directly ahead. Under SSD guidance, participants navigated without collision in 93% of trials. No collisions occurred under visual guidance. Buffer space (clearance between the obstacle and shoulder) was larger by a factor of 2.1 with SSD guidance than with visual guidance, movement times were longer by a factor of 9.4, and numbers of velocity corrections were larger by a factor of 5 (all p<0.05). Participants passed the obstacle on the side affording the most space in the majority of trials for both SSD and visual guidance conditions. The results are consistent with the idea that SSD information can be used to generate a protective envelope during locomotion in order to avoid collisions when navigating around obstacles, and to pass on the side of the obstacle affording the most space in the majority of trials.

Multisensory simultaneity judgment and proximity to the body

The integration of information across different sensory modalities is known to be dependent upon the statistical characteristics of the stimuli to be combined. For example, the spatial and temporal proximity of stimuli are important determinants with stimuli that are close in space and time being more likely to be bound. These multisensory interactions occur not only for singular points in space/time, but over “windows” of space and time that likely relate to the ecological statistics of real-world stimuli. Relatedly, human psychophysical work has demonstrated that individuals are highly prone to judge multisensory stimuli as co-occurring over a wide range of time—a so-called simultaneity window (SW). Similarly, there exists a spatial representation of peripersonal space (PPS) surrounding the body in which stimuli related to the body and to external events occurring near the body are highly likely to be jointly processed. In the current study, we sought to examine the interaction between these temporal and spatial dimensions of multisensory representation by measuring the SW for audiovisual stimuli through proximal–distal space (i.e., PPS and extrapersonal space). Results demonstrate that the audiovisual SWs within PPS are larger than outside PPS. In addition, we suggest that this effect is likely due to an automatic and additional computation of these multisensory events in a body-centered reference frame. We discuss the current findings in terms of the spatiotemporal constraints of multisensory interactions and the implication of distinct reference frames on this process.

Depth: the Forgotten Dimension in Multisensory Research

The last quarter of a century has seen a dramatic rise of interest in the spatial constraints on multisensory integration. However, until recently, the majority of this research has investigated integration in the space directly in front of the observer. The space around us, however, extends in three spatial dimensions in the front and to the rear beyond such a limited area. The question to be addressed in this review concerns whether multisensory integration operates according to the same rules throughout the whole of three-dimensional space. The results reviewed here not only show that the space around us seems to be divided into distinct functional regions, but they also suggest that multisensory interactions are modulated by the region of space in which stimuli happen to be presented. We highlight a number of key limitations with previous research in this area, including: (1) The focus on only a very narrow region of two-dimensional space in front of the observer; (2) the use of static stimuli in most research; (3) the study of observers who themselves have been mostly static; and (4) the study of isolated observers. All of these factors may change the way in which the senses interact at any given distance, as can the emotional state/personality of the observer. In summarizing these salient issues, we hope to encourage researchers to consider these factors in their own research in order to gain a better understanding of the spatial constraints on multisensory integration as they affect us in our everyday life.

Body part-centered and full body-centered peripersonal space representations

Dedicated neural systems represent the space surrounding the body, termed Peripersonal space (PPS), by integrating visual or auditory stimuli occurring near the body with somatosensory information. As a behavioral proxy to PPS, we measured participants' reaction time to tactile stimulation while task-irrelevant auditory or visual stimuli were presented at different distances from their body. In 7 experiments we delineated the critical distance at which auditory or visual stimuli boosted tactile processing on the hand, face, and trunk as a proxy of the PPS extension. Three main findings were obtained. First, the size of PPS varied according to the stimulated body part, being progressively bigger for the hand, then face, and largest for the trunk. Second, while approaching stimuli always modulated tactile processing in a space-dependent manner, receding stimuli did so only for the hand. Finally, the extension of PPS around the hand and the face varied according to their relative positioning and stimuli congruency, whereas the trunk PPS was constant. These results suggest that at least three body-part specific PPS representations exist, differing in extension and directional tuning. These distinct PPS representations, however, are not fully independent from each other, but referenced to the common reference frame of the trunk.

Spatiotemporal Processing in Crossmodal Interactions for Perception of the External World: A Review

Research regarding crossmodal interactions has garnered much interest in the last few decades. A variety of studies have demonstrated that multisensory information (vision, audition, tactile sensation, and so on) can perceptually interact with each other in the spatial and temporal domains. Findings regarding crossmodal interactions in the spatiotemporal domain (i.e., motion processing) have also been reported, with updates in the last few years. In this review, we summarize past and recent findings on spatiotemporal processing in crossmodal interactions regarding perception of the external world. A traditional view regarding crossmodal interactions holds that vision is superior to audition in spatial processing, but audition is dominant over vision in temporal processing. Similarly, vision is considered to have dominant effects over the other sensory modalities (i.e., visual capture) in spatiotemporal processing. However, recent findings demonstrate that sound could have a driving effect on visual motion perception. Moreover, studies regarding perceptual associative learning reported that, after association is established between a sound sequence without spatial information and visual motion information, the sound sequence could trigger visual motion perception. Other sensory information, such as motor action or smell, has also exhibited similar driving effects on visual motion perception. Additionally, recent brain imaging studies demonstrate that similar activation patterns could be observed in several brain areas, including the motion processing areas, between spatiotemporal information from different sensory modalities. Based on these findings, we suggest that multimodal information could mutually interact in spatiotemporal processing in the percept of the external world and that common perceptual and neural underlying mechanisms would exist for spatiotemporal processing.

On the relative contributions of multisensory integration and crossmodal exogenous spatial attention to multisensory response enhancement

Two processes that can give rise to multisensory response enhancement (MRE) are multisensory integration (MSI) and crossmodal exogenous spatial attention. It is, however, currently unclear what the relative contribution of each of these is to MRE. We investigated this issue using two tasks that are generally assumed to measure MSI (a redundant target effect task) and crossmodal exogenous spatial attention (a spatial cueing task). One block of trials consisted of unimodal auditory and visual targets designed to provide a unimodal baseline. In two other blocks of trials, the participants were presented with spatially and temporally aligned and misaligned audiovisual (AV) targets (0, 50, 100, and 200ms SOA). In the integration block, the participants were instructed to respond to the onset of the first target stimulus that they detected (A or V). The instruction for the cueing block was to respond only to the onset of the visual targets. The targets could appear at one of three locations: left, center, and right. The participants were instructed to respond only to lateral targets. The results indicated that MRE was caused by MSI at 0ms SOA. At 50ms SOA, both crossmodal exogenous spatial attention and MSI contributed to the observed MRE, whereas the MRE observed at the 100 and 200ms SOAs was attributable to crossmodal exogenous spatial attention, alerting, and temporal preparation. These results therefore suggest that there may be a temporal window in which both MSI and exogenous crossmodal spatial attention can contribute to multisensory response enhancement.

A geometric model of defensive peripersonal space

Potentially harmful stimuli occurring within the defensive peripersonal space (DPPS), a protective area surrounding the body, elicit stronger defensive reactions. The spatial features of the DPPS are poorly defined and limited to descriptive estimates of its extent along a single dimension. Here we postulated a family of geometric models of the DPPS, to address two important questions with respect to its spatial features: What is its fine-grained topography? How does the nervous system represent the body area to be defended? As a measure of the DPPS, we used the strength of the defensive blink reflex elicited by electrical stimulation of the hand (hand-blink reflex, HBR), which is reliably modulated by the position of the stimulated hand in egocentric coordinates. We tested the goodness of fit of the postulated models to HBR data from six experiments in which we systematically explored the HBR modulation by hand position in both head-centered and body-centered coordinates. The best-fitting model indicated that 1) the nervous system's representation of the body area defended by the HBR can be approximated by a half-ellipsoid centered on the face and 2) the DPPS extending from this area has the shape of a bubble elongated along the vertical axis. Finally, the empirical observation that the HBR is modulated by hand position in head-centered coordinates indicates that the DPPS is anchored to the face. The modeling approach described in this article can be generalized to describe the spatial modulation of any defensive response.

Multisensory Stimulation to Improve Low- and Higher-Level Sensory Deficits after Stroke: A Systematic Review

The aim of this systematic review was to integrate and assess evidence for the effectiveness of multisensory stimulation (i.e., stimulating at least two of the following sensory systems: visual, auditory, and somatosensory) as a possible rehabilitation method after stroke. Evidence was considered with a focus on low-level, perceptual (visual, auditory and somatosensory deficits), as well as higher-level, cognitive, sensory deficits. We referred to the electronic databases Scopus and PubMed to search for articles that were published before May 2015. Studies were included which evaluated the effects of multisensory stimulation on patients with low- or higher-level sensory deficits caused by stroke. Twenty-one studies were included in this review and the quality of these studies was assessed (based on eight elements: randomization, inclusion of control patient group, blinding of participants, blinding of researchers, follow-up, group size, reporting effect sizes, and reporting time post-stroke). Twenty of the twenty-one included studies demonstrate beneficial effects on low- and/or higher-level sensory deficits after stroke. Notwithstanding these beneficial effects, the quality of the studies is insufficient for valid conclusion that multisensory stimulation can be successfully applied as an effective intervention. A valuable and necessary next step would be to set up well-designed randomized controlled trials to examine the effectiveness of multisensory stimulation as an intervention for low- and/or higher-level sensory deficits after stroke. Finally, we consider the potential mechanisms of multisensory stimulation for rehabilitation to guide this future research.

Speed of reaction to sensory stimulation is enhanced during movement

We report four experiments on the speed of people's reactions to sensory stimulation while throwing and catching a basketball. Thirty participants participated in Experiment 1, split according to basketball expertise: none, intermediate (6years on average), or advanced (20years or more). The participants had to catch a bouncing basketball. The movement triggered a short tactile pulse in a tactor attached to their wrist to which they made a speeded vocal response (RT). The pulse could be presented either at rest, at two time-points during the reaching movement, or when the hand reached forward to catch the ball. The results indicated that participants responded more rapidly to vibrations on the moving hand relative to preparing or catching the ball, with expert athletes responding significantly faster than novices. In a second experiment, participants made a speeded vocal response to an auditory signal. As in Experiment 1, faster auditory RTs were observed when the hand was moving, as compared to the other time-points. In a third study, the participants responded to a pulse delivered at their resting hand at various time-points corresponding to the average timings of stimulation in Experiment 1. The results revealed comparable RTs across the tested time-points. In a final experiment, the participants made a vocal response to a pulse presented at various time-points while they were throwing the basketball. The results indicated faster tactile RTs while the ball was being thrown. These results are discussed with reference to the literature on goal-directed movements and in terms of current theories of attention and sensory suppression.

Audiovisual crossmodal cuing effects in front and rear space

The participants in the present study had to make speeded elevation discrimination responses to visual targets presented to the left or right of central fixation following the presentation of a task-irrelevant auditory cue on either the same or opposite side. In Experiment 1, the cues were presented from in front of the participants (from the same azimuthal positions as the visual targets). A standard crossmodal exogenous spatial cuing effect was observed, with participants responding significantly faster in the elevation discrimination task to visual targets when both the auditory cues and the visual targets were presented on the same side. Experiment 2 replicated the exogenous spatial cuing effect for frontal visual targets following both front and rear auditory cues. The results of Experiment 3 demonstrated that the participants had little difficulty in correctly discriminating the location from which the sounds were presented. Thus, taken together, the results of the three experiments reported here demonstrate that the exact co-location of auditory cues and visual targets is not necessary to attract spatial attention. Implications of these results for the design of real-world warning signals are discussed.