Visual-somatotopic interactions in spatial perception

Rescaling perceptual hand maps by visual-tactile recalibration

After concurrent visual and tactile stimuli have been presented repeatedly with a spatial offset, unisensory tactile stimuli, too, are perceived with a spatial bias towards the previously presented visual stimuli. This so-called visual-tactile ventriloquism aftereffect reflects crossmodal recalibration. As touch is intrinsically linked to body parts, we asked here whether recalibration occurs at the level of individual stimuli or at a higher, integrated, map-like level. We applied tactile stimuli to participants' hidden left hand and simultaneously presented visual stimuli with spatial offsets that, if integrated with the tactile stimuli, implied a larger hand. After recalibration, participants pointed to tactile-only stimuli and judged the distance between two tactile stimuli on the hand. The pattern of changes in tactile localization after recalibration was consistent with participants aiming at targets on an enlarged hand. This effect was evident also for new, tactile-only locations that had not been paired with visual stimuli during recalibration. In contrast, distance judgements were not consistently affected by recalibration. The generalization of recalibration to new, non-trained stimulus sites, but not across tasks and responses, suggests a link of low-level multisensory processing and map-like body representations that may, however, be purpose-specific and not organized as a general-purpose "body schema".

Quantifying accuracy and precision from continuous response data in studies of spatial perception and crossmodal recalibration

The ability to detect the absolute location of sensory stimuli can be quantified with either error-based metrics derived from single-trial localization errors or regression-based metrics derived from a linear regression of localization responses on the true stimulus locations. Here we tested the agreement between these two approaches in estimating accuracy and precision in a large sample of 188 subjects who localized auditory stimuli from different azimuthal locations. A subsample of 57 subjects was subsequently exposed to audiovisual stimuli with a consistent spatial disparity before performing the sound localization test again, allowing us to additionally test which of the different metrics best assessed correlations between the amount of crossmodal spatial recalibration and baseline localization performance. First, our findings support a distinction between accuracy and precision. Localization accuracy was mainly reflected in the overall spatial bias and was moderately correlated with precision metrics. However, in our data, the variability of single-trial localization errors (variable error in error-based metrics) and the amount by which the eccentricity of target locations was overestimated (slope in regression-based metrics) were highly correlated, suggesting that intercorrelations between individual metrics need to be carefully considered in spatial perception studies. Secondly, exposure to spatially discrepant audiovisual stimuli resulted in a shift in bias toward the side of the visual stimuli (ventriloquism aftereffect) but did not affect localization precision. The size of the aftereffect shift in bias was at least partly explainable by unspecific test repetition effects, highlighting the need to account for inter-individual baseline differences in studies of spatial learning.

Direction-selective modulation of visual motion rivalry by collocated tactile motion

Early models of multisensory integration posited that cross-modal signals only converged in higher-order association cortices and that vision automatically dominates. However, recent studies have challenged this view. In this study, the significance of the alignment of motion axes and spatial alignment across visual and tactile stimuli, as well as the effect of hand visibility on visuo-tactile interactions were examined. Using binocular rivalry, opposed motions were presented to each eye and participants were required to track the perceived visual direction. A tactile motion that was either a leftward or rightward sweep across the fingerpad was intermittently presented. Results showed that tactile effects on visual percepts were dependent on the alignment of motion axes: rivalry between up/down visual motions was not modulated at all by left/right tactile motion. On the other hand, visual percepts could be altered by tactile motion signals when both modalities shared a common axis of motion: a tactile stimulus could maintain the dominance duration of a congruent visual stimulus and shorten its suppression period. The effects were also conditional on the spatial alignment of the visual and tactile stimuli, being eliminated when the tactile device was displaced 15 cm away to the right of the visual stimulus. In contrast, visibility of the hand touching the tactile stimulus facilitated congruent switches relative to a visual-only baseline but did not present a significant advantage overall. In sum, these results show a low-level sensory interaction that is conditional on visual and tactile stimuli sharing a common motion axis and location in space.

Evaluating the Effect of Stimulus Duration on Vibrotactile Cue Localizability With a Tactile Sleeve

Vibrotactile arrays are appealing as wearable haptic devices, since designers can vary parameters including cue location and duration to create distinct haptic icons to represent a wide range of information. Vibrotactile sleeves have typically used cues that vary in duration from 100 to 400 ms, but it is not well understood how cue duration might affect localizability of stimuli. Using an experimental protocol typically employed to understand how our visual system can localize stimuli, we examined localization of tactile cues for tactors spaced at fixed locations along the forearm while we varied cue duration between 100 and 400 ms. To validate our experimental methods and hardware, we also evaluated visual cue localization performance. Our visual cue localization results were in agreement with prior experiments showing that varying noise in visual cues affects cue localization. More importantly, this experimental paradigm allowed us to verify that participants could successfully localize tactile cues regardless of duration. Response variance in tactile localizability was much greater than the visual case. There was also an effect of stimulus location on tactile localization performance. Our findings support the variation of tactile cue duration in the 100 to 400 ms range for tactile arrays positioned on the forearm.

Modality-specific attention attenuates visual-tactile integration and recalibration effects by reducing prior expectations of a common source for vision and touch

At any moment in time, streams of information reach the brain through the different senses. Given this wealth of noisy information, it is essential that we select information of relevance - a function fulfilled by attention - and infer its causal structure to eventually take advantage of redundancies across the senses. Yet, the role of selective attention during causal inference in cross-modal perception is unknown. We tested experimentally whether the distribution of attention across vision and touch enhances cross-modal spatial integration (visual-tactile ventriloquism effect, Expt. 1) and recalibration (visual-tactile ventriloquism aftereffect, Expt. 2) compared to modality-specific attention, and then used causal-inference modeling to isolate the mechanisms behind the attentional modulation. In both experiments, we found stronger effects of vision on touch under distributed than under modality-specific attention. Model comparison confirmed that participants used Bayes-optimal causal inference to localize visual and tactile stimuli presented as part of a visual-tactile stimulus pair, whereas simultaneously collected unity judgments - indicating whether the visual-tactile pair was perceived as spatially-aligned - relied on a sub-optimal heuristic. The best-fitting model revealed that attention modulated sensory and cognitive components of causal inference. First, distributed attention led to an increase of sensory noise compared to selective attention toward one modality. Second, attending to both modalities strengthened the stimulus-independent expectation that the two signals belong together, the prior probability of a common source for vision and touch. Yet, only the increase in the expectation of vision and touch sharing a common source was able to explain the observed enhancement of visual-tactile integration and recalibration effects with distributed attention. In contrast, the change in sensory noise explained only a fraction of the observed enhancements, as its consequences vary with the overall level of noise and stimulus congruency. Increased sensory noise leads to enhanced integration effects for visual-tactile pairs with a large spatial discrepancy, but reduced integration effects for stimuli with a small or no cross-modal discrepancy. In sum, our study indicates a weak a priori association between visual and tactile spatial signals that can be strengthened by distributing attention across both modalities.

The Ventriloquist Illusion as a Tool to Study Multisensory Processing: An Update

Ventriloquism, the illusion that a voice appears to come from the moving mouth of a puppet rather than from the actual speaker, is one of the classic examples of multisensory processing. In the laboratory, this illusion can be reliably induced by presenting simple meaningless audiovisual stimuli with a spatial discrepancy between the auditory and visual components. Typically, the perceived location of the sound source is biased toward the location of the visual stimulus (the ventriloquism effect). The strength of the visual bias reflects the relative reliability of the visual and auditory inputs as well as prior expectations that the two stimuli originated from the same source. In addition to the ventriloquist illusion, exposure to spatially discrepant audiovisual stimuli results in a subsequent recalibration of unisensory auditory localization (the ventriloquism aftereffect). In the past years, the ventriloquism effect and aftereffect have seen a resurgence as an experimental tool to elucidate basic mechanisms of multisensory integration and learning. For example, recent studies have: (a) revealed top-down influences from the reward and motor systems on cross-modal binding; (b) dissociated recalibration processes operating at different time scales; and (c) identified brain networks involved in the neuronal computations underlying multisensory integration and learning. This mini review article provides a brief overview of established experimental paradigms to measure the ventriloquism effect and aftereffect before summarizing these pathbreaking new advancements. Finally, it is pointed out how the ventriloquism effect and aftereffect could be utilized to address some of the current open questions in the field of multisensory research.

Recalibrating the body: visuotactile ventriloquism aftereffect

Visuotactile ventriloquism is a recently reported effect showing that somatotopic tactile representations (namely, representation of location along the surface of one’s arm) can be biased by simultaneous presentation of a visual stimulus in a spatial localization task along the surface of the skin. Here we investigated whether the exposure to discrepancy between tactile and visual stimuli on the skin can induce lasting changes in the somatotopic representations of space. We conducted an experiment investigating this question by asking participants to perform a localization task that included unisensory and bisensory trials, before and after exposure to spatially discrepant visuotactile stimuli. Participants localized brief flashes of light and brief vibrations that were presented along the surface of their forearms, and were presented either individually (unisensory conditions) or were presented simultaneously at the same location or different locations. We then compared the localization of tactile stimuli in unisensory tactile conditions before and after the exposure to discrepant bisensory stimuli. After exposure, participants exhibited a shift in their tactile localizations in the direction of the visual stimulus that was presented during the exposure block. These results demonstrate that the somatotopic spatial representations are capable of rapidly recalibrating after a very brief exposure to visually discrepant stimuli.