Evidence for dissociation between the perceptual and visuomotor systems in humans

Distinct temporal developments of visual motion and position representations for multi-stream visuomotor coordination

A fundamental but controversial question in information coding of moving visual target is which of 'motion' or 'position' signal is employed in the brain for producing quick motor reactions. Prevailing theory assumed that visually guided reaching is driven always via target position representation influenced by various motion signals (e.g., target texture and surroundings). To rigorously examine this theory, we manipulated the nature of the influence of internal texture motion on the position representation of the target in reaching correction tasks. By focusing on the difference in illusory position shift of targets with the soft- and hard-edges, we succeeded in extracting the temporal development of an indirect effect only ascribed to changes in position representation. Our data revealed that the onset of indirect effect is significantly slower than the adjustment onset itself. This evidence indicates multi-stream processing in visuomotor control: fast and direct contribution of visual motion for quick action initiation, and relatively slow contribution of position representation updated by relevant motion signals for continuous action regulation. The distinctive visuomotor mechanism would be crucial in successfully interacting with time-varying environments in the real world.

The faster you decide, the more accurate localization is possible: Position representation of "curveball illusion" in perception and eye movements

When the inside texture of a moving object moves, the perceived motion of the object is often distorted toward the direction of the texture's motion (motion-induced position shift), and such perceptual distortion accumulates while the object is watched, causing what is known as the curveball illusion. In a recent study, however, the accumulation of the position error was not observed in saccadic eye movements. Here, we examined whether the position of the illusory object is represented independently in the perceptual and saccadic systems. In the experiments, the stimulus of the curveball illusion was adopted to examine the temporal change in the position representation for saccadic eye movements and for perception by varying the elapsed time from the input of visual information to saccade onset and perceptual judgment, respectively. The results showed that the temporal accumulation of the motion-induced position shift is observed not only in perception but also in saccadic eye movements. In the saccade tasks, the landing positions of saccades gradually shifted to the illusory perceived position as the elapsed time from the target offset to the saccade "go" signal increased. Furthermore, in the perception task, shortening the time between the target offset and the perceptual judgment reduced the size of the illusion effect. Therefore, these results argue against the idea of dissociation between saccadic and perceptual localization of a moving object suggested in the previous study, in which saccades were measured in a rushed way while perceptual responses were measured without time constraint. Instead, the similar temporal trends of these effects imply a common or similar target representation for perception and eye movements which dynamically changes over the course of evidence accumulation.

Optic ataxia: beyond the dorsal stream cliché

This chapter reviews clinical and scientific approaches to optic ataxia. This double historic track allows us to address important issues such as the link between Bálint syndrome and optic ataxia, the alleged double dissociation between optic ataxia and visual agnosia, and the use of optic ataxia to argue for a specific vision-for-action occipitoposterior parietal stream. Clinical cases are described and reveal that perceptual deficits have been long shown to accompany ataxia. Importantly, the term ataxia appears to be misleading as patients exhibit a combination of visual and nonvisual perceptual, attentional, and visuomotor guidance deficits, which are confirmed by experimental approaches. Three major features of optic ataxia are described. The first is a spatial feature whereby the deficits exhibited by patients appear to be specific to peripheral vision, akin to the field effect. Visuomotor field examination allows us to quantify this deficit and reveals that it consists of a highly reliable retinocentric hypometria. The third is a temporal feature whereby these deficits are exacerbated under temporal constraints, i.e., when attending to dynamic stimuli. These two aspects combine in a situation where patients have to quickly respond to a target presented in peripheral vision that is experimentally displaced upon movement onset. In addition to the field effect, a hand effect can be described in conditions where the hand is not visible. Spatial and temporal aspects as well as field and hand effects may rely on several posterior parietal modules that remain to be precisely identified both anatomically and functionally. It is concluded that optic ataxia is not a visuomotor deficit and there is no dissociation between perception and action capacities in optic ataxia, hence a fortiori no double dissociation between optic ataxia and visual agnosia. Future directions for understanding the basic pathophysiology of optic ataxia are proposed.

Errors in interception can be predicted from errors in perception

It has been hypothesised that our actions are less susceptible to visual illusions than our perceptual judgements because similar information is processed for perception and action in separate pathways. We test this hypothesis for subjects intercepting a moving object that appears to move at a different speed than its true speed due to an illusion. The object was a moving Gabor patch: a sinusoidal grating of which the luminance contrast is modulated by a two-dimensional Gaussian. We manipulated the patch's apparent speed by moving the grating relative to the Gaussian. We used separate two-interval forced choice discrimination tasks to determine how moving the grating influenced ten people's judgements of the object's position and velocity while they were fixating. Based on their perceptual judgements, and knowing that our ability to correct for errors that arise from relying on incorrect judgements are limited by a sensorimotor delay of about 100 msec, we predicted the extent to which subjects would tap ahead of or behind similar targets when trying to intercept them at the fixation location. The predicted errors closely matched the actual errors that subjects made when trying to intercept the targets. This finding does not support the two visual streams hypothesis. The results are consistent with the idea that the extent to which an illusion influences an action tells us something about the extent to which the action relies on the percept in question.

Asymmetric Mislocalization of a Visual Flash Ahead of and behind a Moving Object

When subjects localize a flash relative to another stationary stimulus, the flash appears displaced in the direction of nearby motion signals (position capture; Whitney and Cavanagh, 2000 Nature Neuroscience3 954–959). Our previous study had suggested that the position capture is larger for a flash presented ahead of a moving stimulus than for a flash behind it (Watanabe et al, 2003 Perception32 545–559). In the present study, I investigated the spatial asymmetry of position capture. Experiment 1 demonstrated that asymmetric position capture occurs primarily in a moving-object-centered coordinate. Experiment 2 showed evidence that the asymmetric position capture operates after individuation of single visual objects. Finally, experiment 3 demonstrated that, when attention was reduced with a dual-task procedure, the asymmetric position capture increased. These results suggest that the spatial asymmetry of position capture occurs without attention but the spatial bias can be reduced by attention. Therefore, the underlying mechanism for the asymmetric spatial bias may be different from attentive tracking (Cavanagh, 1992 Science257 1563–1565) and mislocalization during smooth pursuit (Brenner et al, 2001 Vision Research41 2253–2259).

Action can amplify motion-induced illusory displacement

Local motion is known to produce strong illusory displacement in the perceived position of globally static objects. For example, if a dot-cloud or grating drifts to the left within a stationary aperture, the perceived position of the whole aperture will also be shifted to the left. Previously, we used a simple tracking task to demonstrate that active control over the global position of an object did not eliminate this form of illusion. Here, we used a new iPad task to directly compare the magnitude of illusory displacement under active and passive conditions. In the active condition, participants guided a drifting Gabor patch along a virtual slalom course by using the tilt control of an iPad. The task was to position the patch so that it entered each gate at the direct center, and we used the left/right deviations from that point as our dependent measure. In the passive condition, participants watched playback of standardized trajectories along the same course. We systematically varied deviation from midpoint at gate entry, and participants made 2AFC left/right judgments. We fitted cumulative normal functions to individual distributions and extracted the point of subjective equality (PSE) as our dependent measure. To our surprise, the magnitude of displacement was consistently larger under active than under passive conditions. Importantly, control conditions ruled out the possibility that such amplification results from lack of motor control or differences in global trajectories as performance estimates were equivalent in the two conditions in the absence of local motion. Our results suggest that the illusion penetrates multiple levels of the perception-action cycle, indicating that one important direction for the future of perceptual illusions may be to more fully explore their influence during active vision.

Visual motion shifts saccade targets

Saccades are made thousands of times a day and are the principal means of localizing objects in our environment. However, the saccade system faces the challenge of accurately localizing objects as they are constantly moving relative to the eye and head. Any delays in processing could cause errors in saccadic localization. To compensate for these delays, the saccade system might use one or more sources of information to predict future target locations, including changes in position of the object over time, or its motion. Another possibility is that motion influences the represented position of the object for saccadic targeting, without requiring an actual change in target position. We tested whether the saccade system can use motion-induced position shifts to update the represented spatial location of a saccade target, by using static drifting Gabor patches with either a soft or a hard aperture as saccade targets. In both conditions, the aperture always remained at a fixed retinal location. The soft aperture Gabor patch resulted in an illusory position shift, whereas the hard aperture stimulus maintained the motion signals but resulted in a smaller illusory position shift. Thus, motion energy and target location were equated, but a position shift was generated in only one condition. We measured saccadic localization of these targets and found that saccades were indeed shifted, but only with a soft-aperture Gabor patch. Our results suggest that motion shifts the programmed locations of saccade targets, and this remapped location guides saccadic localization.

The influence of visual motion on interceptive actions and perception

Visual information is an essential guide when interacting with moving objects, yet it can also be deceiving. For instance, motion can induce illusory position shifts, such that a moving ball can seem to have bounced past its true point of contact with the ground. Some evidence suggests illusory motion-induced position shifts bias pointing tasks to a greater extent than they do perceptual judgments. This, however, appears at odds with other findings and with our success when intercepting moving objects. Here we examined the accuracy of interceptive movements and of perceptual judgments in relation to simulated bounces. Participants were asked to intercept a moving disc at its bounce location by positioning a virtual paddle, and then to report where the disc had landed. Results showed that interceptive actions were accurate whereas perceptual judgments were inaccurate, biased in the direction of motion. Successful interceptions necessitated accurate information concerning both the location and timing of the bounce, so motor planning evidently had privileged access to an accurate forward model of bounce timing and location. This would explain why people can be accurate when intercepting a moving object, but lack insight into the accurate information that had guided their actions when asked to make a perceptual judgment.

EEG correlates of haptic feedback in a visuomotor tracking task

This study investigates the temporal brain dynamics associated with haptic feedback in a visuomotor tracking task. Haptic feedback with deviation-related forces was used throughout tracking experiments in which subjects' behavioral responses and electroencephalogram (EEG) data were simultaneously measured. Independent component analysis was employed to decompose the acquired EEG signals into temporally independent time courses arising from distinct brain sources. Clustering analysis was used to extract independent components that were comparable across participants. The resultant independent brain processes were further analyzed via time-frequency analysis (event-related spectral perturbation) and event-related coherence (ERCOH) to contrast brain activity during tracking experiments with or without haptic feedback. Across subjects, in epochs with haptic feedback, components with equivalent dipoles in or near the right motor region exhibited greater alpha band power suppression. Components with equivalent dipoles in or near the left frontal, central, left motor, right motor, and parietal regions exhibited greater beta-band power suppression, while components with equivalent dipoles in or near the left frontal, left motor, and right motor regions showed greater gamma-band power suppression relative to non-haptic conditions. In contrast, the right occipital component cluster exhibited less beta-band power suppression in epochs with haptic feedback compared to non-haptic conditions. The results of ERCOH analysis of the six component clusters showed that there were significant increases in coherence between different brain networks in response to haptic feedback relative to the coherence observed when haptic feedback was not present. The results of this study provide novel insight into the effects of haptic feedback on the brain and may aid the development of new tools to facilitate the learning of motor skills.

Implicit online corrections of reaching movements

Many neuroscience studies of arm control have attempted to explain how aimed movements are planned, generated, and corrected. The mechanisms of subconscious online reaching correction to a target shift are now being widely examined from computational, physiological, and pathological viewpoints. Recent evidence of a quick manual response to surrounding visual motion suggests an additional online compensatory mechanism in reaching movements for bodily and/or external environmental changes, although the computational principle underlying this process remains controversial. Together with preprogrammed voluntary motor command generation, it appears that multiple online visually guided correction mechanisms implicitly govern reaching control to bring the hand to the goal. It is important to reveal unknown mechanisms and underlying neural substrates of generating the response to visual motion, which is additionally modulated by action contexts.

Position Perception: Influence of Motion With Displacement Dissociated From the Influence of Motion Alone

When humans view a moving object, the spatial lag in perception expected from neural delays may be partially corrected by motion mechanisms biasing perceived position. The drifting-Gabor illusion seems to support this view: the perceived location of a static envelope filled with a moving pattern is shifted in the direction of motion. To test whether this shifting mechanism also extrapolates the position of moving displacing objects, we compared the perceptual position shift for drifting versus displacing Gabors when the motion is toward the fovea and when the motion is away from the fovea. For displacing Gabors, the shift was much greater for motion toward the fovea, whereas for drifting Gabors, the shift was greater for motion away from the fovea. This dissociation suggests that the illusions are caused by different mechanisms.

Egocentric and allocentric localization during induced motion

This research examined motor measures of the apparent egocentric location and perceptual measures of the apparent allocentric location of a target that was being seen to undergo induced motion (IM). In Experiments 1 and 3, subjects fixated a stationary dot (IM target) while a rectangular surround stimulus (inducing stimulus) oscillated horizontally. The inducing stimulus motion caused the IM target to appear to move in the opposite direction. In Experiment 1, two dots (flashed targets) were flashed above and below the IM target when the surround had reached its leftmost or rightmost displacement from the subject's midline. Subjects pointed open-loop at either the apparent egocentric location of the IM target or at the bottom of the two flashed targets. On separate trials, subjects made judgments of the Vernier alignment of the IM target with the flashed targets at the endpoints of the surround's oscillation. The pointing responses were displaced in the direction of the previously seen IM for the IM target and to a lesser degree for the bottom flashed target. However, the allocentric Vernier judgments demonstrated no perceptual displacement of the IM target relative to the flashed targets. Thus, IM results in a dissociation of egocentric location measures from allocentric location measures. In Experiment 2, pointing and Vernier measures were obtained with stationary horizontally displaced surrounds and there was no dissociation of egocentric location measures from allocentric location measures. These results indicate that the Roelofs effect did not produce the pattern of results in Experiment 1. In Experiment 3, pointing and Vernier measures were obtained when the surround was at the midpoint of an oscillation. In this case, egocentric pointing responses were displaced in the direction of surround motion (opposite IM) for the IM target and to a greater degree for the bottom flashed target. However, there was no apparent displacement of the IM target relative to the flashed targets in the allocentric Vernier judgments. Therefore, in Experiment 3 egocentric location measures were again dissociated from allocentric location measures. The results of this experiment also demonstrate that IM does not generate an allocentric displacement illusion analogous to the "flash-lag" effect.

Local motion inside an object affects pointing less than smooth pursuit

During smooth pursuit eye movements, briefly presented objects are mislocalized in the direction of motion. It has been proposed that the localization error is the sum of the pursuit signal and the retinal motion signal in a ~200 ms interval after flash onset. To evaluate contributions of retinal motion signals produced by the entire object (global motion) and elements within the object (local motion), we asked observers to reach to flashed Gabor patches (Gaussian-windowed sine-wave gratings). Global motion was manipulated by varying the duration of a stationary flash, and local motion was manipulated by varying the motion of the sine-wave. Our results confirm that global retinal motion reduces the localization error. The effect of local retinal motion on object localization was far smaller, even though local and global motion had equal effects on eye velocity. Thus, local retinal motion has differential access to manual and oculomotor control circuits. Further, we observed moderate correlations between smooth pursuit gain and localization error.

The relative contributions of colour and luminance signals towards the visuomotor localisation of targets in human peripheral vision

We sought to determine the extent to which colour (and luminance) signals contribute towards the visuomotor localization of targets. To do so we exploited the movement-related illusory displacement a small stationary window undergoes when it has a continuously moving carrier grating behind it. We used drifting (1.0-4.2 Hz) red/green-modulated isoluminant gratings or yellow/black luminance-modulated gratings as carriers, each curtailed in space by a stationary, two-dimensional window. After each trial, the perceived location of the window was recorded with reference to an on-screen ruler (perceptual task) or the on-screen touch of a ballistic pointing movement made without visual feedback (visuomotor task). Our results showed that the perceptual displacement measures were similar for each stimulus type and weakly dependent on stimulus drift rate. However, while the visuomotor displacement measures were similar for each stimulus type at low drift rates (<4 Hz), they were significantly larger for luminance than colour stimuli at high drift rates (>4 Hz). We show that the latter cannot be attributed to differences in perceived speed between stimulus types. We assume, therefore, that our visuomotor localization judgements were more susceptible to the (carrier) motion of luminance patterns than colour patterns. We suggest that, far from being detrimental, this susceptibility may indicate the operation of mechanisms designed to counter the temporal asynchrony between perceptual experiences and the physical changes in the environment that give rise to them. We propose that perceptual localisation is equally supported by both colour and luminance signals but that visuomotor localisation is predominantly supported by luminance signals. We discuss the neural pathways that may be involved with visuomotor localization.

Visually guided reaching depends on motion area MT+

Visual information is crucial for goal-directed reaching. A number of studies have recently shown that motion in particular is an important source of information for the visuomotor system. For example, when reaching a stationary object, movement of the background can influence the trajectory of the hand, even when the background motion is irrelevant to the object and task. This manual following response may be a compensatory response to changes in body position, but the underlying mechanism remains unclear. Here we tested whether visual motion area MT+ is necessary to generate the manual following response. We found that stimulation of MT+ with transcranial magnetic stimulation significantly reduced a strong manual following response. MT+ is therefore necessary for generating the manual following response, indicating that it plays a crucial role in guiding goal-directed reaching movements by taking into account background motion in scenes.

Visual motion due to eye movements helps guide the hand

Movement of the body, head, or eyes with respect to the world creates one of the most common yet complex situations in which the visuomotor system must localize objects. In this situation, vestibular, proprioceptive, and extra-retinal information contribute to accurate visuomotor control. The utility of retinal motion information, on the other hand, is questionable, since a single pattern of retinal motion can be produced by any number of head or eye movements. Here we investigated whether retinal motion during a smooth pursuit eye movement contributes to visuomotor control. When subjects pursued a moving object with their eyes and reached to the remembered location of a separate stationary target, the presence of a moving background significantly altered the endpoints of their reaching movements. A background that moved with the pursuit, creating a retinally stationary image (no retinal slip), caused the endpoints of the reaching movements to deviate in the direction of pursuit, overshooting the target. A physically stationary background pattern, however, producing retinal image motion opposite to the direction of pursuit, caused reaching movements to become more accurate. The results indicate that background retinal motion is used by the visuomotor system in the control of visually guided action.

Motion-induced illusory displacement reexamined: differences between perception and action?

The position of a drifting sine-wave grating enveloped by a stationary Gaussian is misperceived in the direction of motion. Previous research indicated that the illusion was larger when observers pointed to the center of the stimulus than when they indicated the stimulus position on a ruler. This conclusion was reexamined. Observers pointed to the position of a small Gabor patch on the screen or compared its position to moving patches, stationary lines, or flashed lines. With moving patches, the illusion was larger with probe than with motor judgments; with stationary lines, the illusion was about the same size; and with flashed lines, the illusion was smaller with probe than with motor judgments. Thus, the comparison between perceptual and motor measures depended strongly on the methods used. Further, the target was mislocalized toward the fovea with motor judgments, whereas the target was displaced away from the fovea relative to line probes.

Action-specific extrapolation of target motion in human visual system

Neuropsychological studies have indicated two distinct visual pathways in our brain, one dedicated to conscious perception and one to visuomotor control. Some psychophysical results support this idea with normal subjects, but they are still controversial. This study provides new psychophysical evidence for the dissociation by showing action-specific extrapolation of the visual target trajectory. When a moving target disappears, the perceived final position is liable to be shifted forward (representational momentum). In experiment 1, larger and more robust forward shifts were found when the position was directly touched without seeing the screen (open-loop pointing) than when the position was judged perceptually. The most striking dissociation was that fixation did not affect the forward shift in open-loop pointing while it almost abolished the shifts in perceptual judgements. In experiment 2, this action-specific result was found to disappear after a response delay of 4000 ms. Experiments 3 and 4 confirmed that the results were not affected by the external reference frames. The specific forward shifts found in open-loop pointing suggest that the visuomotor system compensates for the neural delays by extrapolating the target motion. The results, together with earlier findings, lead to a psychophysical double dissociation of the two visual pathways.

Abstraction from a sensori-motor perspective: can we get a quick hold on simple perception?

Two main types of dissociation can be considered in order to articulate action and abstraction. Vision for action and vision for perception are often described as dissociated systems at both anatomical and functional levels. Within this framework, abstraction should be specific to perceptual representation, whereas the action system would simply analyse the objective metric of space. By contrast, one may focus on dissociations within the action system. In this case, one will accept that action may involve abstract representations, at least during movement preparation. But a specific visuomotor level of processing can be described that appears to comply with the spatial properties of the relationship between the actor and the environment. This system would be specialized for fast movement guidance towards pre-defined goals. Such an automatic piloting system would thus be free of abstraction.

The influence of visual motion on fast reaching movements to a stationary object

One of the most important functions of vision is to direct actions to objects. However, every time that vision is used to guide an action, retinal motion signals are produced by the movement of the eye and head as the person looks at the object or by the motion of other objects in the scene. To reach for the object accurately, the visuomotor system must separate information about the position of the stationary target from background retinal motion signals-a long-standing problem that is poorly understood. Here we show that the visuomotor system does not distinguish between these two information sources: when observers made fast reaching movements to a briefly presented stationary target, their hand shifted in a direction consistent with the motion of a distant and unrelated stimulus, a result contrary to most other findings. This can be seen early in the hand's trajectory (approximately 120 ms) and occurs continuously from programming of the movement through to its execution. The visuomotor system might make use of the motion signals arising from eye and head movements to update the positions of targets rapidly and redirect the hand to compensate for body movements.

The influence of visual motion on perceived position

The ability of the visual system to localize objects is one of its most important functions and yet remains one of the least understood, especially when either the object or the surrounding scene is in motion. The specific process that assigns positions under these circumstances is unknown, but two major classes of mechanism have emerged: spatial mechanisms that directly influence the coded locations of objects, and temporal mechanisms that influence the speed of perception. Disentangling these mechanisms is one of the first steps towards understanding how the visual system assigns locations to objects when there are motion signals present in the scene.