The role of vision in the on-line correction of illusion effects on action

Coming to grips with reality: Real grasps, but not pantomimed grasps, resist a simultaneous tilt illusion

Investigations of grasping real, 3D objects subjected to illusory effects from a pictorial background often choose in-flight grasp aperture as the primary variable to test the hypothesis that the visuomotor system resists the illusion. Here we test an equally important feature of grasps that has received less attention: in-flight grasp orientation. The current study tested a variant of the simultaneous tilt illusion using a mirror-apparatus to manipulate the availability of haptic feedback. Participants performed grasps with haptic feedback (real grasps) and without it (pantomime grasps), reaching for the reflection of a real, 3D bar atop a background grating that induced a 1.1° bias in the perceived orientation of the bar in a separate sample of participants. Analysis of the hand's in-flight grasp orientation at early, late, and end stages of the reach showed that at no point were the real grasps biased by the illusion. In contrast, pantomimed grasps were affected by the illusion at the late and end stages of the reach. At each stage, the effect on the real grasps was significantly weaker than the effect of the illusion as measured by the mean point of subjective equality (PSE) in a two-alternative forced-choice task. In contrast, the effect on the pantomime grasps was statistically indistinguishable from the mean PSE at all three stages of the reach. These findings reinforce the idea that in-flight grasp orientation, like grasp aperture to pictorial illusions of target size, is refractory to pictorial backgrounds that bias perceived orientation.

A double dissociation between action and perception in bimanual grasping: evidence from the Ponzo and the Wundt-Jastrow illusions

Research on visuomotor control suggests that visually guided actions toward objects rely on functionally distinct computations with respect to perception. For example, a double dissociation between grasping and between perceptual estimates was reported in previous experiments that pit real against illusory object size differences in the context of the Ponzo illusion. While most previous research on the relation between action and perception focused on one-handed grasping, everyday visuomotor interactions also entail the simultaneous use of both hands to grasp objects that are larger in size. Here, we examined whether this double dissociation extends to bimanual movement control. In Experiment 1, participants were presented with different-sized objects embedded in the Ponzo Illusion. In Experiment 2, we tested whether the dissociation between perception and action extends to a different illusion, the Wundt–Jastrow illusion, which has not been previously used in grasping experiments. In both experiments, bimanual grasping trajectories reflected the differences in physical size between the objects; At the same time, perceptual estimates reflected the differences in illusory size between the objects. These results suggest that the double dissociation between action and perception generalizes to bimanual movement control. Unlike conscious perception, bimanual grasping movements are tuned to real-world metrics, and can potentially resist irrelevant information on relative size and depth.

Numerical magnitude affects online execution, and not planning of visuomotor control

Recent literature has established a directional influence of irrelevant numerical magnitude on actions performed toward neutral objects. For example, fingers' aperture during grasping is larger when associated with large compared with small numerical digits. This interaction between symbolic magnitude and visuomotor control has been attributed to the planning stage of the action prior to motor execution. However, this assumption has not been directly tested. In two experiments, we tested whether the effects of numerical magnitude on grasping derive from action planning or from action execution. Participants were asked to grasp an object following a short visual (Experiment 1) or auditory (Experiment 2) presentation of small (1/2) or large (8/9) digits. Grasping was performed under either closed-loop (CL) or open-loop (OL) visuomotor control, for which online vision was prevented during action execution. Digit magnitude affected grip apertures in the CL condition, when online vision was allowed. However, magnitude had no effects on grip aperture in the OL condition. This pattern of results strongly suggests that the processing of numerical magnitude originates from interactions between numerical magnitude and real object size during online motor execution. Unlike previously assumed, the findings also suggest that the effect of magnitude on visuomotor control is not likely to be attributed to the motor planning stage prior to action initiation.

Neural activity in the medial parietal area V6A while grasping with or without visual feedback

Recent works have reported that grasping movements are controlled not only by the dorsolateral visual stream, as generally thought, but also by the dorsomedial visual stream, and in particular by the medial posterior parietal area V6A. To date, the grasping activity of V6A neurons has been studied only in darkness. Here we studied the effect of visual feedback on grasp-related discharges of V6A neurons while the monkey was preparing and executing the grasping of a handle. We found that V6A grasping activity could be excited or inhibited by visual information. The neural population was divided into Visual, Motor, and Visuomotor cells. The majority of Visual and Visuomotor neurons did not respond to passive observation of the handle, suggesting that vision of action, rather than object vision, is the most effective factor. The present findings highlight the role of the dorsomedial visual stream in integrating visual and motor signals to monitor and correct grasping.

Visual control of action directed toward two-dimensional objects relies on holistic processing of object shape

Visual perception relies on holistic processing of object shape. In contrast to perception, previous studies demonstrated that vision-for-action operates in a fundamentally different manner based on an analytical representation of objects. This notion was mainly supported by the absence of Garner interference for visually guided actions, compared to robust interference effects for perceptual estimations of the same objects. This study examines the nature of the representations that subserve visually guided actions toward two-dimensional (2D) stimuli. Based on recent results suggesting that actions directed toward 2D objects are mediated by different underlying processes compared to normal actions, we predicted that visually guided actions toward 2D stimuli would rely on perceptually driven holistic representations of object shape. To test this idea, we asked participants to grasp 2D rectangular objects presented on a computer monitor along their width while the values of the irrelevant dimension of length were either kept constant (baseline condition) or varied between trials (filtering condition). Worse performance in the filtering blocks is labeled Garner interference, which indicates holistic processing of object shape. Unlike in previous studies that used real objects, the results showed that grasping toward 2D objects produced a significant Garner interference effect, with more variable within-subject performance in the filtering compared to the baseline blocks. This finding suggests that visually guided actions directed toward 2D targets are mediated by different computations compared to visually guided actions directed toward real objects.

Does the size-illusion effect on prehensile movements depend on preview duration for visuomotor process?

Given that visual estimation of an object's size is affected by an illusory figure, the present study investigates the Ebbinghaus size-illusion effect on visuomotor performance within different preview durations for viewing an object (no preview, 300, 700, 1500, and 3000 ms) before initiating the movement. Twenty participants performed the following actions: (a) grasping the object and (b) matching the perceived object size with the finger aperture configuration as in the grasping task. The illusion affected the grasping aperture size only in the no- and 300-ms preview durations, while the matching aperture was affected across all preview conditions. These results suggest that the preview duration influences the size illusion to affect the grasping performance, and subjects adopt different visuomotor processes, depending on preview duration.

Grasping without vision: time normalizing grip aperture profiles yields spurious grip scaling to target size

The analysis of normalized movement trajectories is a popular and informative technique used in investigations of visuomotor control during goal-directed acts like reaching and grasping. This technique typically involves standardizing measures against the amplitude of some other variable - most typically time. Here, we show that this normalizing technique can lead to some surprising results. In the first of two experiments, we asked participants to grasp target objects without ever seeing them from trial to trial. In the second experiment, participants were given a brief preview of the target and were then cued 3s later to pick it up while vision was prevented. Critically, on some trials during the delay period and unbeknownst to the participants, the previewed target was swapped for a new unseen one. The results of both experiments show that time-normalized measures of grip aperture during the closing phase of the movement appear to be scaled to target size well before the fingers make contact with the target - even though participants had no idea what the size of the target was that they were grasping. In contrast, a classical measure of anticipatory grip scaling, maximum grip aperture, did not show scaling to target size. As we demonstrate, however, in both experiments, movement time was longer for the larger target than the smaller ones. Thus, the comparisons of time-normalized grip aperture, particularly during the closing phase of the movements, were made across different points in real time. Taken together, the results of these experiments highlight a need for caution when investigators interpret differences in time-normalized dependent measures - particularly when the effect of interest is correlated with the dependent measure and a third variable (e.g., movement time) that is used to standardize the dependent measure.

The planning and control model (PCM) of motorvisual priming: reconciling motorvisual impairment and facilitation effects

Previous research on dual-tasks has shown that, under some circumstances, actions impair the perception of action-consistent stimuli, whereas, under other conditions, actions facilitate the perception of action-consistent stimuli. We propose a new model to reconcile these contrasting findings. The planning and control model (PCM) of motorvisual priming proposes that action planning binds categorical representations of action features so that their availability for perceptual processing is inhibited. Thus, the perception of categorically action-consistent stimuli is impaired during action planning. Movement control processes, on the other hand, integrate multi-sensory spatial information about the movement and, therefore, facilitate perceptual processing of spatially movement-consistent stimuli. We show that the PCM is consistent with a wider range of empirical data than previous models on motorvisual priming. Furthermore, the model yields previously untested empirical predictions. We also discuss how the PCM relates to motorvisual research paradigms other than dual-tasks.

The prior-antisaccade effect influences the planning and online control of prosaccades

The latency of a prosaccade is increased when completed following an antisaccade (the prior-antisaccade effect). This finding has been attributed to the inhibition of the oculomotor networks necessary for an antisaccade engendering a persistent response set that delays a to-be-executed prosaccade. The goal of the present investigation was to determine whether the prior-antisaccade effect influences not only the planning but also the control of an unfolding prosaccade trajectory. To accomplish that objective, we employed a task-switching paradigm wherein participants alternated between pro- and antisaccades on every second trial (i.e., AABB paradigm). Importantly, trajectory control was evaluated by computing the proportion of variance (R2 values) explained by the spatial position of the eye at decile increments of movement time relative to the response's ultimate movement endpoint: small R2 values indicate a response that unfolds with error-reducing trajectory amendments (i.e., online control), whereas larger R2 values reflect a response that unfolds with few-if any-online corrections. As expected, results showed a prior-antisaccade effect for response planning; that is, prosaccade latencies were increased when completed after an antisaccade. Moreover, prosaccades completed after an antisaccade elicited larger R2 values and less accurate endpoints than trials wherein a prosaccade was completed after another prosaccade. These results provide first evidence of a prior-antisaccade effect for trajectory control and indicate that the persistent and inhibitory response set arising from an antisaccade diminishes the online corrections, and thus endpoint accuracy, of a subsequent prosaccade.

Pro- and antisaccades: dissociating stimulus and response influences the online control of saccade trajectories

The authors examined whether the diminished online control of antisaccades is related to a trade-off between movement planning and control or the remapping of target properties to a mirror-symmetrical location (i.e., vector inversion). Pro- and antisaccades were examined in a standard no-delay schedule wherein target onset served as the movement imperative and a delay cuing schedule wherein responses were initiated 2,000 ms following target onset. Importantly, the delay cuing schedule was employed to equate pro- and antisaccade reaction times. Online control was evaluated by indexing the strength of trajectory amendments at normalized increments of movement time. Antisaccades exhibited fewer online corrections than prosaccades, and this result was consistent across cuing schedules. Thus, the diminished online control of antisaccades cannot be tied to a trade-off between movement planning and control. Rather, the authors propose that the intentional nature of dissociating stimulus and response (i.e., vector inversion) engenders a slow mode of cognitive control that is not optimized for fast oculomotor corrections.

Action without perception in human vision

In 1992, David Milner and I (Goodale & Milner, 1992) proposed a division of labour in the visual pathways of the primate cerebral cortex between a dorsal stream specialized for the visual control of action and a ventral stream dedicated to constructing our percepts of the visual world. Support for the perception-action distinction has come from neuroimaging experiments, human neuropsychology, and monkey neurophysiology. Differences in the timing and spatial metrics of vision-for-perception and vision-for-action have been studied in human psychophysical experiments, particularly in those that have looked at the way in which each system deals with pictorial illusions. Although the literature is not free from controversy, a large number of studies have found that actions such as grasping and reaching are often unaffected by high-level pictorial illusions, which by definition affect perception. Recent experiments have shown that for actions to escape the effects of such illusions, however, they must be highly practised actions, preferably with the right hand, and must be directed in real time at visible targets. But even though the behavioural evidence suggests that the dorsal and ventral streams make use of different timing, different metrics, and different frames of reference in carrying out their computations, there is a seamless interaction between the two streams in the production of adaptive behaviour. A full understanding of the integrated nature of visually guided behaviour will require that we specify the nature of the interactions and information exchange that occur between these two streams of visual processing.

Online corrections can produce illusory bias during closed-loop pointing

This experiment examined whether the impact of pictorial illusions during the execution of goal-directed reaching movements is attributable to ocular motor signaling. We analyzed eye and hand movements directed toward both the vertex of the Müller-Lyer (ML) figure in a closed-loop procedure. Participants pointed to the right vertex of a visual stimulus in two conditions: a control condition wherein the figure (in-ML, neutral, out-ML) presented at response planning remained unchanged throughout the movement, and an experimental condition wherein a neutral figure presented at response planning was perturbed to an illusory figure (in-ML, out-ML) at movement onset. Consistent with previous work from our group (Heath et al. in Exp Brain Res 158:378-384, 2004; Heath et al. in J Mot Behav 37:179-185, 2005b), action-bias present in both conditions; thus illusory bias was introduced into during online control. Although primary saccades were influenced by illusory configurations (control conditions; see Binsted and Elliott in Hum Mov Sci 18:103-117, 1999a), illusory bias developed within the secondary "corrective" saccades during experimental trials (i.e., following a veridical primary saccade). These results support the position that a unitary spatial representation underlies both action and perception and this representation is common to both the manual and oculomotor systems.

Evidence for Automatic On-Line Adjustments of Hand Orientation During Natural Reaching Movements to Stationary Targets

Control of the spatial orientation of the hand is an important component of reaching and grasping movements. We studied the contribution of vision and proprioception to the perception and control of hand orientation in orientation-matching and letter-posting tasks. In the orientation-matching task, subjects aligned a “match” handle to a “target” handle that was fixed in different orientations. In letter-posting task 1, subjects simultaneously reached and rotated the right hand to insert a match handle into a target slot fixed in the same orientations. Similar sensory conditions produced different error patterns in the two tasks. Furthermore, without vision of the hand, final hand-orientation errors were smaller overall in letter-posting task 1 than in the orientation-matching task. In letter-posting task 2, subjects first aligned their hand to the angle of the target and then reached to it with the instruction not to change their initial hand orientation. Nevertheless, hand orientation changed during reaching in a way that reduced the initial orientation errors. This did not occur when there was no explicitly defined target toward which the subjects reached (letter-posting task 3). The reduction in hand-orientation errors during reach, even when told not to change it, suggests the engagement of an automatic error correction mechanism for hand orientation during reaching movements toward stationary targets. The correction mechanism was engaged when the task involved transitive actions directed at the target object. The on-line adjustments can occur without vision of the hand and even when target orientation is defined only by proprioceptive inputs.

Action Rules: Why the Visual Control of Reaching and Grasping is Not Always Influenced by Perceptual Illusions

It is generally accepted that vision first evolved for the distal control of movement and that perception or ‘representational’ vision emerged much later. Vision-for-action operates in real time and uses egocentric frames of reference and the real metrics of the world. Vision-for-perception can operate over longer time scales and is much more scene-based in its computations. These differences in the timing and metrics of the two systems have been examined in experiments that have looked at the way in which each system deals with visual illusions. Although controversial, the consensus is that actions such as grasping and reaching are often unaffected by high-level pictorial illusions, which by definition affect perception. However, recent experiments have shown that, for actions to escape the effects of such illusions, they must be highly practiced actions, preferably with the right hand, and must be directed in real time at visible targets. This latter finding suggests that some of the critical components of the encapsulated (bottom – up) systems that mediate the visual control of skilled reaching and grasping movements are lateralised to the left hemisphere.

The effect of the "rod-and-frame" illusion on grip planning in a sequential object manipulation task

We investigated the effect of visual context (i.e., a visual illusion) on the planning of a sequential object manipulation task. Participants (n = 13) had to grasp a rod embedded in a “rod-and-frame” illusion and insert the rod-end into a tight hole in a pre-defined way. The grip type (defined by start posture, either pronated or supinated; and end posture, either comfortable or uncomfortable) used to grasp the rod was registered as a macroscopic variable of motor planning. Different rod orientations forced the participants to switch between grip types. As expected, most participants switched between pronated and supinated start postures, such that they ended the movement with a comfortable end posture. As it has been argued that planning is dependent on visual context information, we hypothesized that the visual illusion would affect the specific rod orientation at which participants would switch into a different grip type. This hypothesis was confirmed. More specifically, the illusion affected the critical spatial information that is used for action planning. Collectively, these findings are the first to show an effect of an illusion on motor planning in a sequential object manipulation task.

The spatio-temporal tuning of the mechanisms in the control of saccadic eye movements

We compared the spatio-temporal tuning of perception to the mechanisms that drive saccadic eye movements. Detection thresholds were measured for Gabor-targets presented left or right of fixation (4 or 8deg eccentricity), at one of four spatial frequencies (1, 2, 4 or 8cpd) oscillating at one of three temporal frequencies (1, 8 or 16Hz). We then measured saccade latency to each target presented at various multiples of detection threshold. Consistent with previous research, latency decreased as a function of contrast. However, at equal detection performance, we found no systematic difference in saccadic latency and no difference in average oculometric performance (% correct saccade direction) across the different target spatio-temporal frequencies. Furthermore, position error remained fairly constant across all conditions. The results are consistent with the idea that the spatio-temporal signals used for perception are the same as those used by the mechanisms driving saccadic eye movements.

The effect of the Müller-Lyer illusion on the planning and control of manual aiming movements

Two experiments used Müller-Lyer stimuli to test the predictions of the planning-control model (S. Glover, 2002) for aiming movements. In Experiment 1, participants aimed to stimuli that either remained the same or changed upon movement initiation. Experiment 2 was identical except that the duration of visual feedback for online control was manipulated. The authors found that the figures visible during movement planning and online control had additive effects on endpoint bias, even when participants had ample time to use visual feedback to modify their movements (Experiment 2). These findings are problematic not only for the planning-control model but also for A. D. Milner and M. A. Goodale's (1995) two visual system explanation of illusory bias. Although our results are consistent with the idea that a single representation is used for perception, movement planning, and online control (e.g., V. H. Franz, 2001), other work from our laboratory and elsewhere suggests that the manner in which space is coded depends on constraints associated with the specific task, such as the visual cues available to the performer.

Müller-Lyer figures influence the online reorganization of visually guided grasping movements

In advance of grasping a visual object embedded within fins-in and fins-out Müller-Lyer (ML) configurations, participants formulated a premovement grip aperture (GA) based on the size of a neutral preview object. Preview objects were smaller, veridical, or larger than the size of the to-be-grasped target object. As a result, premovement GA associated with the small and large preview objects required significant online reorganization to appropriately grasp the target object. We reasoned that such a manipulation would provide an opportunity to examine the extent to which the visuomotor system engages egocentric and/or allocentric visual cues for the online, feedback-based control of action. It was found that the online reorganization of GA was reliably influenced by the ML figures (i.e., from 20 to 80% of movement time), regardless of the size of the preview object, albeit the small and large preview objects elicited more robust illusory effects than the veridical preview object. These results counter the view that online grasping control is mediated by absolute visual information computed with respect to the observer (e.g., Glover in Behav Brain Sci 27:3-78, 2004; Milner and Goodale in The visual brain in action 1995). Instead, the impact of the ML figures suggests a level of interaction between egocentric and allocentric visual cues in online action control.

Visual feedback schedules influence visuomotor resistance to the Müller-Lyer figures

We examined whether blocked or random visual feedback schedules influence visuomotor resistance to the Müller-Lyer (ML) illusion. Participants completed closed-loop (CL) and open-loop (OL) grasping movements to an object embedded within fins-in and fins-out ML configurations. In the blocked feedback schedule, CL and OL trials were completed in separate blocks of trials, whereas visual conditions were randomly interleaved in the random feedback schedule. The results of the blocked feedback schedule showed that OL, but not CL, trials were influenced in a direction consistent with the perceptual effects of the ML illusion. For the random feedback schedule, however, both CL and OL trials were influenced by the illusion. We have interpreted these results to reflect the fact that participants evoked distinct control strategies based on the predicted availability of visual feedback. Specifically, the refractory nature of CL trials in the blocked feedback schedule suggests that advance knowledge that visual feedback would be available during a response encouraged an online control strategy wherein metrical visual information supported grasping. When visual feedback was unavailable (i.e., blocked OL trials), or could not be predicted in advance of a response (i.e., random CL and OL trials), it is proposed that movements were structured offline via perception-based visual information that was "tricked" by the cognitive properties of the ML illusion.

Manual pointing to remembered targets...but also in a remembered visual context

In this paper we investigated whether visual background information available during target presentation influences manual pointing to remembered targets. Younger and older participants manually pointed with their unseen hands to remembered or visible targets that were presented or not over a structured visible background. The results indicated that a structured visual background biased movement planning processes, but did not influence motor control processes, regardless of the fact that target location and the background were visible or remembered. How one uses visual background information for movement planning is not modified by aging.

Optic ataxia as a deficit specific to the on-line control of actions

Optic ataxia is characterized by inaccuracies in body movements under visual control, and is a common consequence of damage to the posterior parietal lobes in humans. It is argued here that optic ataxia can be characterized as a deficit in the visual on-line guidance of actions, with action planning remaining relatively intact. This contrasts with the common view of optic ataxia as representing a deficit in the transformations that take place between visual inputs and motor outputs. Evidence in support of the planning-control view comes from the pattern of spared and disrupted behaviors in patients with optic ataxia. It is shown that spared behaviors are those that emphasize planning, whereas disrupted behaviors are those that emphasize control. In particular, recent studies have highlighted the inability of a patient with optic ataxia to make on-line adjustments to targets that change position during the movement. Taken in sum, the data from patients with optic ataxia is more consistent with the planning-control interpretation of optic ataxia than with the visuomotor transformation interpretation.

The Müller-Lyer illusion affects the planning and control of manual aiming movements

Participants made perceptual judgments about the length of, and manual aiming movements to the opposite end of, formerly visible Müller-Lyer stimuli. The Müller-Lyer illusion affected both perceptual judgments and aiming amplitude. Manipulations of stimulus duration (10 ms or 3000 ms) and memory delay length (10 ms or 3000 ms) had no impact on the illusory effect. Aiming movements executed with vision of the hand were less affected by the illusion than movements executed without vision of the hand. The effect of the illusion on aiming amplitude remained the same between peak velocity and the end of the movement even though participants were engaged in on-line control between peak deceleration and the end of the movement. This latter finding was counter to the predictions of a hypothesis (Glover 2002) stating that illusions should only affect the early (planning) stages of movement and not the late (control) stages of movement. We conclude that a single visual representation is used for perception, motor planning, and motor control.

Dynamic effects of the Ebbinghaus illusion in grasping: support for a planning/control model of action

A distinction between planning and control can be used to explain the effects of context-induced illusions on actions. The present study tested the effects of the Ebbinghaus illusion on the planning and control of the grip aperture in grasping a disk. In two experiments, the illusion had an effect on grip aperture that decreased as the hand approached the target, whether or not visual feedback was available. These results are taken as evidence in favor of a planning/control model, in which planning is susceptible to context-induced illusions, whereas control is not. It is argued that many dissociations between perception and action may better be explained as dissociations between perception and on-line control.