Pointing to places and spaces in a patient with visual form agnosia

Card posting does not rely on visual orientation: A challenge to past neuropsychological dissociations

A common set of tasks frequently employed in the neuropsychological assessment of patients with visuomotor or perceptual deficits are the card-posting and the perceptual orientation matching tasks. In the posting task, patients have to post a card (or their hand) through a slot of varying orientations while the matching task requires them to indicate the slot's orientation as accurately as possible. Observations that damage to different areas of the brain (dorsal vs. ventral stream) is associated with selective impairment in one of the tasks - but not the other - has led to the suggestion that different cortical pathways process visual orientation information for perception versus action. In three experiments, we show that this conclusion may be premature as posting does not seem to rely on the processing of visual orientation information but is instead performed using obstacle avoidance strategies that require an accurate judgement of egocentric distances between the card's and the slot's edges. Specifically, we found that while matching is susceptible to the oblique effect (i.e., common perceptual orientation bias with higher accuracy for cardinal than oblique orientations), this was not the case for posting, neither in immediate nor in memory-guided conditions. In contrast to matching, posting errors primarily depended on biomechanical demands and reflected a preference for performing efficient and comfortable movements. Thus, we suggest that previous dissociations between perceptual and visuomotor performance in letter posting tasks are better explained by impairments in egocentric and allocentric spatial processing than by independent visual processing systems.

Spatial updating of allocentric landmark information in real-time and memory-guided reaching

The 2-streams model of vision suggests that egocentric and allocentric reference frames are utilized by the dorsal and the ventral stream for real-time and memory-guided movements, respectively. Recent studies argue against such a strict functional distinction and suggest that real-time and memory-guided movements recruit the same spatial maps. In this study we focus on allocentric spatial coding and updating of targets by using landmark information in real-time and memory-guided reaching. We presented participants with a naturalistic scene which consisted of six objects on a table that served as potential reach targets. Participants were informed about the target object after scene encoding, and were prompted by a go cue to reach to its position. After target identification a brief air-puff was applied to the participant's right eye inducing an eye blink. During the blink the target object disappeared from the scene, and in half of the trials the remaining objects, that functioned as landmarks, were shifted horizontally in the same direction. We found that landmark shifts systematically influenced participants' reaching endpoints irrespective of whether the movements were controlled online based on available target information (real-time movement) or memory-guided based on remembered target information (memory-guided movement). Overall, the effect of landmark shift was stronger for memory-guided than real-time reaching. Our findings suggest that humans can encode and update reach targets in an allocentric reference frame for both real-time and memory-guided movements and show stronger allocentric coding when the movement is based on memory.

Independent working memory resources for egocentric and allocentric spatial information

Visuospatial working memory enables us to maintain access to visual information for processing even when a stimulus is no longer present, due to occlusion, our own movements, or transience of the stimulus. Here we show that, when localizing remembered stimuli, the precision of spatial recall does not rely solely on memory for individual stimuli, but additionally depends on the relative distances between stimuli and visual landmarks in the surroundings. Across three separate experiments, we consistently observed a spatially selective improvement in the precision of recall for items located near a persistent landmark. While the results did not require that the landmark be visible throughout the memory delay period, it was essential that it was visible both during encoding and response. We present a simple model that can accurately capture human performance by considering relative (allocentric) spatial information as an independent localization estimate which degrades with distance and is optimally integrated with egocentric spatial information. Critically, allocentric information was encoded without cost to egocentric estimation, demonstrating independent storage of the two sources of information. Finally, when egocentric and allocentric estimates were put in conflict, the model successfully predicted the resulting localization errors. We suggest that the relative distance between stimuli represents an additional, independent spatial cue for memory recall. This cue information is likely to be critical for spatial localization in natural settings which contain an abundance of visual landmarks.

How do the two visual streams interact with each other?

The current consensus divides primate cortical visual processing into two broad networks or "streams" composed of highly interconnected areas (Milner and Goodale 2006, 2008; Goodale 2014). The ventral stream, passing from primary visual cortex (V1) through to inferior parts of the temporal lobe, is considered to mediate the transformation of the contents of the visual signal into the mental furniture that guides memory, recognition and conscious perception. In contrast the dorsal stream, passing from V1 through to various areas in the posterior parietal lobe, is generally considered to mediate the visual guidance of action, primarily in real time. The brain, however, does not work through mutually insulated subsystems, and indeed there are well-documented interconnections between the two streams. Evidence for contributions from ventral stream systems to the dorsal stream comes from human neuropsychological and neuroimaging research, and indicates a crucial role in mediating complex and flexible visuomotor skills. Complementary evidence points to a role for posterior dorsal-stream visual analysis in certain aspects of 3-D perceptual function in the ventral stream. A series of studies of a patient with visual form agnosia has been instrumental in shaping our knowledge of what each stream can achieve in isolation; but it has also helped us to tease apart the relative dependence of parietal visuomotor systems on direct bottom-up visual inputs versus inputs redirected via perceptual systems within the ventral stream.

The Scale Analysis of Number Mapping onto Space: Manual Estimation Study

Previous studies have shown an interference of task-irrelevant numerical information with the spatial parameters of visuomotor behaviour. These findings lend support to the notion that number and space share a common metric with respect to action. Here I argue that the demonstration of the structural similarity between scales for number and space would be a more stringent test for the shared metrics than a mere fact of interference. The present study investigated the scale of number mapping onto space in a manual estimation task. The physical size of target stimuli and the magnitudes of task-irrelevant numbers were parametrically manipulated in the context of the Titchener illusion. The results revealed different scaling schemas for number and space. Whereas estimates in response to changes in stimulus physical size showed a gradual increase, the effect of number was categorical with the largest number (9) showing greater manual estimate than the other numbers (1, 3, and 7). Possible interpretations that are not necessarily incompatible with the hypothesis of shared metrics with respect to action are proposed. However, the present findings suggest that a meticulous scale analysis is required in order to determine the nature of number–space interaction.

The relationship between allocentric and egocentric frames of reference and categorical and coordinate spatial information processing

We report two experiments on the relationship between allocentric/egocentric frames of reference and categorical/coordinate spatial relations. Jager and Postma (2003) suggest two theoretical possibilities about their relationship: categorical judgements are better when combined with an allocentric reference frame and coordinate judgements with an egocentric reference frame (interaction hypothesis); allocentric/egocentric and categorical/coordinate form independent dimensions (independence hypothesis). Participants saw stimuli comprising two vertical bars (targets), one above and the other below a horizontal bar. They had to judge whether the targets appeared on the same side (categorical) or at the same distance (coordinate) with respect either to their body-midline (egocentric) or to the centre of the horizontal bar (allocentric). The results from Experiment 1 showed a facilitation in the allocentric and categorical conditions. In line with the independence hypothesis, no interaction effect emerged. To see whether the results were affected by the visual salience of the stimuli, in Experiment 2 the luminance of the horizontal bar was reduced. As a consequence, a significant interaction effect emerged indicating that categorical judgements were more accurate than coordinate ones, and especially so in the allocentric condition. Furthermore, egocentric judgements were as accurate as allocentric ones with a specific improvement when combined with coordinate spatial relations. The data from Experiment 2 showed that the visual salience of stimuli affected the relationship between allocentric/egocentric and categorical/coordinate dimensions. This suggests that the emergence of a selective interaction between the two dimensions may be modulated by the characteristics of the task.

Interactions between gaze-centered and allocentric representations of reach target location in the presence of spatial updating

Numerous studies have investigated the phenomenon of egocentric spatial updating in gaze-centered coordinates, and some have studied the use of allocentric cues in visually-guided movement, but it is not known how these two mechanisms interact. Here, we tested whether gaze-centered and allocentric information combine at the time of viewing the target, or if the brain waits until the last possible moment. To do this, we took advantage of the well-known fact that pointing and reaching movements show gaze-centered 'retinal magnification' errors (RME) that update across saccades. During gaze fixation, we found that visual landmarks, and hence allocentric information, reduces RME for targets in the left visual hemifield but not in the right. When a saccade was made between viewing and reaching, this landmark-induced reduction in RME only depended on gaze at reach, not at encoding. Based on this finding, we argue that egocentric-allocentric combination occurs after the intervening saccade. This is consistent with previous findings in healthy and brain damaged subjects suggesting that the brain updates early spatial representations during eye movement and combines them at the time of action.

Cue Reliability and a Landmark Stability Heuristic Determine Relative Weighting Between Egocentric and Allocentric Visual Information in Memory-Guided Reach

It is not known how egocentric visual information (location of a target relative to the self) and allocentric visual information (location of a target relative to external landmarks) are integrated to form reach plans. Based on behavioral data from rodents and humans we hypothesized that the degree of stability in visual landmarks would influence the relative weighting. Furthermore, based on numerous cue-combination studies we hypothesized that the reach system would act like a maximum-likelihood estimator (MLE), where the reliability of both cues determines their relative weighting. To predict how these factors might interact we developed an MLE model that weighs egocentric and allocentric information based on their respective reliabilities, and also on an additional stability heuristic. We tested the predictions of this model in 10 human subjects by manipulating landmark stability and reliability (via variable amplitude vibration of the landmarks and variable amplitude gaze shifts) in three reach-to-touch tasks: an egocentric control (reaching without landmarks), an allocentric control (reaching relative to landmarks), and a cue-conflict task (involving a subtle landmark “shift” during the memory interval). Variability from all three experiments was used to derive parameters for the MLE model, which was then used to simulate egocentric–allocentric weighting in the cue-conflict experiment. As predicted by the model, landmark vibration—despite its lack of influence on pointing variability (and thus allocentric reliability) in the control experiment—had a strong influence on egocentric–allocentric weighting. A reduced model without the stability heuristic was unable to reproduce this effect. These results suggest heuristics for extrinsic cue stability are at least as important as reliability for determining cue weighting in memory-guided reaching.

Self-regulation and the hypothesis of experience-based selection: investigating indirect conscious control

The assumption that the contents of our conscious visual experience directly control our fine-tuned, real-time motor activity has been challenged by neurological and psychophysical evidence that suggest the two processes work semi-independently of each other. Clark [Clark, A. (2001). Visual experience and motor action: Are the bonds too tight? The Philosophical Review, 110, 495-519; Clark, A. (2002). Is seeing all it seems? Action, reason and the grand illusion. Journal of Consciousness Studies, 9, 181-202; Clark, A. (2006). Vision as dance? Three challenges for sensori-motor contingency theory. PSYCHE, 12 (1). Available from http://www.psyche.cs.monash.edu.au] argues that such evidence implies a more indirect relationship between conscious visual experience and motor-control where the function of visual consciousness is not to control action but to select what actions are to be controlled. In this paper, I argue that this type of dynamic also exists at the wider level of self-regulation where conscious intent appears to indirectly control the enactment of the intended behaviour. I argue that by drawing parallels between Clark's proposed dynamic and self-regulation, the former is not only bolstered by a previously unrecognised source of support but our understanding of the latter can help to further elucidate Clark's proposed mechanism of indirect conscious control.

Pointing to two imaginary targets at the same time: bimanual allocentric and egocentric localization in visual form agnosic D.F

We have previously shown the visual form agnosic patient D.F. has spared sensorimotor utilization of visual information relative to her poor perceptual processing of the same stimulus attributes. Her visuomotor skills are, however, only normal when egocentric visual coding can be used in the task. In other words, her egocentric sensorimotor processing is intact while her 'allocentric' coding of spatial position is impaired. The current investigation extends these previous observations by comparing D.F.'s performance in bimanual pointing to pairs of stimuli directly (the egocentric task) versus pointing to the homologous positions on an adjacent workspace (pantomimed reaching, the allocentric task). The results showed greatly superior pointing accuracy in direct pointing compared to pantomimed pointing. The mechanisms supporting her limited but remaining sensitivity to spatial relationships during pantomimed pointing remain unknown. These residual skills may reflect partially spared categorical coding and/or internal sensorimotor self-cueing.

Impaired distance perception and size constancy following bilateral occipitoparietal damage

Accurate distance perception depends on the processing and integration of a variety of monocular and binocular cues. Dorsal stream lesions can impair this process, but details of this neurocognitive relationship remain unclear. Here, we tested a patient with bilateral occipitoparietal damage and severely impaired stereopsis. We addressed four related questions: (1) Can distance and size perception survive limitations in perceiving monocular and binocular cues? (2) Are egocentric (self-referential) and allocentric (object-referential) distance judgments similarly impaired? (3) Are distance measurements equally impaired in peripersonal and extrapersonal space? (4) Are size judgments possible when distance processing is impaired? The results demonstrate that the patient's lesions impaired both her distance and size perception, but not uniformly. Her performance when using an egocentric reference frame was more impaired than her performance when using an allocentric reference frame. Likewise, her distance judgments in peripersonal space were more impaired than those in extrapersonal space. The patient showed partial preservation in size processing of novel objects even when familiar size cues were removed.

Shape Processing Area LO and Illusory Contours

Recent functional MRI has demonstrated that illusory contours can activate the primary visual cortex. Our investigation sought to demonstrate whether this correlation reflects computations performed in the primary visual cortex or feedback effects from shape processing area LO. We explored this in a patient who has a bilateral lesion to LO, but a functionally spared V1. Our data indicate that illusory contours are unable to influence behaviour without visual area LO. Whilst we would not claim that our data provide evidence for the ‘cognitive’ nature of illusory contours, they certainly suggest that illusory contours are dependent upon the computations involved in extracting shape representations in LO. Our data highlight the importance of neuropsychological research in interpreting the role of feedforward and feedback effects in the generation of visual illusions.

Immediate and delayed actions share a common visuomotor transformation mechanism: a prism adaptation study

It is a key assumption of the perception/action model that the dorsal stream relies on current visual input and does not store visual information over an extended period of time. Consequently, it is expected that action which is guided by memorized visual information, so-called delayed action, cannot use information from the dorsal stream but must rely instead on the ventral stream input. However, it is currently unclear how the information from the ventral stream can be used to guide an action. This issue is particularly challenging given that the perception/action model also assumes that ventral stream input is not particularly useful for guiding actions since the information it provides is coded relative to the visual scene and not relative to the observer. We describe two possible solutions to this problem and suggest that they can be tested using the prism adaptation paradigm. Subjects in our study were adapted to optical prisms using either an immediate or a delayed pointing task. In both cases, subjects showed the typical post-exposure negative aftereffect. Moreover, there was almost complete transfer of the aftereffect between immediate and delayed pointing. This is particularly surprising given the long history of findings showing little transfer between motor tasks for which separate neural representations are assumed. In this context our findings suggest a substantial overlap in the visuomotor transformation processes used for immediate and delayed pointing.

Object-based attention and visual area LO

We investigated the neural basis of so-called "object-based attention" by examining patient D.F., who has visual form agnosia caused by bilateral damage to the lateral occipital (LO) area of the ventral visual stream. We tested D.F.'s object-based attention in two ways. In the first experiment, we used a spatial cueing procedure to compare the costs associated with shifting attention within versus between two separate outline figures. D.F. did not show the normal advantage of within-object over between-object attention shifts. In the second experiment, we used a complementary paradigm in which two separate stimuli, presented either on the same or on different objects, have to be identified as the same or different, We found no evidence for the normal pattern of superior performance for within versus between figure comparisons. In a third experiment, we checked that D.F. showed normal shift costs for invalid as opposed to valid cueing in a standard Posner spatial attention task. In a final experiment, we compared horizontal versus vertical attention shifting in group of healthy controls without the presence of outline rectangles, and found that their pattern of shift costs was indistinguishable from that seen in D.F. when the rectangles were present (Experiment 1). We conclude that whilst D.F. has a normal spatial orienting system this is completely uninfluenced by object structure. We suggest that area LO may mediate form processing precisely at the stage where visual representations normally influence the spread of attention.

Dissociation of egocentric and allocentric coding of space in visual search after right middle cerebral artery stroke

Spatial representations rely on different frames of reference. Patients with unilateral neglect may behave as suffering from either egocentric or allocentric deficiency. The neural substrates representing these reference frames are still under discussion. Here we used a visual search paradigm to distinguish between egocentric and allocentric deficits in patients with right hemisphere cortical lesions. An attention demanding search task served to divide patients according to egocentric versus allocentric deficits. The results indicate that egocentric impairment was associated with damage in premotor cortex involving the frontal eye fields. Allocentric impairment on the other hand was linked to lesions in more ventral regions near the parahippocampal gyrus (PHG).

Somatosensory processing subserving perception and action: Dissociations, interactions, and integration

The commentaries have raised important points regarding different aspects of our model. Some have queried the nature of the proposed dissociations, whereas others have requested and provided further details regarding aspects we had glossed over. Here we suggest that our approach to identify major processing streams based on the processing goal does not preclude interactions between them. We further specify details regarding body representations, haptic object recognition, and crossmodal processing, but are also aware that several features of the model require further filling in.

Visuomotor system uses target features unavailable to conscious awareness

After lesions to primary visual cortex, patients lack conscious awareness of visual stimuli. Interestingly, however, some retain the ability to make accurate judgments about the visual world (i.e., so-called blindsight). Similarly, damage to inferior occipitotemporal regions of cortex (e.g., lateral occipital cortex) can result in an inability to perceive object properties while retaining the ability to act on them (i.e., visual form agnosia). In the present work, we demonstrate that the ability to interact with objects in the absence of conscious awareness is not isolated to those with restricted neuropathologic conditions. Specifically, neurologically intact individuals are able to program and execute goal-directed reaching movements to a target object without awareness of extrinsic target properties; they accurately tune the dynamics of their movement and modulate it online without conscious access to features of the goal object. Thus, the planning and execution of actions are not dependent on conscious awareness of the environment, suggesting that the phenomenon of blindsight (and agnosia) reflect normal conditions of the visual system.