The Two Visual Systems Hypothesis: New Challenges and Insights from Visual form Agnosic Patient DF

Patient DF, who developed visual form agnosia following carbon monoxide poisoning, is still able to use vision to adjust the configuration of her grasping hand to the geometry of a goal object. This striking dissociation between perception and action in DF provided a key piece of evidence for the formulation of Goodale and Milner's Two Visual Systems Hypothesis (TVSH). According to the TVSH, the ventral stream plays a critical role in constructing our visual percepts, whereas the dorsal stream mediates the visual control of action, such as visually guided grasping. In this review, we discuss recent studies of DF that provide new insights into the functional organization of the dorsal and ventral streams. We confirm recent evidence that DF has dorsal as well as ventral brain damage - and that her dorsal-stream lesions and surrounding atrophy have increased in size since her first published brain scan. We argue that the damage to DF's dorsal stream explains her deficits in directing actions at targets in the periphery. We then focus on DF's ability to accurately adjust her in-flight hand aperture to changes in the width of goal objects (grip scaling) whose dimensions she cannot explicitly report. An examination of several studies of DF's grip scaling under natural conditions reveals a modest though significant deficit. Importantly, however, she continues to show a robust dissociation between form vision for perception and form vision-for-action. We also review recent studies that explore the role of online visual feedback and terminal haptic feedback in the programming and control of her grasping. These studies make it clear that DF is no more reliant on visual or haptic feedback than are neurologically intact individuals. In short, we argue that her ability to grasp objects depends on visual feedforward processing carried out by visuomotor networks in her dorsal stream that function in the much the same way as they do in neurologically intact individuals.

Enhanced auditory spatial localization in blind echolocators

Echolocation is the extraordinary ability to represent the external environment by using reflected sound waves from self-generated auditory pulses. Blind human expert echolocators show extremely precise spatial acuity and high accuracy in determining the shape and motion of objects by using echoes. In the current study, we investigated whether or not the use of echolocation would improve the representation of auditory space, which is severely compromised in congenitally blind individuals (Gori et al., 2014). The performance of three blind expert echolocators was compared to that of 6 blind non-echolocators and 11 sighted participants. Two tasks were performed: (1) a space bisection task in which participants judged whether the second of a sequence of three sounds was closer in space to the first or the third sound and (2) a minimum audible angle task in which participants reported which of two sounds presented successively was located more to the right. The blind non-echolocating group showed a severe impairment only in the space bisection task compared to the sighted group. Remarkably, the three blind expert echolocators performed both spatial tasks with similar or even better precision and accuracy than the sighted group. These results suggest that echolocation may improve the general sense of auditory space, most likely through a process of sensory calibration.

Parahippocampal cortex is involved in material processing via echoes in blind echolocation experts

Some blind humans use sound to navigate by emitting mouth-clicks and listening to the echoes that reflect from silent objects and surfaces in their surroundings. These echoes contain information about the size, shape, location, and material properties of objects. Here we present results from an fMRI experiment that investigated the neural activity underlying the processing of materials through echolocation. Three blind echolocation experts (as well as three blind and three sighted non-echolocating control participants) took part in the experiment. First, we made binaural sound recordings in the ears of each echolocator while he produced clicks in the presence of one of three different materials (fleece, synthetic foliage, or whiteboard), or while he made clicks in an empty room. During fMRI scanning these recordings were played back to participants. Remarkably, all participants were able to identify each of the three materials reliably, as well as the empty room. Furthermore, a whole brain analysis, in which we isolated the processing of just the reflected echoes, revealed a material-related increase in BOLD activation in a region of left parahippocampal cortex in the echolocating participants, but not in the blind or sighted control participants. Our results, in combination with previous findings about brain areas involved in material processing, are consistent with the idea that material processing by means of echolocation relies on a multi-modal material processing area in parahippocampal cortex.

Representation of object weight in human ventral visual cortex

Skilled manipulation requires the ability to predict the weights of viewed objects based on learned associations linking object weight to object visual appearance. However, the neural mechanisms involved in extracting weight information from viewed object properties are unknown. Given that ventral visual pathway areas represent a wide variety of object features, one intriguing but as yet untested possibility is that these areas also represent object weight, a nonvisual motor-relevant object property. Here, using event-related fMRI and pattern classification techniques, we tested the novel hypothesis that object-sensitive regions in occipitotemporal cortex (OTC), in addition to traditional motor-related brain areas, represent object weight when preparing to lift that object. In two studies, the same participants prepared and then executed lifting actions with objects of varying weight. In the first study, we show that when lifting visually identical objects, where predicted weight is based solely on sensorimotor memory, weight is represented in object-sensitive OTC. In the second study, we show that when object weight is associated with a particular surface texture, that texture-sensitive OTC areas also come to represent object weight. Notably, these texture-sensitive areas failed to carry information about weight in the first study, when object surface properties did not specify weight. Our results indicate that the integration of visual and motor-relevant object information occurs at the level of single OTC areas and provide evidence that the ventral visual pathway is actively and flexibly engaged in processing object weight, an object property critical for action planning and control.

How (and why) the visual control of action differs from visual perception

Vision not only provides us with detailed knowledge of the world beyond our bodies, but it also guides our actions with respect to objects and events in that world. The computations required for vision-for-perception are quite different from those required for vision-for-action. The former uses relational metrics and scene-based frames of reference while the latter uses absolute metrics and effector-based frames of reference. These competing demands on vision have shaped the organization of the visual pathways in the primate brain, particularly within the visual areas of the cerebral cortex. The ventral 'perceptual' stream, projecting from early visual areas to inferior temporal cortex, helps to construct the rich and detailed visual representations of the world that allow us to identify objects and events, attach meaning and significance to them and establish their causal relations. By contrast, the dorsal 'action' stream, projecting from early visual areas to the posterior parietal cortex, plays a critical role in the real-time control of action, transforming information about the location and disposition of goal objects into the coordinate frames of the effectors being used to perform the action. The idea of two visual systems in a single brain might seem initially counterintuitive. Our visual experience of the world is so compelling that it is hard to believe that some other quite independent visual signal-one that we are unaware of-is guiding our movements. But evidence from a broad range of studies from neuropsychology to neuroimaging has shown that the visual signals that give us our experience of objects and events in the world are not the same ones that control our actions.

Temporal order judgments are disrupted more by reflexive than by voluntary saccades

We do not always perceive the sequence of events as they actually unfold. For example, when two events occur before a rapid eye movement (saccade), the interval between them is often perceived as shorter than it really is and the order of those events can be sometimes reversed (Morrone MC, Ross J, Burr DC. Nat Neurosci 8: 950-954, 2005). In the present article we show that these misperceptions of the temporal order of events critically depend on whether the saccade is reflexive or voluntary. In the first experiment, participants judged the temporal order of two visual stimuli that were presented one after the other just before a reflexive or voluntary saccadic eye movement. In the reflexive saccade condition, participants moved their eyes to a target that suddenly appeared. In the voluntary saccade condition, participants moved their eyes to a target that was present already. Similarly to the above-cited study, we found that the temporal order of events was often misjudged just before a reflexive saccade to a suddenly appearing target. However, when people made a voluntary saccade to a target that was already present, there was a significant reduction in the probability of misjudging the temporal order of the same events. In the second experiment, the reduction was seen in a memory-delay task. It is likely that the nature of the motor command and its origin determine how time is perceived during the moments preceding the motor act.

Separate visual systems for perception and action: a framework for understanding cortical visual impairment

The visual control of skilled goal-directed movements requires transformations of incoming visual information that are quite different from those required for visual perception. These differences in the required computations have led to the emergence of dedicated visuomotor modules in the dorsal visual stream of the cerebral cortex that are quite separate from the networks in the ventral visual stream that mediate our conscious perception of the world. Although the identification and selection of goal objects and an appropriate course of action depends on the perceptual machinery of the ventral stream and associated cognitive modules in the temporal and frontal lobes, the execution of the subsequent goal-directed action is mediated by dedicated online control systems in the dorsal stream and associated motor areas. This functional distinction can provide a useful framework for interpreting the range of perceptual and visuomotor deficits observed in children with cortical visual impairment.

Two visual streams: Interconnections do not imply duplication of function

Abstract Schenk and McIntosh (S&M) provide a useful review of the perception-action model (PAM), highlighting some of the gaps that need to be filled, and counteracting the erroneous belief held by some that the PAM implies two mutually independent streams. Although we agree with S&M's contention that the functional independence of the two streams has been overestimated, we reject their speculation that "the specializations proposed may be relative rather than absolute." We argue that the contributions made by the two streams are quite distinct, and that establishing how they work together is the key to a full understanding of visually guided behavior.

Decoding visual object categories in early somatosensory cortex

Neurons, even in the earliest sensory areas of cortex, are subject to a great deal of contextual influence from both within and across modality connections. In the present work, we investigated whether the earliest regions of somatosensory cortex (S1 and S2) would contain content-specific information about visual object categories. We reasoned that this might be possible due to the associations formed through experience that link different sensory aspects of a given object. Participants were presented with visual images of different object categories in 2 fMRI experiments. Multivariate pattern analysis revealed reliable decoding of familiar visual object category in bilateral S1 (i.e., postcentral gyri) and right S2. We further show that this decoding is observed for familiar but not unfamiliar visual objects in S1. In addition, whole-brain searchlight decoding analyses revealed several areas in the parietal lobe that could mediate the observed context effects between vision and somatosensation. These results demonstrate that even the first cortical stages of somatosensory processing carry information about the category of visually presented familiar objects.

A brief review of the role of training in near-tool effects

Research suggests that, like near-hand effects, visual targets appearing near the tip of a hand-held real or virtual tool are treated differently than other targets. This paper reviews neurological and behavioral evidence relevant to near-tool effects and describes how the effect varies with the functional properties of the tool and the knowledge of the participant. In particular, the paper proposes that motor knowledge plays a key role in the appearance of near-tool effects.

The nature and limits of orientation and pattern processing supporting visuomotor control in a visual form agnosic

Abstract We have previously reported that a patient (DF) with visual form agnosia shows accurate guidance of hand and finger movements with respect to the size, orientation, and shape of the objects to which her movements are directed. Despite this, she is unable to indicate any knowledge about these object properties. In the present study, we investigated the extent to which DF is able to use visual shape or pattern to guide her hand movements. In the first experiment, we found that when presented with a stimulus aperture cut in the shape of the letter T, DF was able to guide a T-shaped form into it on about half of the trials, across a range of different stimulus orientations. On the remaining trials, her responses were almost always perpendicular to the correct Orientation. Thus, the visual information guiding the rotation of DF's hand appears to be limited to a single orientation. In other words, the visuomotor transformations mediating her hand rotation appear to be unable to combine the orientations of the stem and the top of the T, although they are sensitive to the orientation of the element(s) that comprise the T. In a second experiment, we examined her ability to use different sources of visual information to guide her hand rotation. In this experiment, DF was required to guide the leading edge of a hand-held card onto a rectangular target positioned at dHerent orientations on a flat surface. Here the orientation of her hand was determined primarily by the predominant orientation of the luminance edge elements present in the stimulus, rather than by information about orientation that was conveyed by nonluminance boundaries. Little evidence was found for an ability to use contour boundaries defined by Gestalt principles of grouping (good continuation or similarity) or "nonaccidental" image properties (colinearity) to guide her movements. We have argued elsewhere that the dorsal visual pathway from occipital to parietal cortex may underlie these preserved visuomotor skills in DF. If so, the limitations in her ability to use different kinds of "pattern" information to guide her hand rotation suggest that such information may need to be transmitted from the ventral visual stream to these parietal areas to enable the full range of prehensive acts in the intact individual.

Acute alcohol consumption impairs controlled but not automatic processes in a psychophysical pointing paradigm

Numerous studies have investigated the effects of alcohol consumption on controlled and automatic cognitive processes. Such studies have shown that alcohol impairs performance on tasks requiring conscious, intentional control, while leaving automatic performance relatively intact. Here, we sought to extend these findings to aspects of visuomotor control by investigating the effects of alcohol in a visuomotor pointing paradigm that allowed us to separate the influence of controlled and automatic processes. Six male participants were assigned to an experimental "correction" condition in which they were instructed to point at a visual target as quickly and accurately as possible. On a small percentage of trials, the target "jumped" to a new location. On these trials, the participants' task was to amend their movement such that they pointed to the new target location. A second group of 6 participants were assigned to a "countermanding" condition, in which they were instructed to terminate their movements upon detection of target "jumps". In both the correction and countermanding conditions, participants served as their own controls, taking part in alcohol and no-alcohol conditions on separate days. Alcohol had no effect on participants' ability to correct movements "in flight", but impaired the ability to withhold such automatic corrections. Our data support the notion that alcohol selectively impairs controlled processes in the visuomotor domain.

Grasping without vision: time normalizing grip aperture profiles yields spurious grip scaling to target size

The analysis of normalized movement trajectories is a popular and informative technique used in investigations of visuomotor control during goal-directed acts like reaching and grasping. This technique typically involves standardizing measures against the amplitude of some other variable - most typically time. Here, we show that this normalizing technique can lead to some surprising results. In the first of two experiments, we asked participants to grasp target objects without ever seeing them from trial to trial. In the second experiment, participants were given a brief preview of the target and were then cued 3s later to pick it up while vision was prevented. Critically, on some trials during the delay period and unbeknownst to the participants, the previewed target was swapped for a new unseen one. The results of both experiments show that time-normalized measures of grip aperture during the closing phase of the movement appear to be scaled to target size well before the fingers make contact with the target - even though participants had no idea what the size of the target was that they were grasping. In contrast, a classical measure of anticipatory grip scaling, maximum grip aperture, did not show scaling to target size. As we demonstrate, however, in both experiments, movement time was longer for the larger target than the smaller ones. Thus, the comparisons of time-normalized grip aperture, particularly during the closing phase of the movements, were made across different points in real time. Taken together, the results of these experiments highlight a need for caution when investigators interpret differences in time-normalized dependent measures - particularly when the effect of interest is correlated with the dependent measure and a third variable (e.g., movement time) that is used to standardize the dependent measure.

Connecting the Dots

The perceptual system parses complex scenes into discrete objects. Parsing is also required for planning visually guided movements when more than one potential target is present. To examine whether visual perception and motor planning use the same or different parsing strategies, we used the connectedness illusion, in which observers typically report seeing fewer targets if pairs of targets are connected by short lines. We found that despite this illusion, when observers are asked to make speeded reaches toward targets in such displays, their reaches are unaffected by the presence of the connecting lines. Instead, their movement plans, as revealed by their movement trajectories, are influenced by the number of potential targets irrespective of whether connecting lines are present or not. This suggests that scene parsing for perception depends on mechanisms that are distinct from those that allow observers to plan rapid and efficient target-directed movements in situations with multiple potential targets.

Size matters: a single representation underlies our perceptions of heaviness in the size-weight illusion

In the size-weight illusion (SWI), a small object feels heavier than an equally-weighted larger object. It is thought that this illusion is a consequence of the way that we internally represent objects’ properties – lifters expect one object to outweigh the other, and the subsequent illusion reflects a contrast with their expectations. Similar internal representations are also thought to guide the application of fingertip forces when we grip and lift objects. To determine the nature of the representations underpinning how we lift objects and perceive their weights, we examined weight judgments in addition to the dynamics and magnitudes of the fingertip forces when individuals lifted small and large exemplars of metal and polystyrene cubes, all of which had been adjusted to have exactly the same mass. Prior to starting the experiment, subjects expected the density of the metal cubes to be higher than that of the polystyrene cubes. Their illusions, however, did not reflect their conscious expectations of heaviness; instead subjects experienced a SWI of the same magnitude regardless of the cubes’ material. Nevertheless, they did report that the polystyrene cubes felt heavier than the metal ones (i.e. they experienced a material-weight illusion). Subjects persisted in lifting the large metal cube with more force than the small metal cube, but lifted the large polystyrene cube with roughly the same amount of force that they used to lift the small polystyrene cube. These findings suggest that our perceptual and sensorimotor representations are not only functionally independent from one another, but that the perceptual system represents a more single, simple size-weight relationship which appears to drive the SWI itself.

Retinotopic organization of the visual cortex before and after decompression of the optic chiasm in a patient with pituitary macroadenoma

Compression induced by a pituitary tumor on the optic chiasm can generate visual field deficits, yet it is unknown how this compression affects the retinotopic organization of the visual cortex. It is also not known how the effect of the tumor on the retinotopic organization of the visual cortex changes after decompression. The authors used functional MRI (fMRI) to map the retinotopic organization of the visual cortex in a 68-year-old right-handed woman before and 3 months after surgery for a recurrent pituitary macroadenoma. The authors demonstrated that longitudinal changes in visual field perimetry, as assessed by the automated Humphrey visual field test, correlated with longitudinal changes in fMRI activation in a retinotopic manner. In other words, after decompression of the optic chiasm, fMRI charted the recruitment of the visual cortex in a way that matched gains in visual field perimetry. On the basis of this case, the authors propose that fMRI can chart neural plasticity of the visual cortex on an individual basis and that it can also serve as a complementary tool in decision making with respect to management of patients with chiasmal compression.

Afterimage size is modulated by size-contrast illusions

Traditionally, the perceived size of negative afterimages has been examined in relation to E. Emmert's law (1881), a size-distance equation that states that changes in perceived size of an afterimage are a function of the distance of the surface on which it is projected. Here, we present evidence that the size of an afterimage is also modulated by its surrounding context. We employed a new version of the Ebbinghaus-Titchener illusion with flickering surrounding stimuli and a static inner target that generated a vivid afterimage of the latter but not the former. Observers were asked to give an initial manual estimate of the size of the inner target during the adaptation phase followed by another manual estimate of the size of the afterimage during the test phase. Manual estimates were affected by the size-contrast illusion both when the surrounding contextual elements were present during afterimage induction and when the surrounding elements were absent during the viewing of the afterimage (Experiment 1). Such a modulation in perceived size, however, did not occur when observers viewed only the flickering surrounding context for a prolonged period of time and then estimated the size of a static target presented on the monitor afterward, demonstrating that flickering stimuli by themselves did not produce any aftereffect on perceived size (Experiment 2). Furthermore, in a final experiment, we showed that the modulation observed in the test phase of Experiment 1 was not due to memory of the manual estimates that had been performed during the adaptation phase (Experiment 3). These findings provide clear evidence for the role of high-level cognitive processes on the perceived size of an afterimage beyond the retinal level. Thus, although retinal stimulation is required to induce an afterimage, post-retinal factors influence its perceived size.

Bringing the real world into the fMRI scanner: repetition effects for pictures versus real objects

Our understanding of the neural underpinnings of perception is largely built upon studies employing 2-dimensional (2D) planar images. Here we used slow event-related functional imaging in humans to examine whether neural populations show a characteristic repetition-related change in haemodynamic response for real-world 3-dimensional (3D) objects, an effect commonly observed using 2D images. As expected, trials involving 2D pictures of objects produced robust repetition effects within classic object-selective cortical regions along the ventral and dorsal visual processing streams. Surprisingly, however, repetition effects were weak, if not absent on trials involving the 3D objects. These results suggest that the neural mechanisms involved in processing real objects may therefore be distinct from those that arise when we encounter a 2D representation of the same items. These preliminary results suggest the need for further research with ecologically valid stimuli in other imaging designs to broaden our understanding of the neural mechanisms underlying human vision.