Relative mislocalization of briefly presented stimuli in the retinal periphery

Severe distortion in the representation of foveal visual image locations in short-term memory

The foveal visual image region provides the human visual system with the highest acuity. However, it is unclear whether such a high fidelity representational advantage is maintained when foveal image locations are committed to short-term memory. Here, we describe a paradoxically large distortion in foveal target location recall by humans. We briefly presented small, but high contrast, points of light at eccentricities ranging from 0.1 to 12°, while subjects maintained their line of sight on a stable target. After a brief memory period, the subjects indicated the remembered target locations via computer controlled cursors. The biggest localization errors, in terms of both directional deviations and amplitude percentage overshoots or undershoots, occurred for the most foveal targets, and such distortions were still present, albeit with qualitatively different patterns, when subjects shifted their gaze to indicate the remembered target locations. Foveal visual images are severely distorted in short-term memory.

Sensorimotor serial dependencies in head movements

On average, we redirect our gaze with a frequency at about 3 Hz. In real life, gaze shifts consist of eye and head movements. Much research has focused on how the accuracy of eye movements is monitored and calibrated. By contrast, little is known about how head movements remain accurate. I wondered whether serial dependencies between artificially induced errors in head movement targeting and the immediately following head movement might recalibrate movement accuracy. I also asked whether head movement targeting errors would influence visual localization. To this end, participants wore a head mounted display and performed head movements to targets, which were displaced as soon as the start of the head movement was detected. I found that target displacements influenced head movement amplitudes in the same trial, indicating that participants could adjust their movement online to reach the new target location. However, I also found serial dependencies between the target displacement in trial n-1 and head movements amplitudes in the following trial n. I did not find serial dependencies between target displacements and visuomotor localization. The results reveal that serial dependencies recalibrate head movement accuracy.

Visuomotor learning from postdictive motor error

Sensorimotor learning adapts motor output to maintain movement accuracy. For saccadic eye movements, learning also alters space perception, suggesting a dissociation between the performed saccade and its internal representation derived from corollary discharge (CD). This is critical since learning is commonly believed to be driven by CD-based visual prediction error. We estimate the internal saccade representation through pre- and trans-saccadic target localization, showing that it decouples from the actual saccade during learning. We present a model that explains motor and perceptual changes by collective plasticity of spatial target percept, motor command, and a forward dynamics model that transforms CD from motor into visuospatial coordinates. We show that learning does not follow visual prediction error but instead a postdictive update of space after saccade landing. We conclude that trans-saccadic space perception guides motor learning via CD-based postdiction of motor error under the assumption of a stable world.

The Motor Representation of Sensory Experience

How do we estimate the position of an object in the world around us? Naturally, we would direct our gaze to that object. Accordingly, neural motor coordinates entail the distance of external objects and thus might be used to derive perceptual estimates. Several general frameworks in the history of perceptual science have offered such a view.^1-4 However, a mechanism showing how motor and visual processes communicate remains elusive. Here, we report that every post-saccadic error biases visual localization in a serially dependent manner. In order to simulate a realignment of visual space through motor coordinates, we induced an artificial de-alignment between visual and motor space. We found that when performing saccades under this distortion, post-saccadic error information clearly realigned visual and motor space, again in a serially dependent manner. These results demonstrate that the consequences of every saccade directly influence where we see objects in the world. On a neural basis, this requires that motor signals, which generate close to the saccade production machinery, are reported to cortical areas and arrange visual space. This view is consistent with recent electrophysiological findings of post-saccadic error processing in posterior parietal cortex.⁵.

Separate and overlapping functional roles for efference copies in the human thalamus

How the perception of space is generated from the multiple maps in the brain is still an unsolved mystery in neuroscience. A neural pathway ascending from the superior colliculus through the medio-dorsal (MD) nucleus of thalamus to the frontal eye field has been identified in monkeys that conveys efference copy information about the metrics of upcoming eye movements. Information sent through this pathway stabilizes vision across saccades. We investigated whether this motor plan information might also shape spatial perception even when no saccades are performed. We studied patients with medial or lateral thalamic lesions (likely involving either the MD or the ventrolateral (VL) nuclei). Patients performed a double-step task testing motor updating, a trans-saccadic localization task testing visual updating, and a localization task during fixation testing a general role of motor signals for visual space in the absence of eye movements. Single patients with medial or lateral thalamic lesions showed deficits in the double-step task, reflecting insufficient transfer of efference copy. However, only a patient with a medial lesion showed impaired performance in the trans-saccadic localization task, suggesting that different types of efference copies contribute to motor and visual updating. During fixation, the MD patient localized stationary stimuli more accurately than healthy controls, suggesting that patients compensate the deficit in visual prediction of saccades - induced by the thalamic lesion - by relying on stationary visual references. We conclude that partially separable efference copy signals contribute to motor and visual stability in company of purely visual signals that are equally effective in supporting trans-saccadic perception.

Nonretinocentric localization of successively presented flashes during smooth pursuit eye movements

Keeping track of objects in our environment across body and eye movements is essential for perceptual stability and localization of external objects. As of yet, it is largely unknown how this perceptual stability is achieved. A common behavioral approach to investigate potential neuronal mechanisms underlying spatial vision has been the presentation of one brief visual stimulus across eye movements. Here, we adopted this approach and aimed to determine the reference frame of the perceptual localization of two successively presented flashes during fixation and smooth pursuit eye movements (SPEMs). To this end, eccentric flashes with a stimulus onset asynchrony of zero or ± 200 ms had to be localized with respect to each other during fixation and SPEMs. The results were used to evaluate different models predicting the reference frame in which the spatial information is represented. First, we were able to reproduce the well-known effect of relative mislocalization during fixation. Second, smooth pursuit led to a characteristic relative mislocalization, different from that during fixation. A model assuming that relative localization takes place in a nonretinocentric reference frame described our data best. This suggests that the relative localization judgment is performed at a stage of visual processing in which retinal and nonretinal information is available.

Fixation related shifts of perceptual localization counter to saccade direction

Perisaccadic compression of the perceived location of flashed visual stimuli toward a saccade target occurs from about 50 ms before a saccade. Here we show that between 150 and 80 ms before a saccade, perceived locations are shifted toward the fixation point. To establish the cause of the "reverse" presaccadic perceptual distortion, participants completed several versions of a saccade task. After a cue to saccade, a probe bar stimulus was briefly presented within the saccade trajectory. In Experiment 1 participants made (a) overlap saccades with immediate return saccades, (b) overlap saccades, and (c) step saccades. In Experiment 2 participants made gap saccades in complete darkness. In Experiment 3 participants maintained fixation with the probe stimuli masked at various interstimulus intervals. Participants indicated the bar's location using a mouse cursor. In all conditions in Experiment 1 presaccadic compression was preceded by compression toward the initial fixation. In Experiment 2, saccadic compression was maintained but the preceding countercompression was not observed. Stimuli masked at fixation were not compressed. This suggests the two opposing compression effects are related to the act of executing an eye movement. They are also not caused by the requirement to make two sequential saccades ending at the initial fixation location and are not caused by continuous presence of the fixation markers. We propose that countercompression is related to fixation activity and is part of the sequence of motor preparations to execute a cued saccade.

An Effect of Context on Whether Memory for Initial Position Exhibits a FröHlich Effect or an Onset Repulsion Effect

Memory for the initial and final positions of moving targets was examined. When targets appeared adjacent to the boundary of a larger enclosing window, memory for initial position exhibited a Fröhlich effect (i.e., a displacement forward), and when distance of initial position from the boundary increased, memory for initial position exhibited a smaller Fröhlich effect or an onset repulsion effect (i.e., a displacement backward). When targets vanished adjacent to the boundary of a larger enclosing window, memory for final position was displaced backward, and when distance of final position from the boundary increased, memory for final position did not exhibit significant displacement. These patterns differed from previously reported displacements of initial and final positions of targets presented on a blank background. Possible influences of attention and extrapolation of trajectory on whether memory for initial position exhibits a Fröhlich effect or an onset repulsion effect and on backward displacement in memory for final position are discussed.

Heading representations in primates are compressed by saccades

Perceptual illusions help to understand how sensory signals are decoded in the brain. Here we report that the opposite approach is also applicable, i.e., results from decoding neural activity from monkey extrastriate visual cortex correctly predict a hitherto unknown perceptual illusion in humans. We record neural activity from monkey medial superior temporal (MST) and ventral intraparietal (VIP) area during presentation of self-motion stimuli and concurrent reflexive eye movements. A heading-decoder performs veridically during slow eye movements. During fast eye movements (saccades), however, the decoder erroneously reports compression of heading toward straight ahead. Functional equivalents of macaque areas MST and VIP have been identified in humans, implying a perceptual correlate (illusion) of this perisaccadic decoding error. Indeed, a behavioral experiment in humans shows that perceived heading is perisaccadically compressed toward the direction of gaze. Response properties of primate areas MST and VIP are consistent with being the substrate of the newly described visual illusion.Macaque higher visual areas MST and VIP encode heading direction based on self-motion stimuli. Here the authors show that, while making saccades, the heading direction decoded from the neural responses is compressed toward straight-ahead, and independently demonstrate a perceptual illusion in humans based on this perisaccadic decoding error.

Auditory Localisation Biases Increase with Sensory Uncertainty

Psychophysical studies have frequently found that adults with normal hearing exhibit systematic errors (biases) in their auditory localisation judgments. Here we tested (i) whether systematic localisation errors could reflect reliance on prior knowledge, as has been proposed for other systematic perceptual biases, and (ii) whether auditory localisation biases can be reduced following training with accurate visual feedback. Twenty-four normal hearing participants were asked to localise the position of a noise burst along the azimuth before, during, and after training with visual feedback. Consistent with reliance on prior knowledge to reduce sensory uncertainty, we found that auditory localisation biases increased when auditory localisation uncertainty increased. Specifically, participants mis-localised auditory stimuli as being more eccentric than they were, and did so more when auditory uncertainty was greater. However, biases also increased with eccentricity, despite no corresponding increase in uncertainty, which is not readily explained by use of a simple prior favouring peripheral locations. Localisation biases decreased (improved) following training with visual feedback, but the reliability of the visual feedback stimulus did not change the effects of training. We suggest that further research is needed to identify alternative mechanisms, besides use of prior knowledge, that could account for increased perceptual biases under sensory uncertainty.

Asymmetric Mislocalization of a Visual Flash Ahead of and behind a Moving Object

When subjects localize a flash relative to another stationary stimulus, the flash appears displaced in the direction of nearby motion signals (position capture; Whitney and Cavanagh, 2000 Nature Neuroscience3 954–959). Our previous study had suggested that the position capture is larger for a flash presented ahead of a moving stimulus than for a flash behind it (Watanabe et al, 2003 Perception32 545–559). In the present study, I investigated the spatial asymmetry of position capture. Experiment 1 demonstrated that asymmetric position capture occurs primarily in a moving-object-centered coordinate. Experiment 2 showed evidence that the asymmetric position capture operates after individuation of single visual objects. Finally, experiment 3 demonstrated that, when attention was reduced with a dual-task procedure, the asymmetric position capture increased. These results suggest that the spatial asymmetry of position capture occurs without attention but the spatial bias can be reduced by attention. Therefore, the underlying mechanism for the asymmetric spatial bias may be different from attentive tracking (Cavanagh, 1992 Science257 1563–1565) and mislocalization during smooth pursuit (Brenner et al, 2001 Vision Research41 2253–2259).

Functional MRI Representational Similarity Analysis Reveals a Dissociation between Discriminative and Relative Location Information in the Human Visual System

Neural responses in visual cortex are governed by a topographic mapping from retinal locations to cortical responses. Moreover, at the voxel population level early visual cortex (EVC) activity enables accurate decoding of stimuli locations. However, in many cases information enabling one to discriminate between locations (i.e., discriminative information) may be less relevant than information regarding the relative location of two objects (i.e., relative information). For example, when planning to grab a cup, determining whether the cup is located at the same retinal location as the hand is hardly relevant, whereas the location of the cup relative to the hand is crucial for performing the action. We have previously used multivariate pattern analysis techniques to measure discriminative location information, and found the highest levels in EVC, in line with other studies. Here we show, using representational similarity analysis, that availability of discriminative information in fMRI activation patterns does not entail availability of relative information. Specifically, we find that relative location information can be reliably extracted from activity patterns in posterior intraparietal sulcus (pIPS), but not from EVC, where we find the spatial representation to be warped. We further show that this variability in relative information levels between regions can be explained by a computational model based on an array of receptive fields. Moreover, when the model's receptive fields are extended to include inhibitory surround regions, the model can account for the spatial warping in EVC. These results demonstrate how size and shape properties of receptive fields in human visual cortex contribute to the transformation of discriminative spatial representations into relative spatial representations along the visual stream.

Saccade Adaptation and Visual Uncertainty

Visual uncertainty may affect saccade adaptation in two complementary ways. First, an ideal adaptor should take into account the reliability of visual information for determining the amount of correction, predicting that increasing visual uncertainty should decrease adaptation rates. We tested this by comparing observers' direction discrimination and adaptation rates in an intra-saccadic-step paradigm. Second, clearly visible target steps may generate a slower adaptation rate since the error can be attributed to an external cause, instead of an internal change in the visuo-motor mapping that needs to be compensated. We tested this prediction by measuring saccade adaptation to different step sizes. Most remarkably, we found little correlation between estimates of visual uncertainty and adaptation rates and no slower adaptation rates with more visible step sizes. Additionally, we show that for low contrast targets backward steps are perceived as stationary after the saccade, but that adaptation rates are independent of contrast. We suggest that the saccadic system uses different position signals for adapting dysmetric saccades and for generating a trans-saccadic stable visual percept, explaining that saccade adaptation is found to be independent of visual uncertainty.

Biases in Visual, Auditory, and Audiovisual Perception of Space

Localization of objects and events in the environment is critical for survival, as many perceptual and motor tasks rely on estimation of spatial location. Therefore, it seems reasonable to assume that spatial localizations should generally be accurate. Curiously, some previous studies have reported biases in visual and auditory localizations, but these studies have used small sample sizes and the results have been mixed. Therefore, it is not clear (1) if the reported biases in localization responses are real (or due to outliers, sampling bias, or other factors), and (2) whether these putative biases reflect a bias in sensory representations of space or a priori expectations (which may be due to the experimental setup, instructions, or distribution of stimuli). Here, to address these questions, a dataset of unprecedented size (obtained from 384 observers) was analyzed to examine presence, direction, and magnitude of sensory biases, and quantitative computational modeling was used to probe the underlying mechanism(s) driving these effects. Data revealed that, on average, observers were biased towards the center when localizing visual stimuli, and biased towards the periphery when localizing auditory stimuli. Moreover, quantitative analysis using a Bayesian Causal Inference framework suggests that while pre-existing spatial biases for central locations exert some influence, biases in the sensory representations of both visual and auditory space are necessary to fully explain the behavioral data. How are these opposing visual and auditory biases reconciled in conditions in which both auditory and visual stimuli are produced by a single event? Potentially, the bias in one modality could dominate, or the biases could interact/cancel out. The data revealed that when integration occurred in these conditions, the visual bias dominated, but the magnitude of this bias was reduced compared to unisensory conditions. Therefore, multisensory integration not only improves the precision of perceptual estimates, but also the accuracy.

The interaction of vision and audition in two-dimensional space

Using a mouse-driven visual pointer, 10 participants made repeated open-loop egocentric localizations of memorized visual, auditory, and combined visual-auditory targets projected randomly across the two-dimensional frontal field (2D). The results are reported in terms of variable error, constant error and local distortion. The results confirmed that auditory and visual maps of the egocentric space differ in their precision (variable error) and accuracy (constant error), both from one another and as a function of eccentricity and direction within a given modality. These differences were used, in turn, to make predictions about the precision and accuracy within which spatially and temporally congruent bimodal visual-auditory targets are localized. Overall, the improvement in precision for bimodal relative to the best unimodal target revealed the presence of optimal integration well-predicted by the Maximum Likelihood Estimation (MLE) model. Conversely, the hypothesis that accuracy in localizing the bimodal visual-auditory targets would represent a compromise between auditory and visual performance in favor of the most precise modality was rejected. Instead, the bimodal accuracy was found to be equivalent to or to exceed that of the best unimodal condition. Finally, we described how the different types of errors could be used to identify properties of the internal representations and coordinate transformations within the central nervous system (CNS). The results provide some insight into the structure of the underlying sensorimotor processes employed by the brain and confirm the usefulness of capitalizing on naturally occurring differences between vision and audition to better understand their interaction and their contribution to multimodal perception.

ARM MOVEMENT AS A CUE FOR THE ESTIMATION OF VISUAL LOCATION

Two experiments including 24 (M age=29 yr., SD=9; 6 men) and 25 participants (M age=27 yr., SD=9; 8 men), respectively, examined how arm movement extent affects the perception of visual locations. Linear arm movements were performed on a horizontal plane from a start position until an auditory signal occurred. Subsequently, the position of a visual target located along the movement path was judged. The target was judged as further away with an increase in movement extent. The results indicated that motor-related signals are taken into account in visual perception of locations. There were no indications, though, that changes of location perception prompted subsequent changes of action planning, which demonstrates the short-term nature of action-induced plasticity of space perception under the present conditions.

Spatial distortions in localization and midline estimation in hemianopia and normal vision

Studies have shown that individuals with hemianopia tend to bisect a line toward their blind, contralesional visual field, termed the hemianopic line bisection error (HLBE). One theory proposes that the HLBE is a perceptual distortion resulting from expansion of the central region of visual space. If true, perceptual expansions of the central regions in the intact hemifield should also be present and observable across different tasks. We tested this hypothesis using a peripheral localization task to assess localization and midpoint estimation along the horizontal axis of the visual field. In this task, participants judged the location of a target dot presented inside a Goldmann perimeter relative to their perceived visual field boundary. In Experiment 1, we tested neurologically healthy participants on the peripheral localization task as well as a novel midpoint assessment task in which participants reported their perceived midpoint along the horizontal axis of their left and right visual fields. The results revealed consistency in individual biases across the two tasks. We then used the peripheral localization task to test whether two patients with hemianopia showed a selective expansion of central visual space. For these patients, three axes were tested: the spared temporal horizontal axis and the upper and lower vertical axes. The results support the notion that the HLBE is due to expansion of perceived space along the spared temporal axis. Together, the results of both experiments validate the use of these novel paradigms for exploring perceptual asymmetries in both healthy individuals and patients with visual field loss.

Impairment of saccade adaptation in a patient with a focal thalamic lesion

The frequent jumps of the eyeballs-called saccades-imply the need for a constant correction of motor errors. If systematic errors are detected in saccade landing, the saccade amplitude adapts to compensate for the error. In the laboratory, saccade adaptation can be studied by displacing the saccade target. Functional selectivity of adaptation for different saccade types suggests that adaptation occurs at multiple sites in the oculomotor system. Saccade motor learning might be the result of a comparison between a prediction of the saccade landing position and its actual postsaccadic location. To investigate whether a thalamic feedback pathway might carry such a prediction signal, we studied a patient with a lesion in the posterior ventrolateral thalamic nucleus. Saccade adaptation was tested for reactive saccades, which are performed to suddenly appearing targets, and for scanning saccades, which are performed to stationary targets. For reactive saccades, we found a clear impairment in adaptation retention ipsilateral to the lesioned side and a larger-than-normal adaptation on the contralesional side. For scanning saccades, adaptation was intact on both sides and not different from the control group. Our results provide the first lesion evidence that adaptation of reactive and scanning saccades relies on distinct feedback pathways from cerebellum to cortex. They further demonstrate that saccade adaptation in humans is not restricted to the cerebellum but also involves cortical areas. The paradoxically strong adaptation for outward target steps can be explained by stronger reliance on visual targeting errors when prediction error signaling is impaired.

The spatial profile of mask-induced compression for perception and action

Stimuli briefly flashed just before a saccade are perceived closer to the saccade target, a phenomenon known as saccadic compression of space. We have recently demonstrated that similar mislocalizations of flashed stimuli can be observed in the absence of saccades: brief probes were attracted towards a visual reference when followed by a mask. To examine the spatial profile of this new phenomenon of masked-induced compression, here we used a pair of references that draw the probe into the gap between them. Strong compression was found when we masked the probe and presented it following a reference pair, whereas little or no compression occurred for the probe without the reference pair or without the mask. When the two references were arranged vertically, horizontal mislocalizations prevailed. That is, probes presented to the left or right of the vertically arranged references were "drawn in" to be seen aligned with the references. In contrast, when we arranged the two references horizontally, we found vertical compression for stimuli presented above or below the references. Finally, when participants were to indicate the perceived probe location by making an eye movement towards it, saccade landing positions were compressed in a similar fashion as perceptual judgments, confirming the robustness of mask-induced compression. Our findings challenge pure oculomotor accounts of saccadic compression of space that assume a vital role for saccade-specific signals such as corollary discharge or the updating of eye position. Instead, we suggest that saccade- and mask-induced compression both reflect how the visual system deals with disruptions.

Impact of action planning on spatial perception: attention matters

Previous research suggested that perception of spatial location is biased towards spatial goals of planned hand movements. In the present study I show that an analogous perceptual distortion can be observed if attention is paid to a spatial location in the absence of planning a hand movement. Participants judged the position of a target during preparation of a mouse movement, the end point of which could deviate from the target by a varying degree in Exp. 1. Judgments of target position were systematically affected by movement characteristics consistent with perceptual assimilation between the target and the planned movement goal. This effect was neither due to an impact of motor execution on judgments (Exp. 2) nor due to characteristics of the movement cues or of certain target positions (Exp. 3, Exp. 5A). When the task included deployment of attention to spatial positions (former movement goals) in preparation for a secondary perceptual task, an effect emerged that was comparable with the bias associated with movement planning (Exp. 4, Exp. 5B). These results indicate that visual distortions accompanying manipulations of variables related to action could be mediated by attentional mechanisms.

Perceived displacement explains wolfpack effect

We investigate the influence of perceived displacement of moving agent-like stimuli on the performance in dynamic interactive tasks. In order to reliably measure perceived displacement we utilize multiple tasks with different task demands. The perceived center of an agent's body is displaced in the direction in which the agent is facing and this perceived displacement is larger than the theoretical position of the center of mass would predict. Furthermore, the displacement in the explicit judgment is dissociated from the displacement obtained by the implicit measures. By manipulating the location of the pivot point, we show that it is not necessary to postulate orientation as an additional cue utilized by perception, as has been suggested by earlier studies. These studies showed that the agent's orientation influences the detection of chasing motion and the detection-related performance in interactive tasks. This influence has been labeled wolfpack effect. In one of the demonstrations of the wolfpack effect participants control a green circle on a display with a computer mouse. It has been shown that participants avoid display areas with agents pointing toward the green circle. Participants do so in favor of areas where the agents point in the direction perpendicular to the circle. We show that this avoidance behavior arises because the agent's pivot point selected by the earlier studies is different from where people locate the center of agent's body. As a consequence, the nominal rotation confounds rotation and translation. We show that the avoidance behavior disappears once the pivot point is set to the center of agent's body.

Transcranial direct current stimulation over posterior parietal cortex modulates visuospatial localization

Visual localization is based on the complex interplay of bottom-up and top-down processing. Based on previous work, the posterior parietal cortex (PPC) is assumed to play an essential role in this interplay. In this study, we investigated the causal role of the PPC in visual localization. Specifically, our goal was to determine whether modulation of the PPC via transcranial direct current stimulation (tDCS) could induce visual mislocalization similar to that induced by an exogenous attentional cue (Wright, Morris, & Krekelberg, 2011). We placed one stimulation electrode over the right PPC and the other over the left PPC (dual tDCS) and varied the polarity of the stimulation. We found that this manipulation altered visual localization; this supports the causal involvement of the PPC in visual localization. Notably, mislocalization was more rightward when the cathode was placed over the right PPC than when the anode was placed over the right PPC. This mislocalization was found within a few minutes of stimulation onset, it dissipated during stimulation, but then resurfaced after stimulation offset and lasted for another 10-15 min. On the assumption that excitability is reduced beneath the cathode and increased beneath the anode, these findings support the view that each hemisphere biases processing to the contralateral hemifield and that the balance of activation between the hemispheres contributes to position perception (Kinsbourne, 1977; Szczepanski, Konen, & Kastner, 2010).

Can representational trajectory reveal the nature of an internal model of gravity?

The memory for the vanishing location of a horizontally moving target is usually displaced forward in the direction of motion (representational momentum) and downward in the direction of gravity (representational gravity). Moreover, this downward displacement has been shown to increase with time (representational trajectory). However, the degree to which different kinematic events change the temporal profile of these displacements remains to be determined. The present article attempts to fill this gap. In the first experiment, we replicate the finding that representational momentum for downward-moving targets is bigger than for upward motions, showing, moreover, that it increases rapidly during the first 300 ms, stabilizing afterward. This temporal profile, but not the increased error for descending targets, is shown to be disrupted when eye movements are not allowed. In the second experiment, we show that the downward drift with time emerges even for static targets. Finally, in the third experiment, we report an increased error for upward-moving targets, as compared with downward movements, when the display is compatible with a downward ego-motion by including vection cues. Thus, the errors in the direction of gravity are compatible with the perceived event and do not merely reflect a retinotopic bias. Overall, these results provide further evidence for an internal model of gravity in the visual representational system.

Displacement of location in illusory line motion

Six experiments examined displacement in memory for the location of the line in illusory line motion (ILM; appearance or disappearance of a stationary cue is followed by appearance of a stationary line that is presented all at once, but the stationary line is perceived to "unfold" or "be drawn" from the end closest to the cue to the end most distant from the cue). If ILM was induced by having a single cue appear, then memory for the location of the line was displaced toward the cue, and displacement was larger if the line was closer to the cue. If ILM was induced by having one of two previously visible cues vanish, then memory for the location of the line was displaced away from the cue that vanished. In general, the magnitude of displacement increased and then decreased as retention interval increased from 50 to 250 ms and from 250 to 450 ms, respectively. Displacement of the line (a) is consistent with a combination of a spatial averaging of the locations of the cue and the line with a relatively weaker dynamic in the direction of illusory motion, (b) might be implemented in a spreading activation network similar to networks previously suggested to implement displacement resulting from implied or apparent motion, and (c) provides constraints and challenges for theories of ILM.

Exploring the edges of visual space: the influence of visual boundaries on peripheral localization

Previous studies of localization of stationary targets in the peripheral visual field have found either underestimations (foveal biases) or overestimations (peripheral biases) of target eccentricity. In the present study, we help resolve this inconsistency by demonstrating the influence of visual boundaries on the type of localization bias. Using a Goldmann perimeter (an illuminated half-dome), we presented targets at different eccentricities across the visual field and asked participants to judge the target locations. In Experiments 1 and 2, participants reported target locations relative to their perceived visual field extent using either a manual or verbal response, with both response types producing a peripheral bias. This peripheral localization bias was a non-linear scaling of perceived location when the visual field was not bounded by external borders induced by facial features (i.e., the nose and brow), but location scaling was linear when visual boundaries were present. Experiment 3 added an external border (an aperture edge placed in the Goldmann perimeter) that resulted in a foveal bias and linear scaling. Our results show that boundaries that define a spatial region within the visual field determine both the direction of bias in localization errors for stationary objects and the scaling function of perceived location across visual space.

Weighted integration of visual position information

The ability to localize visual objects is a fundamental component of human behavior and requires the integration of position information from object components. The retinal eccentricity of a stimulus and the locus of spatial attention can affect object localization, but it is unclear whether these factors alter the global localization of the object, the localization of object components, or both. We used psychophysical methods in humans to quantify behavioral responses in a centroid estimation task. Subjects located the centroid of briefly presented random dot patterns (RDPs). A peripheral cue was used to bias attention toward one side of the display. We found that although subjects were able to localize centroid positions reliably, they typically had a bias toward the fovea and a shift toward the locus of attention. We compared quantitative models that explain these effects either as biased global localization of the RDPs or as anisotropic integration of weighted dot component positions. A model that allowed retinal eccentricity and spatial attention to alter the weights assigned to individual dot positions best explained subjects' performance. These results show that global position perception depends on both the retinal eccentricity of stimulus components and their positions relative to the current locus of attention.

When here becomes there: attentional distribution modulates foveal bias in peripheral localization

Much research concerning attention has focused on changes in the perceptual qualities of objects while attentional states were varied. Here, we address a complementary question—namely, how perceived location can be altered by the distribution of sustained attention over the visual field. We also present a new way to assess the effects of distributing spatial attention across the visual field. We measured magnitude judgments relative to an aperture edge to test perceived location across a large range of eccentricities (30°), and manipulated spatial uncertainty in target locations to examine perceived location under three different distributions of spatial attention. Across three experiments, the results showed that changing the distribution of sustained attention significantly alters known foveal biases in peripheral localization.

Attention can retrospectively distort visual space

A brief visual cue that attracts attention repels the perceived location of a subsequent visual stimulus away from the focus of attention (attentional repulsion). In the first experiment reported here, we presented a visual cue after a visual target and found that the perceived location of the target stimulus shifted toward the location of the cue (attentional attraction). The subsequent experiments ruled out nonattentional hypotheses and indicated that the mislocalization effect is attributable to the attentional shift. The results of this study suggest that preceding and succeeding contexts differentially modulate the perceived location of a briefly presented stimulus. Our findings also underscore the importance of retrospective processes in visual attention.

Perceptual judgment and saccadic behavior in a spatial distortion with briefly presented stimuli

When observers are asked to localize the peripheral position of a small probe with respect to the mid-position of a spatially extended comparison stimulus, they tend to judge the probe as being more peripheral than the mid-position of the comparison stimulus. This relative mislocalization seems to emerge from differences in absolute localization, that is the comparison stimulus is localized more towards the fovea than the probe. The present study compared saccadic behaviour and relative localization judgements in three experiments and determined the quantitative relationship between both measures. The results showed corresponding effects in localization errors and saccadic behaviour. Moreover, it was possible to estimate the amount of the relative mislocalization by means of the saccadic amplitude.

Temporal information can influence spatial localization

To localize objects relative to ourselves, we need to combine various sensory and motor signals. When these signals change abruptly, as information about eye orientation does during saccades, small differences in latency between the signals could introduce localization errors. We examine whether independent temporal information can influence such errors. We asked participants to follow a randomly jumping dot with their eyes and to point at flashes that occurred near the time they made saccades. Such flashes are mislocalized. We presented a tone at different times relative to the flash. We found that the flash was mislocalized as if it had occurred closer in time to the tone. This demonstrates that temporal information is taken into consideration when combining sensory information streams for localization.

The onset-repulsion effect and motion-induced mislocalization of a stationary object

The influence of a moving target on memory for the location of a briefly presented stationary object aligned with the initial location of that moving target was examined. Memory for the location of the stationary object was displaced backward (ie in the direction opposite to target motion), and memory for the initial location of the moving target was also displaced backward (consistent with an onset-repulsion effect); displacement of the stationary object did not differ from displacement of the moving target. Displacement in memory for the initial location of a moving target was not influenced by whether or not a stationary object aligned with that initial location was also presented. The results demonstrate that motion-induced mislocalization can occur in a direction other than the direction of motion, and are consistent with the hypothesis that dynamics of a moving target can influence memory for a nearby stationary object.

Effects of saccadic adaptation on visual localization before and during saccades

Short-term saccadic adaptation is a mechanism that adjusts saccade amplitude to accurately reach an intended saccade target. Short-term saccadic adaptation induces a shift of perceived localization of objects flashed before the saccade. This shift, being detectable only before an adapted saccade, disappears at some time around saccade onset. Up to now, the exact time course of this effect has remained unknown. In previous experiments, the mislocalization caused by this adaptation-induced shift was overlapping with the mislocalization caused by a different, saccade-related localization error, the peri-saccadic compression. Due to peri-saccadic compression, objects flashed immediately at saccade onset appear compressed towards the saccade target. First, we tested whether the adaptation-induced shift and the peri-saccadic compression were either independent or related processes. We performed experiments with two different luminance-contrast conditions to separate the adaptation-induced shift and the peri-saccadic compression. Human participants had to indicate the perceived location of briefly presented stimuli before, during or after an adapted saccade. Adaptation-induced shift occurred similarly in either contrast condition, with or without peri-saccadic compression. Second, after validating the premise of both processes being independent and superimposing, we aimed at characterizing the time course of the adaptation-induced shift in more detail. Being present up to 1 s before an adapted saccade, the adaptation-induced shift begins to gradually decline from about 150 ms before saccade onset, and ceases during the saccade. A final experiment revealed that visual references make a major contribution to adaptation-induced mislocalization.

Relative mislocalization of successively presented stimuli

When observers were asked to localize the peripheral position of a briefly presented target with respect to a previously presented comparison stimulus, they tended to judge the target as being more towards the fovea than the comparison stimulus. Three experiments revealed that the mislocalization only emerged when the comparison stimulus and the target were presented successively. Varying the temporal interval between stimuli showed that the mislocalization reversed with longer stimulus-onset asynchronies. Further, the mislocalization was increased with a decrease of the spatial distance between stimuli. These findings suggested that the mislocalization originated from local excitatory and inhibitory mechanisms. Corroborating this idea a neuronal dynamic field model was successfully developed to account for the findings.

Expansion of visual space during optokinetic afternystagmus (OKAN)

The mechanisms underlying visual perceptual stability are usually investigated using voluntary eye movements. In such studies, errors in perceptual stability during saccades and pursuit are commonly interpreted as mismatches between actual eye position and eye-position signals in the brain. The generality of this interpretation could in principle be tested by investigating spatial localization during reflexive eye movements whose kinematics are very similar to those of voluntary eye movements. Accordingly, in this study, we determined mislocalization of flashed visual targets during optokinetic afternystagmus (OKAN). These eye movements are quite unique in that they occur in complete darkness and are generated by subcortical control mechanisms. We found that during horizontal OKAN slow phases, subjects mislocalize targets away from the fovea in the horizontal direction. This corresponds to a perceived expansion of visual space and is unlike mislocalization found for any other voluntary or reflexive eye movement. Around the OKAN fast phases, we found a bias in the direction of the fast phase prior to its onset and opposite to the fast-phase direction thereafter. Such a biphasic modulation has also been reported in the temporal vicinity of saccades and during optokinetic nystagmus (OKN). A direct comparison, however, showed that the modulation during OKAN was much larger and occurred earlier relative to fast-phase onset than during OKN. A simple mismatch between the current eye position and the eye-position signal in the brain is unlikely to explain such disparate results across similar eye movements. Instead, these data support the view that mislocalization arises from errors in eye-centered position information.

Localizing the onset of moving stimuli by pointing or relative judgment: variations in the size of the Fröhlich effect

In the Fröhlich effect, the perceived onset of a moving stimulus is displaced in the direction of motion. Previously, we observed that pointing movements produced a Fröhlich effect only when the onset position was highly predictable. Here, we show that relative judgments are not affected by spatial predictability if the relative judgment task is performed in isolation. However, when the two tasks vary randomly from trial to trial, effects of spatial predictability carry over to the perceptual task. Thus, observers' intentions before stimulus onset determine the way position signals are processed. An account in terms of sensory and motor maps is discussed.

Evidence for attentional processing in spatial localization

Using a dual-task methodology, this study examined the involvement of selective attention in spatial localization. Thirty participants located a single, briefly presented, peripheral target stimulus, appearing in one of 50 positions on either side of a central fixation point, with or without the requirement to identify a simultaneously presented central distractor stimulus. Results revealed a robust interference effect in localization performance at short target durations that depended on the number of the to-be-identified distractor items. This outcome provides convergent support for the role of the attentional system in spatial localization.

Mislocalization of a target toward subjective contours: attentional modulation of location signals

This study examined whether a briefly presented target was mislocalized toward a subjective contour. Observers manually reproduced the position of a briefly presented peripheral target circle above a central fixation cross. A luminance contour, a subjective contour, or a no-contour stimulus was presented in either the left of right visual field, and a no-contour control was presented in the opposite visual field. After these stimuli vanished, a target circle was then presented. Consequently, the degree of mislocalization toward the subjective and luminance contours was the same; this indicated that image integration at a coarse spatial scale cannot explain mislocalization. Experiment 2 revealed that the mislocalization in Experiment 1 was not a result of eye movements. Experiment 3 found that the spatial attention allocated at the location of the luminance and subjective contours was more than that allocated at the no-contour stimulus. An attentional shift toward the task-irrelevant stimulus resulted in a mislocalization of the target.

Visual memory capacity in transsaccadic integration

How we perceive the visual world as stable and unified suggests the existence of transsaccadic integration that retains and integrates visual information from one eye fixation to another eye fixation across saccadic eye movements. However, the capacity of transsaccadic integration is still a subject of controversy. We tested our subjects' memory capacity of two basic visual features, i.e. luminance (Experiment 1) and orientation (Experiment 2), both within a single fixation (i.e. visual working memory) and between separate fixations (i.e. transsaccadic memory). Experiment 2 was repeated, but attention allocation was manipulated using attentional cues at either the target or distracter (Experiment 3). Subjects were able to retain 3-4 objects in transsaccadic memory for luminance and orientation; errors generally increased as saccade size increased; and, subjects were more accurate when attention was allocated to the same location as the impending target. These results were modelled by inputting a noisy extra-retinal signal into an eye-centered feature map. Our results suggest that transsaccadic memory has a similar capacity for storing simple visual features as basic visual memory, but this capacity is dependent both on the metrics of the saccade and allocation of attention.

Bi-directional illusory position shifts toward the end point of apparent motion

In this study, we examined the relation between motion induced position shifts and the position shifts caused by saccades. When a stimulus is flashed briefly around the time of a saccade, its perceived position is mislocalized toward the saccade target: if the flash is in front of the saccade the test flash appears shifted in the direction of eye movement; but a test flashed beyond the saccade target is displaced back toward the saccade target (bi-directional saccadic compression: Ross, J., Morrone, M. C., and Burr, D. C. (1997). Compression of visual space before saccades. Nature, 386, 598-601. Motion induced position shifts (in the absence of eye movements) have been demonstrated for a variety of stimuli but the illusory position shift is always found to be in the same direction as the motion. However, all previous studies presented the tests either along or beside the motion path, never beyond its end point. We now test this region beyond the motion path and find that the apparent location of a test in this region is shifted in the direction opposite to the motion, back toward the motion end point. In contrast, when the flash was presented between the beginning and end of the motion path, it is shifted in the direction of motion, again, toward the motion end point. These shifts together indicate a compression of perceived locations toward the end point of the apparent motion. Control experiments confirmed that this effect was neither due to the Fröhlich effect induced by apparent motion from the test flash to the second disc nor to foveal compression. The correspondence between compression toward the end point of apparent motion and saccadic compression toward the saccade target suggests that attentional shifts or planned eye movement signals may play a role in both.

Mislocalization of perceived saccade target position induced by perisaccadic visual stimulation

The perceptual localization of objects flashed at the time of a saccade often shows large spatial distortions. These perisaccadic mislocalizations exhibit different spatial patterns depending on the experimental condition. In darkness, when only extraretinal information is available, mislocalization is spatially uniform. In light and when visual references are available, mislocalization is directed toward the saccade target, resembling a compression of visual space. These patterns are derived from measurements of the absolute perceived position of the flashed object in egocentric space. Here, we report that also the perceived location of the saccade target is altered when an object is flashed perisaccadically. The mislocalization of the target depends on the presentation time of the flashed object and is directed toward the position of the flash. The resulting compression of the relative distance between target and flash is similar in darkness and in light and can also be found during fixation. When the localization of the flashed object is described relative to the perceived location of the saccade target, spatial compression becomes similar in many experimental conditions. We therefore suggest that perisaccadic compression relies on an encoding of relative spatial locations of objects rather than on localizations in egocentric space.

The role of uncertainty in the systematic spatial mislocalization of moving objects

It only makes sense to talk about the position of a moving object if one specifies the time at which its position is of interest. The authors here show that when a flash or tone specifies the moment of interest, subjects estimate the moving object to be closer to where it passes the fixation point and further in its direction of motion than it really is. The authors propose that these biases arise from a combination of a large temporal uncertainty, a temporal asymmetry related to sampling the moving object's position, and a bias toward believing that one is looking at what one sees.

Transsaccadic integration of visual features in a line intersection task

Transsaccadic integration (TSI) refers to the perceptual integration of visual information collected across separate gaze fixations. Current theories of TSI disagree on whether it relies solely on visual algorithms or also uses extra-retinal signals. We designed a task in which subjects had to rely on internal oculomotor signals to synthesize remembered stimulus features presented within separate fixations. Using a mouse-controlled pointer, subjects estimated the intersection point of two successively presented bars, in the dark, under two conditions: Saccade task (bars viewed in separate fixations) and Fixation task (bars viewed in one fixation). Small, but systematic biases were observed in both intersection tasks, including position-dependent vertical undershoots and order-dependent horizontal biases. However, the magnitude of these errors was statistically indistinguishable in the Saccade and Fixation tasks. Moreover, part of the errors in the Saccade task were dependent on saccade metrics, showing that egocentric oculomotor signals were used to fuse remembered location and orientation features across saccades. We hypothesize that these extra-retinal signals are normally used to reduce the computational load of calculating visual correspondence between fixations. We further hypothesize that TSI may be implemented within dynamically updated recurrent feedback loops that interconnect a common eye-centered map in occipital cortex with both the "dorsal" and "ventral" streams of visual analysis.

Localization and motion perception during smooth pursuit eye movements

We investigated the relationship between compensation for the effects of smooth pursuit eye movements in localization and motion perception. Participants had to indicate the perceived motion direction, the starting point and the end point of a vertically moving stimulus dot presented during horizontal smooth pursuit. The presentation duration of the stimulus was varied. From the indicated starting and end points, the motion direction was predicted and compared with the actual indicated directions. Both the directions predicted from localization and the indicated directions deviated from the physical directions, but the errors in the predicted directions were larger than those in the indicated directions. The results of a control experiment, in which the same tasks were performed during fixation, suggest that this difference reflects different transformations from a retinocentric to a head-centric frame of reference. This difference appears to be mainly due to an asymmetry in the effect of retinal image motion direction on localization during smooth pursuit.

Flashes are localised as if they were moving with the eyes

Targets that are flashed during smooth pursuit are mislocalised in the direction of the pursuit. It has been suggested that a similar mislocalisation of moving targets could help to overcome processing delays when hitting moving objects. But are moving targets really mislocalised in the way that flashed ones are? To find out we asked people to indicate where targets that were visible for different periods of time had appeared. The targets appeared while the subjects' eyes were moving, and were either moving with the eyes or static. For flashed targets we found the usual systematic mislocalisation. For targets that moved with the eyes the mislocalisation was at least as large, irrespective of the presentation time. For static targets the mislocalisation decreased with increasing presentation time, so that by the time the presentations reached about 200 ms the targets were not mislocalised at all. A simple model that combines smooth retinal motion with information about the velocity of smooth pursuit could account for the measured tapping errors. These findings support the notion that the systematic mislocalisation of flashed targets is related to the way in which people intercept moving objects.

Stopping the motion and sleuthing the flash-lag effect: spatial uncertainty is the key to perceptual mislocalization

A moving object is perceived to lie beyond a static object presented at the same time at the same retinal location (flash-lag effect or FLE). Some studies report that if the moving stimulus stops moving (flash-terminated condition or FTC) the instant the flash occurs, a FLE does not occur. Other studies, using different stimuli, report that the FLE does, in fact, occur in the FTC. The FTC is thus a crucial turning point in theories of flash-lag. Unraveling the mystery of the FLE in the FTC will help unravel the mechanisms underpinning flash-lag and perhaps even perceptual localization in general. Our experiments show that eccentricity of the moving stimulus was a contributing factor, as were eccentricity of the flashed stimulus and spatial separation between the two stimuli. Other factors, such as contrast and offset of moving stimulus, also modulate the magnitude of the FLE in the FTC. We surmise that uncertainty in determining the position in space of a moving stimulus is a key requirement for the lag-effect. A lag-effect in the FTC challenges influential models, such as differential latency, motion extrapolation, and postdiction. Based partly on the notion of an asymmetric spread of activity that arises because of the sheer nature of motion and from a combination of established physiological mechanisms, we propose a schematic account of the present findings that subsumes previous psychological models and scaffolds past experimental findings.