On the relativity of perceived motion

Unveiling the time course of visual stabilization through human electrophysiology

Object positions are coded relative to their surroundings, presumably providing visual stability during eye movements. But when does this perceived stability arise? Here we used a visual illusion, the frame-induced position shift, and measured electrophysiological activity elicited by an object whose perceived position was either shifted because of a surrounding frame or not, thus dissociating perceived and physical locations. We found that visually evoked responses were sensitive to only physical location earlier in time (∼70 ms), but both physical and illusory location information was present at a later time point (∼140 ms). Furthermore, location information could be reliably decoded across physical and illusory locations during the later time interval but not during the earlier time interval, demonstrating that neural activity patterns are shared between the two processes at a later stage. These results suggest that visual stability of objects emerges relatively late and is thus dependent on recurrent feedback from higher processing stages.

Differential processing: towards a unified model of direction and speed perception

In two experiments, we demonstrate a misperception of the velocity of a random-dot stimulus moving in the presence of a static line oriented obliquely to the direction of dot motion. As shown in previous studies, the perceived direction of the dots is shifted away from the orientation of the static line, with the size of the shift varying as a function of line orientation relative to dot direction (the statically-induced direction illusion, or 'SDI'). In addition, we report a novel effect - that perceived speed also varies as a function of relative line orientation, decreasing systematically as the angle is reduced from 90° to 0°. We propose that these illusions both stem from the differential processing of object-relative and non-object-relative component velocities, with the latter being perceptually underestimated with respect to the former by a constant ratio. Although previous proposals regarding the SDI have not allowed quantitative accounts, we present a unified formal model of perceived velocity (both direction and speed) with the magnitude of this ratio as the only free parameter. The model was successful in accounting for the angular repulsion of motion direction across line orientations, and in predicting the systematic decrease in perceived velocity as the line's angle was reduced. Although fitting for direction and speed produced different best-fit values of the ratio of underestimation of non-object-relative motion compared to object-relative motion (with the ratio for speed being larger than that for direction) this discrepancy may be due to differences in the psychophysical procedures for measuring direction and speed.

Visuomotor Velocity Transformations for Smooth Pursuit Eye Movements

Smooth pursuit eye movements are driven by retinal motion signals. These retinal motion signals are converted into motor commands that obey Listing's law (i.e., no accumulation of ocular torsion). The fact that smooth pursuit follows Listing's law is often taken as evidence that no explicit reference frame transformation between the retinal velocity input and the head-centered motor command is required. Such eye-position-dependent reference frame transformations between eye- and head-centered coordinates have been well-described for saccades to static targets. Here we suggest that such an eye (and head)-position-dependent reference frame transformation is also required for target motion (i.e., velocity) driving smooth pursuit eye movements. Therefore we tested smooth pursuit initiation under different three-dimensional eye positions and compared human performance to model simulations. We specifically tested if the ocular rotation axis changed with vertical eye position, if the misalignment of the spatial and retinal axes during oblique fixations was taken into account, and if ocular torsion (due to head roll) was compensated for. If no eye-position-dependent velocity transformation was used, the pursuit initiation should follow the retinal direction, independently of eye position; in contrast, a correct visuomotor velocity transformation would result in spatially correct pursuit initiation. Overall subjects accounted for all three components of the visuomotor velocity transformation, but we did observe differences in the compensatory gains between individual subjects. We concluded that the brain does perform a visuomotor velocity transformation but that this transformation was prone to noise and inaccuracies of the internal model.

Motion perception during selfmotion: The direct versus inferential controversy revisited

According to the traditional inferential theory of perception, percepts of object motion or stationarity stem from an evaluation of afferent retinal signals (which encode image motion) with the help of extraretinal signals (which encode eye movements). According to direct perception theory, on the other hand, the percepts derive from retinally conveyed information only. Neither view is compatible with a perceptual phenomenon that occurs during visually induced sensations of ego motion (vection). A modified version of inferential theory yields a model in which the concept of extraretinal signals is replaced by that of reference signals, which do not encode how the eyes move in their orbits but how they move in space. Hence reference signals are produced not only during eye movements but also during ego motion (i.e., in response to vestibular stimulation and to retinal image flow, which may induce vection). The present theory describes the interface between self-motion and object-motion percepts. An experimental paradigm that allows quantitative measurement of the magnitude and gain of reference signals and the size of the just noticeable difference (JND) between retinal and reference signals reveals that the distinction between direct and inferential theories largely depends on: (1) a mistaken belief that perceptual veridicality is evidence that extraretinal information is not involved, and (2) a failure to distinguish between (the perception of) absolute object motion in space and relative motion of objects with respect to each other. The model corrects these errors, and provides a new, unified framework for interpreting many phenomena in the field of motion perception.

Neuronal correlates of perceptual stability during eye movements

We are usually unaware of retinal image motion resulting from our own movement. For instance, during slow-tracking eye movements the world around us remains perceptually stable despite the retinal image slip induced by the eye movement. It is commonly held that this example of perceptual invariance is achieved by subtracting an internal reference signal, reflecting the eye movement, from the retinal motion signal. If the two cancel each other, visual objects, which do not move, will also be perceived as non-moving. If, however, the reference signal is too small or too large, a false eye movement-induced motion of the external world, the Filehne illusion, will be perceived. We have exploited our ability to manipulate the size of the reference signal in an attempt to identify neurons in the visual cortex of monkeys, influenced by the percept of self-induced visual motion or the reference signal rather than the retinal motion signal. We report here that such 'percept-related' neurons can already be found in the primary visual cortex area, although few in numbers. They become more frequent in areas middle temporal and medial superior temporal in the superior temporal sulcus, and comprise almost 50% of all neurons in area visual posterior sylvian (VPS) in the posterior part of the lateral sulcus. In summary, our findings suggest that our ability to perceive a visual world, which is stable despite self-motion, is based on a neuronal network, which culminates in the VPS located in the lateral sulcus below the classical dorsal stream of visual processing.

Human updating of visual motion direction during head rotations

Previous studies have demonstrated that human subjects update the location of visual targets for saccades after head and body movements and in the absence of visual feedback. This phenomenon is known as spatial updating. Here we investigated whether a similar mechanism exists for the perception of motion direction. We recorded eye positions in three dimensions and behavioral responses in seven subjects during a motion task in two different conditions: when the subject's head remained stationary and when subjects rotated their heads around an anteroposterior axis (head tilt). We demonstrated that after head-tilt subjects updated the direction of saccades made in the perceived stimulus direction (direction of motion updating), the amount of updating varied across subjects and stimulus directions, the amount of motion direction updating was highly correlated with the amount of spatial updating during a memory-guided saccade task, subjects updated the stimulus direction during a two-alternative forced-choice direction discrimination task in the absence of saccadic eye movements (perceptual updating), perceptual updating was more accurate than motion direction updating involving saccades, and subjects updated motion direction similarly during active and passive head rotation. These results demonstrate the existence of an updating mechanism for the perception of motion direction in the human brain that operates during active and passive head rotations and that resembles the one of spatial updating. Such a mechanism operates during different tasks involving different motor and perceptual skills (saccade and motion direction discrimination) with different degrees of accuracy.

Unilateral vestibular failure suppresses cortical visual motion processing

Patients with unilateral vestibular failure (UVF) experience oscillopsia (apparent motion of the visual scene) during rapid head movements due to increased retinal slip caused by vestibulo-ocular reflex impairment. Oscillopsia is always smaller than the net retinal slip and decreases over time in patients with acquired vestibular loss; this correlates with increased thresholds for visual motion detection and increased tolerance to retinal slip. We investigated the underlying cortical adaptive processes using visual motion stimulation during blood oxygen level-dependent (BOLD) fMRI. Optokinetic nystagmus was elicited in seven patients with right-sided and seven patients with left-sided unilateral vestibular neurectomy and in seven age- and gender-matched healthy controls. Patients showed diminished activation of bilateral visual cortex areas (including the motion-sensitive area MT/V5, cuneus, middle occipital, fusiform and lingual areas) and ocular motor regions compared to their controls during visual motion stimulation. Concurrent BOLD signal decreases of temporo-parietal and insular multisensory cortical areas occurred in controls and patients. The diminished activation of visual motion processing areas plausibly reflects an adaptive mechanism that suppresses distressing oscillopsia in patients with UVF and thereby stabilizes the perceived visual surroundings. This study provides for the first time neuroimaging evidence of suppressed cortical visual motion processing in patients with vestibulopathy.

Neural noise distorts perceived motion: the special case of the freezing illusion and the Pavard and Berthoz effect

When a slowly moving pattern is presented on a monitor which itself is moved, the pattern appears to freeze on the screen (Mesland and Wertheim in Vis Res 36(20):3325–3328, 1996) even if we move our head with the monitor, as with a head mounted display (Pavard and Berthoz in Perception 6:529–540, 1977). We present a simple model of these phenomena, which states that the perceived relative velocity between two stimuli (the pattern and the moving monitor) is proportional to the difference between the perceived velocities of these stimuli in space, minus a noise factor. The latter reflects the intrinsic noise in the neural signals that encode retinal image velocities. With noise levels derived from the literature the model fits empirical data well and also predicts strong distortions of visually perceived motion during vestibular stimulation, thus explaining both illusions as resulting from the same mechanism.

Direction and extent of perceived motion smear during pursuit eye movement

Smooth pursuit eye movements superimpose additional motion on the retinal image of untracked visual targets, potentially leading to the perception of motion smear and a distortion of the perceived direction of motion. Previously, we demonstrated an attenuation of perceived motion smear when the untracked target moves in the opposite direction of an ongoing pursuit eye movement. In this study, the extent of perceived motion smear and the direction of perceived smear were compared for a single bright dot that moved in a wide range of directions with respective to horizontal pursuit at 8 deg/s. Comparable data were obtained during fixation as a control. The results indicate that a significant attenuation of perceived motion smear occurs when the dot's motion includes a horizontal component in the opposite direction of eye movement. In contrast, the direction of perceived smear approximates the trajectory of the retinal image motion, during both fixation and pursuit. These results suggest a selective application of extra-retinal signals to compensate specific aspects of visual perception that results from the retinal image motion during smooth pursuit eye movements.

Wertheim's hypothesis on 'highway hypnosis': empirical evidence from a study on motorway and conventional road driving

This paper aims to study the phenomenon known as 'highway hypnosis' or 'driving without attention mode', which has been defined as a state showing sleepiness signs and attention slip resulting from driving a motor vehicle for a long period in a highly predictable environment with low event occurrence, this being the case with motorways and very familiar roads [Highway hypnosis: a theoretical analysis. In: Gale, A.G., Brown, I.D., Haslegrave, C.M., Moorhead, I., Taylor, S. (Eds.), Vision in Vehicles-III. Elsevier, North-Holland, pp. 467-472]. According to Wertheim's hypothesis on 'highway hypnosis', long-term driving on motorways and conventional roads, e.g. main roads, secondary roads--implies differences in the predictability of the movement pattern of the visual stimulation, in the eye musculature activity and in the type of feedback used in visual information processing (mostly extra-retinal on motorways and retinal and extra-retinal on conventional roads). All this ultimately leads to alertness differences between both road types. Our research is intended to provide empirical evidence from the hypothesis, based on the data recorded during the actual driving experience of a group of subjects on a motorway and a conventional road. We studied whether or not significant alertness differences were found-measured by EEG data relative to time periods of on-target eye-tracking performance--between motorway and conventional road driving. Our results partially support the hypothesis, as drowsiness proved to be higher on motorways than on conventional roads during the final driving period but not during the starting stage, when the opposite trend was noticed. This result could be explained by the fact that during the first driving periods the effects of the stimulus movement predictability had not yet become apparent, since they tend to show after a long drive.

Systematic distortion of perceived 2D shape during smooth pursuit eye movements

Even when the retinal image of a static scene is constantly shifting, as occurs when the viewer pursues a small moving object with his or her eyes, the scene is usually correctly perceived to be static. Following early suggestions by von Helmholtz, it is commonly believed that this spatial stability is achieved by combining retinal and extra-retinal signals. Here, we report a perceptually salient 2D shape distortion that can arise during pursuit. We provide evidence that the perceived 2D shape reflects retinal image contents alone, implying that the extra-retinal signal is ignored when judging 2D shape.

Velocity not acceleration of self-motion mediates vestibular-visual interaction

We investigated the influence of vestibular stimulation with different angular accelerations and velocities on the perception of visual motion direction. Constant accelerations resulting in different angular velocities and constant angular velocities obtained at different accelerations were combined in twenty healthy subjects. Random-dot kinematograms with coherently moving pixels and randomly moving pixels were used as visual stimuli during whole-body rotations. The smallest percentage of coherently moving pixels leading to a clear perception of motion direction was taken as the perception threshold. Perception thresholds significantly increased with increasing angular velocity. Increased acceleration, however, had no significant effect on the perception thresholds. We conclude that the achieved angular velocity, and not acceleration, is the predominant factor in the processing of vestibular-visual interaction.

Eye movements affect the perceived speed of visual motion

Eye movements add a constant displacement to the visual scene, altering the retinal-image velocity. Therefore, in order to recover the real world motion, eye-movement effects must be compensated. If full compensation occurs, the perceived speed of a moving object should be the same regardless of whether the eye is stationary or moving. Using a pursue-fixate procedure in a perceptual matching paradigm, we found that eye movements systematically bias the perceived speed of the distal stimulus, indicating a lack of compensation. Speed judgments depended on the interaction between the distal stimulus size and the eye velocity relative to the distal stimulus motion. When the eyes and distal stimulus moved in the same direction, speed judgments of the distal stimulus approximately matched its retinal-image motion. When the eyes and distal stimulus moved in the opposite direction, speed judgments depended on the stimulus size. For small sizes, perceived speed was typically overestimated. For large sizes, perceived speed was underestimated. Results are explained in terms of retinal-extraretinal interactions and correlate with recent neurophysiological findings.

An electrophysiological correlate of visual motion awareness in man

It is usually held that perceptual spatial stability, despite smooth pursuit eye movements, is accomplished by comparing a signal reflecting retinal image slip with an internal reference signal, encoding the eye movement. The important consequence of this concept is that our subjective percept of visual motion reflects the outcome of this comparison rather than retinal image slip. In an attempt to localize the cortical networks underlying this comparison and therefore our subjective percept of visual motion, we exploited an imperfection inherent in it, which results in a movement illusion. If smooth pursuit is carried out across a stationary background, we perceive a tiny degree of illusionary background motion (Filehne illusion, or FI), rather than experiencing the ecologically optimal percept of stationarity. We have recently shown that this illusion can be modified substantially and predictably under laboratory conditions by visual motion unrelated to the eye movement. By making use of this finding, we were able to compare cortical potentials evoked by pursuit-induced retinal image slip under two conditions, which differed perceptually, while being identical physically. This approach allowed us to discern a pair of potentials, a parieto-occipital negativity (N300) followed by a frontal positivity (P300), whose amplitudes were solely determined by the subjective perception of visual motion irrespective of the physical attributes of the situation. This finding strongly suggests that subjective awareness of visual motion depends on neuronal activity in a parieto-occipito-frontal network, which excludes the early stages of visual processing.

Perception of direction of visual motion. I. Influence of angular body acceleration and tilt

We investigated, psychophysically, the influence of body rotation on visual motion direction thresholds for both upright sitting and tilted observers. Four angular accelerations (0, 20, 40 and 60 degrees/s2) were combined with 3 concurrent backward-tilt positions (0, 45 and 90 degrees). This led to combined stimulation of the semicircular canals and otoliths. Vestibular stimulation was combined with a visual motion stimulus. Random-dot kinematograms in which varying percentages of pixels coherently moving to the left were presented upon a background of otherwise randomly moving pixels (random walk). The smallest percentage of coherently moving pixels leading to a clear perception of motion direction represented as the perceptual threshold. Angular accelerations about the longitudinal body axis significantly increased motion-direction thresholds. Concurrent backward tilt did not influence thresholds. These results differ from those of studies in which translational linear acceleration was employed. Our results support the view that it is necessary to distinguish between linear acceleration caused by gravitational forces and that caused by additional linear accelerations about the x-, y-, and z-axes.

Speed discrimination of distal stimuli during smooth pursuit eye motion

We evaluated the hypothesis that smooth pursuit eye movements affect speed discrimination thresholds of distal stimuli because they alter the retinal image speed. Subjects judged speed differences of sine-wave gratings while they simultaneously pursued a superimposed moving bar. Speed discrimination thresholds were measured, under conditions of controlled eye movements, for grating speeds of 0.5 and 2.0 deg/sec across a range of eye velocities. Thresholds were stimulated using a Monte Carlo method based on the retinal speed hypothesis, and the simulation predictions were compared to the psychophysically determined thresholds. The simulation results provided a good match to the psychophysical data for conditions where the eye moved at a slower speed than the grating, regardless of whether the eye moved in the same or opposite direction. However, when the eye moved at a faster speed than the grating in the same direction, the psychophysical thresholds were significantly higher than predicted by the simulation. Control experiments and analyses rule out explanations based on relative motion cues, saccadic involvement, and attentional demands.

Speed discrimination under stabilized and normal viewing conditions

To determine whether speed discrimination improves when the retinal image is stabilized against the effects of eye movements, thresholds were measured under stabilized and normal viewing conditions. In the normal viewing conditions, eye movements were recorded and used to estimate retinal-image speeds. Stimulus reference speed for sinusoidal gratings varied from 0.5 to 8.0 deg/sec. Results showed that speed discrimination thresholds, expressed as Weber ratios, decreased with increasing stimulus speed for both the normal and stabilized viewing conditions. Stabilized viewing thresholds were higher than normal viewing thresholds only at the slowest stimulus reference speed. However, when speed discrimination thresholds were expressed as a function of the estimated retinal speed, there was no difference in thresholds for the stabilized and normal viewing conditions. A retinal-image model, whereby speed discrimination depends on retinal-image motion, explains the results.

Effects of background stimulation upon eye-movement information

To investigate the effects of background stimulation upon eye-movement information (EMI), the perceived deceleration of the target motion during pursuit eye movement (Aubert-Fleishl paradox) was analyzed. In the experiment, a striped pattern was used as a background stimulus with various brightness contrasts and spatial frequencies for serially manipulating the attributions of the background stimulus. Analysis showed that the retinal-image motion of the background stimulus (optic flow) affected eye-movement information and that the effects of optic flow became stronger when high contrast and low spatial frequency stripes were presented as the background stimulus. In conclusion, optic flow is one source of eye-movement information in determining real object motion, and the effectiveness of optic flow depends on the attributes of the background stimulus.

Modification of the Filehne illusion by conditioning visual stimuli

During smooth pursuit eye movements made across a stationary background an illusory motion of the background is perceived (Filehne illusion). The present study was undertaken in order to test if the Filehne illusion can be influenced by information unrelated to the retinal image slip prevailing and to the eye movement being executed. The Filehne illusion was measured in eight subjects by determining the amount of external background motion required to compensate for the illusory background motion induced by 12 deg/sec rightward smooth pursuit. Using a two-alternative forced-choice method, test trials, which yielded the estimate of the Filehne illusion, were randomly interleaved with conditioning trials, in which high retinal image slip was created by background stimuli moving at a constant horizontal velocity. There was a highly reproducible monotic relationship between the size and direction of the Filehne illusion and the velocity of the background stimulus in the conditioning trials with the following extremes: large Filehne illusions with illusory motion to the right occurred for conditioning stimuli moving to the left, i.e. opposite to the direction of eye movement in the test trials, while conversely, conditioning stimuli moving to the right yielded Filehne illusions close to zero. Additional controls suggest that passive motion aftereffects are unlikely to account for the modulation of the Filehne illusion by the conditioning stimulus. We hypothesize that this modification might reflect the dynamic character of the networks elaborating spatial constancy.

Detection thresholds for object motion and self-motion during vestibular and visuo-oculomotor stimulation

We compared the detection threshold for object motion with that of self-motion in space in healthy human subjects. Stimuli consisted of horizontal rotations of subjects' body with a fixation spot kept in fixed alignment with their heads (vestibular stimulus), rotation of the fixation spot relative to the stationary subjects (visuo-oculomotor stimulus), and a combination thereof by applying rotations of subjects body relative to the stationary object (sinusoidal oscillations, 0.025-0.4 Hz). Two series of experiments were performed. 1) One group of subjects was instructed to attend to, and to indicate the occurrence of, either object or self-motion. 2) A second group was instructed not only to detect the occurrence of a perception, but also to quality it either as object motion or self-motion, depending on which modality dominated perceptually. With either instruction it was found that all three stimulus conditions could evoke both, either an object motion perception or a self-motion perception. The detection thresholds of both perceptions were essentially similar. Thresholds were highest with the vestibular stimulus, intermediate with the stimulus combination, and lowest with the visuo-oculomotor stimulus. The vestibular threshold depended on stimulus frequency, in that it decreased with increasing frequency. Thereby, it became similar to the visuo-oculomotor one, which was essentially constant across frequency. Probability of occurrence of the perceptions in the first experimental series was considerably higher than in the second series, suggesting an important role of attentional mechanisms. In the second series, percent frequency of occurrence of veridical perception (object motion with visuo-oculomotor stimulus, self-motion with stimulus combination) was at chance level (50%) at low stimulus frequency, but was augmented considerably at high frequency. We assume that the latter effect is brought about by a visual-vestibular conflict measure by which the visual stimulus (light spot) is qualified as representing either a moving object or a spatial reference for self-motion. While at suprathreshold stimulus intensities the conflict can determine perception magnitude, at threshold levels its influence is restricted mainly on the probability of occurrence of object and self-motion perception.

Processing of visual motion direction in the fronto-parallel plane in the stationary or moving observer

To examine the effect of concurrent self-motion on the perception of the direction of object-motion, random-dot kinematograms were employed in which the strength of the directional signal was manipulated by varying the percentage of coherently moving pixels. The subject's task was to indicate the motion direction of briefly presented displays while undergoing whole body rotations with angular accelerations of 0, 5, 15, or 45 degrees/s2. The perception of the direction of visual motion in the horizontal plane was impaired only when visual and vestibular motion directions were incongruous. The impairment increases with both increasing angular acceleration and decreasing percentage of coherently moving pixels. For object-motion in the vertical plane, an impairment was found for both congruous and incongruous combination of visual and vestibular stimulation, although not as pronounced for the latter (i.e., visual upward, vestibular downward stimulation, and vice versa). These results are discussed in terms of postnatal development and neurophysiological optimization processes resulting from intersensory 'updating' through every-day experience of object-motion during self-motion.

Estimating heading during eye movements

In eight experiments, we examined the ability to judge heading during tracking eye movements. To assess the use of retinal-image and extra-retinal information in this task, we compared heading judgments with executed as opposed to simulated eye movements. In general, judgments were much more accurate during executed eye movements. Observers in the simulated eye movement condition misperceived their self-motion as curvilinear translation rather than the linear translation plus eye rotation that was simulated. There were some experimental conditions in which observers could judge heading reasonably accurately during simulated eye movements; these included conditions in which eye movement velocities were 1 deg/sec or less and conditions which made available a horizon cue that exists for locomotion parallel to a ground plane with a visible horizon. Overall, our results imply that extra-retinal, eye-velocity signals are used in determining heading under many, perhaps most, viewing conditions.