Detecting changes in one's own velocity from the optic flow

A bicycle can be balanced by stochastic optimal feedback control but only with accurate speed estimates

Balancing a bicycle is typical for the balance control humans perform as a part of a whole range of behaviors (walking, running, skating, skiing, etc.). This paper presents a general model of balance control and applies it to the balancing of a bicycle. Balance control has both a physics (mechanics) and a neurobiological component. The physics component pertains to the laws that govern the movements of the rider and his bicycle, and the neurobiological component pertains to the mechanisms via which the central nervous system (CNS) uses these laws for balance control. This paper presents a computational model of this neurobiological component, based on the theory of stochastic optimal feedback control (OFC). The central concept in this model is a computational system, implemented in the CNS, that controls a mechanical system outside the CNS. This computational system uses an internal model to calculate optimal control actions as specified by the theory of stochastic OFC. For the computational model to be plausible, it must be robust to at least two inevitable inaccuracies: (1) model parameters that the CNS learns slowly from interactions with the CNS-attached body and bicycle (i.e., the internal noise covariance matrices), and (2) model parameters that depend on unreliable sensory input (i.e., movement speed). By means of simulations, I demonstrate that this model can balance a bicycle under realistic conditions and is robust to inaccuracies in the learned sensorimotor noise characteristics. However, the model is not robust to inaccuracies in the movement speed estimates. This has important implications for the plausibility of stochastic OFC as a model for motor control.

Three-Dimensional Motion Perception: Comparing Speed and Speed Change Discrimination for Looming Stimuli

Judging the speed of objects moving in three dimensions is important in our everyday lives because we interact with objects in a three-dimensional world. However, speed perception has been seldom studied for motion in depth, particularly when using monocular cues such as looming. Here, we compared speed discrimination, and speed change discrimination, for looming stimuli, in order to better understand what visual information is used for these tasks. For the speed discrimination task, we manipulated the distance and duration information available, in order to investigate if participants were specifically using speed information. For speed change discrimination, total distance and duration were held constant; hence, they could not be used to successfully perform that task. For the speed change discrimination task, our data were consistent with observers not responding specifically to speed changes within an interval. Instead, they may have used alternative, arguably less optimal, strategies to complete the task. Evidence suggested that participants used a variety of cues to complete the speed discrimination task, not always solely relying on speed. Further, our data suggested that participants may have switched between cues on a trial to trial basis. We conclude that speed changes in looming stimuli were not used in a speed change discrimination task, and that naïve participants may not always exclusively use speed for speed discrimination.

Speed change discrimination for motion in depth using constant world and retinal speeds

Motion at constant speed in the world maps into retinal motion very differently for lateral motion and motion in depth. The former is close to linear, for the latter, constant speed objects accelerate on the retina as they approach. Motion in depth is frequently studied using speeds that are constant on the retina, and are thus not consistent with real-world constant motion. Our aim here was to test whether this matters: are we more sensitive to real-world motion? We measured speed change discrimination for objects undergoing accelerating retinal motion in depth (consistent with constant real-world speed), and constant retinal motion in depth (consistent with real-world deceleration). Our stimuli contained both looming and binocular disparity cues to motion in depth. We used a speed change discrimination task to obtain thresholds for conditions with and without binocular and looming motion in depth cues. We found that speed change discrimination thresholds were similar for accelerating retinal speed and constant retinal speed and were notably poor compared to classic speed discrimination thresholds. We conclude that the ecologically valid retinal acceleration in our stimuli neither helps, nor hinders, our ability to make judgements in a speed change discrimination task.

A review of human sensory dynamics for application to models of driver steering and speed control

In comparison with the high level of knowledge about vehicle dynamics which exists nowadays, the role of the driver in the driver-vehicle system is still relatively poorly understood. A large variety of driver models exist for various applications; however, few of them take account of the driver's sensory dynamics, and those that do are limited in their scope and accuracy. A review of the literature has been carried out to consolidate information from previous studies which may be useful when incorporating human sensory systems into the design of a driver model. This includes information on sensory dynamics, delays, thresholds and integration of multiple sensory stimuli. This review should provide a basis for further study into sensory perception during driving.

Visual Acceleration Perception for Simple and Complex Motion Patterns

Humans are able to judge whether a target is accelerating in many viewing contexts, but it is an open question how the motion pattern per se affects visual acceleration perception. We measured acceleration and deceleration detection using patterns of random dots with horizontal (simpler) or radial motion (more visually complex). The results suggest that we detect acceleration better when viewing radial optic flow than horizontal translation. However, the direction within each type of pattern has no effect on performance and observers detect acceleration and deceleration similarly within each condition. We conclude that sensitivity to the presence of acceleration is generally higher for more complex patterns, regardless of the direction within each type of pattern or the sign of acceleration.

Perceptual scaling of visual and inertial cues: effects of field of view, image size, depth cues, and degree of freedom

In the field of motion-based simulation, it was found that a visual amplitude equal to the inertial amplitude does not always provide the best perceived match between visual and inertial motion. This result is thought to be caused by the "quality" of the motion cues delivered by the simulator motion and visual systems. This paper studies how different visual characteristics, like field of view (FoV) and size and depth cues, influence the scaling between visual and inertial motion in a simulation environment. Subjects were exposed to simulator visuals with different fields of view and different visual scenes and were asked to vary the visual amplitude until it matched the perceived inertial amplitude. This was done for motion profiles in surge, sway, and yaw. Results showed that the subjective visual amplitude was significantly affected by the FoV, visual scene, and degree-of-freedom. When the FoV and visual scene were closer to what one expects in the real world, the scaling between the visual and inertial cues was closer to one. For yaw motion, the subjective visual amplitudes were approximately the same as the real inertial amplitudes, whereas for sway and especially surge, the subjective visual amplitudes were higher than the inertial amplitudes. This study demonstrated that visual characteristics affect the scaling between visual and inertial motion which leads to the hypothesis that this scaling may be a good metric to quantify the effect of different visual properties in motion-based simulation.

Velocity discrimination thresholds for flowfield motions with moving observers

The visual flow field, produced by forward locomotion, contains useful information about many aspects of visually guided behavior. But locomotion itself also contributes to possible distortions by adding head bobbing motions. Here we examine whether vertical head bobbing affects velocity discrimination thresholds and how the system may compensate for the distortions. Vertical head and eye movements while fixating were recorded during standing, walking or running on a treadmill. Bobbing noise was found to be larger during locomotion. The same observers were equally good at discriminating velocity increases in large accelerating flow fields when standing or walking or running. Simulated head bobbing was compensated when produced by pursuit eye movements, but not when it was part of the flow field. The results showed that these two contributions are additive and dealt with independently before they are combined. Distortions produced by body/head oscillations may also be compensated. Visual performance during running was at least as good as during walking, suggesting more efficient compensation mechanisms for running.

Multisensory Integration in Speed Estimation During Self-Motion

This study assessed the relative contributions of visual and proprioceptive/motor information during self-motion in a virtual environment using a speed discrimination task. Subjects wore a head-mounted display and rode a stationary bicycle along a straight path in an empty, seemingly infinite hallway with random surface texture. For each trial, subjects were required to pedal the bicycle along two paths at two different speeds (a standard speed and a comparison speed) and subsequently report whether the second speed travelled was faster than the first. The standard speed remained the same while the comparison speed was varied between trials according to the method of constant stimuli. When visual and proprioceptive/motor cues were provided separately or in combination, the speed discrimination thresholds were comparable, suggesting that either cue alone is sufficient. When the relation between visual and proprioceptive information was made inconsistent by varying optic flow gain, the resulting psychometric functions shifted along the horizontal axis (pedalling speed). The degree of separation between these functions indicated that both optic flow and proprioceptive cues contributed to speed estimation, with proprioceptive cues being dominant. These results suggest an important role for proprioceptive information in speed estimation during self-motion.

Perception of acceleration with short presentation times: can acceleration be used in interception?

To investigate whether visual judgments of acceleration could be used for intercepting moving targets, we determined how well subjects can detect acceleration when the presentation time is short. In a differential judgment task, two dots were presented successively. One dot accelerated and the other decelerated. Subjects had to indicate which of the two accelerated. In an absolute judgment task, subjects had to adjust the motion of a dot so that it appeared to move at a constant velocity. The results for the two tasks were similar. For most subjects, we could determine a detection threshold even when the presentation time was only 300 msec. However, an analysis of these thresholds suggests that subjects did not detect the acceleration itself but that they detected that a target had accelerated on the basis of the change in velocity between the beginning and the end of the presentation. A change of about 25% was needed to detect acceleration with reasonable confidence. Perhaps the simplest use of acceleration for interception consists of distinguishing between acceleration and deceleration of the optic projection of an approaching ball to determine whether one has to run backward or forward to catch it. We examined the results of a real ball-catching task (Oudejans, Michaels, & Bakker, 1997) and found that subjects reacted before acceleration could have been detected. We conclude that acceleration is not used in this simple manner to intercept moving targets.

Consistent stereoscopic information increases the perceived speed of vection in depth

Previous research found that adding stereoscopic information to radially expanding optic flow decreased vection onsets and increased vection durations (Palmisano, 1996 Perception & Psychophysics 58 1168-1176). In the current experiments, stereoscopic cues were also found to increase perceptions of vection speed and self-displacement during vection in depth--but only when these cues were consistent with monocularly available information about self-motion. Stereoscopic information did not appear to be improving vection by increasing the perceived maximum extent of displays or by making displays appear more three-dimensional. Rather, it appeared that consistent patterns of stereoscopic optic flow provided extra, purely binocular information about vection speed, which resulted in faster/more compelling illusions of self-motion in depth.

Role of lateral acceleration in curve driving: driver model and experiments on a real vehicle and a driving simulator

Experimental studies show that automobile drivers adjust their speed in curves so that maximum vehicle lateral accelerations decrease at high speeds. This pattern of lateral accelerations is described by a new driver model, assuming drivers control a variable safety margin of perceived lateral acceleration according to their anticipated steering deviations. Compared with a minimum time-to-lane-crossing (H. Godthelp, 1986) speed modulation strategy, this model, based on nonvisual cues, predicts that extreme values of lateral acceleration in curves decrease quadratically with speed, in accordance with experimental data obtained in a vehicle driven on a test track and in a motion-based driving simulator. Variations of model parameters can characterize "normal" or "fast" driving styles on the test track. On the simulator, it was found that the upper limits of lateral acceleration decreased less steeply when the motion cuing system was deactivated, although drivers maintained a consistent driving style. This is interpreted per the model as an underestimation of curvilinear speed due to the lack of inertial stimuli. Actual or potential applications of this research include a method to assess driving simulators as well as to identify driving styles for on-board driver aid systems.

Perception of two-dimensional, simulated ego-motion trajectories from optic flow

A veridical percept of ego-motion is normally derived from a combination of visual, vestibular, and proprioceptive signals. A previous study showed that blindfolded subjects can accurately perceive passively travelled straight or curved trajectories provided that the orientation of the head remained constant along the trajectory. When they were turned (whole-body, head-fixed) relative to the trajectory, errors occurred. We ask whether vision allows for better path perception in that situation, to correct or complement vestibular perception. Seated, stationary subjects wore a head mounted display showing optic flow stimuli which simulated linear or curvilinear 2D trajectories over a horizontal ground plane. The observer's orientation was either fixed in space, fixed relative to the path, or changed relative to both. After presentation, subjects reproduced the perceived movement with a model vehicle, of which position and orientation were recorded. They tended to correctly perceive ego-rotation (yaw), but they perceive orientation as fixed relative to trajectory or (unlike in the vestibular study) to space. This caused trajectory misperception when body rotation was wrongly attributed to a rotation of the path. Visual perception was very similar to vestibular perception.

Responses to continuously changing optic flow in area MST

We studied the temporal behavior and tuning properties of medial superior temporal (MST) neurons in response to constant flow-field stimulation and continuously changing flow-field stimulation (transitions), which were obtained by morphing one flow field into another. During transitions, the flow fields resembled the motion pattern seen by an observer during changing ego-motion. Our aim was to explore the behavior of MST cells in response to changes in the flow-field pattern and to establish whether the responses of MST cells are temporally independent or if they are affected by contextual information from preceding stimulation. We first tested whether the responses obtained during transitions were linear with respect to the two stimuli defining the transition. In over half of the transitions, the cell response was nonlinear: the response during the transition could not be predicted by the linear interpolation between the stimulus before and after the transition. Nonlinearities in the responses could arise from a dependence on temporal context or from nonlinearities in the tuning to flow-field patterns. To distinguish between these two hypotheses, we fit the responses during transitions and during continuous stimuli to the predictions of a temporally independent model (temporal-independence test) and we compared the responses during transitions to the responses elicited by inverse transitions (temporal-symmetry test). The effect of temporal context was significant in only 7.2% and 5.5% of cells in the temporal-independence test and in the temporal-symmetry test, respectively. Most of the nonlinearities in the cell responses could be accounted for by nonlinearities in the tuning to flow-field stimuli (i.e., the responses to a restricted set of flow fields did not predict the responses to other flow fields). Tuning nonlinearities indicate that a complete characterization of the tuning properties of MST neurons cannot be obtained by testing only a small number of flow fields. Because the cells' responses do not depend on temporal context, continuously changing stimulation can be used to characterize the receptive field properties of cells more efficiently than constant stimulation. Temporal independence in the responses to transitions indicates that MST cells do not code for second-order temporal properties of flow-field stimuli, i.e., for changes in the flow field through time that can be construed as paths through the environment. Information about ego-motion three-dimensional paths through the environment may either be processed at the population level in MST or in other cortical areas.

A self-organizing neural network architecture for navigation using optic flow

This article describes a self-organizing neural network architecture that transforms optic flow and eye position information into representations of heading, scene depth, and moving object locations. These representations are used to navigate reactively in simulations involving obstacle avoidance and pursuit of a moving target. The network's weights are trained during an action-perception cycle in which self-generated eye and body movements produce optic flow information, thus allowing the network to tune itself without requiring explicit knowledge of sensor geometry. The confounding effect of eye movement during translation is suppressed by learning the relationship between eye movement outflow commands and the optic flow signals that they induce. The remaining optic flow field is due to only observer translation and independent motion of objects in the scene. A self-organizing feature map categorizes normalized translational flow patterns, thereby creating a map of cells that code heading directions. Heading information is then recombined with translational flow patterns in two different ways to form maps of scene depth and moving object locations. Most of the learning processes take place concurrently and evolve through unsupervised learning. Mapping the learned heading representations onto heading labels or motor commands requires additional structure. Simulations of the network verify its performance using both noise-free and noisy optic flow information.

Perceiving self-motion in depth: the role of stereoscopic motion and changing-size cues

During self-motions, different patterns of optic flow are presented to the left and right eyes. Previous research has, however, focused mainly on the self-motion information contained in a single pattern of optic flow. The present experiments investigated the role that binocular disparity plays in the visual perception of self-motion, showing that the addition of stereoscopic cues to optic flow significantly improves forward linear vection in central vision. Improvements were also achieved by adding changing-size cues to sparse (but not dense) flow patterns. These findings showed that assumptions in the heading literature that stereoscopic cues facilitate self-motion only when the optic flow has ambiguous depth ordering do not apply to vection. Rather, it was concluded that both stereoscopic and changing-size cues provide additional motion-in-depth information that is used in perceiving self-motion.