The surface of the empirical horopter

The Statistics of Eye Movements and Binocular Disparities during VR Gaming: Implications for Headset Design

The human visual system evolved in environments with statistical regularities. Binocular vision is adapted to these such that depth perception and eye movements are more precise, faster, and performed comfortably in environments consistent with the regularities. We measured the statistics of eye movements and binocular disparities in virtual-reality (VR) - gaming environments and found that they are quite different from those in the natural environment. Fixation distance and direction are more restricted in VR, and fixation distance is farther. The pattern of disparity across the visual field is less regular in VR and does not conform to a prominent property of naturally occurring disparities. From this we predict that double vision is more likely in VR than in the natural environment. We also determined the optimal screen distance to minimize discomfort due to the vergence-accommodation conflict, and the optimal nasal-temporal positioning of head-mounted display (HMD) screens to maximize binocular field of view. Finally, in a user study we investigated how VR content affects comfort and performance. Content that is more consistent with the statistics of the natural world yields less discomfort than content that is not. Furthermore, consistent content yields slightly better performance than inconsistent content.

Learning bio-inspired head-centric representations of 3D shapes in an active fixation setting

When exploring the surrounding environment with the eyes, humans and primates need to interpret three-dimensional (3D) shapes in a fast and invariant way, exploiting a highly variant and gaze-dependent visual information. Since they have front-facing eyes, binocular disparity is a prominent cue for depth perception. Specifically, it serves as computational substrate for two ground mechanisms of binocular active vision: stereopsis and binocular coordination. To this aim, disparity information, which is expressed in a retinotopic reference frame, is combined along the visual cortical pathways with gaze information and transformed in a head-centric reference frame. Despite the importance of this mechanism, the underlying neural substrates still remain widely unknown. In this work, we investigate the capabilities of the human visual system to interpret the 3D scene exploiting disparity and gaze information. In a psychophysical experiment, human subjects were asked to judge the depth orientation of a planar surface either while fixating a target point or while freely exploring the surface. Moreover, we used the same stimuli to train a recurrent neural network to exploit the responses of a modelled population of cortical (V1) cells to interpret the 3D scene layout. The results for both human performance and from the model network show that integrating disparity information across gaze directions is crucial for a reliable and invariant interpretation of the 3D geometry of the scene.

The blur horopter: Retinal conjugate surface in binocular viewing

From measurements of wavefront aberrations in 16 emmetropic eyes, we calculated where objects in the world create best-focused images across the central 27∘ (diameter) of the retina. This is the retinal conjugate surface. We calculated how the surface changes as the eye accommodates from near to far and found that it mostly maintains its shape. The conjugate surface is pitched top-back, meaning that the upper visual field is relatively hyperopic compared to the lower field. We extended the measurements of best image quality into the binocular domain by considering how the retinal conjugate surfaces for the two eyes overlap in binocular viewing. We call this binocular extension the blur horopter. We show that in combining the two images with possibly different sharpness, the visual system creates a larger depth of field of apparently sharp images than occurs with monocular viewing. We examined similarities between the blur horopter and its analog in binocular vision: the binocular horopter. We compared these horopters to the statistics of the natural visual environment. The binocular horopter and scene statistics are strikingly similar. The blur horopter and natural statistics are qualitatively, but not quantitatively, similar. Finally, we used the measurements to refine what is commonly referred to as the zone of clear single binocular vision.

The vergence horopter

The horopter is defined by the criterion with which it is measured; these include the apparent frontoparallel plane, the region of binocular single vision, and identical visual direction. Another criterion is one that "the positions of the points are such that none provides a stimulus for fusional movement of the eyes". Measuring the horopter using this criterion is the aim of the current paper. Based on previous research investigating corresponding points, we predicted the vergence horopter should trace an arc similar to the empirical horizontal horopter. We further predicted based on the characteristics of vergence eye movements in the periphery, that the range of horizontal disparities that do not elicit a vergence movement would increase with eccentricity. In experiment 1 we conducted the first measurements of the vergence horopter along the horizontal plane of regard and sagittal planes. We found that the vergence horopter consists of a stable range of disparities that do not increase with eccentricity. In experiment 2, we measured Panum's fusional range for the same eccentricities and elevations to determine the placement of the vergence horopter within the fusional ranges. In the horizontal plane, the vergence horopter sits centrally within Panum's fusional range. In the vertical plane, the data are suggestive that the vergence horopter is less slanted than the corresponding fusional range.

A Geometric Theory Integrating Human Binocular Vision With Eye Movement

A theory of the binocular system with asymmetric eyes (AEs) is developed in the framework of bicentric perspective projections. The AE accounts for the eyeball's global asymmetry produced by the foveal displacement from the posterior pole, the main source of the eye's optical aberrations, and the crystalline lens' tilt countering some of these aberrations. In this theory, the horopter curves, which specify retinal correspondence of binocular single vision, are conic sections resembling empirical horopters. This advances the classic model of empirical horopters as conic sections introduced in an ad hoc way by Ogle in 1932. In contrast to Ogle's theory, here, anatomically supported horopteric conics vary with the AEs' position in the visual plane of bifoveal fixations and their transformations are visualized in a computer simulation. Integrating horopteric conics with eye movements can help design algorithms for maintaining a stable perceptual world from visual information captured by a mobile robot's camera head. Further, this paper proposes a neurophysiologically meaningful definition for the eyes' primary position, a concept which has remained elusive despite its theoretical importance to oculomotor research. Finally, because the horopteric conic's shape is dependent on the AE's parameters, this theory allows for changes in retinal correspondence, which is usually considered preformed and stable.

Stereoscopic depth constancy from a different direction

To calibrate stereoscopic depth from disparity our visual system must compensate for an object's egocentric location. Ideally, the perceived three-dimensional shape and size of objects in visual space should be invariant with their location such that rigid objects have a consistent identity and shape. These percepts should be accurate enough to support both perceptual judgments and visually-guided interaction. This theoretical note reviews the relationship of stereoscopic depth constancy to the geometry of stereoscopic space and seemingly esoteric concepts like the horopter. We argue that to encompass the full scope of stereoscopic depth constancy, researchers need to consider not just distance but also direction, that is 3D egocentric location in space. Judgements of surface orientation need to take into account the shape of the horopter and the computation of metric depth (when tasks depend on it) must compensate for direction as well as distance to calibrate disparities. We show that the concept of the horopter underlies these considerations and that the relationship between depth constancy and the horopter should be more explicit in the literature.

The upper disparity limit increases gradually with eccentricity

Stereopsis is important for tasks of daily living such as eye-hand coordination. It is best in central vision but is also mediated by the periphery. Previously we have shown that individuals with central-field loss who have residual stereopsis in the periphery perform better at an eye-hand-coordination task when they perform the task binocularly rather than monocularly. Here we seek to determine what sets the limit of stereopsis, defined as the largest disparity that supports the sustained appearance of depth, in the near periphery in healthy individuals. While stereoacuity thresholds increase sharply with eccentricity, Panum's area increases much more slowly. We used a rigorous method to determine the uppermost limit of disparity. At long durations, the two half-images that define a large disparity appear as two isolated targets in the same flat plane; small incremental changes in disparity produce changes in the separation between the half-images, and disparity magnitude can be judged on the basis of separation, like a monocular width judgment. The disparity limit is the point at which the threshold for judging dichoptic separation between the half-images is equal to the monocular width-discrimination threshold. The disparity limit at 10° was a factor of 2-4 times larger than the fovea, regardless of the meridian tested. The increase in the disparity limit with eccentricity was shallow, similar to that of Panum's area. Within this disparity limit, disparity increment thresholds were comparable for foveal and peripheral targets, illustrating the significance and utility of peripheral stereopsis, especially in the absence of foveal stereopsis.

Binocular Eye Movements Are Adapted to the Natural Environment

Humans and many animals make frequent saccades requiring coordinated movements of the eyes. When landing on the new fixation point, the eyes must converge accurately or double images will be perceived. We asked whether the visual system uses statistical regularities in the natural environment to aid eye alignment at the end of saccades. We measured the distribution of naturally occurring disparities in different parts of the visual field. The central tendency of the distributions was crossed (nearer than fixation) in the lower field and uncrossed (farther) in the upper field in male and female participants. It was uncrossed in the left and right fields. We also measured horizontal vergence after completion of vertical, horizontal, and oblique saccades. When the eyes first landed near the eccentric target, vergence was quite consistent with the natural-disparity distribution. For example, when making an upward saccade, the eyes diverged to be aligned with the most probable uncrossed disparity in that part of the visual field. Likewise, when making a downward saccade, the eyes converged to enable alignment with crossed disparity in that part of the field. Our results show that rapid binocular eye movements are adapted to the statistics of the 3D environment, minimizing the need for large corrective vergence movements at the end of saccades. The results are relevant to the debate about whether eye movements are derived from separate saccadic and vergence neural commands that control both eyes or from separate monocular commands that control the eyes independently.SIGNIFICANCE STATEMENT We show that the human visual system incorporates statistical regularities in the visual environment to enable efficient binocular eye movements. We define the oculomotor horopter: the surface of 3D positions to which the eyes initially move when stimulated by eccentric targets. The observed movements maximize the probability of accurate fixation as the eyes move from one position to another. This is the first study to show quantitatively that binocular eye movements conform to 3D scene statistics, thereby enabling efficient processing. The results provide greater insight into the neural mechanisms underlying the planning and execution of saccadic eye movements.

On the role of ocular torsion in binocular visual matching

When an observer scans the visual surround, the images cast on the two retinae are slightly different due to the different viewpoints of the two eyes. Objects in the horizontal plane of regard can be seen single by aligning the lines of sight without changing the torsional stance of the eyes. Due to the peculiar ocular kinematics this is not possible for objects above or below the horizontal plane of regard. We provide evidence that binocular fusion can be achieved independently of viewing direction by adjusting the mutual torsional orientation of the eyes in the frontal plane. We characterize the fusion positions of the eyes across the oculomotor range by deriving simple trigonometric equations for the required torsion as a function of gaze direction and compute the iso-torsion contours yielding binocular fusion. Finally, we provide experimental evidence that eye positions in far-to-near re-fixation saccades indeed converge towards the predicted positions by adjusting the torsion of the eyes. This is the first report that describes the three-dimensional orientation of the eyes at binocular fusion positions based on the three-dimensional ocular kinematics. It closes a gap between the sensory and the motor side of binocular vision and stereoscopy.

The effect of image position on the Independent Components of natural binocular images

Human visual performance degrades substantially as the angular distance from the fovea increases. This decrease in performance is found for both binocular and monocular vision. Although analysis of the statistics of natural images has provided significant insights into human visual processing, little research has focused on the statistical content of binocular images at eccentric angles. We applied Independent Component Analysis to rectangular image patches cut from locations within binocular images corresponding to different degrees of eccentricity. The distribution of components learned from the varying locations was examined to determine how these distributions varied across eccentricity. We found a general trend towards a broader spread of horizontal and vertical position disparity tunings in eccentric regions compared to the fovea, with the horizontal spread more pronounced than the vertical spread. Eccentric locations above the centroid show a strong bias towards far-tuned components, eccentric locations below the centroid show a strong bias towards near-tuned components. These distributions exhibit substantial similarities with physiological measurements in V1, however in common with previous research we also observe important differences, in particular distributions of binocular phase disparity which do not match physiology.

Angle of gaze and optic flow direction modulate body sway

Optic flow is a crucial signal in maintaining postural stability. We sought to investigate whether the activity of postural muscles and body sway was modulated by eye position during the view of radial optic flow stimuli. We manipulated the spatial distribution of dot speed and the fixation point position to simulate specific heading directions combined with different gaze positions. The experiments were performed using stabilometry and surface electromyography (EMG) on 24 right-handed young, healthy volunteers. Center of pressure (COP) signals were analyzed considering antero-posterior and medio-lateral oscillation, COP speed, COP area, and the prevalent direction of oscillation of body sway. We found a significant main effect of body side in all COP parameters, with the right body side showing greater oscillations. The different combinations of optic flow and eye position evoked a non-uniform direction of oscillations in females. The EMG analysis showed a significant main effect for muscle and body side. The results showed that the eye position modulated body sway without changing the activity of principal leg postural muscles, suggesting that the extraretinal input regarding the eye position is a crucial signal that needs to be integrated with perceptual optic flow processing in order to control body sway.

A dataset of stereoscopic images and ground-truth disparity mimicking human fixations in peripersonal space

Binocular stereopsis is the ability of a visual system, belonging to a live being or a machine, to interpret the different visual information deriving from two eyes/cameras for depth perception. From this perspective, the ground-truth information about three-dimensional visual space, which is hardly available, is an ideal tool both for evaluating human performance and for benchmarking machine vision algorithms. In the present work, we implemented a rendering methodology in which the camera pose mimics realistic eye pose for a fixating observer, thus including convergent eye geometry and cyclotorsion. The virtual environment we developed relies on highly accurate 3D virtual models, and its full controllability allows us to obtain the stereoscopic pairs together with the ground-truth depth and camera pose information. We thus created a stereoscopic dataset: GENUA PESTO-GENoa hUman Active fixation database: PEripersonal space STereoscopic images and grOund truth disparity. The dataset aims to provide a unified framework useful for a number of problems relevant to human and computer vision, from scene exploration and eye movement studies to 3D scene reconstruction.

The Active Side of Stereopsis: Fixation Strategy and Adaptation to Natural Environments

Depth perception in near viewing strongly relies on the interpretation of binocular retinal disparity to obtain stereopsis. Statistical regularities of retinal disparities have been claimed to greatly impact on the neural mechanisms that underlie binocular vision, both to facilitate perceptual decisions and to reduce computational load. In this paper, we designed a novel and unconventional approach in order to assess the role of fixation strategy in conditioning the statistics of retinal disparity. We integrated accurate realistic three-dimensional models of natural scenes with binocular eye movement recording, to obtain accurate ground-truth statistics of retinal disparity experienced by a subject in near viewing. Our results evidence how the organization of human binocular visual system is finely adapted to the disparity statistics characterizing actual fixations, thus revealing a novel role of the active fixation strategy over the binocular visual functionality. This suggests an ecological explanation for the intrinsic preference of stereopsis for a close central object surrounded by a far background, as an early binocular aspect of the figure-ground segregation process.

Three-dimensional ocular kinematics underlying binocular single vision

We have analyzed the binocular coordination of the eyes during far-to-near refixation saccades based on the evaluation of distance ratios and angular directions of the projected target images relative to the eyes' rotation centers. By defining the geometric point of binocular single vision, called Helmholtz point, we found that disparities during fixations of targets at near distances were limited in the subject's three-dimensional visual field to the vertical and forward directions. These disparities collapsed to simple vertical disparities in the projective binocular image plane. Subjects were able to perfectly fuse the vertically disparate target images with respect to the projected Helmholtz point of single binocular vision, independent of the particular location relative to the horizontal plane of regard. Target image fusion was achieved by binocular torsion combined with corrective modulations of the differential half-vergence angles of the eyes in the horizontal plane. Our findings support the notion that oculomotor control combines vergence in the horizontal plane of regard with active torsion in the frontal plane to achieve fusion of the dichoptic binocular target images.

The natural statistics of blur

Blur from defocus can be both useful and detrimental for visual perception: It can be useful as a source of depth information and detrimental because it degrades image quality. We examined these aspects of blur by measuring the natural statistics of defocus blur across the visual field. Participants wore an eye-and-scene tracker that measured gaze direction, pupil diameter, and scene distances as they performed everyday tasks. We found that blur magnitude increases with increasing eccentricity. There is a vertical gradient in the distances that generate defocus blur: Blur below the fovea is generally due to scene points nearer than fixation; blur above the fovea is mostly due to points farther than fixation. There is no systematic horizontal gradient. Large blurs are generally caused by points farther rather than nearer than fixation. Consistent with the statistics, participants in a perceptual experiment perceived vertical blur gradients as slanted top-back whereas horizontal gradients were perceived equally as left-back and right-back. The tendency for people to see sharp as near and blurred as far is also consistent with the observed statistics. We calculated how many observations will be perceived as unsharp and found that perceptible blur is rare. Finally, we found that eye shape in ground-dwelling animals conforms to that required to put likely distances in best focus.

Balance and coordination after viewing stereoscopic 3D television

Manufacturers and the media have raised the possibility that viewing stereoscopic 3D television (S3D TV) may cause temporary disruption to balance and visuomotor coordination. We looked for evidence of such effects in a laboratory-based study. Four hundred and thirty-three people aged 4-82 years old carried out tests of balance and coordination before and after viewing an 80 min movie in either conventional 2D or stereoscopic 3D, while wearing two triaxial accelerometers. Accelerometry produced little evidence of any change in body motion associated with S3D TV. We found no evidence that viewing the movie in S3D causes a detectable impairment in balance or in visuomotor coordination.

How our body influences our perception of the world

Incorporating the fact that the senses are embodied is necessary for an organism to interpret sensory information. Before a unified perception of the world can be formed, sensory signals must be processed with reference to body representation. The various attributes of the body such as shape, proportion, posture, and movement can be both derived from the various sensory systems and can affect perception of the world (including the body itself). In this review we examine the relationships between sensory and motor information, body representations, and perceptions of the world and the body. We provide several examples of how the body affects perception (including but not limited to body perception). First we show that body orientation effects visual distance perception and object orientation. Also, visual-auditory crossmodal-correspondences depend on the orientation of the body: audio "high" frequencies correspond to a visual "up" defined by both gravity and body coordinates. Next, we show that perceived locations of touch is affected by the orientation of the head and eyes on the body, suggesting a visual component to coding body locations. Additionally, the reference-frame used for coding touch locations seems to depend on whether gaze is static or moved relative to the body during the tactile task. The perceived attributes of the body such as body size, affect tactile perception even at the level of detection thresholds and two-point discrimination. Next, long-range tactile masking provides clues to the posture of the body in a canonical body schema. Finally, ownership of seen body parts depends on the orientation and perspective of the body part in view. Together, all of these findings demonstrate how sensory and motor information, body representations, and perceptions (of the body and the world) are interdependent.

Stereopsis is adaptive for the natural environment

Humans and many animals have forward-facing eyes providing different views of the environment. Precise depth estimates can be derived from the resulting binocular disparities, but determining which parts of the two retinal images correspond to one another is computationally challenging. To aid the computation, the visual system focuses the search on a small range of disparities. We asked whether the disparities encountered in the natural environment match that range. We did this by simultaneously measuring binocular eye position and three-dimensional scene geometry during natural tasks. The natural distribution of disparities is indeed matched to the smaller range of correspondence search. Furthermore, the distribution explains the perception of some ambiguous stereograms. Finally, disparity preferences of macaque cortical neurons are consistent with the natural distribution.

The Effect of Stimulus Size on Stereoscopic Fusion Limits and Response Criteria

The stereoscopic fusion limit denotes the largest binocular disparity for which a single fused image is perceived. Several criteria can be employed when judging whether or not a stereoscopic display is fused, and this may be a factor contributing to a discrepancy in the literature. Schor, Wood, and Ogawa (1984 Vision Research, 24, 661–665) reported that fusion limits did not change as a function of bar width, while Roumes, Plantier, Menu, and Thorpe (1997 Human Factors, 39, 359–373) reported higher fusion limits for larger stimuli than for smaller stimuli. Our investigation suggests that differing criteria between the studies could contribute to this discrepancy. In experiment 1 we measured horizontal and vertical disparity fusion limits for thin bars and for the edge of an extended surface, allowing observers to use the criterion of either diplopia or rivalry when evaluating fusion for all stimuli. Fusion limits were equal for thin bars and extended surfaces in both horizontal and vertical disparity conditions. We next measured fusion limits for a range of bar widths and instructed observers to indicate which criterion they employed on each trial. Fusion limits were constant across all stimulus widths. However, there was a sharp change in criterion from diplopia to rivalry when the angular extent of the bar width exceeded about twice the fusion limit, expressed in angular terms. We conclude that stereoscopic fusion limits do not depend on stimulus size in this context, but the criterion for fusion does. Therefore, the criterion for fusion should be clearly defined in any study measuring stereoscopic fusion limits.

Foveational complexity in single word identification: contralateral visual pathways are advantaged over ipsilateral pathways

Recognition of a single word is an elemental task in innumerable cognitive psychology experiments, but involves unexpected complexity. We test a controversial claim that the human fovea is vertically divided, with each half projecting to either the contralateral or ipsilateral hemisphere, thereby influencing foveal word recognition. We report a novel haploscope task: the two halves of a four-letter word were briefly presented to the two eyes in a Both condition (st|ep)(st|ep), a Contralateral condition (st|__)(__|ep), or an Ipsilateral condition (__|ep)(st|__), all yielding the same single word percept (step). The Both condition yielded superior perceptual recognition, followed by the contralateral projection, then the ipsilateral projection. These results demonstrate that the structure of the fovea influences even the recognition of short, foveally presented words. Projecting different parts of the same word to different hemispheres involves unforeseen complexities and opportunities for optimizing hemispheric coordination.

Super deep 3D images from a 3D omnifocus video camera

When using stereographic image pairs to create three-dimensional (3D) images, a deep depth of field in the original scene enhances the depth perception in the 3D image. The omnifocus video camera has no depth of field limitations and produces images that are in focus throughout. By installing an attachment on the omnifocus video camera, real-time super deep stereoscopic pairs of video images were obtained. The deeper depth of field creates a larger perspective image shift, which makes greater demands on the binocular fusion of human vision. A means of reducing the perspective shift without harming the depth of field was found.

Perception of Relative Depth Interval: Systematic Biases in Perceived Depth

Given an estimate of the binocular disparity between a pair of points and an estimate of the viewing distance, or knowledge of eye position, it should be possible to obtain an estimate of their depth separation. Here we show that, when points are arranged in different vertical geometric configurations across two intervals, many observers find this task difficult. Those who can do the task tend to perceive the depth interval in one configuration as very different from depth in the other configuration. We explore two plausible explanations for this effect. The first is the tilt of the empirical vertical horopter: Points perceived along an apparently vertical line correspond to a physical line of points tilted backwards in space. Second, the eyes can rotate in response to a particular stimulus. Without compensation for this rotation, biases in depth perception would result. We measured cyclovergence indirectly, using a standard psychophysical task, while observers viewed our depth configuration. Biases predicted from error due either to cyclovergence or to the tilted vertical horopter were not consistent with the depth configuration results. Our data suggest that, even for the simplest scenes, we do not have ready access to metric depth from binocular disparity.

Horizontal/Vertical Differences in Range and Upper/Lower Visual Field Differences in the Midpoints of Sensory Fusion Limits of Oriented Lines

O'Shea and Crassini (1982, Perception & Psychophysics32 195–196) demonstrated that fusion persists for vertical lines with an orientation disparity of 8°, but diplopia is experienced in simultaneously presented horizontal lines with the same disparity. They concluded that the neural fusion process fuses larger horizontal disparities than vertical disparities. Kertesz criticised their demonstration because it did not quantify the possible motor component associated with fusing their counter-rotated images. Krekling and Blika argued that the demonstrated anisotropy is due to a disparity bias in the visual system, owing to the temporalward tilt of corresponding vertical meridians. We addressed these criticisms with a novel stimulus and presentation protocol, that rendered compensatory cyclovergence eye movements unlikely and explored a wide range of orientation disparities. We confirmed O'Shea and Crassini's vertical/horizontal anisotropy in orientation fusion limits. In addition, our measurements of vertical lines showed that the distributions of fused responses as a function of orientation disparity in the upper and lower visual fields were shifted relative to each other. Therefore, the distributions of fusible orientation disparities are wider for vertical lines than horizontal lines and are relatively shifted as predicted if the fusional range is centred around the vertical horopter.

On the inverse problem of binocular 3D motion perception

It is shown that existing processing schemes of 3D motion perception such as interocular velocity difference, changing disparity over time, as well as joint encoding of motion and disparity, do not offer a general solution to the inverse optics problem of local binocular 3D motion. Instead we suggest that local velocity constraints in combination with binocular disparity and other depth cues provide a more flexible framework for the solution of the inverse problem. In the context of the aperture problem we derive predictions from two plausible default strategies: (1) the vector normal prefers slow motion in 3D whereas (2) the cyclopean average is based on slow motion in 2D. Predicting perceived motion directions for ambiguous line motion provides an opportunity to distinguish between these strategies of 3D motion processing. Our theoretical results suggest that velocity constraints and disparity from feature tracking are needed to solve the inverse problem of 3D motion perception. It seems plausible that motion and disparity input is processed in parallel and integrated late in the visual processing hierarchy.

Humans and many other predators have two eyes that are set a short distance apart so that an extensive region of the world is seen simultaneously by both eyes from slightly different points of view. Although the images of the world are essentially two-dimensional, we vividly see the world as three-dimensional. This is true for static as well as dynamic images. Here we elaborate on how the visual system may establish 3D motion perception from local input in the left and right eye. Using tools from analytic geometry we show that existing 3D motion models offer no general solution to the inverse optics problem of 3D motion perception. We suggest a flexible framework of motion and depth processing and suggest default strategies for local 3D motion estimation. Our results on the aperture and inverse problem of 3D motion are likely to stimulate computational, behavioral, and neuroscientific studies because they address the fundamental issue of how 3D motion is represented in the visual system.

Stereo correspondence is optimized for large viewing distances

Stereo '3D' vision depends on correctly matching up the differing images of objects seen by our two eyes. But vertical disparity between the retinal images changes with binocular eye posture, reflecting for example the different convergence angles required for different viewing distances. Thus, stereo correspondence must either dynamically adapt to take account of changes in viewing distance, or be hard-wired to perform best at one particular viewing distance. Here, using psychophysical experiments, we show for the first time that human stereo correspondence does not adapt to changes in physical viewing distance. We examine performance on a stereo correspondence task at a short viewing distance (30 cm) and show that performance is improved when we simulate the disparity pattern for viewing infinity, even though these disparities are impossible at the physical viewing distance. We estimate the vertical extent of the retinally fixed 'search zones' as < 0.6° at 14° eccentricity, suggesting that most V1 neurons must be tuned to near-zero vertical disparity. We also show that performance on our stereo task at 14° eccentricity is affected by the pattern of vertical disparity beyond 20° eccentricity, even though this is irrelevant to the task. Performance is best when vertical disparities within and beyond 20° eccentricity both indicate the same convergence angle (even if not the physical angle), than when the pattern of vertical disparity across the visual field is globally inconsistent with any single convergence angle. This novel effect of the periphery may indicate cooperative interactions between disparity-selective neurons activated by the same eye postures.

Neuromuscular plasticity and rehabilitation of the ocular near response

The near response is composed of cross-coupled interactions between convergence and other distance-related oculomotor responses including accommodation, vertical vergence, and cyclovergence. The cross-coupling interactions are analogous to the body postural reflexes that maintain balance. Near-response couplings guide involuntary motor responses during voluntary shifts of distance and direction of gaze without feedback from defocus or retinal-image disparity. They optimize the disparity stimulus for stereoscopic depth perception and can be modified by optically induced sensory demands placed on binocular vision. In natural viewing conditions, the near response is determined by passive orbital mechanics and active-adaptable tonic components. For example, the normal coupling of vertical vergence with convergence in tertiary gaze is partly a byproduct of passive orbital mechanics. Both, adapted changes of vertical vergence in response to anisophoria, produced by unequal ocular magnification (aniseikonia), and adapted changes in the orientation of Listing's plane in response to torsional disparities can be achieved by a combination of passive orbital mechanics and neural adjustments for the control of the vertical vergence and cyclovergence. Adaptive adjustments are coupled with gaze direction, convergence angle, and head tilt. Several adaptation studies suggest that it is possible to achieve non-linear changes in the coupling of both vertical vergence and cyclovergence with gaze direction. This coupling can be achieved with changes in neural control signals of ocular elevator muscles that are cross-coupled with both convergence and direction of tertiary gaze. These linear and non-linear coupling interactions can be adapted to compensate for (1) anisophoria induced by spectacle corrections for anisometropia, (2) accommodative esotropia, (3) convergence excess and insufficiency, and (4) non-concomitant deviations with ocular torticollis associated with trochlear palsy. The adaptable near-response couplings form the basis of an area of orthoptics that optimizes visual performance by facilitating our natural ability to calibrate neural pathways underlying binocular postural reflexes.