Depth estimation from retinal disparity requires eye and head orientation signals

To reach for an object, one needs to know its egocentric distance (absolute depth). It remains an unresolved issue which signals are required by the brain to calculate this absolute depth information. We devised a geometric model of binocular 3D eye orientation and investigated the signals necessary to uniquely determine the depth of a non-foveated object accounting for naturalistic variations of eye and head orientations. Our model shows that, in the presence of noisy internal estimates of the ocular vergence angle, horizontal and vertical retinal disparities alone are insufficient to calculate the unique depth of a point-like target. Instead the brain must account for the 3D orientations of the eye and head. We tested the model in a behavioral experiment that involved reaches to targets in depth. Our analysis showed that a target with the same retinal disparity produced different estimates of reach depth that varied consistently with different eye and head orientations. The experimental results showed that subjects accurately account for this extraretinal information when they reach. In summary, when estimating the distance of point-like targets, all available signals about the object's location as well as body configuration are combined to provide accurate information about the object's distance.

The surface of the empirical horopter

The distribution of empirical corresponding points in the two retinas has been well studied along the horizontal and the vertical meridians, but not in other parts of the visual field. Using an apparent-motion paradigm, we measured the positions of those points across the central portion of the visual field. We found that the Hering-Hillebrand deviation (a deviation from the Vieth-Müller circle) and the Helmholtz shear of horizontal disparity (backward slant of the vertical horopter) exist throughout the visual field. We also found no evidence for non-zero vertical disparities in empirical corresponding points. We used the data to find the combination of points in space and binocular eye position that minimizes the disparity between stimulated points on the retinas and the empirical corresponding points. The optimum surface is a top-back slanted surface at medium to far distance depending on the observer. The line in the middle of the surface extending away from the observer comes very close to lying in the plane of the ground as the observer fixates various positions in the ground, a speculation Helmholtz made that has since been misunderstood.

The extended horopter: Quantifying retinal correspondence across changes of 3D eye position

The theoretical horopter is an interesting qualitative tool for conceptualizing binocular correspondence, but its quantitative applications have been limited because they have ignored ocular kinematics and vertical binocular sensory fusion. Here we extend the mathematical definition of the horopter to a full surface over visual space, and we use this extended horopter to quantify binocular alignment and visualize its dependence on eye position. We reproduce the deformation of the theoretical horopter into a spiral shape in tertiary gaze as first described by Helmholtz (1867). We also describe a new effect of ocular torsion, where the Vieth-Müller circle rotates out of the visual plane for symmetric vergence conditions in elevated or depressed gaze. We demonstrate how these deformations are reduced or abolished when the eyes follow the modification of Listing's law during convergence called L2, which enlarges the extended horopter and keeps its location and shape constant across gaze directions.

Influence of eye position on stereo matching

In animals with binocular depth vision, or stereopsis, the visual fields of the two eyes overlap, shrinking the overall field of view. Eye movements increase the field of view, but they also complicate the first stage of stereopsis: the search for corresponding images on the two retinas. If the eyes were stationary in the head, corresponding images would always lie on retina-fixed bands called epipolar lines. Because the eyes rotate, the epipolar lines move on the retinas. Therefore, the stereoptic system has a choice: it may monitor eye position to keep track of the epipolar lines, or it may give up on tracking epipolar lines and instead search for matches over retina-fixed regions, but in that case the search regions must be 2-D patches, large enough to encompass all possible locations of the epipolar lines in all usual eye positions. We use a new type of random-dot stereogram to show that human stereopsis uses large, retina-fixed search zones. We show that the brain somewhat reduces the size of these search zones by rotating the eyes about their lines of sight in a way that reduces the motion of the epipolar lines. These findings show the link between sensory and motor processes: by considering eye motion we can understand why the brain searches for matching images over 2-D retinal regions rather than along epipolar lines; and by considering retinal correspondence we appreciate why the eyes rotate as they do about their lines of sight.