Pooling of vertical disparities by the human visual system

What's special about horizontal disparity

Horizontal disparity has been recognized as the primary signal driving stereoscopic depth since the invention of the stereoscope in the 1830s. It has a unique status in our understanding of binocular vision. The direction of offset of the eyes gives the disparities of corresponding image point locations across the two retinas a strong horizontal bias. Beyond the retina, other factors give shape to the effective disparity direction used by visual mechanisms. The influence of orientation is examined here. I argue that horizontal disparity is an inflection point along a continuum of effective directions, and its role in stereo vision can be reinterpreted. The pointwise geometric justification for its special status neglects the oriented structural elements of spatial vision, its physiological support is equivocal, and psychophysical support of its special status may partially reflect biased stimulus sampling. The literature shows that horizontal disparity plays no particular role in the processing of one-dimensional stimuli, a reflection of the stereo aperture problem. The resulting depth is non-veridical, even non-transitive. Although one-dimensional components contribute to the stereo depth of visual objects generally, two-dimensional stimuli appear not to inherit the aperture problem. However, a look at the two-dimensional stimuli that predominate in experimental studies shows regularities in orientation that give a new perspective on horizontal disparity.

Attentional selection in judgments of stereo depth

Stereoscopic depth is most useful when it comes from relative rather than absolute disparities. However, the depth perceived from relative disparities can vary with stimulus parameters that have no connection with depth or are irrelevant to the task. We investigated observers' ability to judge the stereo depth of task-relevant stimuli while ignoring irrelevant stimuli. The calculation of depth from disparity differs for 1-D and 2-D stimuli and we investigated the role this difference plays in observers' ability to selectively process relevant information. We show that the presence of irrelevant disparities affects perceived depth differently depending on stimulus dimensionality. Observers could not ignore disparities of irrelevant stimuli when they judged the relative depth between a 1-D stimulus (a grating) and a 2-D stimulus (a plaid). Yet these irrelevant disparities did not affect judgments of the relative depth between 2-D stimuli. Two processes contributing to stereo depth were identified, only one of which computes depth from a horizontal disparity metric and permits attentional selection. The other uses all stimuli, relevant and irrelevant, to calculate an effective disparity direction for comparing disparity magnitudes. These processes produce inseparable effects in most data sets. Using multiple disparity directions and comparing 1-D and 2-D stimuli can distinguish them.

Vertical Disparity Affects Shape and Size Judgments across Surfaces Separated in Depth

Vertical binocular disparity provides a useful source of information allowing three-dimensional (3-D) shape to be recovered from horizontal binocular disparity. In order to influence metric shape judgments, a large field of view is required, suggesting that vertical disparity may play a limited role in the perception of objects projecting small retinal images. This limitation could be overcome if vertical disparity information could be pooled over wide areas of 3-D space. This was investigated by assessing the effect of vertical disparity scaling of a large surround surface on the perceived size and 3-D shape of a small, central object. Observers adjusted the size and shape of a virtual, binocularly defined ellipsoid to match those of a real, hand-held tennis ball. The virtual ball was presented at three distances (200, 325, and 450 mm). Vertical disparities in a large surround surface were manipulated to be consistent with a distance of 160 mm or infinity. Both shape and size settings were influenced by this manipulation. This effect did not depend on presenting the surround and target objects at the same distance. These results suggest that the influence of vertical disparity on the perceived distance to a surface also affects the estimated distance of other visible surfaces. Vertical disparities are therefore important in the perception of metric depth, even for objects that in themselves subtend only small retinal images.

Latitude and longitude vertical disparities

The literature on vertical disparity is complicated by the fact that several different definitions of the term "vertical disparity" are in common use, often without a clear statement about which is intended or a widespread appreciation of the properties of the different definitions. Here, we examine two definitions of retinal vertical disparity: elevation-latitude and elevation-longitude disparities. Near the fixation point, these definitions become equivalent, but in general, they have quite different dependences on object distance and binocular eye posture, which have not previously been spelt out. We present analytical approximations for each type of vertical disparity, valid for more general conditions than previous derivations in the literature: we do not restrict ourselves to objects near the fixation point or near the plane of regard, and we allow for non-zero torsion, cyclovergence, and vertical misalignments of the eyes. We use these expressions to derive estimates of the latitude and longitude vertical disparities expected at each point in the visual field, averaged over all natural viewing. Finally, we present analytical expressions showing how binocular eye position-gaze direction, convergence, torsion, cyclovergence, and vertical misalignment-can be derived from the vertical disparity field and its derivatives at the fovea.

Vertical-size disparities are temporally integrated for slant perception

We investigated temporal properties of vertical-size and horizontal-size disparity processing for slant perception. Subjects indicated perceived slants for a stereoscopic stimulus in which the two magnitudes of vertical-size or horizontal-size disparities were oscillated stepwise with various frequencies (from 0.2 to 10 Hz). For the stimulus with vertical-size disparity oscillation, two slants corresponding to the two magnitudes of disparity were perceived for low-frequency conditions, whereas only a static mean slant of the two slants was perceived for high frequencies (5 and 10 Hz). For the stimulus with horizontal-size disparity oscillation, two slants were perceived for all the temporal frequency conditions. These results indicate that temporal properties of vertical- and horizontal-size disparity processing are clearly different and vertical-size disparities are temporally integrated over a period of around 500 ms for slant perception.

Depth aftereffects mediated by vertical disparities: Evidence for vertical disparity driven calibration of extraretinal signals during stereopsis

Perceptual adaptation often results in a repulsive aftereffect: stimuli are seen as biased away from the adaptation stimulus (). Here we report the absence of a repulsive aftereffect for a vertical gradient of vertical disparity (or vertical size ratio, VSR). We exposed observers to a binocular stimulus consisting of horizontal lines. This stimulus contains vertical, but not horizontal disparities. The visual system was able to measure the VSR of this stimulus: although the lines themselves always appeared unslanted, the VSR carried by the lines had a dramatic effect on the apparent slant of a horizontal row of dots, as predicted by recent accounts of Ogle's (1938) induced effect (e.g., Backus, Banks, van Ee, & Crowell, 1999). Yet we observed no repulsive aftereffect for the VSR signal: after adaptation to horizontal lines that were vertically larger in one eye, we found an attractive aftereffect, the magnitude of which was largest in stimuli that did not contain a VSR signal. We interpret these results as a case of recalibration: disagreement between extra-retinal eye position signals (EP) and VSR causes a recalibration in the use of EP as used in the stereoscopic perception of slant.

Vertical-disparity gradients are processed independently in different depth planes

We examined the effects of vertical-disparity gradients on apparent depth curvature of textured surfaces. In Experiment 1, vertical disparities induced expected curvatures when the surface had a horizontal disparity of < +/-40.34'. A central row of elements, lacking vertical disparities, ceased to have the same apparent curvature as the surface when the horizontal disparity between row and surface exceeded +/-5'. In Experiment 2, vertical disparities were not pooled between superimposed surfaces separated by horizontal disparities > +/-10'. Thus, vertical-disparity gradients are not pooled over depth for curvature perception. Our results suggest that vertical disparities are used to determine distances to surfaces directly, rather than to estimate vergence.

Perceptual compensation for eye torsion

To correctly perceive visual directions relative to the head, one needs to compensate for the eye's orientation in the head. In this study we focus on compensation for the eye's torsion regarding objects that contain the line of sight and objects that do not pass through the fixation point. Subjects judged the location of flashed probe points relative to their binocular plane of regard, the mid-sagittal or the transverse plane of the head, while fixating straight ahead, right upward, or right downward at 30 cm distance, to evoke eye torsion according to Listing's law. In addition, we investigated the effects of head-tilt and monocular versus binocular viewing. Flashed probe points were correctly localized in the plane of regard irrespective of eccentric viewing, head-tilt, and monocular or binocular vision in nearly all subjects and conditions. Thus, eye torsion that varied by +/-9 degrees across these different conditions was in general compensated for. However, the position of probes relative to the midsagittal or the transverse plane, both true head-fixed planes, was misjudged. We conclude that judgment of the orientation of the plane of regard, a plane that contains the line of sight, is veridical, indicating accurate compensation for actual eye torsion. However, when judgment has to be made of a head-fixed plane that is offset with respect to the line of sight, eye torsion that accompanies that eye orientation appears not to be taken into account correctly.

Localization of the plane of regard in space

When we fixate an object in space, the rotation centers of the eyes, together with the object, define a plane of regard. People perceive the elevation of objects relative to this plane accurately, irrespective of eye or head orientation (Poljac et al. (2004) Vision Res, in press). Yet, to create a correct representation of objects in space, the orientation of the plane of regard in space is required. Subjects pointed along an eccentric vertical line on a touch screen to the location where their plane of regard intersected the touch screen positioned on their right. The distance of the vertical line to the subject's eyes varied from 10 to 40 cm. Subjects were sitting upright and fixating one of the nine randomly presented directions ranging from 20 degrees left and down to 20 degrees right and up relative to their straight ahead. The eccentricity of fixations relative to the pointing location varied by up to 40 degrees . Subjects underestimated the elevation of their plane of regard (on average by 3.69 cm, SD=1.44 cm), regardless of the fixation direction or pointing distance. However, when the targets were shown on a display mounted in a table, to provide support of the subject's hand throughout the trial, subjects pointed accurately (average error 0.3 cm, SD=0.8 cm). In addition, head tilt 20 degrees to the left or right did not cause any change in accuracy. The bias observed in the first task could be caused by maintained tonus in arm muscles when the arm is raised, that might interfere with the transformation from visual to motor signals needed to perform the pointing movement. We conclude that the plane of regard is correctly localized in space. This may be a good starting point for representing objects in head-centric coordinates.

Adaptation to vertical disparity induced-depth: implications for disparity processing

Depth aftereffects produced by prolonged inspection of an object in depth can be mediated by monocular and binocular depth cues. The adapting mechanisms responsible for such effects have not yet been fully determined. Theories of binocular depth aftereffects typically posit a role of an adaptive horizontal disparity sensitive mechanism, implying multiple cue-specific mechanisms for depth aftereffects. Here we examined whether binocular depth aftereffects can be attributed to such a cue-specific mechanism. In Experiment 1 we did so using a technique allowing us to maintain horizontal disparities and vergence constant for our adaptation stimuli, whilst manipulating simulated depth by virtue of a vertical disparity induced-depth effect. We found that depth aftereffects were almost identical to those produced by adaptation to stimuli of equivalent depth produced by conventional horizontal disparity modulations. In Experiment 2, we examined depth aftereffects following adaptation to apparently frontal surfaces produced by different combinations of horizontal and vertical disparity modulations. Aftereffects were close to zero. These results suggest that binocular depth aftereffects are not due to adaptation of a horizontal disparity sensitive mechanism, and we argue that adaptation occurs at the level of a 3D shape sensitive mechanism derived from multiple cues. Experiment 3 was a control to examine whether the two types of adaptation stimuli in Experiment 1 were indeed perceptually the same, since in theory they may differ if vertical disparities influenced metric depth scaling. We found no evidence of this, and concluded that the two classes of stimuli used in Experiment 1, though consisting of very different patterns of disparity, were perceptually equivalent.

Effects of horizontal and vertical additive disparity noise on stereoscopic corrugation detection

Stereoscopic corrugation detection in the presence of horizontal- and vertical- additive disparity noise was examined using a signal detection paradigm. Random-dot stereograms either represented a 3-D square-wave surface with various amounts of Gaussian-distributed additive disparity noise or had the same disparity values randomly redistributed. Stereoscopic detection of 2 arcmin peak amplitude corrugations was found to tolerate significantly greater amplitudes of vertical-disparity noise than horizontal-disparity noise--irrespective of whether the corrugations were horizontally or vertically oriented. However, this directional difference in tolerance to disparity noise was found to reverse when the corrugation and noise amplitudes were increased (so as to produce equivalent signal-to-noise ratios). These results suggest that horizontal- and vertical-disparity noise pose different problems for dot-matching and post-matching surface reconstruction as corrugation and noise amplitudes increase.

Adaptation to disparity but not to perceived depth

The purpose of the present study was to investigate whether adaptation can occur to disparity per se. The adapting stimuli were large random-dot patterns of which the two half-images were transformed such that the depth effects induced by the vertical transformations were nulled by horizontal transformations. Thus, the adapting stimuli were perceptually the same, whereas the disparity fields differed from each other. The adapting stimuli were presented for five minutes. During that period, the percept of a fronto-parallel surface did not change. After the adapting period, subjects perceived a thin untransformed strip as either slanted or curved depending on the adapting transformation. The thin strips provided negligible information about the vertical disparity field. In a forced-choice task we measured the amount of horizontal transformation that was required to null the acquired adaptation. We found that the amounts of horizontal transformation required to perceive the test strip fronto-parallel were significantly different from zero. We conclude that the visual system can adapt to disparity signals in the absence of a perceptual drive.

Strength of depth effects induced by three types of vertical disparity

The goal of the present study is to compare the strengths of depth effects induced by different types of vertical disparity. We use a nulling task, in which the depth effects induced by vertical disparity are nulled by horizontal disparity. The advantage of this method is that it prevents cue conflicts from arising between disparity and other depth cues. The ratios between horizontal and vertical disparity that evoke the percept of a fronto-parallel stimulus vary per type of vertical disparity. The ratios determined for vertical scale and vertical quadratic mix (vertical scale with a horizontal gradient) vary strongly across subjects. The ratios for vertical shear are constant, since all subjects needed the same amount of horizontal and vertical shear to perceive a fronto-parallel plane. In these experiments, one conflict cannot be avoided, namely the conflict between vertical disparity and oculomotor signals. This conflict may cause differential weighting of vertical disparity and oculomotor signals, which could explain the individual differences. The different ratios for different types of vertical disparity suggest that weighting is specific for each type of vertical disparity and the associated oculomotor signal.

How vertical disparities assist judgements of distance

The ratio of the vertical sizes of corresponding features in the two eyes' retinal images depends both on the associated object's distance and on its horizontal direction relative to the head (eccentricity). It is known that manipulations of vertical size ratio can affect perceived distance, size, depth and shape. We examined how observers use the vertical size ratio to determine the viewing distance. Do they use the horizontal gradient of vertical size ratio, or do they combine the vertical size ratio itself with the eccentricity at which it is found? Distance scaling (as measured by having subjects set an ellipsoid's size and shape to match a tennis ball) was no better when the judged object was 30 degrees to the right of the head (where vertical size ratios vary considerably with distance) than when it was located straight ahead. Distance scaling improved when vertical disparities were presented within larger visual fields, irrespective of where this was relative to the head. Our results support the proposal that subjects use the horizontal gradient of vertical size ratio to estimate the distance of an object that they are looking at.

An orientation anisotropy in the effects of scaling vertical disparities

Gårding et al. (Vis Res 1995;35:703-722) proposed a two-stage theory of stereopsis. The first uses horizontal disparities for relief computations after they have been subjected to a process called disparity correction that utilises vertical disparities. The second stage, termed disparity normalisation, is concerned with computing metric representations from the output of stage one. It uses vertical disparities to a much lesser extent, if at all, for small field stimuli. We report two psychophysical experiments that tested whether human vision implements this two-stage theory. They tested the prediction that scaling vertical disparities to simulate different viewing distances to the fixation point should affect the perceived amplitudes of vertically but not horizontally oriented ridges. The first used elliptical half-cylinders and the 'apparently circular cylinder' judgement task of Johnston (Vis Res 1991;31:1351-1360). The second experiment used parabolic ridges and the amplitude judgement task of Buckley and Frisby (Vis Res 1993;33:919-934). Both studies broadly confirmed the anisotropy prediction by finding that large scalings of vertical disparities simulating near distances had a strong effect on the perceived amplitudes of the vertically oriented stimuli but little effect on the horizontal ones. When distances > 25 cm were simulated there were no significant differential effects and various methodological reasons are offered for this departure from expectations.

Robust and optimal use of information in stereo vision

Differences between the left and right eye's views of the world carry information about three-dimensional scene structure and about the position of the eyes in the head. The contemporary Bayesian approach to perception implies that human performance in using this source of eye-position information can be analysed most usefully by comparison with the performance of a statistically optimal observer. Here we argue that the comparison observer should also be statistically robust, and we find that this requirement leads to qualitatively new behaviours. For example, when presented with a class of stereoscopic stimuli containing inconsistent information about eccentricity of gaze, estimates of this gaze parameter recorded from one robust ideal observer bifurcate at a critical value of stimulus inconsistency. We report an experiment in which human observers also show this phenomenon and we use the experimentally determined critical value to estimate the vertical acuity of the visual system. The Bayesian analysis also provides a highly reliable and biologically plausible algorithm that can recover eye positions even before the classic stereo-correspondence problem is solved, that is, before deciding which features in the left and right images are to be matched.

Types of size disparity and the perception of surface slant

We examined (i) perceived slant of a textured surface about a vertical axis as a function of disparity magnitude for horizontal-size disparity, vertical-size disparity, and overall-size disparity; and (ii) interactions between patterns with various types and magnitudes of size disparity and superimposed or adjacent zero-disparity stimuli. Horizontal-size disparity produced slant which increased with increasing disparity, was enhanced by superimposed zero-disparity stimuli, and induced contrasting slant in superimposed or adjacent zero-disparity stimuli. Vertical-size disparity produced opposite slant (induced effect) which was reduced to near zero by a superimposed zero-disparity pattern and both patterns appeared as one surface. Adjacent vertical-size-disparity and zero-disparity patterns appeared as separate surfaces with a wide curved boundary. Overall-size disparity produced slant which was enhanced by a superimposed zero-disparity pattern and less so by a zero-disparity line, and induced more slant in a zero-disparity line than in a zero-disparity pattern. The results are discussed in terms of depth underestimation of isolated surfaces, depth enhancement, depth contrast, and the processing of deformation disparity.

Neural mechanisms underlying stereoscopic vision

The progressive frontalization of both eyes in mammals causes overlap of the left and right visual fields, having as a consequence a region of binocular field with single vision and stereopsis. The horizontal separation of the eyes makes the retinal images of the objects lying in this binocular field have slight horizontal and vertical differences, termed disparities. Horizontal disparities are the main cue for stereopsis. In the past decades numerous physiological studies made on monkeys, which have in many aspects a similar visual system to humans, showed that a population of visual cells are capable of encoding the amplitude and sign of horizontal disparity. Such disparity detectors were found in cortical visual areas V1, V2, V3, V3A, VP, MT (V5) and MST of monkeys and in the superior colliculus of the cat and opossum. According to their disparity tuning function, these cells were first grouped into tuned excitatory, tuned inhibitory, near and far sub-groups. Subsequent studies added two more categories, tuned near and tuned far cells. Asymmetries between left and right receptive field position, on and off regions, and intra-receptive field wiring are believed to be the neural mechanisms of disparity detection. Because horizontal disparity alone is insufficient to compute reliable stereopsis, additional information about fixation distance and angle of gaze is required. Thus, while there is unequivocal evidence of cells capable of detecting horizontal disparities, it is not known how horizontal disparity is calibrated. Sensitivity to vertical disparity and information about the vergence angle or eye position may be the source of this additional information.

The area of spatial integration for initial horizontal disparity vergence

We investigated over what central area disparity in a random dot stereogram is integrated to stimulate an initial vergence response. Vergence was measured subjectively, with a forced choice dichoptic nonius vernier task following a brief (230 msec) stimulus presentation. Stimuli were random-dot stereograms showing a central circular disc of 12.5 min arc crossed retinal disparity in front of, and occluding, a same density fixation plane surround. The size of the disc was varied. All ten observers responded to the brief stimulus. Initial vergence increased with increasing disc diameter and, for nine out of ten subjects, reached a maximum with the disc ca 6 deg, suggesting this is the extent of the spatial integration region. Below 6 deg diameter, surround and target disparities were averaged together. Initial horizontal vergence responds automatically to a cyclopean target presented in the centre of gaze by pooling disparities within a limited but surprisingly large area.

Spatial limitation of vertical-size disparity processing

We investigated the upper limit of horizontal spatial modulation of vertical-size disparity in a textured surface for the perception of depth. In Experiment 1 subjects matched the appearance of a surface with modulated horizontal-size disparity to that of a surface with modulated vertical-size disparity. In Experiment 2 we determined the threshold amplitude of modulation of vertical-size disparity required for the perception of depth as a function of the spatial frequency of disparity modulation. The results indicate that sensations of depth are not elicited by modulations of vertical-size disparity of any amplitude at spatial frequencies higher than about 0.04 c/deg. We conclude that vertical disparities are averaged within about 20 deg-wide areas and suggest that this global measurement is used to scale local horizontal disparities for the perception of surface slant.