The interpretation of stereo-disparity information: the computation of surface orientation and depth

What's special about horizontal disparity

Horizontal disparity has been recognized as the primary signal driving stereoscopic depth since the invention of the stereoscope in the 1830s. It has a unique status in our understanding of binocular vision. The direction of offset of the eyes gives the disparities of corresponding image point locations across the two retinas a strong horizontal bias. Beyond the retina, other factors give shape to the effective disparity direction used by visual mechanisms. The influence of orientation is examined here. I argue that horizontal disparity is an inflection point along a continuum of effective directions, and its role in stereo vision can be reinterpreted. The pointwise geometric justification for its special status neglects the oriented structural elements of spatial vision, its physiological support is equivocal, and psychophysical support of its special status may partially reflect biased stimulus sampling. The literature shows that horizontal disparity plays no particular role in the processing of one-dimensional stimuli, a reflection of the stereo aperture problem. The resulting depth is non-veridical, even non-transitive. Although one-dimensional components contribute to the stereo depth of visual objects generally, two-dimensional stimuli appear not to inherit the aperture problem. However, a look at the two-dimensional stimuli that predominate in experimental studies shows regularities in orientation that give a new perspective on horizontal disparity.

Does Vergence Affect Perceived Size?

Since Kepler (1604) and Descartes (1637), it has been suggested that 'vergence' (the angular rotation of the eyes) plays a key role in size constancy. However, this has never been tested divorced from confounding cues such as changes in the retinal image. In our experiment, participants viewed a target which grew or shrank in size over 5 s. At the same time, the fixation distance specified by vergence was reduced from 50 to 25 cm. The question was whether this change in vergence affected the participants' judgements of whether the target grew or shrank in size? We found no evidence of any effect, and therefore no evidence that eye movements affect perceived size. If this is correct, then our finding has three implications. First, perceived size is much more reliant on cognitive influences than previously thought. This is consistent with the argument that visual scale is purely cognitive in nature (Linton, 2017; 2018). Second, it leads us to question whether the vergence modulation of V1 contributes to size constancy. Third, given the interaction between vergence, proprioception, and the retinal image in the Taylor illusion, it leads us to ask whether this cognitive approach could also be applied to multisensory integration.

Contributions of Stereopsis and Aviation Experience to Simulated Rotary Wing Altitude Estimation

OBJECTIVE

We examined the contribution of binocular vision and experience to performance on a simulated helicopter flight task.

BACKGROUND

Although there is a long history of research on the role of binocular vision and stereopsis in aviation, there is no consensus on its operational relevance. This work addresses this using a naturalistic task in a virtual environment.

METHOD

Four high-resolution stereoscopic terrain types were viewed monocularly and binocularly. In separate experiments, we evaluated performance of undergraduate students and military aircrew on a simulated low hover altitude judgment task. Observers were asked to judge the distance between a virtual helicopter skid and the ground plane.

RESULTS

Our results show that for both groups, altitude judgments are more accurate in the binocular viewing condition than in the monocular condition. However, in the monocular condition, aircrew were more accurate than undergraduate observers in estimating height of the skid above the ground.

CONCLUSION

At simulated altitudes of 5 ft (1.5 m) or less, binocular vision provides a significant advantage for estimation of the depth separation between the landing skid and the ground, regardless of relevant operational experience. However, when binocular cues are unavailable aircrew outperform undergraduate observers, a result that likely reflects the impact of training on the ability to interpret monocular depth cues.

Development and verification of a snapshot dental intraoral three-dimensional scanner based on chromatic confocal imaging

We describe the development and verification of an optical, powder-free, intraoral scanner based on a chromatic confocal imaging system, which has been realized in a single-shot multifocal approach. The system is based on a combination of micro-optical and dispersion optical elements. The methodology of recording and analyzing the acquired data are discussed in detail. A proof of concept with the application in intraoral scanning is provided. According to the current findings, the measurement uncertainty, scan speed, and overall performance of the device can well compete with the state-of-the-art of commercially available intraoral scanners.

Stereo-vision: head-centric coding of retinal signals

Stereo-vision is generally considered to provide information about depth in a visual scene derived from disparities in the positions of an image on the two eyes; a new study has found evidence that retinal-image coding relative to the head is also important.

Latitude and longitude vertical disparities

The literature on vertical disparity is complicated by the fact that several different definitions of the term "vertical disparity" are in common use, often without a clear statement about which is intended or a widespread appreciation of the properties of the different definitions. Here, we examine two definitions of retinal vertical disparity: elevation-latitude and elevation-longitude disparities. Near the fixation point, these definitions become equivalent, but in general, they have quite different dependences on object distance and binocular eye posture, which have not previously been spelt out. We present analytical approximations for each type of vertical disparity, valid for more general conditions than previous derivations in the literature: we do not restrict ourselves to objects near the fixation point or near the plane of regard, and we allow for non-zero torsion, cyclovergence, and vertical misalignments of the eyes. We use these expressions to derive estimates of the latitude and longitude vertical disparities expected at each point in the visual field, averaged over all natural viewing. Finally, we present analytical expressions showing how binocular eye position-gaze direction, convergence, torsion, cyclovergence, and vertical misalignment-can be derived from the vertical disparity field and its derivatives at the fovea.

Cyclopean geometry of binocular vision

The geometry of binocular projection is analyzed in relation to the primate visual system. An oculomotor parameterization that includes the classical vergence and version angles is defined. It is shown that the epipolar geometry of the system is constrained by binocular coordination of the eyes. A local model of the scene is adopted in which depth is measured relative to a plane containing the fixation point. These constructions lead to an explicit parameterization of the binocular disparity field involving the gaze angles as well as the scene structure. The representation of visual direction and depth is discussed with reference to the relevant psychophysical and neurophysiological literature.

Effect of vertical disparities on depth representation in macaque monkeys: MT physiology and behavior

Horizontal binocular disparities provide information about the distance of objects relative to the point of ocular fixation and must be combined with an estimate of viewing distance to recover the egocentric distance of an object. Vergence angle and the gradient of vertical disparities across the visual field are thought to provide independent sources of viewing distance information based on human behavioral studies. Although the effect of vergence angle on horizontal disparity selectivity in early visual cortex has been examined (with mixed results), the effect of the vertical disparity field has not been explored. We manipulated the vertical disparities in a large random-dot stimulus to simulate different viewing distances, and we examined the effect of this manipulation on both the responses of neurons in the middle temporal (MT) area and on the psychophysical performance of the animal in a curvature discrimination task. We report here that alterations to the vertical disparity field have no effect on the horizontal disparity tuning of MT neurons. However, the same manipulation strongly and systematically biases the monkey's judgments of curvature, consistent with previous human studies. We conclude that monkeys, like humans, make use of the vertical disparity field to estimate viewing distance, but that the physiological mechanisms for this effect occur either downstream of MT or in a different pathway.

Direct extraction of curvature-based metric shape from stereo by view-modulated receptive fields

Any computation of metric surface structure from horizontal disparities depends on the viewing geometry, and analysing this dependence allows us to narrow down the choice of viable schemes. For example, all depth-based or slant-based schemes (i.e. nearly all existing models) are found to be unrealistically sensitive to natural errors in vergence. Curvature-based schemes avoid these problems and require only moderate, more robust view-dependent corrections to yield local object shape, without any depth coding. This fits the fact that humans are strikingly insensitive to global depth but accurate in discriminating surface curvature. The latter also excludes coding only affine structure. In view of new adaptation results, our goal becomes to directly extract retinotopic fields of metric surface curvatures (i.e. avoiding intermediate disparity curvature). To find a robust neural realisation, we combine new exact analysis with basic neural and psychophysical constraints. Systematic, step-by-step 'design' leads to neural operators which employ a novel family of 'dynamic' receptive fields (RFs), tuned to specific (bi-)local disparity structure. The required RF family is dictated by the non-Euclidean geometry that we identify as inherent in cyclopean vision. The dynamic RF-subfield patterns are controlled via gain modulation by binocular vergence and version, and parameterised by a cell-specific tuning to slant. Our full characterisation of the neural operators invites a range of new neurophysiological tests. Regarding shape perception, the model inverts widely accepted interpretations: It predicts the various types of errors that have often been mistaken for evidence against metric shape extraction.

Vertical-disparity gradients are processed independently in different depth planes

We examined the effects of vertical-disparity gradients on apparent depth curvature of textured surfaces. In Experiment 1, vertical disparities induced expected curvatures when the surface had a horizontal disparity of < +/-40.34'. A central row of elements, lacking vertical disparities, ceased to have the same apparent curvature as the surface when the horizontal disparity between row and surface exceeded +/-5'. In Experiment 2, vertical disparities were not pooled between superimposed surfaces separated by horizontal disparities > +/-10'. Thus, vertical-disparity gradients are not pooled over depth for curvature perception. Our results suggest that vertical disparities are used to determine distances to surfaces directly, rather than to estimate vergence.

A physiological theory of depth perception from vertical disparity

It has been known since the time of Helmholtz that vertical differences between the two retinal images can generate depth perception. Although many ecologically and geometrically inspired theories have been proposed, the neural mechanisms underlying the phenomenon remain elusive. Here we propose a new theory for depth perception from vertical disparity based on the oriented binocular receptive fields of visual cortical cells and on the radial bias of the preferred-orientation distribution in the cortex. The theory suggests that oriented cells may treat a vertical disparity as a weaker, equivalent horizontal disparity. It explains the induced effect, and the quadrant and size dependence of vertical disparity. It predicts that horizontal and vertical disparities should locally enhance or cancel each other according to their depth signs, and that the effect of vertical disparity should be orientation dependent. These predictions were confirmed through psychophysical experiments.

Seeing in three dimensions: the neurophysiology of stereopsis

From the pair of 2-D images formed on the retinas, the brain is capable of synthesizing a rich 3-D representation of our visual surroundings. The horizontal separation of the two eyes gives rise to small positional differences, called binocular disparities, between corresponding features in the two retinal images. These disparities provide a powerful source of information about 3-D scene structure, and alone are sufficient for depth perception. How do visual cortical areas of the brain extract and process these small retinal disparities, and how is this information transformed into non-retinal coordinates useful for guiding action? Although neurons selective for binocular disparity have been found in several visual areas, the brain circuits that give rise to stereoscopic vision are not very well understood. I review recent electrophysiological studies that address four issues: the encoding of disparity at the first stages of binocular processing, the organization of disparity-selective neurons into topographic maps, the contributions of specific visual areas to different stereoscopic tasks, and the integration of binocular disparity and viewing-distance information to yield egocentric distance. Some of these studies combine traditional electrophysiology with psychophysical and computational approaches, and this convergence promises substantial future gains in our understanding of stereoscopic vision.

Horizontal and vertical disparity, eye position, and stereoscopic slant perception

The slant of a stereoscopically defined surface cannot be determined solely from horizontal disparities or from derived quantities such as horizontal size ratio (HSR). There are four other signals that, in combination with horizontal disparity, could in principle allow an unambiguous estimate of slant: the vergence and version of the eyes, the vertical size ratio (VSR), and the horizontal gradient of VSR. Another useful signal is provided by perspective slant cues. The determination of perceived slant can be modeled as a weighted combination of three estimates based on those signals: a perspective estimate, a stereoscopic estimate based on HSR and VSR, and a stereoscopic estimate based on HSR and sensed eye position. In a series of experiments, we examined human observers' use of the two stereoscopic means of estimation. Perspective cues were rendered uninformative. We found that VSR and sensed eye position are both used to interpret the measured horizontal disparities. When the two are placed in conflict, the visual system usually gives more weight to VSR. However, when VSR is made difficult to measure by using short stimuli or stimuli composed of vertical lines, the visual system relies on sensed eye position. A model in which the observer's slant estimate is a weighted average of the slant estimate based on HSR and VSR and the one based on HSR and eye position accounted well for the data. The weights varied across viewing conditions because the informativeness of the signals they employ vary from one situation to another.

An orientation anisotropy in the effects of scaling vertical disparities

Gårding et al. (Vis Res 1995;35:703-722) proposed a two-stage theory of stereopsis. The first uses horizontal disparities for relief computations after they have been subjected to a process called disparity correction that utilises vertical disparities. The second stage, termed disparity normalisation, is concerned with computing metric representations from the output of stage one. It uses vertical disparities to a much lesser extent, if at all, for small field stimuli. We report two psychophysical experiments that tested whether human vision implements this two-stage theory. They tested the prediction that scaling vertical disparities to simulate different viewing distances to the fixation point should affect the perceived amplitudes of vertically but not horizontally oriented ridges. The first used elliptical half-cylinders and the 'apparently circular cylinder' judgement task of Johnston (Vis Res 1991;31:1351-1360). The second experiment used parabolic ridges and the amplitude judgement task of Buckley and Frisby (Vis Res 1993;33:919-934). Both studies broadly confirmed the anisotropy prediction by finding that large scalings of vertical disparities simulating near distances had a strong effect on the perceived amplitudes of the vertically oriented stimuli but little effect on the horizontal ones. When distances > 25 cm were simulated there were no significant differential effects and various methodological reasons are offered for this departure from expectations.

Robust and optimal use of information in stereo vision

Differences between the left and right eye's views of the world carry information about three-dimensional scene structure and about the position of the eyes in the head. The contemporary Bayesian approach to perception implies that human performance in using this source of eye-position information can be analysed most usefully by comparison with the performance of a statistically optimal observer. Here we argue that the comparison observer should also be statistically robust, and we find that this requirement leads to qualitatively new behaviours. For example, when presented with a class of stereoscopic stimuli containing inconsistent information about eccentricity of gaze, estimates of this gaze parameter recorded from one robust ideal observer bifurcate at a critical value of stimulus inconsistency. We report an experiment in which human observers also show this phenomenon and we use the experimentally determined critical value to estimate the vertical acuity of the visual system. The Bayesian analysis also provides a highly reliable and biologically plausible algorithm that can recover eye positions even before the classic stereo-correspondence problem is solved, that is, before deciding which features in the left and right images are to be matched.

A computational model of depth perception based on headcentric disparity

It is now well established that depth is coded by local horizontal disparity and global vertical disparity. We present a computational model which explains how depth is extracted from these two types of disparities. The model uses the two (one for each eye) headcentric directions of binocular targets, derived from retinal signals and oculomotor signals. Headcentric disparity is defined as the difference between headcentric directions of corresponding features in the left and right eye's images. Using Helmholtz's coordinate systems we decompose headcentric disparity into azimuthal and elevational disparity. Elevational disparities of real objects are zero if the signals which contribute to headcentric disparity do not contain any errors. Azimuthal headcentric disparity is a 1D quantity from which an exact equation relating distance and disparity can be derived. The equation is valid for all headcentric directions and for all binocular fixation positions. Such an equation does not exist if disparity is expressed in retinal coordinates. Possible types of errors in oculomotor signals (six) produce global elevational disparity fields which are characterised by different gradients in the azimuthal and elevational directions. Computations show that the elevational disparity fields uniquely characterise both the type and size of the errors in oculomotor signals. Our model uses a measure of the global elevational disparity field together with local azimuthal disparity to accurately derive headcentric distance throughout the visual field. The model explains existing data on whole-field disparity transformations as well as hitherto unexplained aspects of stereoscopic depth perception.

Neural mechanisms underlying stereoscopic vision

The progressive frontalization of both eyes in mammals causes overlap of the left and right visual fields, having as a consequence a region of binocular field with single vision and stereopsis. The horizontal separation of the eyes makes the retinal images of the objects lying in this binocular field have slight horizontal and vertical differences, termed disparities. Horizontal disparities are the main cue for stereopsis. In the past decades numerous physiological studies made on monkeys, which have in many aspects a similar visual system to humans, showed that a population of visual cells are capable of encoding the amplitude and sign of horizontal disparity. Such disparity detectors were found in cortical visual areas V1, V2, V3, V3A, VP, MT (V5) and MST of monkeys and in the superior colliculus of the cat and opossum. According to their disparity tuning function, these cells were first grouped into tuned excitatory, tuned inhibitory, near and far sub-groups. Subsequent studies added two more categories, tuned near and tuned far cells. Asymmetries between left and right receptive field position, on and off regions, and intra-receptive field wiring are believed to be the neural mechanisms of disparity detection. Because horizontal disparity alone is insufficient to compute reliable stereopsis, additional information about fixation distance and angle of gaze is required. Thus, while there is unequivocal evidence of cells capable of detecting horizontal disparities, it is not known how horizontal disparity is calibrated. Sensitivity to vertical disparity and information about the vergence angle or eye position may be the source of this additional information.

Extra-retinal and perspective cues cause the small range of the induced effect

With a horizontal magnifier before one eye, a frontoparallel surface appears rotated about a vertical axis (geometric effect). With a vertical magnifier, apparent rotation is opposite in direction (induced effect); to restore appearance of frontoparallelism, the surface must be rotated away from the magnified eye. The induced effect is interesting because it was thought until recently that vertical disparities do not play an important role in surface perception. As with the geometric effect, the required rotation for the induced effect increases linearly to approximately equal to 4% magnification; unlike the geometric effect, it plateaus at approximately 8%. Current theory explains the linear portion: vertical size ratios (VSRs) are used to compensate for changes in horizontal size ratios (HSRs) that accompany eccentric gaze, so changes in VSR cause changes in perceived slant. The theory does not explain the plateau. We demonstrate that it results from differing slant estimates obtained by use of various retinal and extra-retinal signals. When perspective cues to slant are minimized or sensed eye position is consistent with VSR, the induced and geometric effects have similar magnitudes even at large magnifications.

Stereoscopic depth perception and vertical disparity: neural mechanisms

The additivity assumption relates to the various stereo-disparity components in the vertical and horizontal meridians, each of which is assumed to be independent of the other, with the total disparity in each dimension being the linear sum of the separate components. Information about the position of the eyes provided by the corollary discharge leads to compensatory changes in the lateral geniculate nuclei whereby the angle of gaze disparity component at retinal level is offset by equal and opposite changes at geniculate level. These geniculate changes concern only eye position. Changes in the retinal images such as those produced by lenses (i.e. induced effect) are passed on to the cortex without modification at the geniculate level. Discrimination of the local depth disparity component can be achieved by subtracting the local vertical eccentricity component from the total horizontal disparity.

Can random-dot stereograms serve as a model for the perception of depth in relation to real three-dimensional objects?

The ability to perceive depth in a random-dot stereogram is a valuable test for the perception of retinal image disparities, whether they arise from the viewing of a stereogram or from the viewing of a real 3-D object. However, a stereogram cannot be regarded as a proper model for the perception of depth in the case of a real 3-D object. This conclusion comes out most clearly in relation to changes in viewing distance. Whereas the viewing of real objects and stereograms both obey the rules of size constancy, this is not the case with depth constancy. With changes in viewing distance, the viewing of real objects obeys the rules of depth constancy. By contrast, the magnitude of the depth intervals in a stereogram are not constant but appear to increase in direct proportion to the increase in viewing distance. In a stereogram these changes in the amplitude of the depth intervals are based on the same mechanisms as those responsible for size constancy.

Interaction of stereo and texture cues in the perception of three-dimensional steps

A computational method for calibrating stereo using shape-from-texture is described together with five experiments that tested whether the human visual system implements the method. The experiments all tested the prediction that the perceived size of a step between two planar and slanted real surfaces should be affected by texture slant cues projected on to them that are inconsistent with the disparity cues. The predicted effect was observed but the results could be accounted for by a new phenomenon revealed in control conditions: the perceived size of a step between two slanted planes is in part determined by the size of the slants even when texture and stereo cues are held consistent. We conclude that the hypothesis that human stereo is calibrated by texture is not confirmed.

Stereopsis, vertical disparity and relief transformations

The pattern of retinal binocular disparities acquired by a fixating visual system depends on both the depth structure of the scene and the viewing geometry. This paper treats the problem of interpreting the disparity pattern in terms of scene structure without relying on estimates of fixation position from eye movement control and proprioception mechanisms. We propose a sequential decomposition of this interpretation process into disparity correction, which is used to compute three-dimensional structure up to a relief transformation, and disparity normalization, which is used to resolve the relief ambiguity to obtain metric structure. We point out that the disparity normalization stage can often be omitted, since relief transformations preserve important properties such as depth ordering and coplanarity. Based on this framework we analyse three previously proposed computational models of disparity processing; the Mayhew and Longuet-Higgins model, the deformation model and the polar angle disparity model. We show how these models are related, and argue that none of them can account satisfactorily for available psychophysical data. We therefore propose an alternative model, regional disparity correction. Using this model we derive predictions for a number of experiments based on vertical disparity manipulations, and compare them to available experimental data. The paper is concluded with a summary and a discussion of the possible architectures and mechanisms underling stereopsis in the human visual system.

Cortical representation of visual three-dimensional space

Perception of real depth includes information on stereopsis and distance. How both interact in the visual pathway was the subject of a study performed on the behaving monkey. Neurons in the primary visual cortex (area V1) have their activity, visual and/or spontaneous, modulated by the viewing distance. Disparity selectivity may be present or better expressed at a given viewing distance. This modulation is independent of the visual pattern. The use of prisms shows that vergence is implicated in this phenomenon. Consequently, extraretinal signals related to ocular motility have access to area V1. Among them, proprioceptive signals from the eye muscles have been shown to be involved in visual cortical function and in the development of depth perception. It is possible that the same signals may also be involved in the distance modulation shown in V1 neurons, but this remains to be examined. A possible specialisation of disparity-selective cells in different cortical areas is discussed.

Integration of stereo, texture, and outline cues during pinhole viewing of real ridge-shaped objects and stereograms of ridges

Three experiments are reported in which the possible role of blur cues as a factor needing to be taken into account in cue-integration studies involving conflicts between stereo and texture/outline cues was investigated. The earlier suggestion was tested that uncontrolled blur cues might have caused the quite different patterns of cue integration reported for real ridge-shaped objects oriented vertically and for stereograms depicting similar surfaces. Blur cues were manipulated by pinhole viewing intended to render accommodation open loop. The results for real ridges were as predicted by the blur-cue hypothesis: pinhole viewing strengthened texture/outline cues in vertically oriented ridges, thereby diminishing the pattern of stereo dominance hitherto observed for these stimuli (and as observed here in non-pinhole-viewing control conditions and in horizontally oriented ridges). The results for the stereograms did not conform to predictions: pinhole viewing, assumed to remove blur cues from the cue-integration process, still produced the pattern observed in control conditions in which a texture/outline cue for a shallow ridge overwhelmed stereo cues for a steep ridge. This result is against the hypothesis that perhaps blur cues for the stereogram projection surface differentially favoured the shallow texture/outline cues. A new variant of the blur-cue hypothesis is offered to account for this result. The main conclusion from the study is: beware drawing firm conclusions from stereograms about the pattern of cue integration that can be expected when real objects are being viewed. The two situations can produce very different results as far as cue integration is concerned. This is a conclusion with serious implications for the use of stereograms for studying the integration of stereo with other cues.

Isovergence surfaces: the conjugacy of vertical eye movements in tertiary positions of gaze

Conjugate gaze is often defined as the equal angle rotation of the two eyes. For fixation at far distances, the optical axes are parallel and conjugacy is defined irrespective of the coordinate system. For nearby or finite fixation distances, the evaluation of conjugacy for many gaze postures depends on the coordinate system used to measure it. For example, if the eye is elevated or depressed and the eye is rotated about a vertical axis, the intersections of lines of sight with a tangent screen will describe either straight lines or arcs depending on whether the vertical axis is fixed with respect to the head or to the eye. Because of the horizontal separation of the two eyes, the binocular fixation of near targets at tertiary positions of gaze will require a vertical vergence component for head-referenced but not eye-referenced measurements. The vertical gaze alignment of three human subjects was measured as they viewed targets placed at secondary and tertiary eye positions at two different distances. Vertical vergence was either held open or closed-loop. The lines of sight were found to intersect (i.e. vertical gaze was aligned) regardless of target position or viewing condition.

A polar coordinate system for describing binocular disparity

When a meridional magnifier is introduced in front of one eye, a planar surface is perceived as slanted about a vertical axis. If the horizontal meridian is magnified, the perceived slant is away from the eye with the magnifier (geometric effect). If the vertical meridian is magnified, the slant is towards the eye with the magnifier (induced effect). While the geometric effect can be explained by the binocular horizontal disparities introduced by the horizontal magnifier, the induced effect has to be explained differently. Various models have been developed and the induced effect has generally been explained as a reinterpretation of horizontal disparity under a new reference of stereoscopic localization which is resultant from the vertical positional disparity introduced by the vertical magnifier. In this paper we describe binocular disparity in a polar coordinate system. Under this system, horizontal and vertical disparities are combined into a single stimulus variable, polar angle disparity. We show that the spatial distribution of polar angle disparity can faithfully describe the three-dimensional slant and inclination of a planar surface relative to the gaze normal plane. Both geometric and induced effects can be explained as direct responses to the polar angle disparity map distorted by the magnifier. Theoretical predictions based on the polar angle disparity are compared with experimental findings.

Size constancy, depth constancy and vertical disparities: a further quantitative interpretation

The size and depth constancies considered here operate only at near distances (< about 2 m) in a static stimulus situation with vergence as the only cue to distance. The innervation of the extraocular muscles, as evidenced by the corollary discharge, provides information about the vergence of the eyes and hence about the egocentric distance both for symmetrical and asymmetrical vergences. Size and depth constancies are regarded as the first and second stages of a linked two-stage process. In the lateral geniculate nuclei compensatory adjustments are separately applied to each retinal image as they are received from the two eyes. The modified ocular images, with their associated vertical and horizontal disparities, now provide synaptic inputs to binocularly activated cells in the visual cortex. Then, by a process akin to the induced effect, cortical cells with geniculate afferents with vertical disparities will have their outputs expressed in terms of horizontal disparities. The horizontal disparity outputs of these cortical cells are then further multiplied by the outputs from cortical cells with geniculate afferents with horizontal disparities. It is this second multiplicative process that brings about the quadratic relationship between horizontal retinal disparity and egocentric distance. The proposed mechanisms involve the known ability of the visual system to detect and respond to vertical as well as horizontal disparities and provide a definite role for the induced effect in the perceptual process. The above neural model is based on fairly simple equations that give a remarkably adequate description of the operation of the two constancies.

Cell responses to vertical and horizontal retinal disparities in the monkey visual cortex

Because of the horizontal separation of both ocular globes, the projection angles are slightly different. These differences are commonly termed retinal disparities. Vertical and horizontal retinal disparities occur constantly in normal life. We have investigated the responses of single cells in cortical areas V1 and V2 of behaving Macaca mulatta monkeys to retinal disparities by using dynamic random dot stereograms. Our findings show that cortical visual cells are sensitive to both vertical and horizontal disparities. To calculate the distance between two objects in a three-dimensional space from horizontal disparities, it is necessary to know the fixation distance. It has been suggested that the horizontal gradient of vertical disparity contains information to estimate the fixation distance and therefore to scale horizontal disparities. We suggest that these cells sensitive to horizontal and vertical disparities represent a neural mechanism that provides information to the visual system in order to achieve a correct eye alignment and depth perception.

Vertical disparities, differential perspective and binocular stereopsis

To calculate the depth difference between a pair of points on a three-dimensional surface from binocular disparities, it is necessary to know the absolute distance to the surface. Traditionally, it has been assumed that this information is derived from non-visual sources such as the vergence angle of the eyes. It has been shown that the horizontal gradient of vertical disparity between the images in the two eyes also contains information about the fixation distance. Recent results, however indicated that manipulations of the vertical disparity gradient have no effect on either the perceived shape or the perceived depth of surfaces defined by horizontal disparities. Following the reasoning of Longuet-Higgins and Tyler, we suggest that vertical disparities are best understood as a consequence of perspective viewing from two different vantage points and the results we report here show that the human visual system is able to exploit vertical disparities and use them to scale the perceived depth and size of stereoscopic surfaces, if the field of view is sufficiently large.

Gaze angle explanations of the induced effect

Mayhew and Longuet-Higgins formulated a computational explanation of the induced effect which successfully predicts the conditions under which the induced effect will occur. Underlying their theories are the assumptions that disparity information is separated into horizontal and vertical components and that the vertical disparities are used to calculate the gaze angles. An implementation of the fusional explanation introduced by Petrov makes similar predictions for the induced effect, but does not depend on these two assumptions.

Corneal lens goggles and visual space perception

Night vision goggles are head-mounted, unity-power systems designed to allow the human operator to see and operate at night. Field experience and experimental studies have revealed many drawbacks in conventional designs that impair performance. One major drawback is the poor space perception provided by the goggles. The Hadani et al. [J. Opt. Soc. Am. 70, 60-65 (1980)] model for space perception attributes this drawback to the fact that the conventional designs shift the observer's effective center of perspective approximately 15 cm ahead and also predicts the resulting impairments. An innovative redesign is presented in this paper-the corneal lens goggles (CLG)-which brings the effective center of perspective of the goggles to coincide with the center of perspective of the eyes, thus annulling the optical length of the device. Qualitative and quantitative laboratory studies have compared the performance of the CLG and conventional goggles (type AN/PVS-5). These studies have revealed better visual and visual-motor performance with the CLG. The implications to optical design of the Hadani et al. theory and the CLG concept are discussed.

Does vertical disparity scale the perception of stereoscopic depth?

It has been suggested that a measure of the gradients of vertical disparity over a surface may scale the mapping between horizontal disparity and perceived depth. We have investigated this possibility by obtaining estimates of the depth within stereograms that simulated two apposed fronto-parallel planes placed at different distances from an observer. The gradients of vertical disparity in a stereogram were set to simulate those appropriate to a viewing distance of 12.5 cm, 25 cm, 50 cm or 100 cm, whereas the distance specified by vergence and accommodative cues was always fixed at 50 cm. Judgements of the perceived depth between the two planes were uninfluenced by changes in the gradients of vertical disparity. It thus seems that the human visual system does not employ vertical disparity as a scaling parameter in stereoscopic depth judgements.

Vertical disparities and perception of three-dimensional shape

The information about depth and three-dimensional shape available from the horizontal component of the stereo disparity field requires interpretation in conjunction with information about egocentric viewing distance (D). A novel computational approach for estimating D was proposed by Mayhew and Longuet-Higgins, who demonstrated that the horizontal gradient of vertical disparities uniquely specifies the viewing distance. We have now used random dot stereograms in a shape judgement task to show that changes in vertical disparities have no effect on perceived three-dimensional shape. Changes in ocular convergence do alter perceived shape, suggesting substantial changes in the subjects' scaling of horizontal disparities. We conclude that vertical disparities are not used to scale disparities for viewing distance, and that extraretinal signals must be considered when analysing human three-dimensional shape perception.

The analogy between stereo depth and brightness

Apparent depth in stereograms exhibits various simultaneous-contrast and induction effects analogous to those reported in the luminance domain. This behavior suggests that stereo depth, like brightness, is reconstructed, ie recovered from higher-order spatial derivatives or differences of the original signal. The extent to which depth is analogous to brightness is examined. There are similarities in terms of contrast effects but dissimilarities in terms of the lateral inhibition effects traditionally attributed to underlying spatial-differentiation operators.

Vertical disparity, egocentric distance and stereoscopic depth constancy: a new interpretation

There has long been a problem concerning the presence in the visual cortex of binocularly activated cells that are selective for vertical stimulus disparities because it is generally believed that only horizontal disparities contribute to stereoscopic depth perception. The accepted view is that stereoscopic depth estimates are only relative to the fixation point and that independent information from an extraretinal source is needed to scale for absolute or egocentric distance. Recently, however, theoretical computations have shown that egocentric distance can be estimated directly from vertical disparities without recourse to extraretinal sources. There has been little impetus to follow up these computations with experimental observations, because the vertical disparities that normally occur between the images in the two eyes have always been regarded as being too small to be of significance for visual perception and because experiments have consistently shown that our conscious appreciation of egocentric distance is rather crude and unreliable. Nevertheless, the veridicality of stereoscopic depth constancy indicates that accurate distance information is available to the visual system and that the information about egocentric distance and horizontal disparity are processed together so as to continually recalibrate the horizontal disparity values for different absolute distances. Computations show that the recalibration can be based directly on vertical disparities without the need for any intervening estimates of absolute distance. This may partly explain the relative crudity of our conscious appreciation of egocentric distance. From published data it has been possible to calculate the magnitude of the vertical disparities that the human visual system must be able to discriminate in order for depth constancy to have the observed level of veridicality. From published data on the induced effect it has also been possible to calculate the threshold values for the detection of vertical disparities by the visual system. These threshold values are smaller than those needed to provide for the recalibration of the horizontal disparities in the interests of veridical depth constancy. An outline is given of the known properties of the binocularly activated cells in the striate cortex that are able to discriminate and assess the vertical disparities. Experiments are proposed that should validate, or otherwise, the concepts put forward in this paper.

Integrating stereopsis with monocular interpretations of planar surfaces

Experiments are reported that involved spatial judgments of planar surfaces that had contradictory stereo and monocular information. Tasks included comparing the relative depths of two points on the depicted surface and judging the surface's apparent spatial orientation. It was found that for planar surfaces the 3D perception was dominated by the monocular interpretation, despite the strongly contradictory stereo information. We propose that stereo information is effectively integrated only where the surface exhibits curvature features or edge discontinuities, i.e. where the second spatial derivatives of disparity are nonzero. Planar surfaces induce constant gradients of disparity and are thus effectively featureless to stereopsis. Further observations are reported regarding nonplanar surfaces, where contradictory monocular information can still be effectively rivalrous with that suggested stereoscopically.

Line correspondence in binocular vision

The mathematical analysis of binocular vision introduced by Helmholtz is applied to the problem of the use of disparity information to position a stimulus in depth. It is shown that matching the images from the left and right eyes along radial directions is an alternative to matching images along the horizontal direction only.

End-stopped cells and binocular depth discrimination in the striate cortex of cats

Proposals concerning neural mechanisms for binocular depth discrimination have been criticized on the grounds that only striate cells with a preferred stimulus orientation not too far from the vertical can make significant horizontal disparity discriminations. We investigated this claim by preparing a two-dimensional array of position-disparity response profiles to moving light and dark bars from each of 18 cells in the simple family. From these arrays, it was possible to reconstruct disparity response profiles along any axis across the receptive field, irrespective of the cell's optimal stimulus orientation. This analysis showed that cells with a predominantly excitatory binocular response (N = 10) can make precise horizontal disparity discriminations, independent of their optimal stimulus orientation, provided that they are sufficiently end stopped. End-free cells, on the other hand, are effective for horizontal disparity discriminations only if their preferred orientation are near the vertical. Nearly all striate cells we examined were end-stopped to some degree and nearly half had an end inhibition sufficient to reduce the monocular response from the dominant eye to half its maximal amplitude. Cells having a predominantly inhibitory disparity response profile of the symmetric type (N = 8) have an inhibitory profile along every axis across the receptive field. An outline is given of a neural mechanism for the determination of absolute viewing distance based on the sensitivities of striate cells to vertical retinal-image disparities.

Monocular aniseikonia: a motion parallax analogue of the disparity-induced effect

Mayhew and Longuet-Higgins have recently outlined a computational model of binocular depth perception in which the small vertical disparities between the two eyes' views of a three-dimensional scene are used to determine the 'viewing parameters' of fixation distance (d) and the angle of asymmetric convergence of the eyes (g). The d/g hypothesis, as it has been called, correctly predicts that a fronto-parallel surface, viewed with a vertically magnifying lens over one eye, should appear to be rotated in depth about a vertical axis. We report here a comparable illusion for surfaces specified by monocular motion parallax information, which can be explained more simply by considering the differential invariants of the optic flow field. In addition, our observations suggest that the disparity-induced effect is not a 'whole field' phenomenon nor one limited to small magnification differences between the eyes.

Computational experiments with a feature based stereo algorithm

Computational models of the human stereo system can provide insight into general information processing constraints that apply to any stereo system, either artificial or biological. In 1977 Marr and Poggio proposed one such computational model, which was characterized as matching certain feature points in difference-of-Gaussian filtered images and using the information obtained by matching coarser resolution representations to restrict the search space for matching finer resolution representations. An implementation of the algorithm and its testing on a range of images was reported in 1980. Since then a number of psychophysical experiments have suggested possible refinements to the model and modifications to the algorithm. As well, recent computational experiments applying the algorithm to a variety of natural images, especially aerial photographs, have led to a number of modifications. In this paper, we present a version of the Marr-Poggio-Grimson algorithm that embodies these modifications, and we illustrate its performance on a series of natural images.

Physical limits of acuity and hyperacuity

An ideal detector is derived for the discrimination of arbitrary stimuli in the two-alternative forced-choice paradigm. The ideal detector's performance is assumed to be limited only by quantal fluctuations, the optics of the eye, and the size and spacing of the receptors in the retinal mosaic. Detailed predictions are presented for two-point acuity and hyperacuity tasks. The ideal detector's two-point resolution, over a wide range of luminances, is approximately 10 times worse than its two-point vernier acuity or separation discrimination. Furthermore, two-point resolution is shown to vary in proportion to the -1/4 power of spot intensity, but vernier acuity and separation discrimination vary in proportion to the -1/2 power of spot intensity. It is shown that this ideal detector can be implemented by the use of appropriately shaped receptive fields. The derivation provides a simple way to determine the shapes of these optimal receptive fields for arbitrary stimuli. The sensitivities of real (human) and ideal detectors are compared.