Combining binocular and monocular curvature features

Shape judgments in natural scenes: Convexity biases versus stereopsis

Determining the relief of upcoming terrain is critical to locomotion over rough or uneven ground. Given the significant contribution of stereopsis to perceived surface shape, it should play a crucial role in determining the shape of ground surfaces. The aim of this series of experiments was to evaluate the relative contribution of monocular and binocular depth cues to judgments of ground relief. To accomplish this goal, we simulated a depth discrimination task using naturalistic imagery. Stimuli consisted of a stereoscopically rendered grassy terrain with a central mound or a dip with varying height. We measured thresholds for discrimination of the direction of the depth offset. To determine the relationship between relief discrimination and measures of stereopsis, we used two stereoacuity tasks performed under the same viewing conditions. To assess the impact of ambiguous two-dimensional shading cues on depth judgments in our terrain task, we manipulated the intensity of the shading (low and high). Our results show that observers reliably discriminated ground reliefs as small as 20 cm at a viewing distance of 9.1 m. As the shading was intensified, a large proportion of observers (30%) exhibited a strong convexity bias, even when stereopsis indicated a concave depression. This finding suggests that there are significant individual differences in the reliance on assumptions of surface curvature that must be considered in experimental conditions. In impoverished viewing environments with limiting depth cues, these convexity biases could persist in judgments of ground relief, especially when shading cues are highly salient.

Virtual slant explains perceived slant, distortion, and motion in pictorial scenes

Virtual slants of obliquely viewed figures consisting of skewed grids, computed as a function of depicted slant and slant of the physical surface, were compared with perceived slants in both monocular and binocular viewing conditions. Computations were based on an assumption of parallelism. Perceived slant was well-correlated with virtual slant, even though slants were generally underestimated. The qualitative relationship between virtual slants and physical surface slants was variable, and liable to produce different degrees and directions of apparent rotation as well as of apparent compensation, depending on the depicted slant of the stimulus. Contributions of screen-related cues were small and mainly limited to small slants. Remarkably, binocular disparity did not have any effect on perceived slant. The results imply that many past findings of both transformation and (apparent) compensation in pictorial viewing may in principle be straightforwardly explained as a function of the virtual stimulus.

The Vision of David Marr

Marr proposed a computational paradigm for studying the visual system, wherein aspects of vision would be amenable to study with what might be regarded a computational-reductionist approach. First, vision would be cleaved into separable ‘computational theories’, in which the visual system is characterized in terms of its computational goals and the strategies by which they are carried out. Each such computational theory could then be investigated in increasingly concrete terms, from symbols and measurements, to representations and algorithms, to processes and neural implementations. This paradigm rests on some general expectations of a symbolic information processing system, including his stated principles of explicit naming, modular design, least commitment, and graceful degradation. In retrospect, the computational framework also tacitly rests on additional assumptions about the nature of biological information processing: (1) separability of computational strategies, (2) separability of representations, (3) a pipeline nature of information processing, and that (4) the representations employ primitives of low dimensionality. These assumptions are discussed in this retrospective.

Transfer of perceptual learning of depth discrimination between local and global stereograms

Several previous studies reported differences when stereothresholds are assessed with local-contour stereograms vs. complex random-dot stereograms (RDSs). Dissimilar thresholds may be due to differences in the properties of the stereograms (e.g. spatial frequency content, contrast, inter-element separation, area) or to different underlying processing mechanisms. This study examined the transfer of perceptual learning of depth discrimination between local and global RDSs with similar properties, and vice versa. If global and local stereograms are processed by separate neural mechanisms, then the magnitude and rate of training for the two types of stimuli are likely to differ, and the transfer of training from one stimulus type to the other should be minimal. Based on previous results, we chose RDSs with element densities of 0.17% and 28.3% to serve as the local and global stereograms, respectively. Fourteen inexperienced subjects with normal binocular vision were randomly assigned to either a local- or global- RDS training group. Stereothresholds for both stimulus types were measured before and after 7700 training trials distributed over 10 sessions. Stereothresholds for the trained condition improve for approximately 3000 trials, by an average of 0.36+/-0.08 for local and 0.29+/-0.10 for global RDSs, and level off thereafter. Neither the rate nor the magnitude of improvement differ statistically between the local- and global-training groups. Further, no significant difference exists in the amount of improvement on the trained vs. the untrained targets for either training group. These results are consistent with the operation of a single mechanism to process both local and global stereograms.

A laminar cortical model for 3D perception of slanted and curved surfaces and of 2D images: development, attention, and bistability

A model of laminar visual cortical dynamics proposes how 3D boundary and surface representations arise from viewing slanted and curved 3D objects and 2D images. The 3D boundary representations emerge from non-classical receptive field interactions within intracortical and intercortical feedback circuits. Such non-classical interactions within cortical areas V1 and V2 contextually disambiguate classical receptive field responses to ambiguous visual cues using cells that are sensitive to colinear contours, angles, and disparity gradients. Remarkably, these cell types can all be explained as variants of a unified perceptual grouping circuit whose most familiar example is a 2D colinear bipole cell. Model simulations show how this circuit can develop cell selectivity to colinear contours and angles, how slanted surfaces can activate 3D boundary representations that are sensitive to angles and disparity gradients, how 3D filling-in occurs across slanted surfaces, how a 2D Necker cube image can be represented in 3D, and how bistable 3D Necker cube percepts occur. The model also explains data about slant aftereffects and 3D neon color spreading. It shows how chemical transmitters that habituate, or depress, in an activity-dependent way can help to control development and also to trigger bistable 3D percepts and slant aftereffects. Attention can influence which of these percepts is perceived by propagating selectively along object boundaries.

Integration of binocular disparity and monocular cues at near threshold level

We examined the dependency of the integration of multiple depth cues upon the combined cues and upon the consistency of depth information from different cues. For each observer, depth thresholds were measured by the use of stimuli in which different depth cues (motion parallax, binocular disparity, and monocular configuration) specified the surface undulating sinusoidally with different spatial frequencies and different phases. Analysis of d(') showed that the performance was better than the prediction of probability summation only when parallax and disparity cues specified an undulation with the same spatial frequency and same phase. The probability summation model overestimated the performance for the other conditions of combination of disparity and parallax, and for all of the conditions of combination of disparity and monocular configuration. These results suggest that the improvement in depth perception caused by integration of multiple cues depends on the type of combined cues, and that the visual system possibly integrates the depth information from different cues at different stages of the visual processing.

Bayesian modeling of cue interaction: bistability in stereoscopic slant perception

Our two eyes receive different views of a visual scene, and the resulting binocular disparities enable us to reconstruct its three-dimensional layout. However, the visual environment is also rich in monocular depth cues. We examined the resulting percept when observers view a scene in which there are large conflicts between the surface slant signaled by binocular disparities and the slant signaled by monocular perspective. For a range of disparity-perspective cue conflicts, many observers experience bistability: They are able to perceive two distinct slants and to flip between the two percepts in a controlled way. We present a Bayesian model that describes the quantitative aspects of perceived slant on the basis of the likelihoods of both perspective and disparity slant information combined with prior assumptions about the shape and orientation of objects in the scene. Our Bayesian approach can be regarded as an overarching framework that allows researchers to study all cue integration aspects-including perceptual decisions--in a unified manner.

Curvature biases in stereoscopic vision: a nasotemporal asymmetry

The reliability of curvature judgments for linear elements was studied, with stereograms that contained a binocular arc with curvature in depth, and either a binocular frontoparallel arc or a monocular one, on a background representing a hemiellipsoid. The subjects made about 15% errors on binocular arcs with curvature in depth, and 60%-80% of these occurred when both the hemiellipsoid and the arc were convex, the arc being perceived as concave, by transparency through the hemiellipsoid. There were also about 15%-30% errors on frontoparallel arcs, but spread among all situations, with a small prevalence of concave judgments. Curvature in depth was assigned to the monocular stimuli in more than 60% of the cases. There was a curvature bias when the monocular arcs were on the nasal side, and were viewed against a concave background. Assuming parallel viewing, nasal ingoing arcs were usually perceived as concave, and nasal outgoing arcs usually perceived as convex, in agreement with geometrical likelihood. Nasal-side elements captured by one eye are, in general, those with the highest likelihood of having matching elements in the other eye. Then the observed nasal bias effect suggests that the matching process in stereopsis could be driven from the nasal sides of the projections in the two cerebral hemispheres.

Determinants of object recognition

We demonstrate that performance on an object recognition task can be explained in terms of observer-specific perceptual profiles. These profiles are derived from a battery of tests, including the effects of stereo, texture, outline (occluding contour), and motion cues on amplitude judgements of curved surfaces. Using a task in which observers learned to recognise 'amoeboid' objects, a multivariate regression analysis revealed that three psychometric variables derived from the test battery account for 74% of the variance in learning rate. These variables are choice reaction time, and the relative dependence of amplitude judgements on motion and outline cues. The implications of these findings for the existence of observer-specific perceptual profiles, and their relation to the fundamental psychophysical competences associated with object recognition are discussed.

Temporal dependencies in resolving monocular and binocular cue conflict in slant perception

Observers viewed large dichoptic patterns undergoing smooth temporal modulations or step changes in simulated slant or inclination under various conditions of disparity-perspective cue conflict and concordance. After presentation of each test surface, subjects adjusted a comparison surface to match the perceived slant or inclination of the test surface. Addition of conflicting perspective to disparity affected slant and inclination perception more for brief than for long presentations. Perspective had more influence for smooth temporal changes than for step changes in slant or inclination and for surfaces presented in isolation rather than with a zero disparity frame. These results indicate that conflicting perspective information plays a dominant role in determining the temporal properties of perceived slant and inclination.

Temporal aspects of slant and inclination perception

Linear transformations (shear or scale transformations) of either horizontal or vertical disparity give rise to the percept of slant or inclination. It has been proposed that the percept of slant induced by vertical size disparity, known as Ogle's induced-size effect, and the analogous induced-shear effect, compensate for scale and shear distortions arising from aniseikonia, eccentric viewing, and cyclodisparity. We hypothesised that these linear transformations of vertical disparity are processed more slowly than equivalent transformations of horizontal disparity (horizontal shear and size disparity). We studied the temporal properties of the stereoscopic slant and inclination percepts that arose when subjects viewed stereograms with various combinations of horizontal and vertical size or shear disparities. We found no evidence to support our hypothesis. There were no clear differences in the build-up of percepts of slant or inclination induced by step changes in horizontal size or shear disparity and those induced by step changes in vertical size or shear disparity. Perceived slant and inclination decreased in a similar manner with increasing temporal frequency for modulations of transformations of both horizontal and vertical disparity. Considerable individual differences were found and several subjects experienced slant reversal, particularly with oscillating stimuli. An interesting finding was that perceived slant induced by modulations of dilation disparity was in the direction of the vertical component. This suggests the vertical size disparity mechanism has a higher temporal bandwidth than the horizontal size disparity mechanism. However, conflicting perspective information may play a dominant role in determining the temporal properties of perceived slant and inclination.

Individual differences in the use of depth cues: implications for computer- and video-based tasks

This paper reports an experimental study of individual differences in the performance of computer-stimulated and 'real world' versions of an interactive depth perception task. The availability of depth cues (disparity, accommodation, luminance, and texture) was manipulated. Results indicate that ability to perceive depth using binocular cues is not associated with ability to perceive depth using monocular cues. Further, ability to integrate cues was a strong correlate of depth perception when multiple cues were available, and appeared more important in this respect than ability to use individual depth cues. Correlations between psychometric measures of fluid intelligence/spatial ability and depth perception were generally weak, but consistent with the broader pattern of results, suggesting that individual differences in depth perception should currently be regarded as 'cue specific'. Hypothesized facilitative effects of paradoxical monocular stereopsis were not found.

The perception of scale-dependent and scale-independent surface structure from binocular disparity, texture, and shading

The integration of binocular disparity, shading, and texture was measured for two different aspects of three-dimensional structure: (1) shape index, which is a measure of scale-independent structure, and (2) curvedness, which is a measure of scale-dependent structure. Binocular disparity was found to contribute significantly more to judged shape index than it does to judged curvedness, and shading and texture were both found to contribute more to judged curvedness than to judged shape index. These results demonstrate that different cues do not contribute equally to different aspects of perceived surface structure. This finding suggests that, for the case of linear integration, multiple cues to three-dimensional structure do not combine on the basis of a single type of representation shared by all the 'shape-from-X' processes in the visual system.

Stereopsis with normal and reversed binocular parallax using a head mounted display in normal and strabismic subjects

A virtual reality system using a head mounted display (HMD) was constructed for the purpose of finding out how human beings perceive the three-dimensional world. Four ophthalmologically normal subjects and 4 strabismic patients (2 with exotropia, 2 with esotropia) were examined. They wore the HMD and viewed some familiar objects (e.g. human face) and an unfamiliar object (e.g. tangled ropes) under normal parallax, reversed parallax, and monocular conditions. They also attempted the ring and hook test under each condition. They recognized the normal familiar objects under each condition, but some normal subjects were confused when they perceived unfamiliar objects. The normal subjects barely passed the ring and book test under the reversed parallax condition. The results of the strabismic subjects showed change under each condition. Stereoptic ability in normal and strabismic patients were considered based on the above. Also considered were some stereoptic keys, binocular parallax, monocular stereoptic keys such as shadows, shades, texture, etc., the near reflex and the fact that the objects were familiar. It was found that the strabismic patients used monocular stereoptic keys more than normal people to supplement their lack of binocular parallax perception.

Does the human visual system implement an ideal observer theory of slant from texture?

Texture information about surface shape can be decomposed into three constituents: compression, density and scaling. Blake, Bülthoff and Sheinberg's (1993, Vision Research, 33, 1723-1737) Ideal Observer theory of slant from texture predicts that the relative contribution of compression and density to the percept of planar surface slant should vary with field of view (FOV). The contribution of compression and density should both increase as FOV increases but statistical analysis shows that the reliability, and hence the expected contribution, of density increases at a greater rate than compression with increasing FOV. Specific predictions are that at FOV < 20 deg compression should be more effective than density, at FOV approximately 20 deg, compression and density should be equally effective, and at FOV > 20 deg, compression should be less effective than density. These predictions were tested by pitting these components of texture against one another. The method used was similar to that described in Frisby and Buckley [1992 In Orban, G. & Nagel H.-H. (Eds) Artificial and biological visual systems. Berlin: Springer]: observers judged binocularly the slant of a large table on to which was projected a texture created using computer graphics to contain various texture component cues to slant. The size of the projected texture patch determined the FOV which was by this means set to either 10, 20 or 30 deg. It was found that compression was the dominant cue irrespective of FOV, even if it was perturbed by noise. Possible reasons for this divergence of human observers from predictions of the Ideal Observer theory of slant from texture are discussed.

How is motion disparity integrated with binocular disparity in depth perception?

Two experiments presented motion disparity conflicting with binocular disparity to examine how these cues determined apparent depth order (convex, concave) and depth magnitude. In each experiment, 8 subjects estimated the depth order and depth magnitude. The first experiment showed the following. (1) The visual system used one of these cues exclusively in selecting a depth order for each display. (2) The visual system integrated the depth magnitude information from these cues by a weighted additive fashion if it selected the binocular disparity in depth order perception and if the depth magnitude specified by motion disparity was small relative to that specified by binocular disparity. (3) The visual system ignored the depth magnitude information of binocular disparity if it selected the motion disparity in depth order perception. The second experiment showed that these three points were consistent whether the subject's head movement or object movement generated motion disparity.

Integration by association: combining three-dimensional cues to extrinsic surface shape

Surface shape is visually derived from multiple sources of three-dimensional (3-D) information. The apparent unity of the percept creates an assumption that there are perceptual processes that resolve and combine the different channels of information within some common 3-D representation. However, integration by the explicit conversion of the information from different sources into a common representation is computationally difficult and unnecessary. An alternative, integration by association, is discussed and related to a variety of experimental results on 3-D-cue conflict.

Measurement and modeling of depth cue combination: in defense of weak fusion

Various visual cues provide information about depth and shape in a scene. When several of these cues are simultaneously available in a single location in the scene, the visual system attempts to combine them. In this paper, we discuss three key issues relevant to the experimental analysis of depth cue combination in human vision: cue promotion, dynamic weighting of cues, and robustness of cue combination. We review recent psychophysical studies of human depth cue combination in light of these issues. We organize the discussion and review as the development of a model of the depth cue combination process termed modified weak fusion (MWF). We relate the MWF framework to Bayesian theories of cue combination. We argue that the MWF model is consistent with previous experimental results and is a parsimonious summary of these results. While the MWF model is motivated by normative considerations, it is primarily intended to guide experimental analysis of depth cue combination in human vision. We describe experimental methods, analogous to perturbation analysis, that permit us to analyze depth cue combination in novel ways. In particular these methods allow us to investigate the key issues we have raised. We summarize recent experimental tests of the MWF framework that use these methods.

Integration of shading and texture cues: testing the linear model

One of the first attempts to develop a formal model of depth cue integration is to be found in Maloney and Landy's [(1989) Proceedings of the SPIE: Visual communications and image processing, Part 2 (pp. 1154-1163)] "human depth combination rule". They advocate that the combination of depth cues by the visual system is best described by a weighted linear model. The present experiments tested whether the linear combination rule applies to the integration of texture and shading. As would be predicted by a linear combination rule, the weight assigned to the shading cue did not vary as a function of its curvature value. However, the weight assigned to the texture cue varied systematically as a function of the curvature values of both cues. Here we describe a non-linear model which provides a better fit to the data. Redescribing the stimuli in terms of depth rather than curvature reduced the goodness of fit for all models tested. These results support the hypothesis that the locus of cue integration is a curvature map, rather than a depth map. We conclude that the linear combination rule does not generalize to the integration of shading and texture, and that for these cues it is likely that integration occurs after the recovery of surface curvature.

Interaction of stereo, texture and outline cues in the shape perception of three-dimensional ridges

We report five psychophysical experiments that employed a cue conflict paradigm to investigate integration by the human visual system of surface shape information from stereo, texture and outline cues. The experiments used convex parabolic and triangular three-dimensional ridge stimuli, with amplitudes (base to peak) in the range 3-9 cm, viewed from 57 cm. The observers' task was to judge ridge amplitude using a scale of two-dimensional drawings of ridge profiles. Cue integration was studied using both vertically and horizontally oriented ridges and both real ridges and stereograms of ridges. The main findings were: (a) stereo strongly dominated all horizontal ridge stereograms; (b) texture and outline cues strongly dominated low (3-6 cm) but not high (9 cm) amplitude vertical ridge stereograms; (c) stereo strongly dominated all real ridge stimuli. These results are evidence against explanations of the vertical/horizontal stereo anisotropy which propose that it derives from stereo mechanisms being tuned only to disparity cues with non-zero second-order spatial derivatives or to disparity discontinuities. They also show that radically different results can be obtained when stereo mechanisms are explored using stereograms and real surfaces and possible reasons for this are discussed.