Interpolation in structure from motion

“What” and “where” in spatial language and spatial cognition

Fundamental to spatial knowledge in all species are the representations underlying object recognition, object search, and navigation through space. But what sets humans apart from other species is our ability to express spatial experience through language. This target article explores the language ofobjectsandplaces, asking what geometric properties are preserved in the representations underlying object nouns and spatial prepositions in English. Evidence from these two aspects of language suggests there are significant differences in the geometric richness with which objects and places are encoded. When an object is named (i.e., with count nouns), detailed geometric properties – principally the object's shape (axes, solid and hollow volumes, surfaces, and parts) – are represented. In contrast, when an object plays the role of either “figure” (located object) or “ground” (reference object) in a locational expression, only very coarse geometric object properties are represented, primarily the main axes. In addition, the spatial functions encoded by spatial prepositions tend to be nonmetric and relatively coarse, for example, “containment,” “contact,” “relative distance,” and “relative direction.” These properties are representative of other languages as well. The striking differences in the way language encodes objects versus places lead us to suggest two explanations: First, there is a tendency for languages to level out geometric detail from both object and place representations. Second, a nonlinguistic disparity between the representations of “what” and “where” underlies how language represents objects and places. The language of objects and places converges with and enriches our understanding of corresponding spatial representations.

Depth capture by kinetic depth and by stereopsis

The perceived depth of regions within a stereogram lacking explicit disparity information can be captured by the surface structure of regions where disparity is explicit: stereo capture. In two experiments, observers estimated surface curvature/depth of an untextured object (a 'ribbon') superimposed on a cylinder textured with dots, the cylinder curvature being defined by disparity (stereo depth) or by motion parallax (kinetic depth: KD). With the stereo-defined cylinder, depth capture was obtained under conditions where the disparity of the ribbon was ambiguous; with the KD, cylinder depth capture was obtained under conditions where the motion flow of the cylinder was in a direction parallel to that of the ribbon. These results demonstrate yet another similarity between KD and stereopsis.

Curved trajectory prediction using a self-organizing neural network

Existing neural network models are capable of tracking linear trajectories of moving visual objects. This paper describes an additional neural mechanism, disfacilitation, that enhances the ability of a visual system to track curved trajectories. The added mechanism combines information about an object's trajectory with information about changes in the object's trajectory, to improve the estimates for the object's next probable location. Computational simulations are presented that show how the neural mechanism can learn to track the speed of objects and how the network operates to predict the trajectories of accelerating and decelerating objects.

Monocular discrimination of rigidly and nonrigidly moving objects

We measured thresholds for the monocular discrimination of rigidly and nonrigidly moving objects defined by motion parallax. The retinal projections of rigidly moving objects are subject to certain constraints. By applying smooth 2-D transformations to the projections of rigidly moving objects, we created stimuli in which these constraints were affected. Thresholds for (generic) nonrigid transformations that in theory can be detected from rigid ones by processing pairs of views depended not only on the extent to which the rigidity constraints were affected, but also on the structure and the movement of the simulated object. Nonrigid transformations under which every three successive views had a rigid interpretation were not discriminable from rigid transformations, except in cases where the distortions were very large. Under the rigidity assumption, this would mean that a large class of nonrigidly moving objects is erroneously perceived as rigidly moving.

Structure-from-motion: perceptual evidence for surface interpolation

Dynamic random-dot displays representing a rotating cylinder were used to investigate surface interpolation in the perception of structure-from-motion (SFM) in humans. Surface interpolation refers to a process in which a complete surface in depth is reconstructed from the object depth values extracted at the stimulus features. Surface interpolation will assign depth values even in parts of the object that contain no features. Such a "fill-in" process should make the detection of featureless stimulus areas ("holes") difficult. Indeed, we demonstrate that such holes in our rotating cylinder can be as wide as one-quarter of the stimulus before subjects can reliably detect their presence. Subjects were presented with a variation on the rotating cylinder in which all dots were oscillating either in synchrony or asynchronously. Subjects perceive a rigidly rotating cylinder even when such a percept is not in agreement with the physical stimulus. To reconcile this discrepancy between actual and perceived stimulus we propose that individual points contribute to a surface based object representation and that in this process the visual system looses access to the identity of the individual features that make up the surface. Finally we are able to explain a variety of previously documented perceptual peculiarities in the perception of structure-from-motion by arguing that the perceptual interpretation of the object's boundaries influences the surface interpolation process. These findings offer strong perceptual evidence for a process of surface interpolation and are also physiologically plausible given results from recordings in awake behaving monkey cortical areas V1 and MT. The companion paper demonstrates how such a surface interpolation process can be incorporated into a structure-from-motion algorithm and how object boundaries can influence the perception of structure-from-motion as has been demonstrated before and in this paper.

Interpolation across surface discontinuities in structure from motion

Interpolation across orientation discontinuities in simulated three-dimensional (3-D) surfaces was studied in three experiments with the use of structure-from-motion (SFM) displays. The displays depicted dots on two slanted planes with a region devoid of dots (a gap) between them. If extended through the gap at constant slope, the planes would meet at a dihedral edge. Subjects were required to place an SFM probe dot, located within the gap, on the perceived surface. Probe dot placements indicated that subjects perceived a smooth surface connecting the planes rather than a surface with a discontinuity. Probe dot placements varied with slope of the planes, density of the dots, and gap size, but not with orientation (horizontal or vertical) of the dihedral edge or of the axis of rotation. Smoothing was consistent with models of 2-D interpolation proposed by Ullman (1976) and Kellman and Shipley (1991) and with a model of 3-D interpolation proposed by Grimson (1981).

Discrimination of 3-D shape and 3-D curvature from motion in active vision

We examined the ability of human observers to discriminate between different 3-D quadratic surfaces defined by motion, and with head position fed back to the stimulus to provide an up-to-date dynamical perspective view. We tested whether 3-D shape or 3-D curvature would affect discrimination performance. It appeared that discrimination of 3-D quadratic shape clearly depended on shape but not on the amount of curvature. Even when the amount of curvature was randomized, subjects' performance was not altered. On the other hand, the discrimination of 3-D curvature clearly depended linearly on curvature with Weber fractions of 20% on the average and, to a small degree, on 3-D shape. The experiment shows that observers can easily separate 3-D shape and 3-D curvature, and that Koenderink's shape index and curvedness provide a convenient way to specify shape. These results warn us against using just any arbitrary 3-D shape in 3-D shape perception tasks and indicate, for example, that emphasizing 3-D shape in computer displays by exaggerating curvature does not have any effect.

Three-dimensional illusory contours and surfaces

Under general viewing conditions, objects are often partially camouflaged, obscured or occluded, thereby limiting information about their three-dimensional position, orientation and shape to incomplete and variable image cues. When presented with such partial cues, observers report perceiving 'illusory' contours and surfaces (forms) in regions having no physical image contrast. Here we report that three-dimensional illusory forms share three fundamental properties with 'real' forms: (1) the same forms are perceived using either stereo or motion parallax cues (cue invariance); (2) they retain their shape over changes in position and orientation relative to an observer (view stability); and (3) they can take the shape of general contours and surfaces in three dimensions (morphic generality). We hypothesize that illusory contours and surfaces are manifestations of a previously unnoticed visual process which constructs a representation of three-dimensional position, orientation and shape of objects from available image cues.