Surface gradients, contours and the perception of surface attitude in images of complex scenes

How little do we need for 3-D shape perception?

How little do we need to perceive 3-D shape in monocular natural images? The shape-from-texture and shape-from-shading perspectives would motivate that 3-D perception vanishes once low-level cues are disrupted. Is this the case in human vision? Or can top-down influences salvage the percept? In this study we probe this question by employing a gauge-figure paradigm similar to that used by Koenderink et al (1992, Perception & Psychophysics 52 487-496). Subjects were presented degraded natural images and instructed to make local assessments of slant and tilt at various locations thereby quantifying their internal 3-D percept. Analysis of subjects' responses reveals recognition to be a significant influence thereby allowing subjects to perceive 3-D shape at high levels of degradation. Specifically, we identify the 'medium-blur' condition, images approximately 32 pixels on a side, to be the limit for accurate 3-D shape perception. In addition, we find that degradation affects the perceived slant of point-estimates making images look flatter as degradation increases. A subsequent condition that eliminates texture and shading but preserves contour and recognition reveals how bottom-up and top-down cues can combine for accurate 3-D shape perception.

Inconsistency of perceived 3D shape

Internal consistency of local depth, slant, and curvature judgments was studied by asking participants to match two 3D surfaces rendered by different mixtures of 3D cues (velocity, texture, and shading). We found that perceptual judgments were not consistent with each other, with cue-specific distortions. Adding multiple cues did not eliminate the inconsistencies of the judgments. These results can be predicted by the Intrinsic Constraint (IC) model according to which the perceptual metric local estimates are a monotonically increasing function of the Signal-to-Noise Ratio of the optimal combination of direct information of 3D shape (Domini, Caudek, & Tassinari, 2006).

Effect of planar cuts' orientation on the perceived surface layout and object's shape

The effect of the orientation of the cutting planes producing planar curves over the surface of an object on its perceived pose and shape was investigated for line drawings representing three-dimensional objects. The results suggest that the orientational flow produced by the surface curves introduces an apparent object rotation in depth and in the image plane and changes in its perceived elongation. The apparent location of the nearest points is determined by the points of maximal view-dependent unsigned curvature of the surface curves. The data are discussed in relation to the interaction of the shape-from-silhouette system and shape-from-contour system and its effect on the interpretation of the surface contours with respect to the surface geometry.

Reality-augmented virtual fluoroscopy for computer-assisted diaphyseal long bone fracture osteosynthesis: A novel technique and feasibility study results

In this paper, a novel technique to create a reality-augmented virtual fluoroscopy for computer-assisted diaphyseal long bone fracture osteosynthesis and feasibility study results are presented. With this novel technique, repositioning of bone fragments during closed fracture reduction and osteosynthesis can lead to image updates in the virtual imaging planes of all acquired images without any radiation. The technique is achieved with a two-stage method. After acquiring a few (normally two) calibrated fluoroscopic images and before fracture reduction, the first stage, data preparation, interactively identifies and segments the bone fragments from the background in each image. After that, the second stage, image updates, repositions the fragment projection on to each virtual imaging plane in real time during fracture reduction and osteosynthesis using an OpenGL-based texture warping. Combined with a photorealistic virtual implant model rendering technique, the present technique allows the control of a closed indirect fracture osteosynthesis in the real world through direct insight into the virtual world. The first clinical study results show the reduction in the X-ray radiation to the patient as well as to the surgical team, and the improved operative precision, guaranteeing more safety for the patient. Furthermore, based on the experiences gained from this clinical study, two technical enhancements are proposed. One focuses on eliminating the user interactions with automated identifications and segmentations of bone fragments. The other focuses on providing non-photorealistic implant visualization. Further experiments are performed to validate the effectiveness of the proposed enhancements.

Texture properties affecting the accuracy of surface attitude judgements

Prior studies of the perception of surface shape and attitude from texture have focused on measuring the sensitivity of the visual system to the various geometric deformations induced by projection. Studies that examine variations in accuracy caused by spatial properties of the texture itself are fewer, and often confound multiple, potentially important properties. Here we examine the perception of surface attitude for a broad range of synthetic textures that may represent the types of structure encountered in the natural world. These stimuli allow us to isolate the respective roles of texels, spatial scale structure, discrete symmetries and regularity in the judgement of both the slant and tilt of textured surfaces. Texels, spatial scale structure and discrete symmetries were all found to play a role. Discrete rotational symmetries were found to be particularly important for accurate tilt estimation, likely mediated by skew symmetry and/or linear perspective cues. The operational range of viewing distances over which accurate attitude judgements can be made is greatly extended when texture structure is distributed over multiple scales. Small biases caused by variations in the spin of symmetric textures are observed and are consistent, at least qualitatively, with a Bayesian cue combination model previously proposed by .

Recognizing faces defined by texture gradients

Texture gradients can reveal surface orientation in a manner similar to shape from shading, and therefore provide an important cue for object recognition. In this study, we tested whether a complex 3-D object, such as a face, can be identified from texture gradients alone. The stimuli were laser-scanned faces for which the texture element was a fractal-noise pattern mapped onto the 3-D surface. An eight-alternative forced choice task was used in which participants matched a face defined by texture gradients to one of eight faces defined by shape from shading (Experiment 1) or by texture gradients (Experiment 2). On average, participants scored 24% and 18%, respectively, above chance in these experiments. Although this performance was much poorer than the performance based entirely on shape-from-shading stimuli (Experiment 3), the results suggest that texture gradient information may be used to recover surface geometry of complex objects.

A contour propagation approach to surface filling-in and volume formation

A new approach to surface and volume formation is introduced in response to the question, "Why do some silhouettes look 3 dimensional (3D) and others look 2D?" The central idea is that form information can propagate away from a "propagable segment" (PS) of occluding contour that could have projected onto the image from the visible portion of a cross-section of a surface. A key property of a PS is that it exhibits abrupt curvature changes where it meets the rest of the occluding contour. An algorithm is described for filling in curved surfaces from a PS: When copies of a PS are propagated into the interior, they act as cross-sectional surface contours that also exhibit abrupt curvature changes with the rest of the occluding contour. The result is a nonmetric coding of 3D-shape in terms of local ordinal surface curvature and orientation relationships that is scale, translation, and rotation invariant.

Informational primacy of visual dimensions: specialized roles for luminance and chromaticity in figure-ground perception

Three experiments were conducted to examine the distinct contributions of two visual dimensions to figure-ground segregation. In each experiment, pattern identification was assessed by asking observers to judge whether a near-threshold test pattern was the same or different in shape to a high-contrast comparison pattern. A test pattern could differ from its background along one dimension, either luminance (luminance tasks) or chromaticity (chromaticity tasks). In each task, performance in a baseline condition, in which the test pattern was intact, was compared with performance in each of several degradation conditions, in which either the contour or the surface of the figure was degraded, using either partial occlusion (Experiment 1) or ramping (Experiments 2 and 3) of figure-ground differences. In each experiment, performance in luminance tasks was worst when the contour was degraded, whereas performance in chromaticity tasks was worst when the surface was degraded. This interaction was found even when spatial frequencies were fixed across test patterns by low-pass filtering. The results are consistent with a late (postfiltering) dual-mechanism system that processes luminance information to extract boundary representations and chromaticity information to extract surface representations.

The contributions of slant and tilt to the detection of local surface orientation in structure from motion

The contribution of slant and tilt to the detection of differences in local surface orientation was examined for structure-from-motion (SFM) displays of a complex sinusoidal surface. Observers judged whether an elliptical SFM gauge figure appeared to be lying on the surface or intersecting it. The gauge figure orientation either matched the local surface orientation or differed from it in slant, tilt, or both. Similar sensitivity was found for deviations in slant and tilt, but greater biases and variability were found when the gauge figure deviated from the local surface orientation in slant, depending on the sign of the difference between the gauge figure and local surface orientation and the position of the gauge figure. The results are consistent with Stevens' (Biological Cybernetics, 46 (1983) 183-195) discussion of the computational advantages of slant and tilt contributing independently to the detection of differences in local surface orientation. The effects of changes in perceived surface slant and tilt during rotation and of the misperception of surface depth on the detection of local orientation in dynamic images are discussed.

Inter-attribute tilt effects and orientation analysis in the visual brain

A test grating appears to be tilted away from an inducing grating for small angular separations (repulsion), but towards the inducing grating for larger angular separations (attraction). Previous research on luminance gratings suggests that repulsion is caused by local inhibition in cortical areas V1 and/or V2, and that the attraction involves global interactions beyond V1, in extrastriate areas. Experiments reported here demonstrate attribute invariant attraction and repulsion effects for gratings specified by luminance, motion, and disparity contrasts. A frame surrounding the inducing grating abolishes only the attraction effect, but a spatial frequency difference, or a small gap between the inducer and test gratings, abolishes only the repulsion effect, irrespective of the attributes that specify the gratings. It is proposed that detectors selectively sensitive to attribute invariant orientation and size exist in early cortical sites such as V1 and/or V2.

Haptic perception of texture gradients

To pick up 3-D aspects of pictures is arguably the most difficult problem concerning tactile pictorial perception by the blind. The aim of the experiments reported was to examine the potential utility of texture gradients in this context. Since there is no theoretical basis for predicting absolute values of 3-D properties from 2-D patterns read by the finger pads, the abilities of participants to perceive gradients lying between known maxima and minima were assessed. Experiment 1 involved blindfolded sighted participants making verbal magnitude estimations of texture-gradient magnitudes corresponding to plane surfaces at different slants. In experiment 2 the participants' task was to orient a surface at a slant corresponding to the texture gradients depicted tactually, and experiment 3 required early-blind participants to attempt the same task. The results revealed that participants can scale the magnitudes of texture gradients with high precision and that they can also accurately produce surface slants from depictions, providing the extreme conditions are clearly defined and there are opportunities for learning. Texture gradients appear as informative to the blind as they do to the sighted. To what extent these data can be generalised to other gradients and textures or to other projections of 3-D scenes remains to be investigated.