The generic viewpoint assumption in a framework for visual perception

General lighting can overcome accidental viewing

When seeing an object in a scene, the presumption of seeing that object from a general viewpoint (as opposed to an accidental viewpoint) is a useful heuristic to decide which of many interpretations of this object is correct. Similar heuristic assumptions on illumination quality might also be used for scene interpretation. Here we tested that assumption and asked if illumination information helps determine object properties when seen from an accidental viewpoint. Test objects were placed on a flat surface and illumination was varied while keeping the objects' images constant. Observers judged the shape or rigidity of static or moving simple objects presented in accidental view. They also chose which of two seemingly very similar faces was familiar. We found: (1) Objects might appear flat without shadow information but were perceived to be volumetric objects or non-planar in the presence of cast shadows. (2) Apparently non-rigid objects became rigid with shadow information. (3) Shading and shadows helped to infer which of two face was the familiar one. Previous results had shown that cast shadows help determine spatial layout of objects. Our study shows that other properties of objects like rigidity or 3D-shape can be disambiguated by shadow information.

3D shape recovery algorithm from image orientations of textured surfaces

Previous psychophysical studies have demonstrated that the image orientation of textured surfaces guides human 3D shape perception. However, the accuracy of computational 3D shape reconstruction solely from image orientation requires further study. This paper proposes a 3D shape recovery algorithm from the image orientation of a single textured surface image. The evaluation of the proposed algorithm uses computer-generated textured complex 3D surfaces. The depth correlations between the recovered and true surface shapes achieved or exceeded 0.8, which is similar to the accuracy of human shape perception, as shown in a previous psychophysical study, indicating that the image orientations contain adequate information for 3D shape recovery from textured surface images.

Visual and haptic cues in processing occlusion

Introduction

Although shape is effective in processing occlusion, ambiguities in segmentation can also be addressed using depth discontinuity given visually and haptically. This study elucidates the contribution of visual and haptic cues to depth discontinuity in processing occlusion.

Methods

A virtual reality experiment was conducted with 15 students as participants. Word stimuli were presented on a head-mounted display for recognition. The central part of the words was masked with a virtual ribbon placed at different depths so that the ribbon appeared as an occlusion. The visual depth cue was either present with binocular stereopsis or absent with monocular presentation. The haptic cue was either missing, provided consecutively, or concurrently, by actively tracing a real off-screen bar edge that was positionally aligned with the ribbon in the virtual space. Recognition performance was compared between depth cue conditions.

Results

We found that word recognition was better with the stereoscopic cue but not with the haptic cue, although both cues contributed to greater confidence in depth estimation. The performance was better when the ribbon was at the farther depth plane to appear as a hollow, rather than when it was at the nearer depth plane to cover the word.

Discussion

The results indicate that occlusion is processed in the human brain by visual input only despite the apparent effectiveness of haptic space perception, reflecting a complex set of natural constraints.

The case against probabilistic inference: a new deterministic theory of 3D visual processing

How the brain derives 3D information from inherently ambiguous visual input remains the fundamental question of human vision. The past two decades of research have addressed this question as a problem of probabilistic inference, the dominant model being maximum-likelihood estimation (MLE). This model assumes that independent depth-cue modules derive noisy but statistically accurate estimates of 3D scene parameters that are combined through a weighted average. Cue weights are adjusted based on the system representation of each module's output variability. Here I demonstrate that the MLE model fails to account for important psychophysical findings and, importantly, misinterprets the just noticeable difference, a hallmark measure of stimulus discriminability, to be an estimate of perceptual uncertainty. I propose a new theory, termed Intrinsic Constraint, which postulates that the visual system does not derive the most probable interpretation of the visual input, but rather, the most stable interpretation amid variations in viewing conditions. This goal is achieved with the Vector Sum model, which represents individual cue estimates as components of a multi-dimensional vector whose norm determines the combined output. This model accounts for the psychophysical findings cited in support of MLE, while predicting existing and new findings that contradict the MLE model. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Aesthetic appeal is no coincidence: Preference for generic over specific viewpoints

Generic viewpoints produce images where the visible elements, and their spatial relationships, are largely preserved with small alterations in viewpoint. In contrast, specific viewpoints result from the observer being in a special position relative to the scene, causing spatial alignments between elements that are readily broken with a change in viewpoint. It has been proposed that generic viewpoints are aesthetically more pleasing, due to their lack of suspicious spatial coincidences. We investigated this idea for cartoon foreground and background objects, and found images where foreground and background objects were not suspiciously aligned were judged to be more appealing. A similar, albeit weaker, effect was seen when we digitally manipulated a Monet painting that contained a prominent spatial alignment between a foreground and a background object: the appeal of the painting increased with the magnitude of an introduced spatial offset. A further experiment on an independent observer group found a strong preference for generic viewpoints when tested on a range of cartoon images featuring foregrounds and backgrounds. Our results provide empirical support for the idea that suspicious spatial coincidences between foreground and background objects reduces the aesthetic appeal of images.

From boundaries to bumps: When closed (extremal) contours are critical

Invariants underlying shape inference are elusive: A variety of shapes can give rise to the same image, and a variety of images can be rendered from the same shape. The occluding contour is a rare exception: It has both image salience, in terms of isophotes, and surface meaning, in terms of surface normal. We relax the notion of occluding contour and, more accurately, the rim on the object that projects to it, to define closed extremal curves. This new shape descriptor is invariant over different renderings. It exists at the topological level, which guarantees an image-based counterpart. It surrounds bumps and dents, as well as common interior shape components, and formalizes the qualitative nature of bump perception. The invariants are biologically computable, unify shape inferences from shading and specular materials, and predict new phenomena in bump and dent perception. Most important, working at the topological level allows us to capture the elusive aspect of bump boundaries.

The illusion of absence: how a common feature of magic shows can explain a class of road accidents

The purpose of the present note is to draw attention to the potential role of a recently discovered visual illusion in creating traffic accidents. The illusion consists in a compelling and immediate experience that the space behind an occluding object in the foreground is empty. Although the illusion refers to a region of space, which is invisible due to occlusion (a blind spot), there is evidence to suggest that it is nevertheless driven by visual mechanisms and that it can be just as deceptive and powerful as ordinary visual illusions. We suggest that this novel illusion can make situations involving blind spots in a road user's field of view even more dangerous than one would expect based on the lack of visibility by itself. This could be because it erroneously makes the road user feel that (s)he has actually seen everything there is to see, and thus has verified that the blind spot is empty. This hypothesis requires further testing before definitive conclusions can be drawn, but we wish to make researchers and authorities involved in the analysis of traffic accidents and on-the-spot crash investigations aware of its potential role in order to encourage registration of relevant data and facilitate further research.

Mental geometry of three-dimensional size perception

Judging the poses, sizes, and shapes of objects accurately is necessary for organisms and machines to operate successfully in the world. Retinal images of three-dimensional objects are mapped by the rules of projective geometry and preserve the invariants of that geometry. Since Plato, it has been debated whether geometry is innate to the human brain, and Poincare and Einstein thought it worth examining whether formal geometry arises from experience with the world. We examine if humans have learned to exploit projective geometry to estimate sizes and aspects of three-dimensional shape that are related to relative lengths and aspect ratios.

Numerous studies have examined size invariance as a function of physical distance, which changes scale on the retina. However, it is surprising that possible constancy or inconstancy of relative size seems not to have been investigated for object pose, which changes retinal image size differently along different axes. We show systematic underestimation of length for extents pointing toward or away from the observer, both for static objects and dynamically rotating objects. Observers do correct for projected shortening according to the optimal back-transform, obtained by inverting the projection function, but the correction is inadequate by a multiplicative factor. The clue is provided by the greater underestimation for longer objects, and the observation that they seem to be more slanted toward the observer. Adding a multiplicative factor for perceived slant in the back-transform model provides good fits to the corrections used by observers. We quantify the slant illusion with two different slant matching measurements, and use a dynamic demonstration to show that the slant illusion perceptually dominates length nonrigidity.

In biological and mechanical objects, distortions of shape are manifold, and changes in aspect ratio and relative limb sizes are functionally important. Our model shows that observers try to retain invariance of these aspects of shape to three-dimensional rotation by correcting retinal image distortions due to perspective projection, but the corrections can fall short. We discuss how these results imply that humans have internalized particular aspects of projective geometry through evolution or learning, and if humans assume that images are preserving the continuity, collinearity, and convergence invariances of projective geometry, that would simply explain why illusions such as Ames’ chair appear cohesive despite being a projection of disjointed elements, and thus supplement the generic viewpoint assumption.

The Illusion of Absence in Magic Tricks

Recently, a curious illusion of absence has been described, where the space behind an occluder is compellingly experienced as empty. This illusion is similar to illusions based on amodal completion in the sense that it refers to occluded portions of a visual scene and informal observations suggest that it may also be largely impervious to conscious knowledge. The aim of the present experiment was to test the hypothesis that the illusion of absence is cognitively impenetrable in the same way as amodal completion. Participants viewed magic tricks based on amodal completion, the illusion of absence, or attentional and reasoning misdirection and tried to infer the secret behind the tricks after one, two, or three presentations. The results show that the tricks based on the illusion of absence are very difficult to debunk, even after repeated presentations. In this regard, they are similar to tricks based on amodal completion but different from tricks based on attentional and reasoning misdirection. The participants also rated how magical they felt the tricks were. Surprisingly, the magic ratings tended to be quite high even in trials where the participants had already discovered the secret behind the trick. This unexpected finding may be taken to suggest that there may be two magical moments in the lifetime of a magic trick: In addition to the magical experience evoked by trick itself, discovering the secret behind the trick may also evoke an experience of impossibility.

A Perceptual Illusion of Empty Space Can Create a Perceptual Illusion of Levitation

A recent analysis of magic tricks suggests the existence of a perceptual illusion where the space hidden behind an occluding object is experienced as empty in a strangely compelling way. Here, we show that this illusion of absence is not just a trivial consequence of the lack of retinal stimulation but rather the result of an active process of perceptual construction. The results of a simple experiment show that this perceptual illusion of absence can in turn trigger perceptual processes which generate an immediate perceptual impression of levitation via a percept–percept coupling. This suggests that magical illusions of levitation are partially driven by an immediate perceptual impression of floating in thin air. The perceptual mechanisms underlying the illusion of absence are hitherto unknown, but our results provide support for a potential explanation based on the generic view principle.

Optimality and heuristics in perceptual neuroscience

The foundation for modern understanding of how we make perceptual decisions about what we see or where to look comes from considering the optimal way to perform these behaviors. While statistical computation is useful for deriving the optimal solution to a perceptual problem, optimality requires perfect knowledge of priors and often complex computation. Accumulating evidence, however, suggests that optimal perceptual goals can be achieved or approximated more simply by human observers using heuristic approaches. Perceptual neuroscientists captivated by optimal explanations of sensory behaviors will fail in their search for the neural circuits and cortical processes that implement an optimal computation whenever that behavior is actually achieved through heuristics. This article provides a cross-disciplinary review of decision-making with the aim of building perceptual theory that uses optimality to set the computational goals for perceptual behavior but, through consideration of ecological, computational, and energetic constraints, incorporates how these optimal goals can be achieved through heuristic approximation.

Deformation-induced transparency resolves color scission

Dynamic image deformation produces the perception of a transparent material that appears to deform the background image by light refraction. Since past studies on this phenomenon have mainly used subjective judgment about the presence of a transparent layer, it remains unsolved whether this is a real perceptual transparency effect in the sense that it forms surface representations, as do conventional transparency effects. Visual computation for color and luminance transparency, induced mainly by surface-contour information, can be decomposed into two components: surface formation to determine foreground and background layers, and scission to assign color and luminance to each layer. Here we show that deformation-induced perceptual transparency aids surface formation by color transparency and consequently resolves color scission. We asked observers to report the color of the front layer in a spatial region with a neutral physical color. The layer color could be seen as either reddish or greenish depending on the spatial context producing the color transparency, which was, however, ambiguous about the order of layers. We found that adding to the display a deformation-induced transparency that could specify the front layer significantly biased color scission in the predicted way if and only if the deformation-induced transparency was spatially coincident with the interpretation of color transparency. The results indicate that deformation-induced transparency is indeed a novel type of perceptual transparency that plays a role in surface formation in cooperation with color transparency.

A Switching Observer for Human Perceptual Estimation

Human perceptual inference has been fruitfully characterized as a normative Bayesian process in which sensory evidence and priors are multiplicatively combined to form posteriors from which sensory estimates can be optimally read out. We tested whether this basic Bayesian framework could explain human subjects' behavior in two estimation tasks in which we varied the strength of sensory evidence (motion coherence or contrast) and priors (set of directions or orientations). We found that despite excellent agreement of estimates mean and variability with a Basic Bayesian observer model, the estimate distributions were bimodal with unpredicted modes near the prior and the likelihood. We developed a model that switched between prior and sensory evidence rather than integrating the two, which better explained the data than the Basic and several other Bayesian observers. Our data suggest that humans can approximate Bayesian optimality with a switching heuristic that forgoes multiplicative combination of priors and likelihoods.

The Other Side of Magic

When magicians perform spectacles that seem to defy the laws of nature, they do so by manipulating psychological reality. Hence, the principles underlying the art of conjuring are potentially of interest to psychological science. Here, we argue that perceptual and cognitive principles governing how humans experience hidden things and reason about them play a central role in many magic tricks. Different from tricks based on many other forms of misdirection, which require considerable skill on the part of the magician, many elements of these tricks are essentially self-working because they rely on automatic perceptual and cognitive processes. Since these processes are not directly observable, even experienced magicians may be oblivious to their central role in creating strong magical experiences and tricks that are almost impossible to debunk, even after repeated presentations. We delineate how insights from perceptual psychology provide a framework for understanding why these tricks work so well. Conversely, we argue that studying magic tricks that work much better than one intuitively would believe provides a promising heuristic for charting unexplored aspects of perception and cognition.

The Role of Junctions in Surface Completion and Contour Matching

It has been suggested that contour junctions may be used as cues for occlusion. Ecologically, T-junctions and L-junctions are concurrent with situations of occlusion: they arise when the bounding contour of the occluding surface intersects with that of the occluded surface. However, there are other image properties that can be used as cues for occlusion. Here the role of junctions is directly compared with other occlusion cues—specifically, relatability and surface-similarity—in the emergence of amodal completion and illusory contour perception. Stimuli have been constructed that differ only in the junction structure, with the other occlusion cues kept unchanged. L-junctions and T-junctions were eliminated from the image or manipulated so as to be locally inconsistent with the (still valid) global occlusion interpretation. Although the other occlusion cues of relatability and surface similarity still existed in the image, subjects reported not perceiving illusory contours or amodal completion in junction-manipulated images. Junction manipulation also affected the perceived stereoscopic depth and motion of image regions, depending on whether they were perceived to amodally complete with a disjoint region in the image. These results are interpreted in terms of the role of junctions in the processes of surface completion and contour matching. It is proposed that junctions, being a local cue for occlusion, are used to launch completion processes. Other, more global occlusion cues, such as relatability, play a part at a later stage, once completion processes have been launched.

Part Boundaries Alter the Perception of Transparency

The perception of transparency is a remarkable feat of human vision: A single stimulation at the retina is interpreted as arising from two (or more) distinct surfaces, separated in depth, in the same visual direction. This feat is intriguing because physical transparency is neither necessary nor sufficient for phenomenal transparency. Many conditions for phenomenal transparency have been studied, including luminance, chromaticity, stereo depth, apparent motion, and structure from motion. Figural conditions have also been studied, primarily by Gestalt psychologists, resulting in descriptive laws. Here we extend, and make precise, these laws using the genericity principle and the minima rule for part boundaries. We report experiments that support the psychological plausibility of these refinements. The results suggest that the formation of visual objects and their parts is an early process in human vision that can precede the representation of transparency.

Will understanding vision require a wholly empirical paradigm?

Based on electrophysiological and anatomical studies, a prevalent conception is that the visual system recovers features of the world from retinal images to generate perceptions and guide behavior. This paradigm, however, is unable to explain why visual perceptions differ from physical measurements, or how behavior could routinely succeed on this basis. An alternative is that vision does not recover features of the world, but assigns perceptual qualities empirically by associating frequently occurring stimulus patterns with useful responses on the basis of survival and reproductive success. The purpose of the present article is to briefly describe this strategy of vision and the evidence for it.

The Put-and-Fetch Ambiguity: How Magicians Exploit the Principle of Exclusive Allocation of Movements to Intentions

In many magic tricks, magicians fool their audience by performing a mock action (a so-called “ruse”), which merely serves the purpose of providing a seemingly natural explanation for visible movements that are actually part of the secret move they want to hide from the audience. Here, we discuss a special magic ruse in which the action of secretly putting something somewhere is “explained away” by the mock action of fetching something from the same place, or vice versa. Interestingly, the psychological principles underlying the amazing potency and robustness of this technique seem to be very similar to the general perceptual principles underlying figure–ground perception and the assignment of border ownership. This analogy may be useful for exploring the possibility that this and similar magical effects involve immediate “unconscious inferences” about intentions more akin to perceptual processing than to explicit deliberations based on a reflective “theory” of mind.

Competition with and without priority control: linking rivalry to attention through winner-take-all networks with memory

Competition is ubiquitous in perception. For example, items in the visual field compete for processing resources, and attention controls their priority (biased competition). The inevitable ambiguity in the interpretation of sensory signals yields another form of competition: distinct perceptual interpretations compete for access to awareness. Rivalry, where two equally likely percepts compete for dominance, explicates the latter form of competition. Building upon the similarity between attention and rivalry, we propose to model rivalry by a generic competitive circuit that is widely used in the attention literature-a winner-take-all (WTA) network. Specifically, we show that a network of two coupled WTA circuits replicates three common hallmarks of rivalry: the distribution of dominance durations, their dependence on input strength ("Levelt's propositions"), and the effects of stimulus removal (blanking). This model introduces a form of memory by forming discrete states and explains experimental data better than competitive models of rivalry without memory. This result supports the crucial role of memory in rivalry specifically and in competitive processes in general. Our approach unifies the seemingly distinct phenomena of rivalry, memory, and attention in a single model with competition as the common underlying principle.

Coincidence avoidance principle in surface haptic interpretation

When multiple fingertips experience force sensations, how does the brain interpret the combined sensation? In particular, under what conditions are the sensations perceived as separate or, alternatively, as an integrated whole? In this work, we used a custom force-feedback device to display force signals to two fingertips (index finger and thumb) as they traveled along collinear paths. Each finger experienced a pattern of forces that, taken individually, produced illusory virtual bumps, and subjects reported whether they felt zero, one, or two bumps. We varied the spatial separation between these bump-like force-feedback regions, from being much greater than the finger span to nearly exactly the finger span. When the bump spacing was the same as the finger span, subjects tended to report only one bump. We found that the results are consistent with a quantitative model of perception in which the brain selects a structural interpretation of force signals that relies on minimizing coincidence stemming from accidental alignments between fingertips and inferred surface structures.

Effects of specular highlights on perceived surface convexity

Shading is known to produce vivid perceptions of depth. However, the influence of specular highlights on perceived shape is unclear: some studies have shown that highlights improve quantitative shape perception while others have shown no effect. Here we ask how specular highlights combine with Lambertian shading cues to determine perceived surface curvature, and to what degree this is based upon a coherent model of the scene geometry. Observers viewed ambiguous convex/concave shaded surfaces, with or without highlights. We show that the presence/absence of specular highlights has an effect on qualitative shape, their presence biasing perception toward convex interpretations of ambiguous shaded objects. We also find that the alignment of a highlight with the Lambertian shading modulates its effect on perceived shape; misaligned highlights are less likely to be perceived as specularities, and thus have less effect on shape perception. Increasing the depth of the surface or the slant of the illuminant also modulated the effect of the highlight, increasing the bias toward convexity. The effect of highlights on perceived shape can be understood probabilistically in terms of scene geometry: for deeper objects and/or highly slanted illuminants, highlights will occur on convex but not concave surfaces, due to occlusion of the illuminant. Given uncertainty about the exact object depth and illuminant direction, the presence of a highlight increases the probability that the surface is convex.

A primary goal of the human visual system is to reconstruct the three-dimensional structure of the environment from two-dimensional retinal images. This process is under-determined: an infinite number of combinations of shape, material properties and illumination conditions could give rise to any single image. Rather than determining the true three-dimensional scene in a deductive manner, the visual system must make its ‘best guess’ based on the image, probabilistic models of image formation, and the stored probability of various scene configurations. For example, the visual system appears to assume that convex surfaces are more common than concave ones, biasing perception toward convex surfaces when the image is ambiguous. Here we identify a new probabilistic cue for surface shape: a shape with a visible specular highlight is more likely to be convex than one without. Highlights occur when light is reflected in a mirror-like way from glossy surfaces such as polished marble or metal. Due to the geometry of reflection, however, highlights are more likely to be occluded on concave objects. We show that the human visual system makes use of this constraint: shape perception is biased toward convex surfaces when highlights are apparent.

Side effect of acting on the world: acquisition of action-outcome statistic relation alters visual interpretation of action outcome

Humans can acquire the statistical features of the external world and employ them to control behaviors. Some external events occur in harmony with an agent's action, and thus, humans should also be able to acquire the statistical features between an action and its external outcome. We report that the acquired action-outcome statistical features alter the visual appearance of the action outcome. Pressing either of two assigned keys triggered visual motion whose direction was statistically biased either upward or downward, and observers judged the stimulus motion direction. Points of subjective equality (PSE) for judging motion direction were shifted repulsively from the mean of the distribution associated with each key. Our Bayesian model accounted for the PSE shifts, indicating the optimal acquisition of the action-effect statistical relation. The PSE shifts were moderately attenuated when the action-outcome contingency was reduced. The Bayesian model again accounted for the attenuated PSE shifts. On the other hand, when the action-outcome contiguity was reduced, the PSE shifts were greatly attenuated, and however, the Bayesian model could not accounted for the shifts. The results indicate that visual appearance can be modified by prediction based on the optimal acquisition of action-effect causal relation.

The perception of cast shadows

When an object casts its shadow on a background surface, the shadow can be informative about the shape of the object, the shape of the background surface and the spatial arrangement of the object relative to the background. Among all these roles, we found that cast shadows were perceptually most relevant for the recovery of spatial arrangement, especially when the shadow is in motion. This finding is intriguing when one considers the ambiguities in the possible ways that shadow motion can be interpreted. We reasoned that the visual system must use a priori constraints to disambiguate the cast shadow motion. One of these constraints is that the light source is stationary. Though simple, the stationary-light-source constraint supports rich, reliable inferences about the qualitative motions of objects in three dimensions.

Visual cognition

Visual cognition, high-level vision, mid-level vision and top-down processing all refer to decision-based scene analyses that combine prior knowledge with retinal input to generate representations. The label "visual cognition" is little used at present, but research and experiments on mid- and high-level, inference-based vision have flourished, becoming in the 21st century a significant, if often understated part, of current vision research. How does visual cognition work? What are its moving parts? This paper reviews the origins and architecture of visual cognition and briefly describes some work in the areas of routines, attention, surfaces, objects, and events (motion, causality, and agency). Most vision scientists avoid being too explicit when presenting concepts about visual cognition, having learned that explicit models invite easy criticism. What we see in the literature is ample evidence for visual cognition, but few or only cautious attempts to detail how it might work. This is the great unfinished business of vision research: at some point we will be done with characterizing how the visual system measures the world and we will have to return to the question of how vision constructs models of objects, surfaces, scenes, and events.

Non-Bayesian contour synthesis

Recent research has witnessed an explosive increase in models that treat percepts as optimal probabilistic inference. The ubiquity of partial camouflage and occlusion in natural scenes, and the demonstrated capacity of the visual system to synthesize coherent contours and surfaces from fragmented image data, has inspired numerous attempts to model visual interpolation processes as rational inference. Here, we report striking new forms of visual interpolation that generate highly improbable percepts. We present motion displays depicting simple occlusion sequences that elicit vivid percepts of illusory contours (ICs) in displays for which they play no necessary explanatory role. These ICs define a second, redundant occluding surface, even though all of the image data can be fully explained by an occluding surface that is clearly visible. The formation of ICs in these images therefore entails an extraordinarily improbable co-occurrence of two occluding surfaces that arise from the same local occlusion events. The perceived strength of the ICs depends on simple low-level image properties, which suggests that they emerge as the outputs of mechanisms that automatically synthesize contours from the pattern of occlusion and disocclusion of local contour segments. These percepts challenge attempts to model visual interpolation as a form of rational inference and suggest the need to consider a broader space of computational problems and/or implementation level constraints to understand their genesis.

The shading cue in context

The shading cue is supposed to be a major factor in monocular stereopsis. However, the hypothesis is hardly corroborated by available data. For instance, the conventional stimulus used in perception research, which involves a circular disk with monotonic luminance gradient on a uniform surround, is theoretically 'explained' by any quadric surface, including spherical caps or cups (the conventional response categories), cylindrical ruts or ridges, and saddle surfaces. Whereas cylindrical ruts or ridges are reported when the outline is changed from circular to square, saddle surfaces are never reported. We introduce a method that allows us to differentiate between such possible responses. We report observations on a number of variations of the conventional stimulus, including variations of shape and quality of the boundary, and contexts that allow the observer to infer illumination direction. We find strong and expected influences of outline shape, but, perhaps surprisingly, we fail to find any influence of context, and only partial influence of outline quality. Moreover, we report appreciable differences within the generic population. We trace some of the idiosyncrasies (as compared to shape from shading algorithms) of the human observer to generic properties of the environment, in particular the fact that many objects are limited in size and elliptically convex over most of their boundaries.

Asymmetries in perception of 3D orientation

Visual scene interpretation depends on assumptions based on the statistical regularities of the world. People have some preference for seeing ambiguously oriented objects (Necker cubes) as if tilted down or viewed from above. This bias is a near certainty in the first instant (∼1 s) of viewing and declines over the course of many seconds. In addition, we found that there is modulation of perceived orientation that varies with position—for example objects on the left are more likely to be interpreted as viewed from the right. Therefore there is both a viewed-from-above prior and a scene position-dependent modulation of perceived 3-D orientation. These results are consistent with the idea that ambiguously oriented objects are initially assigned an orientation consistent with our experience of an asymmetric world in which objects most probably sit on surfaces below eye level.

Boundaries, shading, and border ownership: A cusp at their interaction

The association of borders with "figure" rather than "background" provides a topological organizing principle for early vision. Such global influences have recently been shown to have local effects, with neuronal activity modulated by stimulus properties from well outside the classical receptive field. We extend the theoretical analysis of such phenomena by developing the geometry of interaction between shading, boundaries, and boundary ownership for smooth surfaces. The purely exterior edges of smooth objects enjoy a fold-type relationship between shading and boundary, due to foreshortening, while the background is cut off transversely. However, at cusp points in the image mapping the exterior boundary ends abruptly. Since such singular points are notoriously unstable, we conjecture that this process is regularized by a natural quantization of suggestive contours due to physiological boundary-detection mechanisms. The result extends a theorem about how contours must end to one that characterizes surface (Gaussian) curvature in the neighborhood of where they appear to end. Apparent contours and their interaction with local shading thus provide important monocular shape cues.

The Effects of Mirror Reflections and Planar Rotations of Pictures on the Shape Percept of the Depicted Object

Mirror reflections and planar rotations of a picture do not result in any variations concerning the internal geometrical layout of the objects depicted in the picture. We examined to what extent these picture plane transformations gave rise to perceptual differences. A large set of pictures was generated by mirror-reflecting and rotating a set of six original photographs in the picture plane. We externalised the percepts of the depicted objects by using a direct perceptual method: the gauge-figure method. Participants had to adjust a gauge figure so that it seemed to be painted on the surface of the depicted object. From an extensive set of settings collected this way, we computed for each picture the three-dimensional interpretation—or pictorial relief—of the depicted object. On the basis of this set of pictorial reliefs, we addressed the effects of mirror reflections and rotations of pictures on the shape percept of the depicted object. Mirror-reflecting a picture around the horizontal axis resulted in large differences in pictorial reliefs, whereas mirror-reflecting pictures around the vertical axis resulted in only small differences in pictorial reliefs. Clockwise 90°, 180°, and 270° rotation affected the pictorial relief significantly. In all cases, the differences between the pictorial reliefs could be resolved by affine transformations, and could thus be ascribed to different solutions of the depth ambiguities inherent in pictures.

First impressions and last resorts: how listeners adjust to speaker variability

Perceptual theories must explain how perceivers extract meaningful information from a continuously variable physical signal. In the case of speech, the puzzle is that little reliable acoustic invariance seems to exist. We tested the hypothesis that speech-perception processes recover invariants not about the signal, but rather about the source that produced the signal. Findings from two manipulations suggest that the system learns those properties of speech that result from idiosyncratic characteristics of the speaker; the same properties are not learned when they can be attributed to incidental factors. We also found evidence for how the system determines what is characteristic: In the absence of other information about the speaker, the system relies on episodic order, representing those properties present during early experience as characteristic of the speaker. This "first-impressions" bias can be overridden, however, when variation is an incidental consequence of a temporary state (a pen in the speaker's mouth), rather than characteristic of the speaker.

Optimal predictions in everyday cognition

Human perception and memory are often explained as optimal statistical inferences that are informed by accurate prior probabilities. In contrast, cognitive judgments are usually viewed as following error-prone heuristics that are insensitive to priors. We examined the optimality of human cognition in a more realistic context than typical laboratory studies, asking people to make predictions about the duration or extent of everyday phenomena such as human life spans and the box-office take of movies. Our results suggest that everyday cognitive judgments follow the same optimal statistical principles as perception and memory, and reveal a close correspondence between people's implicit probabilistic models and the statistics of the world.

Comparison of Bayesian and empirical ranking approaches to visual perception

Much current vision research is predicated on the idea--and a rapidly growing body of evidence--that visual percepts are generated according to the empirical significance of light stimuli rather than their physical characteristics. As a result, an increasing number of investigators have asked how visual perception can be rationalized in these terms. Here, we compare two different theoretical frameworks for predicting what observers actually see in response to visual stimuli: Bayesian decision theory and empirical ranking theory. Deciding which of these approaches has greater merit is likely to determine how the statistical operations that apparently underlie visual perception are eventually understood.

Visual interpolation is not scale invariant

According to the scale-dependence hypothesis, the visual interpolation of contour fragments depends on the retinal separation of endpoints: as the retinal size of a partially occluded angle increases, the interpolated contour gradually deviates from the shortest connecting path and approaches the shape of the unoccluded angle. In the field model, as the retinal size increases the strength of good continuation increases while the strength of the minimal-path tendency decreases. To test the scale-dependence hypothesis--as well as other hypotheses connected to inclusion, support-ratio dependence, and extended relatability--we ran two experiments using the probe localization technique. Stimuli were regular polygons with rectilinear contours bounding symmetrically occluded angles. Retinal size was manipulated by changing viewing distance. Observers were asked to judge if a probe, briefly superposed on the occlusion region, was inside or outside the amodally completed angle. Retinal size strongly influenced the penetration of interpolated trajectories in the predicted direction. However, support ratio and interpolated angle size interacted with retinal size, consistently with the idea that unification factors are effective within a spatial window. We modified the field model to include the size of such a window as a new parameter and generated model-based trajectories that fitted empirical data closely.

The perceived motion of a stereokinetic stimulus

An ellipse rotating in the image plane can produce several different percepts. The two-dimensional (2D) percepts are either a rotating rigid ellipse or a constantly deforming non-rigid ellipse. The 3D percept is a rotating rigid circular disk that is tilted relative to the image plane. Stimuli that generate 3D percepts based on purely 2D rotational motion are known as stereokinetic stimuli. We examined the 3D percepts generated by the rotating ellipse stimulus. In theory, the motion of the 3D percept cannot be reliably inferred based on the 2D stimulus. When we quantitatively estimated observers' perceived motion, however, we found that the perceived motion was nearly identical across observers. These results suggest that all observers had similar 3D percepts. We assumed that given the 2D rotating ellipse stimulus the visual system generates a rigid 3D percept that is as slow and smooth as possible. The percepts predicted by these assumptions closely matched the experimental data. These findings suggest that perceptual ambiguity in stereokinetic stimuli is resolved using slow and smooth motion assumptions.

Brain signals for spatial attention predict performance in a motion discrimination task

The reliability of visual perception is thought to reflect the quality of the sensory information. However, we show that subjects' performance can be predicted, trial-by-trial, by neural activity that precedes the onset of a sensory stimulus. Using functional MRI (fMRI), we studied how neural mechanisms that mediate spatial attention affect the accuracy of a motion discrimination judgment. The amplitude of blood oxygen level-dependent (BOLD) signals after a cue directing spatial attention predicted subjects' accuracy on 60-75% of the trials. Widespread predictive signals, which included dorsal parietal, visual extra-striate, prefrontal and sensory-motor cortex, depended on whether the cue correctly specified the stimulus location. Therefore, these signals indicate the degree of utilization of the cued information and play a role in the control of spatial attention. We conclude that variability in perceptual performance can be partly explained by the variability in endogenous, preparatory processes and that BOLD signals can be used to forecast human behavior.

Size contrast and assimilation explained by the statistics of natural scene geometry

The term "size contrast and assimilation" refers to a large class of geometrical illusions in which the apparent sizes of identical visual targets in various contexts are different. Here we have examined whether these intriguing discrepancies between physical and perceived size can be explained by a visual process in which percepts are determined by the probability distribution of the possible real-world sources of retinal stimuli. To test this idea, we acquired a range image database of natural scenes that specified the location of every image point in 3-D space. By sampling the possible physical sources of various size contrast or assimilation stimuli in the database, we determined the probability distributions of the size of the target in the images generated by these sources. For each of the various stimuli tested, these probability distributions of target size in different contexts accurately predicted the perceptual effects reported in psychophysical studies. We conclude that size contrast and assimilation effects are a further manifestation of a fundamentally probabilistic process of visual perception.

Object perception as Bayesian inference

We perceive the shapes and material properties of objects quickly and reliably despite the complexity and objective ambiguities of natural images. Typical images are highly complex because they consist of many objects embedded in background clutter. Moreover, the image features of an object are extremely variable and ambiguous owing to the effects of projection, occlusion, background clutter, and illumination. The very success of everyday vision implies neural mechanisms, yet to be understood, that discount irrelevant information and organize ambiguous or noisy local image features into objects and surfaces. Recent work in Bayesian theories of visual perception has shown how complexity may be managed and ambiguity resolved through the task-dependent, probabilistic integration of prior object knowledge with image features.

Bayesian modeling of cue interaction: bistability in stereoscopic slant perception

Our two eyes receive different views of a visual scene, and the resulting binocular disparities enable us to reconstruct its three-dimensional layout. However, the visual environment is also rich in monocular depth cues. We examined the resulting percept when observers view a scene in which there are large conflicts between the surface slant signaled by binocular disparities and the slant signaled by monocular perspective. For a range of disparity-perspective cue conflicts, many observers experience bistability: They are able to perceive two distinct slants and to flip between the two percepts in a controlled way. We present a Bayesian model that describes the quantitative aspects of perceived slant on the basis of the likelihoods of both perspective and disparity slant information combined with prior assumptions about the shape and orientation of objects in the scene. Our Bayesian approach can be regarded as an overarching framework that allows researchers to study all cue integration aspects-including perceptual decisions--in a unified manner.

Recovering slant and angular velocity from a linear velocity field: modeling and psychophysics

The data from two experiments, both using stimuli simulating orthographically rotating surfaces, are presented, with the primary variable of interest being whether the magnitude of the simulated gradient was from expanding vs. contracting motion. One experiment asked observers to report the apparent slant of the rotating surface, using a gauge figure. The other experiment asked observers to report the angular velocity, using a comparison rotating sphere. The results from both experiments clearly show that observers are less sensitive to expanding than to contracting optic-flow fields. These results are well predicted by a probabilistic model which derives the orientation and angular velocity of the projected surface from the properties of the optic flow computed within an extended time window.

Bayesian models of object perception

The human visual system is the most complex pattern recognition device known. In ways that are yet to be fully understood, the visual cortex arrives at a simple and unambiguous interpretation of data from the retinal image that is useful for the decisions and actions of everyday life. Recent advances in Bayesian models of computer vision and in the measurement and modeling of natural image statistics are providing the tools to test and constrain theories of human object perception. In turn, these theories are having an impact on the interpretation of cortical function.

Toward a perceptual theory of transparency

Theories of perceptual transparency have typically been developed within the context of a physical model that generates the percept of transparency (F. Metelli's episcotister model, 1974b). Here 2 fundamental questions are investigated: (a) When does the visual system initiate the percept of one surface seen through another? (b) How does it assign surface properties to a transparent layer? Results reveal systematic deviations from the predictions of Metelli's model, both for initiating image decomposition into multiple surfaces and for assigning surface attributes. Specifically, results demonstrate that the visual system uses Michelson contrast as a critical image variable to initiate percepts of transparency and to assign transmittance to transparent surfaces. Findings are discussed in relation to previous theories of transparency, lightness, brightness, and contrast-contrast.