Spatial dependencies between local luminance and contrast in natural images

A Unified Framework for Salient Structure Detection by Contour-Guided Visual Search

We define the task of salient structure (SS) detection to unify the saliency-related tasks, such as fixation prediction, salient object detection, and detection of other structures of interest in cluttered environments. To solve such SS detection tasks, a unified framework inspired by the two-pathway-based search strategy of biological vision is proposed in this paper. First, a contour-based spatial prior (CBSP) is extracted based on the layout of edges in the given scene along a fast non-selective pathway, which provides a rough, task-irrelevant, and robust estimation of the locations where the potential SSs are present. Second, another flow of local feature extraction is executed in parallel along the selective pathway. Finally, Bayesian inference is used to auto-weight and integrate the local cues guided by CBSP and to predict the exact locations of SSs. This model is invariant to the size and features of objects. The experimental results on six large datasets (three fixation prediction datasets and three salient object datasets) demonstrate that our system achieves competitive performance for SS detection (i.e., both the tasks of fixation prediction and salient object detection) compared with the state-of-the-art methods. In addition, our system also performs well for salient object construction from saliency maps and can be easily extended for salient edge detection.

Multifeature-based surround inhibition improves contour detection in natural images

To effectively perform visual tasks like detecting contours, the visual system normally needs to integrate multiple visual features. Sufficient physiological studies have revealed that for a large number of neurons in the primary visual cortex (V1) of monkeys and cats, neuronal responses elicited by the stimuli placed within the classical receptive field (CRF) are substantially modulated, normally inhibited, when difference exists between the CRF and its surround, namely, non-CRF, for various local features. The exquisite sensitivity of V1 neurons to the center-surround stimulus configuration is thought to serve important perceptual functions, including contour detection. In this paper, we propose a biologically motivated model to improve the performance of perceptually salient contour detection. The main contribution is the multifeature-based center-surround framework, in which the surround inhibition weights of individual features, including orientation, luminance, and luminance contrast, are combined according to a scale-guided strategy, and the combined weights are then used to modulate the final surround inhibition of the neurons. The performance was compared with that of single-cue-based models and other existing methods (especially other biologically motivated ones). The results show that combining multiple cues can substantially improve the performance of contour detection compared with the models using single cue. In general, luminance and luminance contrast contribute much more than orientation to the specific task of contour extraction, at least in gray-scale natural images.

Population response to natural images in the primary visual cortex encodes local stimulus attributes and perceptual processing

The primary visual cortex (V1) is extensively studied with a large repertoire of stimuli, yet little is known about its encoding of natural images. Using voltage-sensitive dye imaging in behaving monkeys, we measured neural population response evoked in V1 by natural images presented during a face/scramble discrimination task. The population response showed two distinct phases of activity: an early phase that was spread over most of the imaged area, and a late phase that was spatially confined. To study the detailed relation between the stimulus and the population response, we used a simple encoding model to compute a continuous map of the expected neural response based on local attributes of the stimulus (luminance and contrast), followed by an analytical retinotopic transformation. Then, we computed the spatial correlation between the maps of the expected and observed response. We found that the early response was highly correlated with the local luminance of the stimulus and was sufficient to effectively discriminate between stimuli at the single trial level. The late response, on the other hand, showed a much lower correlation to the local luminance, was confined to central parts of the face images, and was highly correlated with the animal's perceptual report. Our study reveals a continuous spatial encoding of low- and high-level features of natural images in V1. The low level is directly linked to the stimulus basic local attributes and the high level is correlated with the perceptual outcome of the stimulus processing.

Stereoacuity in the periphery is limited by internal noise

It is well-established that depth discrimination is finer in the fovea than the periphery. Here, we study the decline in depth discrimination thresholds with distance from the fovea using an equivalent noise analysis to separate the contributions of internal noise and sampling efficiency. Observers discriminated the mean depth of patches of "dead leaves" composed of ellipses varying in size, orientation, and luminance at varying levels of disparity noise between 0.05 and 13.56 arcmin and visual field locations between 0° and 9° eccentricity. At low levels of disparity noise, depth discrimination thresholds were lower in the fovea than in the periphery. At higher noise levels (above 3.39 arcmin), thresholds converged, and there was little difference between foveal and peripheral depth discrimination. The parameters estimated from the equivalent noise model indicate that an increase in internal noise is the limiting factor in peripheral depth discrimination with no decline in sampling efficiency. Sampling efficiency was uniformly low across the visual field. The results indicate that a loss of precision of local disparity estimates early in visual processing limits fine depth discrimination in the periphery.

Representation of surface luminance and contrast in primary visual cortex

In visual perception, object identification requires both the ability to define regions of uniform luminance and zones of luminance contrast. Neural processes underlying contrast detection have been well studied, while those defining luminance remain poorly understood and controversial. Partially because stimuli comprised of uniform luminance are relatively ineffective in driving responses of cortical neurons, little effort has been made to systematically compare responses of individual neurons to both uniform luminance and contrast. Using large static uniform luminance and contrast stimuli, modulated temporally in luminance or contrast, we found a continuum of responses ranging from a few cells modulated only by luminance (luminance-only), to many cells modulated by both luminance and contrast (luminance-contrast), and to many others modulated only by contrast (contrast-only) in primary visual cortex. Moreover, luminance-contrast cells had broader orientation tuning, larger receptive field (RF) and lower spatial frequency Preference, on average, than contrast-only cells. Contrast-only cells had contrast responses more linearly correlated to the spatial structure of their RFs than luminance-contrast cells. Taken together these results suggest that luminance and contrast are represented, to some degree, by independent mechanisms that may be shaped by different classes of subcortical and/or cortical inputs.

Effects of mean luminance changes on human contrast perception: contrast dependence, time-course and spatial specificity

Background

When we are viewing natural scenes, every saccade abruptly changes both the mean luminance and the contrast structure falling on any given retinal location. Thus it would be useful if the two were independently encoded by the visual system, even when they change simultaneously. Recordings from single neurons in the cat visual system have suggested that contrast information may be quite independently represented in neural responses to simultaneous changes in contrast and luminance. Here we test to what extent this is true in human perception.

Methodology/Principal Findings

Small contrast stimuli were presented together with a 7-fold upward or downward step of mean luminance (between 185 and 1295 Td, corresponding to 14 and 98 cd/m²), either simultaneously or with various delays (50–800 ms). The perceived contrast of the target under the different conditions was measured with an adaptive staircase method. Over the contrast range 0.1–0.45, mainly subtractive attenuation was found. Perceived contrast decreased by 0.052±0.021 (N = 3) when target onset was simultaneous with the luminance increase. The attenuation subsided within 400 ms, and even faster after luminance decreases, where the effect was also smaller. The main results were robust against differences in target types and the size of the field over which luminance changed.

Conclusions/Significance

Perceived contrast is attenuated mainly by a subtractive term when coincident with a luminance change. The effect is of ecologically relevant magnitude and duration; in other words, strict contrast constancy must often fail during normal human visual behaviour. Still, the relative robustness of the contrast signal is remarkable in view of the limited dynamic response range of retinal cones. We propose a conceptual model for how early retinal signalling may allow this.

Measuring perceived differences in surface texture due to changes in higher order statistics

We investigate the ability of humans to perceive changes in the appearance of images of surface texture caused by the variation of their higher order statistics. We incrementally randomize their phase spectra while holding their first and second order statistics constant in order to ensure that the change in the appearance is due solely to changes in third and other higher order statistics. Stimuli comprise both natural and synthetically generated naturalistic images, with the latter being used to prevent observers from making pixel-wise comparisons. A difference scaling method is used to derive the perceptual scales for each observer, which show a sigmoidal relationship with the degree of randomization. Observers were maximally sensitive to changes within the 20%-60% randomization range. In order to account for this behavior we propose a biologically plausible model that computes the variance of local measurements of phase congruency.

Natural images dominate in binocular rivalry

Ecological approaches to perception have demonstrated that information encoding by the visual system is informed by the natural environment, both in terms of simple image attributes like luminance and contrast, and more complex relationships corresponding to Gestalt principles of perceptual organization. Here, we ask if this optimization biases perception of visual inputs that are perceptually bistable. Using the binocular rivalry paradigm, we designed stimuli that varied in either their spatiotemporal amplitude spectra or their phase spectra. We found that noise stimuli with "natural" amplitude spectra (i.e., amplitude content proportional to 1/f, where f is spatial or temporal frequency) dominate over those with any other systematic spectral slope, along both spatial and temporal dimensions. This could not be explained by perceived contrast measurements, and occurred even though all stimuli had equal energy. Calculating the effective contrast following attenuation by a model contrast sensitivity function suggested that the strong contrast dependency of rivalry provides the mechanism by which binocular vision is optimized for viewing natural images. We also compared rivalry between natural and phase-scrambled images and found a strong preference for natural phase spectra that could not be accounted for by observer biases in a control task. We propose that this phase specificity relates to contour information, and arises either from the activity of V1 complex cells, or from later visual areas, consistent with recent neuroimaging and single-cell work. Our findings demonstrate that human vision integrates information across space, time, and phase to select the input most likely to hold behavioral relevance.