Distinct mechanisms mediate visual detection and identification

Color discrimination repetition distorts color representations

Perceptual learning is the improvement of perceptual performance after repeated practice on a perceptual task. Studies on perceptual learning in color vision are limited. In this study, we measured the impact of color discrimination repetitions at a specific base color on color perception for entire hues. Participants performed five sessions of color discrimination training (200 or 300 trials per session) over five days, at colors on either the negative or positive direction of the L-M color axis, based on group assignment. We administered three color perception assessments (unique hues, color category boundaries, and color appearance) before and after the sessions to evaluate perceptual changes after training. The results showed declines in color discrimination thresholds after training, as expected. Additionally, the training influenced outcomes across all three assessment types. After the training, the perceived color appearance changed near the trained color along the stimulus hue, and some of the unique hues and the color category boundaries moved significantly toward the trained color. These findings indicate that short-term repetitions of color discrimination training can alter color representations in the visual system, distorting color perception around the trained color.

Equivalent noise characterization of human lightness constancy

A goal of visual perception is to provide stable representations of task-relevant scene properties (e.g. object reflectance) despite variation in task-irrelevant scene properties (e.g. illumination and reflectance of other nearby objects). To study such stability in the context of the perceptual representation of lightness, we introduce a threshold-based psychophysical paradigm. We measure how thresholds for discriminating the achromatic reflectance of a target object (task-relevant property) in rendered naturalistic scenes are impacted by variation in the reflectance functions of background objects (task-irrelevant property), using a two-alternative forced-choice paradigm in which the reflectance of the background objects is randomized across the two intervals of each trial. We control the amount of background reflectance variation by manipulating a statistical model of naturally occurring surface reflectances. For low background object reflectance variation, discrimination thresholds were nearly constant, indicating that observers’ internal noise determines threshold in this regime. As background object reflectance variation increases, its effects start to dominate performance. A model based on signal detection theory allows us to express the effects of task-irrelevant variation in terms of the equivalent noise, that is relative to the intrinsic precision of the task-relevant perceptual representation. The results indicate that although naturally occurring background object reflectance variation does intrude on the perceptual representation of target object lightness, the effect is modest – within a factor of two of the equivalent noise level set by internal noise.

Fluctuating environmental light limits number of surfaces visually recognizable by colour

Small changes in daylight in the environment can produce large changes in reflected light, even over short intervals of time. Do these changes limit the visual recognition of surfaces by their colour? To address this question, information-theoretic methods were used to estimate computationally the maximum number of surfaces in a sample that can be identified as the same after an interval. Scene data were taken from successive hyperspectral radiance images. With no illumination change, the average number of surfaces distinguishable by colour was of the order of 10,000. But with an illumination change, the average number still identifiable declined rapidly with change duration. In one condition, the number after two minutes was around 600, after 10 min around 200, and after an hour around 70. These limits on identification are much lower than with spectral changes in daylight. No recoding of the colour signal is likely to recover surface identity lost in this uncertain environment.

Measuring and Modeling Visual Appearance

In studying visual perception, we seek to develop models of processing that accurately predict perceptual judgments. Much of this work is focused on judgments of discrimination, and there is a large literature concerning models of visual discrimination. There are, however, non-threshold visual judgments, such as judgments of the magnitude of differences between visual stimuli, that provide a means to bridge the gap between threshold and appearance. We describe two such models of suprathreshold judgments, maximum likelihood difference scaling and maximum likelihood conjoint measurement, and review recent literature that has exploited them.

Luminance gradients and non-gradients as a cue for distinguishing reflectance and illumination in achromatic images: A computational approach

The brain analyses the visual world through the luminance patterns that reach the retina. Formally, luminance (as measured by the retina) is the product of illumination and reflectance. Whereas illumination is highly variable, reflectance is a physical property that characterizes each object surface. Due to memory constraints, it seems plausible that the visual system suppresses illumination patterns before object recognition takes place. Since many combinations of reflectance and illumination can give rise to identical luminance values, finding the correct reflectance value of a surface is an ill-posed problem, and it is still an open question how it is solved by the brain. Here we propose a computational approach that first learns filter kernels ("receptive fields") for slow and fast variations in luminance, respectively, from achromatic real-world images. Distinguishing between luminance gradients (slow variations) and non-gradients (fast variations) could serve to constrain the mentioned ill-posed problem. The second stage of our approach successfully segregates luminance gradients and non-gradients from real-world images. Our approach furthermore predicts that visual illusions that contain luminance gradients (such as Adelson's checker-shadow display or grating induction) may occur as a consequence of this segregation process.

Neuronal population mechanisms of lightness perception

The way that humans and animals perceive the lightness of an object depends on its physical luminance as well as its surrounding context. While neuronal responses throughout the visual pathway are modulated by context, the relationship between neuronal responses and lightness perception is poorly understood. We searched for a neuronal mechanism of lightness by recording responses of neuronal populations in monkey primary visual cortex (V1) and area V4 to stimuli that produce a lightness illusion in humans, in which the lightness of a disk depends on the context in which it is embedded. We found that the way individual units encode the luminance (or equivalently for our stimuli, contrast) of the disk and its context is extremely heterogeneous. This motivated us to ask whether the population representation in either V1 or V4 satisfies three criteria: 1) disk luminance is represented with high fidelity, 2) the context surrounding the disk is also represented, and 3) the representations of disk luminance and context interact to create a representation of lightness that depends on these factors in a manner consistent with human psychophysical judgments of disk lightness. We found that populations of units in both V1 and V4 fulfill the first two criteria but that we cannot conclude that the two types of information in either area interact in a manner that clearly predicts human psychophysical measurements: the interpretation of our population measurements depends on how subsequent areas read out lightness from the population responses. NEW & NOTEWORTHY A core question in visual neuroscience is how the brain extracts stable representations of object properties from the retinal image. We searched for a neuronal mechanism of lightness perception by determining whether the responses of neuronal populations in primary visual cortex and area V4 could account for a lightness illusion measured using human psychophysics. Our results suggest that comparing psychophysics with population recordings will yield insight into neuronal mechanisms underlying a variety of perceptual phenomena.

Different Electrophysiological Correlates of Visual Awareness for Detection and Identification

Detecting the presence of an object is a different process than identifying the object as a particular object. This difference has not been taken into account in designing experiments on the neural correlates of consciousness. We compared the electrophysiological correlates of conscious detection and identification directly by measuring ERPs while participants performed either a task only requiring the conscious detection of the stimulus or a higher-level task requiring its conscious identification. Behavioral results showed that, even if the stimulus was consciously detected, it was not necessarily identified. A posterior electrophysiological signature 200–300 msec after stimulus onset was sensitive for conscious detection but not for conscious identification, which correlated with a later widespread activity. Thus, we found behavioral and neural evidence for elementary visual experiences, which are not yet enriched with higher-level knowledge. The search for the mechanisms of consciousness should focus on the early elementary phenomenal experiences to avoid the confounding effects of higher-level processes.

Infants' perception of lightness changes related to cast shadows

When humans perceive the lightness of an object’s surface in shadows there is an implicit assumption that cast shadows dim the surface. In two experiments, we investigated whether 5- to 8-month-old infants make this assumption about shadows. According to this shadow assumption, the apparent change in lightness produced by shadows on an object’s surface are attributed to blocked light sources. If infants can use the shadow assumption to perceive the object’s lightness in shadows, they will also be able to detect unnatural lightness changes in shadows. We compared the infants’ looking times to the unnatural and the natural lightness changes in the shadow when an object (duck) goes through the cast shadow. In Experiment 1, we examined whether infants could detect the unnatural lightness changes of the object’s surface in shadows. We created computer-graphic movies of unnatural and natural lightness changes to the duck’s surface. Our results showed that 7- to 8-month-olds but not 5- to 6-month-olds significantly preferred the movie with the unnatural changes in lightness, indicating that only the older infants could detect these changes. In Experiment 2, we confirmed that the infants’ preference was based on the detection of unnatural lightness changes according to the shadow assumption. The natural and the unnatural lightness changes of Experiment 1 were presented without cast shadows. Under these conditions, neither younger nor older infants showed a significant preference. Taken together, the experiments showed that 7- to 8-month-old infants could detect the unnaturalness of a surface’s lightness changes produced by shadows. In conclusion, our findings suggest that 7- to 8-month-old infants can perceive an object’s lightness in shadows by using an assumption that cast shadows dim the surface of an object.

Context-dependent lightness affects perceived contrast

Perceived contrast of a grating varies with its background (or mean) luminance: of the two gratings with the same photometric contrast the one on higher luminance background appears to have higher contrast. Does perceived contrast also vary with context-dependent background lightness even when the luminance remains constant? We investigated this question using a stimulus in which two equiluminant patches ("context squares", CSs) appear different in lightness. First we measured the lightness effect in a behavioral experiment. After ensuring that it was present for all participants, we conducted perceived contrast experiments, where participants judged the contrast of rectified incremental and decremental square-wave gratings superimposed on the CSs. For the incremental gratings participants' settings were significantly different for the two CSs. Specifically, perceived contrast was higher when the gratings were placed on the context square that was perceived lighter. In a follow-up experiment we measured perceived contrast of rectified gratings on isolated patches that differed in luminance. The pattern of results of the two experiments was consistent, demonstrating that possibly shared mechanisms underpin the effects of background luminance and context-dependent lightness on perceived contrast.

Dissociation of equilibrium points for color-discrimination and color-appearance mechanisms in incomplete chromatic adaptation

We compared the color-discrimination thresholds and supra-threshold color differences (STCDs) obtained in complete chromatic adaptation (gray) and incomplete chromatic adaptation (red). The color-difference profiles were examined by evaluating the perceptual distances between various color pairs using maximum likelihood difference scaling. In the gray condition, the chromaticities corresponding with the smallest threshold and the largest color difference were almost identical. In contrast, in the red condition, they were dissociated. The peaks of the sensitivity functions derived from the color-discrimination thresholds and STCDs along the L-M axis were systematically different between the adaptation conditions. These results suggest that the color signals involved in color discrimination and STCD tasks are controlled by separate mechanisms with different characteristic properties.

Visual Adaptation

Sensory systems continuously mold themselves to the widely varying contexts in which they must operate. Studies of these adaptations have played a long and central role in vision science. In part this is because the specific adaptations remain a powerful tool for dissecting vision, by exposing the mechanisms that are adapting. That is, "if it adapts, it's there." Many insights about vision have come from using adaptation in this way, as a method. A second important trend has been the realization that the processes of adaptation are themselves essential to how vision works, and thus are likely to operate at all levels. That is, "if it's there, it adapts." This has focused interest on the mechanisms of adaptation as the target rather than the probe. Together both approaches have led to an emerging insight of adaptation as a fundamental and ubiquitous coding strategy impacting all aspects of how we see.

What visual illusions tell us about underlying neural mechanisms and observer strategies for tackling the inverse problem of achromatic perception

Research in lightness perception centers on understanding the prior assumptions and processing strategies the visual system uses to parse the retinal intensity distribution (the proximal stimulus) into the surface reflectance and illumination components of the scene (the distal stimulus—ground truth). It is agreed that the visual system must compare different regions of the visual image to solve this inverse problem; however, the nature of the comparisons and the mechanisms underlying them are topics of intense debate. Perceptual illusions are of value because they reveal important information about these visual processing mechanisms. We propose a framework for lightness research that resolves confusions and paradoxes in the literature, and provides insight into the mechanisms the visual system employs to tackle the inverse problem. The main idea is that much of the debate and confusion in the literature stems from the fact that lightness, defined as apparent reflectance, is underspecified and refers to three different types of judgments that are not comparable. Under stimulus conditions containing a visible illumination component, such as a shadow boundary, observers can distinguish and match three independent dimensions of achromatic experience: apparent intensity (brightness), apparent local intensity ratio (brightness-contrast), and apparent reflectance (lightness). In the absence of a visible illumination boundary, however, achromatic vision reduces to two dimensions and, depending on stimulus conditions and observer instructions, judgments of lightness are identical to judgments of brightness or brightness-contrast. Furthermore, because lightness judgments are based on different information under different conditions, they can differ greatly in their degree of difficulty and in their accuracy. This may, in part, explain the large variability in lightness constancy across studies.

The Oriented Difference of Gaussians (ODOG) model of brightness perception: Overview and executable Mathematica notebooks

The Oriented Difference of Gaussians (ODOG) model of brightness (perceived intensity) by Blakeslee and McCourt (Vision Research 39:4361-4377, 1999), which is based on linear spatial filtering by oriented receptive fields followed by contrast normalization, has proven highly successful in parsimoniously predicting the perceived intensity (brightness) of regions in complex visual stimuli such as White's effect, which had been believed to defy filter-based explanations. Unlike competing explanations such as anchoring theory, filling-in, edge-integration, or layer decomposition, the spatial filtering approach embodied by the ODOG model readily accounts for the often overlooked but ubiquitous gradient structure of induction which, while most striking in grating induction, also occurs within the test fields of classical simultaneous brightness contrast and the White stimulus. Also, because the ODOG model does not require defined regions of interest, it is generalizable to any stimulus, including natural images. The ODOG model has motivated other researchers to develop modified versions (LODOG and FLODOG), and has served as an important counterweight and proof of concept to constrain high-level theories which rely on less well understood or justified mechanisms such as unconscious inference, transparency, perceptual grouping, and layer decomposition. Here we provide a brief but comprehensive description of the ODOG model as it has been implemented since 1999, as well as working Mathematica (Wolfram, Inc.) notebooks which users can employ to generate ODOG model predictions for their own stimuli.

A unified account of perceptual layering and surface appearance in terms of gamut relativity

When we look at the world--or a graphical depiction of the world--we perceive surface materials (e.g. a ceramic black and white checkerboard) independently of variations in illumination (e.g. shading or shadow) and atmospheric media (e.g. clouds or smoke). Such percepts are partly based on the way physical surfaces and media reflect and transmit light and partly on the way the human visual system processes the complex patterns of light reaching the eye. One way to understand how these percepts arise is to assume that the visual system parses patterns of light into layered perceptual representations of surfaces, illumination and atmospheric media, one seen through another. Despite a great deal of previous experimental and modelling work on layered representation, however, a unified computational model of key perceptual demonstrations is still lacking. Here we present the first general computational model of perceptual layering and surface appearance--based on a boarder theoretical framework called gamut relativity--that is consistent with these demonstrations. The model (a) qualitatively explains striking effects of perceptual transparency, figure-ground separation and lightness, (b) quantitatively accounts for the role of stimulus- and task-driven constraints on perceptual matching performance, and (c) unifies two prominent theoretical frameworks for understanding surface appearance. The model thereby provides novel insights into the remarkable capacity of the human visual system to represent and identify surface materials, illumination and atmospheric media, which can be exploited in computer graphics applications.

The effect of background and illumination on color identification of real, 3D objects

For the surface reflectance of an object to be a useful cue to object identity, judgments of its color should remain stable across changes in the object's environment. In 2D scenes, there is general consensus that color judgments are much more stable across illumination changes than background changes. Here we investigate whether these findings generalize to real 3D objects. Observers made color matches to cubes as we independently varied both the illumination impinging on the cube and the 3D background of the cube. As in 2D scenes, we found relatively high but imperfect stability of color judgments under an illuminant shift. In contrast to 2D scenes, we found that background had little effect on average color judgments. In addition, variability of color judgments was increased by an illuminant shift and decreased by embedding the cube within a background. Taken together, these results suggest that in real 3D scenes with ample cues to object segregation, the addition of a background may improve stability of color identification.

Linking appearance to neural activity through the study of the perception of lightness in naturalistic contexts

The present paper deals with the classical question how a psychological experience, in this case apparent lightness, is linked by intervening neural processing to physical variables. We address two methodological issues: (a) how does one know the appropriate physical variable (what is the right x?) to look at, and (b) how can behavioral measurements be used to probe the internal transformation that leads to psychological experience. We measured so-called lightness transfer functions (LTFs), that is the functions that describe the mapping between retinal luminance and perceived lightness for naturalistic checkerboard stimuli. The LTFs were measured for different illumination situations: plain view, a cast shadow, and an intervening transparent medium. Observers adjusted the luminance of a comparison patch such that it had the same lightness as each of the test patches. When the data were plotted in luminance-luminance space, we found qualitative differences between mapping functions in different contexts. These differences were greatly diminished when the data were plotted in terms of contrast. On contrast-contrast coordinates, the data were compatible with a single linear generative model. This result is an indication that, for the naturalistic scenes used here, lightness perception depends mostly on local contrast. We further discuss that, in addition to the mean adjustments, one may find it useful to consider also the variability of an observer's adjustments in order to infer the true luminance-to-lightness mapping function.

When is spatial filtering enough? Investigation of brightness and lightness perception in stimuli containing a visible illumination component

Brightness (perceived intensity) and lightness (perceived reflectance) matching were investigated in seven well-known visual stimuli that contain a visible shadow or transparent overlay. These stimuli are frequently upheld as evidence that low-level spatial filtering is inadequate to explain brightness/lightness illusions and that additional mid- or high-level mechanisms are required. The argument in favor of rejecting low-level spatial filtering explanations has been founded on the erroneous assumption that equating test patch and near surround luminance is sufficient to control for and rule out this type of mechanism. We tested this idea by comparing the matching behavior of four observers to the predictions of the ODOG multiscale filtering model (Blakeslee & McCourt, 1999). Lightness and brightness matching differed significantly only when test patches appeared in shadow or beneath a transparency. Lightness and brightness matches were both significantly larger under these conditions; however, the lightness matches greatly exceeded the brightness matches. Lightness matches were greater for test patches in shadow or beneath a transparency because lightness matches under these conditions were based on conscious inferential (not sensory-level) judgments where observers attempted to discount the difference in illumination. The ODOG model accounted for approximately 80% of the total variance in the brightness matches (as well as in the lightness matches for targets not in shadow or beneath a transparency), and successfully predicted the relative magnitude of these matches in five of the seven stimulus sets. These results indicate that multiscale spatial filtering provides a unified and parsimonious explanation for brightness perception in these stimuli and imply that higher-level mechanisms are not required to explain them. The model was not as successful for the argyle and wall of blocks illusions in that it incorrectly rank-ordered the relative magnitude of the effects across different versions of the stimuli. It is an important question whether such model failures are due to known but corrigible limitations of the ODOG model or whether they will require other (possibly higher-level) explanations.

The dynamic range of human lightness perception

Natural viewing challenges the visual system with images that have a dynamic range of light intensity (luminance) that can approach 1,000,000:1 and that often exceeds 10,000:1 [1, 2]. The range of perceived surface reflectance (lightness), however, can be well approximated by the Munsell matte neutral scale (N 2.0/ to N 9.5/), consisting of surfaces whose reflectance varies by about 30:1. Thus, the visual system must map a large range of surface luminance onto a much smaller range of surface lightness. We measured this mapping in images with a dynamic range close to that of natural images. We studied simple images that lacked segmentation cues that would indicate multiple regions of illumination. We found a remarkable degree of compression: at a single image location, a stimulus luminance range of 5,905:1 can be mapped onto an extended lightness scale that has a reflectance range of 100:1. We characterized how the luminance-to-lightness mapping changes with stimulus context. Our data rule out theories that predict perceived lightness from luminance ratios or Weber contrast. A mechanistic model connects our data to theories of adaptation and provides insight about how the underlying visual response varies with context.

The human visual system's assumption that light comes from above is weak

Every biological or artificial visual system faces the problem that images are highly ambiguous, in the sense that every image depicts an infinite number of possible 3D arrangements of shapes, surface colors, and light sources. When estimating 3D shape from shading, the human visual system partly resolves this ambiguity by relying on the light-from-above prior, an assumption that light comes from overhead. However, light comes from overhead only on average, and most images contain visual information that contradicts the light-from-above prior, such as shadows indicating oblique lighting. How does the human visual system perceive 3D shape when there are contradictions between what it assumes and what it sees? Here we show that the visual system combines the light-from-above prior with visual lighting cues using an efficient statistical strategy that assigns a weight to the prior and to the cues and finds a maximum-likelihood lighting direction estimate that is a compromise between the two. The prior receives surprisingly little weight and can be overridden by lighting cues that are barely perceptible. Thus, the light-from-above prior plays a much more limited role in shape perception than previously thought, and instead human vision relies heavily on lighting cues to recover 3D shape. These findings also support the notion that the visual system efficiently integrates priors with cues to solve the difficult problem of recovering 3D shape from 2D images.

Task-irrelevant memory load induces inattentional blindness without temporo-parietal suppression

We often fail to consciously detect an unexpected object when we are engaged in an attention-demanding task (inattentional blindness). The inattentional blindness which is induced by visual short-term memory (VSTM) load has been proposed to result from a suppression of temporo-parietal junction (TPJ) activity that involves stimulus-driven attention. However, the fact that, inversely proportional to TPJ activity, intraparietal sulcus (IPS) activity correlates with VSTM load renders questionable the account of inattentional blindness based only on TPJ activity. Here, we investigated whether the TPJ is solely responsible for inattentional blindness by decoupling IPS and TPJ responses to VSTM load and then using the same manipulation to test the behavioral inattentional blindness performance. Experiment 1 showed that TPJ activity was not suppressed by task-irrelevant load while the IPS responded to both task-relevant and task-irrelevant load. Although the TPJ account of inattentional blindness predicts that the degree of inattentional blindness should track TPJ activity, we found in Experiment 2 that inattentional blindness was induced not only by task-relevant load but also by task-irrelevant load, showing inconsistency between the extent of inattentional blindness and TPJ response. These findings suggest that inattentional blindness can be induced without suppression of TPJ activity and seem to offer the possibility that the IPS contributes to conscious perception.

Contrast, constancy, and measurements of perceived lightness under parametric manipulation of surface slant and surface reflectance

Across many scenes, local contrast provides a valid cue to surface reflectance, but it is not the only such cue. To generalize beyond theories of lightness that rely exclusively on local contrast, we need to know which other potential cues matter. We had observers make lightness matches between two scene locations, and varied the surface slant and local surround reflectance of one of the locations. When local contrast was a valid cue to reflectance, all observers were approximately lightness constant. When it was not, observers' lightness matches were intermediate between contrast matching and lightness constancy. For most observers, surface slant exerted an effect on perceived lightness beyond that explainable by local contrast.

Filling-in and suppression of visual perception from context: a Bayesian account of perceptual biases by contextual influences

Visual object recognition and sensitivity to image features are largely influenced by contextual inputs. We study influences by contextual bars on the bias to perceive or infer the presence of a target bar, rather than on the sensitivity to image features. Human observers judged from a briefly presented stimulus whether a target bar of a known orientation and shape is present at the center of a display, given a weak or missing input contrast at the target location with or without a context of other bars. Observers are more likely to perceive a target when the context has a weaker rather than stronger contrast. When the context can perceptually group well with the would-be target, weak contrast contextual bars bias the observers to perceive a target relative to the condition without contexts, as if to fill in the target. Meanwhile, high-contrast contextual bars, regardless of whether they group well with the target, bias the observers to perceive no target. A Bayesian model of visual inference is shown to account for the data well, illustrating that the context influences the perception in two ways: (1) biasing observers' prior belief that a target should be present according to visual grouping principles, and (2) biasing observers' internal model of the likely input contrasts caused by a target bar. According to this model, our data suggest that the context does not influence the perceived target contrast despite its influence on the bias to perceive the target's presence, thereby suggesting that cortical areas beyond the primary visual cortex are responsible for the visual inferences.

Colour vision: cortical circuitry for appearance

Directly stimulating certain cortical neurons can produce a color sensation; a case is reported in which the color perceived by stimulation is the same as the color that most effectively excites the cortical circuitry.