Representation of Perceptual Color Space in Macaque Posterior Inferior Temporal Cortex (the V4 Complex)

Hue and orientation pinwheels in macaque area V4

Area V4 is an intermediate-level area of the macaque visual cortical hierarchy that serves key functions in visual processing by integrating inputs from lower areas such as V1 and V2 and providing feedforward inputs to many higher cortical areas. Previous V4 imaging studies have focused on differential responses to color, orientation, disparity, and motion stimuli, but many details of the spatial organization of significant hue and orientation tuning have not been fully described. We used support vector machine (SVM) decoding of intrinsic cortical single-condition responses to generate high-resolution maps of hue and orientation tuning and to describe the organization of hue and orientation pinwheels in V4. Like V1 and V2, V4 contains maps of orientation that are organized as pinwheels. V4 also contains maps of hue that are organized as pinwheels, whose circular organization more closely represents the perception of hue than is observed in antecedent cortical areas. Unlike V1, where orientation is continuously mapped across the surface, V4 hue and orientation pinwheels are organized in limited numbers of pinwheel sequences. The organization of these sequences and the distance between pinwheels may provide insight into the functional organization of V4. Regions significantly tuned for hue occupy roughly four times that of the orientation, are largely separated from each other, and overlap by roughly 5%. This spatial organization is largely consistent with segregated inputs arising from V2 thin and interstripes. This modular organization of V4 suggests that further integration of color and shape might occur in higher areas in inferotemporal cortical.NEW & NOTEWORTHY The representation of hue and orientation in macaque monkey area V4 was determined by intrinsic cortical imaging of responses to isoluminant hues and achromatic grating stimuli. Vector summation of support vector machine (SVM) decoded single-condition responses was used to generate hue and orientation maps that, like V1 orientation maps, were both characterized by distinct pinwheel patterns. These data suggest that pinwheels are an important structure to represent different stimulus features across multiple visual cortical areas.

Representational structures as a unifying framework for attention

Our visual system consciously processes only a subset of the incoming information. Selective attention allows us to prioritize relevant inputs, and can be allocated to features, locations, and objects. Recent advances in feature-based attention suggest that several selection principles are shared across these domains and that many differences between the effects of attention on perceptual processing can be explained by differences in the underlying representational structures. Moving forward, it can thus be useful to assess how attention changes the structure of the representational spaces over which it operates, which include the spatial organization, feature maps, and object-based coding in visual cortex. This will ultimately add to our understanding of how attention changes the flow of visual information processing more broadly.

Naturalistic Object Representations Depend on Distance and Size Cues

Egocentric distance and real-world size are important cues for object perception and action. Nevertheless, most studies of human vision rely on two-dimensional pictorial stimuli that convey ambiguous distance and size information. Here, we use fMRI to test whether pictures are represented differently in the human brain from real, tangible objects that convey unambiguous distance and size cues. Participants directly viewed stimuli in two display formats (real objects and matched printed pictures of those objects) presented at different egocentric distances (near and far). We measured the effects of format and distance on fMRI response amplitudes and response patterns. We found that fMRI response amplitudes in the lateral occipital and posterior parietal cortices were stronger overall for real objects than for pictures. In these areas and many others, including regions involved in action guidance, responses to real objects were stronger for near vs. far stimuli, whereas distance had little effect on responses to pictures-suggesting that distance determines relevance to action for real objects, but not for pictures. Although stimulus distance especially influenced response patterns in dorsal areas that operate in the service of visually guided action, distance also modulated representations in ventral cortex, where object responses are thought to remain invariant across contextual changes. We observed object size representations for both stimulus formats in ventral cortex but predominantly only for real objects in dorsal cortex. Together, these results demonstrate that whether brain responses reflect physical object characteristics depends on whether the experimental stimuli convey unambiguous information about those characteristics.

Significance Statement

Classic frameworks of vision attribute perception of inherent object characteristics, such as size, to the ventral visual pathway, and processing of spatial characteristics relevant to action, such as distance, to the dorsal visual pathway. However, these frameworks are based on studies that used projected images of objects whose actual size and distance from the observer were ambiguous. Here, we find that when object size and distance information in the stimulus is less ambiguous, these characteristics are widely represented in both visual pathways. Our results provide valuable new insights into the brain representations of objects and their various physical attributes in the context of naturalistic vision.

Pupil responses to colorfulness are selectively reduced in healthy older adults

The alignment between visual pathway signaling and pupil dynamics offers a promising non-invasive method to further illuminate the mechanisms of human color perception. However, only limited research has been done in this area and the effects of healthy aging on pupil responses to the different color components have not been studied yet. Here we aim to address this by modelling the effects of color lightness and chroma (colorfulness) on pupil responses in young and older adults, in a closely controlled passive viewing experiment with 26 broad-spectrum digital color fields. We show that pupil responses to color lightness and chroma are independent from each other in both young and older adults. Pupil responses to color lightness levels are unaffected by healthy aging, when correcting for smaller baseline pupil sizes in older adults. Older adults exhibit weaker pupil responses to chroma increases, predominantly along the Green-Magenta axis, while relatively sparing the Blue-Yellow axis. Our findings complement behavioral studies in providing physiological evidence that colors fade with age, with implications for color-based applications and interventions both in healthy aging and later-life neurodegenerative disorders.

Color appearance and the end of Hering's Opponent-Colors Theory

Hering's Opponent-Colors Theory has been central to understanding color appearance for 150 years. It aims to explain the phenomenology of colors with two linked propositions. First, a psychological hypothesis stipulates that any color is described necessarily and sufficiently by the extent to which it appears reddish-versus-greenish, bluish-versus-yellowish, and blackish-versus-whitish. Second, a physiological hypothesis stipulates that these perceptual mechanisms are encoded by three innate brain mechanisms. We review the evidence and conclude that neither side of the linking proposition is accurate: the theory is wrong. We sketch out an alternative, Utility-Based Coding, by which the known retinal cone-opponent mechanisms represent optimal encoding of spectral information given competing selective pressure to extract high-acuity spatial information; and phenomenological color categories represent an adaptive, efficient, output of the brain governed by behavioral demands.

It's not easy seeing green: The veridical perception of small spots

When single cones are stimulated with spots of 543-nm light presented against a white background, subjects report percepts that vary between predominately red, white, and green. However, light of the same spectral composition viewed over a large field under normal viewing conditions looks invariably green and highly saturated. It remains unknown what stimulus parameters are most important for governing the color appearance in the transition between these two extreme cases. The current study varied the size, intensity and retinal motion of stimuli presented in an adaptive optics scanning laser ophthalmoscope. Stimuli were either stabilized on target locations or allowed to drift across the retina with the eye's natural motion. Increasing both stimulus size and intensity led to higher likelihoods that monochromatic spots of light were perceived as green, whereas only higher intensities led to increases in perceived saturation. The data also show an interaction between size and intensity, suggesting that the balance between magnocellular and parvocellular activation may be critical factors for color perception. Surprisingly, under the range of conditions tested, color appearance did not depend on whether stimuli were stabilized. Sequential activation of many cones does not appear to drive hue and saturation perception as effectively as simultaneous activation of many cones.

Comparative connectomics reveals noncanonical wiring for color vision in human foveal retina

The Old World macaque monkey and New World common marmoset provide fundamental models for human visual processing, yet the human ancestral lineage diverged from these monkey lineages over 25 Mya. We therefore asked whether fine-scale synaptic wiring in the nervous system is preserved across these three primate families, despite long periods of independent evolution. We applied connectomic electron microscopy to the specialized foveal retina where circuits for highest acuity and color vision reside. Synaptic motifs arising from the cone photoreceptor type sensitive to short (S) wavelengths and associated with "blue-yellow" (S-ON and S-OFF) color-coding circuitry were reconstructed. We found that distinctive circuitry arises from S cones for each of the three species. The S cones contacted neighboring L and M (long- and middle-wavelength sensitive) cones in humans, but such contacts were rare or absent in macaques and marmosets. We discovered a major S-OFF pathway in the human retina and established its absence in marmosets. Further, the S-ON and S-OFF chromatic pathways make excitatory-type synaptic contacts with L and M cone types in humans, but not in macaques or marmosets. Our results predict that early-stage chromatic signals are distinct in the human retina and imply that solving the human connectome at the nanoscale level of synaptic wiring will be critical for fully understanding the neural basis of human color vision.

Feature-based attention warps the perception of visual features

Selective attention improves sensory processing of relevant information but can also impact the quality of perception. For example, attention increases visual discrimination performance and at the same time boosts apparent stimulus contrast of attended relative to unattended stimuli. Can attention also lead to perceptual distortions of visual representations? Optimal tuning accounts of attention suggest that processing is biased towards "off-tuned" features to maximize the signal-to-noise ratio in favor of the target, especially when targets and distractors are confusable. Here, we tested whether such tuning gives rise to phenomenological changes of visual features. We instructed participants to select a color among other colors in a visual search display and subsequently asked them to judge the appearance of the target color in a 2-alternative forced choice task. Participants consistently judged the target color to appear more dissimilar from the distractor color in feature space. Critically, the magnitude of these perceptual biases varied systematically with the similarity between target and distractor colors during search, indicating that attentional tuning quickly adapts to current task demands. In control experiments we rule out possible non-attentional explanations such as color contrast or memory effects. Overall, our results demonstrate that selective attention warps the representational geometry of color space, resulting in profound perceptual changes across large swaths of feature space. Broadly, these results indicate that efficient attentional selection can come at a perceptual cost by distorting our sensory experience.

Decoding of EEG signals reveals non-uniformities in the neural geometry of colour

The idea of colour opponency maintains that colour vision arises through the comparison of two chromatic mechanisms, red versus green and yellow versus blue. The four unique hues, red, green, blue, and yellow, are assumed to appear at the null points of these the two chromatic systems. Here we hypothesise that, if unique hues represent a tractable cortical state, they should elicit more robust activity compared to other, non-unique hues. We use a spatiotemporal decoding approach to report that electroencephalographic (EEG) responses carry robust information about the tested isoluminant unique hues within a 100-350 ms window from stimulus onset. Decoding is possible in both passive and active viewing tasks, but is compromised when concurrent high luminance contrast is added to the colour signals. For large hue-differences, the efficiency of hue decoding can be predicted by mutual distance in a nominally uniform perceptual colour space. However, for small perceptual neighbourhoods around unique hues, the encoding space shows pivotal non-uniformities which suggest that anisotropies in neurometric hue-spaces may reflect perceptual unique hues.

Perception of color in primates: A conceptual color neurons hypothesis

Perception of color by humans and other primates is a complex problem, studied by neurophysiology, psychophysiology, psycholinguistics, and even philosophy. Being mostly trichromats, simian primates have three types of opsin proteins, expressed in cone neurons in the eye, which allow for the sensing of color as the physical wavelength of light. Further, in neural networks of the retina, the coding principle changes from three types of sensor proteins to two opponent channels: activity of one type of neuron encode the evolutionarily ancient blue-yellow axis of color stimuli, and another more recent evolutionary channel, encoding the axis of red-green color stimuli. Both color channels are distinctive in neural organization at all levels from the eye to the neocortex, where it is thought that the perception of color (as philosophical qualia) emerges from the activity of some neuron ensembles. Here, using data from neurophysiology as a starting point, we propose a hypothesis on how the perception of color can be encoded in the activity of certain neurons in the neocortex. These conceptual neurons, herein referred to as 'color neurons', code only the hue of the color of visual stimulus, similar to place cells and number neurons, already described in primate brains. A case study with preliminary, but direct, evidence for existing conceptual color neurons in the human brain was published in 2008. We predict that the upcoming studies in non-human primates will be more extensive and provide a more detailed description of conceptual color neurons.

From cones to color vision: a neurobiological model that explains the unique hues

The irreducible unique hues-red, green, blue, and yellow-remain one of the great mysteries of vision science. Attempts to create a physiologically parsimonious model that can predict the spectral locations of the unique hues all rely on at least one post hoc adjustment to produce appropriate loci for unique green and unique red, and struggle to explain the non-linearity of the Blue/Yellow system. We propose a neurobiological color vision model that overcomes these challenges by using physiological cone ratios, cone-opponent normalization to equal-energy white, and a simple adaptation mechanism to produce color-opponent mechanisms that accurately predict the spectral locations and variability of the unique hues.

Color-biased regions in the ventral visual pathway are food selective

Color-biased regions have been found between face- and place-selective areas in the ventral visual pathway. To investigate the function of the color-biased regions in a pathway responsible for object recognition, we analyzed the natural scenes dataset (NSD), a large 7T fMRI dataset from 8 participants who each viewed up to 30,000 trials of images of colored natural scenes over more than 30 scanning sessions. In a whole-brain analysis, we correlated the average color saturation of the images with voxel responses, revealing color-biased regions that diverge into two streams, beginning in V4 and extending medially and laterally relative to the fusiform face area in both hemispheres. We drew regions of interest (ROIs) for the two streams and found that the images for each ROI that evoked the largest responses had certain characteristics: they contained food, circular objects, warmer hues, and had higher color saturation. Further analyses showed that food images were the strongest predictor of activity in these regions, implying the existence of medial and lateral ventral food streams (VFSs). We found that color also contributed independently to voxel responses, suggesting that the medial and lateral VFSs use both color and form to represent food. Our findings illustrate how high-resolution datasets such as the NSD can be used to disentangle the multifaceted contributions of many visual features to the neural representations of natural scenes.

Adaptive coding across visual features during free-viewing and fixation conditions

Theoretical studies have long proposed that adaptation allows the brain to effectively use the limited response range of sensory neurons to encode widely varying natural inputs. However, despite this influential view, experimental studies have exclusively focused on how the neural code adapts to a range of stimuli lying along a single feature axis, such as orientation or contrast. Here, we performed electrical recordings in macaque visual cortex (area V4) to reveal significant adaptive changes in the neural code of single cells and populations across multiple feature axes. Both during free viewing and passive fixation, populations of cells improved their ability to encode image features after rapid exposure to stimuli lying on orthogonal feature axes even in the absence of initial tuning to these stimuli. These results reveal a remarkable adaptive capacity of visual cortical populations to improve network computations relevant for natural viewing despite the modularity of the functional cortical architecture.

Exploring the role of structuralist methodology in the neuroscience of consciousness: a defense and analysis

Traditional contrastive analysis has been the foundation of consciousness science, but its limitations due to the lack of a reliable method for measuring states of consciousness have prompted the exploration of alternative approaches. Structuralist theories have gained attention as an alternative that focuses on the structural properties of phenomenal experience and seeks to identify their neural encoding via structural similarities between quality spaces and neural state spaces. However, the intertwining of philosophical assumptions about structuralism and structuralist methodology may pose a challenge to those who are skeptical of the former. In this paper, I offer an analysis and defense of structuralism as a methodological approach in consciousness science, which is partly independent of structuralist assumptions on the nature of consciousness. By doing so, I aim to make structuralist methodology more accessible to a broader scientific and philosophical audience. I situate methodological structuralism in the context of questions concerning mental representation, psychophysical measurement, holism, and functional relevance of neural processes. At last, I analyze the relationship between the structural approach and the distinction between conscious and unconscious states.

Temporal dynamics of the neural representation of hue and luminance polarity

Hue and luminance contrast are basic visual features. Here we use multivariate analyses of magnetoencephalography data to investigate the timing of the neural computations that extract them, and whether they depend on common neural circuits. We show that hue and luminance-contrast polarity can be decoded from MEG data and, with lower accuracy, both features can be decoded across changes in the other feature. These results are consistent with the existence of both common and separable neural mechanisms. The decoding time course is earlier and more temporally precise for luminance polarity than hue, a result that does not depend on task, suggesting that luminance contrast is an updating signal that separates visual events. Meanwhile, cross-temporal generalization is slightly greater for representations of hue compared to luminance polarity, providing a neural correlate of the preeminence of hue in perceptual grouping and memory. Finally, decoding of luminance polarity varies depending on the hues used to obtain training and testing data. The pattern of results is consistent with observations that luminance contrast is mediated by both L-M and S cone sub-cortical mechanisms.

Pop-out search instigates beta-gated feature selectivity enhancement across V4 layers

Visual search is a workhorse for investigating how attention interacts with processing of sensory information. Attentional selection has been linked to altered cortical sensory responses and feature preferences (i.e., tuning). However, attentional modulation of feature selectivity during search is largely unexplored. Here we map the spatiotemporal profile of feature selectivity during singleton search. Monkeys performed a search where a pop-out feature determined the target of attention. We recorded laminar neural responses from visual area V4. We first identified "feature columns" which showed preference for individual colors. In the unattended condition, feature columns were significantly more selective in superficial relative to middle and deep layers. Attending a stimulus increased selectivity in all layers but not equally. Feature selectivity increased most in the deep layers, leading to higher selectivity in extragranular layers as compared to the middle layer. This attention-induced enhancement was rhythmically gated in phase with the beta-band local field potential. Beta power dominated both extragranular laminar compartments, but current source density analysis pointed to an origin in superficial layers, specifically. While beta-band power was present regardless of attentional state, feature selectivity was only gated by beta in the attended condition. Neither the beta oscillation nor its gating of feature selectivity varied with microsaccade production. Importantly, beta modulation of neural activity predicted response times, suggesting a direct link between attentional gating and behavioral output. Together, these findings suggest beta-range synaptic activation in V4's superficial layers rhythmically gates attentional enhancement of feature tuning in a way that affects the speed of attentional selection.

Hemifield-specific color perception deficits after unilateral V4α lesions

Functional neuroimaging and patient studies demonstrated significant involvement of ventral area V4α, located in the anterior ventral pathway, in color vision. A low number of case studies reported lesions in close vicinity to this region leading to symptoms of hemiachromatopsia indicating hemifield-specific processing of color information. With the present study, we present the first group study investigating hemiachromatopsia after injury to anterior ventral brain areas. In lateral stimulus presentations with several color perception tasks, we observed symptoms of hemiachromatopsia, which were specific to patients with unilateral lesions to the ventral pathway. Particularly, we identified unilateral lesions to area V4α as an important contribution to color perception deficits under demanding viewing conditions. Our results suggest that color information processed along the anterior ventral path is hemifield-specific and that selective deficits in color perception cannot be fully compensated by the intact contralesional visual stream.

Visual Neuropsychology in Development: Anatomo-Functional Brain Mechanisms of Action/Perception Binding in Health and Disease

Vision is the main entrance for environmental input to the human brain. Even if vision is our most used sensory modality, its importance is not limited to environmental exploration. Rather it has strong links to motor competences, further extending to cognitive and social aspects of human life. These multifaceted relationships are particularly important in developmental age and become dramatically evident in presence of complex deficits originating from visual aberrancies. The present review summarizes the available neuropsychological evidence on the development of visual competences, with a particular focus on the associated visuo-motor integration skills in health and disease. With the aim of supporting future research and interventional settings, the goal of the present review is to constitute a solid base to help the translation of neuropsychological hypotheses into straightforward empirical investigations and rehabilitation/training protocols. This approach will further increase the impact, ameliorate the acceptance, and ease the use and implementation of lab-derived intervention protocols in real-life situations.

Color Tuning of Face-Selective Neurons in Macaque Inferior Temporal Cortex

What role does color play in the neural representation of complex shapes? We approached the question by measuring color responses of face-selective neurons, using fMRI-guided microelectrode recording of the middle and anterior face patches of inferior temporal cortex (IT) in rhesus macaques. Face-selective cells responded weakly to pure color (equiluminant) photographs of faces. But many of the cells nonetheless showed a bias for warm colors when assessed using images that preserved the luminance contrast relationships of the original photographs. This bias was also found for non-face-selective neurons. Fourier analysis uncovered two components: the first harmonic, accounting for most of the tuning, was biased toward reddish colors, corresponding to the L>M pole of the L-M cardinal axis. The second harmonic showed a bias for modulation between blue and yellow colors axis, corresponding to the S-cone axis. To test what role face-selective cells play in behavior, we related the information content of the neural population with the distribution of face colors. The analyses show that face-selective cells are not optimally tuned to discriminate face colors, but are consistent with the idea that face-selective cells contribute selectively to processing the green-red contrast of faces. The research supports the hypothesis that color-specific information related to the discrimination of objects, including faces, is handled by neural circuits that are independent of shape-selective cortex, as captured by the multistage parallel processing framework of IT (Lafer-Sousa and Conway, 2013).

Color Space Geometry Uncovered with Magnetoencephalography

The geometry that describes the relationship among colors, and the neural mechanisms that support color vision, are unsettled. Here, we use multivariate analyses of measurements of brain activity obtained with magnetoencephalography to reverse-engineer a geometry of the neural representation of color space. The analyses depend upon determining similarity relationships among the spatial patterns of neural responses to different colors and assessing how these relationships change in time. We evaluate the approach by relating the results to universal patterns in color naming. Two prominent patterns of color naming could be accounted for by the decoding results: the greater precision in naming warm colors compared to cool colors evident by an interaction of hue and lightness, and the preeminence among colors of reddish hues. Additional experiments showed that classifiers trained on responses to color words could decode color from data obtained using colored stimuli, but only at relatively long delays after stimulus onset. These results provide evidence that perceptual representations can give rise to semantic representations, but not the reverse. Taken together, the results uncover a dynamic geometry that provides neural correlates for color appearance and generates new hypotheses about the structure of color space.

A distinct population of heterogeneously color-tuned neurons in macaque visual cortex

Color is a key feature of natural environments that higher mammals routinely use to detect food, avoid predators, and interpret social signals. The distribution of color signals in natural scenes is widely variable, ranging from uniform patches to highly nonuniform regions in which different colors lie in close proximity. Whether individual neurons are tuned to this high degree of variability of color signals is unknown. Here, we identified a distinct population of cells in macaque visual cortex (area V4) that have a heterogeneous receptive field (RF) structure in which individual subfields are tuned to different colors even though the full RF is only weakly tuned. This spatial heterogeneity in color tuning indicates a higher degree of complexity of color-encoding mechanisms in visual cortex than previously believed to efficiently extract chromatic information from the environment.

Color Naming and Categorization Depend on Distinct Functional Brain Networks

Naming a color can be understood as an act of categorization, that is, identifying it as a member of a category of colors that are referred to by the same name. But are naming and categorization equivalent cognitive processes and consequently rely on same neural substrates? Here, we used task and resting-state functional magnetic resonance imaging as well as behavioral measures to identify functional brain networks that modulated naming and categorization of colors. We first identified three bilateral color-sensitive regions in the ventro-occipital cortex. We then showed that, across participants, color naming and categorization response times (RTs) were correlated with different resting state connectivity networks seeded from the color-sensitive regions. Color naming RTs correlated with the connectivity between the left posterior color region, the left middle temporal gyrus, and the left angular gyrus. In contrast, color categorization RTs correlated with the connectivity between the bilateral posterior color regions, and left frontal, right temporal and bilateral parietal areas. The networks supporting naming and categorization had a minimal overlap, indicating that the 2 processes rely on different neural mechanisms.

Color vision

Color is a fundamental aspect of normal visual experience. This chapter provides an overview of the role of color in human behavior, a survey of current knowledge regarding the genetic, retinal, and neural mechanisms that enable color vision, and a review of inherited and acquired defects of color vision including a discussion of diagnostic tests.

Testing the Cross-Cultural Generality of Hering's Theory of Color Appearance

This study examines the cross-cultural generality of Hering's (1878/1964) color-opponent theory of color appearance. English-speaking and Somali-speaking observers performed variants of two paradigms classically used to study color-opponency. First, both groups identified similar red, green, blue, and yellow unique hues. Second, 25 English-speaking and 34 Somali-speaking observers decomposed the colors present in 135 Munsell color samples into their component Hering elemental sensations-red,green,blue, yellow, white, and black-or else responded "no term." Both groups responded no term for many samples, notably purples. Somali terms for yellow were often used to name colors all around the color circle, including colors that are bluish according to Hering's theory. Four Somali Grue speakers named both green and blue elicitation samples by their term for green. However, that term did not name the union of all samples called blue or green by English speakers. A similar pattern was found among three Somali Achromatic speakers, who called the blue elicitation sample black or white. Thus, color decomposition by these Somali-speaking observers suggests a lexically influenced re-dimensionalization of color appearance space, rather than a simple reduction of the one proposed by Hering. Even some Somali Green-Blue speakers, whose data were otherwise similar to English, showed similar trends in yellow and blue usage. World Color Survey data mirror these results. These within- and cross-cultural violations of Hering's theory do not challenge the long-standing view that universal sensory processes mediate color appearance. However, they do demonstrate an important contribution of language in the human understanding of color.

Hierarchical Representation for Chromatic Processing across Macaque V1, V2, and V4

The perception of color is an internal label for the inferred spectral reflectance of visible surfaces. To study how spectral representation is transformed through modular subsystems of successive cortical areas, we undertook simultaneous optical imaging of intrinsic signals in macaque V1, V2, and V4, supplemented by higher-resolution electrophysiology and two-photon imaging in awake macaques. We find a progressive evolution in the scale and precision of chromotopic maps, expressed by a uniform blob-like architecture of hue responses within each area. Two-photon imaging reveals enhanced hue-specific cell clustering in V2 compared with V1. A phenomenon of endspectral (red and blue) responses that is clear in V1, recedes in V2, and is virtually absent in V4. The increase in mid- and extra-spectral hue representations through V2 and V4 reflects the nature of hierarchical processing as higher areas read out locations in chromatic space from progressive integration of signals relayed by V1.

Coherence of achromatic, primary and basic classes of colour categories

A range of explanations have been advanced for the systems of colour names found in different languages. Some explanations give special, fundamental status to a subset of colour categories. We argue that a subset of colour categories, if fundamental, will be coherent - meaning that a non-trivial criterion distinguishes them from the other colour categories. We test the coherence of subsets of achromatic (white, black and grey), primary (white, black, red, green, yellow, blue) and basic (primaries plus brown, orange, purple, pink and grey) colour categories in English. Criteria for defining colour categories were expressed in terms of behavioural, linguistic and geometric features derived from colour naming and linguistic usage data; and were discovered using machine learning methods. We find that achromatic and basic colour categories are coherent subsets but not primaries. These results support claims that the basic colour categories have special status, and undermine claims about the fundamental role of primaries in colour naming systems.

A neurodynamic model of the interaction between color perception and color memory

The memory color effect and Spanish castle illusion have been taken as evidence of the cognitive penetrability of vision. In the same manner, the successful decoding of color-related brain signals in functional neuroimaging studies suggests the retrieval of memory colors associated with a perceived gray object. Here, we offer an alternative account of these findings based on the design principles of adaptive resonance theory (ART). In ART, conscious perception is a consequence of a resonant state. Resonance emerges in a recurrent cortical circuit when a bottom-up spatial pattern agrees with the top-down expectation. When they do not agree, a special control mechanism is activated that resets the network and clears off erroneous expectation, thus allowing the bottom-up activity to always dominate in perception. We developed a color ART circuit and evaluated its behavior in computer simulations. The model helps to explain how traces of erroneous expectations about incoming color are eventually removed from the color perception, although their transient effect may be visible in behavioral responses or in brain imaging. Our results suggest that the color ART circuit, as a predictive computational system, is almost never penetrable, because it is equipped with computational mechanisms designed to constrain the impact of the top-down predictions on ongoing perceptual processing.

Build-up of serial dependence in color working memory

Serial dependence, how immediately preceding experiences bias our current estimations, has been described experimentally during delayed-estimation of many different visual features, with subjects tending to make estimates biased towards previous ones. It has been proposed that these attractive biases help perception stabilization in the face of correlated natural scene statistics, although this remains mostly theoretical. Color, which is strongly correlated in natural scenes, has never been studied with regard to its serial dependencies. Here, we found significant serial dependence in 7 out of 8 datasets with behavioral data of humans (total n = 760) performing delayed-estimation of color with uncorrelated sequential stimuli. Moreover, serial dependence strength built up through the experimental session, suggesting metaplastic mechanisms operating at a slower time scale than previously proposed (e.g. short-term synaptic facilitation). Because, in contrast with natural scenes, stimuli were temporally uncorrelated, this build-up casts doubt on serial dependencies being an ongoing adaptation to the stable statistics of the environment.

Multiplicative modulations enhance diversity of hue-selective cells

There is still much to understand about the brain's colour processing mechanisms and the transformation from cone-opponent representations to perceptual hues. Moreover, it is unclear which area(s) in the brain represent unique hues. We propose a hierarchical model inspired by the neuronal mechanisms in the brain for local hue representation, which reveals the contributions of each visual cortical area in hue representation. Hue encoding is achieved through incrementally increasing processing nonlinearities beginning with cone input. Besides employing nonlinear rectifications, we propose multiplicative modulations as a form of nonlinearity. Our simulation results indicate that multiplicative modulations have significant contributions in encoding of hues along intermediate directions in the MacLeod-Boynton diagram and that our model V2 neurons have the capacity to encode unique hues. Additionally, responses of our model neurons resemble those of biological colour cells, suggesting that our model provides a novel formulation of the brain's colour processing pathway.

Face Representations via Tensorfaces of Various Complexities

Neurons selective for faces exist in humans and monkeys. However, characteristics of face cell receptive fields are poorly understood. In this theoretical study, we explore the effects of complexity, defined as algorithmic information (Kolmogorov complexity) and logical depth, on possible ways that face cells may be organized. We use tensor decompositions to decompose faces into a set of components, called tensorfaces, and their associated weights, which can be interpreted as model face cells and their firing rates. These tensorfaces form a high-dimensional representation space in which each tensorface forms an axis of the space. A distinctive feature of the decomposition algorithm is the ability to specify tensorface complexity. We found that low-complexity tensorfaces have blob-like appearances crudely approximating faces, while high-complexity tensorfaces appear clearly face-like. Low-complexity tensorfaces require a larger population to reach a criterion face reconstruction error than medium- or high-complexity tensorfaces, and thus are inefficient by that criterion. Low-complexity tensorfaces, however, generalize better when representing statistically novel faces, which are faces falling beyond the distribution of face description parameters found in the tensorface training set. The degree to which face representations are parts based or global forms a continuum as a function of tensorface complexity, with low and medium tensorfaces being more parts based. Given the computational load imposed in creating high-complexity face cells (in the form of algorithmic information and logical depth) and in the absence of a compelling advantage to using high-complexity cells, we suggest face representations consist of a mixture of low- and medium-complexity face cells.

Communication efficiency of color naming across languages provides a new framework for the evolution of color terms

Languages vary in their number of color terms. A widely accepted theory proposes that languages evolve, acquiring color terms in a stereotyped sequence. This theory, by Berlin and Kay (BK), is supported by analyzing best exemplars ("focal colors") of basic color terms in the World Color Survey (WCS) of 110 languages. But the instructions of the WCS were complex and the color chips confounded hue and saturation, which likely impacted focal-color selection. In addition, it is now known that even so-called early-stage languages nonetheless have a complete representation of color distributed across the population. These facts undermine the BK theory. Here we revisit the evolution of color terms using original color-naming data obtained with simple instructions in Tsimane', an Amazonian culture that has limited contact with industrialized society. We also collected data in Bolivian-Spanish speakers and English speakers. We discovered that information theory analysis of color-naming data was not influenced by color-chip saturation, which motivated a new analysis of the WCS data. Embedded within a universal pattern in which warm colors (reds, oranges) are always communicated more efficiently than cool colors (blues, greens), as languages increase in overall communicative efficiency about color, some colors undergo greater increases in communication efficiency compared to others. Communication efficiency increases first for yellow, then brown, then purple. The present analyses and results provide a new framework for understanding the evolution of color terms: what varies among cultures is not whether colors are seen differently, but the extent to which color is useful.

What Cognitive Neurology Teaches Us about Our Experience of Color

Color provides valuable information about the environment, yet the exact mechanisms explaining how colors appear to us remain poorly understood. Retinal signals are processed in the visual cortex through high-level mechanisms that link color perception with top-down expectations and knowledge. Here, we review the neuroimaging evidence about color processing in the brain, and how it is affected by acquired brain lesions in humans. Evidence from patients with brain-damage suggests that high-level color processing may be divided into at least three modules: perceptual color experience, color naming, and color knowledge. These modules appear to be functionally independent but richly interconnected, and serve as cortical relays linking sensory and semantic information, with the final goal of directing object-related behavior. We argue that the relations between colors and their objects are key mechanisms to understand high-level color processing.

Visual working memory directly alters perception

Visual working memory (VWM), the ability to temporarily maintain and manipulate information, underlies a variety of critical high-level behaviours from directing attention^1-4 to making complex decisions⁵. Here we show that its impact extends to even the most basic levels of perceptual processing, directly interacting with and even distorting the physical appearance of visual features. This interference results from and can be predicted by the recruitment of posterior perceptual cortices to maintain information in VWM^6-9, which causes an overlap with the neuronal populations supporting perceptual processing^10-15. Across three sets of experiments, we demonstrated bidirectional interference between VWM and low-level perception. Specifically, for both maintained colours and orientations, presenting a distractor created bias in VWM representation depending on the similarity between incoming and maintained information, consistent with the known tuning curves for these features. Moreover, holding an item in mind directly altered the appearance of new stimuli, demonstrated by changes in psychophysical discrimination thresholds. Thus, as a consequence of sharing the early visual cortices, what you see and what you are holding in mind are intertwined at even the most fundamental stages of processing.

How Does Spatial Attention Influence the Probability and Fidelity of Colour Perception?

Existing research has found that spatial attention alters how various stimulus properties are perceived (e.g., luminance, saturation), but few have explored whether it improves the accuracy of perception. To address this question, we performed two experiments using modified Posner cueing tasks, wherein participants made speeded detection responses to peripheral colour targets and then indicated their perceived colours on a colour wheel. In E1, cues were central and endogenous (i.e., prompted voluntary attention) and the interval between cues and targets (stimulus onset asynchrony, or SOA) was always 800 ms. In E2, cues were peripheral and exogenous (i.e., captured attention involuntarily) and the SOA varied between short (100 ms) and long (800 ms). A Bayesian mixed-model analysis was used to isolate the effects of attention on the probability and the fidelity of colour encoding. Both endogenous and short-SOA exogenous spatial cueing improved the probability of encoding the colour of targets. Improved fidelity of encoding was observed in the endogenous but not in the exogenous cueing paradigm. With exogenous cues, inhibition of return (IOR) was observed in both RT and probability at the long SOA. Overall, our findings reinforce the utility of continuous response variables in the research of attention.

The spectral identity of foveal cones is preserved in hue perception

Organisms are faced with the challenge of making inferences about the physical world from incomplete incoming sensory information. One strategy to combat ambiguity in this process is to combine new information with prior experiences. We investigated the strategy of combining these information sources in color vision. Single cones in human subjects were stimulated and the associated percepts were recorded. Subjects rated each flash for brightness, hue, and saturation. Brightness ratings were proportional to stimulus intensity. Saturation was independent of intensity, but varied between cones. Hue, in contrast, was assigned in a stereotyped manner that was predicted by cone type. These experiments revealed that, near the fovea, long and middle wavelength sensitive cones produce sensations that can be reliably distinguished on the basis of hue, but not saturation or brightness. Taken together, these observations implicate the high-resolution, color-opponent parvocellular pathway in this low-level visual task.

Color statistics of objects, and color tuning of object cortex in macaque monkey

We hypothesized that the parts of scenes identified by human observers as “objects” show distinct color properties from backgrounds, and that the brain uses this information towards object recognition. To test this hypothesis, we examined the color statistics of naturally and artificially colored objects and backgrounds in a database of over 20,000 images annotated with object labels. Objects tended to be warmer colored (L-cone response > M-cone response) and more saturated compared to backgrounds. That the distinguishing chromatic property of objects was defined mostly by the L-M post-receptoral mechanism, rather than the S mechanism, is consistent with the idea that trichromatic color vision evolved in response to a selective pressure to identify objects. We also show that classifiers trained using only color information could distinguish animate versus inanimate objects, and at a performance level that was comparable to classification using shape features. Animate/inanimate is considered a fundamental superordinate category distinction, previously thought to be computed by the brain using only shape information. Our results show that color could contribute to animate/inanimate, and likely other, object-category assignments. Finally, color-tuning measured in two macaque monkeys with functional magnetic resonance imaging (fMRI), and confirmed by fMRI-guided microelectrode recording, supports the idea that responsiveness to color reflects the global functional organization of inferior temporal cortex, the brain region implicated in object vision. More strongly in IT than in V1, colors associated with objects elicited higher responses than colors less often associated with objects.

A tour of contemporary color vision research

The study of color vision encompasses many disciplines, including art, biochemistry, biophysics, brain imaging, cognitive neuroscience, color preferences, colorimetry, computer modelling, design, electrophysiology, language and cognition, molecular genetics, neuroscience, physiological optics, psychophysics and physiological optics. Coupled with the elusive nature of the subjective experience of color, this wide range of disciplines makes the study of color as challenging as it is fascinating. This overview of the special issue Color: Cone Opponency and Beyond outlines the state of the science of color, and points to some of the many questions that remain to be answered in this exciting field.

The Organization and Operation of Inferior Temporal Cortex

Inferior temporal cortex (IT) is a key part of the ventral visual pathway implicated in object, face, and scene perception. But how does IT work? Here, I describe an organizational scheme that marries form and function and provides a framework for future research. The scheme consists of a series of stages arranged along the posterior-anterior axis of IT, defined by anatomical connections and functional responses. Each stage comprises a complement of subregions that have a systematic spatial relationship. The organization of each stage is governed by an eccentricity template, and corresponding eccentricity representations across stages are interconnected. Foveal representations take on a role in high-acuity object vision (including face recognition); intermediate representations compute other aspects of object vision such as behavioral valence (using color and surface cues); and peripheral representations encode information about scenes. This multistage, parallel-processing model invokes an innately determined organization refined by visual experience that is consistent with principles of cortical development. The model is also consistent with principles of evolution, which suggest that visual cortex expanded through replication of retinotopic areas. Finally, the model predicts that the most extensively studied network within IT-the face patches-is not unique but rather one manifestation of a canonical set of operations that reveal general principles of how IT works.

Long-wavelength (reddish) hues induce unusually large gamma oscillations in the primate primary visual cortex

Electrophysiological signals recorded from the brain exhibit prominent gamma oscillations (∼30–80 Hz) under sensory stimulation. These oscillations are modulated by stimulus properties and behavioral state, and implicated to play a role in cognitive functions, such as attention and object-binding. In visual areas, gamma oscillations have mainly been studied using achromatic gratings/gabors. Here we show that color stimuli generate unprecedented levels of gamma oscillations in the local field potentials recorded from primate area V1. The strongest oscillations are induced by long-wavelength (reddish) hues. Importantly, their strength depends on color saturation but not on luminance, and is strongly correlated with the L−M cone contrast produced by stimuli, suggesting that gamma oscillations may represent key components of the processing of visual chromatic information.

Gamma oscillations (∼30–80 Hz) are a prominent signature of electrophysiological signals, with a purported role in natural vision. Previous studies in the primary visual cortex (area V1) have shown that achromatic gratings or gabor stimuli generate salient gamma oscillations, whose strength and frequency depend on stimulus properties such as their size, contrast, and orientation. Surprisingly, although natural images are rarely achromatic, the effect of chromatic input on gamma has not been thoroughly investigated. Recording from primate V1, we show that gamma oscillations of extremely high magnitude (peak increase of ∼300-fold in some cases), far exceeding the gamma generated by optimally tuned achromatic gratings, are induced in the local field potentials by full-field color stimuli of different hues. Furthermore, gamma oscillations are sensitive to the hue of the chromatic input, with the strongest oscillations for long-wavelength (reddish) hues and another, smaller gamma response peak for hues in the short-wavelength (bluish) range, which lie approximately on the two cardinal chromatic response axes of the upstream lateral geniculate nucleus neurons. These oscillations depended critically on the purity of the hue, decreasing with hue desaturation, but remained robust for pure hue stimuli even at reduced luminance. Importantly, the magnitude of gamma oscillations was highly correlated with positive L−M cone contrast produced by the stimuli, suggesting that gamma could be a marker of the specific mechanisms underlying this computation. These findings provide insights into the generation of gamma oscillations, as well as the processing of color along the visual pathway.

Consciousness, Representation, Action: The Importance of Being Goal-Directed

Recent years have witnessed fierce debates on the dependence of consciousness on interactions between a subject and the environment. Reviewing neuroscientific, computational, and clinical evidence, I will address three questions. First, does conscious experience necessarily depend on acute interactions between a subject and the environment? Second, does it depend on specific perception-action loops in the longer run? Third, which types of action does consciousness cohere with, if not with all of them? I argue that conscious contents do not necessarily depend on acute or long-term brain-environment interactions. Instead, consciousness is proposed to be specifically associated with, and subserve, deliberate, goal-directed behavior (GDB). Brain systems implied in conscious representation are highly connected to, but distinct from, neural substrates mediating GDB and declarative memory.

Measurements of neuronal color tuning: Procedures, pitfalls, and alternatives

Measuring the color tuning of visual neurons is important for understanding the neural basis of vision, but it is challenging because of the inherently three-dimensional nature of color. Color tuning cannot be represented by a one-dimensional curve, and measuring three-dimensional tuning curves is difficult. One approach to addressing this challenge is to analyze neuronal color tuning data through the lens of mathematical models that make assumptions about the shapes of tuning curves. In this paper, we discuss the linear-nonlinear cascade model as a platform for measuring neuronal color tuning. We compare fitting this model by three techniques: two using response-weighted averaging and one using numerical optimization of likelihood. We highlight the advantages and disadvantages of each technique and emphasize the effects of the stimulus distribution on color tuning measurements.

Color naming across languages reflects color use

What determines how languages categorize colors? We analyzed results of the World Color Survey (WCS) of 110 languages to show that despite gross differences across languages, communication of chromatic chips is always better for warm colors (yellows/reds) than cool colors (blues/greens). We present an analysis of color statistics in a large databank of natural images curated by human observers for salient objects and show that objects tend to have warm rather than cool colors. These results suggest that the cross-linguistic similarity in color-naming efficiency reflects colors of universal usefulness and provide an account of a principle (color use) that governs how color categories come about. We show that potential methodological issues with the WCS do not corrupt information-theoretic analyses, by collecting original data using two extreme versions of the color-naming task, in three groups: the Tsimane', a remote Amazonian hunter-gatherer isolate; Bolivian-Spanish speakers; and English speakers. These data also enabled us to test another prediction of the color-usefulness hypothesis: that differences in color categorization between languages are caused by differences in overall usefulness of color to a culture. In support, we found that color naming among Tsimane' had relatively low communicative efficiency, and the Tsimane' were less likely to use color terms when describing familiar objects. Color-naming among Tsimane' was boosted when naming artificially colored objects compared with natural objects, suggesting that industrialization promotes color usefulness.

Modeling color preference using color space metrics

Studying color preferences provides a means to discover how perceptual experiences map onto cognitive and affective judgments. A challenge is finding a parsimonious way to describe and predict patterns of color preferences, which are complex with rich individual differences. One approach has been to model color preferences using factors from metric color spaces to establish direct correspondences between dimensions of color and preference. Prior work established that substantial, but not all, variance in color preferences could be captured by weights on color space dimensions using multiple linear regression. The question we address here is whether model fits may be improved by using different color metric specifications. We therefore conducted a large-scale analysis of color space models, and focused in-depth analysis on models that differed in color space (cone-contrast vs. CIELAB), coordinate system within the color space (Cartesian vs. cylindrical), and factor degrees (1st degree only, or 1st and 2nd degree). We used k-fold cross validation to avoid over-fitting the data and to ensure fair comparisons across models. The best model was the 2nd-harmonic Lch model ("LabC Cyl2"). Specified in CIELAB space, it included 1st and 2nd harmonics of hue (capturing opponency in hue preferences and simultaneous liking/disliking of both hues on an opponent axis, respectively), lightness, and chroma. These modeling approaches can be used to characterize and compare patterns for group averages and individuals in future datasets on color preference, or other measures in which correspondences between color appearance and cognitive or affective judgments may exist.

Color-Change Detection Activity in the Primate Superior Colliculus

The primate superior colliculus (SC) is a midbrain structure that participates in the control of spatial attention. Previous studies examining the role of the SC in attention have mostly used luminance-based visual features (e.g., motion, contrast) as the stimuli and saccadic eye movements as the behavioral response, both of which are known to modulate the activity of SC neurons. To explore the limits of the SC’s involvement in the control of spatial attention, we recorded SC neuronal activity during a task using color, a visual feature dimension not traditionally associated with the SC, and required monkeys to detect threshold-level changes in the saturation of a cued stimulus by releasing a joystick during maintained fixation. Using this color-based spatial attention task, we found substantial cue-related modulation in all categories of visually responsive neurons in the intermediate layers of the SC. Notably, near-threshold changes in color saturation, both increases and decreases, evoked phasic bursts of activity with magnitudes as large as those evoked by stimulus onset. This change-detection activity had two distinctive features: activity for hits was larger than for misses, and the timing of change-detection activity accounted for 67% of joystick release latency, even though it preceded the release by at least 200 ms. We conclude that during attention tasks, SC activity denotes the behavioral relevance of the stimulus regardless of feature dimension and that phasic event-related SC activity is suitable to guide the selection of manual responses as well as saccadic eye movements.

A neural signature of the unique hues

Since at least the 17^th century there has been the idea that there are four simple and perceptually pure “unique” hues: red, yellow, green, and blue, and that all other hues are perceived as mixtures of these four hues. However, sustained scientific investigation has not yet provided solid evidence for a neural representation that separates the unique hues from other colors. We measured event-related potentials elicited from unique hues and the ‘intermediate’ hues in between them. We find a neural signature of the unique hues 230 ms after stimulus onset at a post-perceptual stage of visual processing. Specifically, the posterior P2 component over the parieto-occipital lobe peaked significantly earlier for the unique than for the intermediate hues (Z = −2.9, p = 0.004). Having identified a neural marker for unique hues, fundamental questions about the contribution of neural hardwiring, language and environment to the unique hues can now be addressed.