Perceptual color map in macaque visual area V4

Theory of morphodynamic information processing: Linking sensing to behaviour

The traditional understanding of brain function has predominantly focused on chemical and electrical processes. However, new research in fruit fly (Drosophila) binocular vision reveals ultrafast photomechanical photoreceptor movements significantly enhance information processing, thereby impacting a fly's perception of its environment and behaviour. The coding advantages resulting from these mechanical processes suggest that similar physical motion-based coding strategies may affect neural communication ubiquitously. The theory of neural morphodynamics proposes that rapid biomechanical movements and microstructural changes at the level of neurons and synapses enhance the speed and efficiency of sensory information processing, intrinsic thoughts, and actions by regulating neural information in a phasic manner. We propose that morphodynamic information processing evolved to drive predictive coding, synchronising cognitive processes across neural networks to match the behavioural demands at hand effectively.

Orientation selectivity mapping in the visual cortex

The orientation map is one of the most well-studied functional maps of the visual cortex. However, results from the literature are of different qualities. Clear boundaries among different orientation domains and blurred uncertain distinctions were shown in different studies. These unclear imaging results will lead to an inaccuracy in depicting cortical structures, and the lack of consideration in experimental design will also lead to biased depictions of the cortical features. How we accurately define orientation domains will impact the entire field of research. In this study, we test how spatial frequency (SF), stimulus size, location, chromatic, and data processing methods affect the orientation functional maps (including a large area of dorsal V4, and parts of dorsal V1) acquired by intrinsic signal optical imaging. Our results indicate that, for large imaging fields, large grating stimuli with mixed SF components should be considered to acquire the orientation map. A diffusion model image enhancement based on the difference map could further improve the map quality. In addition, the similar outcomes of achromatic and chromatic gratings indicate two alternative types of afferents from LGN, pooling in V1 to generate cue-invariant orientation selectivity.

Hue and orientation pinwheels in macaque area V4

Area V4 is an intermediate-level area of the macaque visual cortical hierarchy that serves key functions in visual processing by integrating inputs from lower areas such as V1 and V2 and providing feedforward inputs to many higher cortical areas. Previous V4 imaging studies have focused on differential responses to color, orientation, disparity, and motion stimuli, but many details of the spatial organization of significant hue and orientation tuning have not been fully described. We used support vector machine (SVM) decoding of intrinsic cortical single-condition responses to generate high-resolution maps of hue and orientation tuning and to describe the organization of hue and orientation pinwheels in V4. Like V1 and V2, V4 contains maps of orientation that are organized as pinwheels. V4 also contains maps of hue that are organized as pinwheels, whose circular organization more closely represents the perception of hue than is observed in antecedent cortical areas. Unlike V1, where orientation is continuously mapped across the surface, V4 hue and orientation pinwheels are organized in limited numbers of pinwheel sequences. The organization of these sequences and the distance between pinwheels may provide insight into the functional organization of V4. Regions significantly tuned for hue occupy roughly four times that of the orientation, are largely separated from each other, and overlap by roughly 5%. This spatial organization is largely consistent with segregated inputs arising from V2 thin and interstripes. This modular organization of V4 suggests that further integration of color and shape might occur in higher areas in inferotemporal cortical.NEW & NOTEWORTHY The representation of hue and orientation in macaque monkey area V4 was determined by intrinsic cortical imaging of responses to isoluminant hues and achromatic grating stimuli. Vector summation of support vector machine (SVM) decoded single-condition responses was used to generate hue and orientation maps that, like V1 orientation maps, were both characterized by distinct pinwheel patterns. These data suggest that pinwheels are an important structure to represent different stimulus features across multiple visual cortical areas.

Mesoscale organization of ventral and dorsal visual pathways in macaque monkey revealed by 7T fMRI

In human and nonhuman primate brains, columnar (mesoscale) organization has been demonstrated to underlie both lower and higher order aspects of visual information processing. Previous studies have focused on identifying functional preferences of mesoscale domains in specific areas; but there has been little understanding of how mesoscale domains may cooperatively respond to single visual stimuli across dorsal and ventral pathways. Here, we have developed ultrahigh-field 7 T fMRI methods to enable simultaneous mapping, in individual macaque monkeys, of response in both dorsal and ventral pathways to single simple color and motion stimuli. We provide the first evidence that anatomical V2 cytochrome oxidase-stained stripes are well aligned with fMRI maps of V2 stripes, settling a long-standing controversy. In the ventral pathway, a systematic array of paired color and luminance processing domains across V4 was revealed, suggesting a novel organization for surface information processing. In the dorsal pathway, in addition to high quality motion direction maps of MT, MST and V3A, alternating color and motion direction domains in V3 are revealed. As well, submillimeter motion domains were observed in peripheral LIPd and LIPv. In sum, our study provides a novel global snapshot of how mesoscale networks in the ventral and dorsal visual pathways form the organizational basis of visual objection recognition and vision for action.

The effect of memory load on object reconstruction: Insights from an online mouse-tracking task

Why can't we remember everything that we experience? Previous work in the domain of object memory has suggested that our ability to resolve interference between relevant and irrelevant object features may limit how much we can remember at any given moment. Here, we developed an online mouse-tracking task to study how memory load influences object reconstruction, testing participants synchronously over virtual conference calls. We first tested up to 18 participants concurrently, replicating memory findings from a condition where participants were tested individually. Next, we examined how memory load influenced mouse trajectories as participants reconstructed target objects. We found interference between the contents of working memory and what was perceived during object reconstruction, an effect that interacted with visual similarity and memory load. Furthermore, we found interference from previously studied but currently irrelevant objects, providing evidence of object-to-location binding errors. At the greatest memory load, participants were nearly three times more likely to move their mouse cursor over previously studied nontarget objects, an effect observed primarily during object reconstruction rather than in the period before the final response. As evidence of the dynamic interplay between working memory and perception, these results show that object reconstruction behavior may be altered by (i) interference between what is represented in mind and what is currently being viewed, and (ii) interference from previously studied but currently irrelevant information. Finally, we discuss how mouse tracking can provide a rich characterization of participant behavior at millisecond temporal resolution, enormously increasing power in cognitive psychology experiments.

Pattern forming mechanisms of color vision

While our understanding of the way single neurons process chromatic stimuli in the early visual pathway has advanced significantly in recent years, we do not yet know how these cells interact to form stable representations of hue. Drawing on physiological studies, we offer a dynamical model of how the primary visual cortex tunes for color, hinged on intracortical interactions and emergent network effects. After detailing the evolution of network activity through analytical and numerical approaches, we discuss the effects of the model's cortical parameters on the selectivity of the tuning curves. In particular, we explore the role of the model's thresholding nonlinearity in enhancing hue selectivity by expanding the region of stability, allowing for the precise encoding of chromatic stimuli in early vision. Finally, in the absence of a stimulus, the model is capable of explaining hallucinatory color perception via a Turing-like mechanism of biological pattern formation.

Perception of color in primates: A conceptual color neurons hypothesis

Perception of color by humans and other primates is a complex problem, studied by neurophysiology, psychophysiology, psycholinguistics, and even philosophy. Being mostly trichromats, simian primates have three types of opsin proteins, expressed in cone neurons in the eye, which allow for the sensing of color as the physical wavelength of light. Further, in neural networks of the retina, the coding principle changes from three types of sensor proteins to two opponent channels: activity of one type of neuron encode the evolutionarily ancient blue-yellow axis of color stimuli, and another more recent evolutionary channel, encoding the axis of red-green color stimuli. Both color channels are distinctive in neural organization at all levels from the eye to the neocortex, where it is thought that the perception of color (as philosophical qualia) emerges from the activity of some neuron ensembles. Here, using data from neurophysiology as a starting point, we propose a hypothesis on how the perception of color can be encoded in the activity of certain neurons in the neocortex. These conceptual neurons, herein referred to as 'color neurons', code only the hue of the color of visual stimulus, similar to place cells and number neurons, already described in primate brains. A case study with preliminary, but direct, evidence for existing conceptual color neurons in the human brain was published in 2008. We predict that the upcoming studies in non-human primates will be more extensive and provide a more detailed description of conceptual color neurons.

Developmental neuronal origin regulates neocortical map formation

Sensory neurons in the neocortex exhibit distinct functional selectivity to constitute the neural map. While neocortical map of the visual cortex in higher mammals is clustered, it displays a striking "salt-and-pepper" pattern in rodents. However, little is known about the origin and basis of the interspersed neocortical map. Here we report that the intricate excitatory neuronal kinship-dependent synaptic connectivity influences precise functional map organization in the mouse primary visual cortex. While sister neurons originating from the same neurogenic radial glial progenitors (RGPs) preferentially develop synapses, cousin neurons derived from amplifying RGPs selectively antagonize horizontal synapse formation. Accordantly, cousin neurons in similar layers exhibit clear functional selectivity differences, contributing to a salt-and-pepper architecture. Removal of clustered protocadherins (cPCDHs), the largest subgroup of the diverse cadherin superfamily, eliminates functional selectivity differences between cousin neurons and alters neocortical map organization. These results suggest that developmental neuronal origin regulates neocortical map formation via cPCDHs.

Fundamentally different representations of color and motion revealed by individual differences in perceptual scaling

The coordinate frames for color and motion are often defined by three dimensions (e.g., responses from the three types of human cone photoreceptors for color and the three dimensions of space for motion). Does this common dimensionality lead to similar perceptual representations? Here we show that the organizational principles for the representation of hue and motion direction are instead profoundly different. We compared observers' judgments of hue and motion direction using functionally equivalent stimulus metrics, behavioral tasks, and computational analyses, and used the pattern of individual differences to decode the underlying representational structure for these features. Hue judgments were assessed using a standard "hue-scaling" task (i.e., judging the proportion of red/green and blue/yellow in each hue). Motion judgments were measured using a "motion-scaling" task (i.e., judging the proportion of left/right and up/down motion in moving dots). Analyses of the interobserver variability in hue scaling revealed multiple independent factors limited to different local regions of color space. This is inconsistent with the influences across a broad range of hues predicted by conventional color-opponent models. In contrast, variations in motion scaling were characterized by more global factors plausibly related to variation in the relative weightings of the cardinal spatial axes. These results suggest that although the coordinate frames for specifying color and motion share a common dimensional structure, the perceptual coding principles for hue and motion direction are distinct. These differences might reflect a distinction between the computational strategies required for the visual analysis of spatial vs. nonspatial attributes of the world.

Spatial frequency representation in V2 and V4 of macaque monkey

Spatial frequency (SF) is an important attribute in the visual scene and is a defining feature of visual processing channels. However, there remain many unsolved questions about how extrastriate areas in primate visual cortex code this fundamental information. Here, using intrinsic signal optical imaging in visual areas of V2 and V4 of macaque monkeys, we quantify the relationship between SF maps and (1) visual topography and (2) color and orientation maps. We find that in orientation regions, low to high SF is mapped orthogonally to orientation; in color regions, which are reported to contain orthogonal axes of color and lightness, low SFs tend to be represented more frequently than high SFs. This supports a population-based SF fluctuation related to the 'color/orientation' organizations. We propose a generalized hypercolumn model across cortical areas, comprised of two orthogonal parameters with additional parameters.

The Paton prize lecture 2021: A colourful experience leading to a reassessment of colour vision and its theories

In this lecture, given in honour of Sir William Paton, a brilliant scientist and one of Britain's great patrons of biology, I give a personal account of the fundamental issues in colour vision that I have tackled since 1973, when I discovered a cortical zone lying outside the primary visual cortex that is rich in cells with chromatic properties. I do not provide an exhaustive review of colour vision but summarise how my views on colour vision and theories surrounding it have changed in light of that discovery.

Representation of Cone-Opponent Color Space in Macaque Early Visual Cortices

In primate vision, the encoding of color perception arises from three types of retinal cone cells (L, M, and S cones). The inputs from these cones are linearly integrated into two cone-opponent channels (cardinal axes) before the lateral geniculate nucleus. In subsequent visual cortical stages, color-preferring neurons cluster into functional domains within "blobs" in V1, "thin/color stripes" in V2, and "color bands" in V4. Here, we hypothesize that, with increasing cortical hierarchy, the functional organization of hue representation becomes more balanced and less dependent on cone opponency. To address this question, we used intrinsic signal optical imaging in macaque V1, V2, and V4 cortices to examine the domain-based representation of specific hues (here referred to as "hue domains") in cone-opponent color space (4 cardinal and 4 intermediate hues). Interestingly, we found that in V1, the relative size of S-cone hue preference domain was significantly smaller than that for other hues. This notable difference was less prominent in V2, and, in V4 was virtually absent, resulting in a more balanced representation of hues. In V2, hue clusters contained sequences of shifting preference, while in V4 the organization of hue clusters was more complex. Pattern classification analysis of these hue maps showed that accuracy of hue classification improved from V1 to V2 to V4. These results suggest that hue representation by domains in the early cortical hierarchy reflects a transformation away from cone-opponency and toward a full-coverage representation of hue.

Color discrimination thresholds vary throughout color space in a reef fish (Rhinecanthus aculeatus)

Animal use color vision in a range of behaviours. Visual performance is limited by thresholds, which are set by noise in photoreceptors and subsequent neural processing. The receptor noise limited (RNL) model of color discrimination is widely used for modelling color vision and accounts well for experimental data from many species. In one of the most comprehensive tests yet of color discrimination in a non-human species, we using Ishihara-style stimulus patterns to examine thresholds for 21 directions at five locations in color space for the fish Rhineacanthus aculeatus. Thresholds matched RNL model predictions most closely for stimuli near to the the achromatic point, but exceeded predictions (indicating a decline in sensitivity) with distance from this point. Thresholds were also usually higher for saturation than for hue differences. These changes in color threshold with color space location and direction may give insight into photoreceptor non-linearities and post-receptoral mechanisms of color vision in fish. Our results highlight the need for a cautious interpretation of the RNL model - especially for modelling colours that differ from one another in saturation (rather than hue), and especially for highly saturated colours distant from the achromatic point in colour space.

Perceptual hue, lightness, and chroma are represented in a multidimensional functional anatomical map in macaque V1

Humans perceive millions of colors along three dimensions of color space: hue, lightness, and chroma. A major gap in knowledge is where the brain represents these specific dimensions in cortex, and how they relate to each other. Previous studies have shown that brain areas V4 and the posterior inferotemporal cortex (PIT) are central to computing color dimensions. To determine the contribution of V1 to setting up these downstream processing mechanisms, we studied cortical color responses in macaques-who share color vision mechanisms with humans. We used two-photon calcium imaging at both meso- and micro-scales and found that hue and lightness are laid out in orthogonal directions on the cortical map, with chroma represented by the strength of neuronal responses, as previously shown in PIT. These findings suggest that the earliest cortical stages of vision determine the three primary dimensions of human color perception.

Color Modulates Feature Integration

Bayesian models of object recognition propose the resolution of ambiguity through probabilistic integration of prior experience with available sensory information. Color, even when task-irrelevant, has been shown to modulate high-level cognitive control tasks. However, it remains unclear how color modulations affect lower-level perceptual processing. We investigated whether color affects feature integration using the flash-jump illusion. This illusion occurs when an apparent motion stimulus, a rectangular bar appearing at different locations along a motion trajectory, changes color at a single position. Observers misperceive this color change as occurring farther along the trajectory of motion. This mislocalization error is proposed to be produced by a Bayesian perceptual framework dependent on responses in area V4. Our results demonstrated that the color of the flash modulated the magnitude of the flash-jump illusion such that participants reported less of a shift, i.e., a more veridical flash location, for both red and blue flashes, as compared to green and yellow. Our findings extend color-dependent modulation effects found in higher-order executive functions into lower-level Bayesian perceptual processes. Our results also support the theory that feature integration is a Bayesian process. In this framework, color modulations play an inherent and automatic role as different colors have different weights in Bayesian perceptual processing.

Clustered functional domains for curves and corners in cortical area V4

The ventral visual pathway is crucially involved in integrating low-level visual features into complex representations for objects and scenes. At an intermediate stage of the ventral visual pathway, V4 plays a crucial role in supporting this transformation. Many V4 neurons are selective for shape segments like curves and corners; however, it remains unclear whether these neurons are organized into clustered functional domains, a structural motif common across other visual cortices. Using two-photon calcium imaging in awake macaques, we confirmed and localized cortical domains selective for curves or corners in V4. Single-cell resolution imaging confirmed that curve- or corner-selective neurons were spatially clustered into such domains. When tested with hexagonal-segment stimuli, we find that stimulus smoothness is the cardinal difference between curve and corner selectivity in V4. Combining cortical population responses with single-neuron analysis, our results reveal that curves and corners are encoded by neurons clustered into functional domains in V4. This functionally specific population architecture bridges the gap between the early and late cortices of the ventral pathway and may serve to facilitate complex object recognition.

Toward the development of a color visual prosthesis

Objective. All of the human prosthetic visual systems implanted so far have been achromatic. Schmidtet al(1996Brain119507-22) reported that at low stimulation intensities their subject reported that phosphenes usually had a specific hue, but when the stimulus intensity was increased, they desaturated to white. We speculate here that previous B/W prosthetic systems were unnecessarily over-stimulating the visual cortex to obtain white phosphenes, which may be why unexpected alterations in phosphenes and seizures were not an uncommon occurrence. A color prosthesis would have the advantage of being elicited by lower levels of stimulation, reducing the probability of causing epileptogenic responses.Approach.A 'hybrid' mode of stimulation is suggested, involving a combination of B/W and color stimulation, which could provide color information without reducing spatial resolution.Main results.Colors in the real world are spread along intensity and chromatic gradients.Significance.Software implementation strategies are discussed, as are the advantages and challenges for possible color prosthetic systems.

A distinct population of heterogeneously color-tuned neurons in macaque visual cortex

Color is a key feature of natural environments that higher mammals routinely use to detect food, avoid predators, and interpret social signals. The distribution of color signals in natural scenes is widely variable, ranging from uniform patches to highly nonuniform regions in which different colors lie in close proximity. Whether individual neurons are tuned to this high degree of variability of color signals is unknown. Here, we identified a distinct population of cells in macaque visual cortex (area V4) that have a heterogeneous receptive field (RF) structure in which individual subfields are tuned to different colors even though the full RF is only weakly tuned. This spatial heterogeneity in color tuning indicates a higher degree of complexity of color-encoding mechanisms in visual cortex than previously believed to efficiently extract chromatic information from the environment.

Curvature-processing domains in primate V4

Neurons in primate V4 exhibit various types of selectivity for contour shapes, including curves, angles, and simple shapes. How are these neurons organized in V4 remains unclear. Using intrinsic signal optical imaging and two-photon calcium imaging, we observed submillimeter functional domains in V4 that contained neurons preferring curved contours over rectilinear ones. These curvature domains had similar sizes and response amplitudes as orientation domains but tended to separate from these regions. Within the curvature domains, neurons that preferred circles or curve orientations clustered further into finer scale subdomains. Nevertheless, individual neurons also had a wide range of contour selectivity, and neighboring neurons exhibited a substantial diversity in shape tuning besides their common shape preferences. In strong contrast to V4, V1 and V2 did not have such contour-shape-related domains. These findings highlight the importance and complexity of curvature processing in visual object recognition and the key functional role of V4 in this process.

Hierarchical Representation for Chromatic Processing across Macaque V1, V2, and V4

The perception of color is an internal label for the inferred spectral reflectance of visible surfaces. To study how spectral representation is transformed through modular subsystems of successive cortical areas, we undertook simultaneous optical imaging of intrinsic signals in macaque V1, V2, and V4, supplemented by higher-resolution electrophysiology and two-photon imaging in awake macaques. We find a progressive evolution in the scale and precision of chromotopic maps, expressed by a uniform blob-like architecture of hue responses within each area. Two-photon imaging reveals enhanced hue-specific cell clustering in V2 compared with V1. A phenomenon of endspectral (red and blue) responses that is clear in V1, recedes in V2, and is virtually absent in V4. The increase in mid- and extra-spectral hue representations through V2 and V4 reflects the nature of hierarchical processing as higher areas read out locations in chromatic space from progressive integration of signals relayed by V1.

A neurodynamic model of the interaction between color perception and color memory

The memory color effect and Spanish castle illusion have been taken as evidence of the cognitive penetrability of vision. In the same manner, the successful decoding of color-related brain signals in functional neuroimaging studies suggests the retrieval of memory colors associated with a perceived gray object. Here, we offer an alternative account of these findings based on the design principles of adaptive resonance theory (ART). In ART, conscious perception is a consequence of a resonant state. Resonance emerges in a recurrent cortical circuit when a bottom-up spatial pattern agrees with the top-down expectation. When they do not agree, a special control mechanism is activated that resets the network and clears off erroneous expectation, thus allowing the bottom-up activity to always dominate in perception. We developed a color ART circuit and evaluated its behavior in computer simulations. The model helps to explain how traces of erroneous expectations about incoming color are eventually removed from the color perception, although their transient effect may be visible in behavioral responses or in brain imaging. Our results suggest that the color ART circuit, as a predictive computational system, is almost never penetrable, because it is equipped with computational mechanisms designed to constrain the impact of the top-down predictions on ongoing perceptual processing.

Neural representations of perceptual color experience in the human ventral visual pathway

Color is a perceptual construct that arises from neural processing in hierarchically organized cortical visual areas. Previous research, however, often failed to distinguish between neural responses driven by stimulus chromaticity versus perceptual color experience. An unsolved question is whether the neural responses at each stage of cortical processing represent a physical stimulus or a color we see. The present study dissociated the perceptual domain of color experience from the physical domain of chromatic stimulation at each stage of cortical processing by using a switch rivalry paradigm that caused the color percept to vary over time without changing the retinal stimulation. Using functional MRI (fMRI) and a model-based encoding approach, we found that neural representations in higher visual areas, such as V4 and VO1, corresponded to the perceived color, whereas responses in early visual areas V1 and V2 were modulated by the chromatic light stimulus rather than color perception. Our findings support a transition in the ascending human ventral visual pathway, from a representation of the chromatic stimulus at the retina in early visual areas to responses that correspond to perceptually experienced colors in higher visual areas.

Multiplicative modulations enhance diversity of hue-selective cells

There is still much to understand about the brain's colour processing mechanisms and the transformation from cone-opponent representations to perceptual hues. Moreover, it is unclear which area(s) in the brain represent unique hues. We propose a hierarchical model inspired by the neuronal mechanisms in the brain for local hue representation, which reveals the contributions of each visual cortical area in hue representation. Hue encoding is achieved through incrementally increasing processing nonlinearities beginning with cone input. Besides employing nonlinear rectifications, we propose multiplicative modulations as a form of nonlinearity. Our simulation results indicate that multiplicative modulations have significant contributions in encoding of hues along intermediate directions in the MacLeod-Boynton diagram and that our model V2 neurons have the capacity to encode unique hues. Additionally, responses of our model neurons resemble those of biological colour cells, suggesting that our model provides a novel formulation of the brain's colour processing pathway.

Reward-Related Suppression of Neural Activity in Macaque Visual Area V4

In order for organisms to survive, they need to detect rewarding stimuli, for example, food or a mate, in a complex environment with many competing stimuli. These rewarding stimuli should be detected even if they are nonsalient or irrelevant to the current goal. The value-driven theory of attentional selection proposes that this detection takes place through reward-associated stimuli automatically engaging attentional mechanisms. But how this is achieved in the brain is not very well understood. Here, we investigate the effect of differential reward on the multiunit activity in visual area V4 of monkeys performing a perceptual judgment task. Surprisingly, instead of finding reward-related increases in neural responses to the perceptual target, we observed a large suppression at the onset of the reward indicating cues. Therefore, while previous research showed that reward increases neural activity, here we report a decrease. More suppression was caused by cues associated with higher reward than with lower reward, although neither cue was informative about the perceptually correct choice. This finding of reward-associated neural suppression further highlights normalization as a general cortical mechanism and is consistent with predictions of the value-driven attention theory.

Unique objects attract attention even when faint

Locally contrasting objects, e.g. a red apple surrounded by green apples, attract attention. Does this generalize to differences in feature space? That is, do unique objects-regardless of their location-stand out from a collection of objects that are similar to one another, even when the unique object has lower local contrast with the background than the other objects? Behavioral data show indeed a preference for unique items but previous experiments enabled viewers to anticipate what response they were "supposed" to give. We developed a new experimental paradigm that minimizes such top-down effects. Pitting local contrast against global uniqueness, we show that unique stimuli attract attention even in not-anticipated, never-seen images, and even when the unique stimuli are faint (low contrast). A computational model explains how competition between objects in feature space favors dissimilar objects over those with similar features. The model explains how humans select unique objects, without a loss of performance on natural scenes.

Invariance of surface color representations across illuminant changes in the human cortex

A central problem in color vision is that the light reaching the eye from a given surface can vary dramatically depending on the illumination. Despite this, our color percept, the brain's estimate of surface reflectance, remains remarkably stable. This phenomenon is called color constancy. Here we investigated which human brain regions represent surface color in a way that is invariant with respect to illuminant changes. We used physically realistic rendering methods to display natural yet abstract 3D scenes that were displayed under three distinct illuminants. The scenes embedded, in different conditions, surfaces that differed in their surface color (i.e. in their reflectance property). We used multivariate fMRI pattern analysis to probe neural coding of surface reflectance and illuminant, respectively. While all visual regions encoded surface color when viewed under the same illuminant, we found that only in V1 and V4α surface color representations were invariant to illumination changes. Along the visual hierarchy there was a gradient from V1 to V4α to increasingly encode surface color rather than illumination. Finally, effects of a stimulus manipulation on individual behavioral color constancy indices correlated with neural encoding of the illuminant in hV4. This provides neural evidence for the Equivalent Illuminant Model. Our results provide a principled characterization of color constancy mechanisms across the visual hierarchy, and demonstrate complementary contributions in early and late processing stages.

Enhancing spatio-chromatic representation with more-than-three color coding for image description

The extraction of spatio-chromatic features from color images is usually performed independently on each color channel. Usual 3D color spaces, such as RGB, present a high inter-channel correlation for natural images. This correlation can be reduced using color-opponent representations, but the spatial structure of regions with small color differences is not fully captured in two generic Red-Green and Blue-Yellow channels. To overcome these problems, we propose new color coding that is adapted to the specific content of each image. Our proposal is based on two steps: (a) setting the number of channels to the number of distinctive colors we find in each image (avoiding the problem of channel correlation), and (b) building a channel representation that maximizes contrast differences within each color channel (avoiding the problem of low local contrast). We call this approach more-than-three color coding (MTT) to emphasize the fact that the number of channels is adapted to the image content. The higher the color complexity of an image, the more channels can be used to represent it. Here we select distinctive colors as the most predominant in the image, which we call color pivots, and we build the new color coding strategy using these color pivots as a basis. To evaluate the proposed approach, we measure the efficiency in an image categorization task. We show how a generic descriptor improves performance at the description level when applied to the MTT coding.

Representation of Perceptual Color Space in Macaque Posterior Inferior Temporal Cortex (the V4 Complex)

The lateral geniculate nucleus is thought to represent color using two populations of cone-opponent neurons [L vs M; S vs (L + M)], which establish the cardinal directions in color space (reddish vs cyan; lavender vs lime). How is this representation transformed to bring about color perception? Prior work implicates populations of glob cells in posterior inferior temporal cortex (PIT; the V4 complex), but the correspondence between the neural representation of color in PIT/V4 complex and the organization of perceptual color space is unclear. We compared color-tuning data for populations of glob cells and interglob cells to predictions obtained using models that varied in the color-tuning narrowness of the cells, and the color preference distribution across the populations. Glob cells were best accounted for by simulated neurons that have nonlinear (narrow) tuning and, as a population, represent a color space designed to be perceptually uniform (CIELUV). Multidimensional scaling and representational similarity analyses showed that the color space representations in both glob and interglob populations were correlated with the organization of CIELUV space, but glob cells showed a stronger correlation. Hue could be classified invariant to luminance with high accuracy given glob responses and above-chance accuracy given interglob responses. Luminance could be read out invariant to changes in hue in both populations, but interglob cells tended to prefer stimuli having luminance contrast, regardless of hue, whereas glob cells typically retained hue tuning as luminance contrast was modulated. The combined luminance/hue sensitivity of glob cells is predicted for neurons that can distinguish two colors of the same hue at different luminance levels (orange/brown).

Multisensory integration and cross-modal learning in synaesthesia: A unifying model

Recent research into synaesthesia has highlighted the role of learning, yet synaesthesia is clearly a genetic condition. Here we ask how can the idea that synaesthesia reflects innate, genetic differences be reconciled with models that suggest it is driven by learning. A number of lines of evidence suggest that synaesthesia relies on, or at least interacts with, processes of multisensory integration that are common across all people. These include multisensory activations that arise in early regions of the brain as well as feedback from longer-term cross-modal associations generated in memory. These cognitive processes may interact independently to influence the phenomenology of the synaesthetic experience, as well as the individual differences within particular types of synaesthesia. The theoretical framework presented here is consistent with both an innate difference as the fundamental driver of the condition of synaesthesia, and with experiential and semantic influences on the eventual phenotype that emerges. In particular, it proposes that the internally generated synaesthetic percepts are treated similarly to other sensory information as the brain is learning the multisensory attributes of objects and developing cross-modal associations that merge in the concept of the object.

Progressive thinning of visual cortex in primary open-angle glaucoma of varying severity

The aim of this study was to investigate possible changes of cortical thickness in the visual cortex in primary open-angle glaucoma (POAG) of varying severity. Twenty normal controls (NC), 20 mild (MP) and 17 severe (SP) POAG patients were recruited and scanned using magnetic resonance imaging. Cortical thickness analyses with regions of interest (V1, V2, ventral V3, V4 and V5/MT+) were used to assess the cortical changes among the three groups. Furthermore, the associations of cortical thickness with retinal nerve fiber layer (RNFL) thickness and mean deviation of visual field were analyzed. Compared with the NC group, decreased cortical thickness was detected in the bilateral V5/MT+ areas in the MP group and the left V1, bilateral V2 and V5/MT+ areas in the SP group. Cortical thinning of the bilateral V2 areas was detected in the SP group compared with the MP group. In addition, cortical thinning of these visual areas was related to the ophthalmologic measurements. In conclusion, POAG patients exhibit cortical thinning in the bilateral V5/MT+ in the early stage of disease. The cortical degeneration in visual areas is discrepant with disease progressing and the dorsal pathway might be selectively damaged in POAG. Therefore, the cortical thinning of these visual areas may play a key role in the progression of POAG and can serve as a novel biomarker for accurately evaluating the severity of POAG.

Stimulus competition mediates the joint effects of spatial and feature-based attention

Distinct attentional mechanisms enhance the sensory processing of visual stimuli that appear at task-relevant locations and have task-relevant features. We used a combination of psychophysics and computational modeling to investigate how these two types of attention--spatial and feature based--interact to modulate sensitivity when combined in one task. Observers monitored overlapping groups of dots for a target change in color saturation, which they had to localize as being in the upper or lower visual hemifield. Pre-cues indicated the target's most likely location (left/right), color (red/green), or both location and color. We measured sensitivity (d') for every combination of the location cue and the color cue, each of which could be valid, neutral, or invalid. When three competing saturation changes occurred simultaneously with the target change, there was a clear interaction: The spatial cueing effect was strongest for the cued color, and the color cueing effect was strongest at the cued location. In a second experiment, only the target dot group changed saturation, such that stimulus competition was low. The resulting cueing effects were statistically independent and additive: The color cueing effect was equally strong at attended and unattended locations. We account for these data with a computational model in which spatial and feature-based attention independently modulate the gain of sensory responses, consistent with measurements of cortical activity. Multiple responses then compete via divisive normalization. Sufficient competition creates interactions between the two cueing effects, although the attentional systems are themselves independent. This model helps reconcile seemingly disparate behavioral and physiological findings.