Evolution of neural computations: Mantis shrimp and human color decoding

Hue selectivity from recurrent circuitry in Drosophila

In the perception of color, wavelengths of light reflected off objects are transformed into the derived quantities of brightness, saturation and hue. Neurons responding selectively to hue have been reported in primate cortex, but it is unknown how their narrow tuning in color space is produced by upstream circuit mechanisms. We report the discovery of neurons in the Drosophila optic lobe with hue-selective properties, which enables circuit-level analysis of color processing. From our analysis of an electron microscopy volume of a whole Drosophila brain, we construct a connectomics-constrained circuit model that accounts for this hue selectivity. Our model predicts that recurrent connections in the circuit are critical for generating hue selectivity. Experiments using genetic manipulations to perturb recurrence in adult flies confirm this prediction. Our findings reveal a circuit basis for hue selectivity in color vision.

Color Perspectives in Aquatic Explorations: Unveiling Innate Color Preferences and Psychoactive Responses in Freshwater Crayfish

Color preference assay is a test for an animal's innate and adaptive response to differentiate colors and can be used as an endpoint for psychoactive activity evaluation. Several color preference test methods in aquatic animals that can be used to perform behavioral screening have been established. However, the color preference test conditions have yet to be extensively studied and standardized in aquatic invertebrates. This study aimed to replicate and optimize the previously published method to evaluate the potential color preference in freshwater crayfish based on four different approaches: species, life stages, sex, and pharmaceutical exposure. Using the optimized setup, two crayfish species display color preferences to some specific colors. P. clarkii displays more dominant color preference behavior than C. quadricarinatus in terms of color preference ranking and index. P. clarkii prefers the red color compared to other colors (red > green > blue > yellow), while C. quadricarinatus dislikes yellow compared to other colors (blue = green = red > yellow). Since P. clarkii has a more obvious color index ranking and several advantages compared to C. quadricarinatus, we conducted further tests using P. clarkii as an animal model. In the juvenile and adult stages of P. clarkii, they prefer red and avoid yellow. However, the juvenile one did not display a strong color preference like the adult one. Different sex of crayfish displayed no significant differences in their color preference responses. In addition, we also evaluated the potential effect of the antidepressant sertraline on color preference in P. clarkii and found that waterborne antidepressant exposure can significantly alter their color preference. This fundamental information collected from this study supports the crayfish color preference test as a good behavioral test to address environmental pollution.

Gaining the system: limits to compensating color deficiencies through post-receptoral gain changes

Color percepts of anomalous trichromats are often more similar to normal trichromats than predicted from their receptor spectral sensitivities, suggesting that post-receptoral mechanisms can compensate for chromatic losses. The basis for these adjustments and the extent to which they could discount the deficiency are poorly understood. We modeled the patterns of compensation that might result from increasing the gains in post-receptoral neurons to offset their weakened inputs. Individual neurons and the population responses jointly encode luminance and chromatic signals. As a result, they cannot independently adjust for a change in the chromatic inputs, predicting only partial recovery of the chromatic responses and increased responses to achromatic contrast. These analyses constrain the potential sites and mechanisms of compensation for a color loss and characterize the utility and limits of neural gain changes for calibrating color vision.

Colour vision in stomatopod crustaceans

The stomatopod crustaceans, or mantis shrimps, are colourful marine invertebrate predators. Their unusual compound eyes have dorsal and ventral regions resembling typical crustacean apposition designs separated by a unique region called the midband that consists of from two to six parallel rows of ommatidia. In species with six-row midbands, the dorsal four rows are themselves uniquely specialized for colour analysis. Rhabdoms of ommatidia in these rows are longitudinally divided into three distinct regions: an apical ultraviolet (UV) receptor, a shorter-wavelength middle tier receptor and a longer-wavelength proximal tier receptor. Each of the total of 12 photoreceptors has a different spectral sensitivity, potentially contributing to a colour-vision system with 12 channels. Mantis shrimps can discriminate both human-visible and UV colours, but with limited precision compared to other colour-vision systems. Here, we review the structure and function of stomatopod colour vision, examining the types of receptors present in a species, the spectral tuning of photoreceptors both within and across species, the neural analysis of colour and the genetics underlying the multiple visual pigments used for colour vision. Even today, after many decades of research into the colour vision of stomatopods, much of its operation and its use in nature remain a mystery. This article is part of the theme issue 'Understanding colour vision: molecular, physiological, neuronal and behavioural studies in arthropods'.

Colour vision in stomatopod crustaceans: more questions than answers

Stomatopod crustaceans, or mantis shrimps, are known for their extensive range of spectral sensitivities but relatively poor spectral discrimination. Instead of the colour-opponent mechanism of other colour vision systems, the 12 narrow-band colour channels they possess may underlie a different method of colour processing. We investigated one hypothesis, in which the photoreceptors are proposed to act as individual wave-band detectors, interpreting colour as a parallel pattern of photoreceptor activation, rather than a ratiometric comparison of individual signals. This different form of colour detection has been used to explain previous behavioural tests in which low saturation blue was not discriminated from grey, potentially because of similar activation patterns. Results here, however, indicate that the stomatopod, Haptosquilla trispinosa was able to easily distinguish several colours, including blue of both high and low saturation, from greys. The animals did show a decrease in performance over time in an artificially lit environment, indicating plasticity in colour discrimination ability. This rapid plasticity, most likely the result of a change in opsin (visual pigment) expression, has now been noted in several animal lineages (both invertebrate and vertebrate) and is a factor we suggest needing care and potential re-examination in any colour-based behavioural tests. As for stomatopods, it remains unclear why they achieve poor colour discrimination using the most comprehensive set of spectral sensitivities in the animal kingdom and also what form of colour processing they may utilise.

Color Space Geometry Uncovered with Magnetoencephalography

The geometry that describes the relationship among colors, and the neural mechanisms that support color vision, are unsettled. Here, we use multivariate analyses of measurements of brain activity obtained with magnetoencephalography to reverse-engineer a geometry of the neural representation of color space. The analyses depend upon determining similarity relationships among the spatial patterns of neural responses to different colors and assessing how these relationships change in time. We evaluate the approach by relating the results to universal patterns in color naming. Two prominent patterns of color naming could be accounted for by the decoding results: the greater precision in naming warm colors compared to cool colors evident by an interaction of hue and lightness, and the preeminence among colors of reddish hues. Additional experiments showed that classifiers trained on responses to color words could decode color from data obtained using colored stimuli, but only at relatively long delays after stimulus onset. These results provide evidence that perceptual representations can give rise to semantic representations, but not the reverse. Taken together, the results uncover a dynamic geometry that provides neural correlates for color appearance and generates new hypotheses about the structure of color space.

A neurodynamic model of the interaction between color perception and color memory

The memory color effect and Spanish castle illusion have been taken as evidence of the cognitive penetrability of vision. In the same manner, the successful decoding of color-related brain signals in functional neuroimaging studies suggests the retrieval of memory colors associated with a perceived gray object. Here, we offer an alternative account of these findings based on the design principles of adaptive resonance theory (ART). In ART, conscious perception is a consequence of a resonant state. Resonance emerges in a recurrent cortical circuit when a bottom-up spatial pattern agrees with the top-down expectation. When they do not agree, a special control mechanism is activated that resets the network and clears off erroneous expectation, thus allowing the bottom-up activity to always dominate in perception. We developed a color ART circuit and evaluated its behavior in computer simulations. The model helps to explain how traces of erroneous expectations about incoming color are eventually removed from the color perception, although their transient effect may be visible in behavioral responses or in brain imaging. Our results suggest that the color ART circuit, as a predictive computational system, is almost never penetrable, because it is equipped with computational mechanisms designed to constrain the impact of the top-down predictions on ongoing perceptual processing.

Multiplicative modulations enhance diversity of hue-selective cells

There is still much to understand about the brain's colour processing mechanisms and the transformation from cone-opponent representations to perceptual hues. Moreover, it is unclear which area(s) in the brain represent unique hues. We propose a hierarchical model inspired by the neuronal mechanisms in the brain for local hue representation, which reveals the contributions of each visual cortical area in hue representation. Hue encoding is achieved through incrementally increasing processing nonlinearities beginning with cone input. Besides employing nonlinear rectifications, we propose multiplicative modulations as a form of nonlinearity. Our simulation results indicate that multiplicative modulations have significant contributions in encoding of hues along intermediate directions in the MacLeod-Boynton diagram and that our model V2 neurons have the capacity to encode unique hues. Additionally, responses of our model neurons resemble those of biological colour cells, suggesting that our model provides a novel formulation of the brain's colour processing pathway.

Communication efficiency of color naming across languages provides a new framework for the evolution of color terms

Languages vary in their number of color terms. A widely accepted theory proposes that languages evolve, acquiring color terms in a stereotyped sequence. This theory, by Berlin and Kay (BK), is supported by analyzing best exemplars ("focal colors") of basic color terms in the World Color Survey (WCS) of 110 languages. But the instructions of the WCS were complex and the color chips confounded hue and saturation, which likely impacted focal-color selection. In addition, it is now known that even so-called early-stage languages nonetheless have a complete representation of color distributed across the population. These facts undermine the BK theory. Here we revisit the evolution of color terms using original color-naming data obtained with simple instructions in Tsimane', an Amazonian culture that has limited contact with industrialized society. We also collected data in Bolivian-Spanish speakers and English speakers. We discovered that information theory analysis of color-naming data was not influenced by color-chip saturation, which motivated a new analysis of the WCS data. Embedded within a universal pattern in which warm colors (reds, oranges) are always communicated more efficiently than cool colors (blues, greens), as languages increase in overall communicative efficiency about color, some colors undergo greater increases in communication efficiency compared to others. Communication efficiency increases first for yellow, then brown, then purple. The present analyses and results provide a new framework for understanding the evolution of color terms: what varies among cultures is not whether colors are seen differently, but the extent to which color is useful.

Geometrical structure of perceptual color space: Mental representations and adaptation invariance

Similarity between percepts and concepts is used to accomplish many everyday tasks, e.g., object identification; so this similarity is widely used to construct geometrical spaces that represent stimulus qualities, but the intrinsic validity of the geometry, i.e., whether similarity operations support a particular geometry, is almost never tested critically. We introduce an experimental approach for equating relative similarities by setting perceived midpoints between pairs of stimuli. Midpoint settings are used with Varignon's Theorem to test the intrinsic geometry of a representation space, and its mapping to a physical space of stimuli. For perceptual color space, we demonstrate that geometrical structure depends on the mental representation used in judging similarity: An affine geometry was valid when observers used an opponent-color mental representation. Similarities based on a conceptual space of complementary colors thus power a geometric coordinate system. An affine geometry implies that similarity can be judged within straight lines and across parallel lines, and its neural coding could involve ratios of responses. We show that this perceptual space is invariant to changes in illumination color, providing a formal justification to generalize color constancy results measured for color categories, to all of color space. The midpoint measurements deviate significantly from midpoints in the extensively used "uniform" color spaces CIELAB and CIELUV, showing that these spaces do not provide adequate metric representation of perceived colors. Our paradigm can thus test for intrinsic geometrical assumptions underlying the representation space for many perceptual modalities, and for the extrinsic perceptual geometry of the space of physical stimuli.

Neuronal Adaptation Reveals a Suboptimal Decoding of Orientation Tuned Populations in the Mouse Visual Cortex

Sensory information is encoded by populations of cortical neurons. Yet, it is unknown how this information is used for even simple perceptual choices such as discriminating orientation. To determine the computation underlying this perceptual choice, we took advantage of the robust visual adaptation in mouse primary visual cortex (V1). We first designed a stimulus paradigm in which we could vary the degree of neuronal adaptation measured in V1 during an orientation discrimination task. We then determined how adaptation affects task performance for mice of both sexes and tested which neuronal computations are most consistent with the behavioral results given the adapted population responses in V1. Despite increasing the reliability of the population representation of orientation among neurons, and improving the ability of a variety of optimal decoders to discriminate target from distractor orientations, adaptation increases animals' behavioral thresholds. Decoding the animals' choice from neuronal activity revealed that this unexpected effect on behavior could be explained by an overreliance of the perceptual choice circuit on target preferring neurons and a failure to appropriately discount the activity of neurons that prefer the distractor. Consistent with this all-positive computation, we find that animals' task performance is susceptible to subtle perturbations of distractor orientation and optogenetic suppression of neuronal activity in V1. This suggests that to solve this task the circuit has adopted a suboptimal and task-specific computation that discards important task-related information.SIGNIFICANCE STATEMENT A major goal in systems neuroscience is to understand how sensory signals are used to guide behavior. This requires determining what information in sensory cortical areas is used, and how it is combined, by downstream perceptual choice circuits. Here we demonstrate that when performing a go/no-go orientation discrimination task, mice suboptimally integrate signals from orientation tuned visual cortical neurons. While they appropriately positively weight target-preferring neurons, they fail to negatively weight distractor-preferring neurons. We propose that this all-positive computation may be adopted because of its simple learning rules and faster processing, and may be a common approach to perceptual decision-making when task conditions allow.

Representation of the stomatopod's retinal midband in the optic lobes: Putative neural substrates for integrating chromatic, achromatic and polarization information

Stomatopods have an elaborate visual system served by a retina that is unique to this class of pancrustaceans. Its upper and lower eye hemispheres encode luminance and linear polarization while an equatorial band of photoreceptors termed the midband detects color, circularly polarized light and linear polarization in the ultraviolet. In common with many malacostracan crustaceans, stomatopods have stalked eyes, but they can move these independently within three degrees of rotational freedom. Both eyes separately use saccadic and scanning movements but they can also move in a coordinated fashion to track selected targets or maintain a forward eyestalk posture during swimming. Visual information is initially processed in the first two optic neuropils, the lamina and the medulla, where the eye's midband is represented by enlarged regions within each neuropil that contain populations of neurons, the axons of which are segregated from the neuropil regions subtending the hemispheres. Neuronal channels representing the midband extend from the medulla to the lobula where populations of putative inhibitory glutamic acid decarboxylase-positive neurons and tyrosine hydroxylase-positive neurons intrinsic to the lobula have specific associations with the midband. Here we investigate the organization of the midband representation in the medulla and the lobula in the context of their overall architecture. We discuss the implications of observed arrangements, in which midband inputs to the lobula send out collaterals that extend across the retinotopic mosaic pertaining to the hemispheres. This organization suggests an integrative design that diverges from the eumalacostracan ground pattern and, for the stomatopod, enables color and polarization information to be integrated with luminance information that presumably encodes shape and motion.

Multichannel spectrometers in animals

Multispectral, hyperspectral, polarimetric, and other types of multichannel imaging spectrometers are coming into common use for a variety of applications, including remote sensing, material identification, forensics, and medical diagnosis. These instruments are often bulky and intolerant of field abuse, so designing compact, reliable, portable, and robust devices is a priority. In contrast to most engineering designs, animals have been building compact and robust multichannel imaging systems for millennia-their eyes. Biological sensors arise by evolution, of course, and are not designed 'for' a particular use; they exist because the creatures that were blessed with useful mutations were better able to survive and reproduce than their competitors. While this is an inefficient process for perfecting a sensor, it brings unexpected innovations and novel concepts into visual system design-concepts that may be useful in the inspiration of new engineered solutions to problematic challenges, like the ones mentioned above. Here, we review a diversity of multichannel visual systems from both vertebrate and invertebrate animals, considering the receptor molecules and cells, spectral sensitivity and its tuning, and some aspects of the higher-level processing systems used to shape spectral (and polarizational) channels in vision. The eyes of mantis shrimps are presented as potential models for biomimetic multichannel imaging systems. We end with a description of a bioinspired, newly developed multichannel spectral/polarimetric imaging system based on mantis shrimp vision that is highly adaptable to field application.

Intracellular Recordings of Spectral Sensitivities in Stomatopods: a Comparison across Species

Stomatopods (mantis shrimps) possess one of the most complex eyes in the world with photoreceptors detecting up to 12 different colors. It is not yet understood why stomatopods have almost four times the number of spectral photoreceptors compared with most other animals. It has, however, been suggested that these seemingly redundant photoreceptors could encode color through a new mechanism. Here we compare the spectral sensitivities across five species of stomatopods within the superfamily Gonodactyloidea using intracellular electrophysiological recordings. The results show that the spectral sensitivities across species of stomatopods are remarkably similar apart from some variation in the long-wavelength receptors. We relate these results to spectral sensitivity estimates previously obtained using microspectrophotometry and discuss the variation in the spectral sensitivity maxima (λmax) of the long-wavelength receptors in regard to the previous findings that stomatopods are able to tune their spectral sensitivities according to their respective light environment. We further discuss the similarities of the spectral sensitivities across species of stomatopods in regard to how color information might be processed by their visual systems.

Neural organization of afferent pathways from the stomatopod compound eye

Crustaceans and insects share many similarities of brain organization suggesting that their common ancestor possessed some components of those shared features. Stomatopods (mantis shrimps) are basal eumalacostracan crustaceans famous for their elaborate visual system, the most complex of which possesses 12 types of color photoreceptors and the ability to detect both linearly and circularly polarized light. Here, using a palette of histological methods we describe neurons and their neuropils most immediately associated with the stomatopod retina. We first provide a general overview of the major neuropil structures in the eyestalks lateral protocerebrum, with respect to the optical pathways originating from the six rows of specialized ommatidia in the stomatopod's eye, termed the midband. We then focus on the structure and neuronal types of the lamina, the first optic neuropil in the stomatopod visual system. Using Golgi impregnations to resolve single neurons we identify cells in different parts of the lamina corresponding to the three different regions of the stomatopod eye (midband and the upper and lower eye halves). While the optic cartridges relating to the spectral and polarization sensitive midband ommatidia show some specializations not found in the lamina serving the upper and lower eye halves, the general morphology of the midband lamina reflects cell types elsewhere in the lamina and cell types described for other species of Eumalacostraca.

Insect-Like Organization of the Stomatopod Central Complex: Functional and Phylogenetic Implications

One approach to investigating functional attributes of the central complex is to relate its various elaborations to pancrustacean phylogeny, to taxon-specific behavioral repertoires and ecological settings. Here we review morphological similarities between the central complex of stomatopod crustaceans and the central complex of dicondylic insects. We discuss whether their central complexes possess comparable functional properties, despite the phyletic distance separating these taxa, with mantis shrimp (Stomatopoda) belonging to the basal branch of Eumalacostraca. Stomatopods possess the most elaborate visual receptor system in nature and display a fascinating behavioral repertoire, including refined appendicular dexterity such as independently moving eyestalks. They are also unparalleled in their ability to maneuver during both swimming and substrate locomotion. Like other pancrustaceans, stomatopods possess a set of midline neuropils, called the central complex, which in dicondylic insects have been shown to mediate the selection of motor actions for a range of behaviors. As in dicondylic insects, the stomatopod central complex comprises a modular protocerebral bridge (PB) supplying decussating axons to a scalloped fan-shaped body (FB) and its accompanying ellipsoid body (EB), which is linked to a set of paired noduli and other recognized satellite regions. We consider the functional implications of these attributes in the context of stomatopod behaviors, particularly of their eyestalks that can move independently or conjointly depending on the visual scene.

Variations in normal color vision. VI. Factors underlying individual differences in hue scaling and their implications for models of color appearance

Observers with normal color vision vary widely in their judgments of color appearance, such as the specific spectral stimuli they perceive as pure or unique hues. We examined the basis of these individual differences by using factor analysis to examine the variations in hue-scaling functions from both new and previously published data. Observers reported the perceived proportion of red, green, blue or yellow in chromatic stimuli sampling angles at fixed intervals within the LM and S cone-opponent plane. These proportions were converted to hue angles in a perceptual-opponent space defined by red vs. green and blue vs. yellow axes. Factors were then extracted from the correlation matrix using PCA and Varimax rotation. These analyses revealed that inter-observer differences depend on seven or more narrowly-tuned factors. Moreover, although the task required observers to decompose the stimuli into four primary colors, there was no evidence for factors corresponding to these four primaries, or for opponent relationships between primaries. Perceptions of "redness" in orange, red, and purple, for instance, involved separate factors rather than one shared process for red. This pattern was compared to factor analyses of Monte Carlo simulations of the individual differences in scaling predicted by variations in standard opponent mechanisms, such as their spectral tuning or relative sensitivity. The observed factor pattern is inconsistent with these models and thus with conventional accounts of color appearance based on the Hering primaries. Instead, our analysis points to a perceptual representation of color in terms of multiple mechanisms or decision rules that each influence the perception of only a relatively narrow range of hues, potentially consistent with a population code for color suggested by cortical physiology.

Representation of Perceptual Color Space in Macaque Posterior Inferior Temporal Cortex (the V4 Complex)

The lateral geniculate nucleus is thought to represent color using two populations of cone-opponent neurons [L vs M; S vs (L + M)], which establish the cardinal directions in color space (reddish vs cyan; lavender vs lime). How is this representation transformed to bring about color perception? Prior work implicates populations of glob cells in posterior inferior temporal cortex (PIT; the V4 complex), but the correspondence between the neural representation of color in PIT/V4 complex and the organization of perceptual color space is unclear. We compared color-tuning data for populations of glob cells and interglob cells to predictions obtained using models that varied in the color-tuning narrowness of the cells, and the color preference distribution across the populations. Glob cells were best accounted for by simulated neurons that have nonlinear (narrow) tuning and, as a population, represent a color space designed to be perceptually uniform (CIELUV). Multidimensional scaling and representational similarity analyses showed that the color space representations in both glob and interglob populations were correlated with the organization of CIELUV space, but glob cells showed a stronger correlation. Hue could be classified invariant to luminance with high accuracy given glob responses and above-chance accuracy given interglob responses. Luminance could be read out invariant to changes in hue in both populations, but interglob cells tended to prefer stimuli having luminance contrast, regardless of hue, whereas glob cells typically retained hue tuning as luminance contrast was modulated. The combined luminance/hue sensitivity of glob cells is predicted for neurons that can distinguish two colors of the same hue at different luminance levels (orange/brown).

Segregating animals in naturalistic surroundings: interaction of color distributions and mechanisms

Humans have been shown to rapidly detect animals in naturalistic scenes, but the role of color in this task is unclear. We first analyze the color information contained in a large number of images of salient and camouflaged animals in generic backgrounds. We found that color distributions of most animals and of their immediate backgrounds were oriented along other than the cardinal directions of color space. In addition, the maximum distances between animals and background distributions also tended to be along noncardinal directions, suggesting a role for higher-order cortical color mechanisms whose preferred axes are distributed widely in color space. We measured temporal thresholds for segmenting animal color distributions from background distributions in the absence of spatial cues. Combined over all observers and all images in our sample, thresholds for segmenting isoluminant projections of these distributions were lower than for segmenting the original distributions and considerably lower than for segmenting achromatic projections. Color information is thus likely to be useful in segregating animals in generic views, i.e., views not purposely chosen by the photographer to enhance the visibility of the animal. However, a comparison of thresholds with distances between distributions failed to reveal any advantage conferred by higher-order color mechanisms.

Salience of unique hues and implications for color theory

The unique hues--blue, green, yellow, red--form the fundamental dimensions of opponent-color theories, are considered universal across languages, and provide useful mental representations for structuring color percepts. However, there is no neural evidence for them from neurophysiology or low-level psychophysics. Tapping a higher prelinguistic perceptual level, we tested whether unique hues are particularly salient in search tasks. We found no advantage for unique hues over their nonunique complementary colors. However, yellowish targets were detected faster, more accurately, and with fewer saccades than their complementary bluish targets (including unique blue), while reddish-greenish pairs were not significantly different in salience. Similarly, local field potentials in primate V1 exhibited larger amplitudes and shorter latencies for yellowish versus bluish stimuli, whereas this effect was weaker for reddish versus greenish stimuli. Consequently, color salience is affected more by early neural response asymmetries than by any possible mental or neural representation of unique hues.