Complete sparing of high-contrast color input to motion perception in cortical color blindness

Clarifying status of DNNs as models of human vision

On several key issues we agree with the commentators. Perhaps most importantly, everyone seems to agree that psychology has an important role to play in building better models of human vision, and (most) everyone agrees (including us) that deep neural networks (DNNs) will play an important role in modelling human vision going forward. But there are also disagreements about what models are for, how DNN-human correspondences should be evaluated, the value of alternative modelling approaches, and impact of marketing hype in the literature. In our view, these latter issues are contributing to many unjustified claims regarding DNN-human correspondences in vision and other domains of cognition. We explore all these issues in this response.

Deep problems with neural network models of human vision

Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images of objects and are often described as the best models of biological vision. This conclusion is largely based on three sets of findings: (1) DNNs are more accurate than any other model in classifying images taken from various datasets, (2) DNNs do the best job in predicting the pattern of human errors in classifying objects taken from various behavioral datasets, and (3) DNNs do the best job in predicting brain signals in response to images taken from various brain datasets (e.g., single cell responses or fMRI data). However, these behavioral and brain datasets do not test hypotheses regarding what features are contributing to good predictions and we show that the predictions may be mediated by DNNs that share little overlap with biological vision. More problematically, we show that DNNs account for almost no results from psychological research. This contradicts the common claim that DNNs are good, let alone the best, models of human object recognition. We argue that theorists interested in developing biologically plausible models of human vision need to direct their attention to explaining psychological findings. More generally, theorists need to build models that explain the results of experiments that manipulate independent variables designed to test hypotheses rather than compete on making the best predictions. We conclude by briefly summarizing various promising modeling approaches that focus on psychological data.

Honeybees as a model for the study of visually guided flight, navigation, and biologically inspired robotics

Research over the past century has revealed the impressive capacities of the honeybee, Apis mellifera, in relation to visual perception, flight guidance, navigation, and learning and memory. These observations, coupled with the relative ease with which these creatures can be trained, and the relative simplicity of their nervous systems, have made honeybees an attractive model in which to pursue general principles of sensorimotor function in a variety of contexts, many of which pertain not just to honeybees, but several other animal species, including humans. This review begins by describing the principles of visual guidance that underlie perception of the world in three dimensions, obstacle avoidance, control of flight speed, and orchestrating smooth landings. We then consider how navigation over long distances is accomplished, with particular reference to how bees use information from the celestial compass to determine their flight bearing, and information from the movement of the environment in their eyes to gauge how far they have flown. Finally, we illustrate how some of the principles gleaned from these studies are now being used to design novel, biologically inspired algorithms for the guidance of unmanned aerial vehicles.

The contribution of human cortical area V3A to the perception of chromatic motion: a transcranial magnetic stimulation study

Area V3A was identified in five human subjects on both a functional and retinotopic basis using functional magnetic resonance imaging techniques. V3A, along with other visual areas responsive to motion, was then targeted for disruption by repetitive transcranial magnetic stimulation (rTMS) whilst the participants performed a delayed speed matching task. The stimuli used for this task included chromatic, isoluminant motion stimuli that activated either the L-M or S-(L+M) cone-opponent mechanisms, in addition to moving stimuli that contained only luminance contrast (L+M). The speed matching task was performed for chromatic and luminance stimuli that moved at slow (2 degrees/s) or faster (8 degrees/s) speeds. The application of rTMS to area V3A produced a perceived slowing of all chromatic and luminance stimuli at both slow and fast speeds. Similar deficits were found when rTMS was applied to V5/MT+. No deficits in performance were found when areas V3B and V3d were targeted by rTMS. These results provide evidence of a causal link between neural activity in human area V3A and the perception of chromatic isoluminant motion. They establish area V3A, alongside V5/MT+, as a key area in a cortical network that underpins the analysis of not only luminance but also chromatically-defined motion.

Failure of colour and contrast polarity identification at threshold for detection of motion and global form

We used identification at threshold to systematically measure binding costs in two visual modalities. We presented a conjunction of two features as a signal stimulus and concurrently measured detection and identification performance as a function of three threshold variables: duration, contrast and coherence. Discrepancies between detection and identification sensitivity functions demonstrated a consistent processing cost to visual feature binding. Our findings suggest that feature binding is indeed a genuine problem for the brain to solve. This simple paradigm can transfer across arbitrary feature combinations and is therefore suitable to use in experiments addressing mechanisms of sensory integration.

The perception of speed based on L-M and S-(L+M) cone opponent processing

We have measured perceived speed and speed discrimination thresholds for stimuli that selectively activate the L-M, S-(L+M) cone opponent and L+M (luminance) post-receptoral pathways. For low speeds and low contrasts speed discrimination thresholds for L-M and S-(L+M) are similar but are higher and have a greater dependency upon contrast than those for luminance motion. These differences between chromatic and luminance speed perception can be eliminated when stimuli are equated with respect to their individual motion detection thresholds (MDTs). For fast moving gratings speed perception based upon L-M, S-(L+M) and L+M signals is similar in terms of threshold performance and contrast dependency. These results are consistent with the view that there are separate mechanisms for the analysis of chromatic and luminance motion, the relative contributions of which may change as a function of stimulus contrast and speed. The similarity in performance for S-(L+M) and L+M chromatic stimuli across a range of stimulus parameters suggests that signals derived from the two cone opponent pathways can be used equally well. Our results argue against the idea that speed perception is compromised when it is based upon information derived from the S-(L+M) cone opponent pathway.

Motion vision is independent of color in Drosophila

Whether motion vision uses color contrast is a controversial issue that has been investigated in several species, from insects to humans. We used Drosophila to answer this question, monitoring the optomotor response to moving color stimuli in WT and genetic variants. In the fly eye, a motion channel (outer photoreceptors R1-R6) and a color channel (inner photoreceptors R7 and R8) have been distinguished. With moving bars of alternating colors and high color contrast, a brightness ratio of the two colors can be found, at which the optomotor response is largely missing (point of equiluminance). Under these conditions, mutant flies lacking functional rhodopsin in R1-R6 cells do not respond at all. Furthermore, genetically eliminating the function of photoreceptors R7 and R8 neither alters the strength of the optomotor response nor shifts the point of equiluminance. We conclude that the color channel (R7/R8) does not contribute to motion detection as monitored by the optomotor response.

Misperceptions of speed for chromatic and luminance grating stimuli

Errors in the perception of speed of moving visual stimuli can occur when presented stimuli are of unequal contrast and when they appear alongside additional modifier stimuli that move at different speeds. We have examined these misperceptions for chromatic and luminance grating stimuli in order to assess to what extent these different kinds of motion cue might be utilised in the analysis of speed of moving objects. We show that the dependence on contrast of speed matching for luminance and chromatic stimuli is similar over a range of stimulus speeds greater than 4 deg/s. Differences between the contrast dependencies of speed perception for chromatic and luminance stimuli are only evident at slow speeds (< 4 deg/s) and low contrasts. The presence of modifier stimuli can directly influence the perceived speed at both high and low velocities and contrasts. This influence was found to be independent of the modifiers' chromaticity and was greatest when the modifiers were adjacent to and presented simultaneously with the test and reference stimuli. However, the modifiers were still able to induce measurable changes in perceived speed for increased separations over space and time. Taken together these results indicate that whilst differences do exist in the contrast dependencies of speed perception for chromatic and luminance stimuli, they are evident only for a narrow range of stimulus parameters (i.e. low speed and low contrast). There appears to be ample scope for interactions between chromatic and luminance contrast in speed perception where there is the capacity to pool this information over a relatively broad spatio-temporal extent.

The perception of motion in chromatic stimuli

The issue of whether there is a motion mechanism sensitive to purely chromatic stimuli has been pertinent for the past 30 or more years. The aim of this review is to examine why such different conclusions have been drawn in the literature and to reach some reconciliation. The review critically examines the behavioral evidence and concludes that there is a purely chromatic motion mechanism but that it is limited to the fovea. Examination of motion performance for chromatic and luminance stimuli provides convincing evidence that there are at least two different mechanisms for the two kinds of stimuli. The authors further argue that the chromatic mechanism may be at a particular disadvantage when the integration of multiple local motion signals is required. Finally, the authors present a descriptive model that may go some way toward explaining the reasons for the differences in collected data outlined in this article.

Neocortical disconnectivity disrupts sensory integration in Alzheimer's disease

The cortical pathology in Alzheimer's disease (AD) should lead to the loss of effective interaction between distinct neocortical areas. This study compared 2 conditions within a single sensory integration task that differed in the demands placed on effective cross-cortical interaction. AD patients were impaired in their ability to bind distinct visual features of a stimulus when this binding placed greater demands on cross-cortical interaction (i.e., motion and color) but were not impaired when this binding placed lesser demands on such interaction (i.e., motion and luminance). In contrast, neurologically intact individuals and patients with Huntington's disease were able to effectively bind features under both conditions. These results provide psychophysical support for the presence of functional disconnectivity in AD and demonstrate the utility of AD for investigating the neurocognitive substrates of sensory integration.

The Ferrier Lecture 1995 behind the seen: the functional specialization of the brain in space and time

The visual brain consists of many different visual areas, which are functionally specialized to process and perceive different attributes of the visual scene. However, the time taken to process different attributes varies; consequently, we see some attributes before others. It follows that there is a perceptual asynchrony and hierarchy in visual perception. Because perceiving an attribute is tantamount to becoming conscious of it, it follows that we become conscious of different attributes at different times. Visual consciousness is therefore distributed in time. Given that we become conscious of different visual attributes because of activity at different, functionally specialized, areas of the visual brain, it follows that visual consciousness is also distributed in space. Therefore, visual consciousness is not a single unified entity, but consists of many microconsciousnesses.

Behavioral Deficits and Cortical Damage Loci in Cerebral Achromatopsia

Lesions to ventral occipital cortex can produce severe deficits in color vision, a syndrome known as cerebral achromatopsia. Because most studies examine relatively few cases, however, uncertainty remains about precisely which cortical loci, when damaged, produce the syndrome. In addition, the extents of the associated perceptual deficits remain unclear. To address these issues, we performed a meta-analysis of 92 case reports from the literature. The severity of color vision deficits of the cases varied greatly, although nearly all showed some deficit in color discrimination. Almost all cases tested also showed some loss of spatial vision. Lesion overlap analyses revealed a relatively small region of high overlap in ventral occipital cortex. The region of high overlap was located near areas identified by neuroimaging studies as important for color perception. For comparison, we performed a similar analysis of prosopagnosia, a disorder of face perception, and found several regions of high lesion overlap adjacent to the region associated with achromatopsia. Because the behavioral deficits in achromatopsia are often incomplete and never restricted to color vision, the region of high lesion overlap may be one critical stage within a stream of many visual areas that participate nonexclusively in color perception.

Achromatopsia, color vision, and cortex

Brain damage can entirely abolish color vision in cases of complete achromatopsia. Other processes that depend on wavelength differences, however, can be retained. Form and motion defined by pure color differences can be perceived readily even when the colors themselves cannot be told apart. The loss of color vision in cerebral achromatopsia has been equated with the loss of a "color center" presumed indispensable for the phenomenal experience of hue. The "color center" has been assigned a role in the cortical construction of color, specifically in implementing the computations that underlie color constancy. Many features of the condition are consistent with this account. Other neurologic patients, however, retain conscious experience of hue, yet fail to disentangle the illuminant and the reflectance properties of surfaces. For them, color experience is determined by the wavelength composition of light reflected from a surface. If their wavelength-dependent vision is mediated by activity in early visual areas, then it is difficult to understand why these areas are unable to perform a similar role when they remain intact in achromatopsic observers. The prevalence of cells in the ventral visual areas of the monkey brain that code color and the further fractionation of color-related areas in human observers revealed by functional imaging suggest multiple color areas. Their different contributions are only just beginning to become apparent.

Attentional capture by colour and motion in cerebral achromatopsia

Cerebral achromatopsia is a rare condition in which damage to the ventromedial occipital area of the cortex results in the loss of colour experience. Nevertheless, cortically colour-blind patients can still use wavelength variation to perceive form and motion. In a series of six experiments we examined whether colour could also direct exogenous attention in an achromatopsic observer. We employed the colour singleton paradigm, the phi motion effect, and the correspondence process to assess attentional modulation. Although colour singletons failed to capture attention, a motion signal, based solely on chromatic information, was able to direct attention in the patient. We then show that the effect is abolished when the chromatic contours of stimuli are masked with simultaneous luminance contrast. We argue that the motion effect is dependent on chromatic contrast mediated via intact colour-opponent mechanisms. The results are taken as further evidence for the processing of wavelength variation in achromatopsia despite the absence of colour experience.

The influence of stimulus chromaticity on the isoluminant motion-onset VEP

Motion-onset visual evoked potentials (VEPs) were elicited by low spatial frequency chromatic isoluminant gratings presented in a central 7 degrees circular field. The chromatic composition of the stimuli was varied so as to modulate along different axes in colour space. For slow speeds (<5 degrees/s) changing the chromatic axis induced large response differences between the S- and L/M-cone VEPs. At faster speeds (5-12 degrees/s) the effects were not as marked. A dichotomy between the slow and fast responses was also shown to exist in terms of their contrast dependencies, the former exhibiting a stronger dependency on contrast than the latter. These findings suggest that neural substrates with chromatic sensitivity are involved in the generation of S- and L/M-cone mediated motion-onset VEPs at low velocities. At higher velocities, responses are generated by different mechanisms that possess little or no chromatic sensitivity.

The finger in flight: real-time motor control by visually masked color stimuli

Current theories of dual visual systems suggest that color is processed in a ventral cortical stream that eventually gives rise to visual awareness but is only indirectly involved in visuomotor control mediated by the dorsal stream. If the dorsal stream is indeed less sensitive to color than the ventral stream, color stimuli blocked from awareness by visual masking should also be blocked from guiding fast motor responses. In this study, pointing movements to one of two isoluminant color targets were preceded by consistent or inconsistent color primes. Trajectories were strongly affected by priming, with kinematics implying a continuous flow of color information into executive brain areas while the finger was already moving. Motor effects were more sensitive to color of the primes than were deliberate attempts to identify the primes in forced-choice tasks based on visual awareness. Priming was observed even when masking was complete.

The dissociation of color from form and function knowledge

We report on two brain-damaged subjects who exhibit the uncommon pattern of loss of object color knowledge, but spared color perception and naming. The subject P.C.O., as in previously reported patients, is also impaired in processing other perceptual and functional properties of objects. I.O.C., in contrast, is the first subject on record to have impaired object color knowledge, but spared knowledge of object form, size and function. This pattern of performance is consistent with the view that semantic information about color and other perceptual properties of objects is grounded in modality-specific systems. Lesion analysis suggests that such grounding requires the integrity of the mesial temporal regions of the left hemisphere.

Position-based motion perception for color and texture stimuli: effects of contrast and speed

Motion can be perceived either through low-level, motion-energy detection or through tracking the change in position of features. Previously we have shown that, while luminance-based motion likely is detected with velocity-sensitive motion-energy units, patterns defined by texture or binocular disparity ('second-order' stimuli) were tracked by a position-sensitive mechanism (Seiffert & Cavanagh (1998) Vision Research, 38, 3569-3582). Here, we use the same technique, measuring motion amplitude thresholds of oscillating gratings over a range of temporal frequencies and find that the motion of low-contrast equiluminant red/green gratings is also detected with position tracking. In addition, we find that as contrast or speed increases these results change: high-contrast or high-speed equiluminant color or texture-based motion is detected by velocity-sensitive mechanisms. These results help resolve the dispute over the processes which detect the motion of non-luminance based stimuli. Both systems are available, but their relative efficiency changes as a function of contrast and speed. A position-tracking process is more sensitive at low contrasts and low speeds whereas a motion-energy system is more sensitive at high contrasts and high speeds.

Color signals in human motion-selective cortex

The neural basis for the effects of color and contrast on perceived speed was examined using functional magnetic resonance imaging (fMRI). Responses to S cone (blue-yellow) and L + M cone (luminance) patterns were measured in area V1 and in the motion area MT+. The MT+ responses were quantitatively similar to perceptual speed judgments of color patterns but not to color detection measures. We also measured cortical motion responses in individuals lacking L and M cone function (S cone monochromats). The S cone monochromats have clear motion-responsive regions in the conventional MT+ position, and their contrast-response functions there have twice the responsivity of S cone contrast-response functions in normal controls. But, their responsivity is far lower than the normals' responsivity to luminance contrast. Thus, the powerful magnocellular input to MT+ is either weak or silent during photopic vision in S cone monochromats.

Perceived speed of colored stimuli

The influence of contrast and color on perceived motion was measured using a speed-matching task. Observers adjusted the speed of an L cone contrast pattern to match that of a variety of colored test patterns. The dependence of speed on test contrast was the same for all test colors measured, differing only by a sensitivity factor. This result suggests that the reduced apparent speed of low contrast targets and certain colored targets is caused by a common cortical mechanism. The cone contrast levels that equate perceived speed differ substantially from those that equate visibility. This result suggests that the neural mechanisms governing speed perception and visibility differ. Perceived speed differences caused by variations in color can be explained by color responses that are characteristic of motion-selective cortex.

Vision: modular analysis--or not?

It has commonly been assumed that the many separate areas of the visual system perform modular analyses, each restricted to a single attribute of the image. A recent paper advocates a radically different approach, where all areas in the hierarchy analyse all attributes of the image to extract perceptually relevant decisions.