Calculating the contrasts that retinal ganglion cells and LGN neurones encounter in natural scenes

Hierarchical Perceptual Noise Injection for Social Media Fingerprint Privacy Protection

Billions of people share images from their daily lives on social media every day. However, their biometric information (e.g., fingerprints) could be easily stolen from these images. The threat of fingerprint leakage from social media has created a strong desire to anonymize shared images while maintaining image quality, since fingerprints act as a lifelong individual biometric password. To guard the fingerprint leakage, adversarial attack that involves adding imperceptible perturbations to fingerprint images have emerged as a feasible solution. However, existing works of this kind are either weak in black-box transferability or cause the images to have an unnatural appearance. Motivated by the visual perception hierarchy (i.e., high-level perception exploits model-shared semantics that transfer well across models while low-level perception extracts primitive stimuli that result in high visual sensitivity when a suspicious stimulus is provided), we propose FingerSafe, a hierarchical perceptual protective noise injection framework to address the above mentioned problems. For black-box transferability, we inject protective noises into the fingerprint orientation field to perturb the model-shared high-level semantics (i.e., fingerprint ridges). Considering visual naturalness, we suppress the low-level local contrast stimulus by regularizing the response of the Lateral Geniculate Nucleus. Our proposed FingerSafe is the first to provide feasible fingerprint protection in both digital (up to 94.12%) and realistic scenarios (Twitter and Facebook, up to 68.75%). Our code can be found at https://github.com/nlsde-safety-team/FingerSafe.

Hierarchies in Visual Pathway: Functions and Inspired Artificial Vision

The development of artificial intelligence has posed a challenge to machine vision based on conventional complementary metal-oxide semiconductor (CMOS) circuits owing to its high latency and inefficient power consumption originating from the data shuffling between memory and computation units. Gaining more insights into the function of every part of the visual pathway for visual perception can bring the capabilities of machine vision in terms of robustness and generality. Hardware acceleration of more energy-efficient and biorealistic artificial vision highly necessitates neuromorphic devices and circuits that are able to mimic the function of each part of the visual pathway. In this paper, we review the structure and function of the entire class of visual neurons from the retina to the primate visual cortex within reach (Chapter 2) are reviewed. Based on the extraction of biological principles, the recent hardware-implemented visual neurons located in different parts of the visual pathway are discussed in detail in Chapters 3 and 4. Furthermore, valuable applications of inspired artificial vision in different scenarios (Chapter 5) are provided. The functional description of the visual pathway and its inspired neuromorphic devices/circuits are expected to provide valuable insights for the design of next-generation artificial visual perception systems.

Spatiotemporal visual statistics of aquatic environments in the natural habitats of zebrafish

Animal sensory systems are tightly adapted to the demands of their environment. In the visual domain, research has shown that many species have circuits and systems that exploit statistical regularities in natural visual signals. The zebrafish is a popular model animal in visual neuroscience, but relatively little quantitative data is available about the visual properties of the aquatic habitats where zebrafish reside, as compared to terrestrial environments. Improving our understanding of the visual demands of the aquatic habitats of zebrafish can enhance the insights about sensory neuroscience yielded by this model system. We analyzed a video dataset of zebrafish habitats captured by a stationary camera and compared this dataset to videos of terrestrial scenes in the same geographic area. Our analysis of the spatiotemporal structure in these videos suggests that zebrafish habitats are characterized by low visual contrast and strong motion when compared to terrestrial environments. Similar to terrestrial environments, zebrafish habitats tended to be dominated by dark contrasts, particularly in the lower visual field. We discuss how these properties of the visual environment can inform the study of zebrafish visual behavior and neural processing and, by extension, can inform our understanding of the vertebrate brain.

Learning the Vector Coding of Egocentric Boundary Cells from Visual Data

The use of spatial maps to navigate through the world requires a complex ongoing transformation of egocentric views of the environment into position within the allocentric map. Recent research has discovered neurons in retrosplenial cortex and other structures that could mediate the transformation from egocentric views to allocentric views. These egocentric boundary cells respond to the egocentric direction and distance of barriers relative to an animal's point of view. This egocentric coding based on the visual features of barriers would seem to require complex dynamics of cortical interactions. However, computational models presented here show that egocentric boundary cells can be generated with a remarkably simple synaptic learning rule that forms a sparse representation of visual input as an animal explores the environment. Simulation of this simple sparse synaptic modification generates a population of egocentric boundary cells with distributions of direction and distance coding that strikingly resemble those observed within the retrosplenial cortex. Furthermore, some egocentric boundary cells learnt by the model can still function in new environments without retraining. This provides a framework for understanding the properties of neuronal populations in the retrosplenial cortex that may be essential for interfacing egocentric sensory information with allocentric spatial maps of the world formed by neurons in downstream areas, including the grid cells in entorhinal cortex and place cells in the hippocampus.SIGNIFICANCE STATEMENT The computational model presented here demonstrates that the recently discovered egocentric boundary cells in retrosplenial cortex can be generated with a remarkably simple synaptic learning rule that forms a sparse representation of visual input as an animal explores the environment. Additionally, our model generates a population of egocentric boundary cells with distributions of direction and distance coding that strikingly resemble those observed within the retrosplenial cortex. This transformation between sensory input and egocentric representation in the navigational system could have implications for the way in which egocentric and allocentric representations interface in other brain areas.

A model of colour appearance based on efficient coding of natural images

An object's colour, brightness and pattern are all influenced by its surroundings, and a number of visual phenomena and "illusions" have been discovered that highlight these often dramatic effects. Explanations for these phenomena range from low-level neural mechanisms to high-level processes that incorporate contextual information or prior knowledge. Importantly, few of these phenomena can currently be accounted for in quantitative models of colour appearance. Here we ask to what extent colour appearance is predicted by a model based on the principle of coding efficiency. The model assumes that the image is encoded by noisy spatio-chromatic filters at one octave separations, which are either circularly symmetrical or oriented. Each spatial band's lower threshold is set by the contrast sensitivity function, and the dynamic range of the band is a fixed multiple of this threshold, above which the response saturates. Filter outputs are then reweighted to give equal power in each channel for natural images. We demonstrate that the model fits human behavioural performance in psychophysics experiments, and also primate retinal ganglion responses. Next, we systematically test the model's ability to qualitatively predict over 50 brightness and colour phenomena, with almost complete success. This implies that much of colour appearance is potentially attributable to simple mechanisms evolved for efficient coding of natural images, and is a well-founded basis for modelling the vision of humans and other animals.

Evaluation of visual functions in Iranian hypothyroid adults

INTRODUCTION

The aim of this study was to investigate the effects of hypothyroidism on visual functions such as visual acuity, refractive errors, colour vision, and contrast sensitivity, among hypothyroid adults.

METHODS

Forty-three patients with clinical hypothyroidism along with 43 age- and sex-matched healthy individuals underwent visual examinations, including visual acuity, refractive errors, eye deviations with the cover test, colour vision with the D15 test, and contrast sensitivity with Pelli-Robson test.

RESULTS

It was indicated that visual acuity, refractive errors, phoria, and colour vision had no significant difference between the hypothyroid and control groups. Contrast sensitivity decreased in hypothyroid subjects as compared with controls. The mean values of binocular contrast sensitivity were 1.85 ± 0.09 log in the hypothyroid group and 1.93 ± 0.09 log in controls, which showed a statistically significant difference (p = .03).

CONCLUSIONS

Our findings illustrated a reduced contrast sensitivity in adult hypothyroidism. Since CS is related to functioning and quality of life, a comprehensive and detailed eye examination may be beneficial for hypothyroidism patients.

Optic flow in the natural habitats of zebrafish supports spatial biases in visual self-motion estimation

Animals benefit from knowing if and how they are moving. Across the animal kingdom, sensory information in the form of optic flow over the visual field is used to estimate self-motion. However, different species exhibit strong spatial biases in how they use optic flow. Here, we show computationally that noisy natural environments favor visual systems that extract spatially biased samples of optic flow when estimating self-motion. The performance associated with these biases, however, depends on interactions between the environment and the animal's brain and behavior. Using the larval zebrafish as a model, we recorded natural optic flow associated with swimming trajectories in the animal's habitat with an omnidirectional camera mounted on a mechanical arm. An analysis of these flow fields suggests that lateral regions of the lower visual field are most informative about swimming speed. This pattern is consistent with the recent findings that zebrafish optomotor responses are preferentially driven by optic flow in the lateral lower visual field, which we extend with behavioral results from a high-resolution spherical arena. Spatial biases in optic-flow sampling are likely pervasive because they are an effective strategy for determining self-motion in noisy natural environments.

Unrestricted eye movements strengthen effective connectivity from hippocampal to oculomotor regions during scene construction

Scene construction is a key component of memory recall, navigation, and future imagining, and relies on the medial temporal lobes (MTL). A parallel body of work suggests that eye movements may enable the imagination and construction of scenes, even in the absence of external visual input. There are vast structural and functional connections between regions of the MTL and those of the oculomotor system. However, the directionality of connections between the MTL and oculomotor control regions, and how it relates to scene construction, has not been studied directly in human neuroimaging. In the current study, we used dynamic causal modeling (DCM) to interrogate effective connectivity between the MTL and oculomotor regions using a scene construction task in which participants' eye movements were either restricted (fixed-viewing) or unrestricted (free-viewing). By omitting external visual input, and by contrasting free- versus fixed- viewing, the directionality of neural connectivity during scene construction could be determined. As opposed to when eye movements were restricted, allowing free-viewing during construction of scenes strengthened top-down connections from the MTL to the frontal eye fields, and to lower-level cortical visual processing regions, suppressed bottom-up connections along the visual stream, and enhanced vividness of the constructed scenes. Taken together, these findings provide novel, non-invasive evidence for the underlying, directional, connectivity between the MTL memory system and oculomotor system associated with constructing vivid mental representations of scenes.

Synchronous inhibitory pathways create both efficiency and diversity in the retina

Complex connections in neural circuits make it difficult to quantitatively assign even the most basic neural computations to the actions of specific neurons. Retinal ganglion cells are most sensitive to changes in intensity across space and over time. This property, caused by a region known as the receptive field surround, improves information transmission about natural scenes. We dynamically manipulated individual interneurons to directly measure their effect on retinal receptive fields, finding that two inhibitory neuron types, horizontal cells and amacrine cells, synchronously create the same contribution to the receptive field surround at different spatial scales. By analyzing large populations of ganglion cells, we show that this arrangement increases diversity in retinal signaling while preserving maximal information transmission about natural scenes.

Sensory receptive fields combine features that originate in different neural pathways. Retinal ganglion cell receptive fields compute intensity changes across space and time using a peripheral region known as the surround, a property that improves information transmission about natural scenes. The visual features that construct this fundamental property have not been quantitatively assigned to specific interneurons. Here, we describe a generalizable approach using simultaneous intracellular and multielectrode recording to directly measure and manipulate the sensory feature conveyed by a neural pathway to a downstream neuron. By directly controlling the gain of individual interneurons in the circuit, we show that rather than transmitting different temporal features, inhibitory horizontal cells and linear amacrine cells synchronously create the linear surround at different spatial scales and that these two components fully account for the surround. By analyzing a large population of ganglion cells, we observe substantial diversity in the relative contribution of amacrine and horizontal cell visual features while still allowing individual cells to increase information transmission under the statistics of natural scenes. Established theories of efficient coding have shown that optimal information transmission under natural scenes allows a diverse set of receptive fields. Our results give a mechanism for this theory, showing how distinct neural pathways synthesize a sensory computation and how this architecture both generates computational diversity and achieves the objective of high information transmission.

Intrasaccadic motion streaks jump-start gaze correction

Rapid eye movements (saccades) incessantly shift objects across the retina. To establish object correspondence, the visual system is thought to match surface features of objects across saccades. Here, we show that an object's intrasaccadic retinal trace-a signal previously considered unavailable to visual processing-facilitates this match making. Human observers made saccades to a cued target in a circular stimulus array. Using high-speed visual projection, we swiftly rotated this array during the eyes' flight, displaying continuous intrasaccadic target motion. Observers' saccades landed between the target and a distractor, prompting secondary saccades. Independently of the availability of object features, which we controlled tightly, target motion increased the rate and reduced the latency of gaze-correcting saccades to the initial presaccadic target, in particular when the target's stimulus features incidentally gave rise to efficient motion streaks. These results suggest that intrasaccadic visual information informs the establishment of object correspondence and jump-starts gaze correction.

An Image Contrast Measure Based on Retinex Principles

The image contrast is a feature capturing the variation of the image signal across the space. Such a feature is very useful to describe the local image structure at different scales and thus it is relevant to many computer vision applications, like image/texture retrieval and object recognition. In this work, we present MiRCo, a novel measure of image contrast derived from the Retinex theory. MiRCo is robust against in-plane rotations and light changes at multiple scales. Thanks to these properties, MiRCo enables an accurate and robust description of the local image structure. Here we describe and discuss the mathematical insights of MiRCo also in comparison with other popular contrast measures.

Learning receptive field properties of complex cells in V1

There are two distinct classes of cells in the primary visual cortex (V1): simple cells and complex cells. One defining feature of complex cells is their spatial phase invariance; they respond strongly to oriented grating stimuli with a preferred orientation but with a wide range of spatial phases. A classical model of complete spatial phase invariance in complex cells is the energy model, in which the responses are the sum of the squared outputs of two linear spatially phase-shifted filters. However, recent experimental studies have shown that complex cells have a diverse range of spatial phase invariance and only a subset can be characterized by the energy model. While several models have been proposed to explain how complex cells could learn to be selective to orientation but invariant to spatial phase, most existing models overlook many biologically important details. We propose a biologically plausible model for complex cells that learns to pool inputs from simple cells based on the presentation of natural scene stimuli. The model is a three-layer network with rate-based neurons that describes the activities of LGN cells (layer 1), V1 simple cells (layer 2), and V1 complex cells (layer 3). The first two layers implement a recently proposed simple cell model that is biologically plausible and accounts for many experimental phenomena. The neural dynamics of the complex cells is modeled as the integration of simple cells inputs along with response normalization. Connections between LGN and simple cells are learned using Hebbian and anti-Hebbian plasticity. Connections between simple and complex cells are learned using a modified version of the Bienenstock, Cooper, and Munro (BCM) rule. Our results demonstrate that the learning rule can describe a diversity of complex cells, similar to those observed experimentally.

Many cortical functions originate from the learning ability of the brain. How the properties of cortical cells are learned is vital for understanding how the brain works. There are many models that explain how V1 simple cells can be learned. However, how V1 complex cells are learned still remains unclear. In this paper, we propose a model of learning in complex cells based on the Bienenstock, Cooper, and Munro (BCM) rule. We demonstrate that properties of receptive fields of complex cells can be learned using this biologically plausible learning rule. Quantitative comparisons between the model and experimental data are performed. Results show that model complex cells can account for the diversity of complex cells found in experimental studies. In summary, this study provides a plausible explanation for how complex cells can be learned using biologically plausible plasticity mechanisms. Our findings help us to better understand biological vision processing and provide us with insights into the general signal processing principles that the visual cortex employs to process visual information.

Image luminance changes contrast sensitivity in visual cortex

Accurate measures of contrast sensitivity are important for evaluating visual disease progression and for navigation safety. Previous measures suggested that cortical contrast sensitivity was constant across widely different luminance ranges experienced indoors and outdoors. Against this notion, here, we show that luminance range changes contrast sensitivity in both cat and human cortex, and the changes are different for dark and light stimuli. As luminance range increases, contrast sensitivity increases more within cortical pathways signaling lights than those signaling darks. Conversely, when the luminance range is constant, light-dark differences in contrast sensitivity remain relatively constant even if background luminance changes. We show that a Naka-Rushton function modified to include luminance range and light-dark polarity accurately replicates both the statistics of light-dark features in natural scenes and the cortical responses to multiple combinations of contrast and luminance. We conclude that differences in light-dark contrast increase with luminance range and are largest in bright environments.

The statistics of how natural images drive the responses of neurons

To model the responses of neurons in the early visual system, at least three basic components are required: a receptive field, a normalization term, and a specification of encoding noise. Here, we examine how the receptive field, the normalization factor, and the encoding noise affect the drive to model-neuron responses when stimulated with natural images. We show that when these components are modeled appropriately, the response drives elicited by natural stimuli are Gaussian-distributed and scale invariant, and very nearly maximize the sensitivity (d') for natural-image discrimination. We discuss the statistical models of natural stimuli that can account for these response statistics, and we show how some commonly used modeling practices may distort these results. Finally, we show that normalization can equalize important properties of neural response across different stimulus types. Specifically, narrowband (stimulus- and feature-specific) normalization causes model neurons to yield Gaussian response-drive statistics when stimulated with natural stimuli, 1/f noise stimuli, and white-noise stimuli. The current work makes recommendations for best practices and lays a foundation, grounded in the response statistics to natural stimuli, upon which to build principled models of more complex visual tasks.

Functional diversity among sensory neurons from efficient coding principles

In many sensory systems the neural signal is coded by the coordinated response of heterogeneous populations of neurons. What computational benefit does this diversity confer on information processing? We derive an efficient coding framework assuming that neurons have evolved to communicate signals optimally given natural stimulus statistics and metabolic constraints. Incorporating nonlinearities and realistic noise, we study optimal population coding of the same sensory variable using two measures: maximizing the mutual information between stimuli and responses, and minimizing the error incurred by the optimal linear decoder of responses. Our theory is applied to a commonly observed splitting of sensory neurons into ON and OFF that signal stimulus increases or decreases, and to populations of monotonically increasing responses of the same type, ON. Depending on the optimality measure, we make different predictions about how to optimally split a population into ON and OFF, and how to allocate the firing thresholds of individual neurons given realistic stimulus distributions and noise, which accord with certain biases observed experimentally.

No evidence for abnormal priors in early vision in schizophrenia

The predictive coding account of psychosis postulates the abnormal formation of prior beliefs in schizophrenia, resulting in psychotic symptoms. One domain in which priors play a crucial role is visual perception. For instance, our perception of brightness, line length, and motion direction are not merely based on a veridical extraction of sensory input but are also determined by expectation (or prior) of the stimulus. Formation of such priors is thought to be governed by the statistical regularities within natural scenes. Recently, the use of such priors has been attributed to a specific set of well-documented visual illusions, supporting the idea that perception is biased toward what is statistically more probable within the environment. The Predictive Coding account of psychosis proposes that patients form abnormal representations of statistical regularities in natural scenes, leading to altered perceptual experiences. Here we use classical vision experiments involving a specific set of visual illusions to directly test this hypothesis. We find that perceptual judgments for both patients and control participants are biased in accordance with reported probability distributions of natural scenes. Thus, despite there being a suggested link between visual abnormalities and psychotic symptoms in schizophrenia, our results provide no support for the notion that altered formation of priors is a general feature of the disorder. These data call for a refinement in the predictions of quantitative models of psychosis.

Area V1 responses to illusory corner-folds in Vasarely's nested squares and the Alternating Brightness Star illusions

Vasarely’s nested squares illusion shows that the corners of concentric squares, arranged in a gradient of increasing or decreasing luminance, generate illusory “corner-folds,” which appear more salient (either brighter or darker) than the adjacent flat (non- corner) regions of each individual square. The Alternating Brightness Star (ABS) illusion, based on Vasarely’s classic nested squares, further shows that the strength of these corner-folds depends on corner angle. Previous psychophysical studies showed the relationship between corner angle and perceived contrast in the ABS illusion to be linear, with sharp angles looking higher in contrast, and shallow angles lower in contrast. Center-surround difference-of-Gaussians (DOG) modeling did not replicate this linear relationship, however, suggesting that a full neural explanation of the nested squares and ABS illusions might be found in the visual cortex, rather than at subcortical stages. Here we recorded the responses from single area V1 neurons in the awake primate, during the presentation of visual stimuli containing illusory corner-folds of various angles. Our results showed stronger neural responses for illusory corner-folds made from sharper than from shallower corners, consistent with predictions from the previous psychophysical work. The relationship between corner angle and strength of the neuronal responses, albeit parametric, was apparently non-linear. This finding was in line with the previous DOG data, but not with the psychophysical data. Our combined results suggest that, whereas corner-fold illusions likely originate from center-surround retinogeniculate processes, their complete neural explanation may be found in extrastriate visual cortical areas.

Olfactory Navigation and the Receptor Nonlinearity

The demands on a sensory system depend not only on the statistics of its inputs but also on the task. In olfactory navigation, for example, the task is to find the plume source; allocation of sensory resources may therefore be driven by aspects of the plume that are informative about source location, rather than concentration per se. Here we explore the implications of this idea for encoding odor concentration. To formalize the notion that sensory resources are limited, we considered coding strategies that partitioned the odor concentration range into a set of discriminable intervals. We developed a dynamic programming algorithm that, given the distribution of odor concentrations at several locations, determines the partitioning that conveys the most information about location. We applied this analysis to planar laser-induced fluorescence measurements of spatiotemporal odor fields with realistic advection speeds (5-20 cm/s), with or without a nearby boundary or obstacle. Across all environments, the optimal coding strategy allocated more resources (i.e., more and finer discriminable intervals) to the upper end of the concentration range than would be expected from histogram equalization, the optimal strategy if the goal were to reconstruct the plume, rather than to navigate. Finally, we show that ligand binding, as captured by the Hill equation, transforms odorant concentration into response levels in a way that approximates information maximization for navigation. This behavior occurs when the Hill dissociation constant is near the mean odor concentration, an adaptive set-point that has been observed in the olfactory system of flies.SIGNIFICANCE STATEMENT The first step of olfactory processing is receptor binding, and the resulting relationship between odorant concentration and the bound receptor fraction is a saturating one. While this Hill nonlinearity can be viewed as a distortion that is imposed by the biophysics of receptor binding, here we show that it also plays an important information-processing role in olfactory navigation. Specifically, by combining a novel dynamic-programming algorithm with physical measurements of turbulent plumes, we determine the optimal strategy for encoding odor concentration when the goal is to determine location. This strategy is distinct from histogram equalization, the strategy that maximizes information about plume concentration, and is closely approximated by the Hill nonlinearity when the binding constant is near the ambient mean.

Can Contrast-Response Functions Indicate Visual Processing Levels?

Many visual effects are believed to be processed at several functional and anatomical levels of cortical processing. Determining if and how the levels contribute differentially to these effects is a leading problem in visual perception and visual neuroscience. We review and analyze a combination of extant psychophysical findings in the context of neurophysiological and brain-imaging results. Specifically using findings relating to visual illusions, crowding, and masking as exemplary cases, we develop a theoretical rationale for showing how relative levels of cortical processing contributing to these effects can already be deduced from the psychophysically determined functions relating respectively the illusory, crowding and masking strengths to the contrast of the illusion inducers, of the flankers producing the crowding, and of the mask. The wider implications of this rationale show how it can help to settle or clarify theoretical and interpretive inconsistencies and how it can further psychophysical, brain-recording and brain-imaging research geared to explore the relative functional and cortical levels at which conscious and unconscious processing of visual information occur. Our approach also allows us to make some specific predictions for future studies, whose results will provide empirical tests of its validity.

Perceived Duration Increases with Contrast, but Only a Little

Recent adaptation studies provide evidence for early visual areas playing a role in duration perception. One explanation for the pronounced duration compression commonly found with adaptation is that it reflects adaptation-driven stimulus-specific reduction in neural activity in early visual areas. If this level of stimulus-associated neural activity does drive duration, then we would expect a strong effect of contrast on perceived duration as electrophysiological studies shows neural activity in early visual areas to be strongly related to contrast. We employed a spatially isotropic noise stimulus where the luminance of each noise element was independently sinusoidally modulated at 4 Hz. Participants matched the perceived duration of a high (0.9) or low (0.1) contrast stimulus to a previously presented standard stimulus (600 ms, contrast = 0.3). To achieve perceptually equivalent durations, the low contrast stimulus had to be presented for longer than the high contrast stimulus. This occurred when we controlled for stimulus size and when we adjusted for individual differences in perceived temporal frequency. Further, we show that the effect cannot be explained by shifts in perceived onset and offset and is not explained by a simple contrast-driven response bias. The direction of our results is clearly consistent with the idea that level of neural activity drives duration. However, the magnitude of the effect (~10% duration difference over a 0.9-0.1 contrast reduction) is in marked contrast to the larger duration distortions that can be found with repetition suppression and the oddball effect; particularly when these may be associated with smaller differences in neural activity than that expected from our contrast difference. Taken together, these results indicate that level of stimulus-related neural activity in early visual areas is unlikely to provide a general mechanism for explaining differences in perceived duration.

Efficient Neural Codes That Minimize Lp Reconstruction Error

The efficient coding hypothesis assumes that biological sensory systems use neural codes that are optimized to best possibly represent the stimuli that occur in their environment. Most common models use information-theoretic measures, whereas alternative formulations propose incorporating downstream decoding performance. Here we provide a systematic evaluation of different optimality criteria using a parametric formulation of the efficient coding problem based on the [Formula: see text] reconstruction error of the maximum likelihood decoder. This parametric family includes both the information maximization criterion and squared decoding error as special cases. We analytically derived the optimal tuning curve of a single neuron encoding a one-dimensional stimulus with an arbitrary input distribution. We show how the result can be generalized to a class of neural populations by introducing the concept of a meta-tuning curve. The predictions of our framework are tested against previously measured characteristics of some early visual systems found in biology. We find solutions that correspond to low values of [Formula: see text], suggesting that across different animal models, neural representations in the early visual pathways optimize similar criteria about natural stimuli that are relatively close to the information maximization criterion.

Contrast response functions in the visual wulst of the alert burrowing owl: a single-unit study

The neuronal representation of luminance contrast has not been thoroughly studied in birds. Here we present a detailed quantitative analysis of the contrast response of 120 individual neurons recorded from the visual wulst of awake burrowing owls (Athene cunicularia). Stimuli were sine-wave gratings presented within the cell classical receptive field and optimized in terms of eye preference, direction of drift, and spatiotemporal frequency. As contrast intensity was increased from zero to near 100%, most cells exhibited a monotonic response profile with a compressive, at times saturating, nonlinearity at higher contrasts. However, contrast response functions were found to have a highly variable shape across cells. With the view to capture a systematic trend in the data, we assessed the performance of four plausible models (linear, power, logarithmic, and hyperbolic ratio) using classical goodness-of-fit measures and more rigorous statistical tools for multimodel inferences based on the Akaike information criterion. From this analysis, we conclude that a high degree of model uncertainty is present in our data, meaning that no single descriptor is able on its own to capture the heterogeneous nature of single-unit contrast responses in the wulst. We further show that the generalizability of the hyperbolic ratio model established, for example, in the primary visual cortex of cats and monkeys is not tenable in the owl wulst mainly because most neurons in this area have a much wider dynamic range that starts at low contrast. The challenge for future research will be to understand the functional implications of these findings.

A normalized contrast-encoding model exhibits bright/dark asymmetries similar to early visual neurons

Biological sensory systems share a number of organizing principles. One such principle is the formation of parallel streams. In the visual system, information about bright and dark features is largely conveyed via two separate streams: the ON and OFF pathways. While brightness and darkness can be considered symmetric and opposite forms of visual contrast, the response properties of cells in the ON and OFF pathways are decidedly asymmetric. Here, we ask whether a simple contrast‐encoding model predicts asymmetries for brights and darks that are similar to the asymmetries found in the ON and OFF pathways. Importantly, this model does not include any explicit differences in how the visual system represents brights and darks, but it does include a common normalization mechanism. The phenomena captured by the model include (1) nonlinear contrast response functions, (2) greater nonlinearities in the responses to darks, and (3) larger responses to dark contrasts. We report a direct, quantitative comparison between these model predictions and previously published electrophysiological measurements from the retina and thalamus (guinea pig and cat, respectively). This work suggests that the simple computation of visual contrast may account for a range of early visual processing nonlinearities. Assessing explicit models of sensory representations is essential for understanding which features of neuronal activity these models can and cannot predict, and for investigating how early computations may reverberate through the sensory pathways.

Novel Visual Illusions Related to Vasarely's ‘Nested Squares’ Show That Corner Salience Varies with Corner Angle

Vasarely's ‘nested-squares’ illusion shows that 90° corners can be more salient perceptually than straight edges. On the basis of this illusion we have developed a novel visual illusion, the ‘Alternating Brightness Star’, which shows that sharp corners are more salient than shallow corners (an effect we call ‘corner angle salience variation’) and that the same corner can be perceived as either bright or dark depending on the polarity of the angle (ie whether concave or convex: ‘corner angle brightness reversal’). Here we quantify the perception of corner angle salience variation and corner angle brightness reversal effects in twelve naive human subjects, in a two-alternative forced-choice brightness discrimination task. The results show that sharp corners generate stronger percepts than shallow corners, and that corner gradients appear bright or dark depending on whether the corner is concave or convex. Basic computational models of center – surround receptive fields predict the results to some degree, but not fully.

A Central Role for Mixed Acetylcholine/GABA Transmission in Direction Coding in the Retina

A surprisingly large number of neurons throughout the brain are endowed with the ability to co-release both a fast excitatory and inhibitory transmitter. The computational benefits of dual transmitter release, however, remain poorly understood. Here, we address the role of co-transmission of acetylcholine (ACh) and GABA from starburst amacrine cells (SACs) to direction-selective ganglion cells (DSGCs). Using a combination of pharmacology, optogenetics, and linear regression methods, we estimated the spatiotemporal profiles of GABA, ACh, and glutamate receptor-mediated synaptic activity in DSGCs evoked by motion. We found that ACh initiates responses to motion in natural scenes or under low-contrast conditions. In contrast, classical glutamatergic pathways play a secondary role, amplifying cholinergic responses via NMDA receptor activation. Furthermore, under these conditions, the network of SACs differentially transmits ACh and GABA to DSGCs in a directional manner. Thus, mixed transmission plays a central role in shaping directional responses of DSGCs.

Conjectures regarding the nonlinear geometry of visual neurons

From the earliest stages of sensory processing, neurons show inherent non-linearities: the response to a complex stimulus is not a sum of the responses to a set of constituent basis stimuli. These non-linearities come in a number of forms and have been explained in terms of a number of functional goals. The family of spatial non-linearities have included interactions that occur both within and outside of the classical receptive field. They include, saturation, cross orientation inhibition, contrast normalization, end-stopping and a variety of non-classical effects. In addition, neurons show a number of facilitatory and invariance related effects such as those exhibited by complex cells (integration across position). Here, we describe an approach that attempts to explain many of the non-linearities under a single geometric framework. In line with Zetzsche and colleagues (e.g., Zetzsche et al., 1999) we propose that many of the principal non-linearities can be described by a geometry where the neural response space has a simple curvature. In this paper, we focus on the geometry that produces both increased selectivity (curving outward) and increased tolerance (curving inward). We demonstrate that overcomplete sparse coding with both low-dimensional synthetic data and high-dimensional natural scene data can result in curvature that is responsible for a variety of different known non-classical effects including end-stopping and gain control. We believe that this approach provides a more fundamental explanation of these non-linearities and does not require that one postulate a variety of explanations (e.g., that gain must be controlled or the ends of lines must be detected). In its standard form, sparse coding does not however, produce invariance/tolerance represented by inward curvature. We speculate on some of the requirements needed to produce such curvature.

Synchronized amplification of local information transmission by peripheral retinal input

Sensory stimuli have varying statistics influenced by both the environment and by active sensing behaviors that rapidly and globally change the sensory input. Consequently, sensory systems often adjust their neural code to the expected statistics of their sensory input to transmit novel sensory information. Here, we show that sudden peripheral motion amplifies and accelerates information transmission in salamander ganglion cells in a 50 ms time window. Underlying this gating of information is a transient increase in adaptation to contrast, enhancing sensitivity to a broader range of stimuli. Using a model and natural images, we show that this effect coincides with an expected increase in information in bipolar cells after a global image shift. Our findings reveal the dynamic allocation of energy resources to increase neural activity at times of expected high information content, a principle of adaptation that balances the competing requirements of conserving spikes and transmitting information.

A higher order visual neuron tuned to the spatial amplitude spectra of natural scenes

Animal sensory systems are optimally adapted to those features typically encountered in natural surrounds, thus allowing neurons with limited bandwidth to encode challengingly large input ranges. Natural scenes are not random, and peripheral visual systems in vertebrates and insects have evolved to respond efficiently to their typical spatial statistics. The mammalian visual cortex is also tuned to natural spatial statistics, but less is known about coding in higher order neurons in insects. To redress this we here record intracellularly from a higher order visual neuron in the hoverfly. We show that the cSIFE neuron, which is inhibited by stationary images, is maximally inhibited when the slope constant of the amplitude spectrum is close to the mean in natural scenes. The behavioural optomotor response is also strongest to images with naturalistic image statistics. Our results thus reveal a close coupling between the inherent statistics of natural scenes and higher order visual processing in insects.

Predicting cortical dark/bright asymmetries from natural image statistics and early visual transforms

The nervous system has evolved in an environment with structure and predictability. One of the ubiquitous principles of sensory systems is the creation of circuits that capitalize on this predictability. Previous work has identified predictable non-uniformities in the distributions of basic visual features in natural images that are relevant to the encoding tasks of the visual system. Here, we report that the well-established statistical distributions of visual features -- such as visual contrast, spatial scale, and depth -- differ between bright and dark image components. Following this analysis, we go on to trace how these differences in natural images translate into different patterns of cortical input that arise from the separate bright (ON) and dark (OFF) pathways originating in the retina. We use models of these early visual pathways to transform natural images into statistical patterns of cortical input. The models include the receptive fields and non-linear response properties of the magnocellular (M) and parvocellular (P) pathways, with their ON and OFF pathway divisions. The results indicate that there are regularities in visual cortical input beyond those that have previously been appreciated from the direct analysis of natural images. In particular, several dark/bright asymmetries provide a potential account for recently discovered asymmetries in how the brain processes visual features, such as violations of classic energy-type models. On the basis of our analysis, we expect that the dark/bright dichotomy in natural images plays a key role in the generation of both cortical and perceptual asymmetries.

Sensory systems must contend with a tremendous amount of diversity in the natural world. Gaining a detailed description of the natural world’s statistical regularities is a critical part of understanding how the nervous system is adapted to its environment. Here, we report that the well-established statistical distributions of basic visual features—such as visual contrast and spatial scale—diverge when separated into bright and dark components. Operations such as dark/bright segregation are key features of early visual pathways. By modeling these pathways, we demonstrate that the dark and bright visual patterns driving cortical networks are asymmetric across a number of visual features, producing previously unappreciated second-order regularities. The results provide a parsimonious account for recently discovered asymmetries in cortical activity.

Benefits of pathway splitting in sensory coding

In many sensory systems, the neural signal splits into multiple parallel pathways. For example, in the mammalian retina, ~20 types of retinal ganglion cells transmit information about the visual scene to the brain. The purpose of this profuse and early pathway splitting remains unknown. We examine a common instance of splitting into ON and OFF neurons excited by increments and decrements of light intensity in the visual scene, respectively. We test the hypothesis that pathway splitting enables more efficient encoding of sensory stimuli. Specifically, we compare a model system with an ON and an OFF neuron to one with two ON neurons. Surprisingly, the optimal ON-OFF system transmits the same information as the optimal ON-ON system, if one constrains the maximal firing rate of the neurons. However, the ON-OFF system uses fewer spikes on average to transmit this information. This superiority of the ON-OFF system is also observed when the two systems are optimized while constraining their mean firing rate. The efficiency gain for the ON-OFF split is comparable with that derived from decorrelation, a well known processing strategy of early sensory systems. The gain can be orders of magnitude larger when the ecologically important stimuli are rare but large events of either polarity. The ON-OFF system also provides a better code for extracting information by a linear downstream decoder. The results suggest that the evolution of ON-OFF diversification in sensory systems may be driven by the benefits of lowering average metabolic cost, especially in a world in which the relevant stimuli are sparse.

Perceptual Contrast Enhancement with Dynamic Range Adjustment

Recent years, although great efforts have been made to improve its performance, few Histogram equalization (HE) methods take human visual perception (HVP) into account explicitly. The human visual system (HVS) is more sensitive to edges than brightness. This paper proposes to take use of this nature intuitively and develops a perceptual contrast enhancement approach with dynamic range adjustment through histogram modification. The use of perceptual contrast connects the image enhancement problem with the HVS. To pre-condition the input image before the HE procedure is implemented, a perceptual contrast map (PCM) is constructed based on the modified Difference of Gaussian (DOG) algorithm. As a result, the contrast of the image is sharpened and high frequency noise is suppressed. A modified Clipped Histogram Equalization (CHE) is also developed which improves visual quality by automatically detecting the dynamic range of the image with improved perceptual contrast. Experimental results show that the new HE algorithm outperforms several state-of-the-art algorithms in improving perceptual contrast and enhancing details. In addition, the new algorithm is simple to implement, making it suitable for real-time applications.

From image statistics to scene gist: evoked neural activity reveals transition from low-level natural image structure to scene category

The visual system processes natural scenes in a split second. Part of this process is the extraction of "gist," a global first impression. It is unclear, however, how the human visual system computes this information. Here, we show that, when human observers categorize global information in real-world scenes, the brain exhibits strong sensitivity to low-level summary statistics. Subjects rated a specific instance of a global scene property, naturalness, for a large set of natural scenes while EEG was recorded. For each individual scene, we derived two physiologically plausible summary statistics by spatially pooling local contrast filter outputs: contrast energy (CE), indexing contrast strength, and spatial coherence (SC), indexing scene fragmentation. We show that behavioral performance is directly related to these statistics, with naturalness rating being influenced in particular by SC. At the neural level, both statistics parametrically modulated single-trial event-related potential amplitudes during an early, transient window (100-150 ms), but SC continued to influence activity levels later in time (up to 250 ms). In addition, the magnitude of neural activity that discriminated between man-made versus natural ratings of individual trials was related to SC, but not CE. These results suggest that global scene information may be computed by spatial pooling of responses from early visual areas (e.g., LGN or V1). The increased sensitivity over time to SC in particular, which reflects scene fragmentation, suggests that this statistic is actively exploited to estimate scene naturalness.

A no-reference perceptual based contrast enhancement metric for ocean scenes in fog

In this paper, we develop a perceptually based contrast enhancement metric as a means to solve the problem of autonomously enhancing images degraded by fog that are perceptually pleasing to humans. A learning based approach is considered to develop the contrast enhancement using human observations and low-level contrast enhancement metrics based on the human vision system. In addition, we provide new low-level metrics based on the physics of the scene to improve the performance of existing contrast enhancement metrics. This paper shows that a contrast enhancement metric can be designed to mimic human preference.

Effect of stimulus width on simultaneous contrast

Perceived brightness of a stimulus depends on the background against which the stimulus is set, a phenomenon known as simultaneous contrast. For instance, the same gray stimulus can look light against a black background or dark against a white background. Here we quantified the perceptual strength of simultaneous contrast as a function of stimulus width. Previous studies have reported that wider stimuli result in weaker simultaneous contrast, whereas narrower stimuli result in stronger simultaneous contrast. However, no previous research has quantified this relationship. Our results show a logarithmic relationship between stimulus width and perceived brightness. This relationship is well matched by the normalized output of a Difference-of-Gaussians (DOG) filter applied to stimuli of varied widths.

A two-stage cascade model of BOLD responses in human visual cortex

Visual neuroscientists have discovered fundamental properties of neural representation through careful analysis of responses to controlled stimuli. Typically, different properties are studied and modeled separately. To integrate our knowledge, it is necessary to build general models that begin with an input image and predict responses to a wide range of stimuli. In this study, we develop a model that accepts an arbitrary band-pass grayscale image as input and predicts blood oxygenation level dependent (BOLD) responses in early visual cortex as output. The model has a cascade architecture, consisting of two stages of linear and nonlinear operations. The first stage involves well-established computations—local oriented filters and divisive normalization—whereas the second stage involves novel computations—compressive spatial summation (a form of normalization) and a variance-like nonlinearity that generates selectivity for second-order contrast. The parameters of the model, which are estimated from BOLD data, vary systematically across visual field maps: compared to primary visual cortex, extrastriate maps generally have larger receptive field size, stronger levels of normalization, and increased selectivity for second-order contrast. Our results provide insight into how stimuli are encoded and transformed in successive stages of visual processing.

Much has been learned about how stimuli are represented in the visual system from measuring responses to carefully designed stimuli. Typically, different studies focus on different types of stimuli. Making sense of the large array of findings requires integrated models that explain responses to a wide range of stimuli. In this study, we measure functional magnetic resonance imaging (fMRI) responses in early visual cortex to a wide range of band-pass filtered images, and construct a computational model that takes the stimuli as input and predicts the fMRI responses as output. The model has a cascade architecture, consisting of two stages of linear and nonlinear operations. A novel component of the model is a nonlinear operation that generates selectivity for second-order contrast, that is, variations in contrast-energy across the visual field. We find that this nonlinearity is stronger in extrastriate areas V2 and V3 than in primary visual cortex V1. Our results provide insight into how stimuli are encoded and transformed in the visual system.

Functional characterization of the extraclassical receptive field in macaque V1: contrast, orientation, and temporal dynamics

Neurons in primary visual cortex, V1, very often have extraclassical receptive fields (eCRFs). The eCRF is defined as the region of visual space where stimuli cannot elicit a spiking response but can modulate the response of a stimulus in the classical receptive field (CRF). We investigated the dependence of the eCRF on stimulus contrast and orientation in macaque V1 cells for which the laminar location was determined. The eCRF was more sensitive to contrast than the CRF across the whole population of V1 cells with the greatest contrast differential in layer 2/3. We confirmed that many V1 cells experience stronger suppression for collinear than orthogonal stimuli in the eCRF. Laminar analysis revealed that the predominant bias for collinear suppression was found in layers 2/3 and 4b. The laminar pattern of contrast and orientation dependence suggests that eCRF suppression may derive from different neural circuits in different layers, and may be comprised of two distinct components: orientation-tuned and untuned suppression. On average tuned suppression was delayed by ∼25 ms compared with the onset of untuned suppression. Therefore, response modulation by the eCRF develops dynamically and rapidly in time.

Spatially pooled contrast responses predict neural and perceptual similarity of naturalistic image categories

The visual world is complex and continuously changing. Yet, our brain transforms patterns of light falling on our retina into a coherent percept within a few hundred milliseconds. Possibly, low-level neural responses already carry substantial information to facilitate rapid characterization of the visual input. Here, we computationally estimated low-level contrast responses to computer-generated naturalistic images, and tested whether spatial pooling of these responses could predict image similarity at the neural and behavioral level. Using EEG, we show that statistics derived from pooled responses explain a large amount of variance between single-image evoked potentials (ERPs) in individual subjects. Dissimilarity analysis on multi-electrode ERPs demonstrated that large differences between images in pooled response statistics are predictive of more dissimilar patterns of evoked activity, whereas images with little difference in statistics give rise to highly similar evoked activity patterns. In a separate behavioral experiment, images with large differences in statistics were judged as different categories, whereas images with little differences were confused. These findings suggest that statistics derived from low-level contrast responses can be extracted in early visual processing and can be relevant for rapid judgment of visual similarity. We compared our results with two other, well- known contrast statistics: Fourier power spectra and higher-order properties of contrast distributions (skewness and kurtosis). Interestingly, whereas these statistics allow for accurate image categorization, they do not predict ERP response patterns or behavioral categorization confusions. These converging computational, neural and behavioral results suggest that statistics of pooled contrast responses contain information that corresponds with perceived visual similarity in a rapid, low-level categorization task.

Humans excel in rapid and accurate processing of visual scenes. However, it is unclear which computations allow the visual system to convert light hitting the retina into a coherent representation of visual input in a rapid and efficient way. Here we used simple, computer-generated image categories with similar low-level structure as natural scenes to test whether a model of early integration of low-level information can predict perceived category similarity. Specifically, we show that summarized (spatially pooled) responses of model neurons covering the entire visual field (the population response) to low-level properties of visual input (contrasts) can already be informative about differences in early visual evoked activity as well as behavioral confusions of these categories. These results suggest that low-level population responses can carry relevant information to estimate similarity of controlled images, and put forward the exciting hypothesis that the visual system may exploit these responses to rapidly process real natural scenes. We propose that the spatial pooling that allows for the extraction of this information may be a plausible first step in extracting scene gist to form a rapid impression of the visual input.

Automatic exact histogram specification for contrast enhancement and visual system based quantitative evaluation

Histogram equalization, which aims at information maximization, is widely used in different ways to perform contrast enhancement in images. In this paper, an automatic exact histogram specification technique is proposed and used for global and local contrast enhancement of images. The desired histogram is obtained by first subjecting the image histogram to a modification process and then by maximizing a measure that represents increase in information and decrease in ambiguity. A new method of measuring image contrast based upon local band-limited approach and center-surround retinal receptive field model is also devised in this paper. This method works at multiple scales (frequency bands) and combines the contrast measures obtained at different scales using L(p)-norm. In comparison to a few existing methods, the effectiveness of the proposed automatic exact histogram specification technique in enhancing contrasts of images is demonstrated through qualitative analysis and the proposed image contrast measure based quantitative analysis.

Statistical comparison of spike responses to natural stimuli in monkey area V1 with simulated responses of a detailed laminar network model for a patch of V1

A major goal of computational neuroscience is the creation of computer models for cortical areas whose response to sensory stimuli resembles that of cortical areas in vivo in important aspects. It is seldom considered whether the simulated spiking activity is realistic (in a statistical sense) in response to natural stimuli. Because certain statistical properties of spike responses were suggested to facilitate computations in the cortex, acquiring a realistic firing regimen in cortical network models might be a prerequisite for analyzing their computational functions. We present a characterization and comparison of the statistical response properties of the primary visual cortex (V1) in vivo and in silico in response to natural stimuli. We recorded from multiple electrodes in area V1 of 4 macaque monkeys and developed a large state-of-the-art network model for a 5 × 5-mm patch of V1 composed of 35,000 neurons and 3.9 million synapses that integrates previously published anatomical and physiological details. By quantitative comparison of the model response to the "statistical fingerprint" of responses in vivo, we find that our model for a patch of V1 responds to the same movie in a way which matches the statistical structure of the recorded data surprisingly well. The deviation between the firing regimen of the model and the in vivo data are on the same level as deviations among monkeys and sessions. This suggests that, despite strong simplifications and abstractions of cortical network models, they are nevertheless capable of generating realistic spiking activity. To reach a realistic firing state, it was not only necessary to include both N-methyl-d-aspartate and GABA(B) synaptic conductances in our model, but also to markedly increase the strength of excitatory synapses onto inhibitory neurons (>2-fold) in comparison to literature values, hinting at the importance to carefully adjust the effect of inhibition for achieving realistic dynamics in current network models.

Retina is structured to process an excess of darkness in natural scenes

Retinal ganglion cells that respond selectively to a dark spot on a brighter background (OFF cells) have smaller dendritic fields than their ON counterparts and are more numerous. OFF cells also branch more densely, and thus collect more synapses per visual angle. That the retina devotes more resources to processing dark contrasts predicts that natural images contain more dark information. We confirm this across a range of spatial scales and trace the origin of this phenomenon to the statistical structure of natural scenes. We show that the optimal mosaics for encoding natural images are also asymmetric, with OFF elements smaller and more numerous, matching retinal structure. Finally, the concentration of synapses within a dendritic field matches the information content, suggesting a simple principle to connect a concrete fact of neuroanatomy with the abstract concept of information: equal synapses for equal bits.

Robust models for optic flow coding in natural scenes inspired by insect biology

The extraction of accurate self-motion information from the visual world is a difficult problem that has been solved very efficiently by biological organisms utilizing non-linear processing. Previous bio-inspired models for motion detection based on a correlation mechanism have been dogged by issues that arise from their sensitivity to undesired properties of the image, such as contrast, which vary widely between images. Here we present a model with multiple levels of non-linear dynamic adaptive components based directly on the known or suspected responses of neurons within the visual motion pathway of the fly brain. By testing the model under realistic high-dynamic range conditions we show that the addition of these elements makes the motion detection model robust across a large variety of images, velocities and accelerations. Furthermore the performance of the entire system is more than the incremental improvements offered by the individual components, indicating beneficial non-linear interactions between processing stages. The algorithms underlying the model can be implemented in either digital or analog hardware, including neuromorphic analog VLSI, but defy an analytical solution due to their dynamic non-linear operation. The successful application of this algorithm has applications in the development of miniature autonomous systems in defense and civilian roles, including robotics, miniature unmanned aerial vehicles and collision avoidance sensors.

Receptive fields and functional architecture in the retina

Functional architecture of the striate cortex is known mostly at the tissue level--how neurons of different function distribute across its depth and surface on a scale of millimetres. But explanations for its design--why it is just so--need to be addressed at the synaptic level, a much finer scale where the basic description is still lacking. Functional architecture of the retina is known from the scale of millimetres down to nanometres, so we have sought explanations for various aspects of its design. Here we review several aspects of the retina's functional architecture and find that all seem governed by a single principle: represent the most information for the least cost in space and energy. Specifically: (i) why are OFF ganglion cells more numerous than ON cells? Because natural scenes contain more negative than positive contrasts, and the retina matches its neural resources to represent them equally well; (ii) why do ganglion cells of a given type overlap their dendrites to achieve 3-fold coverage? Because this maximizes total information represented by the array--balancing signal-to-noise improvement against increased redundancy; (iii) why do ganglion cells form multiple arrays? Because this allows most information to be sent at lower rates, decreasing the space and energy costs for sending a given amount of information. This broad principle, operating at higher levels, probably contributes to the brain's immense computational efficiency.

Direction and contrast tuning of macaque MSTd neurons during saccades

Saccades are rapid eye movements that change the direction of gaze, although the full-field image motion associated with these movements is rarely perceived. The attenuation of visual perception during saccades is referred to as saccadic suppression. The mechanisms that produce saccadic suppression are not well understood. We recorded from neurons in the dorsal medial superior temporal area (MSTd) of alert macaque monkeys and compared the neural responses produced by the retinal slip associated with saccades (active motion) to responses evoked by identical motion presented during fixation (passive motion). We provide evidence for a neural correlate of saccadic suppression and expand on two contentious results from previous studies. First, we confirm the finding that some neurons in MSTd reverse their preferred direction during saccades. We quantify this effect by calculating changes in direction tuning index for a large cell population. Second, it has been noted that neural activity associated with saccades can arrive in the parietal cortex <or=30 ms earlier than activity produced by similar visual stimulation during fixation. This led to the question of whether the saccade-related responses were visual in origin or were motor signals arising from saccade-planning areas of the brain. By comparing the responses to saccades made over textured backgrounds of different contrasts, we provide strong evidence that saccade-related responses were visual in origin. Refinements of the possible models of saccadic suppression are discussed.

Image statistics at the point of gaze during human navigation

Theories of efficient sensory processing have considered the regularities of image properties due to the structure of the environment in order to explain properties of neuronal representations of the visual world. The regularities imposed on the input to the visual system due to the regularities of the active selection process mediated by the voluntary movements of the eyes have been considered to a much lesser degree. This is surprising, given that the active nature of vision is well established. The present article investigates statistics of image features at the center of gaze of human subjects navigating through a virtual environment and avoiding and approaching different objects. The analysis shows that contrast can be significantly higher or lower at fixation location compared to random locations, depending on whether subjects avoid or approach targets. Similarly, significant differences in the distribution of responses of model simple and complex cells between horizontal and vertical orientations are found over timescales of tens of seconds. By clustering the model simple cell responses, it is established that gaze was directed toward three distinct features of intermediate complexity the vast majority of time. Thus, this study demonstrates and quantifies how the visuomotor tasks of approaching and avoiding objects during navigation determine feature statistics of the input to the visual system through the combined influence on body and eye movements.

Adaptation and selective information transmission in the cricket auditory neuron AN2

Sensory systems adapt their neural code to changes in the sensory environment, often on multiple time scales. Here, we report a new form of adaptation in a first-order auditory interneuron (AN2) of crickets. We characterize the response of the AN2 neuron to amplitude-modulated sound stimuli and find that adaptation shifts the stimulus–response curves toward higher stimulus intensities, with a time constant of 1.5 s for adaptation and recovery. The spike responses were thus reduced for low-intensity sounds. We then address the question whether adaptation leads to an improvement of the signal's representation and compare the experimental results with the predictions of two competing hypotheses: infomax, which predicts that information conveyed about the entire signal range should be maximized, and selective coding, which predicts that “foreground” signals should be enhanced while “background” signals should be selectively suppressed. We test how adaptation changes the input–response curve when presenting signals with two or three peaks in their amplitude distributions, for which selective coding and infomax predict conflicting changes. By means of Bayesian data analysis, we quantify the shifts of the measured response curves and also find a slight reduction of their slopes. These decreases in slopes are smaller, and the absolute response thresholds are higher than those predicted by infomax. Most remarkably, and in contrast to the infomax principle, adaptation actually reduces the amount of encoded information when considering the whole range of input signals. The response curve changes are also not consistent with the selective coding hypothesis, because the amount of information conveyed about the loudest part of the signal does not increase as predicted but remains nearly constant. Less information is transmitted about signals with lower intensity.

Sensory systems have the ability to adapt to changes in the environment. In a quiet room, the nervous system is very responsive, so that even a whisper can be easily understood. In contrast, the perceived loudness on a crowded street will be reduced to prevent an overload of the nervous system. Two different hypotheses have been proposed to explain how the nervous system achieves this adaptation. According to one idea, all present sensory signals are equally enhanced, so that the whole range of input signals is reliably represented. On the other hand, the aim of the nervous system may be to extract the most important parts of the acoustic signal, for example, an approaching car, and thus abolish the irrelevant rest. To address which of these two principles is implemented in the auditory system of the cricket, we investigated the responses of a single auditory neuron, called interneuron AN2, to different sound signals. We found that adaptation actually reduces the amount of encoded information when considering the whole range of input signals. However, the changes were also not in agreement with the idea that only the most important signal is transmitted, because the amount of information conveyed about the loudest part of the signal does not increase. Thus, we here report the unusual case of a reduction of information transfer by adaptation, while in most other systems reported of so far adaptation actually enhances coding of sensory information.