Adaptation of retinal processing to image contrast and spatial scale

The tuning of tuning: How adaptation influences single cell information transfer

Sensory neurons reconstruct the world from action potentials (spikes) impinging on them. To effectively transfer information about the stimulus to the next processing level, a neuron needs to be able to adapt its working range to the properties of the stimulus. Here, we focus on the intrinsic neural properties that influence information transfer in cortical neurons and how tightly their properties need to be tuned to the stimulus statistics for them to be effective. We start by measuring the intrinsic information encoding properties of putative excitatory and inhibitory neurons in L2/3 of the mouse barrel cortex. Excitatory neurons show high thresholds and strong adaptation, making them fire sparsely and resulting in a strong compression of information, whereas inhibitory neurons that favour fast spiking transfer more information. Next, we turn to computational modelling and ask how two properties influence information transfer: 1) spike-frequency adaptation and 2) the shape of the IV-curve. We find that a subthreshold (but not threshold) adaptation, the 'h-current', and a properly tuned leak conductance can increase the information transfer of a neuron, whereas threshold adaptation can increase its working range. Finally, we verify the effect of the IV-curve slope in our experimental recordings and show that excitatory neurons form a more heterogeneous population than inhibitory neurons. These relationships between intrinsic neural features and neural coding that had not been quantified before will aid computational, theoretical and systems neuroscientists in understanding how neuronal populations can alter their coding properties, such as through the impact of neuromodulators. Why the variability of intrinsic properties of excitatory neurons is larger than that of inhibitory ones is an exciting question, for which future research is needed.

Predictive extrapolation effects can have a greater impact on visual decisions, while visual adaptation has a greater impact on conscious visual experience

Human vision is shaped by historic and by predictive processes. The lingering impact of visual adaptation, for instance, can act to exaggerate differences between past and present inputs, whereas predictive processes can promote extrapolation effects that allow us to anticipate the near future. It is unclear to what extent either of these effects manifest in changes to conscious visual experience. It is also unclear how these influences combine, when acting in concert or opposition. We had people make decisions about the sizes of inputs, and report on levels of decisional confidence. Tests were either selectively subject to size adaptation, to an extrapolation effect, or to both of these effects. When these two effects were placed in opposition, extrapolation had a greater impact on decision making. However, our data suggest the influence of extrapolation is primarily decisional, whereas size adaptation more fully manifests in changes to conscious visual awareness.

Two Sides of the Same Coin: Efficient and Predictive Neural Coding

Some visual properties are consistent across a wide range of environments, while other properties are more labile. The efficient coding hypothesis states that many of these regularities in the environment can be discarded from neural representations, thus allocating more of the brain's dynamic range to properties that are likely to vary. This paradigm is less clear about how the visual system prioritizes different pieces of information that vary across visual environments. One solution is to prioritize information that can be used to predict future events, particularly those that guide behavior. The relationship between the efficient coding and future prediction paradigms is an area of active investigation. In this review, we argue that these paradigms are complementary and often act on distinct components of the visual input. We also discuss how normative approaches to efficient coding and future prediction can be integrated.

Interpreting the retinal neural code for natural scenes: From computations to neurons

Understanding the circuit mechanisms of the visual code for natural scenes is a central goal of sensory neuroscience. We show that a three-layer network model predicts retinal natural scene responses with an accuracy nearing experimental limits. The model's internal structure is interpretable, as interneurons recorded separately and not modeled directly are highly correlated with model interneurons. Models fitted only to natural scenes reproduce a diverse set of phenomena related to motion encoding, adaptation, and predictive coding, establishing their ethological relevance to natural visual computation. A new approach decomposes the computations of model ganglion cells into the contributions of model interneurons, allowing automatic generation of new hypotheses for how interneurons with different spatiotemporal responses are combined to generate retinal computations, including predictive phenomena currently lacking an explanation. Our results demonstrate a unified and general approach to study the circuit mechanisms of ethological retinal computations under natural visual scenes.

Sources of Calcium at Connexin 36 Gap Junctions in the Retina

Synaptic plasticity is a fundamental feature of the CNS that controls the magnitude of signal transmission between communicating cells. Many electrical synapses exhibit substantial plasticity that modulates the degree of coupling within groups of neurons, alters the fidelity of signal transmission, or even reconfigures functional circuits. In several known examples, such plasticity depends on calcium and is associated with neuronal activity. Calcium-driven signaling is known to promote potentiation of electrical synapses in fish Mauthner cells, mammalian retinal AII amacrine cells, and inferior olive neurons, and to promote depression in thalamic reticular neurons. To measure local calcium dynamics in situ, we developed a transgenic mouse expressing a GCaMP calcium biosensor fused to Connexin 36 (Cx36) at electrical synapses. We examined the sources of calcium for activity-dependent plasticity in retina slices using confocal or Super-Resolution Radial Fluctuations imaging. More than half of Cx36-GCaMP gap junctions responded to puffs of glutamate with transient increases in fluorescence. The responses were strongly dependent on NMDA receptors, in keeping with known activity-dependent signaling in some amacrine cells. We also found that some responses depended on the activity of voltage-gated calcium channels, representing a previously unrecognized source of calcium to control retinal electrical synaptic plasticity. The high prevalence of calcium signals at electrical synapses in response to glutamate application indicates that a large fraction of electrical synapses has the potential to be regulated by neuronal activity. This provides a means to tune circuit connectivity dynamically based on local activity.

Contrast normalization affects response time-course of visual interneurons

In natural environments, light intensities and visual contrasts vary widely, yet neurons have a limited response range for encoding them. Neurons accomplish that by flexibly adjusting their dynamic range to the statistics of the environment via contrast normalization. The effect of contrast normalization is usually measured as a reduction of neural signal amplitudes, but whether it influences response dynamics is unknown. Here, we show that contrast normalization in visual interneurons of Drosophila melanogaster not only suppresses the amplitude but also alters the dynamics of responses when a dynamic surround is present. We present a simple model that qualitatively reproduces the simultaneous effect of the visual surround on the response amplitude and temporal dynamics by altering the cells' input resistance and, thus, their membrane time constant. In conclusion, single-cell filtering properties as derived from artificial stimulus protocols like white-noise stimulation cannot be transferred one-to-one to predict responses under natural conditions.

Rapid adaptation of primate LGN neurons to drifting grating stimulation

The visual system needs to dynamically adapt to changing environments. Much is known about the adaptive effects of constant stimulation over prolonged periods. However, there are open questions regarding adaptation to stimuli that are changing over time, interrupted, or repeated. Feature-specific adaptation to repeating stimuli has been shown to occur as early as primary visual cortex (V1), but there is also evidence for more generalized, fatigue-like adaptation that might occur at an earlier stage of processing. Here, we show adaptation in the lateral geniculate nucleus (LGN) of awake, fixating monkeys following brief (1 s) exposure to repeated cycles of a 4-Hz drifting grating. We examined the relative change of each neuron's response across successive (repeated) grating cycles. We found that neurons from all cell classes (parvocellular, magnocellular, and koniocellular) showed significant adaptation. However, only magnocellular neurons showed adaptation when responses were averaged to a population response. In contrast to firing rates, response variability was largely unaffected. Finally, adaptation was comparable between monocular and binocular stimulation, suggesting that rapid LGN adaptation is monocular in nature.NEW & NOTEWORTHY Neural adaptation can be defined as reduction of spiking responses following repeated or prolonged stimulation. Adaptation helps adjust neural responsiveness to avoid saturation and has been suggested to improve perceptual selectivity, information transmission, and predictive coding. Here, we report rapid adaptation to repeated cycles of gratings drifting over the receptive field of neurons at the earliest site of postretinal processing, the lateral geniculate nucleus of the thalamus.

Multifaceted luminance gain control beyond photoreceptors in Drosophila

Animals navigating in natural environments must handle vast changes in their sensory input. Visual systems, for example, handle changes in luminance at many timescales, from slow changes across the day to rapid changes during active behavior. To maintain luminance-invariant perception, visual systems must adapt their sensitivity to changing luminance at different timescales. We demonstrate that luminance gain control in photoreceptors alone is insufficient to explain luminance invariance at both fast and slow timescales and reveal the algorithms that adjust gain past photoreceptors in the fly eye. We combined imaging and behavioral experiments with computational modeling to show that downstream of photoreceptors, circuitry taking input from the single luminance-sensitive neuron type L3 implements gain control at fast and slow timescales. This computation is bidirectional in that it prevents the underestimation of contrasts in low luminance and overestimation in high luminance. An algorithmic model disentangles these multifaceted contributions and shows that the bidirectional gain control occurs at both timescales. The model implements a nonlinear interaction of luminance and contrast to achieve gain correction at fast timescales and a dark-sensitive channel to improve the detection of dim stimuli at slow timescales. Together, our work demonstrates how a single neuronal channel performs diverse computations to implement gain control at multiple timescales that are together important for navigation in natural environments.

Spike timing-dependent plasticity alters electrosensory neuron synaptic strength in vitro but does not consistently predict changes in sensory tuning in vivo

How do sensory systems optimize detection of behaviorally relevant stimuli when the sensory environment is constantly changing? We addressed the role of spike timing-dependent plasticity (STDP) in driving changes in synaptic strength in a sensory pathway and whether those changes in synaptic strength could alter sensory tuning. It is challenging to precisely control temporal patterns of synaptic activity in vivo and replicate those patterns in vitro in behaviorally relevant ways. This makes it difficult to make connections between STDP-induced changes in synaptic physiology and plasticity in sensory systems. Using the mormyrid species Brevimyrus niger and Brienomyrus brachyistius, which produce electric organ discharges for electrolocation and communication, we can precisely control the timing of synaptic input in vivo and replicate these same temporal patterns of synaptic input in vitro. In central electrosensory neurons in the electric communication pathway, using whole cell intracellular recordings in vitro, we paired presynaptic input with postsynaptic spiking at different delays. Using whole cell intracellular recordings in awake, behaving fish, we paired sensory stimulation with postsynaptic spiking using the same delays. We found that Hebbian STDP predictably alters sensory tuning in vitro and is mediated by NMDA receptors. However, the change in synaptic responses induced by sensory stimulation in vivo did not adhere to the direction predicted by the STDP observed in vitro. Further analysis suggests that this difference is influenced by polysynaptic activity, including inhibitory interneurons. Our findings suggest that STDP rules operating at identified synapses may not drive predictable changes in sensory responses at the circuit level.NEW & NOTEWORTHY We replicated behaviorally relevant temporal patterns of synaptic activity in vitro and used the same patterns during sensory stimulation in vivo. There was a Hebbian spike timing-dependent plasticity (STDP) pattern in vitro, but sensory responses in vivo did not shift according to STDP predictions. Analysis suggests that this disparity is influenced by differences in polysynaptic activity, including inhibitory interneurons. These results suggest that STDP rules at synapses in vitro do not necessarily apply to circuits in vivo.

Wild animals suppress the spread of socially transmitted misinformation

Understanding the mechanisms by which information and misinformation spread through groups of individual actors is essential to the prediction of phenomena ranging from coordinated group behaviors to misinformation epidemics. Transmission of information through groups depends on the rules that individuals use to transform the perceived actions of others into their own behaviors. Because it is often not possible to directly infer decision-making strategies in situ, most studies of behavioral spread assume that individuals make decisions by pooling or averaging the actions or behavioral states of neighbors. However, whether individuals may instead adopt more sophisticated strategies that exploit socially transmitted information, while remaining robust to misinformation, is unknown. Here, we study the relationship between individual decision-making and misinformation spread in groups of wild coral reef fish, where misinformation occurs in the form of false alarms that can spread contagiously through groups. Using automated visual field reconstruction of wild animals, we infer the precise sequences of socially transmitted visual stimuli perceived by individuals during decision-making. Our analysis reveals a feature of decision-making essential for controlling misinformation spread: dynamic adjustments in sensitivity to socially transmitted cues. This form of dynamic gain control can be achieved by a simple and biologically widespread decision-making circuit, and it renders individual behavior robust to natural fluctuations in misinformation exposure.

Modality predictability modulation confirms the sensorial readiness function of the pre-stimulus activity in sensory brain areas

The auditory Positivity (aP) and the visual Negativity (vN) are recently discovered modality-specific event-related potential (ERP) components associated with sensory readiness, which seems promising to study anticipatory perception and attention. However, a crucial aspect of these waves remains to be determined since it is still unclear if these components are indeed related to sensory readiness or represent the result of stimulus predictably. Indeed, earlier studies found these components in tasks where stimuli were repeatedly presented uniquely in the same sensory modality. To disentangle this issue, we used an experimental design consisting of three passive tasks: a unimodal auditory condition, a unimodal visual condition, and an intermodal condition in which the visual and auditory stimuli were unpredictably alternated. Then, we compared the amplitudes of the aP and vN in the three conditions and performed correlation analyses between pre-stimulus and post-stimulus components. Crucially, results showed that in the intermodal condition the components still occur, but their amplitudes are decreased compared to unimodal condition, providing evidence that they are only partially dependent on the task and that expectancy might modulate them. This result is in line with the "modality-shift effect" costs phenomenon which can occur also for passive tasks even before stimulus presentation. In addition, the amplitude of the post-stimulus components correlated with pre-stimulus ERP. Collectively, the present study confirms that the aP and the vN reflect sensory readiness processes that "boost" post-stimulus auditory N1 and visual P1 components.

After-image formation by adaptation to dynamic color gradients

The eye's retinotopic exposure to an adapter typically produces an after-image. For example, an observer who fixates a red adapter on a gray background will see an illusory cyan after-image after removing the adapter. The after-image's content, like its color or intensity, gives insight into mechanisms responsible for adaptation and processing of a specific feature. To facilitate adaptation, vision scientists traditionally present stable, unchanging adapters for prolonged durations. How adaptation affects perception when features (e.g., color) dynamically change over time is not understood. To investigate adaptation to a dynamically changing feature, participants viewed a colored patch that changed from a color to gray, following either a direct or curved path through the (roughly) equiluminant color plane of CIE LAB space. We varied the speed and curvature of color changes across trials and experiments. Results showed that dynamic adapters produce after-images, vivid enough to be reported by the majority of participants. An after-image consisted of a color complementary to the average of the adapter's colors with a small bias towards more recent rather than initial adapter colors. The modelling of the reported after-image colors further confirmed that adaptation rapidly instigates and gradually dissipates. A second experiment replicated these results and further showed that the probability of observing an after-image diminishes only slightly when the adapter displays transient (stepwise, abrupt) color transitions. We conclude from the results that the visual system can adapt to dynamic colors, to a degree that is robust to the potential interference of transient changes in adapter content.

Calibrating Vision: Concepts and Questions

The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.

Memoryless Optimality: Neurons Do Not Need Adaptation to Optimally Encode Stimuli With Arbitrarily Complex Statistics

Our neurons seem capable of handling any type of data, regardless of its scale or statistical properties. In this letter, we suggest that optimal coding may occur at the single-neuron level without requiring memory, adaptation, or evolutionary-driven fit to the stimuli. We refer to a neural circuit as optimal if it maximizes the mutual information between its inputs and outputs. We show that often encountered differentiator neurons, or neurons that respond mainly to changes in the input, are capable of using all their information capacity when handling samples of any statistical distribution. We demonstrate this optimality using both analytical methods and simulations. In addition to demonstrating the simplicity and elegance of neural processing, this result might provide a way to improve the handling of data by artificial neural networks.

Nonequilibrium dynamics of adaptation in sensory systems

Adaptation is used by biological sensory systems to respond to a wide range of environmental signals, by adapting their response properties to the statistics of the stimulus in order to maximize information transmission. We derive rules of optimal adaptation to changes in the mean and variance of a continuous stimulus in terms of Bayesian filters and map them onto stochastic equations that couple the state of the environment to an internal variable controlling the response function. We calculate numerical and exact results for the speed and accuracy of adaptation and its impact on information transmission. We find that, in the regime of efficient adaptation, the speed of adaptation scales sublinearly with the rate of change of the environment. Finally, we exploit the mathematical equivalence between adaptation and stochastic thermodynamics to quantitatively relate adaptation to the irreversibility of the adaptation time course, defined by the rate of entropy production. Our results suggest a means to empirically quantify adaptation in a model-free and nonparametric way.

An Irradiance-Adaptable Near-Infrared Vertical Heterojunction Phototransistor

Bioinspired artificial visual perception devices with the optical environment-adaptable function have attracted significant attention for their promising potential in applications like robotics and machine vision. In this regard, a photodetector with in-sensor adaptability is longed for in terms of complexity, efficiency, and cost. Here, a near-infrared phototransistor with a benign light irradiance-adaptability is presented. The phototransistor uses a vertically stacking graphene/lead sulfide quantum dots/graphene heterojunction as the conductive channel. Compared with ordinary lead sulfide quantum dots-decorated graphene phototransistors, the present device demonstrates a faster photoresponse speed and an abnormal transfer characteristic. The latter characteristic is induced by the gate voltage-tunable Fermi level in the heterojunction and the abundant electron trap states in the quantum dot film, which jointly results in an intense dependence of the photoresponse on the gate voltage. The dynamic trapping and de-trapping processes in the quantum dot film enable the inhibition or potentiation of the photoresponse, based on which the photopic or scotopic adaptation behavior of the human retina is successfully mimicked, respectively. By providing an irradiance-adaptable photodetector with a spectral response beyond visible light, this work should inspire future research on artificial environment-adaptable perception devices.

Comparing adaptive coding of reward in bipolar I disorder and schizophrenia

Deficits in neural processing of reward have been described in both bipolar disorder (BD) and schizophrenia (SZ), but it remains unclear to what extent these deficits are caused by similar mechanisms. Efficient reward processing relies on adaptive coding which allows representing large input spans by limited neuronal encoding ranges. Deficits in adaptive coding of reward have previously been observed across the SZ spectrum and correlated with total symptom severity. In the present work, we sought to establish whether adaptive coding is similarly affected in patients with BD. Twenty-five patients with BD, 27 patients with SZ and 25 healthy controls performed a variant of the Monetary Incentive Delay task during functional magnetic resonance imaging in two reward range conditions. Adaptive coding was impaired in the posterior part of the right caudate in BD and SZ (trend level). In contrast, BD did not show impaired adaptive coding in the anterior caudate and right precentral gyrus/insula, where SZ showed deficits compared to healthy controls. BD patients show adaptive coding deficits that are similar to those observed in SZ in the right posterior caudate. Adaptive coding in BD appeared more preserved as compared to SZ participants especially in the more anterior part of the right caudate and to a lesser extent also in the right precentral gyrus. Thus, dysfunctional adaptive coding could constitute a fundamental deficit in severe mental illnesses that extends beyond the SZ spectrum.

Opposite forms of adaptation in mouse visual cortex are controlled by distinct inhibitory microcircuits

Sensory processing in the cortex adapts to the history of stimulation but the mechanisms are not understood. Imaging the primary visual cortex of mice we find here that an increase in stimulus contrast is not followed by a simple decrease in gain of pyramidal cells; as many cells increase gain to improve detection of a subsequent decrease in contrast. Depressing and sensitizing forms of adaptation also occur in different types of interneurons (PV, SST and VIP) and the net effect within individual pyramidal cells reflects the balance of PV inputs, driving depression, and a subset of SST interneurons driving sensitization. Changes in internal state associated with locomotion increase gain across the population of pyramidal cells while maintaining the balance between these opposite forms of plasticity, consistent with activation of both VIP->SST and SST->PV disinhibitory pathways. These results reveal how different inhibitory microcircuits adjust the gain of pyramidal cells signalling changes in stimulus strength.

The authors describe the role of inhibitory microcircuits in the visual cortex of mice in adaptation to contrast. They show how external stimuli and internal state interact to adjust processing in the visual cortex.

Polarized image near-natural color fusion algorithm for target detection

In some automatic systems, target detection is a common task, and visible images are common sources of raw data. Researchers have confirmed that polarization information highlights manmade targets. We propose an algorithm that fuses polarized and visible images to improve detection accuracy. First, the polarization parameter and visible images are simultaneously converted to the HSV color space. The initial fused image after adjusting the hue and saturation will be transformed into the lab color space. Then, the bisecting k-means algorithm is employed to segment the visible image. The visible and initial images are divided into three types of regions for color transfer in lab color space. Finally, the fused image is transformed back to the RGB color space, and the PolarLITIS data set is applied. The experimental results show that the gradient and contrast of the fused image are improved by 115% and 235.3%, respectively, compared with the visible image, and the final fused image is suitable to view with the naked eye. The proposed algorithm significantly improves accuracy.

Novel stimuli evoke excess activity in the mouse primary visual cortex

Rapid detection and processing of stimulus novelty are key elements of adaptive behavior. Predictive coding theories postulate that novel stimuli should be encoded differently from familiar stimuli. Here, we show that the majority of neurons in layer 2/3 of the mouse primary visual cortex exhibit a significant excess response to novel visual stimuli. The distinction between novel and familiar images developed rapidly, requiring only a few repeated presentations. We show that this phenomenon can be described by a model of cascading adaptation. This ubiquitous mechanism makes it likely that similar computations could be carried out in many brain areas.

To explore how neural circuits represent novel versus familiar inputs, we presented mice with repeated sets of images with novel images sparsely substituted. Using two-photon calcium imaging to record from layer 2/3 neurons in the mouse primary visual cortex, we found that novel images evoked excess activity in the majority of neurons. This novelty response rapidly emerged, arising with a time constant of 2.6 ± 0.9 s. When a new image set was repeatedly presented, a majority of neurons had similarly elevated activity for the first few presentations, which decayed to steady state with a time constant of 1.4 ± 0.4 s. When we increased the number of images in the set, the novelty response’s amplitude decreased, defining a capacity to store ∼15 familiar images under our conditions. These results could be explained quantitatively using an adaptive subunit model in which presynaptic neurons have individual tuning and gain control. This result shows that local neural circuits can create different representations for novel versus familiar inputs using generic, widely available mechanisms.

Advances in materials and devices for mimicking sensory adaptation

Adaptive devices, which aim to adjust electrical behaviors autonomically to external stimuli, are considered to be attractive candidates for next-generation artificial perception systems. Compared with typical electronic devices with stable signal output, adaptive devices possess unique features in exhibiting dynamic fitness to varying environments. To meet this requirement, increasing efforts have been made focusing on developing new materials, functional interfaces and novel device geometry for sensory perception applications. In this review, we summarize the recent advances in materials and devices for mimicking sensory adaptation. Keeping this in mind, we first introduce the fundamentals of biological sensory adaptation. Thereafter, the recent progress in mimicking sensory adaptation, such as tactile and visual adaptive systems, is overviewed. Moreover, we suggest five strategies to construct adaptive devices. Finally, challenges and perspectives are proposed to highlight the directions that deserve focused attention in this flourishing field.

Sensory Adaptation in the Whisker-Mediated Tactile System: Physiology, Theory, and Function

In the natural environment, organisms are constantly exposed to a continuous stream of sensory input. The dynamics of sensory input changes with organism's behaviour and environmental context. The contextual variations may induce >100-fold change in the parameters of the stimulation that an animal experiences. Thus, it is vital for the organism to adapt to the new diet of stimulation. The response properties of neurons, in turn, dynamically adjust to the prevailing properties of sensory stimulation, a process known as "neuronal adaptation." Neuronal adaptation is a ubiquitous phenomenon across all sensory modalities and occurs at different stages of processing from periphery to cortex. In spite of the wealth of research on contextual modulation and neuronal adaptation in visual and auditory systems, the neuronal and computational basis of sensory adaptation in somatosensory system is less understood. Here, we summarise the recent finding and views about the neuronal adaptation in the rodent whisker-mediated tactile system and further summarise the functional effect of neuronal adaptation on the response dynamics and encoding efficiency of neurons at single cell and population levels along the whisker-mediated touch system in rodents. Based on direct and indirect pieces of evidence presented here, we suggest sensory adaptation provides context-dependent functional mechanisms for noise reduction in sensory processing, salience processing and deviant stimulus detection, shift between integration and coincidence detection, band-pass frequency filtering, adjusting neuronal receptive fields, enhancing neural coding and improving discriminability around adapting stimuli, energy conservation, and disambiguating encoding of principal features of tactile stimuli.

A mechanistically interpretable model of the retinal neural code for natural scenes with multiscale adaptive dynamics

The visual system processes stimuli over a wide range of spatiotemporal scales, with individual neurons receiving input from tens of thousands of neurons whose dynamics range from milliseconds to tens of seconds. This poses a challenge to create models that both accurately capture visual computations and are mechanistically interpretable. Here we present a model of salamander retinal ganglion cell spiking responses recorded with a multielectrode array that captures natural scene responses and slow adaptive dynamics. The model consists of a three-layer convolutional neural network (CNN) modified to include local recurrent synaptic dynamics taken from a linear-nonlinear-kinetic (LNK) model [1]. We presented alternating natural scenes and uniform field white noise stimuli designed to engage slow contrast adaptation. To overcome difficulties fitting slow and fast dynamics together, we first optimized all fast spatiotemporal parameters, then separately optimized recurrent slow synaptic parameters. The resulting full model reproduces a wide range of retinal computations and is mechanistically interpretable, having internal units that correspond to retinal interneurons with biophysically modeled synapses. This model allows us to study the contribution of model units to any retinal computation, and examine how long-term adaptation changes the retinal neural code for natural scenes through selective adaptation of retinal pathways.

A precise and adaptive neural mechanism for predictive temporal processing in the frontal cortex

The theory of predictive processing posits that the brain computes expectations to process information predictively. Empirical evidence in support of this theory, however, is scarce and largely limited to sensory areas. Here, we report a precise and adaptive mechanism in the frontal cortex of non-human primates consistent with predictive processing of temporal events. We found that the speed of neural dynamics is precisely adjusted according to the average time of an expected stimulus. This speed adjustment, in turn, enables neurons to encode stimuli in terms of deviations from expectation. This lawful relationship was evident across multiple experiments and held true during learning: when temporal statistics underwent covert changes, neural responses underwent predictable changes that reflected the new mean. Together, these results highlight a precise mathematical relationship between temporal statistics in the environment and neural activity in the frontal cortex that may serve as a mechanism for predictive temporal processing.

Dynamics of Visual Adaptation With Simultaneous Stimulation of Two Visual Pathways

Primates’ retinal ganglion cells in different visual pathways have been shown to adapt independently (Current Biology 22 (2012) 220–224). However, the manner in which adaptation occurs under simultaneous stimulation of two visual pathways has not yet been explored. In this study, the dynamics of color afterimages were measured while stimulating one or two visual pathway using a time-varying afterimage paradigm. The dynamics of adaptation was approximately equivalent among the three primary visual pathways, but adaptation was slower for simultaneous stimulation of two visual pathways compared to the stimulation of one visual pathway. In addition, we found that the speed of adaptation also depends upon which two pathways are combined. We developed a two-stage adaptation model, both with the same dynamics, to account for the results with simultaneous stimulation of two pathways.

Matching visual induction effects on screens of different size

In the film industry, the same movie is expected to be watched on displays of vastly different sizes, from cinema screens to mobile phones. But visual induction, the perceptual phenomenon by which the appearance of a scene region is affected by its surroundings, will be different for the same image shown on two displays of different dimensions. This phenomenon presents a practical challenge for the preservation of the artistic intentions of filmmakers, because it can lead to shifts in image appearance between viewing destinations. In this work, we show that a neural field model based on the efficient representation principle is able to predict induction effects and how, by regularizing its associated energy functional, the model is still able to represent induction but is now invertible. From this finding, we propose a method to preprocess an image in a screen-size dependent way so that its perception, in terms of visual induction, may remain constant across displays of different size. The potential of the method is demonstrated through psychophysical experiments on synthetic images and qualitative examples on natural images.

Efficient and adaptive sensory codes

The ability to adapt to changes in stimulus statistics is a hallmark of sensory systems. Here, we developed a theoretical framework that can account for the dynamics of adaptation from an information processing perspective. We use this framework to optimize and analyze adaptive sensory codes, and we show that codes optimized for stationary environments can suffer from prolonged periods of poor performance when the environment changes. To mitigate the adversarial effects of these environmental changes, sensory systems must navigate tradeoffs between the ability to accurately encode incoming stimuli and the ability to rapidly detect and adapt to changes in the distribution of these stimuli. We derive families of codes that balance these objectives, and we demonstrate their close match to experimentally observed neural dynamics during mean and variance adaptation. Our results provide a unifying perspective on adaptation across a range of sensory systems, environments, and sensory tasks.

Unsupervised learning predicts human perception and misperception of gloss

Reflectance, lighting and geometry combine in complex ways to create images. How do we disentangle these to perceive individual properties, such as surface glossiness? We suggest that brains disentangle properties by learning to model statistical structure in proximal images. To test this hypothesis, we trained unsupervised generative neural networks on renderings of glossy surfaces and compared their representations with human gloss judgements. The networks spontaneously cluster images according to distal properties such as reflectance and illumination, despite receiving no explicit information about these properties. Intriguingly, the resulting representations also predict the specific patterns of ‘successes’ and ‘errors’ in human perception. Linearly decoding specular reflectance from the model’s internal code predicts human gloss perception better than ground truth, supervised networks or control models, and it predicts, on an image-by-image basis, illusions of gloss perception caused by interactions between material, shape and lighting. Unsupervised learning may underlie many perceptual dimensions in vision and beyond.

Storrs et al. train unsupervised generative neural networks on glossy surfaces and show how gloss perception in humans may emerge in an unsupervised fashion from learning to model statistical structure.

High Contrast Allows the Retina to Compute More Than Just Contrast

The goal of sensory processing is to represent the environment of an animal. All sensory systems share a similar constraint: they need to encode a wide range of stimulus magnitudes within their narrow neuronal response range. The most efficient way, exploited by even the simplest nervous systems, is to encode relative changes in stimulus magnitude rather than the absolute magnitudes. For instance, the retina encodes contrast, which are the variations of light intensity occurring in time and in space. From this perspective, it is easy to understand why the bright plumage of a moving bird gains a lot of attention, while an octopus remains motionless and mimics its surroundings for concealment. Stronger contrasts simply cause stronger visual signals. However, the gains in retinal performance associated with higher contrast are far more than what can be attributed to just a trivial linear increase in signal strength. Here we discuss how this improvement in performance is reflected throughout different parts of the neural circuitry, within its neural code and how high contrast activates many non-linear mechanisms to unlock several sophisticated retinal computations that are virtually impossible in low contrast conditions.

Pupillary Responses for Cognitive Load Measurement to Classify Difficulty Levels in an Educational Video Game: Empirical Study

Background

A learning task recurrently perceived as easy (or hard) may cause poor learning results. Gamer data such as errors, attempts, or time to finish a challenge are widely used to estimate the perceived difficulty level. In other contexts, pupillometry is widely used to measure cognitive load (mental effort); hence, this may describe the perceived task difficulty.

Objective

This study aims to assess the use of task-evoked pupillary responses to measure the cognitive load measure for describing the difficulty levels in a video game. In addition, it proposes an image filter to better estimate baseline pupil size and to reduce the screen luminescence effect.

Methods

We conducted an experiment that compares the baseline estimated from our filter against that estimated from common approaches. Then, a classifier with different pupil features was used to classify the difficulty of a data set containing information from students playing a video game for practicing math fractions.

Results

We observed that the proposed filter better estimates a baseline. Mauchly’s test of sphericity indicated that the assumption of sphericity had been violated (χ²₁₄=0.05; P=.001); therefore, a Greenhouse-Geisser correction was used (ε=0.47). There was a significant difference in mean pupil diameter change (MPDC) estimated from different baseline images with the scramble filter (F_5,78=30.965; P<.001). Moreover, according to the Wilcoxon signed rank test, pupillary response features that better describe the difficulty level were MPDC (z=−2.15; P=.03) and peak dilation (z=−3.58; P<.001). A random forest classifier for easy and hard levels of difficulty showed an accuracy of 75% when the gamer data were used, but the accuracy increased to 87.5% when pupillary measurements were included.

Conclusions

The screen luminescence effect on pupil size is reduced with a scrambled filter on the background video game image. Finally, pupillary response data can improve classifier accuracy for the perceived difficulty of levels in educational video games.

Top-down influence affects the response adaptation of V1 neurons in cats

The visual system lowers its perceptual sensitivity to a prolonged presentation of the same visual signal. This brain plasticity, called visual adaptation, is generally attributed to the response adaptation of neurons in the visual cortex. Although well-studied in the neurons of the primary visual cortex (V1), the contribution of high-level visual cortical regions to the response adaptation of V1 neurons is unclear. In the present study, we measured the response adaptation strength of V1 neurons before and after the top-down influence of the area 21a (A21a), a higher-order visual cortex homologous to the primate V4 area, was modulated with a noninvasive tool of transcranial direct current stimulation (tDCS). Our results showed that the response adaptation of V1 neurons enhanced significantly after applying anode (a-) tDCS in A21a when compared with that before a-tDCS, whereas the response adaptation of V1 neurons weakened after cathode (c-) tDCS relative to before c-tDCS in A21a. By contrast, sham (s-) tDCS in A21a had no significant impact on the response adaptation of V1 neurons. Further analysis indicated that a-tDCS in A21a significantly increased both the initial response (IR) of V1 neurons to the first several (five) trails of visual stimulation and the plateau response (PR) to the prolonged visual stimulation; the increase in PR was lower than in IR, which caused an enhancement in response adaptation. Conversely, c-tDCS significantly decreased both IR and PR of V1 neurons; the reduction in PR was smaller than in IR, which resulted in a weakness in response adaptation. Furthermore, the tDCS-induced changes of V1 neurons in response and response adaptation could recover after tDCS effect vanished, but did not occur after the neuronal activity in A21a was silenced by electrolytic lesions. These results suggest that the top-down influence of A21a may alter the response adaptation of V1 neurons through activation of local inhibitory circuitry, which enhances network inhibition in the V1 area upon an increased top-down input, weakens inhibition upon a decreased top-down input, and thus maintains homeostasis of V1 neurons in response to the long-presenting visual signals.

Computational challenges and opportunities for a bi-directional artificial retina

A future artificial retina that can restore high acuity vision in blind people will rely on the capability to both read (observe) and write (control) the spiking activity of neurons using an adaptive, bi-directional and high-resolution device. Although current research is focused on overcoming the technical challenges of building and implanting such a device, exploiting its capabilities to achieve more acute visual perception will also require substantial computational advances. Using high-density large-scale recording and stimulation in the primate retina with an ex vivo multi-electrode array lab prototype, we frame several of the major computational problems, and describe current progress and future opportunities in solving them. First, we identify cell types and locations from spontaneous activity in the blind retina, and then efficiently estimate their visual response properties by using a low-dimensional manifold of inter-retina variability learned from a large experimental dataset. Second, we estimate retinal responses to a large collection of relevant electrical stimuli by passing current patterns through an electrode array, spike sorting the resulting recordings and using the results to develop a model of evoked responses. Third, we reproduce the desired responses for a given visual target by temporally dithering a diverse collection of electrical stimuli within the integration time of the visual system. Together, these novel approaches may substantially enhance artificial vision in a next-generation device.

What the salamander eye has been telling the vision scientist's brain

Salamanders have been habitual residents of research laboratories for more than a century, and their history in science is tightly interwoven with vision research. Nevertheless, many vision scientists - even those working with salamanders - may be unaware of how much our knowledge about vision, and particularly the retina, has been shaped by studying salamanders. In this review, we take a tour through the salamander history in vision science, highlighting the main contributions of salamanders to our understanding of the vertebrate retina. We further point out specificities of the salamander visual system and discuss the perspectives of this animal system for future vision research.

Evidence for the intrinsically nonlinear nature of receptive fields in vision

The responses of visual neurons, as well as visual perception phenomena in general, are highly nonlinear functions of the visual input, while most vision models are grounded on the notion of a linear receptive field (RF). The linear RF has a number of inherent problems: it changes with the input, it presupposes a set of basis functions for the visual system, and it conflicts with recent studies on dendritic computations. Here we propose to model the RF in a nonlinear manner, introducing the intrinsically nonlinear receptive field (INRF). Apart from being more physiologically plausible and embodying the efficient representation principle, the INRF has a key property of wide-ranging implications: for several vision science phenomena where a linear RF must vary with the input in order to predict responses, the INRF can remain constant under different stimuli. We also prove that Artificial Neural Networks with INRF modules instead of linear filters have a remarkably improved performance and better emulate basic human perception. Our results suggest a change of paradigm for vision science as well as for artificial intelligence.

Dynamic Contextual Modulation in Superior Colliculus of Awake Mouse

The responses of neurons in the visual pathway depend on the context in which a stimulus is presented. Responses to predictable stimuli are usually suppressed, highlighting responses to unexpected stimuli that might be important for behavior. Here, we established how context modulates the response of neurons in the superior colliculus (SC), a region important in orienting toward or away from visual stimuli. We made extracellular recordings from single units in the superficial layers of SC in awake mice. We found strong suppression of visual response by spatial context (surround suppression) and temporal context (adaptation). Neurons showing stronger surround suppression also showed stronger adaptation effects. In neurons where it was present, surround suppression was dynamic and was reduced by adaptation. Adaptation's effects further revealed two components to surround suppression: one component that was weakly tuned for orientation and adaptable, and another component that was more strongly tuned but less adaptable. The selectivity of the tuned component was flexible, such that suppression was stronger when the stimulus over the surround matched that over the receptive field. Our results therefore reveal strong interactions between spatial and temporal context in regulating the flow of signals through mouse SC, and suggest the presence of a subpopulation of neurons that might signal novelty in either space or time.

Unaltered Perception of Suprathreshold Contrast in Early Glaucoma Despite Sensitivity Loss

Purpose

Glaucoma raises contrast detection thresholds, but our natural visual environment is dominated by high contrast that may remain suprathreshold in early to moderate glaucoma. This study investigates the effect of glaucoma on the apparent contrast of visible stimuli.

Methods

Twenty participants with glaucoma with partial visual field defects (mean age, 72 ± 7 years) and 20 age‐similar healthy controls (mean age, 70 ± 7 years) took part. Contrast detection thresholds for Gabor stimuli (SD, 0.75°) of four spatial frequencies (0.5, 1.0, 2.0, and 4.0 c/deg) were first measured at 10° eccentricity, both within and outside of visual field defects for participants with glaucoma. Subsequently, the contrast of a central Gabor was matched to that of a peripheral Gabor with contrast fixed at two times or four times the detection threshold. Data were analyzed by linear mixed modelling.

Results

Compared with controls, detection thresholds for participants with glaucoma were raised by 0.05 ± 0.025 (Michelson units, ± SE; P = 0.12) and by 0.141 ± 0.026 (P < 0.001) outside and within visual field defects, respectively. For reference stimuli at two times the detection contrast, matched contrast ratios (matched/reference contrast) were 0.16 ± 0.039 (P < 0.001) higher outside compared with within visual field defects in participants with glaucoma. Matched contrast ratios within visual field defects were similar to controls (mean 0.033 ± 0.066 lower; P = 0.87). For reference stimuli at four times the detection contrast, matched contrast ratios were similar across all three groups (P = 0.58). Spatial frequency had a minimal effect on matched contrast ratios.

Conclusions

Despite decreased contrast sensitivity, people with glaucoma perceive the contrast of visible suprathreshold stimuli similarly to healthy controls. These results suggest possible compensation for sensitivity loss in the visual system.

Magnitude, time course, and specificity of rapid adaptation across mouse visual areas

Adaptation is a ubiquitous feature of sensory processing whereby recent experience shapes future responses. The mouse primary visual cortex (V1) is particularly sensitive to recent experience, where a brief stimulus can suppress subsequent responses for seconds. This rapid adaptation profoundly impacts perception, suggesting that its effects are propagated along the visual hierarchy. To understand how rapid adaptation influences sensory processing, we measured its effects at key nodes in the visual system: in V1, three higher visual areas (HVAs: lateromedial, anterolateral, and posteromedial), and the superior colliculus (SC) in awake mice of both sexes using single-unit recordings. Consistent with the feed-forward propagation of adaptation along the visual hierarchy, we find that neurons in layer 4 adapt less strongly than those in other layers of V1. Furthermore, neurons in the HVAs adapt more strongly, and recover more slowly, than those in V1. The magnitude and time course of adaptation was comparable in each of the HVAs and in the SC, suggesting that adaptation may not linearly accumulate along the feed-forward visual processing hierarchy. Despite the increase in adaptation in the HVAs compared with V1, the effects were similarly orientation specific across all areas. These data reveal that adaptation profoundly shapes cortical processing, with increasing impact at higher levels in the cortical hierarchy, and also strongly influencing computations in the SC. Thus, we find robust, brain-wide effects of rapid adaptation on sensory processing.NEW & NOTEWORTHY Rapid adaptation dynamically alters sensory signals to account for recent experience. To understand how adaptation affects sensory processing and perception, we must determine how it impacts the diverse set of cortical and subcortical areas along the hierarchy of the mouse visual system. We find that rapid adaptation strongly impacts neurons in primary visual cortex, the higher visual areas, and the colliculus, consistent with its profound effects on behavior.

Accommodation responses following contrast adaptation

The current study explored the effects of contrast adaptation on the accommodation response (AR), using low- and high-pass filtered video clips as stimuli. Ten young myopic (mean ± standard deviation: -2.91 ± 1.36D) and 10 near emmetropic subjects (-0.19 ± 0.14D) participated in the study. The AR was monitored under monocular viewing conditions using an eccentric infrared photorefractor. A 2-stage procedure was used: (1) the minimum spatial frequency content necessary to produce a proper individual AR; and (2) the AR was compared before and after adaptation to low-pass (s = -0.5), control (s = 0) and high-pass (s = +0.5) filtered videos. We found that (1) the average threshold Sinc-blur of both myopes and emmetropes necessary to evoke accommodation was (mean ± standard deviation) λ = 7.40 ± 4.05 cpd. Myopes required a higher Sinc blur (average, 10.00 ± 4.05 cpd) compared to emmetropes (average, 4.80 ± 1.60 cpd). (2) Adaptation to low-pass filtered videos increased the AR by 0.41 ± 0.33D in the myopic group and reduced it in the emmetropic group by 0.31 ± 0.25D. Adaptation to high pass-filtered videos induced similar changes in both refractive groups (an increase of 0.41 ± 0.40D and 0.46 ± 0.29D for myopes and emmetropes, respectively). Our measurements show that the human AR can be modified by spatial frequency selective contrast adaptation although these were short-term effects. The perhaps most striking finding was that adaptation to low pass filtered videos had opposite effects on the AR in emmetropes and myopes. It remains to be studied whether these differences were a consequence of myopia or a contributing factor in myopia development.

Clustering of Neural Activity: A Design Principle for Population Codes

We propose that correlations among neurons are generically strong enough to organize neural activity patterns into a discrete set of clusters, which can each be viewed as a population codeword. Our reasoning starts with the analysis of retinal ganglion cell data using maximum entropy models, showing that the population is robustly in a frustrated, marginally sub-critical, or glassy, state. This leads to an argument that neural populations in many other brain areas might share this structure. Next, we use latent variable models to show that this glassy state possesses well-defined clusters of neural activity. Clusters have three appealing properties: (i) clusters exhibit error correction, i.e., they are reproducibly elicited by the same stimulus despite variability at the level of constituent neurons; (ii) clusters encode qualitatively different visual features than their constituent neurons; and (iii) clusters can be learned by downstream neural circuits in an unsupervised fashion. We hypothesize that these properties give rise to a "learnable" neural code which the cortical hierarchy uses to extract increasingly complex features without supervision or reinforcement.

Visual illusions via neural dynamics: Wilson-Cowan-type models and the efficient representation principle

We reproduce suprathreshold perception phenomena, specifically visual illusions, by Wilson-Cowan (WC)-type models of neuronal dynamics. Our findings show that the ability to replicate the illusions considered is related to how well the neural activity equations comply with the efficient representation principle. Our first contribution consists in showing that the WC equations can reproduce a number of brightness and orientation-dependent illusions. Then we formally prove that there cannot be an energy functional that the WC dynamics are minimizing. This leads us to consider an alternative, variational modeling, which has been previously employed for local histogram equalization (LHE) tasks. To adapt our model to the architecture of V1, we perform an extension that has an explicit dependence on local image orientation. Finally, we report several numerical experiments showing that LHE provides a better reproduction of visual illusions than the original WC formulation, and that its cortical extension is capable also to reproduce complex orientation-dependent illusions.NEW & NOTEWORTHY We show that the Wilson-Cowan equations can reproduce a number of brightness and orientation-dependent illusions. Then we formally prove that there cannot be an energy functional that the Wilson-Cowan equations are minimizing, making them suboptimal with respect to the efficient representation principle. We thus propose a slight modification that is consistent with such principle and show that this provides a better reproduction of visual illusions than the original Wilson-Cowan formulation. We also consider the cortical extension of both models to deal with more complex orientation-dependent illusions.

Flies Exploit Predictable Perspectives and Backgrounds to Enhance Iridescent Signal Salience and Mating Success

Communication requires both the encoding of information and its effective transmission, but little is known about display traits that primarily serve to enhance efficacy. Here we examined the visual courtships of Lispe cana, a cursorial fly that lives and mates in heterogeneous foreshores, and tested the prediction that males should seek to enhance signal salience and consequent fitness through the flexible choice of display locations. We show that courting males access the field of view of females by straddling them and holding their wings closed before moving ahead to present their structurally colored faces in ritualized dances. Males preferentially present these UV-white signals against darker backgrounds and the magnitude of contrast predicts female attention, which in turn predicts mating success. Our results demonstrate a striking interplay between the physical and attentional manipulation of receivers and reveal novel routes to the enhancement of signal efficacy in noisy environments.

Visual Perception of Moisture Is a Pathogen Detection Mechanism of the Behavioral Immune System

The behavioral immune system (BIS) includes perceptual mechanisms for detecting cues of contamination. Former studies have indicated that moisture has a disgusting property. Therefore, moisture could be a target for detecting contamination cues by the BIS. We conducted two experiments to examine the psychophysical basis of moisture perception and clarify the relationship between the perception of moisture and the BIS. We assumed that the number of high luminance areas in a visual image provided optical information that would enable the visual perception of moisture. In two experiments, we presented eight images of dough that contained different amounts of moisture as experimental stimuli. The amount of moisture shown in the images was increased in eight steps, from 28.6 to 42.9% of the total weight of the dough. In Experiment 1, the images were randomly presented on a computer display, and the participants (n = 22) were asked to rank the images in the order of the visually perceived moisture content. In Experiment 2, the participants (n = 15) completed pairwise comparisons based on the perceived moistness of the images. Furthermore, to examine the BIS responses, the participants rated the strength of disgust evoked by the stimuli, their motivation to avoid touching the stimuli, and the estimated magnitude of the risk of contamination by physical contact with the stimuli. The results indicated that the moisture content and the numbers of high luminance areas in the images accurately predicted the perception of moisture, suggesting that the detection of visual moisture was highly accurate, and the optical information served as an essential perceptual cue for detecting moisture. On the other hand, the BIS responses peaked in response to stimuli having approximately 33 to 39% moisture content. These results show that objects containing a moderate amount of moisture could be the target of visually detecting pathogens by the BIS.

Controlling and measuring dynamic odorant stimuli in the laboratory

Animals experience complex odorant stimuli that vary widely in composition, intensity and temporal properties. However, stimuli used to study olfaction in the laboratory are much simpler. This mismatch arises from the challenges in measuring and controlling them precisely and accurately. Even simple pulses can have diverse kinetics that depend on their molecular identity. Here, we introduce a model that describes how stimulus kinetics depend on the molecular identity of the odorant and the geometry of the delivery system. We describe methods to deliver dynamic odorant stimuli of several types, including broadly distributed stimuli that reproduce some of the statistics of naturalistic plumes, in a reproducible and precise manner. Finally, we introduce a method to calibrate a photo-ionization detector to any odorant it can detect, using no additional components. Our approaches are affordable and flexible and can be used to advance our understanding of how olfactory neurons encode real-world odor signals.

Population adaptation in efficient balanced networks

Adaptation is a key component of efficient coding in sensory neurons. However, it remains unclear how neurons can provide a stable representation of external stimuli given their history-dependent responses. Here we show that a stable representation is maintained if efficiency is optimized by a population of neurons rather than by neurons individually. We show that spike-frequency adaptation and E/I balanced recurrent connectivity emerge as solutions to a global cost-accuracy tradeoff. The network will redistribute sensory responses from highly excitable neurons to less excitable neurons as the cost of neural activity increases. This does not change the representation at the population level despite causing dynamic changes in individual neurons. By applying this framework to an orientation coding network, we reconcile neural and behavioral findings. Our approach underscores the common mechanisms behind the diversity of neural adaptation and its role in producing a reliable representation of the stimulus while minimizing metabolic cost.

Humans see, hear, feel, taste and smell the world as spiking electrical signals in the brain encoded by sensory neurons. Sensory neurons learn from experience to adjust their activity when exposed repeatedly to the same stimuli. A loud noise or that strange taste in your mouth might be alarming at first but soon sensory neurons dial down their response as the sensations become familiar, saving energy.

This neural adaptation has been observed experimentally in individual cells, but it raises questions about how the brain deciphers signals from sensory neurons. How do downstream neurons learn whether the reduced activity from sensory neurons is a result of getting used to a feeling, or a signal encoding a weaker stimulus? The energy saved through neural adaptation cannot come at the expense of sensing the world less accurately. Neural networks in our brain have evidently evolved to code information in a way that is both efficient and accurate, and computational neuroscientists want to know how. There has been great interest in reproducing neural networks for machine learning, but computer models have not yet captured the mechanisms of neural coding with the same eloquence as the brain.

Gutierrez and Denève used computational models to test how networks of sensory neurons encode a sensible signal whilst adapting to new or repeated stimuli. The experiments showed that optimal neural networks are highly cooperative and share the load when encoding information. Individual neurons are more sensitive to certain stimuli but the information is encoded across the network so that if one neuron becomes fatigued, others receptive to the same stimuli can respond. In this way, the network is both responsive and reliable, producing a steady output which can be readily interpreted by downstream neurons.

Exploring how stimuli are encoded in the brain, Gutierrez and Denève have shown that the activity of one neuron does not represent the whole picture of neural adaptation. The brain has evolved to adapt to continuous stimuli for efficiency at both the level of individual neurons and across balanced networks of interconnected neurons. It takes many neurons to accurately represent the world, but only as a network can the brain sustain a steady picture.

Neural sensitization improves encoding fidelity in the primate retina

An animal’s motion through the environment can induce large and frequent fluctuations in light intensity on the retina. These fluctuations pose a major challenge to neural circuits tasked with encoding visual information, as they can cause cells to adapt and lose sensitivity. Here, we report that sensitization, a short-term plasticity mechanism, solves this difficult computational problem by maintaining neuronal sensitivity in the face of these fluctuations. The numerically dominant output pathway in the macaque monkey retina, the midget (parvocellular-projecting) pathway, undergoes sensitization under specific conditions, including simulated eye movements. Sensitization is present in the excitatory synaptic inputs from midget bipolar cells and is mediated by presynaptic disinhibition from a wide-field mechanism extending >0.5 mm along the retinal surface. Direct physiological recordings and a computational model indicate that sensitization in the midget pathway supports accurate sensory encoding and prevents a loss of responsiveness during dynamic visual processing.

Light intensity on the retina can fluctuate rapidly during natural vision, posing a challenge for encoding visual information. Here, the authors report that mechanisms of sensitization/facilitation maintain the sensitivity of the numerically dominant neural pathway in the primate retina during dynamic vision.

Coding Principles in Adaptation

Adaptation is a common principle that recurs throughout the nervous system at all stages of processing. This principle manifests in a variety of phenomena, from spike frequency adaptation, to apparent changes in receptive fields with changes in stimulus statistics, to enhanced responses to unexpected stimuli. The ubiquity of adaptation leads naturally to the question: What purpose do these different types of adaptation serve? A diverse set of theories, often highly overlapping, has been proposed to explain the functional role of adaptive phenomena. In this review, we discuss several of these theoretical frameworks, highlighting relationships among them and clarifying distinctions. We summarize observations of the varied manifestations of adaptation, particularly as they relate to these theoretical frameworks, focusing throughout on the visual system and making connections to other sensory systems.

Slow Temporal Dynamics of Motion-Induced Brightness Shift Reveals Impact of Adaptation

Brightness of an object is determined by various factors including ambient illumination, surface reflectance of the object, and spatial and temporal relation between the object and its surrounding context. Recently, it has been demonstrated that the motion of an object alters its own and nearby object’s appearance such as brightness and color. This study aims to unveil mechanisms of the motion-induced brightness shift by measuring its temporal dynamics. We found that the motion-induced brightness shift occurred instantaneously with the motion onset when the motion was introduced abruptly. However, the brightness of a stationary object was altered gradually by a nearby moving object in about 2 s time window when the stationary dot was introduced abruptly. Two distinct temporal dynamics (slow vs. fast) of the motion-induced brightness shift demonstrate that both slow neural adaptation and fast neural normalization processes determine the brightness shift induced by the object’s motion.

Evaluation of Image Enhancement Techniques for Vision-Based Navigation under Low Illumination

Cameras are valuable sensors for robotics perception tasks. Among these perception tasks are motion estimation, localization, and object detection. Cameras are attractive sensors because they are passive and relatively cheap and can provide rich information. However, being passive sensors, they rely on external illumination from the environment which means that their performance degrades in low-light conditions. In this paper, we present and investigate four methods to enhance images under challenging night conditions. The findings are relevant to a wide range of feature-based vision systems, such as tracking for augmented reality, image registration, localization, and mapping, as well as deep learning-based object detectors. As autonomous mobile robots are expected to operate under low-illumination conditions at night, evaluation is based on state-of-the-art systems for motion estimation, localization, and object detection.